2023-10-16T19:48:23.858 INFO:root:teuthology version: 0.0.1.dev197+g8cdab07 2023-10-16T19:48:23.860 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm} --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2023-10-16_18:49:15-fs-rishabh-2023oct13-testing-default-smithi/7430091 --verbose --name rishabh-2023-10-16_18:49:15-fs-rishabh-2023oct13-testing-default-smithi -- /home/teuthworker/archive/rishabh-2023-10-16_18:49:15-fs-rishabh-2023oct13-testing-default-smithi/7430091/orig.config.yaml 2023-10-16T19:48:23.903 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-16T19:48:23.985 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-10-16_18:49:15-fs-rishabh-2023oct13-testing-default-smithi/7430091 branch: rishabh-2023oct13 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm} email: null first_in_suite: false job_id: '7430091' kernel: flavor: default kdb: true sha1: 47672119149b103ec7cec2cc65c063ccbb001ac7 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-10-16_18:49:15-fs-rishabh-2023oct13-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: rishabh-2023oct13 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rishabh-2023oct13-qa2 sha1: 19fd96fe3581520b6d9f50ffdabf29051aee5d4d owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 6006 sha1: 4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 sleep_before_teardown: 0 subset: 402/512 suite: fs suite_branch: rishabh-2023oct13-qa2 suite_path: /home/teuthworker/src/github.com_rishabh-d-dave_ceph_19fd96fe3581520b6d9f50ffdabf29051aee5d4d/qa suite_relpath: qa suite_repo: https://github.com/rishabh-d-dave/ceph suite_sha1: 19fd96fe3581520b6d9f50ffdabf29051aee5d4d syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi088.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO3u8ck309lL1eF1slemCDnL7evTbk1vSlxyuVhiJ0jwjFWr/00avoczBkwVMMA35fzd5AfNup9oIrkl12E+ihA= smithi094.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMmVItP9PTvUPm1HK+zbQSrkxi0N+SNhl6uG+l41BT706NG4MjhexTB9Ww7pdf+KZC/ZJNRNAt9HxF8lUnSjKy8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_pool_perm teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 8cdab074dcca9a68965bc5a50e9c30b691949723 timestamp: 2023-10-16_18:49:15 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-16T19:48:23.986 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_rishabh-d-dave_ceph_19fd96fe3581520b6d9f50ffdabf29051aee5d4d/qa; will attempt to use it 2023-10-16T19:48:23.987 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_rishabh-d-dave_ceph_19fd96fe3581520b6d9f50ffdabf29051aee5d4d/qa/tasks 2023-10-16T19:48:23.987 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-16T19:48:23.989 INFO:teuthology.task.internal:Checking packages... 2023-10-16T19:48:24.011 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0' 2023-10-16T19:48:24.011 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-16T19:48:24.012 INFO:teuthology.packaging:ref: None 2023-10-16T19:48:24.012 INFO:teuthology.packaging:tag: None 2023-10-16T19:48:24.012 INFO:teuthology.packaging:branch: rishabh-2023oct13 2023-10-16T19:48:24.013 INFO:teuthology.packaging:sha1: 4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 2023-10-16T19:48:24.013 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=rishabh-2023oct13 2023-10-16T19:48:24.162 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6730-g4ba8a190-1jammy 2023-10-16T19:48:24.163 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-16T19:48:24.169 INFO:teuthology.task.internal:no buildpackages task found 2023-10-16T19:48:24.169 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-16T19:48:24.196 INFO:teuthology.task.internal:Saving configuration 2023-10-16T19:48:24.212 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-16T19:48:24.224 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-16T19:48:24.248 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi088.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-16_18:49:15-fs-rishabh-2023oct13-testing-default-smithi/7430091', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-16 19:42:32.886281', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIP/zYcYzQkbdm2YNLkoqq8OgP2fFLCL+vQe6hin4KLvE'} 2023-10-16T19:48:24.270 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi094.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-16_18:49:15-fs-rishabh-2023oct13-testing-default-smithi/7430091', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-16 19:42:32.884956', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIG9NhWz5kHWfzI+FkrUn4acS8PqaKztR5QjDCtl9q+Pe'} 2023-10-16T19:48:24.270 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-16T19:48:24.277 INFO:teuthology.task.internal:roles: ubuntu@smithi088.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-16T19:48:24.277 INFO:teuthology.task.internal:roles: ubuntu@smithi094.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-16T19:48:24.277 INFO:teuthology.run_tasks:Running task console_log... 2023-10-16T19:48:24.353 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6da4200a60>, signals=[15]) 2023-10-16T19:48:24.354 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-16T19:48:24.363 INFO:teuthology.task.internal:Opening connections... 2023-10-16T19:48:24.363 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi088.front.sepia.ceph.com 2023-10-16T19:48:24.365 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T19:48:24.442 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi094.front.sepia.ceph.com 2023-10-16T19:48:24.443 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T19:48:24.514 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-16T19:48:24.522 DEBUG:teuthology.orchestra.run.smithi088:> uname -m 2023-10-16T19:48:24.527 INFO:teuthology.orchestra.run.smithi088.stdout:x86_64 2023-10-16T19:48:24.528 DEBUG:teuthology.orchestra.run.smithi088:> cat /etc/os-release 2023-10-16T19:48:24.574 INFO:teuthology.orchestra.run.smithi088.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-10-16T19:48:24.574 INFO:teuthology.orchestra.run.smithi088.stdout:NAME="Ubuntu" 2023-10-16T19:48:24.574 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION_ID="22.04" 2023-10-16T19:48:24.575 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-10-16T19:48:24.575 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION_CODENAME=jammy 2023-10-16T19:48:24.575 INFO:teuthology.orchestra.run.smithi088.stdout:ID=ubuntu 2023-10-16T19:48:24.575 INFO:teuthology.orchestra.run.smithi088.stdout:ID_LIKE=debian 2023-10-16T19:48:24.575 INFO:teuthology.orchestra.run.smithi088.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-16T19:48:24.576 INFO:teuthology.orchestra.run.smithi088.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-16T19:48:24.576 INFO:teuthology.orchestra.run.smithi088.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-16T19:48:24.576 INFO:teuthology.orchestra.run.smithi088.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-16T19:48:24.576 INFO:teuthology.orchestra.run.smithi088.stdout:UBUNTU_CODENAME=jammy 2023-10-16T19:48:24.577 INFO:teuthology.lock.ops:Updating smithi088.front.sepia.ceph.com on lock server 2023-10-16T19:48:24.602 DEBUG:teuthology.orchestra.run.smithi094:> uname -m 2023-10-16T19:48:24.607 INFO:teuthology.orchestra.run.smithi094.stdout:x86_64 2023-10-16T19:48:24.607 DEBUG:teuthology.orchestra.run.smithi094:> cat /etc/os-release 2023-10-16T19:48:24.654 INFO:teuthology.orchestra.run.smithi094.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-10-16T19:48:24.655 INFO:teuthology.orchestra.run.smithi094.stdout:NAME="Ubuntu" 2023-10-16T19:48:24.655 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION_ID="22.04" 2023-10-16T19:48:24.655 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-10-16T19:48:24.655 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION_CODENAME=jammy 2023-10-16T19:48:24.656 INFO:teuthology.orchestra.run.smithi094.stdout:ID=ubuntu 2023-10-16T19:48:24.656 INFO:teuthology.orchestra.run.smithi094.stdout:ID_LIKE=debian 2023-10-16T19:48:24.656 INFO:teuthology.orchestra.run.smithi094.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-16T19:48:24.656 INFO:teuthology.orchestra.run.smithi094.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-16T19:48:24.656 INFO:teuthology.orchestra.run.smithi094.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-16T19:48:24.657 INFO:teuthology.orchestra.run.smithi094.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-16T19:48:24.657 INFO:teuthology.orchestra.run.smithi094.stdout:UBUNTU_CODENAME=jammy 2023-10-16T19:48:24.657 INFO:teuthology.lock.ops:Updating smithi094.front.sepia.ceph.com on lock server 2023-10-16T19:48:24.677 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-16T19:48:24.689 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-16T19:48:24.700 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-16T19:48:24.701 DEBUG:teuthology.orchestra.run.smithi088:> test '!' -e /home/ubuntu/cephtest 2023-10-16T19:48:24.704 DEBUG:teuthology.orchestra.run.smithi094:> test '!' -e /home/ubuntu/cephtest 2023-10-16T19:48:24.708 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-16T19:48:24.716 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-16T19:48:24.717 DEBUG:teuthology.orchestra.run.smithi088:> test -z $(ls -A /var/lib/ceph) 2023-10-16T19:48:24.752 DEBUG:teuthology.orchestra.run.smithi094:> test -z $(ls -A /var/lib/ceph) 2023-10-16T19:48:24.784 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-16T19:48:24.838 INFO:teuthology.run_tasks:Running task kernel... 2023-10-16T19:48:24.853 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '47672119149b103ec7cec2cc65c063ccbb001ac7'} 2023-10-16T19:48:24.853 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '47672119149b103ec7cec2cc65c063ccbb001ac7'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '47672119149b103ec7cec2cc65c063ccbb001ac7'}}, timeout 300 2023-10-16T19:48:24.854 DEBUG:teuthology.orchestra.run.smithi088:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-16T19:48:24.855 DEBUG:teuthology.orchestra.run.smithi094:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-16T19:48:24.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T19:48:24.860 DEBUG:teuthology.task.kernel:sha1 for mon.a is 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-16T19:48:24.860 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "47672119149b103ec7cec2cc65c063ccbb001ac7"... 2023-10-16T19:48:24.860 DEBUG:teuthology.orchestra.run.smithi088:> uname -r 2023-10-16T19:48:24.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T19:48:24.862 DEBUG:teuthology.task.kernel:sha1 for mon.b is 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-16T19:48:24.862 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "47672119149b103ec7cec2cc65c063ccbb001ac7"... 2023-10-16T19:48:24.862 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2023-10-16T19:48:24.907 INFO:teuthology.orchestra.run.smithi088.stdout:5.15.0-56-generic 2023-10-16T19:48:24.908 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-16T19:48:24.908 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-16T19:48:24.908 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-16T19:48:24.913 INFO:teuthology.orchestra.run.smithi094.stdout:5.15.0-56-generic 2023-10-16T19:48:24.913 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-16T19:48:24.913 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-16T19:48:24.914 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-16T19:48:25.057 INFO:teuthology.task.kernel:Installing linux-firmware on mon.a... 2023-10-16T19:48:25.057 DEBUG:teuthology.orchestra.run.smithi088:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2023-10-16T19:48:25.060 INFO:teuthology.task.kernel:Installing linux-firmware on mon.b... 2023-10-16T19:48:25.060 DEBUG:teuthology.orchestra.run.smithi094:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2023-10-16T19:48:25.076 INFO:teuthology.orchestra.run.smithi088.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2023-10-16T19:48:25.077 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2023-10-16T19:48:25.190 INFO:teuthology.orchestra.run.smithi094.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2023-10-16T19:48:25.191 INFO:teuthology.orchestra.run.smithi094.stderr:hint: is subject to change. To configure the initial branch name to use in all 2023-10-16T19:48:25.191 INFO:teuthology.orchestra.run.smithi094.stderr:hint: of your new repositories, which will suppress this warning, call: 2023-10-16T19:48:25.191 INFO:teuthology.orchestra.run.smithi094.stderr:hint: 2023-10-16T19:48:25.191 INFO:teuthology.orchestra.run.smithi094.stderr:hint: git config --global init.defaultBranch 2023-10-16T19:48:25.191 INFO:teuthology.orchestra.run.smithi094.stderr:hint: 2023-10-16T19:48:25.191 INFO:teuthology.orchestra.run.smithi094.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2023-10-16T19:48:25.192 INFO:teuthology.orchestra.run.smithi094.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2023-10-16T19:48:25.192 INFO:teuthology.orchestra.run.smithi094.stderr:hint: 2023-10-16T19:48:25.192 INFO:teuthology.orchestra.run.smithi094.stderr:hint: git branch -m 2023-10-16T19:48:25.193 INFO:teuthology.orchestra.run.smithi094.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2023-10-16T19:48:25.193 DEBUG:teuthology.orchestra.run.smithi094:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2023-10-16T19:48:25.224 INFO:teuthology.orchestra.run.smithi088.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2023-10-16T19:48:25.224 INFO:teuthology.orchestra.run.smithi088.stderr:hint: is subject to change. To configure the initial branch name to use in all 2023-10-16T19:48:25.225 INFO:teuthology.orchestra.run.smithi088.stderr:hint: of your new repositories, which will suppress this warning, call: 2023-10-16T19:48:25.225 INFO:teuthology.orchestra.run.smithi088.stderr:hint: 2023-10-16T19:48:25.225 INFO:teuthology.orchestra.run.smithi088.stderr:hint: git config --global init.defaultBranch 2023-10-16T19:48:25.225 INFO:teuthology.orchestra.run.smithi088.stderr:hint: 2023-10-16T19:48:25.225 INFO:teuthology.orchestra.run.smithi088.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2023-10-16T19:48:25.225 INFO:teuthology.orchestra.run.smithi088.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2023-10-16T19:48:25.226 INFO:teuthology.orchestra.run.smithi088.stderr:hint: 2023-10-16T19:48:25.226 INFO:teuthology.orchestra.run.smithi088.stderr:hint: git branch -m 2023-10-16T19:48:25.226 INFO:teuthology.orchestra.run.smithi088.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2023-10-16T19:48:25.228 DEBUG:teuthology.orchestra.run.smithi088:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2023-10-16T19:48:25.262 DEBUG:teuthology.orchestra.run.smithi094:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2023-10-16T19:48:25.294 DEBUG:teuthology.orchestra.run.smithi088:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2023-10-16T19:48:25.313 DEBUG:teuthology.orchestra.run.smithi094:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2023-10-16T19:48:25.351 DEBUG:teuthology.orchestra.run.smithi088:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2023-10-16T19:49:32.492 INFO:teuthology.orchestra.run.smithi094.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2023-10-16T19:49:32.493 INFO:teuthology.orchestra.run.smithi094.stderr: * [new branch] main -> origin/main 2023-10-16T19:49:32.515 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20190312 -> 20190312 2023-10-16T19:49:32.516 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20190416 -> 20190416 2023-10-16T19:49:32.516 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20190514 -> 20190514 2023-10-16T19:49:32.516 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20190618 -> 20190618 2023-10-16T19:49:32.516 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20190717 -> 20190717 2023-10-16T19:49:32.517 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20190815 -> 20190815 2023-10-16T19:49:32.517 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20190923 -> 20190923 2023-10-16T19:49:32.517 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20191022 -> 20191022 2023-10-16T19:49:32.517 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20191215 -> 20191215 2023-10-16T19:49:32.520 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20200122 -> 20200122 2023-10-16T19:49:32.521 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20200316 -> 20200316 2023-10-16T19:49:32.521 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20200421 -> 20200421 2023-10-16T19:49:32.521 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20200519 -> 20200519 2023-10-16T19:49:32.521 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20200619 -> 20200619 2023-10-16T19:49:32.521 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20200721 -> 20200721 2023-10-16T19:49:32.522 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20200817 -> 20200817 2023-10-16T19:49:32.522 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20200918 -> 20200918 2023-10-16T19:49:32.522 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20201022 -> 20201022 2023-10-16T19:49:32.522 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20201118 -> 20201118 2023-10-16T19:49:32.522 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20201218 -> 20201218 2023-10-16T19:49:32.523 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20210208 -> 20210208 2023-10-16T19:49:32.523 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20210315 -> 20210315 2023-10-16T19:49:32.523 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20210511 -> 20210511 2023-10-16T19:49:32.523 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20210716 -> 20210716 2023-10-16T19:49:32.523 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20210818 -> 20210818 2023-10-16T19:49:32.524 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20210919 -> 20210919 2023-10-16T19:49:32.524 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20211027 -> 20211027 2023-10-16T19:49:32.524 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20211216 -> 20211216 2023-10-16T19:49:32.524 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20220209 -> 20220209 2023-10-16T19:49:32.524 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20220310 -> 20220310 2023-10-16T19:49:32.524 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20220411 -> 20220411 2023-10-16T19:49:32.525 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20220509 -> 20220509 2023-10-16T19:49:32.525 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20220610 -> 20220610 2023-10-16T19:49:32.525 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20220708 -> 20220708 2023-10-16T19:49:32.525 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20220815 -> 20220815 2023-10-16T19:49:32.525 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20220913 -> 20220913 2023-10-16T19:49:32.526 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20221012 -> 20221012 2023-10-16T19:49:32.526 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20221109 -> 20221109 2023-10-16T19:49:32.526 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20221214 -> 20221214 2023-10-16T19:49:32.526 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20230117 -> 20230117 2023-10-16T19:49:32.526 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20230210 -> 20230210 2023-10-16T19:49:32.527 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20230310 -> 20230310 2023-10-16T19:49:32.527 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20230404 -> 20230404 2023-10-16T19:49:32.528 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20230515 -> 20230515 2023-10-16T19:49:32.528 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20230625 -> 20230625 2023-10-16T19:49:32.528 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20230804 -> 20230804 2023-10-16T19:49:32.528 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20230919 -> 20230919 2023-10-16T19:49:36.238 INFO:teuthology.orchestra.run.smithi094.stderr:Updating files: 50% (1592/3161) Updating files: 51% (1613/3161) Updating files: 52% (1644/3161) Updating files: 53% (1676/3161) Updating files: 54% (1707/3161) Updating files: 55% (1739/3161) Updating files: 56% (1771/3161) Updating files: 57% (1802/3161) Updating files: 58% (1834/3161) Updating files: 59% (1865/3161) Updating files: 60% (1897/3161) Updating files: 61% (1929/3161) Updating files: 62% (1960/3161) Updating files: 63% (1992/3161) Updating files: 64% (2024/3161) Updating files: 65% (2055/3161) Updating files: 66% (2087/3161) Updating files: 66% (2106/3161) Updating files: 67% (2118/3161) Updating files: 68% (2150/3161) Updating files: 69% (2182/3161) Updating files: 70% (2213/3161) Updating files: 71% (2245/3161) Updating files: 72% (2276/3161) Updating files: 73% (2308/3161) Updating files: 74% (2340/3161) Updating files: 75% (2371/3161) Updating files: 76% (2403/3161) Updating files: 77% (2434/3161) Updating files: 78% (2466/3161) Updating files: 79% (2498/3161) Updating files: 80% (2529/3161) Updating files: 81% (2561/3161) Updating files: 81% (2581/3161) Updating files: 82% (2593/3161) Updating files: 83% (2624/3161) Updating files: 84% (2656/3161) Updating files: 85% (2687/3161) Updating files: 86% (2719/3161) Updating files: 87% (2751/3161) Updating files: 88% (2782/3161) Updating files: 89% (2814/3161) Updating files: 90% (2845/3161) Updating files: 91% (2877/3161) Updating files: 92% (2909/3161) Updating files: 93% (2940/3161) Updating files: 94% (2972/3161) Updating files: 95% (3003/3161) Updating files: 96% (3035/3161) Updating files: 97% (3067/3161) Updating files: 98% (3098/3161) Updating files: 99% (3130/3161) Updating files: 100% (3161/3161) Updating files: 100% (3161/3161), done. 2023-10-16T19:49:36.243 INFO:teuthology.orchestra.run.smithi094.stdout:HEAD is now at 63e8aa40 Merge branch 'main' into 'main' 2023-10-16T19:49:36.262 INFO:teuthology.task.kernel:Downloading kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on mon.b... 2023-10-16T19:49:36.263 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-16T19:49:36.533 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-rc5-g47672119149b/ 2023-10-16T19:49:36.534 DEBUG:teuthology.orchestra.run.smithi094:> rm -f /tmp/linux-image.deb && echo linux-image-6.6.0-rc5-g47672119149b_6.6.0-rc5-g47672119149b-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-rc5-g47672119149b/ --input-file=- 2023-10-16T19:49:36.537 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi094.front.sepia.ceph.com to complete... 2023-10-16T19:49:36.654 INFO:teuthology.orchestra.run.smithi088.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2023-10-16T19:49:36.654 INFO:teuthology.orchestra.run.smithi088.stderr: * [new branch] main -> origin/main 2023-10-16T19:49:36.685 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20190312 -> 20190312 2023-10-16T19:49:36.685 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20190416 -> 20190416 2023-10-16T19:49:36.685 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20190514 -> 20190514 2023-10-16T19:49:36.685 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20190618 -> 20190618 2023-10-16T19:49:36.686 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20190717 -> 20190717 2023-10-16T19:49:36.686 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20190815 -> 20190815 2023-10-16T19:49:36.686 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20190923 -> 20190923 2023-10-16T19:49:36.686 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20191022 -> 20191022 2023-10-16T19:49:36.691 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20191215 -> 20191215 2023-10-16T19:49:36.691 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20200122 -> 20200122 2023-10-16T19:49:36.691 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20200316 -> 20200316 2023-10-16T19:49:36.691 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20200421 -> 20200421 2023-10-16T19:49:36.692 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20200519 -> 20200519 2023-10-16T19:49:36.692 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20200619 -> 20200619 2023-10-16T19:49:36.692 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20200721 -> 20200721 2023-10-16T19:49:36.692 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20200817 -> 20200817 2023-10-16T19:49:36.692 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20200918 -> 20200918 2023-10-16T19:49:36.692 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20201022 -> 20201022 2023-10-16T19:49:36.693 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20201118 -> 20201118 2023-10-16T19:49:36.693 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20201218 -> 20201218 2023-10-16T19:49:36.693 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20210208 -> 20210208 2023-10-16T19:49:36.693 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20210315 -> 20210315 2023-10-16T19:49:36.693 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20210511 -> 20210511 2023-10-16T19:49:36.694 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20210716 -> 20210716 2023-10-16T19:49:36.694 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20210818 -> 20210818 2023-10-16T19:49:36.694 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20210919 -> 20210919 2023-10-16T19:49:36.694 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20211027 -> 20211027 2023-10-16T19:49:36.694 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20211216 -> 20211216 2023-10-16T19:49:36.695 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20220209 -> 20220209 2023-10-16T19:49:36.695 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20220310 -> 20220310 2023-10-16T19:49:36.695 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20220411 -> 20220411 2023-10-16T19:49:36.695 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20220509 -> 20220509 2023-10-16T19:49:36.695 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20220610 -> 20220610 2023-10-16T19:49:36.696 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20220708 -> 20220708 2023-10-16T19:49:36.696 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20220815 -> 20220815 2023-10-16T19:49:36.696 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20220913 -> 20220913 2023-10-16T19:49:36.696 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20221012 -> 20221012 2023-10-16T19:49:36.696 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20221109 -> 20221109 2023-10-16T19:49:36.696 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20221214 -> 20221214 2023-10-16T19:49:36.697 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20230117 -> 20230117 2023-10-16T19:49:36.697 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20230210 -> 20230210 2023-10-16T19:49:36.697 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20230310 -> 20230310 2023-10-16T19:49:36.697 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20230404 -> 20230404 2023-10-16T19:49:36.697 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20230515 -> 20230515 2023-10-16T19:49:36.698 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20230625 -> 20230625 2023-10-16T19:49:36.698 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20230804 -> 20230804 2023-10-16T19:49:36.698 INFO:teuthology.orchestra.run.smithi088.stderr: * [new tag] 20230919 -> 20230919 2023-10-16T19:49:40.391 INFO:teuthology.orchestra.run.smithi088.stderr:Updating files: 51% (1624/3161) Updating files: 52% (1644/3161) Updating files: 53% (1676/3161) Updating files: 54% (1707/3161) Updating files: 55% (1739/3161) Updating files: 56% (1771/3161) Updating files: 57% (1802/3161) Updating files: 58% (1834/3161) Updating files: 59% (1865/3161) Updating files: 60% (1897/3161) Updating files: 61% (1929/3161) Updating files: 62% (1960/3161) Updating files: 63% (1992/3161) Updating files: 64% (2024/3161) Updating files: 65% (2055/3161) Updating files: 66% (2087/3161) Updating files: 66% (2106/3161) Updating files: 67% (2118/3161) Updating files: 68% (2150/3161) Updating files: 69% (2182/3161) Updating files: 70% (2213/3161) Updating files: 71% (2245/3161) Updating files: 72% (2276/3161) Updating files: 73% (2308/3161) Updating files: 74% (2340/3161) Updating files: 75% (2371/3161) Updating files: 76% (2403/3161) Updating files: 77% (2434/3161) Updating files: 78% (2466/3161) Updating files: 79% (2498/3161) Updating files: 80% (2529/3161) Updating files: 81% (2561/3161) Updating files: 81% (2589/3161) Updating files: 82% (2593/3161) Updating files: 83% (2624/3161) Updating files: 84% (2656/3161) Updating files: 85% (2687/3161) Updating files: 86% (2719/3161) Updating files: 87% (2751/3161) Updating files: 88% (2782/3161) Updating files: 89% (2814/3161) Updating files: 90% (2845/3161) Updating files: 91% (2877/3161) Updating files: 92% (2909/3161) Updating files: 93% (2940/3161) Updating files: 94% (2972/3161) Updating files: 95% (3003/3161) Updating files: 96% (3035/3161) Updating files: 97% (3067/3161) Updating files: 98% (3098/3161) Updating files: 99% (3130/3161) Updating files: 100% (3161/3161) Updating files: 100% (3161/3161), done. 2023-10-16T19:49:40.396 INFO:teuthology.orchestra.run.smithi088.stdout:HEAD is now at 63e8aa40 Merge branch 'main' into 'main' 2023-10-16T19:49:40.415 INFO:teuthology.task.kernel:Downloading kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on mon.a... 2023-10-16T19:49:40.416 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-16T19:49:40.560 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-rc5-g47672119149b/ 2023-10-16T19:49:40.561 DEBUG:teuthology.orchestra.run.smithi088:> rm -f /tmp/linux-image.deb && echo linux-image-6.6.0-rc5-g47672119149b_6.6.0-rc5-g47672119149b-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-rc5-g47672119149b/ --input-file=- 2023-10-16T19:49:40.563 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi088.front.sepia.ceph.com to complete... 2023-10-16T19:49:41.067 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16 19:49:40 URL:https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-rc5-g47672119149b/linux-image-6.6.0-rc5-g47672119149b_6.6.0-rc5-g47672119149b-1_amd64.deb [38909314/38909314] -> "/tmp/linux-image.deb" [1] 2023-10-16T19:49:41.067 INFO:teuthology.orchestra.run.smithi094.stderr:FINISHED --2023-10-16 19:49:40-- 2023-10-16T19:49:41.067 INFO:teuthology.orchestra.run.smithi094.stderr:Total wall clock time: 4.5s 2023-10-16T19:49:41.068 INFO:teuthology.orchestra.run.smithi094.stderr:Downloaded: 1 files, 37M in 4.2s (8.81 MB/s) 2023-10-16T19:49:41.070 INFO:teuthology.task.kernel:Installing kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on mon.b... 2023-10-16T19:49:41.070 DEBUG:teuthology.orchestra.run.smithi094:> sudo dpkg -i /tmp/linux-image.deb 2023-10-16T19:49:41.317 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package linux-image-6.6.0-rc5-g47672119149b. 2023-10-16T19:49:43.312 INFO:teuthology.orchestra.run.smithi094.stdout:(Reading database ... 129542 files and directories currently installed.) 2023-10-16T19:49:43.316 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack /tmp/linux-image.deb ... 2023-10-16T19:49:43.471 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking linux-image-6.6.0-rc5-g47672119149b (6.6.0-rc5-g47672119149b-1) ... 2023-10-16T19:49:44.531 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16 19:49:44 URL:https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-rc5-g47672119149b/linux-image-6.6.0-rc5-g47672119149b_6.6.0-rc5-g47672119149b-1_amd64.deb [38909314/38909314] -> "/tmp/linux-image.deb" [1] 2023-10-16T19:49:44.532 INFO:teuthology.orchestra.run.smithi088.stderr:FINISHED --2023-10-16 19:49:44-- 2023-10-16T19:49:44.532 INFO:teuthology.orchestra.run.smithi088.stderr:Total wall clock time: 3.9s 2023-10-16T19:49:44.532 INFO:teuthology.orchestra.run.smithi088.stderr:Downloaded: 1 files, 37M in 3.5s (10.6 MB/s) 2023-10-16T19:49:44.534 INFO:teuthology.task.kernel:Installing kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on mon.a... 2023-10-16T19:49:44.535 DEBUG:teuthology.orchestra.run.smithi088:> sudo dpkg -i /tmp/linux-image.deb 2023-10-16T19:49:44.809 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package linux-image-6.6.0-rc5-g47672119149b. 2023-10-16T19:49:46.821 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up linux-image-6.6.0-rc5-g47672119149b (6.6.0-rc5-g47672119149b-1) ... 2023-10-16T19:49:46.822 INFO:teuthology.orchestra.run.smithi088.stdout:(Reading database ... 129542 files and directories currently installed.) 2023-10-16T19:49:46.826 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack /tmp/linux-image.deb ... 2023-10-16T19:49:46.923 INFO:teuthology.orchestra.run.smithi094.stderr:update-initramfs: Generating /boot/initrd.img-6.6.0-rc5-g47672119149b 2023-10-16T19:49:47.067 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking linux-image-6.6.0-rc5-g47672119149b (6.6.0-rc5-g47672119149b-1) ... 2023-10-16T19:49:50.443 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up linux-image-6.6.0-rc5-g47672119149b (6.6.0-rc5-g47672119149b-1) ... 2023-10-16T19:49:50.534 INFO:teuthology.orchestra.run.smithi088.stderr:update-initramfs: Generating /boot/initrd.img-6.6.0-rc5-g47672119149b 2023-10-16T19:50:07.395 INFO:teuthology.orchestra.run.smithi094.stderr:Sourcing file `/etc/default/grub' 2023-10-16T19:50:07.411 INFO:teuthology.orchestra.run.smithi094.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-16T19:50:07.437 INFO:teuthology.orchestra.run.smithi094.stderr:Generating grub configuration file ... 2023-10-16T19:50:07.749 INFO:teuthology.orchestra.run.smithi094.stderr:Found linux image: /boot/vmlinuz-6.6.0-rc5-g47672119149b 2023-10-16T19:50:07.765 INFO:teuthology.orchestra.run.smithi094.stderr:Found initrd image: /boot/initrd.img-6.6.0-rc5-g47672119149b 2023-10-16T19:50:08.044 INFO:teuthology.orchestra.run.smithi094.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-10-16T19:50:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-10-16T19:50:08.114 INFO:teuthology.orchestra.run.smithi094.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-10-16T19:50:08.119 INFO:teuthology.orchestra.run.smithi094.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-10-16T19:50:08.255 INFO:teuthology.orchestra.run.smithi094.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-10-16T19:50:08.255 INFO:teuthology.orchestra.run.smithi094.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-10-16T19:50:08.256 INFO:teuthology.orchestra.run.smithi094.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-10-16T19:50:08.309 INFO:teuthology.orchestra.run.smithi094.stderr:done 2023-10-16T19:50:08.475 DEBUG:teuthology.orchestra.run.smithi094:> dpkg-deb -c /tmp/linux-image.deb 2023-10-16T19:50:08.482 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./ 2023-10-16T19:50:08.482 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./boot/ 2023-10-16T19:50:08.483 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 4415097 2023-10-10 19:56 ./boot/System.map-6.6.0-rc5-g47672119149b 2023-10-16T19:50:08.493 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 169298 2023-10-10 19:56 ./boot/config-6.6.0-rc5-g47672119149b 2023-10-16T19:50:08.493 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11352928 2023-10-10 19:56 ./boot/vmlinuz-6.6.0-rc5-g47672119149b 2023-10-16T19:50:08.509 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/ 2023-10-16T19:50:08.509 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/kernel/ 2023-10-16T19:50:08.510 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/kernel/postinst.d/ 2023-10-16T19:50:08.510 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/kernel/postrm.d/ 2023-10-16T19:50:08.510 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/kernel/preinst.d/ 2023-10-16T19:50:08.510 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/kernel/prerm.d/ 2023-10-16T19:50:08.510 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/ 2023-10-16T19:50:08.511 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/ 2023-10-16T19:50:08.511 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/ 2023-10-16T19:50:08.511 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/ 2023-10-16T19:50:08.511 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/ 2023-10-16T19:50:08.511 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/ 2023-10-16T19:50:08.512 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/ 2023-10-16T19:50:08.512 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 490233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/aesni-intel.ko 2023-10-16T19:50:08.514 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-16T19:50:08.514 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-16T19:50:08.515 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-16T19:50:08.515 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 93113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-16T19:50:08.515 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 106729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-16T19:50:08.515 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 152241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-16T19:50:08.515 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-16T19:50:08.516 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-16T19:50:08.516 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 83889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-16T19:50:08.516 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-16T19:50:08.516 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-16T19:50:08.516 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-16T19:50:08.517 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 51801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-16T19:50:08.517 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-16T19:50:08.517 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-16T19:50:08.517 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 58585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-16T19:50:08.517 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 50497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-16T19:50:08.518 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-16T19:50:08.518 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-16T19:50:08.518 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/ 2023-10-16T19:50:08.518 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/ 2023-10-16T19:50:08.518 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce/ 2023-10-16T19:50:08.518 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-16T19:50:08.519 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpuid.ko 2023-10-16T19:50:08.519 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/msr.ko 2023-10-16T19:50:08.519 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/ 2023-10-16T19:50:08.519 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 353761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-16T19:50:08.519 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 881713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-16T19:50:08.520 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 2276817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm.ko 2023-10-16T19:50:08.581 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ 2023-10-16T19:50:08.582 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/af_alg.ko 2023-10-16T19:50:08.582 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/algif_hash.ko 2023-10-16T19:50:08.582 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/algif_skcipher.ko 2023-10-16T19:50:08.582 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ansi_cprng.ko 2023-10-16T19:50:08.583 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/anubis.ko 2023-10-16T19:50:08.583 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/arc4.ko 2023-10-16T19:50:08.583 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/asymmetric_keys/ 2023-10-16T19:50:08.583 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2023-10-16T19:50:08.583 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/ 2023-10-16T19:50:08.585 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-16T19:50:08.585 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_pq.ko 2023-10-16T19:50:08.586 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-16T19:50:08.586 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_tx.ko 2023-10-16T19:50:08.586 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_xor.ko 2023-10-16T19:50:08.586 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/raid6test.ko 2023-10-16T19:50:08.587 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authenc.ko 2023-10-16T19:50:08.587 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authencesn.ko 2023-10-16T19:50:08.587 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blake2b_generic.ko 2023-10-16T19:50:08.587 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_common.ko 2023-10-16T19:50:08.587 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_generic.ko 2023-10-16T19:50:08.588 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/camellia_generic.ko 2023-10-16T19:50:08.588 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast5_generic.ko 2023-10-16T19:50:08.588 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast6_generic.ko 2023-10-16T19:50:08.588 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast_common.ko 2023-10-16T19:50:08.589 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ccm.ko 2023-10-16T19:50:08.589 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cmac.ko 2023-10-16T19:50:08.589 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crc32_generic.ko 2023-10-16T19:50:08.589 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cryptd.ko 2023-10-16T19:50:08.589 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_engine.ko 2023-10-16T19:50:08.590 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_simd.ko 2023-10-16T19:50:08.590 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_user.ko 2023-10-16T19:50:08.590 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cts.ko 2023-10-16T19:50:08.590 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/des_generic.ko 2023-10-16T19:50:08.590 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/dh_generic.ko 2023-10-16T19:50:08.591 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/echainiv.ko 2023-10-16T19:50:08.591 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/essiv.ko 2023-10-16T19:50:08.591 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/fcrypt.ko 2023-10-16T19:50:08.591 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/khazad.ko 2023-10-16T19:50:08.591 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lrw.ko 2023-10-16T19:50:08.592 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4.ko 2023-10-16T19:50:08.592 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4hc.ko 2023-10-16T19:50:08.592 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/md4.ko 2023-10-16T19:50:08.592 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/michael_mic.ko 2023-10-16T19:50:08.593 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcbc.ko 2023-10-16T19:50:08.593 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcrypt.ko 2023-10-16T19:50:08.593 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/rmd160.ko 2023-10-16T19:50:08.593 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/seed.ko 2023-10-16T19:50:08.593 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/serpent_generic.ko 2023-10-16T19:50:08.594 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 149193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tcrypt.ko 2023-10-16T19:50:08.594 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tea.ko 2023-10-16T19:50:08.594 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_common.ko 2023-10-16T19:50:08.594 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_generic.ko 2023-10-16T19:50:08.594 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/vmac.ko 2023-10-16T19:50:08.595 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/wp512.ko 2023-10-16T19:50:08.595 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xcbc.ko 2023-10-16T19:50:08.595 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xor.ko 2023-10-16T19:50:08.595 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xts.ko 2023-10-16T19:50:08.596 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xxhash_generic.ko 2023-10-16T19:50:08.596 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ 2023-10-16T19:50:08.596 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/ 2023-10-16T19:50:08.596 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_extlog.ko 2023-10-16T19:50:08.596 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-16T19:50:08.597 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_pad.ko 2023-10-16T19:50:08.597 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/apei/ 2023-10-16T19:50:08.597 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/apei/einj.ko 2023-10-16T19:50:08.597 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/ec_sys.ko 2023-10-16T19:50:08.597 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/platform_profile.ko 2023-10-16T19:50:08.598 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbs.ko 2023-10-16T19:50:08.598 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbshc.ko 2023-10-16T19:50:08.598 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 103025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/video.ko 2023-10-16T19:50:08.598 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ 2023-10-16T19:50:08.598 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/acard-ahci.ko 2023-10-16T19:50:08.599 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 88865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ahci.ko 2023-10-16T19:50:08.599 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ahci_platform.ko 2023-10-16T19:50:08.599 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 88289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/libahci.ko 2023-10-16T19:50:08.599 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/libahci_platform.ko 2023-10-16T19:50:08.599 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_acpi.ko 2023-10-16T19:50:08.600 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ali.ko 2023-10-16T19:50:08.600 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_amd.ko 2023-10-16T19:50:08.600 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_artop.ko 2023-10-16T19:50:08.600 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atiixp.ko 2023-10-16T19:50:08.600 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atp867x.ko 2023-10-16T19:50:08.601 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd640.ko 2023-10-16T19:50:08.601 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-16T19:50:08.601 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cypress.ko 2023-10-16T19:50:08.601 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_efar.ko 2023-10-16T19:50:08.602 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt366.ko 2023-10-16T19:50:08.602 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-16T19:50:08.602 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-16T19:50:08.602 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-16T19:50:08.602 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it8213.ko 2023-10-16T19:50:08.603 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it821x.ko 2023-10-16T19:50:08.603 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_jmicron.ko 2023-10-16T19:50:08.603 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 51481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_legacy.ko 2023-10-16T19:50:08.603 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_marvell.ko 2023-10-16T19:50:08.603 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_mpiix.ko 2023-10-16T19:50:08.604 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_netcell.ko 2023-10-16T19:50:08.604 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ninja32.ko 2023-10-16T19:50:08.604 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87410.ko 2023-10-16T19:50:08.605 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87415.ko 2023-10-16T19:50:08.605 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-16T19:50:08.605 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_opti.ko 2023-10-16T19:50:08.605 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_optidma.ko 2023-10-16T19:50:08.605 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-16T19:50:08.606 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-16T19:50:08.606 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-16T19:50:08.606 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_piccolo.ko 2023-10-16T19:50:08.606 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_radisys.ko 2023-10-16T19:50:08.606 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_rdc.ko 2023-10-16T19:50:08.607 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_rz1000.ko 2023-10-16T19:50:08.607 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sch.ko 2023-10-16T19:50:08.607 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_serverworks.ko 2023-10-16T19:50:08.607 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sil680.ko 2023-10-16T19:50:08.607 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-16T19:50:08.608 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_triflex.ko 2023-10-16T19:50:08.608 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_via.ko 2023-10-16T19:50:08.608 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pdc_adma.ko 2023-10-16T19:50:08.608 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_inic162x.ko 2023-10-16T19:50:08.609 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 83345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_mv.ko 2023-10-16T19:50:08.609 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 70393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_nv.ko 2023-10-16T19:50:08.609 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_promise.ko 2023-10-16T19:50:08.609 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_qstor.ko 2023-10-16T19:50:08.609 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil.ko 2023-10-16T19:50:08.610 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil24.ko 2023-10-16T19:50:08.610 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sis.ko 2023-10-16T19:50:08.610 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_svw.ko 2023-10-16T19:50:08.610 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sx4.ko 2023-10-16T19:50:08.610 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_uli.ko 2023-10-16T19:50:08.610 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_via.ko 2023-10-16T19:50:08.611 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_vsc.ko 2023-10-16T19:50:08.611 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay/ 2023-10-16T19:50:08.611 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-16T19:50:08.611 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma/ 2023-10-16T19:50:08.611 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 124473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma/bcma.ko 2023-10-16T19:50:08.612 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/ 2023-10-16T19:50:08.612 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe/ 2023-10-16T19:50:08.612 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 113665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe/aoe.ko 2023-10-16T19:50:08.612 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd/ 2023-10-16T19:50:08.613 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 965585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd/drbd.ko 2023-10-16T19:50:08.613 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 173225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/floppy.ko 2023-10-16T19:50:08.613 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx/ 2023-10-16T19:50:08.613 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 94297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-16T19:50:08.613 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 115793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/nbd.ko 2023-10-16T19:50:08.613 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk/ 2023-10-16T19:50:08.614 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 110449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-16T19:50:08.614 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 98193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/pktcdvd.ko 2023-10-16T19:50:08.614 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 263769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/rbd.ko 2023-10-16T19:50:08.614 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback/ 2023-10-16T19:50:08.615 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 98145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-16T19:50:08.615 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram/ 2023-10-16T19:50:08.615 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 60217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram/zram.ko 2023-10-16T19:50:08.615 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ 2023-10-16T19:50:08.615 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/applicom.ko 2023-10-16T19:50:08.615 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hangcheck-timer.ko 2023-10-16T19:50:08.616 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/ 2023-10-16T19:50:08.616 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-16T19:50:08.616 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-16T19:50:08.616 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-16T19:50:08.616 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-16T19:50:08.616 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-16T19:50:08.617 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ 2023-10-16T19:50:08.617 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-16T19:50:08.617 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 148601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-16T19:50:08.617 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-16T19:50:08.618 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 139313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-16T19:50:08.618 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 70921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-16T19:50:08.618 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-16T19:50:08.618 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/lp.ko 2023-10-16T19:50:08.619 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave/ 2023-10-16T19:50:08.619 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 84097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave/mwave.ko 2023-10-16T19:50:08.619 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/nvram.ko 2023-10-16T19:50:08.619 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ppdev.ko 2023-10-16T19:50:08.619 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 51113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tlclk.ko 2023-10-16T19:50:08.620 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/ 2023-10-16T19:50:08.620 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-16T19:50:08.620 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2023-10-16T19:50:08.620 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2023-10-16T19:50:08.620 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2023-10-16T19:50:08.621 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-16T19:50:08.621 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-16T19:50:08.621 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/xen-tpmfront.ko 2023-10-16T19:50:08.621 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/xillybus/ 2023-10-16T19:50:08.621 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/xillybus/xillybus_class.ko 2023-10-16T19:50:08.622 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 50945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/xillybus/xillybus_core.ko 2023-10-16T19:50:08.622 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/xillybus/xillybus_pcie.ko 2023-10-16T19:50:08.622 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/ 2023-10-16T19:50:08.622 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/clk-palmas.ko 2023-10-16T19:50:08.623 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/clk-si5351.ko 2023-10-16T19:50:08.623 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/clk-twl6040.ko 2023-10-16T19:50:08.623 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/clk-wm831x.ko 2023-10-16T19:50:08.623 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/ 2023-10-16T19:50:08.623 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-16T19:50:08.624 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-16T19:50:08.624 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-16T19:50:08.624 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ 2023-10-16T19:50:08.624 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ 2023-10-16T19:50:08.624 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 70801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-16T19:50:08.625 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 243025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-16T19:50:08.625 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/ 2023-10-16T19:50:08.625 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/ 2023-10-16T19:50:08.625 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common/ 2023-10-16T19:50:08.625 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 438761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-16T19:50:08.625 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2023-10-16T19:50:08.626 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-16T19:50:08.626 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-aes.ko 2023-10-16T19:50:08.626 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-sha.ko 2023-10-16T19:50:08.626 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio/ 2023-10-16T19:50:08.627 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 68857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-16T19:50:08.627 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca/ 2023-10-16T19:50:08.627 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca/dca.ko 2023-10-16T19:50:08.627 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ 2023-10-16T19:50:08.627 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw/ 2023-10-16T19:50:08.628 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-16T19:50:08.628 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat/ 2023-10-16T19:50:08.628 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 122569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-16T19:50:08.628 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/ 2023-10-16T19:50:08.629 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 79953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/amd64_edac.ko 2023-10-16T19:50:08.629 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/e752x_edac.ko 2023-10-16T19:50:08.629 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 63617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-16T19:50:08.629 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3000_edac.ko 2023-10-16T19:50:08.630 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3200_edac.ko 2023-10-16T19:50:08.630 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5100_edac.ko 2023-10-16T19:50:08.630 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5400_edac.ko 2023-10-16T19:50:08.630 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7300_edac.ko 2023-10-16T19:50:08.630 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 52041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7core_edac.ko 2023-10-16T19:50:08.631 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i82975x_edac.ko 2023-10-16T19:50:08.631 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/ie31200_edac.ko 2023-10-16T19:50:08.631 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 56825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/sb_edac.ko 2023-10-16T19:50:08.631 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/x38_edac.ko 2023-10-16T19:50:08.631 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/ 2023-10-16T19:50:08.631 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-gpio.ko 2023-10-16T19:50:08.632 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-max14577.ko 2023-10-16T19:50:08.632 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-max77693.ko 2023-10-16T19:50:08.632 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-max8997.ko 2023-10-16T19:50:08.633 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-palmas.ko 2023-10-16T19:50:08.633 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-rt8973a.ko 2023-10-16T19:50:08.633 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-sm5502.ko 2023-10-16T19:50:08.633 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/ 2023-10-16T19:50:08.633 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/dmi-sysfs.ko 2023-10-16T19:50:08.634 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/efi/ 2023-10-16T19:50:08.634 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/efi/efi-pstore.ko 2023-10-16T19:50:08.634 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-16T19:50:08.634 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/ 2023-10-16T19:50:08.634 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-adp5520.ko 2023-10-16T19:50:08.635 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-amd8111.ko 2023-10-16T19:50:08.635 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-arizona.ko 2023-10-16T19:50:08.635 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-da9052.ko 2023-10-16T19:50:08.635 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-da9055.ko 2023-10-16T19:50:08.635 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-dln2.ko 2023-10-16T19:50:08.636 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 77849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-f7188x.ko 2023-10-16T19:50:08.636 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-generic.ko 2023-10-16T19:50:08.636 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-ich.ko 2023-10-16T19:50:08.636 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-janz-ttl.ko 2023-10-16T19:50:08.636 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-kempld.ko 2023-10-16T19:50:08.637 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-lp3943.ko 2023-10-16T19:50:08.637 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-max7300.ko 2023-10-16T19:50:08.637 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-max730x.ko 2023-10-16T19:50:08.637 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-max732x.ko 2023-10-16T19:50:08.637 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-ml-ioh.ko 2023-10-16T19:50:08.638 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-pca953x.ko 2023-10-16T19:50:08.638 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-pcf857x.ko 2023-10-16T19:50:08.638 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-rdc321x.ko 2023-10-16T19:50:08.638 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-sch.ko 2023-10-16T19:50:08.638 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-sch311x.ko 2023-10-16T19:50:08.639 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-tps65912.ko 2023-10-16T19:50:08.639 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-twl4030.ko 2023-10-16T19:50:08.639 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-twl6040.ko 2023-10-16T19:50:08.639 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-16T19:50:08.639 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-vx855.ko 2023-10-16T19:50:08.640 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-wm831x.ko 2023-10-16T19:50:08.640 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-wm8350.ko 2023-10-16T19:50:08.640 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-wm8994.ko 2023-10-16T19:50:08.640 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/ 2023-10-16T19:50:08.640 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-a4tech.ko 2023-10-16T19:50:08.641 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-apple.ko 2023-10-16T19:50:08.641 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-appleir.ko 2023-10-16T19:50:08.641 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-aureal.ko 2023-10-16T19:50:08.641 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-axff.ko 2023-10-16T19:50:08.641 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-belkin.ko 2023-10-16T19:50:08.642 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-cherry.ko 2023-10-16T19:50:08.642 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-chicony.ko 2023-10-16T19:50:08.642 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 60649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-cp2112.ko 2023-10-16T19:50:08.642 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-cypress.ko 2023-10-16T19:50:08.642 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-dr.ko 2023-10-16T19:50:08.643 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elecom.ko 2023-10-16T19:50:08.643 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elo.ko 2023-10-16T19:50:08.643 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-emsff.ko 2023-10-16T19:50:08.643 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-ezkey.ko 2023-10-16T19:50:08.643 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gaff.ko 2023-10-16T19:50:08.644 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-generic.ko 2023-10-16T19:50:08.644 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gt683r.ko 2023-10-16T19:50:08.644 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gyration.ko 2023-10-16T19:50:08.644 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-16T19:50:08.644 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-16T19:50:08.644 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtekff.ko 2023-10-16T19:50:08.645 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-hyperv.ko 2023-10-16T19:50:08.645 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-icade.ko 2023-10-16T19:50:08.645 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-kensington.ko 2023-10-16T19:50:08.645 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-keytouch.ko 2023-10-16T19:50:08.645 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-kye.ko 2023-10-16T19:50:08.646 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lcpower.ko 2023-10-16T19:50:08.646 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-led.ko 2023-10-16T19:50:08.646 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lenovo.ko 2023-10-16T19:50:08.646 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lg-g15.ko 2023-10-16T19:50:08.646 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 58185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-16T19:50:08.647 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 101369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-16T19:50:08.647 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 80993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech.ko 2023-10-16T19:50:08.647 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-magicmouse.ko 2023-10-16T19:50:08.647 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-microsoft.ko 2023-10-16T19:50:08.647 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-monterey.ko 2023-10-16T19:50:08.647 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 50161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-multitouch.ko 2023-10-16T19:50:08.648 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-ntrig.ko 2023-10-16T19:50:08.648 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-ortek.ko 2023-10-16T19:50:08.648 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-penmount.ko 2023-10-16T19:50:08.648 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-petalynx.ko 2023-10-16T19:50:08.649 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 72937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-picolcd.ko 2023-10-16T19:50:08.649 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-pl.ko 2023-10-16T19:50:08.649 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-plantronics.ko 2023-10-16T19:50:08.649 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-primax.ko 2023-10-16T19:50:08.649 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-rmi.ko 2023-10-16T19:50:08.650 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-16T19:50:08.650 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-16T19:50:08.650 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-16T19:50:08.650 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-16T19:50:08.650 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-16T19:50:08.651 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-16T19:50:08.651 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-16T19:50:08.651 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-16T19:50:08.651 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-16T19:50:08.651 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-16T19:50:08.652 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-16T19:50:08.652 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat.ko 2023-10-16T19:50:08.652 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-saitek.ko 2023-10-16T19:50:08.652 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-samsung.ko 2023-10-16T19:50:08.652 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-16T19:50:08.652 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sjoy.ko 2023-10-16T19:50:08.653 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 51249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sony.ko 2023-10-16T19:50:08.653 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-speedlink.ko 2023-10-16T19:50:08.653 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-steelseries.ko 2023-10-16T19:50:08.653 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sunplus.ko 2023-10-16T19:50:08.653 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-16T19:50:08.654 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tivo.ko 2023-10-16T19:50:08.654 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tmff.ko 2023-10-16T19:50:08.654 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-topseed.ko 2023-10-16T19:50:08.654 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-twinhan.ko 2023-10-16T19:50:08.654 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 71073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-uclogic.ko 2023-10-16T19:50:08.655 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-waltop.ko 2023-10-16T19:50:08.655 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 143473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-wiimote.ko 2023-10-16T19:50:08.655 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-xinmo.ko 2023-10-16T19:50:08.655 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zpff.ko 2023-10-16T19:50:08.655 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zydacron.ko 2023-10-16T19:50:08.656 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 267825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid.ko 2023-10-16T19:50:08.656 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/uhid.ko 2023-10-16T19:50:08.656 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/usbhid/ 2023-10-16T19:50:08.656 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 125721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/usbhid/usbhid.ko 2023-10-16T19:50:08.656 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/usbhid/usbkbd.ko 2023-10-16T19:50:08.657 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/usbhid/usbmouse.ko 2023-10-16T19:50:08.657 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 217873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/wacom.ko 2023-10-16T19:50:08.657 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/ 2023-10-16T19:50:08.657 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 64201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_balloon.ko 2023-10-16T19:50:08.657 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 89625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_utils.ko 2023-10-16T19:50:08.658 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 305465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_vmbus.ko 2023-10-16T19:50:08.658 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ 2023-10-16T19:50:08.658 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 66521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru.ko 2023-10-16T19:50:08.658 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 61393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru3.ko 2023-10-16T19:50:08.658 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-16T19:50:08.659 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7414.ko 2023-10-16T19:50:08.659 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7418.ko 2023-10-16T19:50:08.659 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adc128d818.ko 2023-10-16T19:50:08.659 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1025.ko 2023-10-16T19:50:08.659 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 97977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1026.ko 2023-10-16T19:50:08.660 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1029.ko 2023-10-16T19:50:08.660 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 45553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1031.ko 2023-10-16T19:50:08.660 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm9240.ko 2023-10-16T19:50:08.660 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ads7828.ko 2023-10-16T19:50:08.660 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7410.ko 2023-10-16T19:50:08.661 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7411.ko 2023-10-16T19:50:08.661 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 74433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7462.ko 2023-10-16T19:50:08.661 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7470.ko 2023-10-16T19:50:08.661 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 67929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7475.ko 2023-10-16T19:50:08.661 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7x10.ko 2023-10-16T19:50:08.662 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/amc6821.ko 2023-10-16T19:50:08.662 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 55265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/applesmc.ko 2023-10-16T19:50:08.662 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asb100.ko 2023-10-16T19:50:08.662 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 58617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asc7621.ko 2023-10-16T19:50:08.662 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-16T19:50:08.663 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/atxp1.ko 2023-10-16T19:50:08.663 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/coretemp.ko 2023-10-16T19:50:08.663 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/da9052-hwmon.ko 2023-10-16T19:50:08.663 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/da9055-hwmon.ko 2023-10-16T19:50:08.663 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-16T19:50:08.663 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 99601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dme1737.ko 2023-10-16T19:50:08.664 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds1621.ko 2023-10-16T19:50:08.664 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds620.ko 2023-10-16T19:50:08.664 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc1403.ko 2023-10-16T19:50:08.664 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc2103.ko 2023-10-16T19:50:08.664 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc6w201.ko 2023-10-16T19:50:08.665 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 66985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71805f.ko 2023-10-16T19:50:08.665 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 94457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71882fg.ko 2023-10-16T19:50:08.665 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f75375s.ko 2023-10-16T19:50:08.665 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-16T19:50:08.665 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 55449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fschmd.ko 2023-10-16T19:50:08.665 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g760a.ko 2023-10-16T19:50:08.666 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g762.ko 2023-10-16T19:50:08.666 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 46801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl518sm.ko 2023-10-16T19:50:08.666 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 45529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl520sm.ko 2023-10-16T19:50:08.666 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/hih6130.ko 2023-10-16T19:50:08.666 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-16T19:50:08.667 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-16T19:50:08.667 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-16T19:50:08.667 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmaem.ko 2023-10-16T19:50:08.667 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmpex.ko 2023-10-16T19:50:08.667 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina209.ko 2023-10-16T19:50:08.668 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina2xx.ko 2023-10-16T19:50:08.668 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 130449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/it87.ko 2023-10-16T19:50:08.668 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/jc42.ko 2023-10-16T19:50:08.668 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k10temp.ko 2023-10-16T19:50:08.668 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k8temp.ko 2023-10-16T19:50:08.669 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-16T19:50:08.669 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 48601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm63.ko 2023-10-16T19:50:08.669 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm73.ko 2023-10-16T19:50:08.669 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm75.ko 2023-10-16T19:50:08.669 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm77.ko 2023-10-16T19:50:08.670 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm78.ko 2023-10-16T19:50:08.670 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm80.ko 2023-10-16T19:50:08.670 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm83.ko 2023-10-16T19:50:08.670 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 72833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm85.ko 2023-10-16T19:50:08.670 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 46153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm87.ko 2023-10-16T19:50:08.671 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm90.ko 2023-10-16T19:50:08.671 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm92.ko 2023-10-16T19:50:08.671 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 108841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm93.ko 2023-10-16T19:50:08.671 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95234.ko 2023-10-16T19:50:08.671 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95241.ko 2023-10-16T19:50:08.671 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95245.ko 2023-10-16T19:50:08.672 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc2945.ko 2023-10-16T19:50:08.672 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4151.ko 2023-10-16T19:50:08.672 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4215.ko 2023-10-16T19:50:08.672 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4222.ko 2023-10-16T19:50:08.672 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4245.ko 2023-10-16T19:50:08.673 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4260.ko 2023-10-16T19:50:08.673 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4261.ko 2023-10-16T19:50:08.673 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max16065.ko 2023-10-16T19:50:08.673 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1619.ko 2023-10-16T19:50:08.673 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1668.ko 2023-10-16T19:50:08.674 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max197.ko 2023-10-16T19:50:08.674 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6639.ko 2023-10-16T19:50:08.674 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6650.ko 2023-10-16T19:50:08.674 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6697.ko 2023-10-16T19:50:08.674 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/mc13783-adc.ko 2023-10-16T19:50:08.675 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/mcp3021.ko 2023-10-16T19:50:08.675 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2023-10-16T19:50:08.675 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6683.ko 2023-10-16T19:50:08.675 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 119817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-16T19:50:08.675 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 60705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775.ko 2023-10-16T19:50:08.676 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 59737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct7802.ko 2023-10-16T19:50:08.676 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 82337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87360.ko 2023-10-16T19:50:08.676 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 54433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87427.ko 2023-10-16T19:50:08.676 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pcf8591.ko 2023-10-16T19:50:08.676 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ 2023-10-16T19:50:08.677 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-16T19:50:08.677 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-16T19:50:08.677 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-16T19:50:08.677 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-16T19:50:08.677 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-16T19:50:08.677 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-16T19:50:08.678 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-16T19:50:08.678 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 82577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-16T19:50:08.678 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-16T19:50:08.678 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-16T19:50:08.678 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-16T19:50:08.679 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-16T19:50:08.679 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/powr1220.ko 2023-10-16T19:50:08.679 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5627.ko 2023-10-16T19:50:08.679 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5636.ko 2023-10-16T19:50:08.679 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-16T19:50:08.679 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht15.ko 2023-10-16T19:50:08.680 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht21.ko 2023-10-16T19:50:08.680 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/shtc1.ko 2023-10-16T19:50:08.680 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sis5595.ko 2023-10-16T19:50:08.680 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-16T19:50:08.680 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-16T19:50:08.680 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-16T19:50:08.681 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/thmc50.ko 2023-10-16T19:50:08.681 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp102.ko 2023-10-16T19:50:08.681 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp103.ko 2023-10-16T19:50:08.681 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp401.ko 2023-10-16T19:50:08.681 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp421.ko 2023-10-16T19:50:08.682 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-16T19:50:08.682 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via686a.ko 2023-10-16T19:50:08.682 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 52457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt1211.ko 2023-10-16T19:50:08.682 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 51545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt8231.ko 2023-10-16T19:50:08.682 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 57513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-16T19:50:08.683 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 78873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627hf.ko 2023-10-16T19:50:08.683 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 71345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83781d.ko 2023-10-16T19:50:08.683 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 63289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83791d.ko 2023-10-16T19:50:08.683 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 71321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83792d.ko 2023-10-16T19:50:08.683 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 101473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83793.ko 2023-10-16T19:50:08.684 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 119505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83795.ko 2023-10-16T19:50:08.684 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-16T19:50:08.684 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-16T19:50:08.684 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/wm831x-hwmon.ko 2023-10-16T19:50:08.684 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/wm8350-hwmon.ko 2023-10-16T19:50:08.685 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/ 2023-10-16T19:50:08.685 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/ 2023-10-16T19:50:08.685 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-16T19:50:08.685 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-16T19:50:08.685 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/ 2023-10-16T19:50:08.686 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ali1535.ko 2023-10-16T19:50:08.686 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ali1563.ko 2023-10-16T19:50:08.686 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2023-10-16T19:50:08.686 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-16T19:50:08.687 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-16T19:50:08.687 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-16T19:50:08.687 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2023-10-16T19:50:08.687 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 6929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-16T19:50:08.687 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2023-10-16T19:50:08.688 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 50201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-16T19:50:08.688 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-16T19:50:08.688 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-16T19:50:08.688 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-16T19:50:08.688 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-dln2.ko 2023-10-16T19:50:08.688 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-gpio.ko 2023-10-16T19:50:08.689 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 62817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-16T19:50:08.689 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-16T19:50:08.689 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-16T19:50:08.689 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-kempld.ko 2023-10-16T19:50:08.689 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-16T19:50:08.690 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-16T19:50:08.690 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ocores.ko 2023-10-16T19:50:08.690 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-16T19:50:08.690 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-16T19:50:08.690 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-16T19:50:08.691 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2023-10-16T19:50:08.691 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-16T19:50:08.691 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-16T19:50:08.691 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-sis5595.ko 2023-10-16T19:50:08.691 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-sis630.ko 2023-10-16T19:50:08.692 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-16T19:50:08.692 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2023-10-16T19:50:08.692 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-16T19:50:08.692 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-16T19:50:08.692 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-16T19:50:08.693 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-16T19:50:08.693 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-xiic.ko 2023-10-16T19:50:08.693 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-mux.ko 2023-10-16T19:50:08.693 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-16T19:50:08.693 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-stub.ko 2023-10-16T19:50:08.694 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/muxes/ 2023-10-16T19:50:08.694 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2023-10-16T19:50:08.694 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2023-10-16T19:50:08.694 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2023-10-16T19:50:08.694 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ 2023-10-16T19:50:08.695 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ 2023-10-16T19:50:08.695 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 273609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-16T19:50:08.695 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 879969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-16T19:50:08.695 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 69337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-16T19:50:08.695 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 324217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-16T19:50:08.696 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 97985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-16T19:50:08.696 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 282273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-16T19:50:08.696 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 66201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-16T19:50:08.696 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/ 2023-10-16T19:50:08.696 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4/ 2023-10-16T19:50:08.697 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 493073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-16T19:50:08.697 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4/ 2023-10-16T19:50:08.697 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 419737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-16T19:50:08.697 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5/ 2023-10-16T19:50:08.697 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 739025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-16T19:50:08.698 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca/ 2023-10-16T19:50:08.698 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 296625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-16T19:50:08.698 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/ocrdma/ 2023-10-16T19:50:08.698 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 153697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2023-10-16T19:50:08.699 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/usnic/ 2023-10-16T19:50:08.699 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 168969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2023-10-16T19:50:08.699 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ 2023-10-16T19:50:08.699 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib/ 2023-10-16T19:50:08.699 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 263081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-16T19:50:08.699 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser/ 2023-10-16T19:50:08.700 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 100241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-16T19:50:08.700 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert/ 2023-10-16T19:50:08.700 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 111425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-16T19:50:08.700 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp/ 2023-10-16T19:50:08.700 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 128753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-16T19:50:08.701 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt/ 2023-10-16T19:50:08.701 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 129529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-16T19:50:08.701 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/ 2023-10-16T19:50:08.701 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/evbug.ko 2023-10-16T19:50:08.701 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/ff-memless.ko 2023-10-16T19:50:08.702 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/ 2023-10-16T19:50:08.702 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-16T19:50:08.702 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-16T19:50:08.702 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/gameport.ko 2023-10-16T19:50:08.702 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/joydev.ko 2023-10-16T19:50:08.703 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/ 2023-10-16T19:50:08.703 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/adp5520-keys.ko 2023-10-16T19:50:08.703 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/adp5588-keys.ko 2023-10-16T19:50:08.703 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/adp5589-keys.ko 2023-10-16T19:50:08.703 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2023-10-16T19:50:08.704 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-16T19:50:08.704 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2023-10-16T19:50:08.704 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/lkkbd.ko 2023-10-16T19:50:08.704 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/lm8323.ko 2023-10-16T19:50:08.704 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/lm8333.ko 2023-10-16T19:50:08.705 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/matrix_keypad.ko 2023-10-16T19:50:08.705 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/max7359_keypad.ko 2023-10-16T19:50:08.705 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/mcs_touchkey.ko 2023-10-16T19:50:08.705 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2023-10-16T19:50:08.705 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/newtonkbd.ko 2023-10-16T19:50:08.706 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/opencores-kbd.ko 2023-10-16T19:50:08.706 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/qt1070.ko 2023-10-16T19:50:08.706 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/qt2160.ko 2023-10-16T19:50:08.706 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/samsung-keypad.ko 2023-10-16T19:50:08.706 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/stowaway.ko 2023-10-16T19:50:08.707 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/sunkbd.ko 2023-10-16T19:50:08.707 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/tca6416-keypad.ko 2023-10-16T19:50:08.707 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/tca8418_keypad.ko 2023-10-16T19:50:08.707 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/twl4030_keypad.ko 2023-10-16T19:50:08.707 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/xtkbd.ko 2023-10-16T19:50:08.708 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/matrix-keymap.ko 2023-10-16T19:50:08.708 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/ 2023-10-16T19:50:08.708 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/appletouch.ko 2023-10-16T19:50:08.708 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-16T19:50:08.709 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 123681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-16T19:50:08.709 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 89449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-16T19:50:08.709 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/gpio_mouse.ko 2023-10-16T19:50:08.709 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 335977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/psmouse.ko 2023-10-16T19:50:08.709 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/sermouse.ko 2023-10-16T19:50:08.709 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-16T19:50:08.710 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-16T19:50:08.710 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-16T19:50:08.710 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4/ 2023-10-16T19:50:08.710 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 109873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-16T19:50:08.710 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/ 2023-10-16T19:50:08.711 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-16T19:50:08.711 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-16T19:50:08.711 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/ct82c710.ko 2023-10-16T19:50:08.711 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-16T19:50:08.711 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/parkbd.ko 2023-10-16T19:50:08.712 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/pcips2.ko 2023-10-16T19:50:08.712 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/ps2mult.ko 2023-10-16T19:50:08.712 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/serio_raw.ko 2023-10-16T19:50:08.712 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/serport.ko 2023-10-16T19:50:08.712 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/sparse-keymap.ko 2023-10-16T19:50:08.713 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/ 2023-10-16T19:50:08.713 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd/ 2023-10-16T19:50:08.713 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-16T19:50:08.713 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/ 2023-10-16T19:50:08.713 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-88pm860x.ko 2023-10-16T19:50:08.714 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-adp5520.ko 2023-10-16T19:50:08.714 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 59753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-bd2802.ko 2023-10-16T19:50:08.714 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-blinkm.ko 2023-10-16T19:50:08.714 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-da903x.ko 2023-10-16T19:50:08.714 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-da9052.ko 2023-10-16T19:50:08.715 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-gpio.ko 2023-10-16T19:50:08.715 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm3530.ko 2023-10-16T19:50:08.715 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm3533.ko 2023-10-16T19:50:08.715 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm355x.ko 2023-10-16T19:50:08.715 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm3642.ko 2023-10-16T19:50:08.715 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lp3944.ko 2023-10-16T19:50:08.716 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lp8788.ko 2023-10-16T19:50:08.716 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-max8997.ko 2023-10-16T19:50:08.716 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-mc13783.ko 2023-10-16T19:50:08.716 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-menf21bmc.ko 2023-10-16T19:50:08.716 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-pca9532.ko 2023-10-16T19:50:08.716 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-pca955x.ko 2023-10-16T19:50:08.717 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-pca963x.ko 2023-10-16T19:50:08.717 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-ss4200.ko 2023-10-16T19:50:08.717 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-tca6507.ko 2023-10-16T19:50:08.717 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-wm831x-status.ko 2023-10-16T19:50:08.717 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-wm8350.ko 2023-10-16T19:50:08.718 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ 2023-10-16T19:50:08.718 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-16T19:50:08.718 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-16T19:50:08.718 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-16T19:50:08.718 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-16T19:50:08.719 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-16T19:50:08.719 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-16T19:50:08.719 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-16T19:50:08.719 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-16T19:50:08.719 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/ 2023-10-16T19:50:08.719 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache/ 2023-10-16T19:50:08.720 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 654969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache/bcache.ko 2023-10-16T19:50:08.720 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-bio-prison.ko 2023-10-16T19:50:08.720 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 89601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-bufio.ko 2023-10-16T19:50:08.720 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache-smq.ko 2023-10-16T19:50:08.720 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 156009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache.ko 2023-10-16T19:50:08.721 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 106897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-crypt.ko 2023-10-16T19:50:08.721 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-delay.ko 2023-10-16T19:50:08.721 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-era.ko 2023-10-16T19:50:08.721 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-flakey.ko 2023-10-16T19:50:08.721 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-log-userspace.ko 2023-10-16T19:50:08.722 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-log.ko 2023-10-16T19:50:08.722 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-mirror.ko 2023-10-16T19:50:08.722 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 78033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-multipath.ko 2023-10-16T19:50:08.722 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-queue-length.ko 2023-10-16T19:50:08.722 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 70825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-raid.ko 2023-10-16T19:50:08.723 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-region-hash.ko 2023-10-16T19:50:08.723 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-round-robin.ko 2023-10-16T19:50:08.723 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-service-time.ko 2023-10-16T19:50:08.723 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 109593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-snapshot.ko 2023-10-16T19:50:08.723 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-switch.ko 2023-10-16T19:50:08.724 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 181865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-thin-pool.ko 2023-10-16T19:50:08.724 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 51737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-verity.ko 2023-10-16T19:50:08.724 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-zero.ko 2023-10-16T19:50:08.724 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/faulty.ko 2023-10-16T19:50:08.724 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/linear.ko 2023-10-16T19:50:08.725 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/multipath.ko 2023-10-16T19:50:08.725 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data/ 2023-10-16T19:50:08.725 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 199937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-16T19:50:08.725 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid0.ko 2023-10-16T19:50:08.725 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 113073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid1.ko 2023-10-16T19:50:08.725 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 158681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid10.ko 2023-10-16T19:50:08.726 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 395721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid456.ko 2023-10-16T19:50:08.726 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/ 2023-10-16T19:50:08.726 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/ 2023-10-16T19:50:08.726 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 178137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptbase.ko 2023-10-16T19:50:08.726 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 63385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptctl.ko 2023-10-16T19:50:08.727 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptfc.ko 2023-10-16T19:50:08.727 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptlan.ko 2023-10-16T19:50:08.727 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 123321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptsas.ko 2023-10-16T19:50:08.727 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 79041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-16T19:50:08.728 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptspi.ko 2023-10-16T19:50:08.728 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/ 2023-10-16T19:50:08.728 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/88pm800.ko 2023-10-16T19:50:08.728 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/88pm805.ko 2023-10-16T19:50:08.728 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/88pm80x.ko 2023-10-16T19:50:08.728 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/arizona-i2c.ko 2023-10-16T19:50:08.729 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 123721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/arizona.ko 2023-10-16T19:50:08.729 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/bcm590xx.ko 2023-10-16T19:50:08.729 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/cros_ec_dev.ko 2023-10-16T19:50:08.729 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/dln2.ko 2023-10-16T19:50:08.729 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/janz-cmodio.ko 2023-10-16T19:50:08.730 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/kempld-core.ko 2023-10-16T19:50:08.730 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lm3533-core.ko 2023-10-16T19:50:08.730 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lm3533-ctrlbank.ko 2023-10-16T19:50:08.730 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lp3943.ko 2023-10-16T19:50:08.730 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 54569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_ich.ko 2023-10-16T19:50:08.731 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_sch.ko 2023-10-16T19:50:08.731 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/max8907.ko 2023-10-16T19:50:08.731 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/mc13xxx-core.ko 2023-10-16T19:50:08.731 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/mc13xxx-i2c.ko 2023-10-16T19:50:08.731 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/menf21bmc.ko 2023-10-16T19:50:08.732 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/pcf50633-adc.ko 2023-10-16T19:50:08.732 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/pcf50633-gpio.ko 2023-10-16T19:50:08.732 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/pcf50633.ko 2023-10-16T19:50:08.732 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/rdc321x-southbridge.ko 2023-10-16T19:50:08.732 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/retu-mfd.ko 2023-10-16T19:50:08.733 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 80425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/si476x-core.ko 2023-10-16T19:50:08.733 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 48457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/sm501.ko 2023-10-16T19:50:08.733 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/ti_am335x_tscadc.ko 2023-10-16T19:50:08.733 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 51137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/tps65010.ko 2023-10-16T19:50:08.733 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/tps6507x.ko 2023-10-16T19:50:08.733 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/viperboard.ko 2023-10-16T19:50:08.734 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/vx855.ko 2023-10-16T19:50:08.734 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/wl1273-core.ko 2023-10-16T19:50:08.734 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/wm8994.ko 2023-10-16T19:50:08.734 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ 2023-10-16T19:50:08.734 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ad525x_dpot-i2c.ko 2023-10-16T19:50:08.735 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 48489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ad525x_dpot.ko 2023-10-16T19:50:08.735 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl/ 2023-10-16T19:50:08.735 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 46585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-16T19:50:08.735 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds9802als.ko 2023-10-16T19:50:08.736 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds990x.ko 2023-10-16T19:50:08.736 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 45593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/bh1770glc.ko 2023-10-16T19:50:08.736 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/c2port/ 2023-10-16T19:50:08.736 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2023-10-16T19:50:08.736 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/c2port/core.ko 2023-10-16T19:50:08.736 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710/ 2023-10-16T19:50:08.737 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710/cb710.ko 2023-10-16T19:50:08.737 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ds1682.ko 2023-10-16T19:50:08.737 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/dummy-irq.ko 2023-10-16T19:50:08.737 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/echo/ 2023-10-16T19:50:08.737 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/echo/echo.ko 2023-10-16T19:50:08.738 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/ 2023-10-16T19:50:08.738 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/at24.ko 2023-10-16T19:50:08.738 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-16T19:50:08.738 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-16T19:50:08.738 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-16T19:50:08.739 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/enclosure.ko 2023-10-16T19:50:08.739 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/genwqe/ 2023-10-16T19:50:08.739 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 152201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/genwqe/genwqe_card.ko 2023-10-16T19:50:08.739 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/hmc6352.ko 2023-10-16T19:50:08.739 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/hpilo.ko 2023-10-16T19:50:08.739 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ibmasm/ 2023-10-16T19:50:08.740 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 72129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ibmasm/ibmasm.ko 2023-10-16T19:50:08.740 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ics932s401.ko 2023-10-16T19:50:08.740 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29003.ko 2023-10-16T19:50:08.740 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29020.ko 2023-10-16T19:50:08.740 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/ 2023-10-16T19:50:08.741 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-16T19:50:08.741 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-16T19:50:08.741 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/ 2023-10-16T19:50:08.741 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 125209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-me.ko 2023-10-16T19:50:08.741 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 56521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-16T19:50:08.742 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 310113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei.ko 2023-10-16T19:50:08.742 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/phantom.ko 2023-10-16T19:50:08.742 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ti-st/ 2023-10-16T19:50:08.742 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 74265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ti-st/st_drv.ko 2023-10-16T19:50:08.742 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-16T19:50:08.743 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_core.ko 2023-10-16T19:50:08.743 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tsl2550.ko 2023-10-16T19:50:08.743 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 48697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_balloon.ko 2023-10-16T19:50:08.743 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci/ 2023-10-16T19:50:08.743 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 183609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-16T19:50:08.744 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ 2023-10-16T19:50:08.744 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding/ 2023-10-16T19:50:08.744 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 490129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding/bonding.ko 2023-10-16T19:50:08.744 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/ 2023-10-16T19:50:08.744 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53/ 2023-10-16T19:50:08.744 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 106033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-16T19:50:08.745 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-16T19:50:08.745 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-16T19:50:08.745 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx/ 2023-10-16T19:50:08.745 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 453737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-16T19:50:08.745 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dummy.ko 2023-10-16T19:50:08.746 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/eql.ko 2023-10-16T19:50:08.746 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ 2023-10-16T19:50:08.746 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/ 2023-10-16T19:50:08.746 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 45785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-16T19:50:08.746 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-16T19:50:08.747 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 130553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-16T19:50:08.747 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 68849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-16T19:50:08.747 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/ 2023-10-16T19:50:08.747 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-16T19:50:08.747 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 59689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-16T19:50:08.748 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-16T19:50:08.748 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 99113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-16T19:50:08.748 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec/ 2023-10-16T19:50:08.748 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 54521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-16T19:50:08.748 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere/ 2023-10-16T19:50:08.749 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 69201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-16T19:50:08.749 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon/ 2023-10-16T19:50:08.749 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 59585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-16T19:50:08.749 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera/ 2023-10-16T19:50:08.749 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-16T19:50:08.750 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/ 2023-10-16T19:50:08.750 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 54609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-16T19:50:08.750 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-16T19:50:08.750 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 87297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-16T19:50:08.751 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/ 2023-10-16T19:50:08.751 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx/ 2023-10-16T19:50:08.751 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 97337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-16T19:50:08.751 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c/ 2023-10-16T19:50:08.751 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 104673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-16T19:50:08.752 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e/ 2023-10-16T19:50:08.752 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 90257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-16T19:50:08.752 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/ 2023-10-16T19:50:08.752 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 81569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-16T19:50:08.752 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 69729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-16T19:50:08.753 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/ 2023-10-16T19:50:08.753 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 78401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-16T19:50:08.753 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 176761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-16T19:50:08.753 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2023-10-16T19:50:08.753 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1421497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-16T19:50:08.754 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 128569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-16T19:50:08.754 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet/ 2023-10-16T19:50:08.754 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 132089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-16T19:50:08.754 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 312081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-16T19:50:08.754 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/ 2023-10-16T19:50:08.755 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna/ 2023-10-16T19:50:08.755 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 323513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-16T19:50:08.755 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/ 2023-10-16T19:50:08.755 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb/ 2023-10-16T19:50:08.755 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 193497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-16T19:50:08.755 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2023-10-16T19:50:08.756 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 347417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-16T19:50:08.756 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2023-10-16T19:50:08.756 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1044433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-16T19:50:08.756 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2023-10-16T19:50:08.756 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 166193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-16T19:50:08.757 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2023-10-16T19:50:08.757 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-16T19:50:08.757 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/ 2023-10-16T19:50:08.757 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic/ 2023-10-16T19:50:08.757 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 204345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-16T19:50:08.758 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/ 2023-10-16T19:50:08.758 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/ 2023-10-16T19:50:08.758 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 69449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-16T19:50:08.758 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 61489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-16T19:50:08.758 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 141225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-16T19:50:08.759 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 51497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-16T19:50:08.759 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 62817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-16T19:50:08.759 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-16T19:50:08.759 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/ 2023-10-16T19:50:08.759 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 60873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-16T19:50:08.760 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 62833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-16T19:50:08.760 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 45721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dnet.ko 2023-10-16T19:50:08.760 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ec_bhf.ko 2023-10-16T19:50:08.760 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/ 2023-10-16T19:50:08.760 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet/ 2023-10-16T19:50:08.761 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 281137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-16T19:50:08.761 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-16T19:50:08.761 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 51737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-16T19:50:08.761 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/fujitsu/ 2023-10-16T19:50:08.761 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 45345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2023-10-16T19:50:08.762 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ 2023-10-16T19:50:08.762 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 90465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-16T19:50:08.762 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000/ 2023-10-16T19:50:08.762 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 295577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-16T19:50:08.762 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e/ 2023-10-16T19:50:08.763 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 584497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-16T19:50:08.863 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k/ 2023-10-16T19:50:08.863 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 246209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-16T19:50:08.863 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e/ 2023-10-16T19:50:08.863 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 925697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-16T19:50:08.864 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf/ 2023-10-16T19:50:08.864 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 302849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-16T19:50:08.864 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb/ 2023-10-16T19:50:08.864 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 514681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-16T19:50:08.864 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf/ 2023-10-16T19:50:08.865 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 93593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-16T19:50:08.865 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe/ 2023-10-16T19:50:08.865 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 734305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-16T19:50:08.865 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf/ 2023-10-16T19:50:08.865 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 151769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-16T19:50:08.865 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 86921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/jme.ko 2023-10-16T19:50:08.866 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/ 2023-10-16T19:50:08.866 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-16T19:50:08.866 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 102289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-16T19:50:08.866 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 122649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-16T19:50:08.867 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/ 2023-10-16T19:50:08.867 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/ 2023-10-16T19:50:08.867 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 727601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-16T19:50:08.867 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 278937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-16T19:50:08.868 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/ 2023-10-16T19:50:08.868 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2023-10-16T19:50:08.868 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1113993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-16T19:50:08.868 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ 2023-10-16T19:50:08.868 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ks8842.ko 2023-10-16T19:50:08.869 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2023-10-16T19:50:08.869 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2023-10-16T19:50:08.869 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 95665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-16T19:50:08.869 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/ 2023-10-16T19:50:08.870 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge/ 2023-10-16T19:50:08.870 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 108649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-16T19:50:08.870 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/ 2023-10-16T19:50:08.870 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 73441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-16T19:50:08.870 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-16T19:50:08.871 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion/ 2023-10-16T19:50:08.871 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 150905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-16T19:50:08.871 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia/ 2023-10-16T19:50:08.871 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 136721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-16T19:50:08.871 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/ 2023-10-16T19:50:08.872 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 50481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-16T19:50:08.872 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-16T19:50:08.872 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/ 2023-10-16T19:50:08.872 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen/ 2023-10-16T19:50:08.872 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 227897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-16T19:50:08.872 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 87081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-16T19:50:08.873 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2023-10-16T19:50:08.873 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 672049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-16T19:50:08.873 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc/ 2023-10-16T19:50:08.873 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-16T19:50:08.873 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/ 2023-10-16T19:50:08.874 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 67385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-16T19:50:08.874 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 83985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-16T19:50:08.874 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-16T19:50:08.874 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 189801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-16T19:50:08.874 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/samsung/ 2023-10-16T19:50:08.875 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/samsung/sxgbe/ 2023-10-16T19:50:08.875 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 104241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2023-10-16T19:50:08.875 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc/ 2023-10-16T19:50:08.875 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1053921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-16T19:50:08.875 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan/ 2023-10-16T19:50:08.876 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-16T19:50:08.876 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/ 2023-10-16T19:50:08.876 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 54817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-16T19:50:08.876 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 63337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-16T19:50:08.876 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/ 2023-10-16T19:50:08.877 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 61601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-16T19:50:08.877 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 67193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-16T19:50:08.877 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 73409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-16T19:50:08.877 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 68177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-16T19:50:08.877 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/ 2023-10-16T19:50:08.878 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/ 2023-10-16T19:50:08.878 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2023-10-16T19:50:08.878 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-16T19:50:08.878 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2023-10-16T19:50:08.878 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 486961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-16T19:50:08.879 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/ 2023-10-16T19:50:08.879 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 103337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-16T19:50:08.879 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 166161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-16T19:50:08.879 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 66329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-16T19:50:08.879 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 78801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-16T19:50:08.880 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti/ 2023-10-16T19:50:08.880 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 50057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-16T19:50:08.880 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti/ 2023-10-16T19:50:08.880 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 86529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-16T19:50:08.880 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/ 2023-10-16T19:50:08.881 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 80273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-16T19:50:08.881 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 83633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-16T19:50:08.881 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/ 2023-10-16T19:50:08.881 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 46241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-16T19:50:08.881 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-16T19:50:08.882 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom/ 2023-10-16T19:50:08.882 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 60057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-16T19:50:08.882 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/fddi/ 2023-10-16T19:50:08.882 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/fddi/defxx.ko 2023-10-16T19:50:08.882 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/fddi/skfp/ 2023-10-16T19:50:08.883 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 216985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/fddi/skfp/skfp.ko 2023-10-16T19:50:08.883 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 77345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/geneve.ko 2023-10-16T19:50:08.883 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv/ 2023-10-16T19:50:08.883 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 235521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-16T19:50:08.883 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154/ 2023-10-16T19:50:08.884 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-16T19:50:08.884 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ifb.ko 2023-10-16T19:50:08.884 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan/ 2023-10-16T19:50:08.884 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 83769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-16T19:50:08.884 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 62209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvlan.ko 2023-10-16T19:50:08.885 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvtap.ko 2023-10-16T19:50:08.885 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/ 2023-10-16T19:50:08.885 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-16T19:50:08.885 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-16T19:50:08.885 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-gpio.ko 2023-10-16T19:50:08.885 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-16T19:50:08.886 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio.ko 2023-10-16T19:50:08.886 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mii.ko 2023-10-16T19:50:08.886 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/netconsole.ko 2023-10-16T19:50:08.886 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/nlmon.ko 2023-10-16T19:50:08.887 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/ 2023-10-16T19:50:08.887 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-16T19:50:08.887 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-16T19:50:08.887 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/ 2023-10-16T19:50:08.887 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/amd.ko 2023-10-16T19:50:08.888 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 50097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-16T19:50:08.888 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-16T19:50:08.888 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-16T19:50:08.888 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/broadcom.ko 2023-10-16T19:50:08.888 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/cicada.ko 2023-10-16T19:50:08.889 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/davicom.ko 2023-10-16T19:50:08.889 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/et1011c.ko 2023-10-16T19:50:08.889 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/icplus.ko 2023-10-16T19:50:08.889 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/lxt.ko 2023-10-16T19:50:08.889 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 82049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/marvell.ko 2023-10-16T19:50:08.890 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 76177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/micrel.ko 2023-10-16T19:50:08.890 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/national.ko 2023-10-16T19:50:08.890 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 97857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/phylink.ko 2023-10-16T19:50:08.890 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/qsemi.ko 2023-10-16T19:50:08.890 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 50065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/realtek.ko 2023-10-16T19:50:08.890 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/smsc.ko 2023-10-16T19:50:08.891 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-16T19:50:08.891 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/vitesse.ko 2023-10-16T19:50:08.891 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/plip/ 2023-10-16T19:50:08.891 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/plip/plip.ko 2023-10-16T19:50:08.891 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ 2023-10-16T19:50:08.892 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-16T19:50:08.892 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-16T19:50:08.892 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-16T19:50:08.892 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-16T19:50:08.892 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-16T19:50:08.893 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppoe.ko 2023-10-16T19:50:08.893 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppox.ko 2023-10-16T19:50:08.893 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pptp.ko 2023-10-16T19:50:08.893 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/sb1000.ko 2023-10-16T19:50:08.893 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/ 2023-10-16T19:50:08.894 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/slip.ko 2023-10-16T19:50:08.894 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/sungem_phy.ko 2023-10-16T19:50:08.894 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 64385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/tap.ko 2023-10-16T19:50:08.894 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/ 2023-10-16T19:50:08.894 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 106169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team.ko 2023-10-16T19:50:08.895 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-16T19:50:08.895 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-16T19:50:08.895 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-16T19:50:08.895 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_random.ko 2023-10-16T19:50:08.895 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-16T19:50:08.895 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 87017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/veth.ko 2023-10-16T19:50:08.896 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3/ 2023-10-16T19:50:08.896 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 144297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-16T19:50:08.896 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan/ 2023-10-16T19:50:08.896 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 297329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-16T19:50:08.897 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback/ 2023-10-16T19:50:08.897 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 144649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-16T19:50:08.897 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb/ 2023-10-16T19:50:08.897 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb/ntb.ko 2023-10-16T19:50:08.897 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/ 2023-10-16T19:50:08.897 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/ 2023-10-16T19:50:08.898 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 264937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-16T19:50:08.898 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 102209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme.ko 2023-10-16T19:50:08.898 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/ 2023-10-16T19:50:08.898 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 133841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport.ko 2023-10-16T19:50:08.898 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_cs.ko 2023-10-16T19:50:08.899 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 84465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_pc.ko 2023-10-16T19:50:08.899 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_serial.ko 2023-10-16T19:50:08.899 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/ 2023-10-16T19:50:08.899 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/ 2023-10-16T19:50:08.899 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-16T19:50:08.900 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2023-10-16T19:50:08.900 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2023-10-16T19:50:08.900 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/pci-stub.ko 2023-10-16T19:50:08.900 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 46529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/xen-pcifront.ko 2023-10-16T19:50:08.900 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/ 2023-10-16T19:50:08.901 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/i82092.ko 2023-10-16T19:50:08.901 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 145809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pcmcia.ko 2023-10-16T19:50:08.901 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 62137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pcmcia_core.ko 2023-10-16T19:50:08.901 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2023-10-16T19:50:08.902 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pd6729.ko 2023-10-16T19:50:08.902 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 112889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-16T19:50:08.902 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/ 2023-10-16T19:50:08.902 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/broadcom/ 2023-10-16T19:50:08.902 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2023-10-16T19:50:08.903 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/samsung/ 2023-10-16T19:50:08.903 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2023-10-16T19:50:08.903 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/ 2023-10-16T19:50:08.903 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel/ 2023-10-16T19:50:08.903 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 70153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-16T19:50:08.904 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/ 2023-10-16T19:50:08.904 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/ 2023-10-16T19:50:08.904 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 50889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2023-10-16T19:50:08.904 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec.ko 2023-10-16T19:50:08.904 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2023-10-16T19:50:08.905 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2023-10-16T19:50:08.905 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2023-10-16T19:50:08.905 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2023-10-16T19:50:08.905 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2023-10-16T19:50:08.905 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/ 2023-10-16T19:50:08.905 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 67097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-16T19:50:08.906 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-16T19:50:08.906 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-16T19:50:08.906 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 59665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-16T19:50:08.906 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-16T19:50:08.906 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 127569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-16T19:50:08.907 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-16T19:50:08.907 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-16T19:50:08.907 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-16T19:50:08.907 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-16T19:50:08.907 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-16T19:50:08.908 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/hdaps.ko 2023-10-16T19:50:08.908 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/ibm_rtl.ko 2023-10-16T19:50:08.908 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel/ 2023-10-16T19:50:08.908 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-16T19:50:08.908 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2023-10-16T19:50:08.908 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-16T19:50:08.909 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-16T19:50:08.909 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-16T19:50:08.909 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-16T19:50:08.909 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-16T19:50:08.909 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-16T19:50:08.910 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-16T19:50:08.910 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-16T19:50:08.910 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-16T19:50:08.910 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-16T19:50:08.910 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 57569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi.ko 2023-10-16T19:50:08.911 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/ 2023-10-16T19:50:08.911 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ 2023-10-16T19:50:08.911 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/88pm860x_battery.ko 2023-10-16T19:50:08.911 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/88pm860x_charger.ko 2023-10-16T19:50:08.911 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 55057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/bq2415x_charger.ko 2023-10-16T19:50:08.912 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 54081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/bq24190_charger.ko 2023-10-16T19:50:08.912 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/bq24735-charger.ko 2023-10-16T19:50:08.912 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/cros_peripheral_charger.ko 2023-10-16T19:50:08.912 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/da9030_battery.ko 2023-10-16T19:50:08.912 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/da9052-battery.ko 2023-10-16T19:50:08.913 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ds2760_battery.ko 2023-10-16T19:50:08.913 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ds2780_battery.ko 2023-10-16T19:50:08.913 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ds2781_battery.ko 2023-10-16T19:50:08.913 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ds2782_battery.ko 2023-10-16T19:50:08.913 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/gpio-charger.ko 2023-10-16T19:50:08.914 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/isp1704_charger.ko 2023-10-16T19:50:08.914 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/lp8727_charger.ko 2023-10-16T19:50:08.914 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max14577_charger.ko 2023-10-16T19:50:08.914 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max17040_battery.ko 2023-10-16T19:50:08.914 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max17042_battery.ko 2023-10-16T19:50:08.915 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max8903_charger.ko 2023-10-16T19:50:08.915 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max8925_power.ko 2023-10-16T19:50:08.915 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/pcf50633-charger.ko 2023-10-16T19:50:08.915 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/sbs-battery.ko 2023-10-16T19:50:08.915 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/test_power.ko 2023-10-16T19:50:08.916 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/tps65090-charger.ko 2023-10-16T19:50:08.916 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/wm831x_backup.ko 2023-10-16T19:50:08.916 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/wm831x_power.ko 2023-10-16T19:50:08.916 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/wm8350_power.ko 2023-10-16T19:50:08.916 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/ 2023-10-16T19:50:08.916 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 57729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-16T19:50:08.917 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-16T19:50:08.917 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/ 2023-10-16T19:50:08.917 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/ 2023-10-16T19:50:08.917 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-16T19:50:08.917 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-16T19:50:08.918 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-16T19:50:08.918 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/pps_core.ko 2023-10-16T19:50:08.918 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ 2023-10-16T19:50:08.918 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 66385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp.ko 2023-10-16T19:50:08.918 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-16T19:50:08.919 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/ 2023-10-16T19:50:08.919 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-88pm80x.ko 2023-10-16T19:50:08.919 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-88pm860x.ko 2023-10-16T19:50:08.919 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-16T19:50:08.919 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-da9052.ko 2023-10-16T19:50:08.920 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-da9055.ko 2023-10-16T19:50:08.920 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-da9063.ko 2023-10-16T19:50:08.920 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-16T19:50:08.920 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 68577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-16T19:50:08.920 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-16T19:50:08.921 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-16T19:50:08.921 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-16T19:50:08.921 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-16T19:50:08.921 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-16T19:50:08.921 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-16T19:50:08.921 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-16T19:50:08.922 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-16T19:50:08.922 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-16T19:50:08.922 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-16T19:50:08.922 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-16T19:50:08.922 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-lp8788.ko 2023-10-16T19:50:08.923 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-16T19:50:08.923 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-16T19:50:08.923 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-16T19:50:08.923 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t86.ko 2023-10-16T19:50:08.923 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-16T19:50:08.924 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max8907.ko 2023-10-16T19:50:08.924 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max8925.ko 2023-10-16T19:50:08.924 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max8997.ko 2023-10-16T19:50:08.924 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max8998.ko 2023-10-16T19:50:08.924 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-mc13xxx.ko 2023-10-16T19:50:08.924 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-16T19:50:08.925 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-palmas.ko 2023-10-16T19:50:08.925 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 46633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-16T19:50:08.925 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf50633.ko 2023-10-16T19:50:08.925 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-16T19:50:08.925 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-16T19:50:08.926 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-16T19:50:08.926 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-16T19:50:08.926 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rc5t583.ko 2023-10-16T19:50:08.926 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-16T19:50:08.926 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-16T19:50:08.927 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-16T19:50:08.927 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-16T19:50:08.927 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-16T19:50:08.927 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-s35390a.ko 2023-10-16T19:50:08.928 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-16T19:50:08.928 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-tps6586x.ko 2023-10-16T19:50:08.928 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-tps65910.ko 2023-10-16T19:50:08.928 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-wm831x.ko 2023-10-16T19:50:08.928 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-wm8350.ko 2023-10-16T19:50:08.929 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-16T19:50:08.929 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ 2023-10-16T19:50:08.929 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 77617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-16T19:50:08.929 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 57665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-sas.ko 2023-10-16T19:50:08.929 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 55169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-16T19:50:08.930 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 135185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/BusLogic.ko 2023-10-16T19:50:08.930 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/a100u2w.ko 2023-10-16T19:50:08.930 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid/ 2023-10-16T19:50:08.930 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 272305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-16T19:50:08.931 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 100617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/advansys.ko 2023-10-16T19:50:08.931 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/ 2023-10-16T19:50:08.931 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 356265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-16T19:50:08.931 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 295801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-16T19:50:08.931 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic94xx/ 2023-10-16T19:50:08.932 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 189185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic94xx/aic94xx.ko 2023-10-16T19:50:08.932 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/am53c974.ko 2023-10-16T19:50:08.932 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr/ 2023-10-16T19:50:08.932 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 106881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-16T19:50:08.932 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/atp870u.ko 2023-10-16T19:50:08.932 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi/ 2023-10-16T19:50:08.933 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 246985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-16T19:50:08.933 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa/ 2023-10-16T19:50:08.933 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1171945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-16T19:50:08.933 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc/ 2023-10-16T19:50:08.933 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 237273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-16T19:50:08.934 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i/ 2023-10-16T19:50:08.934 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 115369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-16T19:50:08.934 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ch.ko 2023-10-16T19:50:08.934 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor/ 2023-10-16T19:50:08.934 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 308921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-16T19:50:08.935 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/ 2023-10-16T19:50:08.935 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i/ 2023-10-16T19:50:08.935 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 76145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-16T19:50:08.935 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i/ 2023-10-16T19:50:08.935 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 101449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-16T19:50:08.936 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 146689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-16T19:50:08.936 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 75553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dc395x.ko 2023-10-16T19:50:08.936 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dmx3191d.ko 2023-10-16T19:50:08.936 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r/ 2023-10-16T19:50:08.936 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 212425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-16T19:50:08.936 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 60505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esp_scsi.ko 2023-10-16T19:50:08.937 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/ 2023-10-16T19:50:08.937 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 78209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-16T19:50:08.937 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 158505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-16T19:50:08.938 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fdomain.ko 2023-10-16T19:50:08.938 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic/ 2023-10-16T19:50:08.938 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 263489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-16T19:50:08.938 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 201817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hpsa.ko 2023-10-16T19:50:08.938 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 51209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hptiop.ko 2023-10-16T19:50:08.939 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-16T19:50:08.939 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/imm.ko 2023-10-16T19:50:08.939 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/initio.ko 2023-10-16T19:50:08.939 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 266537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ipr.ko 2023-10-16T19:50:08.939 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 101881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ips.ko 2023-10-16T19:50:08.940 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci/ 2023-10-16T19:50:08.940 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 317153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci/isci.ko 2023-10-16T19:50:08.940 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-16T19:50:08.940 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 52961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-16T19:50:08.940 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc/ 2023-10-16T19:50:08.941 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 314865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-16T19:50:08.941 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 145585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi.ko 2023-10-16T19:50:08.941 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-16T19:50:08.941 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas/ 2023-10-16T19:50:08.941 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 216665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-16T19:50:08.942 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc/ 2023-10-16T19:50:08.942 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 2197249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-16T19:50:08.942 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/ 2023-10-16T19:50:08.942 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 73569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-16T19:50:08.942 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-16T19:50:08.943 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 310617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-16T19:50:08.943 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 84721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid.ko 2023-10-16T19:50:08.943 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas/ 2023-10-16T19:50:08.943 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 659777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-16T19:50:08.943 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas/ 2023-10-16T19:50:08.944 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 119561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-16T19:50:08.944 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 78129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvumi.ko 2023-10-16T19:50:08.944 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/ 2023-10-16T19:50:08.944 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 81625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2023-10-16T19:50:08.944 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2023-10-16T19:50:08.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2023-10-16T19:50:08.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2023-10-16T19:50:08.945 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001/ 2023-10-16T19:50:08.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 364105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-16T19:50:08.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 123617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pmcraid.ko 2023-10-16T19:50:08.946 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ppa.ko 2023-10-16T19:50:08.946 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 55361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla1280.ko 2023-10-16T19:50:08.946 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/ 2023-10-16T19:50:08.946 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1876625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-16T19:50:08.946 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 85289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-16T19:50:08.947 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx/ 2023-10-16T19:50:08.947 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 559033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-16T19:50:08.947 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qlogicfas408.ko 2023-10-16T19:50:08.947 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/raid_class.ko 2023-10-16T19:50:08.947 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 238825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_debug.ko 2023-10-16T19:50:08.948 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 173937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-16T19:50:08.948 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 279313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-16T19:50:08.948 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 97729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-16T19:50:08.948 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 73881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-16T19:50:08.948 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-16T19:50:08.949 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ses.ko 2023-10-16T19:50:08.949 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 115889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/st.ko 2023-10-16T19:50:08.949 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 46825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/stex.ko 2023-10-16T19:50:08.949 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2/ 2023-10-16T19:50:08.949 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 162121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-16T19:50:08.950 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-16T19:50:08.950 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 50321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-16T19:50:08.950 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/wd719x.ko 2023-10-16T19:50:08.950 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 50025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/xen-scsifront.ko 2023-10-16T19:50:08.950 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb/ 2023-10-16T19:50:08.951 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 129441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb/ssb.ko 2023-10-16T19:50:08.951 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/ 2023-10-16T19:50:08.951 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi/ 2023-10-16T19:50:08.951 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 736601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-16T19:50:08.951 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback/ 2023-10-16T19:50:08.952 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-16T19:50:08.952 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_file.ko 2023-10-16T19:50:08.952 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_iblock.ko 2023-10-16T19:50:08.952 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 930321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_mod.ko 2023-10-16T19:50:08.952 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_pscsi.ko 2023-10-16T19:50:08.952 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 122521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_user.ko 2023-10-16T19:50:08.953 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc/ 2023-10-16T19:50:08.953 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 75593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-16T19:50:08.953 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/ 2023-10-16T19:50:08.953 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/ 2023-10-16T19:50:08.953 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/ 2023-10-16T19:50:08.954 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-16T19:50:08.954 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-16T19:50:08.954 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-16T19:50:08.954 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-16T19:50:08.955 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-16T19:50:08.955 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-16T19:50:08.955 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-16T19:50:08.955 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-16T19:50:08.956 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-16T19:50:08.956 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-16T19:50:08.956 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-16T19:50:08.956 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-16T19:50:08.956 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_powerclamp.ko 2023-10-16T19:50:08.957 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-16T19:50:08.957 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-16T19:50:08.957 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-16T19:50:08.957 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ 2023-10-16T19:50:08.957 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless/ 2023-10-16T19:50:08.957 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 78425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-16T19:50:08.958 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 57865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/moxa.ko 2023-10-16T19:50:08.958 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 50953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/mxser.ko 2023-10-16T19:50:08.958 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 108601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_gsm.ko 2023-10-16T19:50:08.958 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_hdlc.ko 2023-10-16T19:50:08.958 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 67929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/nozomi.ko 2023-10-16T19:50:08.959 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/ 2023-10-16T19:50:08.959 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250/ 2023-10-16T19:50:08.959 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250/8250_dw.ko 2023-10-16T19:50:08.959 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 59497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-16T19:50:08.959 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/altera_jtaguart.ko 2023-10-16T19:50:08.960 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/altera_uart.ko 2023-10-16T19:50:08.960 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-16T19:50:08.960 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 72425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/fsl_lpuart.ko 2023-10-16T19:50:08.960 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm/ 2023-10-16T19:50:08.960 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 103073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-16T19:50:08.961 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/rp2.ko 2023-10-16T19:50:08.961 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 52817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/sc16is7xx.ko 2023-10-16T19:50:08.961 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/ 2023-10-16T19:50:08.961 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core/ 2023-10-16T19:50:08.961 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 531265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-16T19:50:08.962 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host/ 2023-10-16T19:50:08.962 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-16T19:50:08.962 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2023-10-16T19:50:08.962 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/ 2023-10-16T19:50:08.962 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio.ko 2023-10-16T19:50:08.963 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_aec.ko 2023-10-16T19:50:08.963 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_cif.ko 2023-10-16T19:50:08.963 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_dmem_genirq.ko 2023-10-16T19:50:08.963 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_mf624.ko 2023-10-16T19:50:08.963 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_netx.ko 2023-10-16T19:50:08.964 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-16T19:50:08.964 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_pdrv_genirq.ko 2023-10-16T19:50:08.964 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_sercos3.ko 2023-10-16T19:50:08.964 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/ 2023-10-16T19:50:08.964 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/c67x00/ 2023-10-16T19:50:08.965 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 73297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/c67x00/c67x00.ko 2023-10-16T19:50:08.965 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ 2023-10-16T19:50:08.965 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 144377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ci_hdrc.ko 2023-10-16T19:50:08.965 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2023-10-16T19:50:08.965 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2023-10-16T19:50:08.965 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2023-10-16T19:50:08.966 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/ 2023-10-16T19:50:08.966 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 87089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-16T19:50:08.966 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 51409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-16T19:50:08.966 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usblp.ko 2023-10-16T19:50:08.966 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 89857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usbtmc.ko 2023-10-16T19:50:08.967 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/common/ 2023-10-16T19:50:08.967 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/common/ulpi.ko 2023-10-16T19:50:08.967 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc2/ 2023-10-16T19:50:08.967 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc2/dwc2_pci.ko 2023-10-16T19:50:08.967 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc3/ 2023-10-16T19:50:08.968 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc3/dwc3-haps.ko 2023-10-16T19:50:08.968 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc3/dwc3-pci.ko 2023-10-16T19:50:08.968 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 444553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc3/dwc3.ko 2023-10-16T19:50:08.968 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/ 2023-10-16T19:50:08.968 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/ 2023-10-16T19:50:08.969 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 60177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/u_ether.ko 2023-10-16T19:50:08.969 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 51881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/u_serial.ko 2023-10-16T19:50:08.969 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2023-10-16T19:50:08.969 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2023-10-16T19:50:08.969 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2023-10-16T19:50:08.970 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2023-10-16T19:50:08.970 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 92489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2023-10-16T19:50:08.970 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 51049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2023-10-16T19:50:08.970 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 122081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2023-10-16T19:50:08.970 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 57617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2023-10-16T19:50:08.971 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2023-10-16T19:50:08.971 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2023-10-16T19:50:08.971 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 57729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2023-10-16T19:50:08.971 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 85609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2023-10-16T19:50:08.971 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2023-10-16T19:50:08.971 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 58113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2023-10-16T19:50:08.972 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 67337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2023-10-16T19:50:08.972 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/ 2023-10-16T19:50:08.972 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2023-10-16T19:50:08.972 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2023-10-16T19:50:08.972 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2023-10-16T19:50:08.973 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_ether.ko 2023-10-16T19:50:08.973 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2023-10-16T19:50:08.973 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_hid.ko 2023-10-16T19:50:08.973 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2023-10-16T19:50:08.973 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2023-10-16T19:50:08.974 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2023-10-16T19:50:08.974 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_printer.ko 2023-10-16T19:50:08.974 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_serial.ko 2023-10-16T19:50:08.974 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_zero.ko 2023-10-16T19:50:08.974 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 62185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2023-10-16T19:50:08.975 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2023-10-16T19:50:08.975 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 161225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/libcomposite.ko 2023-10-16T19:50:08.975 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/ 2023-10-16T19:50:08.975 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2023-10-16T19:50:08.975 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/bdc/ 2023-10-16T19:50:08.976 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 157681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2023-10-16T19:50:08.976 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/goku_udc.ko 2023-10-16T19:50:08.976 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 52721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/gr_udc.ko 2023-10-16T19:50:08.976 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 69705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2023-10-16T19:50:08.976 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 56033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/mv_udc.ko 2023-10-16T19:50:08.977 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 76057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/net2272.ko 2023-10-16T19:50:08.977 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 75073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/net2280.ko 2023-10-16T19:50:08.977 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 73169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/pch_udc.ko 2023-10-16T19:50:08.977 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 65809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2023-10-16T19:50:08.977 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 66097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2023-10-16T19:50:08.977 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 81505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2023-10-16T19:50:08.978 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 156073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/udc-core.ko 2023-10-16T19:50:08.978 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/ 2023-10-16T19:50:08.978 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/bcma-hcd.ko 2023-10-16T19:50:08.978 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 80113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/isp116x-hcd.ko 2023-10-16T19:50:08.979 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 87353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/oxu210hp-hcd.ko 2023-10-16T19:50:08.979 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 78873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/r8a66597-hcd.ko 2023-10-16T19:50:08.979 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 51329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-16T19:50:08.979 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/sl811_cs.ko 2023-10-16T19:50:08.979 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/ssb-hcd.ko 2023-10-16T19:50:08.979 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/xhci-plat-hcd.ko 2023-10-16T19:50:08.980 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/ 2023-10-16T19:50:08.980 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/mdc800.ko 2023-10-16T19:50:08.980 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/microtek.ko 2023-10-16T19:50:08.980 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ 2023-10-16T19:50:08.980 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/adutux.ko 2023-10-16T19:50:08.981 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-16T19:50:08.981 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/cypress_cy7c63.ko 2023-10-16T19:50:08.981 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/cytherm.ko 2023-10-16T19:50:08.981 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ehset.ko 2023-10-16T19:50:08.981 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi26.ko 2023-10-16T19:50:08.982 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi62.ko 2023-10-16T19:50:08.982 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ezusb.ko 2023-10-16T19:50:08.982 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/idmouse.ko 2023-10-16T19:50:08.982 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-16T19:50:08.982 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-16T19:50:08.983 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ldusb.ko 2023-10-16T19:50:08.983 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-16T19:50:08.983 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/lvstest.ko 2023-10-16T19:50:08.983 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga/ 2023-10-16T19:50:08.983 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 64841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-16T19:50:08.984 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-16T19:50:08.984 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usb3503.ko 2023-10-16T19:50:08.984 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usblcd.ko 2023-10-16T19:50:08.984 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-16T19:50:08.985 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 70873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usbtest.ko 2023-10-16T19:50:08.985 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/uss720.ko 2023-10-16T19:50:08.985 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/yurex.ko 2023-10-16T19:50:08.985 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/mon/ 2023-10-16T19:50:08.985 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 72953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/mon/usbmon.ko 2023-10-16T19:50:08.986 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/musb/ 2023-10-16T19:50:08.986 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 301545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/musb/musb_hdrc.ko 2023-10-16T19:50:08.986 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/ 2023-10-16T19:50:08.987 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-16T19:50:08.987 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2023-10-16T19:50:08.987 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-isp1301.ko 2023-10-16T19:50:08.987 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-tahvo.ko 2023-10-16T19:50:08.988 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ 2023-10-16T19:50:08.988 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/aircable.ko 2023-10-16T19:50:08.988 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ark3116.ko 2023-10-16T19:50:08.988 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-16T19:50:08.989 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ch341.ko 2023-10-16T19:50:08.989 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 70257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cp210x.ko 2023-10-16T19:50:08.989 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-16T19:50:08.989 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 58217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-16T19:50:08.989 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-16T19:50:08.990 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/empeg.ko 2023-10-16T19:50:08.990 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/f81232.ko 2023-10-16T19:50:08.990 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 175137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-16T19:50:08.990 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-16T19:50:08.990 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 137081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-16T19:50:08.991 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 124833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_ti.ko 2023-10-16T19:50:08.991 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 67769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipaq.ko 2023-10-16T19:50:08.991 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipw.ko 2023-10-16T19:50:08.991 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-16T19:50:08.991 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 58233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-16T19:50:08.991 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 107521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan.ko 2023-10-16T19:50:08.992 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-16T19:50:08.992 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-16T19:50:08.992 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-16T19:50:08.993 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-16T19:50:08.993 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/metro-usb.ko 2023-10-16T19:50:08.993 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 66449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7720.ko 2023-10-16T19:50:08.993 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 78217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7840.ko 2023-10-16T19:50:08.993 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 56337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mxuport.ko 2023-10-16T19:50:08.994 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/navman.ko 2023-10-16T19:50:08.994 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/omninet.ko 2023-10-16T19:50:08.994 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/opticon.ko 2023-10-16T19:50:08.994 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 190633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/option.ko 2023-10-16T19:50:08.995 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/oti6858.ko 2023-10-16T19:50:08.995 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 54353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/pl2303.ko 2023-10-16T19:50:08.995 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcaux.ko 2023-10-16T19:50:08.995 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcserial.ko 2023-10-16T19:50:08.995 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/quatech2.ko 2023-10-16T19:50:08.997 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-16T19:50:08.997 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 51625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/sierra.ko 2023-10-16T19:50:08.997 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-16T19:50:08.997 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ssu100.ko 2023-10-16T19:50:08.998 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-16T19:50:08.998 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 61489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-16T19:50:08.998 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-16T19:50:08.998 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-16T19:50:08.999 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-16T19:50:08.999 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 125609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usbserial.ko 2023-10-16T19:50:08.999 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/visor.ko 2023-10-16T19:50:08.999 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-16T19:50:08.999 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/wishbone-serial.ko 2023-10-16T19:50:09.000 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-16T19:50:09.000 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ 2023-10-16T19:50:09.000 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 52913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/uas.ko 2023-10-16T19:50:09.000 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-16T19:50:09.000 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-16T19:50:09.001 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-16T19:50:09.001 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-16T19:50:09.001 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-16T19:50:09.001 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-16T19:50:09.001 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-16T19:50:09.002 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-16T19:50:09.002 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-16T19:50:09.002 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-16T19:50:09.002 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-16T19:50:09.002 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-16T19:50:09.003 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-16T19:50:09.003 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 179969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-16T19:50:09.003 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/ 2023-10-16T19:50:09.003 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 67481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-16T19:50:09.003 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 72177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-16T19:50:09.004 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 107001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-16T19:50:09.004 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/ 2023-10-16T19:50:09.004 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/ 2023-10-16T19:50:09.004 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 154297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-16T19:50:09.004 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-16T19:50:09.005 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 108737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio.ko 2023-10-16T19:50:09.005 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 77609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-16T19:50:09.005 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/ 2023-10-16T19:50:09.005 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 113873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost.ko 2023-10-16T19:50:09.005 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-16T19:50:09.005 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 58089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_net.ko 2023-10-16T19:50:09.006 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 89249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-16T19:50:09.006 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/ 2023-10-16T19:50:09.006 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/ 2023-10-16T19:50:09.006 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/88pm860x_bl.ko 2023-10-16T19:50:09.006 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/aat2870_bl.ko 2023-10-16T19:50:09.007 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/adp5520_bl.ko 2023-10-16T19:50:09.007 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/adp8860_bl.ko 2023-10-16T19:50:09.007 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/adp8870_bl.ko 2023-10-16T19:50:09.007 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-16T19:50:09.007 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/as3711_bl.ko 2023-10-16T19:50:09.008 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/bd6107.ko 2023-10-16T19:50:09.008 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/da903x_bl.ko 2023-10-16T19:50:09.008 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/da9052_bl.ko 2023-10-16T19:50:09.008 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/gpio_backlight.ko 2023-10-16T19:50:09.008 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/kb3886_bl.ko 2023-10-16T19:50:09.009 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lcd.ko 2023-10-16T19:50:09.009 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lm3533_bl.ko 2023-10-16T19:50:09.009 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lm3639_bl.ko 2023-10-16T19:50:09.009 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lv5207lp.ko 2023-10-16T19:50:09.009 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/max8925_bl.ko 2023-10-16T19:50:09.010 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/pandora_bl.ko 2023-10-16T19:50:09.010 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/pcf50633-backlight.ko 2023-10-16T19:50:09.010 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-16T19:50:09.010 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/wm831x_bl.ko 2023-10-16T19:50:09.010 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/ 2023-10-16T19:50:09.011 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ 2023-10-16T19:50:09.011 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2482.ko 2023-10-16T19:50:09.011 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2490.ko 2023-10-16T19:50:09.011 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/matrox_w1.ko 2023-10-16T19:50:09.011 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/w1-gpio.ko 2023-10-16T19:50:09.012 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/ 2023-10-16T19:50:09.012 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-16T19:50:09.012 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-16T19:50:09.012 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-16T19:50:09.012 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-16T19:50:09.013 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-16T19:50:09.013 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-16T19:50:09.013 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-16T19:50:09.013 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-16T19:50:09.013 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-16T19:50:09.014 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-16T19:50:09.014 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 56425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-16T19:50:09.014 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 102673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/wire.ko 2023-10-16T19:50:09.014 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ 2023-10-16T19:50:09.014 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/acquirewdt.ko 2023-10-16T19:50:09.014 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/advantechwdt.ko 2023-10-16T19:50:09.015 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-16T19:50:09.015 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-16T19:50:09.015 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/cpu5wdt.ko 2023-10-16T19:50:09.015 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/da9052_wdt.ko 2023-10-16T19:50:09.015 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/da9055_wdt.ko 2023-10-16T19:50:09.016 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/da9063_wdt.ko 2023-10-16T19:50:09.016 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/dw_wdt.ko 2023-10-16T19:50:09.016 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/eurotechwdt.ko 2023-10-16T19:50:09.016 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-16T19:50:09.016 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/hpwdt.ko 2023-10-16T19:50:09.017 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/i6300esb.ko 2023-10-16T19:50:09.017 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-16T19:50:09.017 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-16T19:50:09.017 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-16T19:50:09.017 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ibmasr.ko 2023-10-16T19:50:09.018 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-16T19:50:09.018 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-16T19:50:09.018 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-16T19:50:09.018 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/kempld_wdt.ko 2023-10-16T19:50:09.018 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/machzwd.ko 2023-10-16T19:50:09.019 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/mena21_wdt.ko 2023-10-16T19:50:09.019 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/menf21bmc_wdt.ko 2023-10-16T19:50:09.019 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/nv_tco.ko 2023-10-16T19:50:09.019 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/of_xilinx_wdt.ko 2023-10-16T19:50:09.019 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pc87413_wdt.ko 2023-10-16T19:50:09.019 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-16T19:50:09.020 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-16T19:50:09.020 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/retu_wdt.ko 2023-10-16T19:50:09.020 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sbc60xxwdt.ko 2023-10-16T19:50:09.020 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sbc_epx_c3.ko 2023-10-16T19:50:09.020 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-16T19:50:09.021 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sc1200wdt.ko 2023-10-16T19:50:09.021 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-16T19:50:09.021 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/smsc37b787_wdt.ko 2023-10-16T19:50:09.021 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/softdog.ko 2023-10-16T19:50:09.021 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-16T19:50:09.021 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/twl4030_wdt.ko 2023-10-16T19:50:09.022 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/via_wdt.ko 2023-10-16T19:50:09.022 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-16T19:50:09.022 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-16T19:50:09.022 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-16T19:50:09.022 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wafer5823wdt.ko 2023-10-16T19:50:09.023 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-16T19:50:09.023 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wm831x_wdt.ko 2023-10-16T19:50:09.023 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wm8350_wdt.ko 2023-10-16T19:50:09.023 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-16T19:50:09.023 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/ 2023-10-16T19:50:09.024 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-evtchn.ko 2023-10-16T19:50:09.024 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-16T19:50:09.024 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntdev.ko 2023-10-16T19:50:09.024 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback/ 2023-10-16T19:50:09.024 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 176441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-16T19:50:09.024 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-privcmd.ko 2023-10-16T19:50:09.025 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 71249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-scsiback.ko 2023-10-16T19:50:09.025 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs/ 2023-10-16T19:50:09.025 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-16T19:50:09.025 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ 2023-10-16T19:50:09.025 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p/ 2023-10-16T19:50:09.026 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 166425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p/9p.ko 2023-10-16T19:50:09.026 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/adfs/ 2023-10-16T19:50:09.026 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 61073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/adfs/adfs.ko 2023-10-16T19:50:09.026 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs/ 2023-10-16T19:50:09.026 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 163593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs/affs.ko 2023-10-16T19:50:09.027 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/afs/ 2023-10-16T19:50:09.027 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 932817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/afs/kafs.ko 2023-10-16T19:50:09.027 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/autofs/ 2023-10-16T19:50:09.027 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 101057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/autofs/autofs4.ko 2023-10-16T19:50:09.027 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs/ 2023-10-16T19:50:09.028 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 64545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs/befs.ko 2023-10-16T19:50:09.028 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/bfs/ 2023-10-16T19:50:09.028 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/bfs/bfs.ko 2023-10-16T19:50:09.028 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/binfmt_misc.ko 2023-10-16T19:50:09.028 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs/ 2023-10-16T19:50:09.029 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 3402585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs/btrfs.ko 2023-10-16T19:50:09.029 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles/ 2023-10-16T19:50:09.029 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 306073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles/cachefiles.ko 2023-10-16T19:50:09.029 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph/ 2023-10-16T19:50:09.029 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1323873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph/ceph.ko 2023-10-16T19:50:09.030 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda/ 2023-10-16T19:50:09.030 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 100801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda/coda.ko 2023-10-16T19:50:09.030 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/configfs/ 2023-10-16T19:50:09.030 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 108393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/configfs/configfs.ko 2023-10-16T19:50:09.030 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs/ 2023-10-16T19:50:09.030 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs/cramfs.ko 2023-10-16T19:50:09.031 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm/ 2023-10-16T19:50:09.031 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 642577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm/dlm.ko 2023-10-16T19:50:09.031 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/efs/ 2023-10-16T19:50:09.031 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/efs/efs.ko 2023-10-16T19:50:09.031 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs/ 2023-10-16T19:50:09.032 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1505025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs/f2fs.ko 2023-10-16T19:50:09.032 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/ 2023-10-16T19:50:09.032 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/msdos.ko 2023-10-16T19:50:09.032 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/freevxfs/ 2023-10-16T19:50:09.032 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/freevxfs/freevxfs.ko 2023-10-16T19:50:09.032 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache/ 2023-10-16T19:50:09.033 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 226481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache/fscache.ko 2023-10-16T19:50:09.033 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/ 2023-10-16T19:50:09.033 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/cuse.ko 2023-10-16T19:50:09.033 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2/ 2023-10-16T19:50:09.033 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1231529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2/gfs2.ko 2023-10-16T19:50:09.034 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs/ 2023-10-16T19:50:09.034 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 141209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs/hfs.ko 2023-10-16T19:50:09.034 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus/ 2023-10-16T19:50:09.034 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 222817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus/hfsplus.ko 2023-10-16T19:50:09.034 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hpfs/ 2023-10-16T19:50:09.035 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 192329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hpfs/hpfs.ko 2023-10-16T19:50:09.035 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs/ 2023-10-16T19:50:09.035 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 74065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs/isofs.ko 2023-10-16T19:50:09.035 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs/ 2023-10-16T19:50:09.035 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 419497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs/jfs.ko 2023-10-16T19:50:09.036 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd/ 2023-10-16T19:50:09.036 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 280313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd/lockd.ko 2023-10-16T19:50:09.036 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix/ 2023-10-16T19:50:09.036 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 89193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix/minix.ko 2023-10-16T19:50:09.036 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs/ 2023-10-16T19:50:09.037 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 124649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs/netfs.ko 2023-10-16T19:50:09.037 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/ 2023-10-16T19:50:09.037 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout/ 2023-10-16T19:50:09.037 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 81449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-16T19:50:09.037 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout/ 2023-10-16T19:50:09.038 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 59449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-16T19:50:09.038 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout/ 2023-10-16T19:50:09.038 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 117513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-16T19:50:09.038 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1055649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfs.ko 2023-10-16T19:50:09.038 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 66441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv2.ko 2023-10-16T19:50:09.039 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 120177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv3.ko 2023-10-16T19:50:09.039 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 2041689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv4.ko 2023-10-16T19:50:09.039 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/ 2023-10-16T19:50:09.039 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/grace.ko 2023-10-16T19:50:09.039 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-16T19:50:09.040 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd/ 2023-10-16T19:50:09.040 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1516809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd/nfsd.ko 2023-10-16T19:50:09.040 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2/ 2023-10-16T19:50:09.040 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 592833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2/nilfs2.ko 2023-10-16T19:50:09.040 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/ 2023-10-16T19:50:09.041 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-celtic.ko 2023-10-16T19:50:09.041 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-centeuro.ko 2023-10-16T19:50:09.041 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-croatian.ko 2023-10-16T19:50:09.041 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-cyrillic.ko 2023-10-16T19:50:09.041 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-gaelic.ko 2023-10-16T19:50:09.042 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-greek.ko 2023-10-16T19:50:09.042 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-iceland.ko 2023-10-16T19:50:09.042 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-inuit.ko 2023-10-16T19:50:09.042 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-roman.ko 2023-10-16T19:50:09.042 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-romanian.ko 2023-10-16T19:50:09.043 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-turkish.ko 2023-10-16T19:50:09.043 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_ascii.ko 2023-10-16T19:50:09.043 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1250.ko 2023-10-16T19:50:09.043 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1251.ko 2023-10-16T19:50:09.043 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1255.ko 2023-10-16T19:50:09.044 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp737.ko 2023-10-16T19:50:09.044 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp775.ko 2023-10-16T19:50:09.044 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp850.ko 2023-10-16T19:50:09.044 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp852.ko 2023-10-16T19:50:09.044 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp855.ko 2023-10-16T19:50:09.044 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp857.ko 2023-10-16T19:50:09.045 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp860.ko 2023-10-16T19:50:09.045 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp861.ko 2023-10-16T19:50:09.045 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp862.ko 2023-10-16T19:50:09.045 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp863.ko 2023-10-16T19:50:09.045 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp864.ko 2023-10-16T19:50:09.046 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp865.ko 2023-10-16T19:50:09.046 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp866.ko 2023-10-16T19:50:09.046 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp869.ko 2023-10-16T19:50:09.046 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp874.ko 2023-10-16T19:50:09.046 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 94137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp932.ko 2023-10-16T19:50:09.047 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 142673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp936.ko 2023-10-16T19:50:09.047 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 165745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp949.ko 2023-10-16T19:50:09.047 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 118121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp950.ko 2023-10-16T19:50:09.047 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_euc-jp.ko 2023-10-16T19:50:09.047 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-16T19:50:09.047 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-16T19:50:09.048 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-16T19:50:09.048 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-16T19:50:09.048 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-16T19:50:09.048 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-16T19:50:09.048 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-16T19:50:09.049 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-16T19:50:09.049 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-16T19:50:09.049 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-16T19:50:09.049 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-16T19:50:09.049 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-r.ko 2023-10-16T19:50:09.050 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-16T19:50:09.050 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-u.ko 2023-10-16T19:50:09.050 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_ucs2_utils.ko 2023-10-16T19:50:09.050 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_utf8.ko 2023-10-16T19:50:09.050 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ntfs/ 2023-10-16T19:50:09.051 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 219417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ntfs/ntfs.ko 2023-10-16T19:50:09.051 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ 2023-10-16T19:50:09.051 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster/ 2023-10-16T19:50:09.051 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 244409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-16T19:50:09.051 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm/ 2023-10-16T19:50:09.052 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 440737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-16T19:50:09.052 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs/ 2023-10-16T19:50:09.052 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 54049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-16T19:50:09.052 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 2592433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2.ko 2023-10-16T19:50:09.052 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-16T19:50:09.052 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-16T19:50:09.053 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-16T19:50:09.053 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/omfs/ 2023-10-16T19:50:09.053 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/omfs/omfs.ko 2023-10-16T19:50:09.053 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs/ 2023-10-16T19:50:09.053 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 352153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs/overlay.ko 2023-10-16T19:50:09.054 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore/ 2023-10-16T19:50:09.054 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 54809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore/ramoops.ko 2023-10-16T19:50:09.054 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/qnx4/ 2023-10-16T19:50:09.054 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/qnx4/qnx4.ko 2023-10-16T19:50:09.055 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/qnx6/ 2023-10-16T19:50:09.055 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 46689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/qnx6/qnx6.ko 2023-10-16T19:50:09.055 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/quota/ 2023-10-16T19:50:09.055 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/quota/quota_tree.ko 2023-10-16T19:50:09.056 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/quota/quota_v1.ko 2023-10-16T19:50:09.056 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/quota/quota_v2.ko 2023-10-16T19:50:09.056 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs/ 2023-10-16T19:50:09.056 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 532673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs/reiserfs.ko 2023-10-16T19:50:09.056 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs/ 2023-10-16T19:50:09.057 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs/romfs.ko 2023-10-16T19:50:09.057 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/ 2023-10-16T19:50:09.057 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client/ 2023-10-16T19:50:09.057 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 2668121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client/cifs.ko 2023-10-16T19:50:09.057 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/ 2023-10-16T19:50:09.058 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-16T19:50:09.058 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_md4.ko 2023-10-16T19:50:09.058 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs/ 2023-10-16T19:50:09.058 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 134641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs/squashfs.ko 2023-10-16T19:50:09.058 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv/ 2023-10-16T19:50:09.059 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 90721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv/sysv.ko 2023-10-16T19:50:09.059 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf/ 2023-10-16T19:50:09.059 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 247737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf/udf.ko 2023-10-16T19:50:09.059 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs/ 2023-10-16T19:50:09.059 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 162497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs/ufs.ko 2023-10-16T19:50:09.060 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs/ 2023-10-16T19:50:09.060 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 4010729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs/xfs.ko 2023-10-16T19:50:09.060 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/ 2023-10-16T19:50:09.060 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/locking/ 2023-10-16T19:50:09.060 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 75489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/locking/locktorture.ko 2023-10-16T19:50:09.061 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/time/ 2023-10-16T19:50:09.061 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/time/test_udelay.ko 2023-10-16T19:50:09.061 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 60553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/torture.ko 2023-10-16T19:50:09.061 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ 2023-10-16T19:50:09.061 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 6833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc-itu-t.ko 2023-10-16T19:50:09.062 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 6529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc7.ko 2023-10-16T19:50:09.062 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc8.ko 2023-10-16T19:50:09.062 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/ 2023-10-16T19:50:09.062 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libarc4.ko 2023-10-16T19:50:09.063 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libdes.ko 2023-10-16T19:50:09.063 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/interval_tree_test.ko 2023-10-16T19:50:09.063 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/libcrc32c.ko 2023-10-16T19:50:09.063 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lru_cache.ko 2023-10-16T19:50:09.063 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/ 2023-10-16T19:50:09.064 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4_compress.ko 2023-10-16T19:50:09.064 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-16T19:50:09.064 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math/ 2023-10-16T19:50:09.064 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 6617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math/cordic.ko 2023-10-16T19:50:09.065 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/memory-notifier-error-inject.ko 2023-10-16T19:50:09.065 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/notifier-error-inject.ko 2023-10-16T19:50:09.065 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/percpu_test.ko 2023-10-16T19:50:09.065 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/pm-notifier-error-inject.ko 2023-10-16T19:50:09.065 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6/ 2023-10-16T19:50:09.066 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 143433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6/raid6_pq.ko 2023-10-16T19:50:09.066 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/rbtree_test.ko 2023-10-16T19:50:09.066 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon/ 2023-10-16T19:50:09.066 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-16T19:50:09.066 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test-kstrtox.ko 2023-10-16T19:50:09.067 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test-string_helpers.ko 2023-10-16T19:50:09.067 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 4760425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test_bpf.ko 2023-10-16T19:50:09.067 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test_firmware.ko 2023-10-16T19:50:09.067 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 6369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test_module.ko 2023-10-16T19:50:09.068 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test_user_copy.ko 2023-10-16T19:50:09.068 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_bm.ko 2023-10-16T19:50:09.068 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_fsm.ko 2023-10-16T19:50:09.068 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_kmp.ko 2023-10-16T19:50:09.068 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/xz/ 2023-10-16T19:50:09.069 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/xz/xz_dec_test.ko 2023-10-16T19:50:09.069 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/mm/ 2023-10-16T19:50:09.069 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/mm/hwpoison-inject.ko 2023-10-16T19:50:09.069 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ 2023-10-16T19:50:09.069 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/ 2023-10-16T19:50:09.070 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 55737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/6lowpan.ko 2023-10-16T19:50:09.070 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_dest.ko 2023-10-16T19:50:09.070 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-16T19:50:09.070 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_hop.ko 2023-10-16T19:50:09.071 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 6985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-16T19:50:09.071 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-16T19:50:09.071 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_routing.ko 2023-10-16T19:50:09.071 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_udp.ko 2023-10-16T19:50:09.071 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/ 2023-10-16T19:50:09.072 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/garp.ko 2023-10-16T19:50:09.072 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/mrp.ko 2023-10-16T19:50:09.072 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/p8022.ko 2023-10-16T19:50:09.072 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/psnap.ko 2023-10-16T19:50:09.073 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/stp.ko 2023-10-16T19:50:09.073 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q/ 2023-10-16T19:50:09.073 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 81737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q/8021q.ko 2023-10-16T19:50:09.073 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/ 2023-10-16T19:50:09.073 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 198337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet.ko 2023-10-16T19:50:09.074 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_fd.ko 2023-10-16T19:50:09.074 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_rdma.ko 2023-10-16T19:50:09.074 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_virtio.ko 2023-10-16T19:50:09.074 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv/ 2023-10-16T19:50:09.075 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 649297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv/batman-adv.ko 2023-10-16T19:50:09.075 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/ 2023-10-16T19:50:09.075 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 63809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/br_netfilter.ko 2023-10-16T19:50:09.075 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 777689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/bridge.ko 2023-10-16T19:50:09.075 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ 2023-10-16T19:50:09.076 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-16T19:50:09.076 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-16T19:50:09.076 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-16T19:50:09.076 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-16T19:50:09.076 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-16T19:50:09.077 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-16T19:50:09.077 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-16T19:50:09.077 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-16T19:50:09.077 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-16T19:50:09.077 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-16T19:50:09.078 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-16T19:50:09.078 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-16T19:50:09.078 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-16T19:50:09.078 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-16T19:50:09.078 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-16T19:50:09.079 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-16T19:50:09.079 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-16T19:50:09.079 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-16T19:50:09.079 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-16T19:50:09.080 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-16T19:50:09.080 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 55929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-16T19:50:09.080 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-16T19:50:09.080 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-16T19:50:09.080 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph/ 2023-10-16T19:50:09.081 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1047017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph/libceph.ko 2023-10-16T19:50:09.081 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/ 2023-10-16T19:50:09.081 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 125697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/pktgen.ko 2023-10-16T19:50:09.081 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/ 2023-10-16T19:50:09.081 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 194209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp.ko 2023-10-16T19:50:09.082 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_diag.ko 2023-10-16T19:50:09.082 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 57881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv4.ko 2023-10-16T19:50:09.082 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 60337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv6.ko 2023-10-16T19:50:09.082 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/ 2023-10-16T19:50:09.083 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 390473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/dsa_core.ko 2023-10-16T19:50:09.083 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_brcm.ko 2023-10-16T19:50:09.083 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_dsa.ko 2023-10-16T19:50:09.083 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_none.ko 2023-10-16T19:50:09.083 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_trailer.ko 2023-10-16T19:50:09.084 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/hsr/ 2023-10-16T19:50:09.084 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 99729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/hsr/hsr.ko 2023-10-16T19:50:09.084 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ 2023-10-16T19:50:09.084 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan/ 2023-10-16T19:50:09.084 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 45233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-16T19:50:09.085 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 254625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154.ko 2023-10-16T19:50:09.085 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-16T19:50:09.085 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ 2023-10-16T19:50:09.085 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ah4.ko 2023-10-16T19:50:09.086 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/esp4.ko 2023-10-16T19:50:09.086 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 48401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/fou.ko 2023-10-16T19:50:09.086 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/gre.ko 2023-10-16T19:50:09.086 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 48657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/inet_diag.ko 2023-10-16T19:50:09.086 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 60633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_gre.ko 2023-10-16T19:50:09.087 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 61225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_tunnel.ko 2023-10-16T19:50:09.087 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_vti.ko 2023-10-16T19:50:09.087 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipcomp.ko 2023-10-16T19:50:09.087 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipip.ko 2023-10-16T19:50:09.087 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ 2023-10-16T19:50:09.088 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-16T19:50:09.088 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-16T19:50:09.088 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-16T19:50:09.088 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ip_tables.ko 2023-10-16T19:50:09.089 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-16T19:50:09.089 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2023-10-16T19:50:09.089 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-16T19:50:09.089 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-16T19:50:09.089 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-16T19:50:09.089 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_filter.ko 2023-10-16T19:50:09.090 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-16T19:50:09.090 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-16T19:50:09.090 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-16T19:50:09.090 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-16T19:50:09.091 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-16T19:50:09.091 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-16T19:50:09.091 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-16T19:50:09.091 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-16T19:50:09.092 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-16T19:50:09.092 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2023-10-16T19:50:09.092 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-16T19:50:09.092 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-16T19:50:09.092 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-16T19:50:09.093 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-16T19:50:09.093 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-16T19:50:09.093 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_bic.ko 2023-10-16T19:50:09.093 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-16T19:50:09.093 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_diag.ko 2023-10-16T19:50:09.094 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-16T19:50:09.094 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_htcp.ko 2023-10-16T19:50:09.094 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_hybla.ko 2023-10-16T19:50:09.094 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_illinois.ko 2023-10-16T19:50:09.094 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_lp.ko 2023-10-16T19:50:09.095 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_scalable.ko 2023-10-16T19:50:09.095 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_vegas.ko 2023-10-16T19:50:09.095 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_veno.ko 2023-10-16T19:50:09.095 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_westwood.ko 2023-10-16T19:50:09.096 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_yeah.ko 2023-10-16T19:50:09.096 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tunnel4.ko 2023-10-16T19:50:09.096 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_diag.ko 2023-10-16T19:50:09.096 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_tunnel.ko 2023-10-16T19:50:09.096 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-16T19:50:09.097 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ 2023-10-16T19:50:09.097 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ah6.ko 2023-10-16T19:50:09.097 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/esp6.ko 2023-10-16T19:50:09.097 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/fou6.ko 2023-10-16T19:50:09.097 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 80897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_gre.ko 2023-10-16T19:50:09.098 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 93577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-16T19:50:09.098 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-16T19:50:09.098 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_vti.ko 2023-10-16T19:50:09.098 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ipcomp6.ko 2023-10-16T19:50:09.098 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/mip6.ko 2023-10-16T19:50:09.099 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ 2023-10-16T19:50:09.099 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-16T19:50:09.099 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2023-10-16T19:50:09.099 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-16T19:50:09.100 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-16T19:50:09.100 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-16T19:50:09.100 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-16T19:50:09.100 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-16T19:50:09.100 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-16T19:50:09.101 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-16T19:50:09.101 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-16T19:50:09.101 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-16T19:50:09.101 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-16T19:50:09.101 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-16T19:50:09.102 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-16T19:50:09.102 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-16T19:50:09.102 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-16T19:50:09.102 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-16T19:50:09.102 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-16T19:50:09.103 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-16T19:50:09.103 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-16T19:50:09.103 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-16T19:50:09.103 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-16T19:50:09.103 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-16T19:50:09.104 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-16T19:50:09.104 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-16T19:50:09.104 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 74073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/sit.ko 2023-10-16T19:50:09.104 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/tunnel6.ko 2023-10-16T19:50:09.104 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-16T19:50:09.105 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key/ 2023-10-16T19:50:09.105 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 82833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key/af_key.ko 2023-10-16T19:50:09.105 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/ 2023-10-16T19:50:09.105 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 125281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_core.ko 2023-10-16T19:50:09.105 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-16T19:50:09.106 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_eth.ko 2023-10-16T19:50:09.106 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip.ko 2023-10-16T19:50:09.106 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-16T19:50:09.106 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-16T19:50:09.106 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-16T19:50:09.107 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/lapb/ 2023-10-16T19:50:09.107 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/lapb/lapb.ko 2023-10-16T19:50:09.107 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc/ 2023-10-16T19:50:09.107 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc/llc.ko 2023-10-16T19:50:09.108 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 267777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc/llc2.ko 2023-10-16T19:50:09.108 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154/ 2023-10-16T19:50:09.108 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 290417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154/mac802154.ko 2023-10-16T19:50:09.108 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls/ 2023-10-16T19:50:09.108 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls/mpls_gso.ko 2023-10-16T19:50:09.109 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ 2023-10-16T19:50:09.109 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ 2023-10-16T19:50:09.109 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 119585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-16T19:50:09.109 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-16T19:50:09.109 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-16T19:50:09.110 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-16T19:50:09.110 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 100297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-16T19:50:09.110 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 101441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-16T19:50:09.110 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 102817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-16T19:50:09.110 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 103481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-16T19:50:09.111 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 112849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-16T19:50:09.111 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 60329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-16T19:50:09.111 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 109593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-16T19:50:09.111 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 113129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-16T19:50:09.111 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 112585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-16T19:50:09.112 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 111585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-16T19:50:09.112 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 114465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-16T19:50:09.112 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-16T19:50:09.112 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ 2023-10-16T19:50:09.112 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 411729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-16T19:50:09.113 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-16T19:50:09.113 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-16T19:50:09.113 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-16T19:50:09.113 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-16T19:50:09.113 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-16T19:50:09.114 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-16T19:50:09.114 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-16T19:50:09.114 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-16T19:50:09.114 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-16T19:50:09.114 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-16T19:50:09.115 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-16T19:50:09.115 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-16T19:50:09.115 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-16T19:50:09.115 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conncount.ko 2023-10-16T19:50:09.115 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 408257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack.ko 2023-10-16T19:50:09.116 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-16T19:50:09.116 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-16T19:50:09.116 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-16T19:50:09.116 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 118145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-16T19:50:09.116 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-16T19:50:09.117 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-16T19:50:09.117 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 107297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-16T19:50:09.117 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-16T19:50:09.117 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-16T19:50:09.117 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 66641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-16T19:50:09.118 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-16T19:50:09.118 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-16T19:50:09.118 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-16T19:50:09.118 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 89473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table.ko 2023-10-16T19:50:09.119 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-16T19:50:09.119 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-16T19:50:09.119 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 95417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat.ko 2023-10-16T19:50:09.119 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-16T19:50:09.119 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-16T19:50:09.119 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-16T19:50:09.120 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-16T19:50:09.120 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-16T19:50:09.120 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-16T19:50:09.120 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 615313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_tables.ko 2023-10-16T19:50:09.120 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink.ko 2023-10-16T19:50:09.121 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-16T19:50:09.121 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_cttimeout.ko 2023-10-16T19:50:09.121 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-16T19:50:09.121 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-16T19:50:09.121 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-16T19:50:09.122 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-16T19:50:09.122 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_compat.ko 2023-10-16T19:50:09.122 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_ct.ko 2023-10-16T19:50:09.122 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-16T19:50:09.122 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fib.ko 2023-10-16T19:50:09.123 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-16T19:50:09.123 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_hash.ko 2023-10-16T19:50:09.123 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_limit.ko 2023-10-16T19:50:09.123 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_log.ko 2023-10-16T19:50:09.123 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_masq.ko 2023-10-16T19:50:09.123 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_nat.ko 2023-10-16T19:50:09.124 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_numgen.ko 2023-10-16T19:50:09.124 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_osf.ko 2023-10-16T19:50:09.124 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_queue.ko 2023-10-16T19:50:09.124 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_redir.ko 2023-10-16T19:50:09.124 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject.ko 2023-10-16T19:50:09.124 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-16T19:50:09.125 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_socket.ko 2023-10-16T19:50:09.125 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_synproxy.ko 2023-10-16T19:50:09.125 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_tproxy.ko 2023-10-16T19:50:09.125 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 89313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/x_tables.ko 2023-10-16T19:50:09.125 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-16T19:50:09.126 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-16T19:50:09.126 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-16T19:50:09.126 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-16T19:50:09.126 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CT.ko 2023-10-16T19:50:09.126 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_DSCP.ko 2023-10-16T19:50:09.127 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HL.ko 2023-10-16T19:50:09.127 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HMARK.ko 2023-10-16T19:50:09.127 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-16T19:50:09.127 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LED.ko 2023-10-16T19:50:09.127 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LOG.ko 2023-10-16T19:50:09.128 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-16T19:50:09.128 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-16T19:50:09.128 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-16T19:50:09.128 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-16T19:50:09.128 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-16T19:50:09.129 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-16T19:50:09.129 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-16T19:50:09.129 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-16T19:50:09.129 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-16T19:50:09.129 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TEE.ko 2023-10-16T19:50:09.130 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-16T19:50:09.130 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TRACE.ko 2023-10-16T19:50:09.130 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_addrtype.ko 2023-10-16T19:50:09.130 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_bpf.ko 2023-10-16T19:50:09.130 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cgroup.ko 2023-10-16T19:50:09.131 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cluster.ko 2023-10-16T19:50:09.131 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_comment.ko 2023-10-16T19:50:09.131 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connbytes.ko 2023-10-16T19:50:09.131 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlabel.ko 2023-10-16T19:50:09.131 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlimit.ko 2023-10-16T19:50:09.132 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connmark.ko 2023-10-16T19:50:09.132 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_conntrack.ko 2023-10-16T19:50:09.132 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cpu.ko 2023-10-16T19:50:09.132 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dccp.ko 2023-10-16T19:50:09.132 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_devgroup.ko 2023-10-16T19:50:09.133 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dscp.ko 2023-10-16T19:50:09.133 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ecn.ko 2023-10-16T19:50:09.133 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_esp.ko 2023-10-16T19:50:09.133 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-16T19:50:09.133 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_helper.ko 2023-10-16T19:50:09.134 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hl.ko 2023-10-16T19:50:09.134 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-16T19:50:09.134 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_iprange.ko 2023-10-16T19:50:09.134 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipvs.ko 2023-10-16T19:50:09.134 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_l2tp.ko 2023-10-16T19:50:09.135 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_length.ko 2023-10-16T19:50:09.135 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_limit.ko 2023-10-16T19:50:09.135 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mac.ko 2023-10-16T19:50:09.135 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mark.ko 2023-10-16T19:50:09.135 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_multiport.ko 2023-10-16T19:50:09.136 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nat.ko 2023-10-16T19:50:09.136 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nfacct.ko 2023-10-16T19:50:09.136 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_osf.ko 2023-10-16T19:50:09.136 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_owner.ko 2023-10-16T19:50:09.136 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_physdev.ko 2023-10-16T19:50:09.137 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_pkttype.ko 2023-10-16T19:50:09.137 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_policy.ko 2023-10-16T19:50:09.137 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_quota.ko 2023-10-16T19:50:09.137 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_rateest.ko 2023-10-16T19:50:09.137 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_realm.ko 2023-10-16T19:50:09.137 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_recent.ko 2023-10-16T19:50:09.138 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_sctp.ko 2023-10-16T19:50:09.138 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_set.ko 2023-10-16T19:50:09.138 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_socket.ko 2023-10-16T19:50:09.138 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_state.ko 2023-10-16T19:50:09.138 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_statistic.ko 2023-10-16T19:50:09.139 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_string.ko 2023-10-16T19:50:09.139 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-16T19:50:09.139 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_tcpudp.ko 2023-10-16T19:50:09.139 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_time.ko 2023-10-16T19:50:09.139 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_u32.ko 2023-10-16T19:50:09.140 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink/ 2023-10-16T19:50:09.140 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink/netlink_diag.ko 2023-10-16T19:50:09.140 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh/ 2023-10-16T19:50:09.140 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh/nsh.ko 2023-10-16T19:50:09.140 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/ 2023-10-16T19:50:09.141 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 380033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/openvswitch.ko 2023-10-16T19:50:09.141 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-geneve.ko 2023-10-16T19:50:09.141 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-gre.ko 2023-10-16T19:50:09.141 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-16T19:50:09.141 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet/ 2023-10-16T19:50:09.141 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet/af_packet_diag.ko 2023-10-16T19:50:09.142 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/phonet/ 2023-10-16T19:50:09.142 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 110201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/phonet/phonet.ko 2023-10-16T19:50:09.142 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 54697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/phonet/pn_pep.ko 2023-10-16T19:50:09.142 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/ 2023-10-16T19:50:09.142 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 251481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds.ko 2023-10-16T19:50:09.143 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 164937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_rdma.ko 2023-10-16T19:50:09.143 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 67377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_tcp.ko 2023-10-16T19:50:09.143 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rxrpc/ 2023-10-16T19:50:09.143 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 819657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rxrpc/rxrpc.ko 2023-10-16T19:50:09.143 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/ 2023-10-16T19:50:09.144 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_csum.ko 2023-10-16T19:50:09.144 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_gact.ko 2023-10-16T19:50:09.144 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_ipt.ko 2023-10-16T19:50:09.144 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_mirred.ko 2023-10-16T19:50:09.144 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_nat.ko 2023-10-16T19:50:09.144 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_pedit.ko 2023-10-16T19:50:09.145 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_police.ko 2023-10-16T19:50:09.145 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_simple.ko 2023-10-16T19:50:09.145 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_skbedit.ko 2023-10-16T19:50:09.145 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_vlan.ko 2023-10-16T19:50:09.145 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_basic.ko 2023-10-16T19:50:09.146 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_bpf.ko 2023-10-16T19:50:09.146 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_cgroup.ko 2023-10-16T19:50:09.146 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_flow.ko 2023-10-16T19:50:09.146 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_fw.ko 2023-10-16T19:50:09.146 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_route.ko 2023-10-16T19:50:09.147 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 61409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_u32.ko 2023-10-16T19:50:09.147 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_cmp.ko 2023-10-16T19:50:09.147 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_ipset.ko 2023-10-16T19:50:09.147 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_meta.ko 2023-10-16T19:50:09.147 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_nbyte.ko 2023-10-16T19:50:09.148 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_text.ko 2023-10-16T19:50:09.148 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_u32.ko 2023-10-16T19:50:09.148 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_choke.ko 2023-10-16T19:50:09.148 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_codel.ko 2023-10-16T19:50:09.148 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_drr.ko 2023-10-16T19:50:09.149 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_fq.ko 2023-10-16T19:50:09.149 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_fq_codel.ko 2023-10-16T19:50:09.149 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_gred.ko 2023-10-16T19:50:09.149 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hfsc.ko 2023-10-16T19:50:09.149 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hhf.ko 2023-10-16T19:50:09.150 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 66393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_htb.ko 2023-10-16T19:50:09.150 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_ingress.ko 2023-10-16T19:50:09.150 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio.ko 2023-10-16T19:50:09.150 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-16T19:50:09.150 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_multiq.ko 2023-10-16T19:50:09.151 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_netem.ko 2023-10-16T19:50:09.151 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_pie.ko 2023-10-16T19:50:09.151 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_plug.ko 2023-10-16T19:50:09.151 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_prio.ko 2023-10-16T19:50:09.151 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 48793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_qfq.ko 2023-10-16T19:50:09.151 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_red.ko 2023-10-16T19:50:09.152 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfb.ko 2023-10-16T19:50:09.152 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfq.ko 2023-10-16T19:50:09.152 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_tbf.ko 2023-10-16T19:50:09.152 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_teql.ko 2023-10-16T19:50:09.153 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/ 2023-10-16T19:50:09.153 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 929745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp.ko 2023-10-16T19:50:09.153 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp_diag.ko 2023-10-16T19:50:09.153 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/ 2023-10-16T19:50:09.153 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/ 2023-10-16T19:50:09.153 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 336529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-16T19:50:09.154 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 54737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-16T19:50:09.154 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1520161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/sunrpc.ko 2023-10-16T19:50:09.154 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma/ 2023-10-16T19:50:09.154 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 707721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-16T19:50:09.154 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/ 2023-10-16T19:50:09.155 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/diag.ko 2023-10-16T19:50:09.155 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 862641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/tipc.ko 2023-10-16T19:50:09.155 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix/ 2023-10-16T19:50:09.155 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix/unix_diag.ko 2023-10-16T19:50:09.156 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/ 2023-10-16T19:50:09.156 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 85337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-16T19:50:09.156 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 68049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-16T19:50:09.156 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 96193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock.ko 2023-10-16T19:50:09.156 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-16T19:50:09.157 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-16T19:50:09.157 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/x25/ 2023-10-16T19:50:09.157 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 125177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/x25/x25.ko 2023-10-16T19:50:09.157 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/ 2023-10-16T19:50:09.157 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/xfrm_algo.ko 2023-10-16T19:50:09.157 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-16T19:50:09.158 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 102257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/xfrm_user.ko 2023-10-16T19:50:09.158 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/ 2023-10-16T19:50:09.158 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib/ 2023-10-16T19:50:09.158 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib/irqbypass.ko 2023-10-16T19:50:09.158 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 639863 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.alias 2023-10-16T19:50:09.159 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 623919 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.alias.bin 2023-10-16T19:50:09.159 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7663 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.builtin 2023-10-16T19:50:09.159 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23428 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.alias.bin 2023-10-16T19:50:09.159 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9821 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.bin 2023-10-16T19:50:09.159 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 70004 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.modinfo 2023-10-16T19:50:09.160 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 146313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.dep 2023-10-16T19:50:09.160 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 232548 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.dep.bin 2023-10-16T19:50:09.160 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 210 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.devname 2023-10-16T19:50:09.160 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 72606 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.order 2023-10-16T19:50:09.160 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 879 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.softdep 2023-10-16T19:50:09.160 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 254438 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.symbols 2023-10-16T19:50:09.161 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 309363 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.symbols.bin 2023-10-16T19:50:09.161 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./usr/ 2023-10-16T19:50:09.161 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./usr/share/ 2023-10-16T19:50:09.161 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./usr/share/doc/ 2023-10-16T19:50:09.161 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./usr/share/doc/linux-image-6.6.0-rc5-g47672119149b/ 2023-10-16T19:50:09.162 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 175 2023-10-10 19:56 ./usr/share/doc/linux-image-6.6.0-rc5-g47672119149b/changelog.Debian.gz 2023-10-16T19:50:09.162 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 691 2023-10-10 19:56 ./usr/share/doc/linux-image-6.6.0-rc5-g47672119149b/copyright 2023-10-16T19:50:09.163 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g47672119149b 2023-10-16T19:50:09.163 INFO:teuthology.task.kernel:searching for kernel 6.6.0-rc5-g47672119149b 2023-10-16T19:50:09.163 DEBUG:teuthology.orchestra.run.smithi094:> egrep '(submenu|menuentry.*6.6.0-rc5-g47672119149b).*{' /boot/grub/grub.cfg 2023-10-16T19:50:09.170 INFO:teuthology.orchestra.run.smithi094.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-10-16T19:50:09.170 INFO:teuthology.orchestra.run.smithi094.stdout: menuentry 'Ubuntu, with Linux 6.6.0-rc5-g47672119149b' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-rc5-g47672119149b-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-10-16T19:50:09.171 INFO:teuthology.orchestra.run.smithi094.stdout: menuentry 'Ubuntu, with Linux 6.6.0-rc5-g47672119149b (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-rc5-g47672119149b-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-10-16T19:50:09.171 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2023-10-16T19:50:09.172 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.6.0-rc5-g47672119149b 2023-10-16T19:50:09.172 DEBUG:teuthology.orchestra.run.smithi094:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2023-10-16T19:50:09.215 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi094.front.sepia.ceph.com to complete... 2023-10-16T19:50:09.669 INFO:teuthology.orchestra.run.smithi094.stderr:Sourcing file `/etc/default/grub' 2023-10-16T19:50:09.670 INFO:teuthology.orchestra.run.smithi094.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-16T19:50:09.691 INFO:teuthology.orchestra.run.smithi094.stderr:Generating grub configuration file ... 2023-10-16T19:50:09.867 INFO:teuthology.orchestra.run.smithi094.stderr:Found linux image: /boot/vmlinuz-6.6.0-rc5-g47672119149b 2023-10-16T19:50:09.873 INFO:teuthology.orchestra.run.smithi094.stderr:Found initrd image: /boot/initrd.img-6.6.0-rc5-g47672119149b 2023-10-16T19:50:10.093 INFO:teuthology.orchestra.run.smithi094.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-10-16T19:50:10.098 INFO:teuthology.orchestra.run.smithi094.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-10-16T19:50:10.139 INFO:teuthology.orchestra.run.smithi094.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-10-16T19:50:10.144 INFO:teuthology.orchestra.run.smithi094.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-10-16T19:50:10.270 INFO:teuthology.orchestra.run.smithi094.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-10-16T19:50:10.270 INFO:teuthology.orchestra.run.smithi094.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-10-16T19:50:10.270 INFO:teuthology.orchestra.run.smithi094.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-10-16T19:50:10.284 INFO:teuthology.orchestra.run.smithi094.stderr:done 2023-10-16T19:50:11.157 INFO:teuthology.orchestra.run.smithi088.stderr:Sourcing file `/etc/default/grub' 2023-10-16T19:50:11.176 INFO:teuthology.orchestra.run.smithi088.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-16T19:50:11.245 INFO:teuthology.orchestra.run.smithi088.stderr:Generating grub configuration file ... 2023-10-16T19:50:11.606 INFO:teuthology.orchestra.run.smithi088.stderr:Found linux image: /boot/vmlinuz-6.6.0-rc5-g47672119149b 2023-10-16T19:50:11.629 INFO:teuthology.orchestra.run.smithi088.stderr:Found initrd image: /boot/initrd.img-6.6.0-rc5-g47672119149b 2023-10-16T19:50:11.961 INFO:teuthology.orchestra.run.smithi088.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-10-16T19:50:11.972 INFO:teuthology.orchestra.run.smithi088.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-10-16T19:50:12.040 INFO:teuthology.orchestra.run.smithi088.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-10-16T19:50:12.045 INFO:teuthology.orchestra.run.smithi088.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-10-16T19:50:12.196 INFO:teuthology.orchestra.run.smithi088.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-10-16T19:50:12.196 INFO:teuthology.orchestra.run.smithi088.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-10-16T19:50:12.196 INFO:teuthology.orchestra.run.smithi088.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-10-16T19:50:12.250 INFO:teuthology.orchestra.run.smithi088.stderr:done 2023-10-16T19:50:12.464 DEBUG:teuthology.orchestra.run.smithi088:> dpkg-deb -c /tmp/linux-image.deb 2023-10-16T19:50:12.472 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./ 2023-10-16T19:50:12.473 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./boot/ 2023-10-16T19:50:12.473 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 4415097 2023-10-10 19:56 ./boot/System.map-6.6.0-rc5-g47672119149b 2023-10-16T19:50:12.485 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 169298 2023-10-10 19:56 ./boot/config-6.6.0-rc5-g47672119149b 2023-10-16T19:50:12.485 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11352928 2023-10-10 19:56 ./boot/vmlinuz-6.6.0-rc5-g47672119149b 2023-10-16T19:50:12.504 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/ 2023-10-16T19:50:12.504 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/kernel/ 2023-10-16T19:50:12.505 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/kernel/postinst.d/ 2023-10-16T19:50:12.505 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/kernel/postrm.d/ 2023-10-16T19:50:12.505 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/kernel/preinst.d/ 2023-10-16T19:50:12.505 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/kernel/prerm.d/ 2023-10-16T19:50:12.506 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/ 2023-10-16T19:50:12.506 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/ 2023-10-16T19:50:12.506 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/ 2023-10-16T19:50:12.506 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/ 2023-10-16T19:50:12.506 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/ 2023-10-16T19:50:12.507 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/ 2023-10-16T19:50:12.507 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/ 2023-10-16T19:50:12.507 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 490233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/aesni-intel.ko 2023-10-16T19:50:12.510 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-16T19:50:12.510 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-16T19:50:12.510 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-16T19:50:12.510 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 93113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-16T19:50:12.511 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 106729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-16T19:50:12.511 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 152241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-16T19:50:12.511 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-16T19:50:12.511 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-16T19:50:12.512 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 83889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-16T19:50:12.512 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-16T19:50:12.512 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 47225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-16T19:50:12.512 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 47209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-16T19:50:12.512 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 51801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-16T19:50:12.513 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 38937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-16T19:50:12.513 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 40529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-16T19:50:12.513 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 58585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-16T19:50:12.513 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 50497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-16T19:50:12.513 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-16T19:50:12.514 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-16T19:50:12.514 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/ 2023-10-16T19:50:12.514 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/ 2023-10-16T19:50:12.514 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce/ 2023-10-16T19:50:12.514 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 40113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-16T19:50:12.515 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpuid.ko 2023-10-16T19:50:12.515 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/msr.ko 2023-10-16T19:50:12.515 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/ 2023-10-16T19:50:12.515 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 353761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-16T19:50:12.515 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 881713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-16T19:50:12.516 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 2276817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm.ko 2023-10-16T19:50:12.593 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ 2023-10-16T19:50:12.593 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 53761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/af_alg.ko 2023-10-16T19:50:12.593 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/algif_hash.ko 2023-10-16T19:50:12.594 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/algif_skcipher.ko 2023-10-16T19:50:12.594 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ansi_cprng.ko 2023-10-16T19:50:12.594 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/anubis.ko 2023-10-16T19:50:12.594 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/arc4.ko 2023-10-16T19:50:12.594 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/asymmetric_keys/ 2023-10-16T19:50:12.595 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2023-10-16T19:50:12.595 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/ 2023-10-16T19:50:12.595 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-16T19:50:12.595 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_pq.ko 2023-10-16T19:50:12.595 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-16T19:50:12.596 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_tx.ko 2023-10-16T19:50:12.596 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_xor.ko 2023-10-16T19:50:12.596 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/raid6test.ko 2023-10-16T19:50:12.596 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authenc.ko 2023-10-16T19:50:12.596 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authencesn.ko 2023-10-16T19:50:12.597 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blake2b_generic.ko 2023-10-16T19:50:12.597 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_common.ko 2023-10-16T19:50:12.597 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_generic.ko 2023-10-16T19:50:12.597 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 38873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/camellia_generic.ko 2023-10-16T19:50:12.597 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast5_generic.ko 2023-10-16T19:50:12.598 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast6_generic.ko 2023-10-16T19:50:12.598 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast_common.ko 2023-10-16T19:50:12.598 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ccm.ko 2023-10-16T19:50:12.598 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cmac.ko 2023-10-16T19:50:12.598 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crc32_generic.ko 2023-10-16T19:50:12.599 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 47505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cryptd.ko 2023-10-16T19:50:12.599 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 39505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_engine.ko 2023-10-16T19:50:12.599 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_simd.ko 2023-10-16T19:50:12.599 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_user.ko 2023-10-16T19:50:12.599 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cts.ko 2023-10-16T19:50:12.600 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/des_generic.ko 2023-10-16T19:50:12.600 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/dh_generic.ko 2023-10-16T19:50:12.600 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/echainiv.ko 2023-10-16T19:50:12.600 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/essiv.ko 2023-10-16T19:50:12.600 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/fcrypt.ko 2023-10-16T19:50:12.600 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/khazad.ko 2023-10-16T19:50:12.601 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lrw.ko 2023-10-16T19:50:12.601 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4.ko 2023-10-16T19:50:12.601 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4hc.ko 2023-10-16T19:50:12.601 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/md4.ko 2023-10-16T19:50:12.601 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/michael_mic.ko 2023-10-16T19:50:12.602 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcbc.ko 2023-10-16T19:50:12.602 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcrypt.ko 2023-10-16T19:50:12.602 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/rmd160.ko 2023-10-16T19:50:12.602 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/seed.ko 2023-10-16T19:50:12.603 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/serpent_generic.ko 2023-10-16T19:50:12.603 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 149193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tcrypt.ko 2023-10-16T19:50:12.603 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tea.ko 2023-10-16T19:50:12.603 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_common.ko 2023-10-16T19:50:12.603 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_generic.ko 2023-10-16T19:50:12.604 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/vmac.ko 2023-10-16T19:50:12.604 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/wp512.ko 2023-10-16T19:50:12.604 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xcbc.ko 2023-10-16T19:50:12.604 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xor.ko 2023-10-16T19:50:12.604 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xts.ko 2023-10-16T19:50:12.605 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xxhash_generic.ko 2023-10-16T19:50:12.605 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ 2023-10-16T19:50:12.605 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/ 2023-10-16T19:50:12.605 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_extlog.ko 2023-10-16T19:50:12.605 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-16T19:50:12.605 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_pad.ko 2023-10-16T19:50:12.606 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/apei/ 2023-10-16T19:50:12.606 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/apei/einj.ko 2023-10-16T19:50:12.606 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/ec_sys.ko 2023-10-16T19:50:12.606 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/platform_profile.ko 2023-10-16T19:50:12.607 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbs.ko 2023-10-16T19:50:12.607 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbshc.ko 2023-10-16T19:50:12.607 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 103025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/video.ko 2023-10-16T19:50:12.607 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ 2023-10-16T19:50:12.607 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/acard-ahci.ko 2023-10-16T19:50:12.608 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 88865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ahci.ko 2023-10-16T19:50:12.608 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ahci_platform.ko 2023-10-16T19:50:12.608 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 88289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/libahci.ko 2023-10-16T19:50:12.608 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/libahci_platform.ko 2023-10-16T19:50:12.608 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_acpi.ko 2023-10-16T19:50:12.609 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ali.ko 2023-10-16T19:50:12.609 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_amd.ko 2023-10-16T19:50:12.609 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_artop.ko 2023-10-16T19:50:12.609 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atiixp.ko 2023-10-16T19:50:12.609 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atp867x.ko 2023-10-16T19:50:12.610 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd640.ko 2023-10-16T19:50:12.610 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-16T19:50:12.610 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cypress.ko 2023-10-16T19:50:12.610 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_efar.ko 2023-10-16T19:50:12.610 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt366.ko 2023-10-16T19:50:12.611 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-16T19:50:12.611 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-16T19:50:12.611 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-16T19:50:12.611 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it8213.ko 2023-10-16T19:50:12.611 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it821x.ko 2023-10-16T19:50:12.612 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_jmicron.ko 2023-10-16T19:50:12.612 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 51481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_legacy.ko 2023-10-16T19:50:12.612 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_marvell.ko 2023-10-16T19:50:12.612 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_mpiix.ko 2023-10-16T19:50:12.612 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_netcell.ko 2023-10-16T19:50:12.613 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ninja32.ko 2023-10-16T19:50:12.613 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87410.ko 2023-10-16T19:50:12.613 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87415.ko 2023-10-16T19:50:12.613 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-16T19:50:12.613 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_opti.ko 2023-10-16T19:50:12.614 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_optidma.ko 2023-10-16T19:50:12.614 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-16T19:50:12.614 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 37793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-16T19:50:12.614 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-16T19:50:12.614 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_piccolo.ko 2023-10-16T19:50:12.615 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_radisys.ko 2023-10-16T19:50:12.615 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_rdc.ko 2023-10-16T19:50:12.615 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_rz1000.ko 2023-10-16T19:50:12.615 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sch.ko 2023-10-16T19:50:12.615 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_serverworks.ko 2023-10-16T19:50:12.616 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sil680.ko 2023-10-16T19:50:12.616 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-16T19:50:12.616 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_triflex.ko 2023-10-16T19:50:12.616 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_via.ko 2023-10-16T19:50:12.616 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pdc_adma.ko 2023-10-16T19:50:12.617 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_inic162x.ko 2023-10-16T19:50:12.617 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 83345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_mv.ko 2023-10-16T19:50:12.617 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 70393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_nv.ko 2023-10-16T19:50:12.617 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 37377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_promise.ko 2023-10-16T19:50:12.617 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_qstor.ko 2023-10-16T19:50:12.618 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil.ko 2023-10-16T19:50:12.618 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 38337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil24.ko 2023-10-16T19:50:12.618 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sis.ko 2023-10-16T19:50:12.618 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_svw.ko 2023-10-16T19:50:12.618 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 44793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sx4.ko 2023-10-16T19:50:12.619 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_uli.ko 2023-10-16T19:50:12.619 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_via.ko 2023-10-16T19:50:12.619 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_vsc.ko 2023-10-16T19:50:12.619 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay/ 2023-10-16T19:50:12.619 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-16T19:50:12.620 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma/ 2023-10-16T19:50:12.620 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 124473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma/bcma.ko 2023-10-16T19:50:12.620 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/ 2023-10-16T19:50:12.620 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe/ 2023-10-16T19:50:12.620 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 113665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe/aoe.ko 2023-10-16T19:50:12.621 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd/ 2023-10-16T19:50:12.621 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 965585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd/drbd.ko 2023-10-16T19:50:12.621 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 173225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/floppy.ko 2023-10-16T19:50:12.621 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx/ 2023-10-16T19:50:12.621 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 94297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-16T19:50:12.622 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 115793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/nbd.ko 2023-10-16T19:50:12.622 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk/ 2023-10-16T19:50:12.622 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 110449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-16T19:50:12.622 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 98193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/pktcdvd.ko 2023-10-16T19:50:12.622 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 263769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/rbd.ko 2023-10-16T19:50:12.623 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback/ 2023-10-16T19:50:12.623 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 98145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-16T19:50:12.623 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram/ 2023-10-16T19:50:12.623 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 60217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram/zram.ko 2023-10-16T19:50:12.623 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ 2023-10-16T19:50:12.624 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/applicom.ko 2023-10-16T19:50:12.624 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hangcheck-timer.ko 2023-10-16T19:50:12.624 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/ 2023-10-16T19:50:12.624 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-16T19:50:12.624 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-16T19:50:12.625 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-16T19:50:12.625 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-16T19:50:12.625 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-16T19:50:12.625 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ 2023-10-16T19:50:12.625 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-16T19:50:12.625 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 148601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-16T19:50:12.626 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-16T19:50:12.626 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 139313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-16T19:50:12.626 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 70921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-16T19:50:12.626 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 53513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-16T19:50:12.626 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/lp.ko 2023-10-16T19:50:12.627 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave/ 2023-10-16T19:50:12.627 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 84097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave/mwave.ko 2023-10-16T19:50:12.627 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/nvram.ko 2023-10-16T19:50:12.627 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ppdev.ko 2023-10-16T19:50:12.627 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 51113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tlclk.ko 2023-10-16T19:50:12.627 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/ 2023-10-16T19:50:12.628 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-16T19:50:12.628 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2023-10-16T19:50:12.628 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2023-10-16T19:50:12.628 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2023-10-16T19:50:12.629 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-16T19:50:12.629 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-16T19:50:12.629 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/xen-tpmfront.ko 2023-10-16T19:50:12.629 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/xillybus/ 2023-10-16T19:50:12.629 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/xillybus/xillybus_class.ko 2023-10-16T19:50:12.629 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 50945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/xillybus/xillybus_core.ko 2023-10-16T19:50:12.630 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/xillybus/xillybus_pcie.ko 2023-10-16T19:50:12.630 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/ 2023-10-16T19:50:12.630 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/clk-palmas.ko 2023-10-16T19:50:12.630 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 47849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/clk-si5351.ko 2023-10-16T19:50:12.631 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/clk-twl6040.ko 2023-10-16T19:50:12.631 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/clk-wm831x.ko 2023-10-16T19:50:12.631 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/ 2023-10-16T19:50:12.631 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-16T19:50:12.631 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-16T19:50:12.631 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-16T19:50:12.632 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ 2023-10-16T19:50:12.632 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ 2023-10-16T19:50:12.632 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 70801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-16T19:50:12.632 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 243025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-16T19:50:12.632 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/ 2023-10-16T19:50:12.633 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/ 2023-10-16T19:50:12.633 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common/ 2023-10-16T19:50:12.633 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 438761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-16T19:50:12.633 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2023-10-16T19:50:12.633 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-16T19:50:12.634 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-aes.ko 2023-10-16T19:50:12.634 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-sha.ko 2023-10-16T19:50:12.634 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio/ 2023-10-16T19:50:12.634 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 68857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-16T19:50:12.634 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca/ 2023-10-16T19:50:12.635 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca/dca.ko 2023-10-16T19:50:12.635 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ 2023-10-16T19:50:12.635 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw/ 2023-10-16T19:50:12.635 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-16T19:50:12.635 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat/ 2023-10-16T19:50:12.636 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 122569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-16T19:50:12.636 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/ 2023-10-16T19:50:12.636 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 79953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/amd64_edac.ko 2023-10-16T19:50:12.636 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/e752x_edac.ko 2023-10-16T19:50:12.636 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 63617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-16T19:50:12.636 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3000_edac.ko 2023-10-16T19:50:12.637 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3200_edac.ko 2023-10-16T19:50:12.637 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5100_edac.ko 2023-10-16T19:50:12.637 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5400_edac.ko 2023-10-16T19:50:12.637 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7300_edac.ko 2023-10-16T19:50:12.637 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 52041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7core_edac.ko 2023-10-16T19:50:12.638 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i82975x_edac.ko 2023-10-16T19:50:12.638 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/ie31200_edac.ko 2023-10-16T19:50:12.638 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 56825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/sb_edac.ko 2023-10-16T19:50:12.638 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/x38_edac.ko 2023-10-16T19:50:12.638 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/ 2023-10-16T19:50:12.639 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-gpio.ko 2023-10-16T19:50:12.639 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-max14577.ko 2023-10-16T19:50:12.639 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-max77693.ko 2023-10-16T19:50:12.639 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-max8997.ko 2023-10-16T19:50:12.640 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-palmas.ko 2023-10-16T19:50:12.640 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-rt8973a.ko 2023-10-16T19:50:12.640 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-sm5502.ko 2023-10-16T19:50:12.640 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/ 2023-10-16T19:50:12.640 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/dmi-sysfs.ko 2023-10-16T19:50:12.641 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/efi/ 2023-10-16T19:50:12.641 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/efi/efi-pstore.ko 2023-10-16T19:50:12.641 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-16T19:50:12.641 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/ 2023-10-16T19:50:12.641 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-adp5520.ko 2023-10-16T19:50:12.642 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-amd8111.ko 2023-10-16T19:50:12.642 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-arizona.ko 2023-10-16T19:50:12.642 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-da9052.ko 2023-10-16T19:50:12.642 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-da9055.ko 2023-10-16T19:50:12.642 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-dln2.ko 2023-10-16T19:50:12.643 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 77849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-f7188x.ko 2023-10-16T19:50:12.643 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-generic.ko 2023-10-16T19:50:12.643 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-ich.ko 2023-10-16T19:50:12.643 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-janz-ttl.ko 2023-10-16T19:50:12.643 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-kempld.ko 2023-10-16T19:50:12.644 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-lp3943.ko 2023-10-16T19:50:12.644 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-max7300.ko 2023-10-16T19:50:12.644 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-max730x.ko 2023-10-16T19:50:12.644 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-max732x.ko 2023-10-16T19:50:12.644 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-ml-ioh.ko 2023-10-16T19:50:12.645 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 47073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-pca953x.ko 2023-10-16T19:50:12.645 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-pcf857x.ko 2023-10-16T19:50:12.645 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-rdc321x.ko 2023-10-16T19:50:12.645 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-sch.ko 2023-10-16T19:50:12.645 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-sch311x.ko 2023-10-16T19:50:12.646 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-tps65912.ko 2023-10-16T19:50:12.646 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-twl4030.ko 2023-10-16T19:50:12.646 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-twl6040.ko 2023-10-16T19:50:12.646 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-16T19:50:12.646 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-vx855.ko 2023-10-16T19:50:12.647 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-wm831x.ko 2023-10-16T19:50:12.647 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-wm8350.ko 2023-10-16T19:50:12.647 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-wm8994.ko 2023-10-16T19:50:12.647 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/ 2023-10-16T19:50:12.647 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-a4tech.ko 2023-10-16T19:50:12.647 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 39169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-apple.ko 2023-10-16T19:50:12.648 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-appleir.ko 2023-10-16T19:50:12.648 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-aureal.ko 2023-10-16T19:50:12.648 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-axff.ko 2023-10-16T19:50:12.648 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-belkin.ko 2023-10-16T19:50:12.648 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-cherry.ko 2023-10-16T19:50:12.649 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-chicony.ko 2023-10-16T19:50:12.649 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 60649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-cp2112.ko 2023-10-16T19:50:12.649 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-cypress.ko 2023-10-16T19:50:12.649 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-dr.ko 2023-10-16T19:50:12.649 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elecom.ko 2023-10-16T19:50:12.650 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elo.ko 2023-10-16T19:50:12.650 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-emsff.ko 2023-10-16T19:50:12.650 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-ezkey.ko 2023-10-16T19:50:12.650 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gaff.ko 2023-10-16T19:50:12.650 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-generic.ko 2023-10-16T19:50:12.651 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gt683r.ko 2023-10-16T19:50:12.651 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gyration.ko 2023-10-16T19:50:12.651 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-16T19:50:12.651 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-16T19:50:12.651 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtekff.ko 2023-10-16T19:50:12.651 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-hyperv.ko 2023-10-16T19:50:12.652 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-icade.ko 2023-10-16T19:50:12.652 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-kensington.ko 2023-10-16T19:50:12.652 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-keytouch.ko 2023-10-16T19:50:12.652 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-kye.ko 2023-10-16T19:50:12.652 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lcpower.ko 2023-10-16T19:50:12.653 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-led.ko 2023-10-16T19:50:12.653 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 42361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lenovo.ko 2023-10-16T19:50:12.653 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lg-g15.ko 2023-10-16T19:50:12.653 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 58185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-16T19:50:12.654 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 101369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-16T19:50:12.654 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 80993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech.ko 2023-10-16T19:50:12.654 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-magicmouse.ko 2023-10-16T19:50:12.654 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-microsoft.ko 2023-10-16T19:50:12.655 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-monterey.ko 2023-10-16T19:50:12.655 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 50161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-multitouch.ko 2023-10-16T19:50:12.655 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-ntrig.ko 2023-10-16T19:50:12.655 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-ortek.ko 2023-10-16T19:50:12.655 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-penmount.ko 2023-10-16T19:50:12.656 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-petalynx.ko 2023-10-16T19:50:12.656 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 72937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-picolcd.ko 2023-10-16T19:50:12.656 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-pl.ko 2023-10-16T19:50:12.656 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-plantronics.ko 2023-10-16T19:50:12.656 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-primax.ko 2023-10-16T19:50:12.656 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-rmi.ko 2023-10-16T19:50:12.657 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-16T19:50:12.657 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-16T19:50:12.657 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-16T19:50:12.657 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-16T19:50:12.657 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-16T19:50:12.658 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-16T19:50:12.658 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-16T19:50:12.658 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-16T19:50:12.658 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-16T19:50:12.658 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-16T19:50:12.659 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-16T19:50:12.659 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat.ko 2023-10-16T19:50:12.659 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-saitek.ko 2023-10-16T19:50:12.659 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-samsung.ko 2023-10-16T19:50:12.659 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-16T19:50:12.660 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sjoy.ko 2023-10-16T19:50:12.660 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 51249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sony.ko 2023-10-16T19:50:12.660 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-speedlink.ko 2023-10-16T19:50:12.660 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-steelseries.ko 2023-10-16T19:50:12.660 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sunplus.ko 2023-10-16T19:50:12.660 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-16T19:50:12.661 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tivo.ko 2023-10-16T19:50:12.661 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tmff.ko 2023-10-16T19:50:12.661 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-topseed.ko 2023-10-16T19:50:12.661 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-twinhan.ko 2023-10-16T19:50:12.661 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 71073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-uclogic.ko 2023-10-16T19:50:12.662 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-waltop.ko 2023-10-16T19:50:12.662 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 143473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-wiimote.ko 2023-10-16T19:50:12.662 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-xinmo.ko 2023-10-16T19:50:12.662 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zpff.ko 2023-10-16T19:50:12.662 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zydacron.ko 2023-10-16T19:50:12.663 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 267825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid.ko 2023-10-16T19:50:12.663 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/uhid.ko 2023-10-16T19:50:12.663 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/usbhid/ 2023-10-16T19:50:12.663 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 125721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/usbhid/usbhid.ko 2023-10-16T19:50:12.663 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/usbhid/usbkbd.ko 2023-10-16T19:50:12.664 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/usbhid/usbmouse.ko 2023-10-16T19:50:12.664 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 217873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/wacom.ko 2023-10-16T19:50:12.664 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/ 2023-10-16T19:50:12.664 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 64201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_balloon.ko 2023-10-16T19:50:12.664 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 89625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_utils.ko 2023-10-16T19:50:12.665 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 305465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_vmbus.ko 2023-10-16T19:50:12.665 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ 2023-10-16T19:50:12.665 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 66521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru.ko 2023-10-16T19:50:12.665 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 61393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru3.ko 2023-10-16T19:50:12.665 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-16T19:50:12.666 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7414.ko 2023-10-16T19:50:12.666 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7418.ko 2023-10-16T19:50:12.666 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adc128d818.ko 2023-10-16T19:50:12.666 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1025.ko 2023-10-16T19:50:12.666 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 97977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1026.ko 2023-10-16T19:50:12.666 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1029.ko 2023-10-16T19:50:12.667 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 45553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1031.ko 2023-10-16T19:50:12.667 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm9240.ko 2023-10-16T19:50:12.667 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ads7828.ko 2023-10-16T19:50:12.667 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7410.ko 2023-10-16T19:50:12.667 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7411.ko 2023-10-16T19:50:12.668 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 74433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7462.ko 2023-10-16T19:50:12.668 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 39585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7470.ko 2023-10-16T19:50:12.668 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 67929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7475.ko 2023-10-16T19:50:12.668 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7x10.ko 2023-10-16T19:50:12.668 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 39073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/amc6821.ko 2023-10-16T19:50:12.669 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 55265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/applesmc.ko 2023-10-16T19:50:12.669 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 49913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asb100.ko 2023-10-16T19:50:12.669 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 58617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asc7621.ko 2023-10-16T19:50:12.669 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 40889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-16T19:50:12.669 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/atxp1.ko 2023-10-16T19:50:12.670 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/coretemp.ko 2023-10-16T19:50:12.670 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/da9052-hwmon.ko 2023-10-16T19:50:12.670 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/da9055-hwmon.ko 2023-10-16T19:50:12.670 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 44465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-16T19:50:12.671 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 99601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dme1737.ko 2023-10-16T19:50:12.671 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds1621.ko 2023-10-16T19:50:12.671 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds620.ko 2023-10-16T19:50:12.671 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc1403.ko 2023-10-16T19:50:12.671 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc2103.ko 2023-10-16T19:50:12.671 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc6w201.ko 2023-10-16T19:50:12.672 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 66985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71805f.ko 2023-10-16T19:50:12.672 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 94457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71882fg.ko 2023-10-16T19:50:12.672 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 41401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f75375s.ko 2023-10-16T19:50:12.672 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-16T19:50:12.672 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 55449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fschmd.ko 2023-10-16T19:50:12.673 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g760a.ko 2023-10-16T19:50:12.673 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g762.ko 2023-10-16T19:50:12.673 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 46801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl518sm.ko 2023-10-16T19:50:12.673 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 45529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl520sm.ko 2023-10-16T19:50:12.673 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/hih6130.ko 2023-10-16T19:50:12.674 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-16T19:50:12.674 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-16T19:50:12.674 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-16T19:50:12.674 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmaem.ko 2023-10-16T19:50:12.674 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmpex.ko 2023-10-16T19:50:12.675 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina209.ko 2023-10-16T19:50:12.675 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina2xx.ko 2023-10-16T19:50:12.675 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 130449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/it87.ko 2023-10-16T19:50:12.675 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/jc42.ko 2023-10-16T19:50:12.675 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k10temp.ko 2023-10-16T19:50:12.676 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k8temp.ko 2023-10-16T19:50:12.676 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-16T19:50:12.676 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 48601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm63.ko 2023-10-16T19:50:12.676 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm73.ko 2023-10-16T19:50:12.676 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm75.ko 2023-10-16T19:50:12.677 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm77.ko 2023-10-16T19:50:12.677 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm78.ko 2023-10-16T19:50:12.677 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm80.ko 2023-10-16T19:50:12.677 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm83.ko 2023-10-16T19:50:12.677 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 72833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm85.ko 2023-10-16T19:50:12.678 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 46153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm87.ko 2023-10-16T19:50:12.678 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 37913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm90.ko 2023-10-16T19:50:12.678 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm92.ko 2023-10-16T19:50:12.678 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 108841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm93.ko 2023-10-16T19:50:12.678 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95234.ko 2023-10-16T19:50:12.679 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95241.ko 2023-10-16T19:50:12.679 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95245.ko 2023-10-16T19:50:12.679 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc2945.ko 2023-10-16T19:50:12.679 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4151.ko 2023-10-16T19:50:12.679 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4215.ko 2023-10-16T19:50:12.680 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4222.ko 2023-10-16T19:50:12.680 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4245.ko 2023-10-16T19:50:12.680 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4260.ko 2023-10-16T19:50:12.680 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4261.ko 2023-10-16T19:50:12.680 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max16065.ko 2023-10-16T19:50:12.680 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1619.ko 2023-10-16T19:50:12.681 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1668.ko 2023-10-16T19:50:12.681 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max197.ko 2023-10-16T19:50:12.681 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6639.ko 2023-10-16T19:50:12.681 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6650.ko 2023-10-16T19:50:12.681 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6697.ko 2023-10-16T19:50:12.682 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/mc13783-adc.ko 2023-10-16T19:50:12.682 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/mcp3021.ko 2023-10-16T19:50:12.682 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2023-10-16T19:50:12.682 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 44129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6683.ko 2023-10-16T19:50:12.683 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 119817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-16T19:50:12.683 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 60705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775.ko 2023-10-16T19:50:12.683 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 59737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct7802.ko 2023-10-16T19:50:12.683 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 82337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87360.ko 2023-10-16T19:50:12.683 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 54433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87427.ko 2023-10-16T19:50:12.683 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pcf8591.ko 2023-10-16T19:50:12.684 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ 2023-10-16T19:50:12.684 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-16T19:50:12.684 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-16T19:50:12.684 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-16T19:50:12.685 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-16T19:50:12.685 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-16T19:50:12.685 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-16T19:50:12.685 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-16T19:50:12.685 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 82577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-16T19:50:12.686 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-16T19:50:12.686 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-16T19:50:12.686 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-16T19:50:12.686 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-16T19:50:12.686 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/powr1220.ko 2023-10-16T19:50:12.687 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5627.ko 2023-10-16T19:50:12.687 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5636.ko 2023-10-16T19:50:12.687 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-16T19:50:12.687 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht15.ko 2023-10-16T19:50:12.687 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht21.ko 2023-10-16T19:50:12.688 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/shtc1.ko 2023-10-16T19:50:12.688 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 40105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sis5595.ko 2023-10-16T19:50:12.688 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-16T19:50:12.688 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 39193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-16T19:50:12.689 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 37385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-16T19:50:12.689 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/thmc50.ko 2023-10-16T19:50:12.689 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp102.ko 2023-10-16T19:50:12.689 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp103.ko 2023-10-16T19:50:12.689 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp401.ko 2023-10-16T19:50:12.690 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp421.ko 2023-10-16T19:50:12.690 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-16T19:50:12.690 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 41945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via686a.ko 2023-10-16T19:50:12.690 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 52457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt1211.ko 2023-10-16T19:50:12.690 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 51545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt8231.ko 2023-10-16T19:50:12.691 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 57513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-16T19:50:12.691 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 78873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627hf.ko 2023-10-16T19:50:12.691 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 71345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83781d.ko 2023-10-16T19:50:12.691 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 63289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83791d.ko 2023-10-16T19:50:12.691 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 71321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83792d.ko 2023-10-16T19:50:12.692 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 101473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83793.ko 2023-10-16T19:50:12.692 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 119505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83795.ko 2023-10-16T19:50:12.692 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-16T19:50:12.692 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-16T19:50:12.692 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/wm831x-hwmon.ko 2023-10-16T19:50:12.692 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/wm8350-hwmon.ko 2023-10-16T19:50:12.693 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/ 2023-10-16T19:50:12.693 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/ 2023-10-16T19:50:12.693 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-16T19:50:12.693 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-16T19:50:12.693 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/ 2023-10-16T19:50:12.694 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ali1535.ko 2023-10-16T19:50:12.694 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ali1563.ko 2023-10-16T19:50:12.694 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2023-10-16T19:50:12.694 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-16T19:50:12.694 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-16T19:50:12.695 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-16T19:50:12.695 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2023-10-16T19:50:12.695 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 6929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-16T19:50:12.695 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2023-10-16T19:50:12.695 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 50201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-16T19:50:12.697 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-16T19:50:12.697 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-16T19:50:12.697 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-16T19:50:12.697 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-dln2.ko 2023-10-16T19:50:12.698 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-gpio.ko 2023-10-16T19:50:12.698 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 62817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-16T19:50:12.698 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-16T19:50:12.698 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 47889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-16T19:50:12.699 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-kempld.ko 2023-10-16T19:50:12.699 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-16T19:50:12.699 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-16T19:50:12.699 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ocores.ko 2023-10-16T19:50:12.699 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-16T19:50:12.699 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-16T19:50:12.700 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 47241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-16T19:50:12.700 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2023-10-16T19:50:12.700 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-16T19:50:12.700 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-16T19:50:12.700 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-sis5595.ko 2023-10-16T19:50:12.701 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-sis630.ko 2023-10-16T19:50:12.701 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-16T19:50:12.701 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2023-10-16T19:50:12.701 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-16T19:50:12.701 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-16T19:50:12.702 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-16T19:50:12.702 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-16T19:50:12.702 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 47657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-xiic.ko 2023-10-16T19:50:12.702 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-mux.ko 2023-10-16T19:50:12.702 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-16T19:50:12.703 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-stub.ko 2023-10-16T19:50:12.703 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/muxes/ 2023-10-16T19:50:12.703 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2023-10-16T19:50:12.703 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2023-10-16T19:50:12.703 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2023-10-16T19:50:12.704 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ 2023-10-16T19:50:12.704 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ 2023-10-16T19:50:12.704 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 273609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-16T19:50:12.704 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 879969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-16T19:50:12.704 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 69337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-16T19:50:12.704 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 324217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-16T19:50:12.705 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 97985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-16T19:50:12.705 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 282273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-16T19:50:12.705 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 66201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-16T19:50:12.705 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/ 2023-10-16T19:50:12.705 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4/ 2023-10-16T19:50:12.706 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 493073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-16T19:50:12.706 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4/ 2023-10-16T19:50:12.706 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 419737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-16T19:50:12.706 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5/ 2023-10-16T19:50:12.707 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 739025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-16T19:50:12.707 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca/ 2023-10-16T19:50:12.707 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 296625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-16T19:50:12.707 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/ocrdma/ 2023-10-16T19:50:12.707 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 153697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2023-10-16T19:50:12.708 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/usnic/ 2023-10-16T19:50:12.708 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 168969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2023-10-16T19:50:12.708 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ 2023-10-16T19:50:12.708 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib/ 2023-10-16T19:50:12.708 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 263081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-16T19:50:12.709 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser/ 2023-10-16T19:50:12.709 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 100241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-16T19:50:12.709 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert/ 2023-10-16T19:50:12.709 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 111425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-16T19:50:12.709 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp/ 2023-10-16T19:50:12.709 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 128753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-16T19:50:12.710 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt/ 2023-10-16T19:50:12.710 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 129529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-16T19:50:12.710 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/ 2023-10-16T19:50:12.710 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/evbug.ko 2023-10-16T19:50:12.710 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/ff-memless.ko 2023-10-16T19:50:12.711 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/ 2023-10-16T19:50:12.711 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-16T19:50:12.711 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-16T19:50:12.711 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 41241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/gameport.ko 2023-10-16T19:50:12.711 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 41801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/joydev.ko 2023-10-16T19:50:12.712 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/ 2023-10-16T19:50:12.712 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/adp5520-keys.ko 2023-10-16T19:50:12.712 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/adp5588-keys.ko 2023-10-16T19:50:12.712 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/adp5589-keys.ko 2023-10-16T19:50:12.712 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2023-10-16T19:50:12.713 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-16T19:50:12.713 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2023-10-16T19:50:12.713 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/lkkbd.ko 2023-10-16T19:50:12.713 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/lm8323.ko 2023-10-16T19:50:12.713 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/lm8333.ko 2023-10-16T19:50:12.714 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/matrix_keypad.ko 2023-10-16T19:50:12.714 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/max7359_keypad.ko 2023-10-16T19:50:12.714 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/mcs_touchkey.ko 2023-10-16T19:50:12.714 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2023-10-16T19:50:12.714 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/newtonkbd.ko 2023-10-16T19:50:12.715 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/opencores-kbd.ko 2023-10-16T19:50:12.715 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/qt1070.ko 2023-10-16T19:50:12.715 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/qt2160.ko 2023-10-16T19:50:12.715 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/samsung-keypad.ko 2023-10-16T19:50:12.715 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/stowaway.ko 2023-10-16T19:50:12.716 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/sunkbd.ko 2023-10-16T19:50:12.716 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/tca6416-keypad.ko 2023-10-16T19:50:12.716 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/tca8418_keypad.ko 2023-10-16T19:50:12.716 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/twl4030_keypad.ko 2023-10-16T19:50:12.716 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/xtkbd.ko 2023-10-16T19:50:12.717 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/matrix-keymap.ko 2023-10-16T19:50:12.717 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/ 2023-10-16T19:50:12.717 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/appletouch.ko 2023-10-16T19:50:12.717 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-16T19:50:12.717 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 123681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-16T19:50:12.718 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 89449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-16T19:50:12.718 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/gpio_mouse.ko 2023-10-16T19:50:12.718 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 335977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/psmouse.ko 2023-10-16T19:50:12.718 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/sermouse.ko 2023-10-16T19:50:12.718 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-16T19:50:12.719 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-16T19:50:12.719 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-16T19:50:12.719 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4/ 2023-10-16T19:50:12.719 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 109873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-16T19:50:12.719 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/ 2023-10-16T19:50:12.719 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-16T19:50:12.720 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-16T19:50:12.720 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/ct82c710.ko 2023-10-16T19:50:12.720 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-16T19:50:12.720 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/parkbd.ko 2023-10-16T19:50:12.721 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/pcips2.ko 2023-10-16T19:50:12.721 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/ps2mult.ko 2023-10-16T19:50:12.721 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/serio_raw.ko 2023-10-16T19:50:12.721 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/serport.ko 2023-10-16T19:50:12.721 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/sparse-keymap.ko 2023-10-16T19:50:12.722 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/ 2023-10-16T19:50:12.722 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd/ 2023-10-16T19:50:12.722 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 39289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-16T19:50:12.722 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/ 2023-10-16T19:50:12.722 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-88pm860x.ko 2023-10-16T19:50:12.723 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-adp5520.ko 2023-10-16T19:50:12.723 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 59753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-bd2802.ko 2023-10-16T19:50:12.723 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-blinkm.ko 2023-10-16T19:50:12.723 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-da903x.ko 2023-10-16T19:50:12.723 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-da9052.ko 2023-10-16T19:50:12.724 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-gpio.ko 2023-10-16T19:50:12.724 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm3530.ko 2023-10-16T19:50:12.724 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm3533.ko 2023-10-16T19:50:12.724 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm355x.ko 2023-10-16T19:50:12.724 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm3642.ko 2023-10-16T19:50:12.725 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lp3944.ko 2023-10-16T19:50:12.725 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lp8788.ko 2023-10-16T19:50:12.725 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-max8997.ko 2023-10-16T19:50:12.725 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-mc13783.ko 2023-10-16T19:50:12.725 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-menf21bmc.ko 2023-10-16T19:50:12.726 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-pca9532.ko 2023-10-16T19:50:12.726 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-pca955x.ko 2023-10-16T19:50:12.726 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-pca963x.ko 2023-10-16T19:50:12.726 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-ss4200.ko 2023-10-16T19:50:12.726 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-tca6507.ko 2023-10-16T19:50:12.727 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-wm831x-status.ko 2023-10-16T19:50:12.727 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-wm8350.ko 2023-10-16T19:50:12.727 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ 2023-10-16T19:50:12.727 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-16T19:50:12.727 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-16T19:50:12.728 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-16T19:50:12.728 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-16T19:50:12.728 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-16T19:50:12.728 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-16T19:50:12.728 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-16T19:50:12.729 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-16T19:50:12.729 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/ 2023-10-16T19:50:12.729 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache/ 2023-10-16T19:50:12.729 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 654969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache/bcache.ko 2023-10-16T19:50:12.729 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 40577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-bio-prison.ko 2023-10-16T19:50:12.730 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 89601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-bufio.ko 2023-10-16T19:50:12.730 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 47529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache-smq.ko 2023-10-16T19:50:12.730 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 156009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache.ko 2023-10-16T19:50:12.730 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 106897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-crypt.ko 2023-10-16T19:50:12.730 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-delay.ko 2023-10-16T19:50:12.731 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 53857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-era.ko 2023-10-16T19:50:12.731 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-flakey.ko 2023-10-16T19:50:12.731 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 41593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-log-userspace.ko 2023-10-16T19:50:12.731 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-log.ko 2023-10-16T19:50:12.731 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 49545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-mirror.ko 2023-10-16T19:50:12.732 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 78033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-multipath.ko 2023-10-16T19:50:12.732 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-queue-length.ko 2023-10-16T19:50:12.732 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 70825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-raid.ko 2023-10-16T19:50:12.732 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-region-hash.ko 2023-10-16T19:50:12.733 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-round-robin.ko 2023-10-16T19:50:12.733 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-service-time.ko 2023-10-16T19:50:12.733 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 109593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-snapshot.ko 2023-10-16T19:50:12.733 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-switch.ko 2023-10-16T19:50:12.733 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 181865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-thin-pool.ko 2023-10-16T19:50:12.734 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 51737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-verity.ko 2023-10-16T19:50:12.734 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-zero.ko 2023-10-16T19:50:12.734 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/faulty.ko 2023-10-16T19:50:12.734 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/linear.ko 2023-10-16T19:50:12.734 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/multipath.ko 2023-10-16T19:50:12.735 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data/ 2023-10-16T19:50:12.735 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 199937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-16T19:50:12.735 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 37337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid0.ko 2023-10-16T19:50:12.735 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 113073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid1.ko 2023-10-16T19:50:12.735 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 158681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid10.ko 2023-10-16T19:50:12.736 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 395721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid456.ko 2023-10-16T19:50:12.736 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/ 2023-10-16T19:50:12.736 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/ 2023-10-16T19:50:12.736 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 178137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptbase.ko 2023-10-16T19:50:12.736 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 63385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptctl.ko 2023-10-16T19:50:12.737 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 44665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptfc.ko 2023-10-16T19:50:12.737 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 37225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptlan.ko 2023-10-16T19:50:12.737 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 123321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptsas.ko 2023-10-16T19:50:12.737 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 79041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-16T19:50:12.737 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 53081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptspi.ko 2023-10-16T19:50:12.738 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/ 2023-10-16T19:50:12.738 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/88pm800.ko 2023-10-16T19:50:12.738 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/88pm805.ko 2023-10-16T19:50:12.738 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/88pm80x.ko 2023-10-16T19:50:12.738 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/arizona-i2c.ko 2023-10-16T19:50:12.739 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 123721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/arizona.ko 2023-10-16T19:50:12.739 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/bcm590xx.ko 2023-10-16T19:50:12.739 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/cros_ec_dev.ko 2023-10-16T19:50:12.739 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/dln2.ko 2023-10-16T19:50:12.739 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/janz-cmodio.ko 2023-10-16T19:50:12.740 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 53585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/kempld-core.ko 2023-10-16T19:50:12.740 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lm3533-core.ko 2023-10-16T19:50:12.740 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lm3533-ctrlbank.ko 2023-10-16T19:50:12.740 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lp3943.ko 2023-10-16T19:50:12.740 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 54569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_ich.ko 2023-10-16T19:50:12.741 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_sch.ko 2023-10-16T19:50:12.741 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/max8907.ko 2023-10-16T19:50:12.741 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/mc13xxx-core.ko 2023-10-16T19:50:12.741 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/mc13xxx-i2c.ko 2023-10-16T19:50:12.741 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/menf21bmc.ko 2023-10-16T19:50:12.742 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/pcf50633-adc.ko 2023-10-16T19:50:12.742 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/pcf50633-gpio.ko 2023-10-16T19:50:12.742 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/pcf50633.ko 2023-10-16T19:50:12.742 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/rdc321x-southbridge.ko 2023-10-16T19:50:12.742 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/retu-mfd.ko 2023-10-16T19:50:12.743 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 80425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/si476x-core.ko 2023-10-16T19:50:12.743 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 48457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/sm501.ko 2023-10-16T19:50:12.743 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/ti_am335x_tscadc.ko 2023-10-16T19:50:12.743 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 51137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/tps65010.ko 2023-10-16T19:50:12.743 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/tps6507x.ko 2023-10-16T19:50:12.743 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/viperboard.ko 2023-10-16T19:50:12.744 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/vx855.ko 2023-10-16T19:50:12.744 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/wl1273-core.ko 2023-10-16T19:50:12.744 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 44345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/wm8994.ko 2023-10-16T19:50:12.744 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ 2023-10-16T19:50:12.744 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ad525x_dpot-i2c.ko 2023-10-16T19:50:12.745 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 48489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ad525x_dpot.ko 2023-10-16T19:50:12.745 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl/ 2023-10-16T19:50:12.745 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 46585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-16T19:50:12.745 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds9802als.ko 2023-10-16T19:50:12.745 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds990x.ko 2023-10-16T19:50:12.746 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 45593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/bh1770glc.ko 2023-10-16T19:50:12.746 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/c2port/ 2023-10-16T19:50:12.746 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2023-10-16T19:50:12.746 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/c2port/core.ko 2023-10-16T19:50:12.746 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710/ 2023-10-16T19:50:12.747 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710/cb710.ko 2023-10-16T19:50:12.747 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ds1682.ko 2023-10-16T19:50:12.747 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/dummy-irq.ko 2023-10-16T19:50:12.747 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/echo/ 2023-10-16T19:50:12.747 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/echo/echo.ko 2023-10-16T19:50:12.748 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/ 2023-10-16T19:50:12.748 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 37849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/at24.ko 2023-10-16T19:50:12.748 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-16T19:50:12.748 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-16T19:50:12.748 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-16T19:50:12.748 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/enclosure.ko 2023-10-16T19:50:12.749 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/genwqe/ 2023-10-16T19:50:12.749 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 152201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/genwqe/genwqe_card.ko 2023-10-16T19:50:12.749 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/hmc6352.ko 2023-10-16T19:50:12.749 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/hpilo.ko 2023-10-16T19:50:12.749 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ibmasm/ 2023-10-16T19:50:12.750 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 72129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ibmasm/ibmasm.ko 2023-10-16T19:50:12.750 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ics932s401.ko 2023-10-16T19:50:12.750 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29003.ko 2023-10-16T19:50:12.750 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29020.ko 2023-10-16T19:50:12.750 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/ 2023-10-16T19:50:12.751 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 42513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-16T19:50:12.751 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-16T19:50:12.751 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/ 2023-10-16T19:50:12.751 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 125209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-me.ko 2023-10-16T19:50:12.751 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 56521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-16T19:50:12.752 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 310113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei.ko 2023-10-16T19:50:12.752 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/phantom.ko 2023-10-16T19:50:12.752 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ti-st/ 2023-10-16T19:50:12.752 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 74265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ti-st/st_drv.ko 2023-10-16T19:50:12.752 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-16T19:50:12.752 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_core.ko 2023-10-16T19:50:12.753 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tsl2550.ko 2023-10-16T19:50:12.753 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 48697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_balloon.ko 2023-10-16T19:50:12.753 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci/ 2023-10-16T19:50:12.753 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 183609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-16T19:50:12.753 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ 2023-10-16T19:50:12.754 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding/ 2023-10-16T19:50:12.754 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 490129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding/bonding.ko 2023-10-16T19:50:12.754 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/ 2023-10-16T19:50:12.754 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53/ 2023-10-16T19:50:12.754 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 106033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-16T19:50:12.755 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 40297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-16T19:50:12.755 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-16T19:50:12.755 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx/ 2023-10-16T19:50:12.755 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 453737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-16T19:50:12.755 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dummy.ko 2023-10-16T19:50:12.756 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/eql.ko 2023-10-16T19:50:12.756 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ 2023-10-16T19:50:12.756 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/ 2023-10-16T19:50:12.756 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 45785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-16T19:50:12.757 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 42105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-16T19:50:12.757 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 130553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-16T19:50:12.757 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 68849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-16T19:50:12.757 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/ 2023-10-16T19:50:12.757 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 37649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-16T19:50:12.758 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 59689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-16T19:50:12.758 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-16T19:50:12.758 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 99113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-16T19:50:12.758 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec/ 2023-10-16T19:50:12.759 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 54521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-16T19:50:12.759 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere/ 2023-10-16T19:50:12.759 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 69201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-16T19:50:12.759 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon/ 2023-10-16T19:50:12.759 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 59585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-16T19:50:12.760 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera/ 2023-10-16T19:50:12.760 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 41409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-16T19:50:12.760 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/ 2023-10-16T19:50:12.760 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 54609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-16T19:50:12.760 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 49289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-16T19:50:12.761 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 87297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-16T19:50:12.761 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/ 2023-10-16T19:50:12.761 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx/ 2023-10-16T19:50:12.761 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 97337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-16T19:50:12.761 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c/ 2023-10-16T19:50:12.761 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 104673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-16T19:50:12.762 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e/ 2023-10-16T19:50:12.762 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 90257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-16T19:50:12.762 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/ 2023-10-16T19:50:12.762 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 81569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-16T19:50:12.762 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 69729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-16T19:50:12.763 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/ 2023-10-16T19:50:12.763 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 78401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-16T19:50:12.763 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 176761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-16T19:50:12.763 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2023-10-16T19:50:12.763 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1421497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-16T19:50:12.764 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 128569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-16T19:50:12.764 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet/ 2023-10-16T19:50:12.764 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 132089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-16T19:50:12.764 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 312081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-16T19:50:12.764 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/ 2023-10-16T19:50:12.765 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna/ 2023-10-16T19:50:12.765 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 323513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-16T19:50:12.765 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/ 2023-10-16T19:50:12.765 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb/ 2023-10-16T19:50:12.765 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 193497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-16T19:50:12.766 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2023-10-16T19:50:12.766 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 347417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-16T19:50:12.766 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2023-10-16T19:50:12.766 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1044433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-16T19:50:12.893 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2023-10-16T19:50:12.893 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 166193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-16T19:50:12.894 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2023-10-16T19:50:12.894 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-16T19:50:12.894 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/ 2023-10-16T19:50:12.894 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic/ 2023-10-16T19:50:12.895 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 204345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-16T19:50:12.895 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/ 2023-10-16T19:50:12.895 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/ 2023-10-16T19:50:12.895 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 69449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-16T19:50:12.895 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 61489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-16T19:50:12.896 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 141225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-16T19:50:12.896 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 51497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-16T19:50:12.896 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 62817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-16T19:50:12.896 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-16T19:50:12.896 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/ 2023-10-16T19:50:12.897 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 60873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-16T19:50:12.897 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 62833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-16T19:50:12.897 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 45721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dnet.ko 2023-10-16T19:50:12.897 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ec_bhf.ko 2023-10-16T19:50:12.897 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/ 2023-10-16T19:50:12.898 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet/ 2023-10-16T19:50:12.898 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 281137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-16T19:50:12.898 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 49457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-16T19:50:12.898 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 51737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-16T19:50:12.898 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/fujitsu/ 2023-10-16T19:50:12.901 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 45345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2023-10-16T19:50:12.901 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ 2023-10-16T19:50:12.901 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 90465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-16T19:50:12.902 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000/ 2023-10-16T19:50:12.902 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 295577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-16T19:50:12.902 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e/ 2023-10-16T19:50:12.902 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 584497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-16T19:50:12.902 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k/ 2023-10-16T19:50:12.903 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 246209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-16T19:50:12.903 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e/ 2023-10-16T19:50:12.903 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 925697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-16T19:50:12.903 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf/ 2023-10-16T19:50:12.904 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 302849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-16T19:50:12.904 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb/ 2023-10-16T19:50:12.904 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 514681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-16T19:50:12.904 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf/ 2023-10-16T19:50:12.904 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 93593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-16T19:50:12.905 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe/ 2023-10-16T19:50:12.905 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 734305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-16T19:50:12.905 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf/ 2023-10-16T19:50:12.905 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 151769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-16T19:50:12.905 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 86921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/jme.ko 2023-10-16T19:50:12.905 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/ 2023-10-16T19:50:12.906 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-16T19:50:12.906 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 102289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-16T19:50:12.906 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 122649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-16T19:50:12.906 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/ 2023-10-16T19:50:12.906 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/ 2023-10-16T19:50:12.907 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 727601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-16T19:50:12.907 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 278937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-16T19:50:12.907 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/ 2023-10-16T19:50:12.907 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2023-10-16T19:50:12.908 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1113993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-16T19:50:12.908 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ 2023-10-16T19:50:12.908 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 53105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ks8842.ko 2023-10-16T19:50:12.908 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 40993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2023-10-16T19:50:12.908 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2023-10-16T19:50:12.909 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 95665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-16T19:50:12.909 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/ 2023-10-16T19:50:12.909 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge/ 2023-10-16T19:50:12.909 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 108649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-16T19:50:12.909 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/ 2023-10-16T19:50:12.910 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 73441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-16T19:50:12.910 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 47777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-16T19:50:12.910 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion/ 2023-10-16T19:50:12.910 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 150905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-16T19:50:12.910 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia/ 2023-10-16T19:50:12.911 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 136721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-16T19:50:12.911 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/ 2023-10-16T19:50:12.911 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 50481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-16T19:50:12.911 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 43145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-16T19:50:12.911 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/ 2023-10-16T19:50:12.912 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen/ 2023-10-16T19:50:12.912 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 227897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-16T19:50:12.912 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 87081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-16T19:50:12.912 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2023-10-16T19:50:12.912 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 672049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-16T19:50:12.913 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc/ 2023-10-16T19:50:12.913 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 43017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-16T19:50:12.913 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/ 2023-10-16T19:50:12.913 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 67385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-16T19:50:12.913 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 83985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-16T19:50:12.914 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-16T19:50:12.914 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 189801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-16T19:50:12.914 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/samsung/ 2023-10-16T19:50:12.914 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/samsung/sxgbe/ 2023-10-16T19:50:12.914 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 104241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2023-10-16T19:50:12.915 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc/ 2023-10-16T19:50:12.915 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1053921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-16T19:50:12.915 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan/ 2023-10-16T19:50:12.915 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 47097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-16T19:50:12.916 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/ 2023-10-16T19:50:12.916 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 54817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-16T19:50:12.916 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 63337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-16T19:50:12.916 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/ 2023-10-16T19:50:12.916 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 61601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-16T19:50:12.917 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 67193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-16T19:50:12.917 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 73409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-16T19:50:12.917 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 68177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-16T19:50:12.917 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/ 2023-10-16T19:50:12.917 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/ 2023-10-16T19:50:12.918 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2023-10-16T19:50:12.918 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 39905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-16T19:50:12.918 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2023-10-16T19:50:12.918 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 486961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-16T19:50:12.918 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/ 2023-10-16T19:50:12.919 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 103337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-16T19:50:12.919 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 166161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-16T19:50:12.919 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 66329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-16T19:50:12.919 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 78801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-16T19:50:12.919 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti/ 2023-10-16T19:50:12.920 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 50057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-16T19:50:12.920 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti/ 2023-10-16T19:50:12.920 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 86529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-16T19:50:12.920 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/ 2023-10-16T19:50:12.921 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 80273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-16T19:50:12.921 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 83633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-16T19:50:12.921 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/ 2023-10-16T19:50:12.921 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 46241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-16T19:50:12.921 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-16T19:50:12.922 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom/ 2023-10-16T19:50:12.922 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 60057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-16T19:50:12.922 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/fddi/ 2023-10-16T19:50:12.922 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 41065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/fddi/defxx.ko 2023-10-16T19:50:12.922 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/fddi/skfp/ 2023-10-16T19:50:12.923 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 216985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/fddi/skfp/skfp.ko 2023-10-16T19:50:12.923 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 77345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/geneve.ko 2023-10-16T19:50:12.923 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv/ 2023-10-16T19:50:12.923 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 235521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-16T19:50:12.923 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154/ 2023-10-16T19:50:12.924 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-16T19:50:12.924 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ifb.ko 2023-10-16T19:50:12.924 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan/ 2023-10-16T19:50:12.924 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 83769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-16T19:50:12.924 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 62209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvlan.ko 2023-10-16T19:50:12.925 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvtap.ko 2023-10-16T19:50:12.925 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/ 2023-10-16T19:50:12.925 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-16T19:50:12.925 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-16T19:50:12.925 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-gpio.ko 2023-10-16T19:50:12.926 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-16T19:50:12.926 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio.ko 2023-10-16T19:50:12.926 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mii.ko 2023-10-16T19:50:12.926 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 43953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/netconsole.ko 2023-10-16T19:50:12.926 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/nlmon.ko 2023-10-16T19:50:12.927 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/ 2023-10-16T19:50:12.927 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-16T19:50:12.927 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 38977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-16T19:50:12.927 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/ 2023-10-16T19:50:12.927 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/amd.ko 2023-10-16T19:50:12.928 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 50097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-16T19:50:12.928 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 44505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-16T19:50:12.928 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-16T19:50:12.928 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 42241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/broadcom.ko 2023-10-16T19:50:12.928 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/cicada.ko 2023-10-16T19:50:12.929 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/davicom.ko 2023-10-16T19:50:12.929 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/et1011c.ko 2023-10-16T19:50:12.929 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/icplus.ko 2023-10-16T19:50:12.929 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/lxt.ko 2023-10-16T19:50:12.929 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 82049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/marvell.ko 2023-10-16T19:50:12.930 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 76177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/micrel.ko 2023-10-16T19:50:12.930 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/national.ko 2023-10-16T19:50:12.930 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 97857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/phylink.ko 2023-10-16T19:50:12.930 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/qsemi.ko 2023-10-16T19:50:12.930 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 50065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/realtek.ko 2023-10-16T19:50:12.931 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/smsc.ko 2023-10-16T19:50:12.931 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-16T19:50:12.931 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/vitesse.ko 2023-10-16T19:50:12.931 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/plip/ 2023-10-16T19:50:12.931 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 43569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/plip/plip.ko 2023-10-16T19:50:12.932 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ 2023-10-16T19:50:12.932 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-16T19:50:12.932 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-16T19:50:12.932 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-16T19:50:12.932 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-16T19:50:12.933 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-16T19:50:12.933 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 42545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppoe.ko 2023-10-16T19:50:12.933 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppox.ko 2023-10-16T19:50:12.933 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pptp.ko 2023-10-16T19:50:12.933 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/sb1000.ko 2023-10-16T19:50:12.934 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/ 2023-10-16T19:50:12.934 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 42465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/slip.ko 2023-10-16T19:50:12.934 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/sungem_phy.ko 2023-10-16T19:50:12.934 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 64385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/tap.ko 2023-10-16T19:50:12.934 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/ 2023-10-16T19:50:12.935 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 106169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team.ko 2023-10-16T19:50:12.935 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-16T19:50:12.935 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-16T19:50:12.935 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-16T19:50:12.935 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_random.ko 2023-10-16T19:50:12.936 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-16T19:50:12.936 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 87017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/veth.ko 2023-10-16T19:50:12.936 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3/ 2023-10-16T19:50:12.936 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 144297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-16T19:50:12.936 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan/ 2023-10-16T19:50:12.937 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 297329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-16T19:50:12.937 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback/ 2023-10-16T19:50:12.937 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 144649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-16T19:50:12.937 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb/ 2023-10-16T19:50:12.937 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb/ntb.ko 2023-10-16T19:50:12.938 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/ 2023-10-16T19:50:12.938 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/ 2023-10-16T19:50:12.938 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 264937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-16T19:50:12.938 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 102209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme.ko 2023-10-16T19:50:12.938 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/ 2023-10-16T19:50:12.939 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 133841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport.ko 2023-10-16T19:50:12.939 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_cs.ko 2023-10-16T19:50:12.939 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 84465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_pc.ko 2023-10-16T19:50:12.939 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_serial.ko 2023-10-16T19:50:12.939 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/ 2023-10-16T19:50:12.940 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/ 2023-10-16T19:50:12.940 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-16T19:50:12.940 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2023-10-16T19:50:12.940 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2023-10-16T19:50:12.940 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/pci-stub.ko 2023-10-16T19:50:12.940 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 46529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/xen-pcifront.ko 2023-10-16T19:50:12.941 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/ 2023-10-16T19:50:12.941 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/i82092.ko 2023-10-16T19:50:12.941 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 145809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pcmcia.ko 2023-10-16T19:50:12.941 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 62137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pcmcia_core.ko 2023-10-16T19:50:12.941 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2023-10-16T19:50:12.942 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pd6729.ko 2023-10-16T19:50:12.942 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 112889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-16T19:50:12.942 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/ 2023-10-16T19:50:12.942 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/broadcom/ 2023-10-16T19:50:12.942 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2023-10-16T19:50:12.943 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/samsung/ 2023-10-16T19:50:12.943 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2023-10-16T19:50:12.943 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/ 2023-10-16T19:50:12.943 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel/ 2023-10-16T19:50:12.944 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 70153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-16T19:50:12.944 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/ 2023-10-16T19:50:12.944 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/ 2023-10-16T19:50:12.944 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 50889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2023-10-16T19:50:12.944 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec.ko 2023-10-16T19:50:12.944 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2023-10-16T19:50:12.945 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2023-10-16T19:50:12.945 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2023-10-16T19:50:12.945 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2023-10-16T19:50:12.945 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2023-10-16T19:50:12.945 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/ 2023-10-16T19:50:12.946 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 67097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-16T19:50:12.946 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 39145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-16T19:50:12.946 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-16T19:50:12.946 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 59665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-16T19:50:12.946 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-16T19:50:12.947 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 127569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-16T19:50:12.947 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-16T19:50:12.947 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 49121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-16T19:50:12.947 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-16T19:50:12.947 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 40265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-16T19:50:12.948 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-16T19:50:12.948 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/hdaps.ko 2023-10-16T19:50:12.948 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/ibm_rtl.ko 2023-10-16T19:50:12.948 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel/ 2023-10-16T19:50:12.948 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-16T19:50:12.949 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2023-10-16T19:50:12.949 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 43497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-16T19:50:12.949 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-16T19:50:12.949 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-16T19:50:12.949 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-16T19:50:12.950 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 42329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-16T19:50:12.950 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-16T19:50:12.950 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-16T19:50:12.950 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-16T19:50:12.950 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-16T19:50:12.951 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-16T19:50:12.951 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 57569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi.ko 2023-10-16T19:50:12.951 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/ 2023-10-16T19:50:12.951 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ 2023-10-16T19:50:12.951 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/88pm860x_battery.ko 2023-10-16T19:50:12.952 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/88pm860x_charger.ko 2023-10-16T19:50:12.952 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 55057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/bq2415x_charger.ko 2023-10-16T19:50:12.952 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 54081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/bq24190_charger.ko 2023-10-16T19:50:12.952 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/bq24735-charger.ko 2023-10-16T19:50:12.952 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/cros_peripheral_charger.ko 2023-10-16T19:50:12.953 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/da9030_battery.ko 2023-10-16T19:50:12.953 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/da9052-battery.ko 2023-10-16T19:50:12.953 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ds2760_battery.ko 2023-10-16T19:50:12.953 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ds2780_battery.ko 2023-10-16T19:50:12.953 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ds2781_battery.ko 2023-10-16T19:50:12.954 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ds2782_battery.ko 2023-10-16T19:50:12.954 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/gpio-charger.ko 2023-10-16T19:50:12.954 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/isp1704_charger.ko 2023-10-16T19:50:12.954 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/lp8727_charger.ko 2023-10-16T19:50:12.954 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max14577_charger.ko 2023-10-16T19:50:12.955 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max17040_battery.ko 2023-10-16T19:50:12.955 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max17042_battery.ko 2023-10-16T19:50:12.955 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max8903_charger.ko 2023-10-16T19:50:12.955 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max8925_power.ko 2023-10-16T19:50:12.955 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/pcf50633-charger.ko 2023-10-16T19:50:12.956 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 38009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/sbs-battery.ko 2023-10-16T19:50:12.956 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/test_power.ko 2023-10-16T19:50:12.956 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/tps65090-charger.ko 2023-10-16T19:50:12.956 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/wm831x_backup.ko 2023-10-16T19:50:12.957 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/wm831x_power.ko 2023-10-16T19:50:12.957 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/wm8350_power.ko 2023-10-16T19:50:12.957 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/ 2023-10-16T19:50:12.957 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 57729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-16T19:50:12.957 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-16T19:50:12.957 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/ 2023-10-16T19:50:12.958 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/ 2023-10-16T19:50:12.958 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-16T19:50:12.958 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-16T19:50:12.958 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-16T19:50:12.958 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 41873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/pps_core.ko 2023-10-16T19:50:12.959 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ 2023-10-16T19:50:12.959 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 66385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp.ko 2023-10-16T19:50:12.959 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-16T19:50:12.959 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/ 2023-10-16T19:50:12.959 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-88pm80x.ko 2023-10-16T19:50:12.960 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-88pm860x.ko 2023-10-16T19:50:12.960 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-16T19:50:12.960 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-da9052.ko 2023-10-16T19:50:12.960 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-da9055.ko 2023-10-16T19:50:12.960 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-da9063.ko 2023-10-16T19:50:12.961 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-16T19:50:12.961 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 68577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-16T19:50:12.961 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-16T19:50:12.961 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-16T19:50:12.961 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-16T19:50:12.962 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-16T19:50:12.962 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-16T19:50:12.962 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-16T19:50:12.962 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-16T19:50:12.963 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-16T19:50:12.963 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-16T19:50:12.963 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-16T19:50:12.963 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-16T19:50:12.963 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-lp8788.ko 2023-10-16T19:50:12.964 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 38097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-16T19:50:12.964 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-16T19:50:12.964 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-16T19:50:12.964 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t86.ko 2023-10-16T19:50:12.964 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-16T19:50:12.964 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max8907.ko 2023-10-16T19:50:12.965 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max8925.ko 2023-10-16T19:50:12.965 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max8997.ko 2023-10-16T19:50:12.965 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max8998.ko 2023-10-16T19:50:12.965 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-mc13xxx.ko 2023-10-16T19:50:12.965 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-16T19:50:12.966 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-palmas.ko 2023-10-16T19:50:12.966 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 46633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-16T19:50:12.966 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf50633.ko 2023-10-16T19:50:12.966 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-16T19:50:12.966 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-16T19:50:12.967 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-16T19:50:12.967 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-16T19:50:12.967 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rc5t583.ko 2023-10-16T19:50:12.967 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-16T19:50:12.967 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-16T19:50:12.968 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-16T19:50:12.968 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-16T19:50:12.968 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-16T19:50:12.968 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-s35390a.ko 2023-10-16T19:50:12.968 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-16T19:50:12.969 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-tps6586x.ko 2023-10-16T19:50:12.969 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-tps65910.ko 2023-10-16T19:50:12.969 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-wm831x.ko 2023-10-16T19:50:12.969 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-wm8350.ko 2023-10-16T19:50:12.969 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-16T19:50:12.970 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ 2023-10-16T19:50:12.970 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 77617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-16T19:50:12.970 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 57665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-sas.ko 2023-10-16T19:50:12.970 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 55169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-16T19:50:12.970 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 135185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/BusLogic.ko 2023-10-16T19:50:12.971 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/a100u2w.ko 2023-10-16T19:50:12.971 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid/ 2023-10-16T19:50:12.971 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 272305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-16T19:50:12.971 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 100617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/advansys.ko 2023-10-16T19:50:12.971 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/ 2023-10-16T19:50:12.971 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 356265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-16T19:50:12.972 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 295801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-16T19:50:12.972 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic94xx/ 2023-10-16T19:50:12.972 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 189185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic94xx/aic94xx.ko 2023-10-16T19:50:12.972 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/am53c974.ko 2023-10-16T19:50:12.972 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr/ 2023-10-16T19:50:12.973 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 106881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-16T19:50:12.973 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 38185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/atp870u.ko 2023-10-16T19:50:12.973 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi/ 2023-10-16T19:50:12.973 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 246985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-16T19:50:12.973 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa/ 2023-10-16T19:50:12.974 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1171945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-16T19:50:12.974 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc/ 2023-10-16T19:50:12.974 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 237273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-16T19:50:12.974 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i/ 2023-10-16T19:50:12.974 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 115369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-16T19:50:12.974 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 40585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ch.ko 2023-10-16T19:50:12.975 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor/ 2023-10-16T19:50:12.975 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 308921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-16T19:50:12.975 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/ 2023-10-16T19:50:12.975 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i/ 2023-10-16T19:50:12.975 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 76145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-16T19:50:12.976 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i/ 2023-10-16T19:50:12.976 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 101449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-16T19:50:12.976 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 146689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-16T19:50:12.976 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 75553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dc395x.ko 2023-10-16T19:50:12.976 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 38137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dmx3191d.ko 2023-10-16T19:50:12.977 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r/ 2023-10-16T19:50:12.977 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 212425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-16T19:50:12.977 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 60505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esp_scsi.ko 2023-10-16T19:50:12.977 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/ 2023-10-16T19:50:12.977 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 78209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-16T19:50:12.978 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 158505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-16T19:50:12.978 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fdomain.ko 2023-10-16T19:50:12.978 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic/ 2023-10-16T19:50:12.978 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 263489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-16T19:50:12.979 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 201817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hpsa.ko 2023-10-16T19:50:12.979 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 51209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hptiop.ko 2023-10-16T19:50:12.979 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 49041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-16T19:50:12.979 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/imm.ko 2023-10-16T19:50:12.979 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 44057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/initio.ko 2023-10-16T19:50:12.979 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 266537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ipr.ko 2023-10-16T19:50:12.980 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 101881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ips.ko 2023-10-16T19:50:12.980 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci/ 2023-10-16T19:50:12.980 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 317153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci/isci.ko 2023-10-16T19:50:12.980 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-16T19:50:12.981 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 52961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-16T19:50:12.981 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc/ 2023-10-16T19:50:12.981 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 314865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-16T19:50:12.981 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 145585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi.ko 2023-10-16T19:50:12.981 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 53521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-16T19:50:12.982 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas/ 2023-10-16T19:50:12.982 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 216665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-16T19:50:12.982 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc/ 2023-10-16T19:50:12.982 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 2197249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-16T19:50:12.982 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/ 2023-10-16T19:50:12.983 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 73569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-16T19:50:12.983 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-16T19:50:12.983 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 310617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-16T19:50:12.983 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 84721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid.ko 2023-10-16T19:50:12.983 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas/ 2023-10-16T19:50:12.983 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 659777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-16T19:50:12.984 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas/ 2023-10-16T19:50:12.984 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 119561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-16T19:50:12.984 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 78129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvumi.ko 2023-10-16T19:50:12.984 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/ 2023-10-16T19:50:12.984 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 81625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2023-10-16T19:50:12.984 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2023-10-16T19:50:12.985 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2023-10-16T19:50:12.985 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2023-10-16T19:50:12.985 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001/ 2023-10-16T19:50:12.985 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 364105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-16T19:50:12.985 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 123617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pmcraid.ko 2023-10-16T19:50:12.986 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ppa.ko 2023-10-16T19:50:12.986 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 55361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla1280.ko 2023-10-16T19:50:12.986 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/ 2023-10-16T19:50:12.986 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1876625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-16T19:50:12.986 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 85289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-16T19:50:12.987 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx/ 2023-10-16T19:50:12.987 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 559033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-16T19:50:12.987 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qlogicfas408.ko 2023-10-16T19:50:12.987 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/raid_class.ko 2023-10-16T19:50:12.987 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 238825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_debug.ko 2023-10-16T19:50:12.987 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 173937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-16T19:50:12.988 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 279313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-16T19:50:12.988 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 97729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-16T19:50:12.988 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 73881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-16T19:50:12.988 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 44993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-16T19:50:12.988 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ses.ko 2023-10-16T19:50:12.989 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 115889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/st.ko 2023-10-16T19:50:12.989 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 46825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/stex.ko 2023-10-16T19:50:12.989 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2/ 2023-10-16T19:50:12.989 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 162121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-16T19:50:12.989 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 42297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-16T19:50:12.989 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 50321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-16T19:50:12.990 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 40025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/wd719x.ko 2023-10-16T19:50:12.990 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 50025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/xen-scsifront.ko 2023-10-16T19:50:12.990 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb/ 2023-10-16T19:50:12.990 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 129441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb/ssb.ko 2023-10-16T19:50:12.990 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/ 2023-10-16T19:50:12.991 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi/ 2023-10-16T19:50:12.991 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 736601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-16T19:50:12.991 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback/ 2023-10-16T19:50:12.991 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 49985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-16T19:50:12.992 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 39713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_file.ko 2023-10-16T19:50:12.992 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 43025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_iblock.ko 2023-10-16T19:50:12.992 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 930321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_mod.ko 2023-10-16T19:50:12.992 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 44305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_pscsi.ko 2023-10-16T19:50:12.992 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 122521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_user.ko 2023-10-16T19:50:12.992 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc/ 2023-10-16T19:50:12.993 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 75593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-16T19:50:12.993 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/ 2023-10-16T19:50:12.993 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/ 2023-10-16T19:50:12.993 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/ 2023-10-16T19:50:12.993 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-16T19:50:12.994 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-16T19:50:12.994 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-16T19:50:12.994 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-16T19:50:12.994 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-16T19:50:12.994 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-16T19:50:12.995 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-16T19:50:12.995 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-16T19:50:12.995 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-16T19:50:12.995 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-16T19:50:12.995 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-16T19:50:12.996 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 37065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-16T19:50:12.996 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_powerclamp.ko 2023-10-16T19:50:12.996 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-16T19:50:12.996 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-16T19:50:12.996 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-16T19:50:12.997 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ 2023-10-16T19:50:12.997 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless/ 2023-10-16T19:50:12.997 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 78425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-16T19:50:12.997 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 57865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/moxa.ko 2023-10-16T19:50:12.997 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 50953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/mxser.ko 2023-10-16T19:50:12.998 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 108601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_gsm.ko 2023-10-16T19:50:12.998 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_hdlc.ko 2023-10-16T19:50:12.998 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 67929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/nozomi.ko 2023-10-16T19:50:12.998 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/ 2023-10-16T19:50:12.999 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250/ 2023-10-16T19:50:12.999 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250/8250_dw.ko 2023-10-16T19:50:12.999 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 59497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-16T19:50:12.999 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/altera_jtaguart.ko 2023-10-16T19:50:12.999 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/altera_uart.ko 2023-10-16T19:50:13.000 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-16T19:50:13.000 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 72425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/fsl_lpuart.ko 2023-10-16T19:50:13.000 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm/ 2023-10-16T19:50:13.000 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 103073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-16T19:50:13.000 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/rp2.ko 2023-10-16T19:50:13.001 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 52817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/sc16is7xx.ko 2023-10-16T19:50:13.001 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/ 2023-10-16T19:50:13.001 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core/ 2023-10-16T19:50:13.001 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 531265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-16T19:50:13.001 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host/ 2023-10-16T19:50:13.002 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-16T19:50:13.002 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2023-10-16T19:50:13.002 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/ 2023-10-16T19:50:13.002 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 44513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio.ko 2023-10-16T19:50:13.003 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_aec.ko 2023-10-16T19:50:13.003 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_cif.ko 2023-10-16T19:50:13.003 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_dmem_genirq.ko 2023-10-16T19:50:13.003 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_mf624.ko 2023-10-16T19:50:13.003 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_netx.ko 2023-10-16T19:50:13.004 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-16T19:50:13.004 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_pdrv_genirq.ko 2023-10-16T19:50:13.004 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_sercos3.ko 2023-10-16T19:50:13.004 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/ 2023-10-16T19:50:13.004 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/c67x00/ 2023-10-16T19:50:13.005 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 73297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/c67x00/c67x00.ko 2023-10-16T19:50:13.005 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ 2023-10-16T19:50:13.005 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 144377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ci_hdrc.ko 2023-10-16T19:50:13.005 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2023-10-16T19:50:13.005 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2023-10-16T19:50:13.006 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2023-10-16T19:50:13.006 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/ 2023-10-16T19:50:13.006 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 87089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-16T19:50:13.006 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 51409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-16T19:50:13.007 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 47369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usblp.ko 2023-10-16T19:50:13.007 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 89857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usbtmc.ko 2023-10-16T19:50:13.007 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/common/ 2023-10-16T19:50:13.007 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/common/ulpi.ko 2023-10-16T19:50:13.007 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc2/ 2023-10-16T19:50:13.008 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc2/dwc2_pci.ko 2023-10-16T19:50:13.008 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc3/ 2023-10-16T19:50:13.008 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc3/dwc3-haps.ko 2023-10-16T19:50:13.008 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc3/dwc3-pci.ko 2023-10-16T19:50:13.008 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 444553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc3/dwc3.ko 2023-10-16T19:50:13.009 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/ 2023-10-16T19:50:13.009 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/ 2023-10-16T19:50:13.009 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 60177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/u_ether.ko 2023-10-16T19:50:13.009 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 51881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/u_serial.ko 2023-10-16T19:50:13.009 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2023-10-16T19:50:13.010 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 42449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2023-10-16T19:50:13.010 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2023-10-16T19:50:13.010 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2023-10-16T19:50:13.010 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 92489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2023-10-16T19:50:13.010 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 51049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2023-10-16T19:50:13.011 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 122081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2023-10-16T19:50:13.011 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 57617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2023-10-16T19:50:13.011 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2023-10-16T19:50:13.011 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2023-10-16T19:50:13.011 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 57729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2023-10-16T19:50:13.012 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 85609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2023-10-16T19:50:13.012 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2023-10-16T19:50:13.012 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 58113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2023-10-16T19:50:13.012 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 67337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2023-10-16T19:50:13.012 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/ 2023-10-16T19:50:13.013 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2023-10-16T19:50:13.013 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2023-10-16T19:50:13.013 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2023-10-16T19:50:13.013 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_ether.ko 2023-10-16T19:50:13.013 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2023-10-16T19:50:13.014 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_hid.ko 2023-10-16T19:50:13.014 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2023-10-16T19:50:13.014 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2023-10-16T19:50:13.014 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2023-10-16T19:50:13.015 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_printer.ko 2023-10-16T19:50:13.015 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_serial.ko 2023-10-16T19:50:13.015 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_zero.ko 2023-10-16T19:50:13.015 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 62185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2023-10-16T19:50:13.015 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2023-10-16T19:50:13.016 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 161225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/libcomposite.ko 2023-10-16T19:50:13.016 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/ 2023-10-16T19:50:13.016 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2023-10-16T19:50:13.016 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/bdc/ 2023-10-16T19:50:13.017 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 157681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2023-10-16T19:50:13.017 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/goku_udc.ko 2023-10-16T19:50:13.017 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 52721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/gr_udc.ko 2023-10-16T19:50:13.017 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 69705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2023-10-16T19:50:13.017 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 56033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/mv_udc.ko 2023-10-16T19:50:13.018 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 76057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/net2272.ko 2023-10-16T19:50:13.018 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 75073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/net2280.ko 2023-10-16T19:50:13.018 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 73169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/pch_udc.ko 2023-10-16T19:50:13.018 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 65809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2023-10-16T19:50:13.018 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 66097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2023-10-16T19:50:13.019 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 81505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2023-10-16T19:50:13.019 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 156073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/udc-core.ko 2023-10-16T19:50:13.019 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/ 2023-10-16T19:50:13.019 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/bcma-hcd.ko 2023-10-16T19:50:13.020 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 80113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/isp116x-hcd.ko 2023-10-16T19:50:13.020 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 87353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/oxu210hp-hcd.ko 2023-10-16T19:50:13.020 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 78873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/r8a66597-hcd.ko 2023-10-16T19:50:13.020 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 51329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-16T19:50:13.021 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/sl811_cs.ko 2023-10-16T19:50:13.021 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/ssb-hcd.ko 2023-10-16T19:50:13.021 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/xhci-plat-hcd.ko 2023-10-16T19:50:13.021 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/ 2023-10-16T19:50:13.021 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/mdc800.ko 2023-10-16T19:50:13.022 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/microtek.ko 2023-10-16T19:50:13.022 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ 2023-10-16T19:50:13.022 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 41721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/adutux.ko 2023-10-16T19:50:13.022 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-16T19:50:13.023 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/cypress_cy7c63.ko 2023-10-16T19:50:13.023 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/cytherm.ko 2023-10-16T19:50:13.023 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ehset.ko 2023-10-16T19:50:13.023 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi26.ko 2023-10-16T19:50:13.023 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi62.ko 2023-10-16T19:50:13.024 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ezusb.ko 2023-10-16T19:50:13.024 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/idmouse.ko 2023-10-16T19:50:13.024 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-16T19:50:13.024 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-16T19:50:13.024 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ldusb.ko 2023-10-16T19:50:13.025 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-16T19:50:13.025 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/lvstest.ko 2023-10-16T19:50:13.025 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga/ 2023-10-16T19:50:13.025 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 64841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-16T19:50:13.025 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-16T19:50:13.026 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usb3503.ko 2023-10-16T19:50:13.026 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usblcd.ko 2023-10-16T19:50:13.026 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-16T19:50:13.026 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 70873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usbtest.ko 2023-10-16T19:50:13.027 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 41969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/uss720.ko 2023-10-16T19:50:13.027 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/yurex.ko 2023-10-16T19:50:13.027 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/mon/ 2023-10-16T19:50:13.027 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 72953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/mon/usbmon.ko 2023-10-16T19:50:13.027 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/musb/ 2023-10-16T19:50:13.028 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 301545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/musb/musb_hdrc.ko 2023-10-16T19:50:13.028 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/ 2023-10-16T19:50:13.028 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-16T19:50:13.028 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2023-10-16T19:50:13.028 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-isp1301.ko 2023-10-16T19:50:13.029 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-tahvo.ko 2023-10-16T19:50:13.029 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ 2023-10-16T19:50:13.029 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/aircable.ko 2023-10-16T19:50:13.029 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ark3116.ko 2023-10-16T19:50:13.029 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-16T19:50:13.030 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 38105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ch341.ko 2023-10-16T19:50:13.030 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 70257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cp210x.ko 2023-10-16T19:50:13.030 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-16T19:50:13.030 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 58217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-16T19:50:13.030 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 49817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-16T19:50:13.031 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/empeg.ko 2023-10-16T19:50:13.031 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 44337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/f81232.ko 2023-10-16T19:50:13.031 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 175137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-16T19:50:13.031 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 53113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-16T19:50:13.032 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 137081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-16T19:50:13.032 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 124833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_ti.ko 2023-10-16T19:50:13.032 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 67769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipaq.ko 2023-10-16T19:50:13.032 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipw.ko 2023-10-16T19:50:13.032 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-16T19:50:13.033 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 58233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-16T19:50:13.033 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 107521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan.ko 2023-10-16T19:50:13.033 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-16T19:50:13.033 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-16T19:50:13.033 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-16T19:50:13.034 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-16T19:50:13.034 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/metro-usb.ko 2023-10-16T19:50:13.034 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 66449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7720.ko 2023-10-16T19:50:13.034 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 78217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7840.ko 2023-10-16T19:50:13.034 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 56337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mxuport.ko 2023-10-16T19:50:13.035 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/navman.ko 2023-10-16T19:50:13.035 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/omninet.ko 2023-10-16T19:50:13.035 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/opticon.ko 2023-10-16T19:50:13.035 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 190633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/option.ko 2023-10-16T19:50:13.035 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 39073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/oti6858.ko 2023-10-16T19:50:13.036 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 54353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/pl2303.ko 2023-10-16T19:50:13.036 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcaux.ko 2023-10-16T19:50:13.036 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 37825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcserial.ko 2023-10-16T19:50:13.036 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/quatech2.ko 2023-10-16T19:50:13.036 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-16T19:50:13.037 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 51625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/sierra.ko 2023-10-16T19:50:13.037 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-16T19:50:13.037 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ssu100.ko 2023-10-16T19:50:13.037 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-16T19:50:13.037 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 61489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-16T19:50:13.038 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-16T19:50:13.038 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-16T19:50:13.038 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-16T19:50:13.038 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 125609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usbserial.ko 2023-10-16T19:50:13.038 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/visor.ko 2023-10-16T19:50:13.039 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 37409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-16T19:50:13.039 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/wishbone-serial.ko 2023-10-16T19:50:13.039 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-16T19:50:13.039 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ 2023-10-16T19:50:13.039 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 52913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/uas.ko 2023-10-16T19:50:13.040 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-16T19:50:13.040 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-16T19:50:13.040 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-16T19:50:13.040 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-16T19:50:13.040 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-16T19:50:13.041 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-16T19:50:13.041 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-16T19:50:13.041 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-16T19:50:13.041 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-16T19:50:13.041 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-16T19:50:13.041 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-16T19:50:13.042 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-16T19:50:13.042 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-16T19:50:13.042 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 179969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-16T19:50:13.042 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/ 2023-10-16T19:50:13.043 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 67481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-16T19:50:13.043 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 72177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-16T19:50:13.043 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 107001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-16T19:50:13.043 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/ 2023-10-16T19:50:13.043 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/ 2023-10-16T19:50:13.044 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 154297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-16T19:50:13.044 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-16T19:50:13.044 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 108737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio.ko 2023-10-16T19:50:13.044 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 77609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-16T19:50:13.044 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/ 2023-10-16T19:50:13.045 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 113873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost.ko 2023-10-16T19:50:13.045 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-16T19:50:13.045 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 58089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_net.ko 2023-10-16T19:50:13.045 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 89249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-16T19:50:13.045 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/ 2023-10-16T19:50:13.046 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/ 2023-10-16T19:50:13.046 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/88pm860x_bl.ko 2023-10-16T19:50:13.046 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/aat2870_bl.ko 2023-10-16T19:50:13.046 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/adp5520_bl.ko 2023-10-16T19:50:13.046 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/adp8860_bl.ko 2023-10-16T19:50:13.047 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 39993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/adp8870_bl.ko 2023-10-16T19:50:13.047 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-16T19:50:13.047 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/as3711_bl.ko 2023-10-16T19:50:13.047 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/bd6107.ko 2023-10-16T19:50:13.048 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/da903x_bl.ko 2023-10-16T19:50:13.048 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/da9052_bl.ko 2023-10-16T19:50:13.048 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/gpio_backlight.ko 2023-10-16T19:50:13.048 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/kb3886_bl.ko 2023-10-16T19:50:13.048 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lcd.ko 2023-10-16T19:50:13.049 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lm3533_bl.ko 2023-10-16T19:50:13.049 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lm3639_bl.ko 2023-10-16T19:50:13.049 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lv5207lp.ko 2023-10-16T19:50:13.049 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/max8925_bl.ko 2023-10-16T19:50:13.049 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/pandora_bl.ko 2023-10-16T19:50:13.050 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/pcf50633-backlight.ko 2023-10-16T19:50:13.050 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-16T19:50:13.050 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/wm831x_bl.ko 2023-10-16T19:50:13.050 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/ 2023-10-16T19:50:13.050 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ 2023-10-16T19:50:13.051 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2482.ko 2023-10-16T19:50:13.051 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 38985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2490.ko 2023-10-16T19:50:13.051 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/matrox_w1.ko 2023-10-16T19:50:13.051 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/w1-gpio.ko 2023-10-16T19:50:13.051 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/ 2023-10-16T19:50:13.052 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-16T19:50:13.052 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-16T19:50:13.052 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-16T19:50:13.052 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-16T19:50:13.052 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-16T19:50:13.053 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-16T19:50:13.053 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-16T19:50:13.053 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-16T19:50:13.053 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-16T19:50:13.053 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-16T19:50:13.053 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 56425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-16T19:50:13.054 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 102673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/wire.ko 2023-10-16T19:50:13.054 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ 2023-10-16T19:50:13.054 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/acquirewdt.ko 2023-10-16T19:50:13.054 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/advantechwdt.ko 2023-10-16T19:50:13.055 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-16T19:50:13.055 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-16T19:50:13.055 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/cpu5wdt.ko 2023-10-16T19:50:13.055 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/da9052_wdt.ko 2023-10-16T19:50:13.055 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/da9055_wdt.ko 2023-10-16T19:50:13.056 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/da9063_wdt.ko 2023-10-16T19:50:13.056 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/dw_wdt.ko 2023-10-16T19:50:13.056 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/eurotechwdt.ko 2023-10-16T19:50:13.056 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-16T19:50:13.056 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/hpwdt.ko 2023-10-16T19:50:13.057 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/i6300esb.ko 2023-10-16T19:50:13.057 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-16T19:50:13.057 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-16T19:50:13.057 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-16T19:50:13.057 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ibmasr.ko 2023-10-16T19:50:13.058 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-16T19:50:13.058 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-16T19:50:13.058 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-16T19:50:13.058 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/kempld_wdt.ko 2023-10-16T19:50:13.058 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/machzwd.ko 2023-10-16T19:50:13.059 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/mena21_wdt.ko 2023-10-16T19:50:13.059 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/menf21bmc_wdt.ko 2023-10-16T19:50:13.059 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/nv_tco.ko 2023-10-16T19:50:13.059 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/of_xilinx_wdt.ko 2023-10-16T19:50:13.059 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pc87413_wdt.ko 2023-10-16T19:50:13.059 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 38657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-16T19:50:13.060 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-16T19:50:13.060 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/retu_wdt.ko 2023-10-16T19:50:13.060 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sbc60xxwdt.ko 2023-10-16T19:50:13.060 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sbc_epx_c3.ko 2023-10-16T19:50:13.061 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-16T19:50:13.061 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sc1200wdt.ko 2023-10-16T19:50:13.061 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-16T19:50:13.061 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/smsc37b787_wdt.ko 2023-10-16T19:50:13.061 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/softdog.ko 2023-10-16T19:50:13.062 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-16T19:50:13.062 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/twl4030_wdt.ko 2023-10-16T19:50:13.062 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/via_wdt.ko 2023-10-16T19:50:13.062 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-16T19:50:13.063 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-16T19:50:13.063 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-16T19:50:13.063 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wafer5823wdt.ko 2023-10-16T19:50:13.063 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-16T19:50:13.063 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wm831x_wdt.ko 2023-10-16T19:50:13.063 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wm8350_wdt.ko 2023-10-16T19:50:13.064 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-16T19:50:13.064 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/ 2023-10-16T19:50:13.064 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-evtchn.ko 2023-10-16T19:50:13.064 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-16T19:50:13.064 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 47865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntdev.ko 2023-10-16T19:50:13.065 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback/ 2023-10-16T19:50:13.065 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 176441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-16T19:50:13.065 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 40121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-privcmd.ko 2023-10-16T19:50:13.065 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 71249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-scsiback.ko 2023-10-16T19:50:13.065 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs/ 2023-10-16T19:50:13.065 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-16T19:50:13.066 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ 2023-10-16T19:50:13.066 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p/ 2023-10-16T19:50:13.066 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 166425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p/9p.ko 2023-10-16T19:50:13.066 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/adfs/ 2023-10-16T19:50:13.066 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 61073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/adfs/adfs.ko 2023-10-16T19:50:13.067 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs/ 2023-10-16T19:50:13.067 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 163593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs/affs.ko 2023-10-16T19:50:13.067 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/afs/ 2023-10-16T19:50:13.067 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 932817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/afs/kafs.ko 2023-10-16T19:50:13.067 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/autofs/ 2023-10-16T19:50:13.068 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 101057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/autofs/autofs4.ko 2023-10-16T19:50:13.068 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs/ 2023-10-16T19:50:13.068 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 64545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs/befs.ko 2023-10-16T19:50:13.068 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/bfs/ 2023-10-16T19:50:13.068 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 41257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/bfs/bfs.ko 2023-10-16T19:50:13.068 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/binfmt_misc.ko 2023-10-16T19:50:13.069 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs/ 2023-10-16T19:50:13.069 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 3402585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs/btrfs.ko 2023-10-16T19:50:13.069 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles/ 2023-10-16T19:50:13.069 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 306073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles/cachefiles.ko 2023-10-16T19:50:13.070 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph/ 2023-10-16T19:50:13.070 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1323873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph/ceph.ko 2023-10-16T19:50:13.070 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda/ 2023-10-16T19:50:13.070 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 100801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda/coda.ko 2023-10-16T19:50:13.070 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/configfs/ 2023-10-16T19:50:13.071 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 108393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/configfs/configfs.ko 2023-10-16T19:50:13.071 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs/ 2023-10-16T19:50:13.071 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 29625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs/cramfs.ko 2023-10-16T19:50:13.071 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm/ 2023-10-16T19:50:13.072 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 642577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm/dlm.ko 2023-10-16T19:50:13.072 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/efs/ 2023-10-16T19:50:13.072 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/efs/efs.ko 2023-10-16T19:50:13.072 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs/ 2023-10-16T19:50:13.073 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1505025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs/f2fs.ko 2023-10-16T19:50:13.073 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/ 2023-10-16T19:50:13.073 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/msdos.ko 2023-10-16T19:50:13.073 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/freevxfs/ 2023-10-16T19:50:13.073 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 42545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/freevxfs/freevxfs.ko 2023-10-16T19:50:13.074 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache/ 2023-10-16T19:50:13.074 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 226481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache/fscache.ko 2023-10-16T19:50:13.074 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/ 2023-10-16T19:50:13.074 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/cuse.ko 2023-10-16T19:50:13.075 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2/ 2023-10-16T19:50:13.075 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1231529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2/gfs2.ko 2023-10-16T19:50:13.075 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs/ 2023-10-16T19:50:13.075 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 141209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs/hfs.ko 2023-10-16T19:50:13.075 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus/ 2023-10-16T19:50:13.076 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 222817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus/hfsplus.ko 2023-10-16T19:50:13.076 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hpfs/ 2023-10-16T19:50:13.076 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 192329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hpfs/hpfs.ko 2023-10-16T19:50:13.076 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs/ 2023-10-16T19:50:13.076 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 74065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs/isofs.ko 2023-10-16T19:50:13.077 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs/ 2023-10-16T19:50:13.077 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 419497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs/jfs.ko 2023-10-16T19:50:13.077 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd/ 2023-10-16T19:50:13.077 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 280313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd/lockd.ko 2023-10-16T19:50:13.077 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix/ 2023-10-16T19:50:13.078 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 89193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix/minix.ko 2023-10-16T19:50:13.078 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs/ 2023-10-16T19:50:13.078 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 124649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs/netfs.ko 2023-10-16T19:50:13.078 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/ 2023-10-16T19:50:13.079 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout/ 2023-10-16T19:50:13.079 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 81449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-16T19:50:13.079 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout/ 2023-10-16T19:50:13.079 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 59449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-16T19:50:13.079 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout/ 2023-10-16T19:50:13.080 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 117513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-16T19:50:13.080 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1055649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfs.ko 2023-10-16T19:50:13.080 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 66441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv2.ko 2023-10-16T19:50:13.080 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 120177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv3.ko 2023-10-16T19:50:13.080 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 2041689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv4.ko 2023-10-16T19:50:13.081 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/ 2023-10-16T19:50:13.081 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/grace.ko 2023-10-16T19:50:13.081 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-16T19:50:13.081 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd/ 2023-10-16T19:50:13.081 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1516809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd/nfsd.ko 2023-10-16T19:50:13.082 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2/ 2023-10-16T19:50:13.082 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 592833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2/nilfs2.ko 2023-10-16T19:50:13.082 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/ 2023-10-16T19:50:13.082 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-celtic.ko 2023-10-16T19:50:13.082 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-centeuro.ko 2023-10-16T19:50:13.083 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-croatian.ko 2023-10-16T19:50:13.083 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-cyrillic.ko 2023-10-16T19:50:13.083 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-gaelic.ko 2023-10-16T19:50:13.083 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-greek.ko 2023-10-16T19:50:13.083 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-iceland.ko 2023-10-16T19:50:13.084 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-inuit.ko 2023-10-16T19:50:13.084 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-roman.ko 2023-10-16T19:50:13.084 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-romanian.ko 2023-10-16T19:50:13.084 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-turkish.ko 2023-10-16T19:50:13.084 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_ascii.ko 2023-10-16T19:50:13.085 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1250.ko 2023-10-16T19:50:13.085 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1251.ko 2023-10-16T19:50:13.085 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1255.ko 2023-10-16T19:50:13.085 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp737.ko 2023-10-16T19:50:13.085 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp775.ko 2023-10-16T19:50:13.086 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp850.ko 2023-10-16T19:50:13.086 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp852.ko 2023-10-16T19:50:13.086 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp855.ko 2023-10-16T19:50:13.086 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp857.ko 2023-10-16T19:50:13.086 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp860.ko 2023-10-16T19:50:13.087 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp861.ko 2023-10-16T19:50:13.087 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp862.ko 2023-10-16T19:50:13.087 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp863.ko 2023-10-16T19:50:13.087 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp864.ko 2023-10-16T19:50:13.087 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp865.ko 2023-10-16T19:50:13.087 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp866.ko 2023-10-16T19:50:13.088 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp869.ko 2023-10-16T19:50:13.088 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp874.ko 2023-10-16T19:50:13.088 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 94137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp932.ko 2023-10-16T19:50:13.088 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 142673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp936.ko 2023-10-16T19:50:13.088 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 165745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp949.ko 2023-10-16T19:50:13.089 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 118121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp950.ko 2023-10-16T19:50:13.089 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_euc-jp.ko 2023-10-16T19:50:13.089 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-16T19:50:13.089 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-16T19:50:13.089 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-16T19:50:13.090 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-16T19:50:13.090 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-16T19:50:13.090 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-16T19:50:13.090 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-16T19:50:13.090 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-16T19:50:13.091 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-16T19:50:13.091 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-16T19:50:13.091 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-16T19:50:13.091 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-r.ko 2023-10-16T19:50:13.091 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-16T19:50:13.092 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-u.ko 2023-10-16T19:50:13.092 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_ucs2_utils.ko 2023-10-16T19:50:13.092 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_utf8.ko 2023-10-16T19:50:13.092 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ntfs/ 2023-10-16T19:50:13.092 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 219417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ntfs/ntfs.ko 2023-10-16T19:50:13.093 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ 2023-10-16T19:50:13.093 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster/ 2023-10-16T19:50:13.093 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 244409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-16T19:50:13.093 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm/ 2023-10-16T19:50:13.093 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 440737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-16T19:50:13.093 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs/ 2023-10-16T19:50:13.094 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 54049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-16T19:50:13.094 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 2592433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2.ko 2023-10-16T19:50:13.094 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-16T19:50:13.094 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-16T19:50:13.095 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-16T19:50:13.095 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/omfs/ 2023-10-16T19:50:13.095 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 53433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/omfs/omfs.ko 2023-10-16T19:50:13.095 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs/ 2023-10-16T19:50:13.095 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 352153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs/overlay.ko 2023-10-16T19:50:13.095 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore/ 2023-10-16T19:50:13.096 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 54809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore/ramoops.ko 2023-10-16T19:50:13.096 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/qnx4/ 2023-10-16T19:50:13.096 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/qnx4/qnx4.ko 2023-10-16T19:50:13.096 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/qnx6/ 2023-10-16T19:50:13.096 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 46689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/qnx6/qnx6.ko 2023-10-16T19:50:13.097 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/quota/ 2023-10-16T19:50:13.097 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/quota/quota_tree.ko 2023-10-16T19:50:13.097 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/quota/quota_v1.ko 2023-10-16T19:50:13.097 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/quota/quota_v2.ko 2023-10-16T19:50:13.097 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs/ 2023-10-16T19:50:13.097 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 532673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs/reiserfs.ko 2023-10-16T19:50:13.098 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs/ 2023-10-16T19:50:13.098 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs/romfs.ko 2023-10-16T19:50:13.098 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/ 2023-10-16T19:50:13.098 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client/ 2023-10-16T19:50:13.098 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 2668121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client/cifs.ko 2023-10-16T19:50:13.099 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/ 2023-10-16T19:50:13.099 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-16T19:50:13.099 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_md4.ko 2023-10-16T19:50:13.099 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs/ 2023-10-16T19:50:13.100 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 134641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs/squashfs.ko 2023-10-16T19:50:13.100 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv/ 2023-10-16T19:50:13.100 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 90721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv/sysv.ko 2023-10-16T19:50:13.100 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf/ 2023-10-16T19:50:13.100 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 247737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf/udf.ko 2023-10-16T19:50:13.101 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs/ 2023-10-16T19:50:13.101 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 162497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs/ufs.ko 2023-10-16T19:50:13.101 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs/ 2023-10-16T19:50:13.101 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 4010729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs/xfs.ko 2023-10-16T19:50:13.101 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/ 2023-10-16T19:50:13.102 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/locking/ 2023-10-16T19:50:13.102 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 75489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/locking/locktorture.ko 2023-10-16T19:50:13.102 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/time/ 2023-10-16T19:50:13.102 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/time/test_udelay.ko 2023-10-16T19:50:13.102 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 60553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/torture.ko 2023-10-16T19:50:13.103 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ 2023-10-16T19:50:13.103 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 6833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc-itu-t.ko 2023-10-16T19:50:13.103 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 6529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc7.ko 2023-10-16T19:50:13.103 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc8.ko 2023-10-16T19:50:13.103 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/ 2023-10-16T19:50:13.103 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libarc4.ko 2023-10-16T19:50:13.104 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 37081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libdes.ko 2023-10-16T19:50:13.104 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/interval_tree_test.ko 2023-10-16T19:50:13.104 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/libcrc32c.ko 2023-10-16T19:50:13.104 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lru_cache.ko 2023-10-16T19:50:13.104 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/ 2023-10-16T19:50:13.105 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4_compress.ko 2023-10-16T19:50:13.105 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-16T19:50:13.105 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math/ 2023-10-16T19:50:13.105 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 6617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math/cordic.ko 2023-10-16T19:50:13.105 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/memory-notifier-error-inject.ko 2023-10-16T19:50:13.106 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/notifier-error-inject.ko 2023-10-16T19:50:13.106 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/percpu_test.ko 2023-10-16T19:50:13.106 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/pm-notifier-error-inject.ko 2023-10-16T19:50:13.106 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6/ 2023-10-16T19:50:13.106 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 143433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6/raid6_pq.ko 2023-10-16T19:50:13.107 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/rbtree_test.ko 2023-10-16T19:50:13.107 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon/ 2023-10-16T19:50:13.107 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-16T19:50:13.107 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 38841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test-kstrtox.ko 2023-10-16T19:50:13.107 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test-string_helpers.ko 2023-10-16T19:50:13.107 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 4760425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test_bpf.ko 2023-10-16T19:50:13.108 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 49089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test_firmware.ko 2023-10-16T19:50:13.108 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 6369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test_module.ko 2023-10-16T19:50:13.108 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test_user_copy.ko 2023-10-16T19:50:13.108 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_bm.ko 2023-10-16T19:50:13.108 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_fsm.ko 2023-10-16T19:50:13.109 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_kmp.ko 2023-10-16T19:50:13.109 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/xz/ 2023-10-16T19:50:13.109 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/xz/xz_dec_test.ko 2023-10-16T19:50:13.109 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/mm/ 2023-10-16T19:50:13.109 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/mm/hwpoison-inject.ko 2023-10-16T19:50:13.110 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ 2023-10-16T19:50:13.110 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/ 2023-10-16T19:50:13.110 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 55737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/6lowpan.ko 2023-10-16T19:50:13.110 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_dest.ko 2023-10-16T19:50:13.110 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-16T19:50:13.111 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_hop.ko 2023-10-16T19:50:13.111 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 6985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-16T19:50:13.111 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-16T19:50:13.111 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_routing.ko 2023-10-16T19:50:13.111 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_udp.ko 2023-10-16T19:50:13.112 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/ 2023-10-16T19:50:13.112 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/garp.ko 2023-10-16T19:50:13.112 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/mrp.ko 2023-10-16T19:50:13.112 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/p8022.ko 2023-10-16T19:50:13.112 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/psnap.ko 2023-10-16T19:50:13.113 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/stp.ko 2023-10-16T19:50:13.113 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q/ 2023-10-16T19:50:13.113 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 81737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q/8021q.ko 2023-10-16T19:50:13.113 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/ 2023-10-16T19:50:13.113 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 198337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet.ko 2023-10-16T19:50:13.114 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_fd.ko 2023-10-16T19:50:13.114 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_rdma.ko 2023-10-16T19:50:13.114 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_virtio.ko 2023-10-16T19:50:13.114 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv/ 2023-10-16T19:50:13.115 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 649297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv/batman-adv.ko 2023-10-16T19:50:13.115 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/ 2023-10-16T19:50:13.115 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 63809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/br_netfilter.ko 2023-10-16T19:50:13.115 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 777689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/bridge.ko 2023-10-16T19:50:13.115 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ 2023-10-16T19:50:13.116 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-16T19:50:13.116 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-16T19:50:13.116 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-16T19:50:13.116 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-16T19:50:13.116 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-16T19:50:13.116 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-16T19:50:13.117 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-16T19:50:13.117 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-16T19:50:13.117 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-16T19:50:13.117 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-16T19:50:13.117 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-16T19:50:13.117 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-16T19:50:13.118 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-16T19:50:13.118 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-16T19:50:13.118 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-16T19:50:13.118 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-16T19:50:13.118 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-16T19:50:13.119 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-16T19:50:13.119 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-16T19:50:13.119 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-16T19:50:13.119 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 55929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-16T19:50:13.119 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-16T19:50:13.120 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-16T19:50:13.120 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph/ 2023-10-16T19:50:13.120 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1047017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph/libceph.ko 2023-10-16T19:50:13.120 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/ 2023-10-16T19:50:13.120 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 125697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/pktgen.ko 2023-10-16T19:50:13.121 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/ 2023-10-16T19:50:13.121 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 194209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp.ko 2023-10-16T19:50:13.121 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_diag.ko 2023-10-16T19:50:13.121 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 57881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv4.ko 2023-10-16T19:50:13.121 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 60337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv6.ko 2023-10-16T19:50:13.122 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/ 2023-10-16T19:50:13.122 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 390473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/dsa_core.ko 2023-10-16T19:50:13.122 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_brcm.ko 2023-10-16T19:50:13.122 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_dsa.ko 2023-10-16T19:50:13.122 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_none.ko 2023-10-16T19:50:13.122 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_trailer.ko 2023-10-16T19:50:13.123 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/hsr/ 2023-10-16T19:50:13.123 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 99729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/hsr/hsr.ko 2023-10-16T19:50:13.123 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ 2023-10-16T19:50:13.123 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan/ 2023-10-16T19:50:13.123 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 45233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-16T19:50:13.124 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 254625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154.ko 2023-10-16T19:50:13.124 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 47289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-16T19:50:13.124 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ 2023-10-16T19:50:13.124 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ah4.ko 2023-10-16T19:50:13.124 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 38905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/esp4.ko 2023-10-16T19:50:13.125 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 48401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/fou.ko 2023-10-16T19:50:13.125 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/gre.ko 2023-10-16T19:50:13.125 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 48657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/inet_diag.ko 2023-10-16T19:50:13.125 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 60633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_gre.ko 2023-10-16T19:50:13.125 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 61225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_tunnel.ko 2023-10-16T19:50:13.125 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_vti.ko 2023-10-16T19:50:13.126 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipcomp.ko 2023-10-16T19:50:13.126 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipip.ko 2023-10-16T19:50:13.126 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ 2023-10-16T19:50:13.126 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 37641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-16T19:50:13.126 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-16T19:50:13.127 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-16T19:50:13.127 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 41673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ip_tables.ko 2023-10-16T19:50:13.127 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-16T19:50:13.127 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2023-10-16T19:50:13.127 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-16T19:50:13.128 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-16T19:50:13.128 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-16T19:50:13.128 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_filter.ko 2023-10-16T19:50:13.128 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-16T19:50:13.128 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-16T19:50:13.129 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-16T19:50:13.129 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-16T19:50:13.129 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-16T19:50:13.129 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-16T19:50:13.129 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-16T19:50:13.130 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-16T19:50:13.130 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-16T19:50:13.130 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2023-10-16T19:50:13.130 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-16T19:50:13.130 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-16T19:50:13.131 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-16T19:50:13.131 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-16T19:50:13.131 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-16T19:50:13.131 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_bic.ko 2023-10-16T19:50:13.131 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-16T19:50:13.132 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_diag.ko 2023-10-16T19:50:13.132 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-16T19:50:13.132 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_htcp.ko 2023-10-16T19:50:13.132 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_hybla.ko 2023-10-16T19:50:13.132 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_illinois.ko 2023-10-16T19:50:13.133 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_lp.ko 2023-10-16T19:50:13.133 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_scalable.ko 2023-10-16T19:50:13.133 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_vegas.ko 2023-10-16T19:50:13.133 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_veno.ko 2023-10-16T19:50:13.133 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_westwood.ko 2023-10-16T19:50:13.134 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_yeah.ko 2023-10-16T19:50:13.134 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tunnel4.ko 2023-10-16T19:50:13.134 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_diag.ko 2023-10-16T19:50:13.134 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 41665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_tunnel.ko 2023-10-16T19:50:13.134 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-16T19:50:13.135 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ 2023-10-16T19:50:13.135 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ah6.ko 2023-10-16T19:50:13.135 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 39537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/esp6.ko 2023-10-16T19:50:13.135 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/fou6.ko 2023-10-16T19:50:13.135 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 80897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_gre.ko 2023-10-16T19:50:13.136 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 93577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-16T19:50:13.136 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-16T19:50:13.136 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 53273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_vti.ko 2023-10-16T19:50:13.136 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ipcomp6.ko 2023-10-16T19:50:13.136 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/mip6.ko 2023-10-16T19:50:13.136 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ 2023-10-16T19:50:13.137 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 42169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-16T19:50:13.137 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2023-10-16T19:50:13.137 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-16T19:50:13.137 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-16T19:50:13.137 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-16T19:50:13.138 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-16T19:50:13.138 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-16T19:50:13.138 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-16T19:50:13.138 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-16T19:50:13.138 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-16T19:50:13.139 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-16T19:50:13.139 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-16T19:50:13.139 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-16T19:50:13.139 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-16T19:50:13.139 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-16T19:50:13.139 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-16T19:50:13.140 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-16T19:50:13.140 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 39113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-16T19:50:13.140 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-16T19:50:13.140 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-16T19:50:13.140 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-16T19:50:13.141 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-16T19:50:13.141 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-16T19:50:13.141 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-16T19:50:13.141 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-16T19:50:13.141 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 74073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/sit.ko 2023-10-16T19:50:13.142 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/tunnel6.ko 2023-10-16T19:50:13.142 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-16T19:50:13.142 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key/ 2023-10-16T19:50:13.142 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 82833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key/af_key.ko 2023-10-16T19:50:13.142 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/ 2023-10-16T19:50:13.143 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 125281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_core.ko 2023-10-16T19:50:13.143 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-16T19:50:13.143 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 25953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_eth.ko 2023-10-16T19:50:13.143 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip.ko 2023-10-16T19:50:13.143 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-16T19:50:13.143 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-16T19:50:13.144 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 53241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-16T19:50:13.144 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/lapb/ 2023-10-16T19:50:13.144 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 47977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/lapb/lapb.ko 2023-10-16T19:50:13.144 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc/ 2023-10-16T19:50:13.144 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc/llc.ko 2023-10-16T19:50:13.145 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 267777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc/llc2.ko 2023-10-16T19:50:13.145 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154/ 2023-10-16T19:50:13.145 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 290417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154/mac802154.ko 2023-10-16T19:50:13.145 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls/ 2023-10-16T19:50:13.145 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls/mpls_gso.ko 2023-10-16T19:50:13.145 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ 2023-10-16T19:50:13.146 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ 2023-10-16T19:50:13.146 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 119585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-16T19:50:13.146 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-16T19:50:13.146 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-16T19:50:13.146 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-16T19:50:13.147 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 100297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-16T19:50:13.147 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 101441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-16T19:50:13.147 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 102817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-16T19:50:13.147 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 103481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-16T19:50:13.147 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 112849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-16T19:50:13.148 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 60329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-16T19:50:13.148 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 109593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-16T19:50:13.148 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 113129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-16T19:50:13.148 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 112585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-16T19:50:13.148 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 111585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-16T19:50:13.149 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 114465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-16T19:50:13.149 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-16T19:50:13.149 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ 2023-10-16T19:50:13.149 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 411729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-16T19:50:13.149 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-16T19:50:13.150 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-16T19:50:13.150 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-16T19:50:13.150 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-16T19:50:13.150 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-16T19:50:13.150 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-16T19:50:13.151 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-16T19:50:13.151 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-16T19:50:13.151 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-16T19:50:13.151 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-16T19:50:13.151 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-16T19:50:13.152 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-16T19:50:13.152 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-16T19:50:13.152 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 49257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conncount.ko 2023-10-16T19:50:13.152 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 408257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack.ko 2023-10-16T19:50:13.152 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-16T19:50:13.152 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-16T19:50:13.153 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-16T19:50:13.153 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 118145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-16T19:50:13.153 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-16T19:50:13.153 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-16T19:50:13.153 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 107297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-16T19:50:13.154 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-16T19:50:13.154 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-16T19:50:13.154 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 66641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-16T19:50:13.154 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-16T19:50:13.155 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-16T19:50:13.155 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-16T19:50:13.155 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 89473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table.ko 2023-10-16T19:50:13.155 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-16T19:50:13.155 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 37129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-16T19:50:13.156 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 95417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat.ko 2023-10-16T19:50:13.156 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-16T19:50:13.156 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-16T19:50:13.156 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-16T19:50:13.156 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-16T19:50:13.157 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-16T19:50:13.157 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 44065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-16T19:50:13.157 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 615313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_tables.ko 2023-10-16T19:50:13.157 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 42561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink.ko 2023-10-16T19:50:13.157 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-16T19:50:13.158 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_cttimeout.ko 2023-10-16T19:50:13.158 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 39785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-16T19:50:13.158 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-16T19:50:13.158 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 49377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-16T19:50:13.158 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-16T19:50:13.159 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_compat.ko 2023-10-16T19:50:13.159 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 40721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_ct.ko 2023-10-16T19:50:13.159 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-16T19:50:13.159 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fib.ko 2023-10-16T19:50:13.159 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-16T19:50:13.160 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_hash.ko 2023-10-16T19:50:13.160 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_limit.ko 2023-10-16T19:50:13.160 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_log.ko 2023-10-16T19:50:13.160 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_masq.ko 2023-10-16T19:50:13.160 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_nat.ko 2023-10-16T19:50:13.161 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_numgen.ko 2023-10-16T19:50:13.161 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_osf.ko 2023-10-16T19:50:13.161 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_queue.ko 2023-10-16T19:50:13.161 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_redir.ko 2023-10-16T19:50:13.161 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject.ko 2023-10-16T19:50:13.161 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-16T19:50:13.162 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_socket.ko 2023-10-16T19:50:13.162 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_synproxy.ko 2023-10-16T19:50:13.162 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_tproxy.ko 2023-10-16T19:50:13.162 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 89313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/x_tables.ko 2023-10-16T19:50:13.162 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-16T19:50:13.163 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-16T19:50:13.163 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-16T19:50:13.163 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-16T19:50:13.163 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 18169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CT.ko 2023-10-16T19:50:13.163 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_DSCP.ko 2023-10-16T19:50:13.164 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HL.ko 2023-10-16T19:50:13.164 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HMARK.ko 2023-10-16T19:50:13.164 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 38873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-16T19:50:13.164 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LED.ko 2023-10-16T19:50:13.164 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LOG.ko 2023-10-16T19:50:13.164 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-16T19:50:13.165 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-16T19:50:13.165 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-16T19:50:13.165 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-16T19:50:13.165 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-16T19:50:13.165 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-16T19:50:13.166 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 14337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-16T19:50:13.166 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-16T19:50:13.166 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-16T19:50:13.166 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TEE.ko 2023-10-16T19:50:13.166 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-16T19:50:13.167 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TRACE.ko 2023-10-16T19:50:13.167 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_addrtype.ko 2023-10-16T19:50:13.167 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_bpf.ko 2023-10-16T19:50:13.167 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cgroup.ko 2023-10-16T19:50:13.167 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cluster.ko 2023-10-16T19:50:13.168 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_comment.ko 2023-10-16T19:50:13.168 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connbytes.ko 2023-10-16T19:50:13.168 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlabel.ko 2023-10-16T19:50:13.168 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlimit.ko 2023-10-16T19:50:13.168 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connmark.ko 2023-10-16T19:50:13.169 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_conntrack.ko 2023-10-16T19:50:13.169 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cpu.ko 2023-10-16T19:50:13.169 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dccp.ko 2023-10-16T19:50:13.169 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_devgroup.ko 2023-10-16T19:50:13.169 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dscp.ko 2023-10-16T19:50:13.170 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ecn.ko 2023-10-16T19:50:13.170 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_esp.ko 2023-10-16T19:50:13.170 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 43289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-16T19:50:13.170 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_helper.ko 2023-10-16T19:50:13.170 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hl.ko 2023-10-16T19:50:13.171 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-16T19:50:13.171 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_iprange.ko 2023-10-16T19:50:13.171 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipvs.ko 2023-10-16T19:50:13.171 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_l2tp.ko 2023-10-16T19:50:13.171 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_length.ko 2023-10-16T19:50:13.172 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_limit.ko 2023-10-16T19:50:13.172 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mac.ko 2023-10-16T19:50:13.172 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mark.ko 2023-10-16T19:50:13.172 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_multiport.ko 2023-10-16T19:50:13.172 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nat.ko 2023-10-16T19:50:13.173 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nfacct.ko 2023-10-16T19:50:13.173 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_osf.ko 2023-10-16T19:50:13.173 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_owner.ko 2023-10-16T19:50:13.173 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_physdev.ko 2023-10-16T19:50:13.173 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_pkttype.ko 2023-10-16T19:50:13.174 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_policy.ko 2023-10-16T19:50:13.174 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_quota.ko 2023-10-16T19:50:13.174 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_rateest.ko 2023-10-16T19:50:13.174 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_realm.ko 2023-10-16T19:50:13.174 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_recent.ko 2023-10-16T19:50:13.175 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_sctp.ko 2023-10-16T19:50:13.175 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_set.ko 2023-10-16T19:50:13.175 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 16089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_socket.ko 2023-10-16T19:50:13.175 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_state.ko 2023-10-16T19:50:13.176 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_statistic.ko 2023-10-16T19:50:13.176 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_string.ko 2023-10-16T19:50:13.176 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 8553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-16T19:50:13.176 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_tcpudp.ko 2023-10-16T19:50:13.176 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_time.ko 2023-10-16T19:50:13.177 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_u32.ko 2023-10-16T19:50:13.177 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink/ 2023-10-16T19:50:13.177 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink/netlink_diag.ko 2023-10-16T19:50:13.177 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh/ 2023-10-16T19:50:13.177 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 12425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh/nsh.ko 2023-10-16T19:50:13.178 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/ 2023-10-16T19:50:13.178 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 380033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/openvswitch.ko 2023-10-16T19:50:13.178 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-geneve.ko 2023-10-16T19:50:13.178 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-gre.ko 2023-10-16T19:50:13.178 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 11793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-16T19:50:13.179 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet/ 2023-10-16T19:50:13.179 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet/af_packet_diag.ko 2023-10-16T19:50:13.179 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/phonet/ 2023-10-16T19:50:13.179 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 110201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/phonet/phonet.ko 2023-10-16T19:50:13.179 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 54697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/phonet/pn_pep.ko 2023-10-16T19:50:13.179 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/ 2023-10-16T19:50:13.180 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 251481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds.ko 2023-10-16T19:50:13.180 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 164937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_rdma.ko 2023-10-16T19:50:13.180 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 67377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_tcp.ko 2023-10-16T19:50:13.180 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rxrpc/ 2023-10-16T19:50:13.181 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 819657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rxrpc/rxrpc.ko 2023-10-16T19:50:13.181 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/ 2023-10-16T19:50:13.181 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_csum.ko 2023-10-16T19:50:13.181 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_gact.ko 2023-10-16T19:50:13.181 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_ipt.ko 2023-10-16T19:50:13.181 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_mirred.ko 2023-10-16T19:50:13.182 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_nat.ko 2023-10-16T19:50:13.182 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 43449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_pedit.ko 2023-10-16T19:50:13.182 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_police.ko 2023-10-16T19:50:13.182 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_simple.ko 2023-10-16T19:50:13.182 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_skbedit.ko 2023-10-16T19:50:13.183 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_vlan.ko 2023-10-16T19:50:13.183 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_basic.ko 2023-10-16T19:50:13.183 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_bpf.ko 2023-10-16T19:50:13.183 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_cgroup.ko 2023-10-16T19:50:13.183 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_flow.ko 2023-10-16T19:50:13.184 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_fw.ko 2023-10-16T19:50:13.184 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 38489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_route.ko 2023-10-16T19:50:13.184 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 61409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_u32.ko 2023-10-16T19:50:13.184 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_cmp.ko 2023-10-16T19:50:13.184 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 13409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_ipset.ko 2023-10-16T19:50:13.185 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_meta.ko 2023-10-16T19:50:13.185 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_nbyte.ko 2023-10-16T19:50:13.185 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_text.ko 2023-10-16T19:50:13.185 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_u32.ko 2023-10-16T19:50:13.185 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 21649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_choke.ko 2023-10-16T19:50:13.186 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_codel.ko 2023-10-16T19:50:13.186 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 34633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_drr.ko 2023-10-16T19:50:13.186 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 33609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_fq.ko 2023-10-16T19:50:13.186 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_fq_codel.ko 2023-10-16T19:50:13.187 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 35433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_gred.ko 2023-10-16T19:50:13.187 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 49601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hfsc.ko 2023-10-16T19:50:13.187 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hhf.ko 2023-10-16T19:50:13.187 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 66393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_htb.ko 2023-10-16T19:50:13.187 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_ingress.ko 2023-10-16T19:50:13.187 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 30273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio.ko 2023-10-16T19:50:13.188 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-16T19:50:13.188 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_multiq.ko 2023-10-16T19:50:13.188 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 36137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_netem.ko 2023-10-16T19:50:13.188 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 27497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_pie.ko 2023-10-16T19:50:13.188 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_plug.ko 2023-10-16T19:50:13.189 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 24729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_prio.ko 2023-10-16T19:50:13.189 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 48793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_qfq.ko 2023-10-16T19:50:13.189 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 32905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_red.ko 2023-10-16T19:50:13.189 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 26881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfb.ko 2023-10-16T19:50:13.189 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 31353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfq.ko 2023-10-16T19:50:13.190 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_tbf.ko 2023-10-16T19:50:13.190 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 28105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_teql.ko 2023-10-16T19:50:13.190 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/ 2023-10-16T19:50:13.190 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 929745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp.ko 2023-10-16T19:50:13.190 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 19033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp_diag.ko 2023-10-16T19:50:13.191 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/ 2023-10-16T19:50:13.191 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/ 2023-10-16T19:50:13.191 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 336529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-16T19:50:13.191 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 54737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-16T19:50:13.191 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 1520161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/sunrpc.ko 2023-10-16T19:50:13.192 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma/ 2023-10-16T19:50:13.192 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 707721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-16T19:50:13.192 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/ 2023-10-16T19:50:13.192 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/diag.ko 2023-10-16T19:50:13.192 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 862641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/tipc.ko 2023-10-16T19:50:13.193 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix/ 2023-10-16T19:50:13.193 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix/unix_diag.ko 2023-10-16T19:50:13.193 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/ 2023-10-16T19:50:13.193 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 85337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-16T19:50:13.193 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 68049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-16T19:50:13.194 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 96193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock.ko 2023-10-16T19:50:13.194 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 10705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-16T19:50:13.194 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 17521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-16T19:50:13.194 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/x25/ 2023-10-16T19:50:13.195 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 125177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/x25/x25.ko 2023-10-16T19:50:13.195 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/ 2023-10-16T19:50:13.195 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 22657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/xfrm_algo.ko 2023-10-16T19:50:13.195 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 20649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-16T19:50:13.195 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 102257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/xfrm_user.ko 2023-10-16T19:50:13.195 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/ 2023-10-16T19:50:13.196 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib/ 2023-10-16T19:50:13.196 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 15537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib/irqbypass.ko 2023-10-16T19:50:13.196 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 639863 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.alias 2023-10-16T19:50:13.196 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 623919 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.alias.bin 2023-10-16T19:50:13.196 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 7663 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.builtin 2023-10-16T19:50:13.196 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 23428 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.alias.bin 2023-10-16T19:50:13.197 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 9821 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.bin 2023-10-16T19:50:13.197 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 70004 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.modinfo 2023-10-16T19:50:13.197 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 146313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.dep 2023-10-16T19:50:13.197 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 232548 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.dep.bin 2023-10-16T19:50:13.197 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 210 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.devname 2023-10-16T19:50:13.198 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 72606 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.order 2023-10-16T19:50:13.198 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 879 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.softdep 2023-10-16T19:50:13.198 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 254438 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.symbols 2023-10-16T19:50:13.198 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 309363 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.symbols.bin 2023-10-16T19:50:13.198 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./usr/ 2023-10-16T19:50:13.199 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./usr/share/ 2023-10-16T19:50:13.199 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./usr/share/doc/ 2023-10-16T19:50:13.199 INFO:teuthology.orchestra.run.smithi088.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./usr/share/doc/linux-image-6.6.0-rc5-g47672119149b/ 2023-10-16T19:50:13.199 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 175 2023-10-10 19:56 ./usr/share/doc/linux-image-6.6.0-rc5-g47672119149b/changelog.Debian.gz 2023-10-16T19:50:13.199 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r-- root/root 691 2023-10-10 19:56 ./usr/share/doc/linux-image-6.6.0-rc5-g47672119149b/copyright 2023-10-16T19:50:13.200 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g47672119149b 2023-10-16T19:50:13.200 INFO:teuthology.task.kernel:searching for kernel 6.6.0-rc5-g47672119149b 2023-10-16T19:50:13.201 DEBUG:teuthology.orchestra.run.smithi088:> egrep '(submenu|menuentry.*6.6.0-rc5-g47672119149b).*{' /boot/grub/grub.cfg 2023-10-16T19:50:13.208 INFO:teuthology.orchestra.run.smithi088.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-10-16T19:50:13.208 INFO:teuthology.orchestra.run.smithi088.stdout: menuentry 'Ubuntu, with Linux 6.6.0-rc5-g47672119149b' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-rc5-g47672119149b-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-10-16T19:50:13.208 INFO:teuthology.orchestra.run.smithi088.stdout: menuentry 'Ubuntu, with Linux 6.6.0-rc5-g47672119149b (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-rc5-g47672119149b-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-10-16T19:50:13.209 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2023-10-16T19:50:13.210 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.6.0-rc5-g47672119149b 2023-10-16T19:50:13.210 DEBUG:teuthology.orchestra.run.smithi088:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2023-10-16T19:50:13.251 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi088.front.sepia.ceph.com to complete... 2023-10-16T19:50:13.659 INFO:teuthology.orchestra.run.smithi088.stderr:Sourcing file `/etc/default/grub' 2023-10-16T19:50:13.660 INFO:teuthology.orchestra.run.smithi088.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-16T19:50:13.707 INFO:teuthology.orchestra.run.smithi088.stderr:Generating grub configuration file ... 2023-10-16T19:50:14.007 INFO:teuthology.orchestra.run.smithi088.stderr:Found linux image: /boot/vmlinuz-6.6.0-rc5-g47672119149b 2023-10-16T19:50:14.013 INFO:teuthology.orchestra.run.smithi088.stderr:Found initrd image: /boot/initrd.img-6.6.0-rc5-g47672119149b 2023-10-16T19:50:14.303 INFO:teuthology.orchestra.run.smithi088.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-10-16T19:50:14.309 INFO:teuthology.orchestra.run.smithi088.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-10-16T19:50:14.351 INFO:teuthology.orchestra.run.smithi088.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-10-16T19:50:14.357 INFO:teuthology.orchestra.run.smithi088.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-10-16T19:50:14.521 INFO:teuthology.orchestra.run.smithi088.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-10-16T19:50:14.521 INFO:teuthology.orchestra.run.smithi088.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-10-16T19:50:14.521 INFO:teuthology.orchestra.run.smithi088.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-10-16T19:50:14.533 INFO:teuthology.orchestra.run.smithi088.stderr:done 2023-10-16T19:50:41.458 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-10-16T19:50:41.459 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2023-10-16T19:50:41.460 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T19:50:45.674 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-10-16T19:50:45.675 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2023-10-16T19:50:45.675 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T19:50:59.966 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.94 2023-10-16T19:51:04.058 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.88 2023-10-16T19:51:08.970 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2023-10-16T19:51:08.971 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T19:51:12.026 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.94 2023-10-16T19:51:13.060 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2023-10-16T19:51:13.061 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T19:51:16.122 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.88 2023-10-16T19:51:24.034 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2023-10-16T19:51:24.035 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T19:51:28.126 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2023-10-16T19:51:28.127 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T19:51:31.194 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.88 2023-10-16T19:51:46.207 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2023-10-16T19:51:46.208 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T19:51:55.787 DEBUG:teuthology.orchestra.run.smithi094:> true 2023-10-16T19:51:56.496 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2023-10-16T19:51:56.497 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.6.0-rc5-g47672119149b"... 2023-10-16T19:51:56.497 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2023-10-16T19:51:56.551 INFO:teuthology.orchestra.run.smithi094.stdout:6.6.0-rc5-g47672119149b 2023-10-16T19:51:56.552 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g47672119149b vs 6.6.0-rc5-g47672119149b 2023-10-16T19:51:56.552 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-16T19:51:56.552 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-10-16T19:51:57.554 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-10-16T19:51:57.554 DEBUG:teuthology.orchestra.run.smithi094:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-16T19:51:57.637 INFO:teuthology.orchestra.run.smithi094.stdout:ttyS1 2023-10-16T19:51:57.675 DEBUG:teuthology.parallel:result is None 2023-10-16T19:52:18.014 DEBUG:teuthology.orchestra.run.smithi088:> true 2023-10-16T19:52:18.678 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi088.front.sepia.ceph.com' 2023-10-16T19:52:18.678 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.6.0-rc5-g47672119149b"... 2023-10-16T19:52:18.679 DEBUG:teuthology.orchestra.run.smithi088:> uname -r 2023-10-16T19:52:18.728 INFO:teuthology.orchestra.run.smithi088.stdout:6.6.0-rc5-g47672119149b 2023-10-16T19:52:18.729 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g47672119149b vs 6.6.0-rc5-g47672119149b 2023-10-16T19:52:18.729 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-16T19:52:18.729 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-10-16T19:52:19.730 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-10-16T19:52:19.730 DEBUG:teuthology.orchestra.run.smithi088:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-16T19:52:19.818 INFO:teuthology.orchestra.run.smithi088.stdout:ttyS1 2023-10-16T19:52:19.876 DEBUG:teuthology.parallel:result is None 2023-10-16T19:52:19.876 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-16T19:52:19.888 INFO:teuthology.task.internal:Creating test directory... 2023-10-16T19:52:19.888 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-16T19:52:19.892 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-16T19:52:19.902 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-16T19:52:19.908 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-16T19:52:19.924 INFO:teuthology.task.internal:Creating archive directory... 2023-10-16T19:52:19.925 DEBUG:teuthology.orchestra.run.smithi088:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-16T19:52:19.948 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-16T19:52:19.989 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-16T19:52:19.996 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-16T19:52:19.996 DEBUG:teuthology.orchestra.run.smithi088:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-16T19:52:20.004 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-16T19:52:20.031 INFO:teuthology.orchestra.run.smithi088.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-16T19:52:20.046 INFO:teuthology.orchestra.run.smithi088.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-16T19:52:20.059 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-16T19:52:20.073 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-16T19:52:20.076 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-16T19:52:20.083 INFO:teuthology.task.internal:Configuring sudo... 2023-10-16T19:52:20.083 DEBUG:teuthology.orchestra.run.smithi088:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-16T19:52:20.096 DEBUG:teuthology.orchestra.run.smithi094:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-16T19:52:20.145 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-16T19:52:20.154 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-16T19:52:20.155 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-16T19:52:20.164 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-16T19:52:20.196 DEBUG:teuthology.orchestra.run.smithi088:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-16T19:52:20.221 DEBUG:teuthology.orchestra.run.smithi088:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-16T19:52:20.273 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T19:52:20.274 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-16T19:52:20.352 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-16T19:52:20.363 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-16T19:52:20.413 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T19:52:20.413 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-16T19:52:20.488 DEBUG:teuthology.orchestra.run.smithi088:> sudo service rsyslog restart 2023-10-16T19:52:20.492 DEBUG:teuthology.orchestra.run.smithi094:> sudo service rsyslog restart 2023-10-16T19:52:20.629 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-16T19:52:20.637 INFO:teuthology.task.internal:Starting timer... 2023-10-16T19:52:20.637 INFO:teuthology.run_tasks:Running task pcp... 2023-10-16T19:52:20.649 INFO:teuthology.run_tasks:Running task selinux... 2023-10-16T19:52:20.657 DEBUG:teuthology.task.selinux:Excluding smithi088: OS 'ubuntu' does not support SELinux 2023-10-16T19:52:20.658 DEBUG:teuthology.task.selinux:Excluding smithi094: OS 'ubuntu' does not support SELinux 2023-10-16T19:52:20.658 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-16T19:52:20.659 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-10-16T19:52:20.659 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-16T19:52:20.659 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-16T19:52:20.668 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-10-16T19:52:20.673 INFO:teuthology.repo_utils:Fetching main from origin 2023-10-16T19:52:20.808 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-16T19:52:20.825 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-16T19:52:20.827 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi088.front.sepia.ceph.com,smithi094.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-16T20:01:00.434 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi088.front.sepia.ceph.com'), Remote(name='ubuntu@smithi094.front.sepia.ceph.com')] 2023-10-16T20:01:00.435 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2023-10-16T20:01:00.436 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T20:01:00.518 DEBUG:teuthology.orchestra.run.smithi088:> true 2023-10-16T20:01:00.653 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi088.front.sepia.ceph.com' 2023-10-16T20:01:00.654 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2023-10-16T20:01:00.654 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T20:01:00.733 DEBUG:teuthology.orchestra.run.smithi094:> true 2023-10-16T20:01:00.877 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2023-10-16T20:01:00.878 INFO:teuthology.run_tasks:Running task clock... 2023-10-16T20:01:00.889 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-16T20:01:00.890 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-16T20:01:00.891 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-16T20:01:00.894 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-16T20:01:00.895 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-16T20:01:00.931 INFO:teuthology.orchestra.run.smithi088.stdout:16 Oct 20:01:00 ntpd[27971]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-10-16T20:01:00.931 INFO:teuthology.orchestra.run.smithi088.stdout:16 Oct 20:01:00 ntpd[27971]: Command line: ntpd -gq 2023-10-16T20:01:00.931 INFO:teuthology.orchestra.run.smithi088.stdout:16 Oct 20:01:00 ntpd[27971]: ---------------------------------------------------- 2023-10-16T20:01:00.932 INFO:teuthology.orchestra.run.smithi088.stdout:16 Oct 20:01:00 ntpd[27971]: ntp-4 is maintained by Network Time Foundation, 2023-10-16T20:01:00.932 INFO:teuthology.orchestra.run.smithi088.stdout:16 Oct 20:01:00 ntpd[27971]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-10-16T20:01:00.932 INFO:teuthology.orchestra.run.smithi088.stdout:16 Oct 20:01:00 ntpd[27971]: corporation. Support and training for ntp-4 are 2023-10-16T20:01:00.932 INFO:teuthology.orchestra.run.smithi088.stdout:16 Oct 20:01:00 ntpd[27971]: available at https://www.nwtime.org/support 2023-10-16T20:01:00.932 INFO:teuthology.orchestra.run.smithi088.stdout:16 Oct 20:01:00 ntpd[27971]: ---------------------------------------------------- 2023-10-16T20:01:00.933 INFO:teuthology.orchestra.run.smithi088.stdout:16 Oct 20:01:00 ntpd[27971]: proto: precision = 0.041 usec (-24) 2023-10-16T20:01:00.933 INFO:teuthology.orchestra.run.smithi088.stdout:16 Oct 20:01:00 ntpd[27971]: basedate set to 2022-02-04 2023-10-16T20:01:00.934 INFO:teuthology.orchestra.run.smithi088.stdout:16 Oct 20:01:00 ntpd[27971]: gps base set to 2022-02-06 (week 2196) 2023-10-16T20:01:00.934 INFO:teuthology.orchestra.run.smithi088.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-16T20:01:00.937 INFO:teuthology.orchestra.run.smithi088.stdout:16 Oct 20:01:00 ntpd[27971]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-16T20:01:00.937 INFO:teuthology.orchestra.run.smithi088.stdout:16 Oct 20:01:00 ntpd[27971]: restrict ::: KOD does nothing without LIMITED. 2023-10-16T20:01:00.938 INFO:teuthology.orchestra.run.smithi088.stderr:restrict ::: KOD does nothing without LIMITED. 2023-10-16T20:01:00.939 INFO:teuthology.orchestra.run.smithi088.stdout:16 Oct 20:01:00 ntpd[27971]: Listen and drop on 0 v6wildcard [::]:123 2023-10-16T20:01:00.939 INFO:teuthology.orchestra.run.smithi088.stdout:16 Oct 20:01:00 ntpd[27971]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-10-16T20:01:00.939 INFO:teuthology.orchestra.run.smithi088.stdout:16 Oct 20:01:00 ntpd[27971]: Listen normally on 2 lo 127.0.0.1:123 2023-10-16T20:01:00.939 INFO:teuthology.orchestra.run.smithi088.stdout:16 Oct 20:01:00 ntpd[27971]: Listen normally on 3 enp3s0f1 172.21.15.88:123 2023-10-16T20:01:00.939 INFO:teuthology.orchestra.run.smithi088.stdout:16 Oct 20:01:00 ntpd[27971]: Listen normally on 4 lo [::1]:123 2023-10-16T20:01:00.940 INFO:teuthology.orchestra.run.smithi088.stdout:16 Oct 20:01:00 ntpd[27971]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d42d%5]:123 2023-10-16T20:01:00.940 INFO:teuthology.orchestra.run.smithi088.stdout:16 Oct 20:01:00 ntpd[27971]: Listening on routing socket on fd #22 for interface updates 2023-10-16T20:01:00.957 INFO:teuthology.orchestra.run.smithi094.stdout:16 Oct 20:01:00 ntpd[27975]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-10-16T20:01:00.957 INFO:teuthology.orchestra.run.smithi094.stdout:16 Oct 20:01:00 ntpd[27975]: Command line: ntpd -gq 2023-10-16T20:01:00.958 INFO:teuthology.orchestra.run.smithi094.stdout:16 Oct 20:01:00 ntpd[27975]: ---------------------------------------------------- 2023-10-16T20:01:00.958 INFO:teuthology.orchestra.run.smithi094.stdout:16 Oct 20:01:00 ntpd[27975]: ntp-4 is maintained by Network Time Foundation, 2023-10-16T20:01:00.958 INFO:teuthology.orchestra.run.smithi094.stdout:16 Oct 20:01:00 ntpd[27975]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-10-16T20:01:00.958 INFO:teuthology.orchestra.run.smithi094.stdout:16 Oct 20:01:00 ntpd[27975]: corporation. Support and training for ntp-4 are 2023-10-16T20:01:00.959 INFO:teuthology.orchestra.run.smithi094.stdout:16 Oct 20:01:00 ntpd[27975]: available at https://www.nwtime.org/support 2023-10-16T20:01:00.959 INFO:teuthology.orchestra.run.smithi094.stdout:16 Oct 20:01:00 ntpd[27975]: ---------------------------------------------------- 2023-10-16T20:01:00.959 INFO:teuthology.orchestra.run.smithi094.stdout:16 Oct 20:01:00 ntpd[27975]: proto: precision = 0.043 usec (-24) 2023-10-16T20:01:00.960 INFO:teuthology.orchestra.run.smithi094.stdout:16 Oct 20:01:00 ntpd[27975]: basedate set to 2022-02-04 2023-10-16T20:01:00.960 INFO:teuthology.orchestra.run.smithi094.stdout:16 Oct 20:01:00 ntpd[27975]: gps base set to 2022-02-06 (week 2196) 2023-10-16T20:01:00.960 INFO:teuthology.orchestra.run.smithi094.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-16T20:01:00.961 INFO:teuthology.orchestra.run.smithi094.stdout:16 Oct 20:01:00 ntpd[27975]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-16T20:01:00.961 INFO:teuthology.orchestra.run.smithi094.stdout:16 Oct 20:01:00 ntpd[27975]: restrict ::: KOD does nothing without LIMITED. 2023-10-16T20:01:00.961 INFO:teuthology.orchestra.run.smithi094.stderr:restrict ::: KOD does nothing without LIMITED. 2023-10-16T20:01:00.962 INFO:teuthology.orchestra.run.smithi094.stdout:16 Oct 20:01:00 ntpd[27975]: Listen and drop on 0 v6wildcard [::]:123 2023-10-16T20:01:00.963 INFO:teuthology.orchestra.run.smithi094.stdout:16 Oct 20:01:00 ntpd[27975]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-10-16T20:01:00.963 INFO:teuthology.orchestra.run.smithi094.stdout:16 Oct 20:01:00 ntpd[27975]: Listen normally on 2 lo 127.0.0.1:123 2023-10-16T20:01:00.963 INFO:teuthology.orchestra.run.smithi094.stdout:16 Oct 20:01:00 ntpd[27975]: Listen normally on 3 enp3s0f1 172.21.15.94:123 2023-10-16T20:01:00.963 INFO:teuthology.orchestra.run.smithi094.stdout:16 Oct 20:01:00 ntpd[27975]: Listen normally on 4 lo [::1]:123 2023-10-16T20:01:00.963 INFO:teuthology.orchestra.run.smithi094.stdout:16 Oct 20:01:00 ntpd[27975]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d763%5]:123 2023-10-16T20:01:00.964 INFO:teuthology.orchestra.run.smithi094.stdout:16 Oct 20:01:00 ntpd[27975]: Listening on routing socket on fd #22 for interface updates 2023-10-16T20:01:01.932 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:01 ntpd[27971]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:01.933 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:01 ntpd[27971]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:01.933 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:01 ntpd[27971]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:01.933 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:01 ntpd[27971]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:01.933 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:01 ntpd[27971]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:01.934 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:01 ntpd[27971]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:01.934 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:01 ntpd[27971]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T20:01:01.934 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:01 ntpd[27971]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T20:01:01.934 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:01 ntpd[27971]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:01.934 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:01 ntpd[27971]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:01.935 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:01 ntpd[27971]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T20:01:01.935 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:01 ntpd[27971]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T20:01:01.959 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:01 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:01.959 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:01 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:01.959 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:01 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:01.960 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:01 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:01.960 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:01 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:01.960 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:01 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:01.960 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:01 ntpd[27975]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T20:01:01.960 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:01 ntpd[27975]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T20:01:01.961 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:01 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:01.961 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:01 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:01.961 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:01 ntpd[27975]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T20:01:01.961 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:01 ntpd[27975]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T20:01:02.931 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:02 ntpd[27971]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:02.932 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:02 ntpd[27971]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:03.933 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:03 ntpd[27971]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:03.933 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:03 ntpd[27971]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:03.933 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:03 ntpd[27971]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:03.933 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:03 ntpd[27971]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:03.934 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:03 ntpd[27971]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:03.934 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:03 ntpd[27971]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:03.934 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:03 ntpd[27971]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:03.934 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:03 ntpd[27971]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:03.934 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:03 ntpd[27971]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T20:01:03.935 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:03 ntpd[27971]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T20:01:03.935 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:03 ntpd[27971]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:03.935 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:03 ntpd[27971]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:03.935 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:03 ntpd[27971]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T20:01:03.935 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:03 ntpd[27971]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T20:01:03.936 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:03 ntpd[27971]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:03.936 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:03 ntpd[27971]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:03.936 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:03 ntpd[27971]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T20:01:03.936 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:03 ntpd[27971]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T20:01:03.959 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:03 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:03.959 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:03 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:03.959 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:03 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:03.960 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:03 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:03.960 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:03 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:03.960 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:03 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:03.960 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:03 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:03.960 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:03 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:03.961 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:03 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:03.961 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:03 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:03.961 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:03 ntpd[27975]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T20:01:03.961 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:03 ntpd[27975]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T20:01:03.961 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:03 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:03.962 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:03 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:03.962 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:03 ntpd[27975]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T20:01:03.962 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:03 ntpd[27975]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T20:01:03.962 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:03 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:03.962 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:03 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:03.963 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:03 ntpd[27975]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T20:01:03.963 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:03 ntpd[27975]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T20:01:05.932 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:05 ntpd[27971]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:05.933 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:05 ntpd[27971]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:05.933 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:05 ntpd[27971]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:05.933 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:05 ntpd[27971]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:05.933 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:05 ntpd[27971]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:05.934 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:05 ntpd[27971]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:05.934 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:05 ntpd[27971]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:05.934 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:05 ntpd[27971]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:05.934 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:05 ntpd[27971]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T20:01:05.934 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:05 ntpd[27971]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T20:01:05.935 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:05 ntpd[27971]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:05.935 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:05 ntpd[27971]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:05.935 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:05 ntpd[27971]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T20:01:05.935 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:05 ntpd[27971]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T20:01:05.935 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:05 ntpd[27971]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:05.936 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:05 ntpd[27971]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:05.936 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:05 ntpd[27971]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T20:01:05.936 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:05 ntpd[27971]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T20:01:05.959 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:05 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:05.959 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:05 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:05.959 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:05 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:05.960 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:05 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:05.960 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:05 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:05.960 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:05 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:05.960 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:05 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:05.960 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:05 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:05.961 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:05 ntpd[27975]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T20:01:05.961 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:05 ntpd[27975]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T20:01:05.961 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:05 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:05.961 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:05 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:05.961 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:05 ntpd[27975]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T20:01:05.961 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:05 ntpd[27975]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T20:01:05.962 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:05 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:05.962 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:05 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:05.962 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:05 ntpd[27975]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T20:01:05.962 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:05 ntpd[27975]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T20:01:07.932 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:07 ntpd[27971]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:07.933 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:07 ntpd[27971]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:07.933 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:07 ntpd[27971]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:07.933 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:07 ntpd[27971]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:07.933 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:07 ntpd[27971]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:07.933 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:07 ntpd[27971]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:07.934 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:07 ntpd[27971]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:07.934 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:07 ntpd[27971]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:07.934 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:07 ntpd[27971]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T20:01:07.934 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:07 ntpd[27971]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T20:01:07.934 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:07 ntpd[27971]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:07.935 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:07 ntpd[27971]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:07.935 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:07 ntpd[27971]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T20:01:07.935 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:07 ntpd[27971]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T20:01:07.935 INFO:teuthology.orchestra.run.smithi088.stdout:16 Oct 20:01:07 ntpd[27971]: ntpd: time slew +0.000503 s 2023-10-16T20:01:07.936 INFO:teuthology.orchestra.run.smithi088.stdout:ntpd: time slew +0.000503s 2023-10-16T20:01:07.936 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:07 ntpd[27971]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-10-16T20:01:07.936 INFO:teuthology.orchestra.run.smithi088.stderr:16 Oct 20:01:07 ntpd[27971]: can't open /var/log/ntpstats/loopstats.20231016: Permission denied 2023-10-16T20:01:07.958 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:07 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:07.958 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:07 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:07.959 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:07 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:07.959 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:07 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:07.959 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:07 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:07.959 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:07 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:07.959 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:07 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T20:01:07.960 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:07 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T20:01:07.960 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:07 ntpd[27975]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T20:01:07.960 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:07 ntpd[27975]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T20:01:07.960 INFO:teuthology.orchestra.run.smithi094.stdout:16 Oct 20:01:07 ntpd[27975]: ntpd: time slew -0.003861 s 2023-10-16T20:01:07.961 INFO:teuthology.orchestra.run.smithi094.stdout:ntpd: time slew -0.003861s 2023-10-16T20:01:07.961 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:07 ntpd[27975]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-10-16T20:01:07.961 INFO:teuthology.orchestra.run.smithi094.stderr:16 Oct 20:01:07 ntpd[27975]: can't open /var/log/ntpstats/loopstats.20231016: Permission denied 2023-10-16T20:01:08.066 INFO:teuthology.orchestra.run.smithi088.stdout: remote refid st t when poll reach delay offset jitter 2023-10-16T20:01:08.066 INFO:teuthology.orchestra.run.smithi088.stdout:============================================================================== 2023-10-16T20:01:08.067 INFO:teuthology.orchestra.run.smithi088.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-16T20:01:08.067 INFO:teuthology.orchestra.run.smithi088.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-16T20:01:08.067 INFO:teuthology.orchestra.run.smithi088.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-16T20:01:08.067 INFO:teuthology.orchestra.run.smithi088.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-16T20:01:08.069 INFO:teuthology.orchestra.run.smithi094.stdout: remote refid st t when poll reach delay offset jitter 2023-10-16T20:01:08.069 INFO:teuthology.orchestra.run.smithi094.stdout:============================================================================== 2023-10-16T20:01:08.069 INFO:teuthology.orchestra.run.smithi094.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-16T20:01:08.069 INFO:teuthology.orchestra.run.smithi094.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-16T20:01:08.070 INFO:teuthology.orchestra.run.smithi094.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-16T20:01:08.070 INFO:teuthology.orchestra.run.smithi094.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-16T20:01:08.071 INFO:teuthology.run_tasks:Running task install... 2023-10-16T20:01:08.080 DEBUG:teuthology.task.install:project ceph 2023-10-16T20:01:08.080 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0'}} 2023-10-16T20:01:08.081 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0'} 2023-10-16T20:01:08.081 INFO:teuthology.task.install:Using flavor: default 2023-10-16T20:01:08.088 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-16T20:01:08.088 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-16T20:01:08.089 DEBUG:teuthology.orchestra.run.smithi088:> sudo apt-key list | grep Ceph 2023-10-16T20:01:08.089 DEBUG:teuthology.orchestra.run.smithi094:> sudo apt-key list | grep Ceph 2023-10-16T20:01:08.217 INFO:teuthology.orchestra.run.smithi088.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-10-16T20:01:08.225 INFO:teuthology.orchestra.run.smithi094.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-10-16T20:01:08.271 INFO:teuthology.orchestra.run.smithi088.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-10-16T20:01:08.271 INFO:teuthology.orchestra.run.smithi088.stdout:uid [ unknown] Ceph.com (release key) 2023-10-16T20:01:08.272 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-10-16T20:01:08.273 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-10-16T20:01:08.273 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 2023-10-16T20:01:08.279 INFO:teuthology.orchestra.run.smithi094.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-10-16T20:01:08.279 INFO:teuthology.orchestra.run.smithi094.stdout:uid [ unknown] Ceph.com (release key) 2023-10-16T20:01:08.280 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-10-16T20:01:08.280 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-10-16T20:01:08.280 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 2023-10-16T20:01:08.420 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default/ 2023-10-16T20:01:08.421 INFO:teuthology.task.install.deb:Package version is 18.0.0-6730-g4ba8a190-1jammy 2023-10-16T20:01:08.511 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default/ 2023-10-16T20:01:08.511 INFO:teuthology.task.install.deb:Package version is 18.0.0-6730-g4ba8a190-1jammy 2023-10-16T20:01:08.593 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:01:08.594 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-10-16T20:01:08.614 DEBUG:teuthology.orchestra.run.smithi094:> sudo apt-get update 2023-10-16T20:01:08.617 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:01:08.618 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-10-16T20:01:08.638 DEBUG:teuthology.orchestra.run.smithi088:> sudo apt-get update 2023-10-16T20:01:08.808 INFO:teuthology.orchestra.run.smithi094.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-10-16T20:01:08.814 INFO:teuthology.orchestra.run.smithi094.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-10-16T20:01:08.824 INFO:teuthology.orchestra.run.smithi088.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-10-16T20:01:08.830 INFO:teuthology.orchestra.run.smithi088.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-10-16T20:01:08.831 INFO:teuthology.orchestra.run.smithi094.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-10-16T20:01:08.847 INFO:teuthology.orchestra.run.smithi088.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-10-16T20:01:08.870 INFO:teuthology.orchestra.run.smithi094.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-10-16T20:01:08.916 INFO:teuthology.orchestra.run.smithi094.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy InRelease 2023-10-16T20:01:08.934 INFO:teuthology.orchestra.run.smithi088.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy InRelease 2023-10-16T20:01:08.952 INFO:teuthology.orchestra.run.smithi094.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-10-16T20:01:08.970 INFO:teuthology.orchestra.run.smithi088.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-10-16T20:01:08.989 INFO:teuthology.orchestra.run.smithi094.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy Release.gpg 2023-10-16T20:01:08.999 INFO:teuthology.orchestra.run.smithi088.stdout:Hit:6 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-10-16T20:01:09.007 INFO:teuthology.orchestra.run.smithi088.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy Release.gpg 2023-10-16T20:01:09.025 INFO:teuthology.orchestra.run.smithi094.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.5 kB] 2023-10-16T20:01:09.044 INFO:teuthology.orchestra.run.smithi088.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.5 kB] 2023-10-16T20:01:09.119 INFO:teuthology.orchestra.run.smithi094.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,407 B] 2023-10-16T20:01:09.129 INFO:teuthology.orchestra.run.smithi088.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,407 B] 2023-10-16T20:01:09.423 INFO:teuthology.orchestra.run.smithi094.stdout:Fetched 28.6 kB in 1s (48.4 kB/s) 2023-10-16T20:01:09.426 INFO:teuthology.orchestra.run.smithi088.stdout:Fetched 28.6 kB in 1s (50.4 kB/s) 2023-10-16T20:01:10.324 INFO:teuthology.orchestra.run.smithi094.stdout:Reading package lists... 2023-10-16T20:01:10.333 INFO:teuthology.orchestra.run.smithi088.stdout:Reading package lists... 2023-10-16T20:01:10.341 DEBUG:teuthology.orchestra.run.smithi094:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-6730-g4ba8a190-1jammy cephadm=18.0.0-6730-g4ba8a190-1jammy ceph-mds=18.0.0-6730-g4ba8a190-1jammy ceph-mgr=18.0.0-6730-g4ba8a190-1jammy ceph-common=18.0.0-6730-g4ba8a190-1jammy ceph-fuse=18.0.0-6730-g4ba8a190-1jammy ceph-test=18.0.0-6730-g4ba8a190-1jammy ceph-volume=18.0.0-6730-g4ba8a190-1jammy radosgw=18.0.0-6730-g4ba8a190-1jammy python3-rados=18.0.0-6730-g4ba8a190-1jammy python3-rgw=18.0.0-6730-g4ba8a190-1jammy python3-cephfs=18.0.0-6730-g4ba8a190-1jammy python3-rbd=18.0.0-6730-g4ba8a190-1jammy libcephfs2=18.0.0-6730-g4ba8a190-1jammy libcephfs-dev=18.0.0-6730-g4ba8a190-1jammy librados2=18.0.0-6730-g4ba8a190-1jammy librbd1=18.0.0-6730-g4ba8a190-1jammy rbd-fuse=18.0.0-6730-g4ba8a190-1jammy python3-cephfs=18.0.0-6730-g4ba8a190-1jammy cephfs-shell=18.0.0-6730-g4ba8a190-1jammy cephfs-top=18.0.0-6730-g4ba8a190-1jammy cephfs-mirror=18.0.0-6730-g4ba8a190-1jammy 2023-10-16T20:01:10.349 DEBUG:teuthology.orchestra.run.smithi088:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-6730-g4ba8a190-1jammy cephadm=18.0.0-6730-g4ba8a190-1jammy ceph-mds=18.0.0-6730-g4ba8a190-1jammy ceph-mgr=18.0.0-6730-g4ba8a190-1jammy ceph-common=18.0.0-6730-g4ba8a190-1jammy ceph-fuse=18.0.0-6730-g4ba8a190-1jammy ceph-test=18.0.0-6730-g4ba8a190-1jammy ceph-volume=18.0.0-6730-g4ba8a190-1jammy radosgw=18.0.0-6730-g4ba8a190-1jammy python3-rados=18.0.0-6730-g4ba8a190-1jammy python3-rgw=18.0.0-6730-g4ba8a190-1jammy python3-cephfs=18.0.0-6730-g4ba8a190-1jammy python3-rbd=18.0.0-6730-g4ba8a190-1jammy libcephfs2=18.0.0-6730-g4ba8a190-1jammy libcephfs-dev=18.0.0-6730-g4ba8a190-1jammy librados2=18.0.0-6730-g4ba8a190-1jammy librbd1=18.0.0-6730-g4ba8a190-1jammy rbd-fuse=18.0.0-6730-g4ba8a190-1jammy python3-cephfs=18.0.0-6730-g4ba8a190-1jammy cephfs-shell=18.0.0-6730-g4ba8a190-1jammy cephfs-top=18.0.0-6730-g4ba8a190-1jammy cephfs-mirror=18.0.0-6730-g4ba8a190-1jammy 2023-10-16T20:01:10.415 INFO:teuthology.orchestra.run.smithi094.stdout:Reading package lists... 2023-10-16T20:01:10.424 INFO:teuthology.orchestra.run.smithi088.stdout:Reading package lists... 2023-10-16T20:01:10.578 INFO:teuthology.orchestra.run.smithi094.stdout:Building dependency tree... 2023-10-16T20:01:10.578 INFO:teuthology.orchestra.run.smithi094.stdout:Reading state information... 2023-10-16T20:01:10.592 INFO:teuthology.orchestra.run.smithi088.stdout:Building dependency tree... 2023-10-16T20:01:10.593 INFO:teuthology.orchestra.run.smithi088.stdout:Reading state information... 2023-10-16T20:01:10.727 INFO:teuthology.orchestra.run.smithi094.stdout:The following packages were automatically installed and are no longer required: 2023-10-16T20:01:10.728 INFO:teuthology.orchestra.run.smithi094.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-16T20:01:10.728 INFO:teuthology.orchestra.run.smithi094.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-16T20:01:10.728 INFO:teuthology.orchestra.run.smithi094.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-16T20:01:10.728 INFO:teuthology.orchestra.run.smithi094.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-10-16T20:01:10.729 INFO:teuthology.orchestra.run.smithi094.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-10-16T20:01:10.729 INFO:teuthology.orchestra.run.smithi094.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-16T20:01:10.730 INFO:teuthology.orchestra.run.smithi094.stdout:The following additional packages will be installed: 2023-10-16T20:01:10.731 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-10-16T20:01:10.731 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-10-16T20:01:10.731 INFO:teuthology.orchestra.run.smithi094.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-10-16T20:01:10.731 INFO:teuthology.orchestra.run.smithi094.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2023-10-16T20:01:10.731 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2023-10-16T20:01:10.732 INFO:teuthology.orchestra.run.smithi094.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-10-16T20:01:10.732 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-10-16T20:01:10.732 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2023-10-16T20:01:10.732 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2023-10-16T20:01:10.732 INFO:teuthology.orchestra.run.smithi094.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2023-10-16T20:01:10.733 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-10-16T20:01:10.733 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-10-16T20:01:10.733 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-10-16T20:01:10.733 INFO:teuthology.orchestra.run.smithi094.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-10-16T20:01:10.734 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-10-16T20:01:10.734 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-10-16T20:01:10.734 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2023-10-16T20:01:10.734 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2023-10-16T20:01:10.734 INFO:teuthology.orchestra.run.smithi094.stdout: python3-simplejson python3-singledispatch python3-sklearn 2023-10-16T20:01:10.735 INFO:teuthology.orchestra.run.smithi094.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2023-10-16T20:01:10.735 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2023-10-16T20:01:10.735 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2023-10-16T20:01:10.735 INFO:teuthology.orchestra.run.smithi094.stdout: qttranslations5-l10n socat xmlstarlet 2023-10-16T20:01:10.735 INFO:teuthology.orchestra.run.smithi094.stdout:Suggested packages: 2023-10-16T20:01:10.736 INFO:teuthology.orchestra.run.smithi094.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2023-10-16T20:01:10.736 INFO:teuthology.orchestra.run.smithi094.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2023-10-16T20:01:10.736 INFO:teuthology.orchestra.run.smithi094.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2023-10-16T20:01:10.736 INFO:teuthology.orchestra.run.smithi094.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2023-10-16T20:01:10.736 INFO:teuthology.orchestra.run.smithi094.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2023-10-16T20:01:10.737 INFO:teuthology.orchestra.run.smithi094.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2023-10-16T20:01:10.737 INFO:teuthology.orchestra.run.smithi094.stdout: python3-watchdog 2023-10-16T20:01:10.737 INFO:teuthology.orchestra.run.smithi094.stdout:Recommended packages: 2023-10-16T20:01:10.737 INFO:teuthology.orchestra.run.smithi094.stdout: btrfs-tools 2023-10-16T20:01:10.746 INFO:teuthology.orchestra.run.smithi088.stdout:The following packages were automatically installed and are no longer required: 2023-10-16T20:01:10.746 INFO:teuthology.orchestra.run.smithi088.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-16T20:01:10.747 INFO:teuthology.orchestra.run.smithi088.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-16T20:01:10.747 INFO:teuthology.orchestra.run.smithi088.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-16T20:01:10.747 INFO:teuthology.orchestra.run.smithi088.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-10-16T20:01:10.749 INFO:teuthology.orchestra.run.smithi088.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-10-16T20:01:10.749 INFO:teuthology.orchestra.run.smithi088.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-16T20:01:10.749 INFO:teuthology.orchestra.run.smithi088.stdout:The following additional packages will be installed: 2023-10-16T20:01:10.749 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-10-16T20:01:10.749 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-10-16T20:01:10.750 INFO:teuthology.orchestra.run.smithi088.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-10-16T20:01:10.750 INFO:teuthology.orchestra.run.smithi088.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2023-10-16T20:01:10.750 INFO:teuthology.orchestra.run.smithi088.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2023-10-16T20:01:10.750 INFO:teuthology.orchestra.run.smithi088.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-10-16T20:01:10.750 INFO:teuthology.orchestra.run.smithi088.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-10-16T20:01:10.751 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2023-10-16T20:01:10.751 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2023-10-16T20:01:10.751 INFO:teuthology.orchestra.run.smithi088.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2023-10-16T20:01:10.751 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-10-16T20:01:10.751 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-10-16T20:01:10.752 INFO:teuthology.orchestra.run.smithi088.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-10-16T20:01:10.752 INFO:teuthology.orchestra.run.smithi088.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-10-16T20:01:10.752 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-10-16T20:01:10.752 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-10-16T20:01:10.752 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2023-10-16T20:01:10.752 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2023-10-16T20:01:10.753 INFO:teuthology.orchestra.run.smithi088.stdout: python3-simplejson python3-singledispatch python3-sklearn 2023-10-16T20:01:10.753 INFO:teuthology.orchestra.run.smithi088.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2023-10-16T20:01:10.753 INFO:teuthology.orchestra.run.smithi088.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2023-10-16T20:01:10.753 INFO:teuthology.orchestra.run.smithi088.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2023-10-16T20:01:10.753 INFO:teuthology.orchestra.run.smithi088.stdout: qttranslations5-l10n socat xmlstarlet 2023-10-16T20:01:10.754 INFO:teuthology.orchestra.run.smithi088.stdout:Suggested packages: 2023-10-16T20:01:10.754 INFO:teuthology.orchestra.run.smithi088.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2023-10-16T20:01:10.754 INFO:teuthology.orchestra.run.smithi088.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2023-10-16T20:01:10.754 INFO:teuthology.orchestra.run.smithi088.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2023-10-16T20:01:10.755 INFO:teuthology.orchestra.run.smithi088.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2023-10-16T20:01:10.755 INFO:teuthology.orchestra.run.smithi088.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2023-10-16T20:01:10.755 INFO:teuthology.orchestra.run.smithi088.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2023-10-16T20:01:10.755 INFO:teuthology.orchestra.run.smithi088.stdout: python3-watchdog 2023-10-16T20:01:10.755 INFO:teuthology.orchestra.run.smithi088.stdout:Recommended packages: 2023-10-16T20:01:10.755 INFO:teuthology.orchestra.run.smithi088.stdout: btrfs-tools 2023-10-16T20:01:10.871 INFO:teuthology.orchestra.run.smithi094.stdout:The following NEW packages will be installed: 2023-10-16T20:01:10.871 INFO:teuthology.orchestra.run.smithi094.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-10-16T20:01:10.871 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-10-16T20:01:10.871 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-10-16T20:01:10.872 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-10-16T20:01:10.872 INFO:teuthology.orchestra.run.smithi094.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-10-16T20:01:10.872 INFO:teuthology.orchestra.run.smithi094.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2023-10-16T20:01:10.872 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2023-10-16T20:01:10.873 INFO:teuthology.orchestra.run.smithi094.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2023-10-16T20:01:10.874 INFO:teuthology.orchestra.run.smithi094.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2023-10-16T20:01:10.874 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-10-16T20:01:10.874 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2023-10-16T20:01:10.874 INFO:teuthology.orchestra.run.smithi094.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2023-10-16T20:01:10.875 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2023-10-16T20:01:10.875 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2023-10-16T20:01:10.875 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2023-10-16T20:01:10.875 INFO:teuthology.orchestra.run.smithi094.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2023-10-16T20:01:10.875 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2023-10-16T20:01:10.876 INFO:teuthology.orchestra.run.smithi094.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-10-16T20:01:10.876 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2023-10-16T20:01:10.876 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2023-10-16T20:01:10.876 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-10-16T20:01:10.876 INFO:teuthology.orchestra.run.smithi094.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-10-16T20:01:10.877 INFO:teuthology.orchestra.run.smithi094.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-10-16T20:01:10.877 INFO:teuthology.orchestra.run.smithi094.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-10-16T20:01:10.877 INFO:teuthology.orchestra.run.smithi094.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-10-16T20:01:10.877 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2023-10-16T20:01:10.877 INFO:teuthology.orchestra.run.smithi094.stdout: socat xmlstarlet 2023-10-16T20:01:10.890 INFO:teuthology.orchestra.run.smithi088.stdout:The following NEW packages will be installed: 2023-10-16T20:01:10.890 INFO:teuthology.orchestra.run.smithi088.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-10-16T20:01:10.890 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-10-16T20:01:10.891 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-10-16T20:01:10.892 INFO:teuthology.orchestra.run.smithi088.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-10-16T20:01:10.892 INFO:teuthology.orchestra.run.smithi088.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-10-16T20:01:10.892 INFO:teuthology.orchestra.run.smithi088.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2023-10-16T20:01:10.893 INFO:teuthology.orchestra.run.smithi088.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2023-10-16T20:01:10.893 INFO:teuthology.orchestra.run.smithi088.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2023-10-16T20:01:10.893 INFO:teuthology.orchestra.run.smithi088.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2023-10-16T20:01:10.893 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-10-16T20:01:10.893 INFO:teuthology.orchestra.run.smithi088.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2023-10-16T20:01:10.894 INFO:teuthology.orchestra.run.smithi088.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2023-10-16T20:01:10.894 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2023-10-16T20:01:10.894 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2023-10-16T20:01:10.894 INFO:teuthology.orchestra.run.smithi088.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2023-10-16T20:01:10.894 INFO:teuthology.orchestra.run.smithi088.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2023-10-16T20:01:10.895 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2023-10-16T20:01:10.895 INFO:teuthology.orchestra.run.smithi088.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-10-16T20:01:10.895 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2023-10-16T20:01:10.895 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2023-10-16T20:01:10.895 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-10-16T20:01:10.895 INFO:teuthology.orchestra.run.smithi088.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-10-16T20:01:10.896 INFO:teuthology.orchestra.run.smithi088.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-10-16T20:01:10.896 INFO:teuthology.orchestra.run.smithi088.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-10-16T20:01:10.896 INFO:teuthology.orchestra.run.smithi088.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-10-16T20:01:10.896 INFO:teuthology.orchestra.run.smithi088.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2023-10-16T20:01:10.896 INFO:teuthology.orchestra.run.smithi088.stdout: socat xmlstarlet 2023-10-16T20:01:10.960 INFO:teuthology.orchestra.run.smithi094.stdout:0 upgraded, 115 newly installed, 0 to remove and 259 not upgraded. 2023-10-16T20:01:10.961 INFO:teuthology.orchestra.run.smithi094.stdout:Need to get 184 MB of archives. 2023-10-16T20:01:10.961 INFO:teuthology.orchestra.run.smithi094.stdout:After this operation, 826 MB of additional disk space will be used. 2023-10-16T20:01:10.961 INFO:teuthology.orchestra.run.smithi094.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-10-16T20:01:10.983 INFO:teuthology.orchestra.run.smithi094.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-10-16T20:01:10.984 INFO:teuthology.orchestra.run.smithi088.stdout:0 upgraded, 115 newly installed, 0 to remove and 259 not upgraded. 2023-10-16T20:01:10.984 INFO:teuthology.orchestra.run.smithi088.stdout:Need to get 184 MB of archives. 2023-10-16T20:01:10.984 INFO:teuthology.orchestra.run.smithi088.stdout:After this operation, 826 MB of additional disk space will be used. 2023-10-16T20:01:10.984 INFO:teuthology.orchestra.run.smithi088.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-10-16T20:01:10.991 INFO:teuthology.orchestra.run.smithi094.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-10-16T20:01:11.005 INFO:teuthology.orchestra.run.smithi088.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-10-16T20:01:11.018 INFO:teuthology.orchestra.run.smithi088.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-10-16T20:01:11.079 INFO:teuthology.orchestra.run.smithi094.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2023-10-16T20:01:11.099 INFO:teuthology.orchestra.run.smithi094.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-10-16T20:01:11.132 INFO:teuthology.orchestra.run.smithi088.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2023-10-16T20:01:11.150 INFO:teuthology.orchestra.run.smithi088.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-10-16T20:01:11.173 INFO:teuthology.orchestra.run.smithi094.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-10-16T20:01:11.180 INFO:teuthology.orchestra.run.smithi094.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-10-16T20:01:11.182 INFO:teuthology.orchestra.run.smithi094.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.0.0-6730-g4ba8a190-1jammy [3,449 kB] 2023-10-16T20:01:11.208 INFO:teuthology.orchestra.run.smithi088.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.0.0-6730-g4ba8a190-1jammy [3,449 kB] 2023-10-16T20:01:11.223 INFO:teuthology.orchestra.run.smithi088.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-10-16T20:01:11.224 INFO:teuthology.orchestra.run.smithi094.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-10-16T20:01:11.233 INFO:teuthology.orchestra.run.smithi088.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-10-16T20:01:11.277 INFO:teuthology.orchestra.run.smithi088.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-10-16T20:01:11.393 INFO:teuthology.orchestra.run.smithi094.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-10-16T20:01:11.398 INFO:teuthology.orchestra.run.smithi094.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-10-16T20:01:11.440 INFO:teuthology.orchestra.run.smithi088.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-10-16T20:01:11.447 INFO:teuthology.orchestra.run.smithi088.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-10-16T20:01:11.448 INFO:teuthology.orchestra.run.smithi094.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-10-16T20:01:11.457 INFO:teuthology.orchestra.run.smithi094.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-10-16T20:01:11.458 INFO:teuthology.orchestra.run.smithi094.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-10-16T20:01:11.543 INFO:teuthology.orchestra.run.smithi094.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-10-16T20:01:11.544 INFO:teuthology.orchestra.run.smithi094.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-10-16T20:01:11.550 INFO:teuthology.orchestra.run.smithi088.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-10-16T20:01:11.554 INFO:teuthology.orchestra.run.smithi088.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-10-16T20:01:11.554 INFO:teuthology.orchestra.run.smithi088.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-10-16T20:01:11.559 INFO:teuthology.orchestra.run.smithi094.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-10-16T20:01:11.560 INFO:teuthology.orchestra.run.smithi088.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-10-16T20:01:11.561 INFO:teuthology.orchestra.run.smithi094.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-10-16T20:01:11.561 INFO:teuthology.orchestra.run.smithi088.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-10-16T20:01:11.561 INFO:teuthology.orchestra.run.smithi088.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-10-16T20:01:11.563 INFO:teuthology.orchestra.run.smithi088.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-10-16T20:01:11.563 INFO:teuthology.orchestra.run.smithi088.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-10-16T20:01:11.563 INFO:teuthology.orchestra.run.smithi088.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-10-16T20:01:11.563 INFO:teuthology.orchestra.run.smithi094.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-10-16T20:01:11.563 INFO:teuthology.orchestra.run.smithi094.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-10-16T20:01:11.564 INFO:teuthology.orchestra.run.smithi094.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-10-16T20:01:11.564 INFO:teuthology.orchestra.run.smithi094.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-10-16T20:01:11.564 INFO:teuthology.orchestra.run.smithi094.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-10-16T20:01:11.579 INFO:teuthology.orchestra.run.smithi088.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-10-16T20:01:11.583 INFO:teuthology.orchestra.run.smithi094.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-10-16T20:01:11.587 INFO:teuthology.orchestra.run.smithi094.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-10-16T20:01:11.588 INFO:teuthology.orchestra.run.smithi088.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-10-16T20:01:11.588 INFO:teuthology.orchestra.run.smithi094.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-10-16T20:01:11.589 INFO:teuthology.orchestra.run.smithi088.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-10-16T20:01:11.589 INFO:teuthology.orchestra.run.smithi088.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-10-16T20:01:11.591 INFO:teuthology.orchestra.run.smithi088.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-10-16T20:01:11.592 INFO:teuthology.orchestra.run.smithi088.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-10-16T20:01:11.592 INFO:teuthology.orchestra.run.smithi088.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-10-16T20:01:11.593 INFO:teuthology.orchestra.run.smithi088.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-10-16T20:01:11.594 INFO:teuthology.orchestra.run.smithi088.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-10-16T20:01:11.599 INFO:teuthology.orchestra.run.smithi088.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-10-16T20:01:11.602 INFO:teuthology.orchestra.run.smithi094.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-10-16T20:01:11.603 INFO:teuthology.orchestra.run.smithi094.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-10-16T20:01:11.603 INFO:teuthology.orchestra.run.smithi094.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-10-16T20:01:11.604 INFO:teuthology.orchestra.run.smithi094.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-10-16T20:01:11.605 INFO:teuthology.orchestra.run.smithi094.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-10-16T20:01:11.605 INFO:teuthology.orchestra.run.smithi094.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-10-16T20:01:11.606 INFO:teuthology.orchestra.run.smithi094.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-10-16T20:01:11.614 INFO:teuthology.orchestra.run.smithi088.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-10-16T20:01:11.615 INFO:teuthology.orchestra.run.smithi088.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-10-16T20:01:11.616 INFO:teuthology.orchestra.run.smithi088.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-10-16T20:01:11.617 INFO:teuthology.orchestra.run.smithi088.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-10-16T20:01:11.623 INFO:teuthology.orchestra.run.smithi094.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-10-16T20:01:11.627 INFO:teuthology.orchestra.run.smithi088.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-10-16T20:01:11.628 INFO:teuthology.orchestra.run.smithi088.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-10-16T20:01:11.628 INFO:teuthology.orchestra.run.smithi088.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-10-16T20:01:11.629 INFO:teuthology.orchestra.run.smithi088.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-10-16T20:01:11.632 INFO:teuthology.orchestra.run.smithi088.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-10-16T20:01:11.633 INFO:teuthology.orchestra.run.smithi094.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-10-16T20:01:11.633 INFO:teuthology.orchestra.run.smithi094.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-10-16T20:01:11.641 INFO:teuthology.orchestra.run.smithi088.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-10-16T20:01:11.642 INFO:teuthology.orchestra.run.smithi088.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-urllib3 all 1.26.5-1~exp1 [96.4 kB] 2023-10-16T20:01:11.647 INFO:teuthology.orchestra.run.smithi088.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-10-16T20:01:11.649 INFO:teuthology.orchestra.run.smithi094.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-10-16T20:01:11.650 INFO:teuthology.orchestra.run.smithi094.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-10-16T20:01:11.651 INFO:teuthology.orchestra.run.smithi088.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-10-16T20:01:11.656 INFO:teuthology.orchestra.run.smithi094.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-10-16T20:01:11.663 INFO:teuthology.orchestra.run.smithi094.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-10-16T20:01:11.664 INFO:teuthology.orchestra.run.smithi094.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-urllib3 all 1.26.5-1~exp1 [96.4 kB] 2023-10-16T20:01:11.665 INFO:teuthology.orchestra.run.smithi094.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-10-16T20:01:11.667 INFO:teuthology.orchestra.run.smithi094.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-10-16T20:01:11.682 INFO:teuthology.orchestra.run.smithi088.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-10-16T20:01:11.682 INFO:teuthology.orchestra.run.smithi088.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-10-16T20:01:11.687 INFO:teuthology.orchestra.run.smithi088.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-10-16T20:01:11.697 INFO:teuthology.orchestra.run.smithi094.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-10-16T20:01:11.698 INFO:teuthology.orchestra.run.smithi094.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-10-16T20:01:11.705 INFO:teuthology.orchestra.run.smithi094.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-10-16T20:01:11.847 INFO:teuthology.orchestra.run.smithi088.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-10-16T20:01:11.853 INFO:teuthology.orchestra.run.smithi088.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-10-16T20:01:11.855 INFO:teuthology.orchestra.run.smithi088.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-10-16T20:01:11.856 INFO:teuthology.orchestra.run.smithi088.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-10-16T20:01:11.856 INFO:teuthology.orchestra.run.smithi088.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-10-16T20:01:11.857 INFO:teuthology.orchestra.run.smithi088.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-10-16T20:01:11.902 INFO:teuthology.orchestra.run.smithi094.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-10-16T20:01:11.907 INFO:teuthology.orchestra.run.smithi088.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-10-16T20:01:11.911 INFO:teuthology.orchestra.run.smithi088.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-10-16T20:01:11.912 INFO:teuthology.orchestra.run.smithi088.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-10-16T20:01:11.918 INFO:teuthology.orchestra.run.smithi094.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-10-16T20:01:11.924 INFO:teuthology.orchestra.run.smithi094.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-10-16T20:01:11.925 INFO:teuthology.orchestra.run.smithi094.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-10-16T20:01:11.925 INFO:teuthology.orchestra.run.smithi094.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-10-16T20:01:11.928 INFO:teuthology.orchestra.run.smithi094.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-10-16T20:01:11.955 INFO:teuthology.orchestra.run.smithi088.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-10-16T20:01:11.956 INFO:teuthology.orchestra.run.smithi088.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-10-16T20:01:11.956 INFO:teuthology.orchestra.run.smithi088.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-10-16T20:01:11.957 INFO:teuthology.orchestra.run.smithi088.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-10-16T20:01:11.957 INFO:teuthology.orchestra.run.smithi088.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-10-16T20:01:11.958 INFO:teuthology.orchestra.run.smithi088.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-10-16T20:01:11.983 INFO:teuthology.orchestra.run.smithi088.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-10-16T20:01:11.984 INFO:teuthology.orchestra.run.smithi088.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-10-16T20:01:11.989 INFO:teuthology.orchestra.run.smithi088.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-10-16T20:01:12.007 INFO:teuthology.orchestra.run.smithi094.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-10-16T20:01:12.014 INFO:teuthology.orchestra.run.smithi088.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-10-16T20:01:12.017 INFO:teuthology.orchestra.run.smithi088.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-10-16T20:01:12.017 INFO:teuthology.orchestra.run.smithi088.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-10-16T20:01:12.018 INFO:teuthology.orchestra.run.smithi094.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-10-16T20:01:12.020 INFO:teuthology.orchestra.run.smithi094.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-10-16T20:01:12.023 INFO:teuthology.orchestra.run.smithi088.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-10-16T20:01:12.028 INFO:teuthology.orchestra.run.smithi088.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-10-16T20:01:12.028 INFO:teuthology.orchestra.run.smithi088.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-10-16T20:01:12.035 INFO:teuthology.orchestra.run.smithi088.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-10-16T20:01:12.054 INFO:teuthology.orchestra.run.smithi088.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-10-16T20:01:12.054 INFO:teuthology.orchestra.run.smithi088.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-10-16T20:01:12.055 INFO:teuthology.orchestra.run.smithi088.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-10-16T20:01:12.088 INFO:teuthology.orchestra.run.smithi094.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-10-16T20:01:12.089 INFO:teuthology.orchestra.run.smithi094.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-10-16T20:01:12.089 INFO:teuthology.orchestra.run.smithi088.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-10-16T20:01:12.090 INFO:teuthology.orchestra.run.smithi094.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-10-16T20:01:12.091 INFO:teuthology.orchestra.run.smithi088.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-10-16T20:01:12.092 INFO:teuthology.orchestra.run.smithi088.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-10-16T20:01:12.093 INFO:teuthology.orchestra.run.smithi094.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-10-16T20:01:12.095 INFO:teuthology.orchestra.run.smithi094.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-10-16T20:01:12.096 INFO:teuthology.orchestra.run.smithi094.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-10-16T20:01:12.098 INFO:teuthology.orchestra.run.smithi094.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-10-16T20:01:12.100 INFO:teuthology.orchestra.run.smithi094.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-10-16T20:01:12.102 INFO:teuthology.orchestra.run.smithi088.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-10-16T20:01:12.103 INFO:teuthology.orchestra.run.smithi088.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-10-16T20:01:12.103 INFO:teuthology.orchestra.run.smithi088.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-10-16T20:01:12.106 INFO:teuthology.orchestra.run.smithi088.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-10-16T20:01:12.116 INFO:teuthology.orchestra.run.smithi094.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-10-16T20:01:12.124 INFO:teuthology.orchestra.run.smithi094.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-10-16T20:01:12.124 INFO:teuthology.orchestra.run.smithi088.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-10-16T20:01:12.159 INFO:teuthology.orchestra.run.smithi094.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-10-16T20:01:12.160 INFO:teuthology.orchestra.run.smithi094.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-10-16T20:01:12.167 INFO:teuthology.orchestra.run.smithi094.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-10-16T20:01:12.175 INFO:teuthology.orchestra.run.smithi094.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-10-16T20:01:12.176 INFO:teuthology.orchestra.run.smithi094.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-10-16T20:01:12.189 INFO:teuthology.orchestra.run.smithi094.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-10-16T20:01:12.200 INFO:teuthology.orchestra.run.smithi094.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-10-16T20:01:12.200 INFO:teuthology.orchestra.run.smithi094.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-10-16T20:01:12.202 INFO:teuthology.orchestra.run.smithi094.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-10-16T20:01:12.202 INFO:teuthology.orchestra.run.smithi094.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-10-16T20:01:12.208 INFO:teuthology.orchestra.run.smithi094.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-10-16T20:01:12.226 INFO:teuthology.orchestra.run.smithi094.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-10-16T20:01:12.245 INFO:teuthology.orchestra.run.smithi094.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-10-16T20:01:12.246 INFO:teuthology.orchestra.run.smithi094.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-10-16T20:01:12.247 INFO:teuthology.orchestra.run.smithi094.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-10-16T20:01:12.253 INFO:teuthology.orchestra.run.smithi094.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-10-16T20:01:12.255 INFO:teuthology.orchestra.run.smithi094.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-10-16T20:01:13.249 INFO:teuthology.orchestra.run.smithi088.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.0.0-6730-g4ba8a190-1jammy [3,130 kB] 2023-10-16T20:01:13.341 INFO:teuthology.orchestra.run.smithi094.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.0.0-6730-g4ba8a190-1jammy [3,130 kB] 2023-10-16T20:01:14.398 INFO:teuthology.orchestra.run.smithi088.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.0.0-6730-g4ba8a190-1jammy [688 kB] 2023-10-16T20:01:14.519 INFO:teuthology.orchestra.run.smithi094.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.0.0-6730-g4ba8a190-1jammy [688 kB] 2023-10-16T20:01:14.624 INFO:teuthology.orchestra.run.smithi088.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.0.0-6730-g4ba8a190-1jammy [356 kB] 2023-10-16T20:01:14.743 INFO:teuthology.orchestra.run.smithi088.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.0.0-6730-g4ba8a190-1jammy [32.8 kB] 2023-10-16T20:01:14.746 INFO:teuthology.orchestra.run.smithi088.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.0.0-6730-g4ba8a190-1jammy [180 kB] 2023-10-16T20:01:14.755 INFO:teuthology.orchestra.run.smithi094.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.0.0-6730-g4ba8a190-1jammy [356 kB] 2023-10-16T20:01:14.819 INFO:teuthology.orchestra.run.smithi088.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.0.0-6730-g4ba8a190-1jammy [62.4 kB] 2023-10-16T20:01:14.825 INFO:teuthology.orchestra.run.smithi088.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.0.0-6730-g4ba8a190-1jammy [329 kB] 2023-10-16T20:01:14.875 INFO:teuthology.orchestra.run.smithi094.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.0.0-6730-g4ba8a190-1jammy [32.8 kB] 2023-10-16T20:01:14.881 INFO:teuthology.orchestra.run.smithi094.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.0.0-6730-g4ba8a190-1jammy [180 kB] 2023-10-16T20:01:14.934 INFO:teuthology.orchestra.run.smithi088.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.0.0-6730-g4ba8a190-1jammy [6,222 kB] 2023-10-16T20:01:14.945 INFO:teuthology.orchestra.run.smithi094.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.0.0-6730-g4ba8a190-1jammy [62.4 kB] 2023-10-16T20:01:14.963 INFO:teuthology.orchestra.run.smithi094.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.0.0-6730-g4ba8a190-1jammy [329 kB] 2023-10-16T20:01:15.064 INFO:teuthology.orchestra.run.smithi094.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.0.0-6730-g4ba8a190-1jammy [6,222 kB] 2023-10-16T20:01:17.051 INFO:teuthology.orchestra.run.smithi088.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.0.0-6730-g4ba8a190-1jammy [112 kB] 2023-10-16T20:01:17.077 INFO:teuthology.orchestra.run.smithi088.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.0.0-6730-g4ba8a190-1jammy [439 kB] 2023-10-16T20:01:17.250 INFO:teuthology.orchestra.run.smithi088.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.0.0-6730-g4ba8a190-1jammy [22.6 MB] 2023-10-16T20:01:17.264 INFO:teuthology.orchestra.run.smithi094.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.0.0-6730-g4ba8a190-1jammy [112 kB] 2023-10-16T20:01:17.296 INFO:teuthology.orchestra.run.smithi094.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.0.0-6730-g4ba8a190-1jammy [439 kB] 2023-10-16T20:01:17.404 INFO:teuthology.orchestra.run.smithi094.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.0.0-6730-g4ba8a190-1jammy [22.6 MB] 2023-10-16T20:01:21.493 INFO:teuthology.orchestra.run.smithi088.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.0.0-6730-g4ba8a190-1jammy [6,851 kB] 2023-10-16T20:01:21.792 INFO:teuthology.orchestra.run.smithi094.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.0.0-6730-g4ba8a190-1jammy [6,851 kB] 2023-10-16T20:01:22.598 INFO:teuthology.orchestra.run.smithi088.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.0.0-6730-g4ba8a190-1jammy [238 kB] 2023-10-16T20:01:22.634 INFO:teuthology.orchestra.run.smithi088.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.0.0-6730-g4ba8a190-1jammy [128 kB] 2023-10-16T20:01:22.642 INFO:teuthology.orchestra.run.smithi088.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.0.0-6730-g4ba8a190-1jammy [1,496 kB] 2023-10-16T20:01:22.764 INFO:teuthology.orchestra.run.smithi094.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.0.0-6730-g4ba8a190-1jammy [238 kB] 2023-10-16T20:01:22.788 INFO:teuthology.orchestra.run.smithi094.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.0.0-6730-g4ba8a190-1jammy [128 kB] 2023-10-16T20:01:22.805 INFO:teuthology.orchestra.run.smithi094.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.0.0-6730-g4ba8a190-1jammy [1,496 kB] 2023-10-16T20:01:22.836 INFO:teuthology.orchestra.run.smithi088.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.0.0-6730-g4ba8a190-1jammy [9,861 kB] 2023-10-16T20:01:22.972 INFO:teuthology.orchestra.run.smithi094.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.0.0-6730-g4ba8a190-1jammy [9,861 kB] 2023-10-16T20:01:23.958 INFO:teuthology.orchestra.run.smithi088.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.0.0-6730-g4ba8a190-1jammy [31.7 MB] 2023-10-16T20:01:24.057 INFO:teuthology.orchestra.run.smithi094.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.0.0-6730-g4ba8a190-1jammy [31.7 MB] 2023-10-16T20:01:26.379 INFO:teuthology.orchestra.run.smithi088.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.0.0-6730-g4ba8a190-1jammy [14.0 kB] 2023-10-16T20:01:26.379 INFO:teuthology.orchestra.run.smithi088.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.0.0-6730-g4ba8a190-1jammy [839 kB] 2023-10-16T20:01:26.405 INFO:teuthology.orchestra.run.smithi094.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.0.0-6730-g4ba8a190-1jammy [14.0 kB] 2023-10-16T20:01:26.406 INFO:teuthology.orchestra.run.smithi094.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.0.0-6730-g4ba8a190-1jammy [839 kB] 2023-10-16T20:01:26.426 INFO:teuthology.orchestra.run.smithi088.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.0.0-6730-g4ba8a190-1jammy [2,146 kB] 2023-10-16T20:01:26.443 INFO:teuthology.orchestra.run.smithi094.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.0.0-6730-g4ba8a190-1jammy [2,146 kB] 2023-10-16T20:01:26.554 INFO:teuthology.orchestra.run.smithi088.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.0.0-6730-g4ba8a190-1jammy [283 kB] 2023-10-16T20:01:26.572 INFO:teuthology.orchestra.run.smithi094.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.0.0-6730-g4ba8a190-1jammy [283 kB] 2023-10-16T20:01:26.579 INFO:teuthology.orchestra.run.smithi088.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.0.0-6730-g4ba8a190-1jammy [138 kB] 2023-10-16T20:01:26.588 INFO:teuthology.orchestra.run.smithi094.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.0.0-6730-g4ba8a190-1jammy [138 kB] 2023-10-16T20:01:26.594 INFO:teuthology.orchestra.run.smithi088.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.0.0-6730-g4ba8a190-1jammy [1,906 kB] 2023-10-16T20:01:26.606 INFO:teuthology.orchestra.run.smithi094.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.0.0-6730-g4ba8a190-1jammy [1,906 kB] 2023-10-16T20:01:26.720 INFO:teuthology.orchestra.run.smithi088.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.0.0-6730-g4ba8a190-1jammy [8,625 kB] 2023-10-16T20:01:26.730 INFO:teuthology.orchestra.run.smithi094.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.0.0-6730-g4ba8a190-1jammy [8,625 kB] 2023-10-16T20:01:27.294 INFO:teuthology.orchestra.run.smithi088.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.0.0-6730-g4ba8a190-1jammy [14.2 kB] 2023-10-16T20:01:27.294 INFO:teuthology.orchestra.run.smithi088.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.0.0-6730-g4ba8a190-1jammy [47.8 MB] 2023-10-16T20:01:27.303 INFO:teuthology.orchestra.run.smithi094.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.0.0-6730-g4ba8a190-1jammy [14.2 kB] 2023-10-16T20:01:27.303 INFO:teuthology.orchestra.run.smithi094.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.0.0-6730-g4ba8a190-1jammy [47.8 MB] 2023-10-16T20:01:30.423 INFO:teuthology.orchestra.run.smithi094.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.0.0-6730-g4ba8a190-1jammy [120 kB] 2023-10-16T20:01:30.424 INFO:teuthology.orchestra.run.smithi094.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.0.0-6730-g4ba8a190-1jammy [227 kB] 2023-10-16T20:01:30.427 INFO:teuthology.orchestra.run.smithi094.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.0.0-6730-g4ba8a190-1jammy [33.9 kB] 2023-10-16T20:01:30.429 INFO:teuthology.orchestra.run.smithi094.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.0.0-6730-g4ba8a190-1jammy [23.4 kB] 2023-10-16T20:01:30.430 INFO:teuthology.orchestra.run.smithi094.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.0.0-6730-g4ba8a190-1jammy [38.3 kB] 2023-10-16T20:01:30.432 INFO:teuthology.orchestra.run.smithi094.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.0.0-6730-g4ba8a190-1jammy [12.3 MB] 2023-10-16T20:01:30.500 INFO:teuthology.orchestra.run.smithi088.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.0.0-6730-g4ba8a190-1jammy [120 kB] 2023-10-16T20:01:30.501 INFO:teuthology.orchestra.run.smithi088.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.0.0-6730-g4ba8a190-1jammy [227 kB] 2023-10-16T20:01:30.504 INFO:teuthology.orchestra.run.smithi088.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.0.0-6730-g4ba8a190-1jammy [33.9 kB] 2023-10-16T20:01:30.505 INFO:teuthology.orchestra.run.smithi088.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.0.0-6730-g4ba8a190-1jammy [23.4 kB] 2023-10-16T20:01:30.505 INFO:teuthology.orchestra.run.smithi088.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.0.0-6730-g4ba8a190-1jammy [38.3 kB] 2023-10-16T20:01:30.507 INFO:teuthology.orchestra.run.smithi088.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.0.0-6730-g4ba8a190-1jammy [12.3 MB] 2023-10-16T20:01:31.227 INFO:teuthology.orchestra.run.smithi094.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.0.0-6730-g4ba8a190-1jammy [91.6 kB] 2023-10-16T20:01:31.298 INFO:teuthology.orchestra.run.smithi088.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.0.0-6730-g4ba8a190-1jammy [91.6 kB] 2023-10-16T20:01:31.803 INFO:teuthology.orchestra.run.smithi094.stdout:Fetched 184 MB in 20s (9,050 kB/s) 2023-10-16T20:01:31.858 INFO:teuthology.orchestra.run.smithi088.stdout:Fetched 184 MB in 20s (9,026 kB/s) 2023-10-16T20:01:31.973 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-pyperclip. 2023-10-16T20:01:31.977 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pyperclip. 2023-10-16T20:01:32.041 INFO:teuthology.orchestra.run.smithi088.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132932 files and directories currently installed.) 2023-10-16T20:01:32.045 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-10-16T20:01:32.045 INFO:teuthology.orchestra.run.smithi094.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132932 files and directories currently installed.) 2023-10-16T20:01:32.048 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-10-16T20:01:32.080 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-10-16T20:01:32.084 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-10-16T20:01:32.362 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-wcwidth. 2023-10-16T20:01:32.375 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-10-16T20:01:32.383 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-wcwidth. 2023-10-16T20:01:32.397 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-10-16T20:01:32.412 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-10-16T20:01:32.434 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-10-16T20:01:32.723 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-cmd2. 2023-10-16T20:01:32.736 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-10-16T20:01:32.781 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-10-16T20:01:32.786 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-cmd2. 2023-10-16T20:01:32.800 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-10-16T20:01:32.844 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-10-16T20:01:33.100 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libfmt8:amd64. 2023-10-16T20:01:33.113 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2023-10-16T20:01:33.172 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libfmt8:amd64. 2023-10-16T20:01:33.185 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2023-10-16T20:01:33.209 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2023-10-16T20:01:33.264 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2023-10-16T20:01:33.687 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-10-16T20:01:33.700 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-10-16T20:01:33.745 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-10-16T20:01:33.776 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-10-16T20:01:33.789 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-10-16T20:01:33.826 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-10-16T20:01:34.073 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-10-16T20:01:34.086 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-10-16T20:01:34.137 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-10-16T20:01:34.141 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-10-16T20:01:34.151 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-10-16T20:01:34.195 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-10-16T20:01:34.518 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-10-16T20:01:34.532 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-10-16T20:01:34.557 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-10-16T20:01:34.570 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-10-16T20:01:34.576 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-10-16T20:01:34.615 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-10-16T20:01:34.938 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-10-16T20:01:34.952 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-16T20:01:34.996 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T20:01:35.010 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-10-16T20:01:35.024 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-16T20:01:35.068 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T20:01:35.382 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-10-16T20:01:35.396 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-16T20:01:35.440 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T20:01:35.530 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-10-16T20:01:35.544 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-16T20:01:35.589 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T20:01:35.802 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-10-16T20:01:35.817 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-16T20:01:35.860 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T20:01:35.925 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-10-16T20:01:35.939 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-16T20:01:35.983 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T20:01:36.196 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-10-16T20:01:36.211 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-10-16T20:01:36.246 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-10-16T20:01:36.278 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-10-16T20:01:36.292 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-10-16T20:01:36.328 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-10-16T20:01:36.516 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package librados2. 2023-10-16T20:01:36.530 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../011-librados2_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:36.574 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking librados2 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:36.656 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package librados2. 2023-10-16T20:01:36.670 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../011-librados2_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:36.714 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking librados2 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:36.993 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package librbd1. 2023-10-16T20:01:37.008 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../012-librbd1_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:37.051 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking librbd1 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:37.142 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package librbd1. 2023-10-16T20:01:37.157 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../012-librbd1_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:37.200 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking librbd1 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:37.462 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libcephfs2. 2023-10-16T20:01:37.477 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../013-libcephfs2_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:37.520 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libcephfs2 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:37.603 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libcephfs2. 2023-10-16T20:01:37.617 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../013-libcephfs2_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:37.661 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libcephfs2 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:37.798 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-rados. 2023-10-16T20:01:37.812 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../014-python3-rados_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:37.848 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-rados (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:37.897 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-rados. 2023-10-16T20:01:37.911 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../014-python3-rados_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:37.947 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-rados (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:38.192 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-10-16T20:01:38.207 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T20:01:38.250 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-ceph-argparse (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:38.333 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-10-16T20:01:38.347 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T20:01:38.416 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-ceph-argparse (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:38.486 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-cephfs. 2023-10-16T20:01:38.501 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../016-python3-cephfs_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:38.544 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-cephfs (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:38.702 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-cephfs. 2023-10-16T20:01:38.716 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../016-python3-cephfs_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:38.760 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-cephfs (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:38.865 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-ceph-common. 2023-10-16T20:01:38.878 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../017-python3-ceph-common_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T20:01:38.922 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-ceph-common (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:39.113 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-ceph-common. 2023-10-16T20:01:39.127 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../017-python3-ceph-common_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T20:01:39.180 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-ceph-common (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:39.225 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-prettytable. 2023-10-16T20:01:39.240 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2023-10-16T20:01:39.274 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-10-16T20:01:39.577 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-rbd. 2023-10-16T20:01:39.583 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-prettytable. 2023-10-16T20:01:39.592 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../019-python3-rbd_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:39.597 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2023-10-16T20:01:39.635 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-rbd (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:39.649 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-10-16T20:01:39.960 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-rbd. 2023-10-16T20:01:39.974 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../019-python3-rbd_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:39.988 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-10-16T20:01:40.003 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-10-16T20:01:40.027 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-rbd (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:40.046 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-10-16T20:01:40.383 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package librgw2. 2023-10-16T20:01:40.397 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../021-librgw2_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:40.441 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking librgw2 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:40.447 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-10-16T20:01:40.461 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-10-16T20:01:40.505 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-10-16T20:01:40.869 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package librgw2. 2023-10-16T20:01:40.883 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../021-librgw2_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:40.935 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking librgw2 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:40.943 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-rgw. 2023-10-16T20:01:40.957 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../022-python3-rgw_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:41.001 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-rgw (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:41.287 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package liboath0:amd64. 2023-10-16T20:01:41.302 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2023-10-16T20:01:41.337 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-10-16T20:01:41.471 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-rgw. 2023-10-16T20:01:41.484 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../022-python3-rgw_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:41.537 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-rgw (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:41.640 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libradosstriper1. 2023-10-16T20:01:41.654 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../024-libradosstriper1_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:41.698 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libradosstriper1 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:41.907 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package liboath0:amd64. 2023-10-16T20:01:41.921 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2023-10-16T20:01:41.965 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-10-16T20:01:42.126 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-common. 2023-10-16T20:01:42.141 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../025-ceph-common_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:42.201 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-common (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:42.351 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libradosstriper1. 2023-10-16T20:01:42.366 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../024-libradosstriper1_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:42.418 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libradosstriper1 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:42.938 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-common. 2023-10-16T20:01:42.952 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../025-ceph-common_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:43.004 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-common (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:43.306 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-base. 2023-10-16T20:01:43.389 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../026-ceph-base_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:43.389 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-base (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:44.763 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-10-16T20:01:44.778 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-10-16T20:01:44.813 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-10-16T20:01:45.044 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-base. 2023-10-16T20:01:45.058 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../026-ceph-base_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:45.074 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-cheroot. 2023-10-16T20:01:45.089 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-10-16T20:01:45.133 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-10-16T20:01:45.152 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-base (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:45.461 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-10-16T20:01:45.476 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-10-16T20:01:45.519 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-10-16T20:01:45.823 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-10-16T20:01:45.836 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-10-16T20:01:45.839 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-jaraco.text. 2023-10-16T20:01:45.854 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2023-10-16T20:01:45.873 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-10-16T20:01:45.897 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-10-16T20:01:46.183 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-10-16T20:01:46.191 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-cheroot. 2023-10-16T20:01:46.198 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-10-16T20:01:46.205 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-10-16T20:01:46.242 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-10-16T20:01:46.259 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-10-16T20:01:46.536 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-tempora. 2023-10-16T20:01:46.551 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2023-10-16T20:01:46.587 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-10-16T20:01:46.595 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-10-16T20:01:46.600 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-10-16T20:01:46.645 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-10-16T20:01:46.839 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-portend. 2023-10-16T20:01:46.853 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2023-10-16T20:01:46.889 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-10-16T20:01:46.947 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-jaraco.text. 2023-10-16T20:01:46.961 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2023-10-16T20:01:47.006 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-10-16T20:01:47.201 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-10-16T20:01:47.215 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2023-10-16T20:01:47.259 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-10-16T20:01:47.309 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-10-16T20:01:47.322 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-10-16T20:01:47.367 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-10-16T20:01:47.570 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-cherrypy3. 2023-10-16T20:01:47.585 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2023-10-16T20:01:47.629 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-10-16T20:01:47.670 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-tempora. 2023-10-16T20:01:47.684 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2023-10-16T20:01:47.728 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-10-16T20:01:47.966 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-natsort. 2023-10-16T20:01:47.980 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2023-10-16T20:01:48.024 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-10-16T20:01:48.064 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-portend. 2023-10-16T20:01:48.078 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2023-10-16T20:01:48.122 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-10-16T20:01:48.369 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-logutils. 2023-10-16T20:01:48.383 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2023-10-16T20:01:48.400 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-10-16T20:01:48.414 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2023-10-16T20:01:48.427 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-10-16T20:01:48.450 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-10-16T20:01:48.703 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-cherrypy3. 2023-10-16T20:01:48.717 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2023-10-16T20:01:48.730 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-markupsafe. 2023-10-16T20:01:48.745 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-10-16T20:01:48.761 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-10-16T20:01:48.788 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-10-16T20:01:49.091 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-mako. 2023-10-16T20:01:49.106 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-10-16T20:01:49.132 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-natsort. 2023-10-16T20:01:49.145 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2023-10-16T20:01:49.149 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-10-16T20:01:49.190 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-10-16T20:01:49.485 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-logutils. 2023-10-16T20:01:49.499 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2023-10-16T20:01:49.503 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-simplegeneric. 2023-10-16T20:01:49.518 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2023-10-16T20:01:49.534 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-10-16T20:01:49.561 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-10-16T20:01:49.815 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-singledispatch. 2023-10-16T20:01:49.820 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-markupsafe. 2023-10-16T20:01:49.829 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-10-16T20:01:49.835 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-10-16T20:01:49.864 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-10-16T20:01:49.879 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-10-16T20:01:50.159 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-webob. 2023-10-16T20:01:50.173 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-10-16T20:01:50.198 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-mako. 2023-10-16T20:01:50.212 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-10-16T20:01:50.217 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-10-16T20:01:50.256 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-10-16T20:01:50.554 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-waitress. 2023-10-16T20:01:50.559 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-simplegeneric. 2023-10-16T20:01:50.568 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-10-16T20:01:50.573 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2023-10-16T20:01:50.609 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-10-16T20:01:50.615 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-10-16T20:01:50.929 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-singledispatch. 2023-10-16T20:01:50.943 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-10-16T20:01:50.973 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-tempita. 2023-10-16T20:01:50.987 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-10-16T20:01:50.988 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-10-16T20:01:51.032 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-10-16T20:01:51.290 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-webob. 2023-10-16T20:01:51.304 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-10-16T20:01:51.318 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-paste. 2023-10-16T20:01:51.333 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-10-16T20:01:51.348 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-10-16T20:01:51.368 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-10-16T20:01:51.680 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-10-16T20:01:51.684 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-waitress. 2023-10-16T20:01:51.694 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-10-16T20:01:51.698 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-10-16T20:01:51.738 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-10-16T20:01:51.753 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-10-16T20:01:52.024 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pastedeploy. 2023-10-16T20:01:52.037 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-tempita. 2023-10-16T20:01:52.039 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2023-10-16T20:01:52.051 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-10-16T20:01:52.074 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-10-16T20:01:52.087 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-10-16T20:01:52.378 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-webtest. 2023-10-16T20:01:52.390 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-paste. 2023-10-16T20:01:52.392 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2023-10-16T20:01:52.405 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-10-16T20:01:52.435 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-10-16T20:01:52.448 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-10-16T20:01:52.747 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pecan. 2023-10-16T20:01:52.762 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-10-16T20:01:52.784 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-10-16T20:01:52.799 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-10-16T20:01:52.805 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-10-16T20:01:52.851 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-10-16T20:01:53.118 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-certifi. 2023-10-16T20:01:53.133 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2023-10-16T20:01:53.154 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-pastedeploy. 2023-10-16T20:01:53.167 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2023-10-16T20:01:53.176 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-10-16T20:01:53.212 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-10-16T20:01:53.514 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-idna. 2023-10-16T20:01:53.515 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-webtest. 2023-10-16T20:01:53.528 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../051-python3-idna_3.3-1_all.deb ... 2023-10-16T20:01:53.529 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2023-10-16T20:01:53.572 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-idna (3.3-1) ... 2023-10-16T20:01:53.573 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-10-16T20:01:53.883 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-urllib3. 2023-10-16T20:01:53.897 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1_all.deb ... 2023-10-16T20:01:53.918 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-pecan. 2023-10-16T20:01:53.932 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-10-16T20:01:53.941 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1) ... 2023-10-16T20:01:53.976 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-10-16T20:01:54.245 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-requests. 2023-10-16T20:01:54.259 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-10-16T20:01:54.303 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-10-16T20:01:54.339 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-certifi. 2023-10-16T20:01:54.352 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2023-10-16T20:01:54.397 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-10-16T20:01:54.647 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-werkzeug. 2023-10-16T20:01:54.661 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-10-16T20:01:54.706 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-10-16T20:01:54.758 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-idna. 2023-10-16T20:01:54.772 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../051-python3-idna_3.3-1_all.deb ... 2023-10-16T20:01:54.816 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-idna (3.3-1) ... 2023-10-16T20:01:54.992 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-10-16T20:01:55.006 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T20:01:55.051 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mgr-modules-core (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:55.103 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-urllib3. 2023-10-16T20:01:55.117 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1_all.deb ... 2023-10-16T20:01:55.152 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1) ... 2023-10-16T20:01:55.397 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-10-16T20:01:55.411 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:55.422 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-requests. 2023-10-16T20:01:55.436 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-10-16T20:01:55.447 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:55.472 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-10-16T20:01:55.759 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-openssl. 2023-10-16T20:01:55.772 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../057-python3-openssl_21.0.0-1_all.deb ... 2023-10-16T20:01:55.791 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-werkzeug. 2023-10-16T20:01:55.806 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-10-16T20:01:55.817 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-10-16T20:01:55.850 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-10-16T20:01:56.111 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mgr. 2023-10-16T20:01:56.125 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../058-ceph-mgr_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:56.153 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-10-16T20:01:56.161 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mgr (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:56.167 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T20:01:56.211 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-mgr-modules-core (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:56.540 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mon. 2023-10-16T20:01:56.554 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../059-ceph-mon_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:56.557 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-10-16T20:01:56.571 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:56.598 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mon (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:56.615 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:56.944 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-openssl. 2023-10-16T20:01:56.958 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../057-python3-openssl_21.0.0-1_all.deb ... 2023-10-16T20:01:56.993 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-10-16T20:01:57.288 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-mgr. 2023-10-16T20:01:57.302 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../058-ceph-mgr_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:57.346 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-mgr (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:57.435 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-osd. 2023-10-16T20:01:57.450 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../060-ceph-osd_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:57.493 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-osd (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:57.757 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-mon. 2023-10-16T20:01:57.772 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../059-ceph-mon_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:57.816 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-mon (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:58.653 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-osd. 2023-10-16T20:01:58.667 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../060-ceph-osd_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:58.720 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-osd (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:58.722 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph. 2023-10-16T20:01:58.736 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../061-ceph_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:58.772 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:59.258 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-fuse. 2023-10-16T20:01:59.273 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../062-ceph-fuse_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:59.317 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-fuse (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:01:59.687 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mds. 2023-10-16T20:01:59.702 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../063-ceph-mds_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:01:59.745 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mds (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:00.106 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph. 2023-10-16T20:02:00.121 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../061-ceph_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:02:00.163 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:00.173 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package cephadm. 2023-10-16T20:02:00.187 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../064-cephadm_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:02:00.223 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking cephadm (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:00.483 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-fuse. 2023-10-16T20:02:00.497 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../062-ceph-fuse_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:02:00.518 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-asyncssh. 2023-10-16T20:02:00.533 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1_all.deb ... 2023-10-16T20:02:00.541 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-fuse (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:00.576 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-10-16T20:02:00.838 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python-babel-localedata. 2023-10-16T20:02:00.852 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../066-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-10-16T20:02:00.929 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-10-16T20:02:00.961 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-mds. 2023-10-16T20:02:00.976 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../063-ceph-mds_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:02:01.019 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-mds (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:01.364 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package cephadm. 2023-10-16T20:02:01.378 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../064-cephadm_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:02:01.414 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking cephadm (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:01.892 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-asyncssh. 2023-10-16T20:02:01.908 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1_all.deb ... 2023-10-16T20:02:01.926 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-babel. 2023-10-16T20:02:01.940 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../067-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-10-16T20:02:01.950 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-10-16T20:02:01.984 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-10-16T20:02:02.245 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python-babel-localedata. 2023-10-16T20:02:02.260 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../066-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-10-16T20:02:02.303 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-10-16T20:02:02.347 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-jinja2. 2023-10-16T20:02:02.361 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../068-python3-jinja2_3.0.3-1_all.deb ... 2023-10-16T20:02:02.405 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-10-16T20:02:02.750 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-10-16T20:02:02.764 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../069-ceph-mgr-cephadm_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T20:02:02.808 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mgr-cephadm (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:03.137 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-bcrypt. 2023-10-16T20:02:03.152 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../070-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-10-16T20:02:03.195 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-10-16T20:02:03.266 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-babel. 2023-10-16T20:02:03.280 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../067-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-10-16T20:02:03.324 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-10-16T20:02:03.540 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-repoze.lru. 2023-10-16T20:02:03.555 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../071-python3-repoze.lru_0.7-2_all.deb ... 2023-10-16T20:02:03.598 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-10-16T20:02:03.652 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-jinja2. 2023-10-16T20:02:03.666 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../068-python3-jinja2_3.0.3-1_all.deb ... 2023-10-16T20:02:03.711 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-10-16T20:02:03.859 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-routes. 2023-10-16T20:02:03.874 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../072-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-10-16T20:02:03.917 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-10-16T20:02:04.072 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-10-16T20:02:04.085 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../069-ceph-mgr-cephadm_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T20:02:04.130 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-mgr-cephadm (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:04.338 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-10-16T20:02:04.353 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../073-ceph-mgr-dashboard_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T20:02:04.388 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mgr-dashboard (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:04.497 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-bcrypt. 2023-10-16T20:02:04.511 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../070-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-10-16T20:02:04.559 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-10-16T20:02:04.841 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-10-16T20:02:04.853 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-repoze.lru. 2023-10-16T20:02:04.855 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../074-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-10-16T20:02:04.867 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../071-python3-repoze.lru_0.7-2_all.deb ... 2023-10-16T20:02:04.900 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-10-16T20:02:04.911 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-10-16T20:02:05.164 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-routes. 2023-10-16T20:02:05.178 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../072-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-10-16T20:02:05.222 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-10-16T20:02:05.387 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-joblib. 2023-10-16T20:02:05.401 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../075-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-10-16T20:02:05.445 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-10-16T20:02:05.551 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-10-16T20:02:05.564 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../073-ceph-mgr-dashboard_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T20:02:05.601 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-mgr-dashboard (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:05.816 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-10-16T20:02:05.829 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../076-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-10-16T20:02:05.905 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-10-16T20:02:06.021 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-10-16T20:02:06.034 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../074-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-10-16T20:02:06.079 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-10-16T20:02:06.210 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-sklearn. 2023-10-16T20:02:06.224 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../077-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-10-16T20:02:06.268 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-10-16T20:02:06.599 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-joblib. 2023-10-16T20:02:06.613 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../075-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-10-16T20:02:06.657 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-10-16T20:02:06.928 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-10-16T20:02:06.942 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../078-ceph-mgr-diskprediction-local_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T20:02:06.986 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:07.019 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-10-16T20:02:07.032 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../076-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-10-16T20:02:07.077 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-10-16T20:02:07.405 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-sklearn. 2023-10-16T20:02:07.419 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../077-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-10-16T20:02:07.463 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-10-16T20:02:07.931 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-cachetools. 2023-10-16T20:02:07.946 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../079-python3-cachetools_5.0.0-1_all.deb ... 2023-10-16T20:02:07.989 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-10-16T20:02:08.088 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-10-16T20:02:08.102 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../078-ceph-mgr-diskprediction-local_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T20:02:08.138 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:08.251 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pyasn1. 2023-10-16T20:02:08.266 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../080-python3-pyasn1_0.4.8-1_all.deb ... 2023-10-16T20:02:08.309 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-10-16T20:02:08.662 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-10-16T20:02:08.677 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../081-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-10-16T20:02:08.721 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-10-16T20:02:09.048 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-rsa. 2023-10-16T20:02:09.064 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../082-python3-rsa_4.8-1_all.deb ... 2023-10-16T20:02:09.099 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-cachetools. 2023-10-16T20:02:09.106 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-rsa (4.8-1) ... 2023-10-16T20:02:09.115 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../079-python3-cachetools_5.0.0-1_all.deb ... 2023-10-16T20:02:09.158 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-10-16T20:02:09.494 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-pyasn1. 2023-10-16T20:02:09.495 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-google-auth. 2023-10-16T20:02:09.509 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../080-python3-pyasn1_0.4.8-1_all.deb ... 2023-10-16T20:02:09.511 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../083-python3-google-auth_1.5.1-3_all.deb ... 2023-10-16T20:02:09.552 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-10-16T20:02:09.553 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-10-16T20:02:09.882 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-10-16T20:02:09.889 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-10-16T20:02:09.897 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../084-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-10-16T20:02:09.904 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../081-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-10-16T20:02:09.932 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-10-16T20:02:09.947 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-10-16T20:02:10.235 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-websocket. 2023-10-16T20:02:10.250 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../085-python3-websocket_1.2.3-1_all.deb ... 2023-10-16T20:02:10.266 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-rsa. 2023-10-16T20:02:10.281 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../082-python3-rsa_4.8-1_all.deb ... 2023-10-16T20:02:10.293 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-10-16T20:02:10.324 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-rsa (4.8-1) ... 2023-10-16T20:02:10.622 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-kubernetes. 2023-10-16T20:02:10.636 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../086-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-10-16T20:02:10.680 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-10-16T20:02:10.721 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-google-auth. 2023-10-16T20:02:10.736 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../083-python3-google-auth_1.5.1-3_all.deb ... 2023-10-16T20:02:10.781 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-10-16T20:02:11.099 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-10-16T20:02:11.114 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../084-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-10-16T20:02:11.157 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-10-16T20:02:11.311 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-10-16T20:02:11.325 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../087-ceph-mgr-k8sevents_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T20:02:11.369 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:11.494 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-websocket. 2023-10-16T20:02:11.509 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../085-python3-websocket_1.2.3-1_all.deb ... 2023-10-16T20:02:11.552 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-10-16T20:02:11.697 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libonig5:amd64. 2023-10-16T20:02:11.711 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../088-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-10-16T20:02:11.755 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-10-16T20:02:11.838 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-kubernetes. 2023-10-16T20:02:11.853 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../086-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-10-16T20:02:11.888 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-10-16T20:02:12.092 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libjq1:amd64. 2023-10-16T20:02:12.106 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../089-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-10-16T20:02:12.175 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-10-16T20:02:12.428 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package jq. 2023-10-16T20:02:12.442 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../090-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-10-16T20:02:12.478 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-10-16T20:02:12.585 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-10-16T20:02:12.599 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../087-ceph-mgr-k8sevents_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T20:02:12.643 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:12.714 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package socat. 2023-10-16T20:02:12.728 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../091-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-10-16T20:02:12.772 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-10-16T20:02:13.021 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libonig5:amd64. 2023-10-16T20:02:13.037 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../088-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-10-16T20:02:13.068 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package xmlstarlet. 2023-10-16T20:02:13.083 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../092-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-10-16T20:02:13.087 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-10-16T20:02:13.117 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-10-16T20:02:13.455 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-test. 2023-10-16T20:02:13.469 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../093-ceph-test_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:02:13.499 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libjq1:amd64. 2023-10-16T20:02:13.505 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-test (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:13.512 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../089-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-10-16T20:02:13.565 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-10-16T20:02:13.860 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package jq. 2023-10-16T20:02:13.873 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../090-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-10-16T20:02:13.918 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-10-16T20:02:14.229 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package socat. 2023-10-16T20:02:14.243 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../091-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-10-16T20:02:14.296 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-10-16T20:02:14.615 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package xmlstarlet. 2023-10-16T20:02:14.629 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../092-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-10-16T20:02:14.682 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-10-16T20:02:15.070 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-test. 2023-10-16T20:02:15.083 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../093-ceph-test_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:02:15.194 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-test (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:15.651 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-volume. 2023-10-16T20:02:15.666 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../094-ceph-volume_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T20:02:15.701 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-volume (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:16.022 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package cephfs-mirror. 2023-10-16T20:02:16.037 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../095-cephfs-mirror_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:02:16.072 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking cephfs-mirror (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:16.392 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-colorama. 2023-10-16T20:02:16.407 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../096-python3-colorama_0.4.4-1_all.deb ... 2023-10-16T20:02:16.450 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-10-16T20:02:16.711 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package cephfs-shell. 2023-10-16T20:02:16.726 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../097-cephfs-shell_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T20:02:16.769 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking cephfs-shell (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:16.997 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package cephfs-top. 2023-10-16T20:02:17.013 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../098-cephfs-top_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T20:02:17.047 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking cephfs-top (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:17.165 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-volume. 2023-10-16T20:02:17.180 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../094-ceph-volume_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T20:02:17.223 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-volume (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:17.300 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libcephfs-dev. 2023-10-16T20:02:17.315 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../099-libcephfs-dev_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:02:17.358 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libcephfs-dev (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:17.669 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package cephfs-mirror. 2023-10-16T20:02:17.684 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../095-cephfs-mirror_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:02:17.727 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking cephfs-mirror (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:17.762 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package nvme-cli. 2023-10-16T20:02:17.777 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../100-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-10-16T20:02:17.820 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-10-16T20:02:18.080 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-colorama. 2023-10-16T20:02:18.096 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../096-python3-colorama_0.4.4-1_all.deb ... 2023-10-16T20:02:18.147 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-10-16T20:02:18.150 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-10-16T20:02:18.165 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../101-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-10-16T20:02:18.200 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-10-16T20:02:18.483 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package cephfs-shell. 2023-10-16T20:02:18.498 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../097-cephfs-shell_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T20:02:18.549 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking cephfs-shell (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:18.628 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-iniconfig. 2023-10-16T20:02:18.642 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../102-python3-iniconfig_1.1.1-2_all.deb ... 2023-10-16T20:02:18.686 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-10-16T20:02:18.861 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package cephfs-top. 2023-10-16T20:02:18.876 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../098-cephfs-top_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T20:02:18.927 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking cephfs-top (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:18.981 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pastescript. 2023-10-16T20:02:18.996 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../103-python3-pastescript_2.0.2-4_all.deb ... 2023-10-16T20:02:19.039 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-10-16T20:02:19.221 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libcephfs-dev. 2023-10-16T20:02:19.237 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../099-libcephfs-dev_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:02:19.279 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libcephfs-dev (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:19.351 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pluggy. 2023-10-16T20:02:19.366 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../104-python3-pluggy_0.13.0-7.1_all.deb ... 2023-10-16T20:02:19.401 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-10-16T20:02:19.682 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package nvme-cli. 2023-10-16T20:02:19.696 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-psutil. 2023-10-16T20:02:19.698 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../100-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-10-16T20:02:19.710 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../105-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-10-16T20:02:19.749 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-10-16T20:02:19.754 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-10-16T20:02:20.081 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-10-16T20:02:20.096 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../101-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-10-16T20:02:20.116 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-py. 2023-10-16T20:02:20.130 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../106-python3-py_1.10.0-1_all.deb ... 2023-10-16T20:02:20.174 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-py (1.10.0-1) ... 2023-10-16T20:02:20.189 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-10-16T20:02:20.494 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pygments. 2023-10-16T20:02:20.508 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../107-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-10-16T20:02:20.552 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-10-16T20:02:20.617 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-iniconfig. 2023-10-16T20:02:20.631 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../102-python3-iniconfig_1.1.1-2_all.deb ... 2023-10-16T20:02:20.675 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-10-16T20:02:20.961 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-pastescript. 2023-10-16T20:02:20.976 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../103-python3-pastescript_2.0.2-4_all.deb ... 2023-10-16T20:02:21.009 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pyinotify. 2023-10-16T20:02:21.011 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-10-16T20:02:21.023 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../108-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-10-16T20:02:21.067 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-10-16T20:02:21.298 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-pluggy. 2023-10-16T20:02:21.312 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../104-python3-pluggy_0.13.0-7.1_all.deb ... 2023-10-16T20:02:21.348 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-10-16T20:02:21.369 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-toml. 2023-10-16T20:02:21.384 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../109-python3-toml_0.10.2-1_all.deb ... 2023-10-16T20:02:21.419 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-10-16T20:02:21.659 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-psutil. 2023-10-16T20:02:21.674 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../105-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-10-16T20:02:21.697 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pytest. 2023-10-16T20:02:21.711 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../110-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-10-16T20:02:21.717 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-10-16T20:02:21.755 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-10-16T20:02:22.059 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-py. 2023-10-16T20:02:22.075 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../106-python3-py_1.10.0-1_all.deb ... 2023-10-16T20:02:22.084 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-simplejson. 2023-10-16T20:02:22.098 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-10-16T20:02:22.120 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-py (1.10.0-1) ... 2023-10-16T20:02:22.142 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-10-16T20:02:22.395 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-10-16T20:02:22.409 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-10-16T20:02:22.440 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-pygments. 2023-10-16T20:02:22.453 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-10-16T20:02:22.455 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../107-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-10-16T20:02:22.498 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-10-16T20:02:22.935 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-pyinotify. 2023-10-16T20:02:22.949 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../108-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-10-16T20:02:22.985 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-10-16T20:02:23.258 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package radosgw. 2023-10-16T20:02:23.272 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../113-radosgw_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:02:23.296 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-toml. 2023-10-16T20:02:23.310 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../109-python3-toml_0.10.2-1_all.deb ... 2023-10-16T20:02:23.316 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking radosgw (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:23.354 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-10-16T20:02:23.640 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-pytest. 2023-10-16T20:02:23.655 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../110-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-10-16T20:02:23.698 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-10-16T20:02:23.944 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package rbd-fuse. 2023-10-16T20:02:23.959 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../114-rbd-fuse_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:02:24.002 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking rbd-fuse (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:24.043 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-simplejson. 2023-10-16T20:02:24.058 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-10-16T20:02:24.101 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-10-16T20:02:24.278 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-10-16T20:02:24.363 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-10-16T20:02:24.377 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-10-16T20:02:24.421 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-10-16T20:02:24.541 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-10-16T20:02:24.667 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-10-16T20:02:24.930 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-10-16T20:02:25.200 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package radosgw. 2023-10-16T20:02:25.213 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../113-radosgw_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:02:25.257 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking radosgw (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:25.324 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-10-16T20:02:25.869 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package rbd-fuse. 2023-10-16T20:02:25.884 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../114-rbd-fuse_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T20:02:25.913 INFO:teuthology.orchestra.run.smithi094.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-10-16T20:02:25.935 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking rbd-fuse (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:25.968 INFO:teuthology.orchestra.run.smithi094.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2023-10-16T20:02:26.010 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up cephadm (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:26.220 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-10-16T20:02:26.490 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-10-16T20:02:26.547 INFO:teuthology.orchestra.run.smithi094.stdout:Adding system user cephadm....done 2023-10-16T20:02:26.642 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-10-16T20:02:26.708 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-10-16T20:02:26.909 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-10-16T20:02:27.010 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-10-16T20:02:27.253 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-10-16T20:02:27.311 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-10-16T20:02:27.403 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-10-16T20:02:27.697 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-10-16T20:02:27.876 INFO:teuthology.orchestra.run.smithi088.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-10-16T20:02:27.906 INFO:teuthology.orchestra.run.smithi088.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2023-10-16T20:02:27.939 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-10-16T20:02:27.960 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up cephadm (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:28.074 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-py (1.10.0-1) ... 2023-10-16T20:02:28.394 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-10-16T20:02:28.522 INFO:teuthology.orchestra.run.smithi088.stdout:Adding system user cephadm....done 2023-10-16T20:02:28.682 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-10-16T20:02:28.787 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-10-16T20:02:29.009 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-10-16T20:02:29.054 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-10-16T20:02:29.269 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-10-16T20:02:29.306 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-10-16T20:02:29.419 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-10-16T20:02:29.591 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-10-16T20:02:29.695 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-10-16T20:02:29.850 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-10-16T20:02:29.997 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-10-16T20:02:30.126 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-10-16T20:02:30.148 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-py (1.10.0-1) ... 2023-10-16T20:02:30.394 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-ceph-argparse (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:30.661 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-10-16T20:02:30.766 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-10-16T20:02:30.796 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-10-16T20:02:31.047 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-10-16T20:02:31.168 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-10-16T20:02:31.290 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-10-16T20:02:31.416 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-10-16T20:02:31.444 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-10-16T20:02:31.729 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-10-16T20:02:31.784 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-10-16T20:02:31.910 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-10-16T20:02:32.022 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-10-16T20:02:32.186 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-10-16T20:02:32.298 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-10-16T20:02:32.462 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-10-16T20:02:32.582 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-10-16T20:02:32.797 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-10-16T20:02:32.891 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-ceph-argparse (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:33.167 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-10-16T20:02:33.285 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-10-16T20:02:33.475 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-10-16T20:02:33.544 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-10-16T20:02:33.776 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-10-16T20:02:33.829 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-10-16T20:02:33.901 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-10-16T20:02:33.955 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-10-16T20:02:34.027 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-10-16T20:02:34.322 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-10-16T20:02:34.354 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-10-16T20:02:34.423 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-10-16T20:02:34.691 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-10-16T20:02:34.755 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-10-16T20:02:34.958 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-10-16T20:02:35.007 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-10-16T20:02:35.285 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-10-16T20:02:35.391 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-10-16T20:02:35.693 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-10-16T20:02:35.962 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-10-16T20:02:36.010 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-10-16T20:02:36.238 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-10-16T20:02:36.339 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-10-16T20:02:36.370 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-10-16T20:02:36.440 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-10-16T20:02:36.629 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-10-16T20:02:36.730 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-ceph-common (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:36.774 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-10-16T20:02:37.024 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-idna (3.3-1) ... 2023-10-16T20:02:37.159 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-10-16T20:02:37.377 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-10-16T20:02:37.470 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-10-16T20:02:37.870 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-10-16T20:02:38.171 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-10-16T20:02:38.472 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-10-16T20:02:38.606 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T20:02:38.724 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-toml (0.10.2-1) ... 2023-10-16T20:02:38.831 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-10-16T20:02:39.008 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-10-16T20:02:39.092 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-10-16T20:02:39.134 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-10-16T20:02:39.217 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-ceph-common (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:39.260 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-urllib3 (1.26.5-1~exp1) ... 2023-10-16T20:02:39.526 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-idna (3.3-1) ... 2023-10-16T20:02:39.562 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-10-16T20:02:39.822 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-10-16T20:02:39.878 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-10-16T20:02:40.064 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T20:02:40.173 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2023-10-16T20:02:40.300 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-10-16T20:02:40.609 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-10-16T20:02:40.936 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-10-16T20:02:41.100 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T20:02:41.203 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-10-16T20:02:41.242 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-toml (0.10.2-1) ... 2023-10-16T20:02:41.497 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-10-16T20:02:41.535 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-10-16T20:02:41.661 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-10-16T20:02:41.731 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-10-16T20:02:41.787 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-urllib3 (1.26.5-1~exp1) ... 2023-10-16T20:02:41.990 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-10-16T20:02:42.088 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-10-16T20:02:42.126 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-10-16T20:02:42.331 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-10-16T20:02:42.409 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-10-16T20:02:42.552 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-10-16T20:02:42.590 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T20:02:42.691 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2023-10-16T20:02:42.792 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-10-16T20:02:42.854 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-10-16T20:02:43.094 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-10-16T20:02:43.163 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-10-16T20:02:43.361 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-10-16T20:02:43.406 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-10-16T20:02:43.629 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-10-16T20:02:43.850 INFO:teuthology.orchestra.run.smithi094.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-10-16T20:02:43.880 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-10-16T20:02:44.118 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-10-16T20:02:44.139 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-10-16T20:02:44.243 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-10-16T20:02:44.449 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-10-16T20:02:44.549 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-10-16T20:02:44.645 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-10-16T20:02:44.826 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-10-16T20:02:44.913 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-portend (3.0.0-1) ... 2023-10-16T20:02:44.952 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-10-16T20:02:45.181 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T20:02:45.278 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-10-16T20:02:45.307 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-10-16T20:02:45.613 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-10-16T20:02:45.633 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-10-16T20:02:45.863 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-10-16T20:02:45.943 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-10-16T20:02:46.244 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-10-16T20:02:46.252 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-10-16T20:02:46.361 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-rsa (4.8-1) ... 2023-10-16T20:02:46.516 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-10-16T20:02:46.638 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-10-16T20:02:46.642 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-10-16T20:02:46.914 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-10-16T20:02:47.085 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-10-16T20:02:47.173 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-10-16T20:02:47.353 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-portend (3.0.0-1) ... 2023-10-16T20:02:47.567 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-10-16T20:02:47.612 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T20:02:47.713 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-10-16T20:02:47.859 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-10-16T20:02:48.056 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-10-16T20:02:48.212 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-10-16T20:02:48.330 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up librados2 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:48.349 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-10-16T20:02:48.439 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up librgw2 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:48.625 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-10-16T20:02:48.642 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libsqlite3-mod-ceph (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:48.751 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-rsa (4.8-1) ... 2023-10-16T20:02:48.794 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libcephfs2 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:48.920 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libradosstriper1 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:49.027 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-10-16T20:02:49.046 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-10-16T20:02:49.312 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-10-16T20:02:49.322 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up librbd1 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:49.423 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mgr-modules-core (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:49.558 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-fuse (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:49.562 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-10-16T20:02:49.776 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-16T20:02:49.776 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-16T20:02:49.964 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-10-16T20:02:50.265 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-10-16T20:02:50.402 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libcephfs-dev (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:50.528 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-rados (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:50.600 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-10-16T20:02:50.645 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up cephfs-top (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:50.726 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up librados2 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:50.746 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-10-16T20:02:50.852 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up librgw2 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:51.106 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libsqlite3-mod-ceph (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:51.215 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libcephfs2 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:51.316 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libradosstriper1 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:51.425 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-10-16T20:02:51.727 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up librbd1 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:51.878 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-mgr-modules-core (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:52.004 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-fuse (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:52.010 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-rbd (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:52.119 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up rbd-fuse (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:52.214 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-16T20:02:52.214 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-16T20:02:52.245 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-rgw (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:52.371 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-cephfs (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:52.522 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up cephfs-shell (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:52.649 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-common (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:52.839 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libcephfs-dev (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:52.964 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-rados (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:53.090 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up cephfs-top (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:53.178 INFO:teuthology.orchestra.run.smithi094.stdout:Adding group ceph....done 2023-10-16T20:02:53.225 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-10-16T20:02:53.537 INFO:teuthology.orchestra.run.smithi094.stdout:Adding system user ceph....done 2023-10-16T20:02:53.745 INFO:teuthology.orchestra.run.smithi094.stdout:Setting system user ceph properties....done 2023-10-16T20:02:53.761 INFO:teuthology.orchestra.run.smithi094.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-10-16T20:02:53.902 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-10-16T20:02:54.300 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-10-16T20:02:54.504 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-rbd (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:54.631 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up rbd-fuse (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:54.781 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-rgw (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:54.882 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-cephfs (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:54.942 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up cephfs-mirror (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:54.992 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up cephfs-shell (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:55.118 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-common (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:55.176 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-16T20:02:55.176 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-16T20:02:55.663 INFO:teuthology.orchestra.run.smithi088.stdout:Adding group ceph....done 2023-10-16T20:02:55.769 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-test (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:55.886 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up radosgw (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:56.014 INFO:teuthology.orchestra.run.smithi088.stdout:Adding system user ceph....done 2023-10-16T20:02:56.208 INFO:teuthology.orchestra.run.smithi088.stdout:Setting system user ceph properties....done 2023-10-16T20:02:56.223 INFO:teuthology.orchestra.run.smithi088.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-10-16T20:02:56.363 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-10-16T20:02:56.374 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-16T20:02:56.374 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-16T20:02:56.740 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-10-16T20:02:56.955 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-base (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:57.317 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-10-16T20:02:57.402 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up cephfs-mirror (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:57.593 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-16T20:02:57.594 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-16T20:02:58.058 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mds (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:58.195 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-test (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:58.296 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up radosgw (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:58.656 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-16T20:02:58.656 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-16T20:02:58.842 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-16T20:02:58.843 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-16T20:02:59.269 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mgr (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:59.481 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-base (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:02:59.481 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-16T20:02:59.482 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-16T20:02:59.869 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-10-16T20:03:00.095 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-osd (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:03:00.418 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-16T20:03:00.418 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-16T20:03:00.550 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-mds (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:03:00.780 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-16T20:03:00.780 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-16T20:03:01.005 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mgr-k8sevents (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:03:01.131 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:03:01.477 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-mgr (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:03:01.706 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-16T20:03:01.706 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-16T20:03:01.763 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mon (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:03:01.975 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-16T20:03:01.975 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-16T20:03:02.328 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-osd (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:03:02.581 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mgr-cephadm (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:03:02.723 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:03:02.753 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-16T20:03:02.753 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-16T20:03:02.858 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mgr-dashboard (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:03:02.993 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-volume (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:03:03.337 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-10-16T20:03:03.396 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-mgr-k8sevents (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:03:03.522 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:03:03.673 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-mon (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:03:03.882 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-16T20:03:03.882 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-16T20:03:04.447 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-10-16T20:03:04.516 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-mgr-cephadm (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:03:04.658 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:03:04.784 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-mgr-dashboard (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:03:04.921 INFO:teuthology.orchestra.run.smithi094.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-16T20:03:04.926 DEBUG:teuthology.orchestra.run.smithi094:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-10-16T20:03:04.927 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-volume (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T20:03:05.001 INFO:teuthology.orchestra.run.smithi094.stdout:Reading package lists... 2023-10-16T20:03:05.163 INFO:teuthology.orchestra.run.smithi094.stdout:Building dependency tree... 2023-10-16T20:03:05.164 INFO:teuthology.orchestra.run.smithi094.stdout:Reading state information... 2023-10-16T20:03:05.312 INFO:teuthology.orchestra.run.smithi094.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-10-16T20:03:05.312 INFO:teuthology.orchestra.run.smithi094.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-10-16T20:03:05.313 INFO:teuthology.orchestra.run.smithi094.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-10-16T20:03:05.313 INFO:teuthology.orchestra.run.smithi094.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-10-16T20:03:05.313 INFO:teuthology.orchestra.run.smithi094.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-10-16T20:03:05.313 INFO:teuthology.orchestra.run.smithi094.stdout:The following packages were automatically installed and are no longer required: 2023-10-16T20:03:05.313 INFO:teuthology.orchestra.run.smithi094.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-16T20:03:05.314 INFO:teuthology.orchestra.run.smithi094.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-16T20:03:05.314 INFO:teuthology.orchestra.run.smithi094.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-16T20:03:05.314 INFO:teuthology.orchestra.run.smithi094.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-10-16T20:03:05.314 INFO:teuthology.orchestra.run.smithi094.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-10-16T20:03:05.314 INFO:teuthology.orchestra.run.smithi094.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-16T20:03:05.315 INFO:teuthology.orchestra.run.smithi094.stdout:The following additional packages will be installed: 2023-10-16T20:03:05.316 INFO:teuthology.orchestra.run.smithi094.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-10-16T20:03:05.316 INFO:teuthology.orchestra.run.smithi094.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-10-16T20:03:05.316 INFO:teuthology.orchestra.run.smithi094.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-10-16T20:03:05.316 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-10-16T20:03:05.316 INFO:teuthology.orchestra.run.smithi094.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-16T20:03:05.316 INFO:teuthology.orchestra.run.smithi094.stdout:Suggested packages: 2023-10-16T20:03:05.317 INFO:teuthology.orchestra.run.smithi094.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-10-16T20:03:05.317 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-10-16T20:03:05.317 INFO:teuthology.orchestra.run.smithi094.stdout: libengine-pkcs11-openssl 2023-10-16T20:03:05.321 INFO:teuthology.orchestra.run.smithi088.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-10-16T20:03:05.363 INFO:teuthology.orchestra.run.smithi094.stdout:The following NEW packages will be installed: 2023-10-16T20:03:05.364 INFO:teuthology.orchestra.run.smithi094.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-10-16T20:03:05.364 INFO:teuthology.orchestra.run.smithi094.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-10-16T20:03:05.365 INFO:teuthology.orchestra.run.smithi094.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-10-16T20:03:05.365 INFO:teuthology.orchestra.run.smithi094.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-10-16T20:03:05.365 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-10-16T20:03:05.366 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-10-16T20:03:05.366 INFO:teuthology.orchestra.run.smithi094.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-16T20:03:05.366 INFO:teuthology.orchestra.run.smithi094.stdout:The following packages will be upgraded: 2023-10-16T20:03:05.367 INFO:teuthology.orchestra.run.smithi094.stdout: libssl3 2023-10-16T20:03:05.421 INFO:teuthology.orchestra.run.smithi094.stdout:1 upgraded, 39 newly installed, 0 to remove and 258 not upgraded. 2023-10-16T20:03:05.422 INFO:teuthology.orchestra.run.smithi094.stdout:Need to get 54.6 MB of archives. 2023-10-16T20:03:05.422 INFO:teuthology.orchestra.run.smithi094.stdout:After this operation, 202 MB of additional disk space will be used. 2023-10-16T20:03:05.422 INFO:teuthology.orchestra.run.smithi094.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-10-16T20:03:05.523 INFO:teuthology.orchestra.run.smithi094.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.10 [1,901 kB] 2023-10-16T20:03:05.577 INFO:teuthology.orchestra.run.smithi094.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-10-16T20:03:05.584 INFO:teuthology.orchestra.run.smithi094.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-10-16T20:03:05.585 INFO:teuthology.orchestra.run.smithi094.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-10-16T20:03:05.587 INFO:teuthology.orchestra.run.smithi094.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2023-10-16T20:03:05.588 INFO:teuthology.orchestra.run.smithi094.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-10-16T20:03:05.588 INFO:teuthology.orchestra.run.smithi094.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libbluetooth3 amd64 5.64-0ubuntu1 [86.9 kB] 2023-10-16T20:03:05.589 INFO:teuthology.orchestra.run.smithi094.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-10-16T20:03:05.590 INFO:teuthology.orchestra.run.smithi094.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-10-16T20:03:05.591 INFO:teuthology.orchestra.run.smithi094.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-10-16T20:03:05.596 INFO:teuthology.orchestra.run.smithi094.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-10-16T20:03:05.608 INFO:teuthology.orchestra.run.smithi094.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-10-16T20:03:05.817 INFO:teuthology.orchestra.run.smithi094.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-10-16T20:03:05.819 INFO:teuthology.orchestra.run.smithi094.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-10-16T20:03:05.819 INFO:teuthology.orchestra.run.smithi094.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-10-16T20:03:05.822 INFO:teuthology.orchestra.run.smithi094.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-10-16T20:03:05.823 INFO:teuthology.orchestra.run.smithi094.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-10-16T20:03:05.827 INFO:teuthology.orchestra.run.smithi094.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.9-0ubuntu0.22.04.1 [141 kB] 2023-10-16T20:03:05.829 INFO:teuthology.orchestra.run.smithi094.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-10-16T20:03:05.836 INFO:teuthology.orchestra.run.smithi094.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-10-16T20:03:05.836 INFO:teuthology.orchestra.run.smithi094.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.10 [2,372 kB] 2023-10-16T20:03:05.857 INFO:teuthology.orchestra.run.smithi094.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-10-16T20:03:05.858 INFO:teuthology.orchestra.run.smithi094.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-10-16T20:03:05.867 INFO:teuthology.orchestra.run.smithi094.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-10-16T20:03:05.887 INFO:teuthology.orchestra.run.smithi094.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-10-16T20:03:05.890 INFO:teuthology.orchestra.run.smithi094.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-10-16T20:03:05.891 INFO:teuthology.orchestra.run.smithi094.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-10-16T20:03:05.892 INFO:teuthology.orchestra.run.smithi094.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-10-16T20:03:05.892 INFO:teuthology.orchestra.run.smithi094.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.9-0ubuntu0.22.04.1 [1,222 kB] 2023-10-16T20:03:05.904 INFO:teuthology.orchestra.run.smithi094.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-10-16T20:03:05.905 INFO:teuthology.orchestra.run.smithi094.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.9-0ubuntu0.22.04.1 [16.1 MB] 2023-10-16T20:03:06.047 INFO:teuthology.orchestra.run.smithi094.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-10-16T20:03:06.047 INFO:teuthology.orchestra.run.smithi094.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-10-16T20:03:06.047 INFO:teuthology.orchestra.run.smithi094.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-10-16T20:03:06.048 INFO:teuthology.orchestra.run.smithi094.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-10-16T20:03:06.048 INFO:teuthology.orchestra.run.smithi094.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-10-16T20:03:06.048 INFO:teuthology.orchestra.run.smithi094.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-10-16T20:03:06.061 INFO:teuthology.orchestra.run.smithi094.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-10-16T20:03:06.061 INFO:teuthology.orchestra.run.smithi094.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-10-16T20:03:06.443 INFO:teuthology.orchestra.run.smithi094.stdout:Preconfiguring packages ... 2023-10-16T20:03:06.498 INFO:teuthology.orchestra.run.smithi088.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-10-16T20:03:06.609 INFO:teuthology.orchestra.run.smithi094.stdout:Fetched 54.6 MB in 1s (79.4 MB/s) 2023-10-16T20:03:06.906 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package flex. 2023-10-16T20:03:06.967 INFO:teuthology.orchestra.run.smithi088.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-16T20:03:06.972 DEBUG:teuthology.orchestra.run.smithi088:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-10-16T20:03:06.976 INFO:teuthology.orchestra.run.smithi094.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139851 files and directories currently installed.) 2023-10-16T20:03:06.980 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-10-16T20:03:07.021 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking flex (2.6.4-8build2) ... 2023-10-16T20:03:07.047 INFO:teuthology.orchestra.run.smithi088.stdout:Reading package lists... 2023-10-16T20:03:07.212 INFO:teuthology.orchestra.run.smithi088.stdout:Building dependency tree... 2023-10-16T20:03:07.212 INFO:teuthology.orchestra.run.smithi088.stdout:Reading state information... 2023-10-16T20:03:07.360 INFO:teuthology.orchestra.run.smithi088.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-10-16T20:03:07.360 INFO:teuthology.orchestra.run.smithi088.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-10-16T20:03:07.361 INFO:teuthology.orchestra.run.smithi088.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-10-16T20:03:07.361 INFO:teuthology.orchestra.run.smithi088.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-10-16T20:03:07.361 INFO:teuthology.orchestra.run.smithi088.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-10-16T20:03:07.361 INFO:teuthology.orchestra.run.smithi088.stdout:The following packages were automatically installed and are no longer required: 2023-10-16T20:03:07.361 INFO:teuthology.orchestra.run.smithi088.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-16T20:03:07.362 INFO:teuthology.orchestra.run.smithi088.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-16T20:03:07.362 INFO:teuthology.orchestra.run.smithi088.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-16T20:03:07.362 INFO:teuthology.orchestra.run.smithi088.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-10-16T20:03:07.362 INFO:teuthology.orchestra.run.smithi088.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-10-16T20:03:07.362 INFO:teuthology.orchestra.run.smithi088.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-16T20:03:07.363 INFO:teuthology.orchestra.run.smithi088.stdout:The following additional packages will be installed: 2023-10-16T20:03:07.364 INFO:teuthology.orchestra.run.smithi088.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-10-16T20:03:07.364 INFO:teuthology.orchestra.run.smithi088.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-10-16T20:03:07.364 INFO:teuthology.orchestra.run.smithi088.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-10-16T20:03:07.364 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-10-16T20:03:07.364 INFO:teuthology.orchestra.run.smithi088.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-16T20:03:07.365 INFO:teuthology.orchestra.run.smithi088.stdout:Suggested packages: 2023-10-16T20:03:07.365 INFO:teuthology.orchestra.run.smithi088.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-10-16T20:03:07.365 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-10-16T20:03:07.365 INFO:teuthology.orchestra.run.smithi088.stdout: libengine-pkcs11-openssl 2023-10-16T20:03:07.411 INFO:teuthology.orchestra.run.smithi088.stdout:The following NEW packages will be installed: 2023-10-16T20:03:07.412 INFO:teuthology.orchestra.run.smithi088.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-10-16T20:03:07.412 INFO:teuthology.orchestra.run.smithi088.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-10-16T20:03:07.413 INFO:teuthology.orchestra.run.smithi088.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-10-16T20:03:07.413 INFO:teuthology.orchestra.run.smithi088.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-10-16T20:03:07.413 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-10-16T20:03:07.413 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-10-16T20:03:07.414 INFO:teuthology.orchestra.run.smithi088.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-16T20:03:07.414 INFO:teuthology.orchestra.run.smithi088.stdout:The following packages will be upgraded: 2023-10-16T20:03:07.414 INFO:teuthology.orchestra.run.smithi088.stdout: libssl3 2023-10-16T20:03:07.549 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.10_amd64.deb ... 2023-10-16T20:03:07.597 INFO:teuthology.orchestra.run.smithi088.stdout:1 upgraded, 39 newly installed, 0 to remove and 258 not upgraded. 2023-10-16T20:03:07.597 INFO:teuthology.orchestra.run.smithi088.stdout:Need to get 54.6 MB of archives. 2023-10-16T20:03:07.597 INFO:teuthology.orchestra.run.smithi088.stdout:After this operation, 202 MB of additional disk space will be used. 2023-10-16T20:03:07.598 INFO:teuthology.orchestra.run.smithi088.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-10-16T20:03:07.719 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.10) over (3.0.2-0ubuntu1.7) ... 2023-10-16T20:03:08.094 INFO:teuthology.orchestra.run.smithi088.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.10 [1,901 kB] 2023-10-16T20:03:08.109 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.10) ... 2023-10-16T20:03:08.352 INFO:teuthology.orchestra.run.smithi088.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-10-16T20:03:08.386 INFO:teuthology.orchestra.run.smithi088.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-10-16T20:03:08.388 INFO:teuthology.orchestra.run.smithi088.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-10-16T20:03:08.393 INFO:teuthology.orchestra.run.smithi088.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2023-10-16T20:03:08.396 INFO:teuthology.orchestra.run.smithi088.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-10-16T20:03:08.397 INFO:teuthology.orchestra.run.smithi088.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libbluetooth3 amd64 5.64-0ubuntu1 [86.9 kB] 2023-10-16T20:03:08.401 INFO:teuthology.orchestra.run.smithi088.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-10-16T20:03:08.403 INFO:teuthology.orchestra.run.smithi088.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-10-16T20:03:08.567 INFO:teuthology.orchestra.run.smithi088.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-10-16T20:03:08.568 INFO:teuthology.orchestra.run.smithi088.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-10-16T20:03:08.628 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package bison. 2023-10-16T20:03:08.700 INFO:teuthology.orchestra.run.smithi094.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139924 files and directories currently installed.) 2023-10-16T20:03:08.703 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-10-16T20:03:08.756 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-10-16T20:03:08.813 INFO:teuthology.orchestra.run.smithi088.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-10-16T20:03:09.124 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-10-16T20:03:09.138 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-10-16T20:03:09.182 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-10-16T20:03:09.494 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package dump. 2023-10-16T20:03:09.508 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-10-16T20:03:09.552 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking dump (0.4b47-3) ... 2023-10-16T20:03:09.814 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package indent. 2023-10-16T20:03:09.826 INFO:teuthology.orchestra.run.smithi088.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-10-16T20:03:09.828 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2023-10-16T20:03:09.828 INFO:teuthology.orchestra.run.smithi088.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-10-16T20:03:09.829 INFO:teuthology.orchestra.run.smithi088.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-10-16T20:03:09.831 INFO:teuthology.orchestra.run.smithi088.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-10-16T20:03:09.832 INFO:teuthology.orchestra.run.smithi088.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-10-16T20:03:09.860 INFO:teuthology.orchestra.run.smithi088.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.9-0ubuntu0.22.04.1 [141 kB] 2023-10-16T20:03:09.861 INFO:teuthology.orchestra.run.smithi088.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-10-16T20:03:09.868 INFO:teuthology.orchestra.run.smithi088.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-10-16T20:03:09.872 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2023-10-16T20:03:10.032 INFO:teuthology.orchestra.run.smithi088.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.10 [2,372 kB] 2023-10-16T20:03:10.141 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-10-16T20:03:10.155 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-10-16T20:03:10.191 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-10-16T20:03:10.520 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-10-16T20:03:10.535 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1_amd64.deb ... 2023-10-16T20:03:10.578 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-10-16T20:03:10.781 INFO:teuthology.orchestra.run.smithi088.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-10-16T20:03:10.781 INFO:teuthology.orchestra.run.smithi088.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-10-16T20:03:10.808 INFO:teuthology.orchestra.run.smithi088.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-10-16T20:03:10.840 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-10-16T20:03:10.854 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-10-16T20:03:10.885 INFO:teuthology.orchestra.run.smithi088.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-10-16T20:03:10.894 INFO:teuthology.orchestra.run.smithi088.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-10-16T20:03:10.894 INFO:teuthology.orchestra.run.smithi088.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-10-16T20:03:10.895 INFO:teuthology.orchestra.run.smithi088.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-10-16T20:03:10.896 INFO:teuthology.orchestra.run.smithi088.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.9-0ubuntu0.22.04.1 [1,222 kB] 2023-10-16T20:03:10.898 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-10-16T20:03:10.924 INFO:teuthology.orchestra.run.smithi088.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-10-16T20:03:10.927 INFO:teuthology.orchestra.run.smithi088.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.9-0ubuntu0.22.04.1 [16.1 MB] 2023-10-16T20:03:11.301 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libfl2:amd64. 2023-10-16T20:03:11.317 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-10-16T20:03:11.351 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-10-16T20:03:11.557 INFO:teuthology.orchestra.run.smithi088.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-10-16T20:03:11.595 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-10-16T20:03:11.611 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-10-16T20:03:11.639 INFO:teuthology.orchestra.run.smithi088.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-10-16T20:03:11.640 INFO:teuthology.orchestra.run.smithi088.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-10-16T20:03:11.640 INFO:teuthology.orchestra.run.smithi088.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-10-16T20:03:11.654 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-10-16T20:03:11.729 INFO:teuthology.orchestra.run.smithi088.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-10-16T20:03:11.735 INFO:teuthology.orchestra.run.smithi088.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-10-16T20:03:12.199 INFO:teuthology.orchestra.run.smithi088.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-10-16T20:03:12.200 INFO:teuthology.orchestra.run.smithi088.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-10-16T20:03:12.232 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libjson-perl. 2023-10-16T20:03:12.247 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-10-16T20:03:12.290 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-10-16T20:03:12.589 INFO:teuthology.orchestra.run.smithi088.stdout:Preconfiguring packages ... 2023-10-16T20:03:12.627 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libllvm14:amd64. 2023-10-16T20:03:12.642 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-10-16T20:03:12.685 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-10-16T20:03:12.730 INFO:teuthology.orchestra.run.smithi088.stdout:Fetched 54.6 MB in 5s (11.4 MB/s) 2023-10-16T20:03:12.872 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package flex. 2023-10-16T20:03:12.944 INFO:teuthology.orchestra.run.smithi088.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139851 files and directories currently installed.) 2023-10-16T20:03:12.947 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-10-16T20:03:13.021 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking flex (2.6.4-8build2) ... 2023-10-16T20:03:13.599 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.10_amd64.deb ... 2023-10-16T20:03:13.777 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.10) over (3.0.2-0ubuntu1.7) ... 2023-10-16T20:03:14.096 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-10-16T20:03:14.112 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-10-16T20:03:14.154 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-10-16T20:03:14.208 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.10) ... 2023-10-16T20:03:14.415 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libmbim-proxy. 2023-10-16T20:03:14.431 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-10-16T20:03:14.474 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-10-16T20:03:14.718 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package bison. 2023-10-16T20:03:14.778 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-10-16T20:03:14.792 INFO:teuthology.orchestra.run.smithi088.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139924 files and directories currently installed.) 2023-10-16T20:03:14.794 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-10-16T20:03:14.796 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-10-16T20:03:14.836 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-10-16T20:03:14.846 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-10-16T20:03:15.181 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libndp0:amd64. 2023-10-16T20:03:15.196 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-10-16T20:03:15.239 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-10-16T20:03:15.280 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-10-16T20:03:15.294 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-10-16T20:03:15.338 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-10-16T20:03:15.566 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libnm0:amd64. 2023-10-16T20:03:15.582 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-10-16T20:03:15.625 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-10-16T20:03:15.641 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package dump. 2023-10-16T20:03:15.655 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-10-16T20:03:15.691 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking dump (0.4b47-3) ... 2023-10-16T20:03:15.969 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libpq5:amd64. 2023-10-16T20:03:15.985 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../16-libpq5_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-16T20:03:16.003 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package indent. 2023-10-16T20:03:16.017 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2023-10-16T20:03:16.027 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T20:03:16.061 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2023-10-16T20:03:16.322 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-10-16T20:03:16.336 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-10-16T20:03:16.372 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-10-16T20:03:16.388 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-10-16T20:03:16.388 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-10-16T20:03:16.421 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-10-16T20:03:16.709 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-10-16T20:03:16.715 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libqmi-proxy. 2023-10-16T20:03:16.724 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1_amd64.deb ... 2023-10-16T20:03:16.731 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-10-16T20:03:16.759 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-10-16T20:03:16.774 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-10-16T20:03:17.009 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-10-16T20:03:17.020 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-10-16T20:03:17.025 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.10_amd64.deb ... 2023-10-16T20:03:17.035 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-10-16T20:03:17.078 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-10-16T20:03:17.085 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.10) ... 2023-10-16T20:03:17.425 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libfl2:amd64. 2023-10-16T20:03:17.441 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-10-16T20:03:17.483 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-10-16T20:03:17.613 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-10-16T20:03:17.628 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-10-16T20:03:17.671 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-10-16T20:03:17.736 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-10-16T20:03:17.751 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-10-16T20:03:17.803 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-10-16T20:03:17.982 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package modemmanager. 2023-10-16T20:03:17.998 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-10-16T20:03:18.040 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-10-16T20:03:18.047 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libjson-perl. 2023-10-16T20:03:18.062 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-10-16T20:03:18.105 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-10-16T20:03:18.425 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libllvm14:amd64. 2023-10-16T20:03:18.440 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-10-16T20:03:18.483 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-10-16T20:03:18.638 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package network-manager. 2023-10-16T20:03:18.653 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-10-16T20:03:18.696 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-10-16T20:03:19.285 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ppp. 2023-10-16T20:03:19.300 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-10-16T20:03:19.636 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-10-16T20:03:19.836 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-10-16T20:03:19.851 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-10-16T20:03:19.886 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-10-16T20:03:19.957 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package pptp-linux. 2023-10-16T20:03:19.971 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-10-16T20:03:20.032 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-10-16T20:03:20.155 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libmbim-proxy. 2023-10-16T20:03:20.171 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-10-16T20:03:20.213 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-10-16T20:03:20.292 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package network-manager-pptp. 2023-10-16T20:03:20.306 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-10-16T20:03:20.342 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-10-16T20:03:20.541 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-10-16T20:03:20.557 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-10-16T20:03:20.599 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-10-16T20:03:20.629 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package postgresql-client-common. 2023-10-16T20:03:20.643 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-10-16T20:03:20.687 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking postgresql-client-common (238) ... 2023-10-16T20:03:20.924 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package postgresql-client-14. 2023-10-16T20:03:20.938 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../27-postgresql-client-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-16T20:03:20.952 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libndp0:amd64. 2023-10-16T20:03:20.967 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-10-16T20:03:20.982 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T20:03:21.010 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-10-16T20:03:21.304 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libnm0:amd64. 2023-10-16T20:03:21.320 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-10-16T20:03:21.362 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-10-16T20:03:21.611 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package postgresql-common. 2023-10-16T20:03:21.626 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-10-16T20:03:21.695 INFO:teuthology.orchestra.run.smithi094.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-10-16T20:03:21.727 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking postgresql-common (238) ... 2023-10-16T20:03:21.783 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libpq5:amd64. 2023-10-16T20:03:21.799 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../16-libpq5_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-16T20:03:21.841 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T20:03:22.178 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-10-16T20:03:22.193 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-10-16T20:03:22.201 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package postgresql-14. 2023-10-16T20:03:22.215 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../29-postgresql-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-16T20:03:22.236 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-10-16T20:03:22.262 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T20:03:22.513 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libqmi-proxy. 2023-10-16T20:03:22.528 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-10-16T20:03:22.605 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-10-16T20:03:22.832 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-10-16T20:03:22.848 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.10_amd64.deb ... 2023-10-16T20:03:22.916 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.10) ... 2023-10-16T20:03:23.427 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-10-16T20:03:23.429 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package postgresql. 2023-10-16T20:03:23.442 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-10-16T20:03:23.444 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-10-16T20:03:23.477 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-10-16T20:03:23.479 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking postgresql (14+238) ... 2023-10-16T20:03:23.706 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package postgresql-client. 2023-10-16T20:03:23.720 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-10-16T20:03:23.764 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking postgresql-client (14+238) ... 2023-10-16T20:03:23.830 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package modemmanager. 2023-10-16T20:03:23.845 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-10-16T20:03:23.888 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-10-16T20:03:24.033 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package postgresql-contrib. 2023-10-16T20:03:24.047 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-10-16T20:03:24.091 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking postgresql-contrib (14+238) ... 2023-10-16T20:03:24.410 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package tcl8.6. 2023-10-16T20:03:24.424 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-10-16T20:03:24.469 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-10-16T20:03:24.519 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package network-manager. 2023-10-16T20:03:24.535 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-10-16T20:03:24.578 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-10-16T20:03:24.755 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package tcl. 2023-10-16T20:03:24.769 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-10-16T20:03:24.809 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-10-16T20:03:25.157 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package wpasupplicant. 2023-10-16T20:03:25.173 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-10-16T20:03:25.185 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ppp. 2023-10-16T20:03:25.200 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-10-16T20:03:25.215 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-10-16T20:03:25.528 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-10-16T20:03:25.561 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-10-16T20:03:25.577 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-10-16T20:03:25.619 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-10-16T20:03:25.872 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package pptp-linux. 2023-10-16T20:03:25.886 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-10-16T20:03:25.931 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-10-16T20:03:25.997 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package usb-modeswitch. 2023-10-16T20:03:26.013 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-10-16T20:03:26.056 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-10-16T20:03:26.183 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package network-manager-pptp. 2023-10-16T20:03:26.197 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-10-16T20:03:26.242 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-10-16T20:03:26.315 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up postgresql-client-common (238) ... 2023-10-16T20:03:26.536 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up flex (2.6.4-8build2) ... 2023-10-16T20:03:26.578 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package postgresql-client-common. 2023-10-16T20:03:26.592 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-10-16T20:03:26.636 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking postgresql-client-common (238) ... 2023-10-16T20:03:26.662 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-10-16T20:03:26.788 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T20:03:26.914 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-10-16T20:03:27.040 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-10-16T20:03:27.068 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package postgresql-client-14. 2023-10-16T20:03:27.083 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../27-postgresql-client-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-16T20:03:27.126 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T20:03:27.166 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T20:03:27.730 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package postgresql-common. 2023-10-16T20:03:27.745 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-10-16T20:03:27.806 INFO:teuthology.orchestra.run.smithi088.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-10-16T20:03:27.838 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking postgresql-common (238) ... 2023-10-16T20:03:28.175 INFO:teuthology.orchestra.run.smithi094.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-10-16T20:03:28.271 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-10-16T20:03:28.296 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package postgresql-14. 2023-10-16T20:03:28.312 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../29-postgresql-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-16T20:03:28.374 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T20:03:28.666 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-16T20:03:28.666 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-16T20:03:29.333 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-10-16T20:03:29.434 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.10) ... 2023-10-16T20:03:29.516 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package postgresql. 2023-10-16T20:03:29.531 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-10-16T20:03:29.574 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking postgresql (14+238) ... 2023-10-16T20:03:29.576 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-10-16T20:03:29.702 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-10-16T20:03:29.818 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package postgresql-client. 2023-10-16T20:03:29.828 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-10-16T20:03:29.832 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-10-16T20:03:29.876 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking postgresql-client (14+238) ... 2023-10-16T20:03:29.962 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-10-16T20:03:30.088 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-10-16T20:03:30.120 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package postgresql-contrib. 2023-10-16T20:03:30.135 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-10-16T20:03:30.179 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking postgresql-contrib (14+238) ... 2023-10-16T20:03:30.498 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package tcl8.6. 2023-10-16T20:03:30.513 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-10-16T20:03:30.547 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-10-16T20:03:30.809 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package tcl. 2023-10-16T20:03:30.818 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-10-16T20:03:30.848 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-10-16T20:03:30.872 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-10-16T20:03:30.944 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up dump (0.4b47-3) ... 2023-10-16T20:03:31.040 INFO:teuthology.orchestra.run.smithi094.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-10-16T20:03:31.112 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-10-16T20:03:31.229 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package wpasupplicant. 2023-10-16T20:03:31.238 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-10-16T20:03:31.244 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-10-16T20:03:31.287 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-10-16T20:03:31.381 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up tcl (8.6.11+1build2) ... 2023-10-16T20:03:31.507 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libjson-perl (4.04000-1) ... 2023-10-16T20:03:31.633 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-10-16T20:03:31.657 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-10-16T20:03:31.673 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-10-16T20:03:31.716 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-10-16T20:03:31.722 INFO:teuthology.orchestra.run.smithi094.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-10-16T20:03:31.793 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2023-10-16T20:03:31.902 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-10-16T20:03:32.061 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-10-16T20:03:32.196 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-10-16T20:03:32.322 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-10-16T20:03:32.469 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package usb-modeswitch. 2023-10-16T20:03:32.485 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-10-16T20:03:32.490 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up postgresql-client (14+238) ... 2023-10-16T20:03:32.535 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-10-16T20:03:32.641 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-10-16T20:03:32.827 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up postgresql-client-common (238) ... 2023-10-16T20:03:33.017 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up flex (2.6.4-8build2) ... 2023-10-16T20:03:33.142 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-10-16T20:03:33.214 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:33.214 INFO:teuthology.orchestra.run.smithi094.stdout:The following network interfaces were found in /etc/network/interfaces 2023-10-16T20:03:33.215 INFO:teuthology.orchestra.run.smithi094.stdout:which means they are currently configured by ifupdown: 2023-10-16T20:03:33.215 INFO:teuthology.orchestra.run.smithi094.stdout:- enp3s0f1 2023-10-16T20:03:33.215 INFO:teuthology.orchestra.run.smithi094.stdout:If you want to manage those interfaces with NetworkManager instead 2023-10-16T20:03:33.215 INFO:teuthology.orchestra.run.smithi094.stdout:remove their configuration from /etc/network/interfaces. 2023-10-16T20:03:33.215 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:33.294 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T20:03:33.346 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-10-16T20:03:33.511 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-10-16T20:03:33.637 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-10-16T20:03:33.743 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-10-16T20:03:33.763 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T20:03:34.153 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-10-16T20:03:34.774 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-10-16T20:03:34.827 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-10-16T20:03:34.860 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-10-16T20:03:34.946 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-10-16T20:03:35.079 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-10-16T20:03:35.230 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up postgresql-common (238) ... 2023-10-16T20:03:35.271 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-16T20:03:35.271 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-16T20:03:35.955 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-10-16T20:03:36.088 INFO:teuthology.orchestra.run.smithi094.stdout:Adding user postgres to group ssl-cert 2023-10-16T20:03:36.098 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.10) ... 2023-10-16T20:03:36.207 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-10-16T20:03:36.333 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-10-16T20:03:36.347 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:36.347 INFO:teuthology.orchestra.run.smithi094.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-10-16T20:03:36.459 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-10-16T20:03:36.585 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-10-16T20:03:36.609 INFO:teuthology.orchestra.run.smithi094.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-10-16T20:03:36.609 INFO:teuthology.orchestra.run.smithi094.stdout:Removing obsolete dictionary files: 2023-10-16T20:03:36.711 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-10-16T20:03:37.327 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-10-16T20:03:37.742 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-10-16T20:03:37.764 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T20:03:37.843 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up dump (0.4b47-3) ... 2023-10-16T20:03:37.930 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-10-16T20:03:38.011 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-10-16T20:03:38.101 INFO:teuthology.orchestra.run.smithi094.stdout:Creating new PostgreSQL cluster 14/main ... 2023-10-16T20:03:38.137 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-10-16T20:03:38.162 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-10-16T20:03:38.222 INFO:teuthology.orchestra.run.smithi094.stdout:The files belonging to this database system will be owned by user "postgres". 2023-10-16T20:03:38.222 INFO:teuthology.orchestra.run.smithi094.stdout:This user must also own the server process. 2023-10-16T20:03:38.223 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:38.223 INFO:teuthology.orchestra.run.smithi094.stdout:The database cluster will be initialized with locale "en_US". 2023-10-16T20:03:38.223 INFO:teuthology.orchestra.run.smithi094.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-10-16T20:03:38.223 INFO:teuthology.orchestra.run.smithi094.stdout:The default text search configuration will be set to "english". 2023-10-16T20:03:38.223 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:38.224 INFO:teuthology.orchestra.run.smithi094.stdout:Data page checksums are disabled. 2023-10-16T20:03:38.224 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:38.224 INFO:teuthology.orchestra.run.smithi094.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-10-16T20:03:38.225 INFO:teuthology.orchestra.run.smithi094.stdout:creating subdirectories ... ok 2023-10-16T20:03:38.225 INFO:teuthology.orchestra.run.smithi094.stdout:selecting dynamic shared memory implementation ... posix 2023-10-16T20:03:38.248 INFO:teuthology.orchestra.run.smithi094.stdout:selecting default max_connections ... 100 2023-10-16T20:03:38.263 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up tcl (8.6.11+1build2) ... 2023-10-16T20:03:38.282 INFO:teuthology.orchestra.run.smithi094.stdout:selecting default shared_buffers ... 128MB 2023-10-16T20:03:38.296 INFO:teuthology.orchestra.run.smithi094.stdout:selecting default time zone ... Etc/UTC 2023-10-16T20:03:38.297 INFO:teuthology.orchestra.run.smithi094.stdout:creating configuration files ... ok 2023-10-16T20:03:38.389 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libjson-perl (4.04000-1) ... 2023-10-16T20:03:38.466 INFO:teuthology.orchestra.run.smithi094.stdout:running bootstrap script ... ok 2023-10-16T20:03:38.607 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-10-16T20:03:38.697 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-10-16T20:03:38.767 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2023-10-16T20:03:38.868 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-10-16T20:03:39.036 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-10-16T20:03:39.116 INFO:teuthology.orchestra.run.smithi094.stdout:performing post-bootstrap initialization ... ok 2023-10-16T20:03:39.162 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-10-16T20:03:39.288 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-10-16T20:03:39.456 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up postgresql-client (14+238) ... 2023-10-16T20:03:39.582 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-10-16T20:03:39.721 INFO:teuthology.orchestra.run.smithi094.stdout:syncing data to disk ... ok 2023-10-16T20:03:40.170 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:40.170 INFO:teuthology.orchestra.run.smithi088.stdout:The following network interfaces were found in /etc/network/interfaces 2023-10-16T20:03:40.170 INFO:teuthology.orchestra.run.smithi088.stdout:which means they are currently configured by ifupdown: 2023-10-16T20:03:40.170 INFO:teuthology.orchestra.run.smithi088.stdout:- enp3s0f1 2023-10-16T20:03:40.171 INFO:teuthology.orchestra.run.smithi088.stdout:If you want to manage those interfaces with NetworkManager instead 2023-10-16T20:03:40.171 INFO:teuthology.orchestra.run.smithi088.stdout:remove their configuration from /etc/network/interfaces. 2023-10-16T20:03:40.171 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:40.225 INFO:teuthology.orchestra.run.smithi094.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-10-16T20:03:40.302 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-10-16T20:03:40.736 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-10-16T20:03:41.151 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-10-16T20:03:41.851 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-10-16T20:03:42.011 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-10-16T20:03:42.136 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-10-16T20:03:42.263 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up postgresql-common (238) ... 2023-10-16T20:03:43.164 INFO:teuthology.orchestra.run.smithi088.stdout:Adding user postgres to group ssl-cert 2023-10-16T20:03:43.416 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:43.416 INFO:teuthology.orchestra.run.smithi088.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-10-16T20:03:43.581 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-10-16T20:03:43.667 INFO:teuthology.orchestra.run.smithi088.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-10-16T20:03:43.667 INFO:teuthology.orchestra.run.smithi088.stdout:Removing obsolete dictionary files: 2023-10-16T20:03:43.706 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up postgresql-contrib (14+238) ... 2023-10-16T20:03:43.857 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up postgresql (14+238) ... 2023-10-16T20:03:43.983 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-10-16T20:03:44.101 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-10-16T20:03:44.346 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-10-16T20:03:44.346 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-10-16T20:03:44.375 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-10-16T20:03:44.856 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T20:03:45.120 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-10-16T20:03:45.285 INFO:teuthology.orchestra.run.smithi088.stdout:Creating new PostgreSQL cluster 14/main ... 2023-10-16T20:03:45.317 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-10-16T20:03:45.372 INFO:teuthology.orchestra.run.smithi088.stdout:The files belonging to this database system will be owned by user "postgres". 2023-10-16T20:03:45.372 INFO:teuthology.orchestra.run.smithi088.stdout:This user must also own the server process. 2023-10-16T20:03:45.372 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:45.372 INFO:teuthology.orchestra.run.smithi088.stdout:The database cluster will be initialized with locale "en_US". 2023-10-16T20:03:45.373 INFO:teuthology.orchestra.run.smithi088.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-10-16T20:03:45.373 INFO:teuthology.orchestra.run.smithi088.stdout:The default text search configuration will be set to "english". 2023-10-16T20:03:45.373 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:45.373 INFO:teuthology.orchestra.run.smithi088.stdout:Data page checksums are disabled. 2023-10-16T20:03:45.373 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:45.374 INFO:teuthology.orchestra.run.smithi088.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-10-16T20:03:45.374 INFO:teuthology.orchestra.run.smithi088.stdout:creating subdirectories ... ok 2023-10-16T20:03:45.374 INFO:teuthology.orchestra.run.smithi088.stdout:selecting dynamic shared memory implementation ... posix 2023-10-16T20:03:45.398 INFO:teuthology.orchestra.run.smithi088.stdout:selecting default max_connections ... 100 2023-10-16T20:03:45.433 INFO:teuthology.orchestra.run.smithi088.stdout:selecting default shared_buffers ... 128MB 2023-10-16T20:03:45.446 INFO:teuthology.orchestra.run.smithi088.stdout:selecting default time zone ... Etc/UTC 2023-10-16T20:03:45.447 INFO:teuthology.orchestra.run.smithi088.stdout:creating configuration files ... ok 2023-10-16T20:03:45.455 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-10-16T20:03:45.606 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-10-16T20:03:45.615 INFO:teuthology.orchestra.run.smithi088.stdout:running bootstrap script ... ok 2023-10-16T20:03:46.261 INFO:teuthology.orchestra.run.smithi088.stdout:performing post-bootstrap initialization ... ok 2023-10-16T20:03:46.825 INFO:teuthology.orchestra.run.smithi088.stdout:syncing data to disk ... ok 2023-10-16T20:03:47.142 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-10-16T20:03:47.242 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-10-16T20:03:47.352 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-10-16T20:03:47.987 INFO:teuthology.orchestra.run.smithi094.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-16T20:03:47.992 DEBUG:teuthology.parallel:result is None 2023-10-16T20:03:50.338 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-10-16T20:03:50.463 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up postgresql-contrib (14+238) ... 2023-10-16T20:03:50.590 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up postgresql (14+238) ... 2023-10-16T20:03:50.690 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-10-16T20:03:50.791 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-10-16T20:03:51.051 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-10-16T20:03:51.052 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-10-16T20:03:51.794 INFO:teuthology.orchestra.run.smithi088.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-10-16T20:03:52.121 INFO:teuthology.orchestra.run.smithi088.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-10-16T20:03:52.281 INFO:teuthology.orchestra.run.smithi088.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-10-16T20:03:53.659 INFO:teuthology.orchestra.run.smithi088.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-10-16T20:03:53.751 INFO:teuthology.orchestra.run.smithi088.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-10-16T20:03:54.445 INFO:teuthology.orchestra.run.smithi088.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-16T20:03:54.449 DEBUG:teuthology.parallel:result is None 2023-10-16T20:03:54.450 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 2023-10-16T20:03:54.853 DEBUG:teuthology.orchestra.run.smithi088:> dpkg-query -W -f '${Version}' ceph 2023-10-16T20:03:54.875 INFO:teuthology.orchestra.run.smithi088.stdout:18.0.0-6730-g4ba8a190-1jammy 2023-10-16T20:03:54.876 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6730-g4ba8a190-1jammy 2023-10-16T20:03:54.876 INFO:teuthology.task.install:The correct ceph version 18.0.0-6730-g4ba8a190-1jammy is installed. 2023-10-16T20:03:54.876 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 2023-10-16T20:03:55.014 DEBUG:teuthology.orchestra.run.smithi094:> dpkg-query -W -f '${Version}' ceph 2023-10-16T20:03:55.038 INFO:teuthology.orchestra.run.smithi094.stdout:18.0.0-6730-g4ba8a190-1jammy 2023-10-16T20:03:55.038 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6730-g4ba8a190-1jammy 2023-10-16T20:03:55.039 INFO:teuthology.task.install:The correct ceph version 18.0.0-6730-g4ba8a190-1jammy is installed. 2023-10-16T20:03:55.039 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-16T20:03:55.040 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:03:55.040 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-16T20:03:55.061 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:03:55.061 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-16T20:03:55.107 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-16T20:03:55.107 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:03:55.108 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/usr/bin/daemon-helper 2023-10-16T20:03:55.128 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-16T20:03:55.194 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:03:55.195 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/daemon-helper 2023-10-16T20:03:55.217 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-16T20:03:55.278 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-16T20:03:55.279 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:03:55.279 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-16T20:03:55.299 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-16T20:03:55.363 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:03:55.363 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-16T20:03:55.385 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-16T20:03:55.445 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-16T20:03:55.446 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:03:55.446 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/usr/bin/stdin-killer 2023-10-16T20:03:55.466 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-16T20:03:55.526 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:03:55.527 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/stdin-killer 2023-10-16T20:03:55.548 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-16T20:03:55.609 INFO:teuthology.run_tasks:Running task ceph... 2023-10-16T20:03:55.674 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-16T20:03:55.675 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 777 /var/log/ceph 2023-10-16T20:03:55.679 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 777 /var/log/ceph 2023-10-16T20:03:55.700 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-16T20:03:55.701 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-16T20:03:55.739 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-16T20:03:55.766 INFO:tasks.ceph:Creating extra log directories... 2023-10-16T20:03:55.766 DEBUG:teuthology.orchestra.run.smithi088:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-16T20:03:55.799 DEBUG:teuthology.orchestra.run.smithi094:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-16T20:03:55.839 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-16T20:03:55.840 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-16T20:03:55.840 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-16T20:03:55.840 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:03:55.840 DEBUG:teuthology.orchestra.run.smithi088:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-16T20:03:55.866 DEBUG:teuthology.orchestra.run.smithi088:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-16T20:03:55.954 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:03:55.954 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-16T20:03:55.964 DEBUG:teuthology.orchestra.run.smithi094:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-16T20:03:56.060 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-16T20:03:56.061 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-16T20:03:56.061 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-10-16_18:49:15-fs-rishabh-2023oct13-testing-default-smithi/7430091', 'branch': 'rishabh-2023oct13', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm}', 'email': None, 'first_in_suite': False, 'job_id': '7430091', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '47672119149b103ec7cec2cc65c063ccbb001ac7'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-10-16_18:49:15-fs-rishabh-2023oct13-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'rishabh-2023oct13'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'rishabh-2023oct13-qa2', 'sha1': '19fd96fe3581520b6d9f50ffdabf29051aee5d4d'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 6006, 'sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0', 'sleep_before_teardown': 0, 'subset': '402/512', 'suite': 'fs', 'suite_branch': 'rishabh-2023oct13-qa2', 'suite_path': '/home/teuthworker/src/github.com_rishabh-d-dave_ceph_19fd96fe3581520b6d9f50ffdabf29051aee5d4d/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/rishabh-d-dave/ceph', 'suite_sha1': '19fd96fe3581520b6d9f50ffdabf29051aee5d4d', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi088.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO3u8ck309lL1eF1slemCDnL7evTbk1vSlxyuVhiJ0jwjFWr/00avoczBkwVMMA35fzd5AfNup9oIrkl12E+ihA=', 'smithi094.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMmVItP9PTvUPm1HK+zbQSrkxi0N+SNhl6uG+l41BT706NG4MjhexTB9Ww7pdf+KZC/ZJNRNAt9HxF8lUnSjKy8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '47672119149b103ec7cec2cc65c063ccbb001ac7'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_pool_perm']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '8cdab074dcca9a68965bc5a50e9c30b691949723', 'timestamp': '2023-10-16_18:49:15', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-16T20:03:56.062 DEBUG:teuthology.orchestra.run.smithi088:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-16T20:03:56.066 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-16T20:03:56.075 DEBUG:teuthology.orchestra.run.smithi088:> sudo install -d -m0777 -- /var/run/ceph 2023-10-16T20:03:56.119 DEBUG:teuthology.orchestra.run.smithi094:> sudo install -d -m0777 -- /var/run/ceph 2023-10-16T20:03:56.140 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:03:56.140 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/scratch_devs of=/dev/stdout 2023-10-16T20:03:56.184 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-16T20:03:56.184 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_1 2023-10-16T20:03:56.236 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-16T20:03:56.236 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T20:03:56.237 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 5h/5d Inode: 889 Links: 1 2023-10-16T20:03:56.237 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T20:03:56.237 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2023-10-16 19:57:24.799540376 +0000 2023-10-16T20:03:56.237 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2023-10-16 19:57:24.407543876 +0000 2023-10-16T20:03:56.237 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2023-10-16 19:57:24.407543876 +0000 2023-10-16T20:03:56.237 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2023-10-16 19:57:24.407543876 +0000 2023-10-16T20:03:56.238 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-16T20:03:56.298 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2023-10-16T20:03:56.299 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2023-10-16T20:03:56.299 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000345833 s, 1.5 MB/s 2023-10-16T20:03:56.300 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-16T20:03:56.357 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_2 2023-10-16T20:03:56.412 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-16T20:03:56.412 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T20:03:56.413 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 5h/5d Inode: 908 Links: 1 2023-10-16T20:03:56.413 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T20:03:56.413 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2023-10-16 19:57:25.475534337 +0000 2023-10-16T20:03:56.413 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2023-10-16 19:57:25.067537981 +0000 2023-10-16T20:03:56.413 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2023-10-16 19:57:25.067537981 +0000 2023-10-16T20:03:56.414 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2023-10-16 19:57:25.067537981 +0000 2023-10-16T20:03:56.414 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-16T20:03:56.473 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2023-10-16T20:03:56.474 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2023-10-16T20:03:56.474 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000335026 s, 1.5 MB/s 2023-10-16T20:03:56.476 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-16T20:03:56.530 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_3 2023-10-16T20:03:56.580 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-16T20:03:56.581 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T20:03:56.581 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 5h/5d Inode: 926 Links: 1 2023-10-16T20:03:56.582 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T20:03:56.582 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2023-10-16 19:57:26.107528692 +0000 2023-10-16T20:03:56.582 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2023-10-16 19:57:25.723532122 +0000 2023-10-16T20:03:56.583 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2023-10-16 19:57:25.723532122 +0000 2023-10-16T20:03:56.583 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2023-10-16 19:57:25.723532122 +0000 2023-10-16T20:03:56.584 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-16T20:03:56.641 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2023-10-16T20:03:56.642 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2023-10-16T20:03:56.642 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000350076 s, 1.5 MB/s 2023-10-16T20:03:56.643 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-16T20:03:56.693 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_4 2023-10-16T20:03:56.748 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-16T20:03:56.748 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T20:03:56.749 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 5h/5d Inode: 942 Links: 1 2023-10-16T20:03:56.749 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T20:03:56.749 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2023-10-16 19:57:26.787522619 +0000 2023-10-16T20:03:56.749 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2023-10-16 19:57:26.383526228 +0000 2023-10-16T20:03:56.749 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2023-10-16 19:57:26.383526228 +0000 2023-10-16T20:03:56.750 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2023-10-16 19:57:26.383526228 +0000 2023-10-16T20:03:56.750 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-16T20:03:56.809 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2023-10-16T20:03:56.810 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2023-10-16T20:03:56.810 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.00036375 s, 1.4 MB/s 2023-10-16T20:03:56.811 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-16T20:03:56.862 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-16T20:03:56.862 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:03:56.863 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/scratch_devs of=/dev/stdout 2023-10-16T20:03:56.873 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-16T20:03:56.873 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_1 2023-10-16T20:03:56.930 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-16T20:03:56.930 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T20:03:56.930 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 897 Links: 1 2023-10-16T20:03:56.931 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T20:03:56.931 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2023-10-16 19:57:24.781009569 +0000 2023-10-16T20:03:56.931 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2023-10-16 19:57:24.389013071 +0000 2023-10-16T20:03:56.931 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2023-10-16 19:57:24.389013071 +0000 2023-10-16T20:03:56.931 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2023-10-16 19:57:24.389013071 +0000 2023-10-16T20:03:56.932 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-16T20:03:56.996 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2023-10-16T20:03:56.997 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2023-10-16T20:03:56.997 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000280865 s, 1.8 MB/s 2023-10-16T20:03:56.998 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-16T20:03:57.052 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_2 2023-10-16T20:03:57.102 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-16T20:03:57.102 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T20:03:57.103 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 913 Links: 1 2023-10-16T20:03:57.103 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T20:03:57.103 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2023-10-16 19:57:25.473003388 +0000 2023-10-16T20:03:57.103 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2023-10-16 19:57:25.057007104 +0000 2023-10-16T20:03:57.104 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2023-10-16 19:57:25.057007104 +0000 2023-10-16T20:03:57.104 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2023-10-16 19:57:25.057007104 +0000 2023-10-16T20:03:57.104 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-16T20:03:57.164 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2023-10-16T20:03:57.165 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2023-10-16T20:03:57.165 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000269632 s, 1.9 MB/s 2023-10-16T20:03:57.167 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-16T20:03:57.220 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_3 2023-10-16T20:03:57.270 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-16T20:03:57.271 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T20:03:57.271 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 932 Links: 1 2023-10-16T20:03:57.271 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T20:03:57.271 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2023-10-16 19:57:26.128997529 +0000 2023-10-16T20:03:57.271 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2023-10-16 19:57:25.713001245 +0000 2023-10-16T20:03:57.272 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2023-10-16 19:57:25.713001245 +0000 2023-10-16T20:03:57.272 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2023-10-16 19:57:25.713001245 +0000 2023-10-16T20:03:57.272 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-16T20:03:57.336 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2023-10-16T20:03:57.337 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2023-10-16T20:03:57.337 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000272737 s, 1.9 MB/s 2023-10-16T20:03:57.339 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-16T20:03:57.392 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_4 2023-10-16T20:03:57.442 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-16T20:03:57.443 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T20:03:57.443 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 947 Links: 1 2023-10-16T20:03:57.443 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T20:03:57.443 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2023-10-16 19:57:26.832991241 +0000 2023-10-16T20:03:57.443 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2023-10-16 19:57:26.404995064 +0000 2023-10-16T20:03:57.444 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2023-10-16 19:57:26.404995064 +0000 2023-10-16T20:03:57.444 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2023-10-16 19:57:26.404995064 +0000 2023-10-16T20:03:57.444 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-16T20:03:57.504 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2023-10-16T20:03:57.504 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2023-10-16T20:03:57.505 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000271721 s, 1.9 MB/s 2023-10-16T20:03:57.507 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-16T20:03:57.560 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-16T20:03:57.561 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi088.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi094.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-16T20:03:57.561 INFO:tasks.ceph:Generating config... 2023-10-16T20:03:57.562 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-16T20:03:57.563 INFO:tasks.ceph:[client] debug client = 20 2023-10-16T20:03:57.563 INFO:tasks.ceph:[client] debug ms = 1 2023-10-16T20:03:57.563 INFO:tasks.ceph:[client] pid file = 2023-10-16T20:03:57.563 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-16T20:03:57.564 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-16T20:03:57.564 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-16T20:03:57.564 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-10-16T20:03:57.564 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-16T20:03:57.565 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-16T20:03:57.565 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-16T20:03:57.565 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-16T20:03:57.565 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-16T20:03:57.566 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-16T20:03:57.566 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-16T20:03:57.566 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-16T20:03:57.566 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-16T20:03:57.567 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-16T20:03:57.567 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-16T20:03:57.567 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-16T20:03:57.568 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-16T20:03:57.568 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-16T20:03:57.568 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-16T20:03:57.568 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-16T20:03:57.569 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-10-16T20:03:57.569 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-10-16T20:03:57.569 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-16T20:03:57.569 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-16T20:03:57.570 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-10-16T20:03:57.570 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-16T20:03:57.570 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-16T20:03:57.571 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-16T20:03:57.571 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-16T20:03:57.571 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-16T20:03:57.571 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-16T20:03:57.572 INFO:tasks.ceph:Setting up mon.a... 2023-10-16T20:03:57.572 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-16T20:03:57.618 INFO:teuthology.orchestra.run.smithi088.stdout:creating /etc/ceph/ceph.keyring 2023-10-16T20:03:57.637 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-16T20:03:57.732 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-16T20:03:57.795 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.88'), ('mon.b', '172.21.15.94'), ('mon.c', '[v2:172.21.15.94:3301,v1:172.21.15.94:6790]')] 2023-10-16T20:03:57.796 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.88,172.21.15.94,[v2:172.21.15.94:3301,v1:172.21.15.94:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-16T20:03:57.797 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:03:57.798 DEBUG:teuthology.orchestra.run.smithi088:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-16T20:03:57.848 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.88 --add b 172.21.15.94 --addv c '[v2:172.21.15.94:3301,v1:172.21.15.94:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-16T20:03:57.938 INFO:teuthology.orchestra.run.smithi088.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-16T20:03:57.938 INFO:teuthology.orchestra.run.smithi088.stdout:monmaptool: generated fsid d11187c0-3dd8-4c92-8d13-6ec536230748 2023-10-16T20:03:57.938 INFO:teuthology.orchestra.run.smithi088.stdout:setting min_mon_release = pacific 2023-10-16T20:03:57.938 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 0 2023-10-16T20:03:57.939 INFO:teuthology.orchestra.run.smithi088.stdout:fsid d11187c0-3dd8-4c92-8d13-6ec536230748 2023-10-16T20:03:57.939 INFO:teuthology.orchestra.run.smithi088.stdout:last_changed 2023-10-16T20:03:57.940735+0000 2023-10-16T20:03:57.939 INFO:teuthology.orchestra.run.smithi088.stdout:created 2023-10-16T20:03:57.940735+0000 2023-10-16T20:03:57.939 INFO:teuthology.orchestra.run.smithi088.stdout:min_mon_release 16 (pacific) 2023-10-16T20:03:57.939 INFO:teuthology.orchestra.run.smithi088.stdout:election_strategy: 1 2023-10-16T20:03:57.940 INFO:teuthology.orchestra.run.smithi088.stdout:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2023-10-16T20:03:57.940 INFO:teuthology.orchestra.run.smithi088.stdout:1: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.b 2023-10-16T20:03:57.940 INFO:teuthology.orchestra.run.smithi088.stdout:2: [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] mon.c 2023-10-16T20:03:57.940 INFO:teuthology.orchestra.run.smithi088.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-16T20:03:57.942 DEBUG:teuthology.orchestra.run.smithi088:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-16T20:03:57.950 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID d11187c0-3dd8-4c92-8d13-6ec536230748... 2023-10-16T20:03:57.952 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-16T20:03:57.995 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-16T20:03:58.035 INFO:teuthology.orchestra.run.smithi088.stdout:[global] 2023-10-16T20:03:58.036 INFO:teuthology.orchestra.run.smithi088.stdout: chdir = "" 2023-10-16T20:03:58.036 INFO:teuthology.orchestra.run.smithi088.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-16T20:03:58.036 INFO:teuthology.orchestra.run.smithi088.stdout: auth supported = cephx 2023-10-16T20:03:58.036 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:58.036 INFO:teuthology.orchestra.run.smithi088.stdout: filestore xattr use omap = true 2023-10-16T20:03:58.037 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:58.037 INFO:teuthology.orchestra.run.smithi088.stdout: mon clock drift allowed = 1.000 2023-10-16T20:03:58.037 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:58.037 INFO:teuthology.orchestra.run.smithi088.stdout: osd crush chooseleaf type = 0 2023-10-16T20:03:58.037 INFO:teuthology.orchestra.run.smithi088.stdout: auth debug = true 2023-10-16T20:03:58.038 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:58.038 INFO:teuthology.orchestra.run.smithi088.stdout: ms die on old message = true 2023-10-16T20:03:58.038 INFO:teuthology.orchestra.run.smithi088.stdout: ms die on bug = true 2023-10-16T20:03:58.038 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:58.038 INFO:teuthology.orchestra.run.smithi088.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-16T20:03:58.039 INFO:teuthology.orchestra.run.smithi088.stdout: mon pg warn max object skew = 0 2023-10-16T20:03:58.039 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:58.039 INFO:teuthology.orchestra.run.smithi088.stdout: # disable pg_autoscaler by default for new pools 2023-10-16T20:03:58.039 INFO:teuthology.orchestra.run.smithi088.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-16T20:03:58.039 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:58.040 INFO:teuthology.orchestra.run.smithi088.stdout: osd pool default size = 2 2023-10-16T20:03:58.040 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:58.040 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd allow primary affinity = true 2023-10-16T20:03:58.040 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd allow pg remap = true 2023-10-16T20:03:58.040 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on legacy crush tunables = false 2023-10-16T20:03:58.040 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on crush straw calc version zero = false 2023-10-16T20:03:58.041 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on no sortbitwise = false 2023-10-16T20:03:58.041 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on osd down out interval zero = false 2023-10-16T20:03:58.041 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on too few osds = false 2023-10-16T20:03:58.041 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-16T20:03:58.041 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-16T20:03:58.041 INFO:teuthology.orchestra.run.smithi088.stdout: mon_allow_pool_size_one = true 2023-10-16T20:03:58.042 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:58.042 INFO:teuthology.orchestra.run.smithi088.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-16T20:03:58.042 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:58.042 INFO:teuthology.orchestra.run.smithi088.stdout: osd default data pool replay window = 5 2023-10-16T20:03:58.042 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:58.043 INFO:teuthology.orchestra.run.smithi088.stdout: mon allow pool delete = true 2023-10-16T20:03:58.043 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:58.043 INFO:teuthology.orchestra.run.smithi088.stdout: mon cluster log file level = debug 2023-10-16T20:03:58.043 INFO:teuthology.orchestra.run.smithi088.stdout: debug asserts on shutdown = true 2023-10-16T20:03:58.043 INFO:teuthology.orchestra.run.smithi088.stdout: mon health detail to clog = false 2023-10-16T20:03:58.043 INFO:teuthology.orchestra.run.smithi088.stdout: mon host = "172.21.15.88,172.21.15.94,[v2:172.21.15.94:3301,v1:172.21.15.94:6790]" 2023-10-16T20:03:58.044 INFO:teuthology.orchestra.run.smithi088.stdout: fsid = d11187c0-3dd8-4c92-8d13-6ec536230748 2023-10-16T20:03:58.044 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:58.044 INFO:teuthology.orchestra.run.smithi088.stdout:[osd] 2023-10-16T20:03:58.044 INFO:teuthology.orchestra.run.smithi088.stdout: osd journal size = 100 2023-10-16T20:03:58.044 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:58.044 INFO:teuthology.orchestra.run.smithi088.stdout: osd scrub load threshold = 5.0 2023-10-16T20:03:58.045 INFO:teuthology.orchestra.run.smithi088.stdout: osd scrub max interval = 600 2023-10-16T20:03:58.045 INFO:teuthology.orchestra.run.smithi088.stdout: osd mclock profile = high_recovery_ops 2023-10-16T20:03:58.045 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:58.045 INFO:teuthology.orchestra.run.smithi088.stdout: osd recover clone overlap = true 2023-10-16T20:03:58.045 INFO:teuthology.orchestra.run.smithi088.stdout: osd recovery max chunk = 1048576 2023-10-16T20:03:58.045 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:58.046 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug shutdown = true 2023-10-16T20:03:58.046 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug op order = true 2023-10-16T20:03:58.046 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug verify stray on activate = true 2023-10-16T20:03:58.046 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:58.046 INFO:teuthology.orchestra.run.smithi088.stdout: osd open classes on start = true 2023-10-16T20:03:58.047 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug pg log writeout = true 2023-10-16T20:03:58.047 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:58.047 INFO:teuthology.orchestra.run.smithi088.stdout: osd deep scrub update digest min age = 30 2023-10-16T20:03:58.047 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:58.047 INFO:teuthology.orchestra.run.smithi088.stdout: osd map max advance = 10 2023-10-16T20:03:58.047 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:58.048 INFO:teuthology.orchestra.run.smithi088.stdout: journal zero on create = true 2023-10-16T20:03:58.048 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:58.048 INFO:teuthology.orchestra.run.smithi088.stdout: filestore ondisk finisher threads = 3 2023-10-16T20:03:58.048 INFO:teuthology.orchestra.run.smithi088.stdout: filestore apply finisher threads = 3 2023-10-16T20:03:58.048 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:58.049 INFO:teuthology.orchestra.run.smithi088.stdout: bdev debug aio = true 2023-10-16T20:03:58.049 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug misdirected ops = true 2023-10-16T20:03:58.049 INFO:teuthology.orchestra.run.smithi088.stdout: bluestore block size = 96636764160 2023-10-16T20:03:58.049 INFO:teuthology.orchestra.run.smithi088.stdout: bluestore fsck on mount = True 2023-10-16T20:03:58.049 INFO:teuthology.orchestra.run.smithi088.stdout: debug bluefs = 20 2023-10-16T20:03:58.049 INFO:teuthology.orchestra.run.smithi088.stdout: debug bluestore = 20 2023-10-16T20:03:58.050 INFO:teuthology.orchestra.run.smithi088.stdout: debug ms = 1 2023-10-16T20:03:58.050 INFO:teuthology.orchestra.run.smithi088.stdout: debug osd = 20 2023-10-16T20:03:58.050 INFO:teuthology.orchestra.run.smithi088.stdout: debug rocksdb = 10 2023-10-16T20:03:58.050 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd backfillfull_ratio = 0.85 2023-10-16T20:03:58.050 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd full ratio = 0.9 2023-10-16T20:03:58.051 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd nearfull ratio = 0.8 2023-10-16T20:03:58.051 INFO:teuthology.orchestra.run.smithi088.stdout: osd failsafe full ratio = 0.95 2023-10-16T20:03:58.051 INFO:teuthology.orchestra.run.smithi088.stdout: osd objectstore = bluestore 2023-10-16T20:03:58.051 INFO:teuthology.orchestra.run.smithi088.stdout: osd op complaint time = 180 2023-10-16T20:03:58.051 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:58.051 INFO:teuthology.orchestra.run.smithi088.stdout:[mgr] 2023-10-16T20:03:58.052 INFO:teuthology.orchestra.run.smithi088.stdout: debug ms = 1 2023-10-16T20:03:58.052 INFO:teuthology.orchestra.run.smithi088.stdout: debug mgr = 20 2023-10-16T20:03:58.052 INFO:teuthology.orchestra.run.smithi088.stdout: debug mon = 20 2023-10-16T20:03:58.052 INFO:teuthology.orchestra.run.smithi088.stdout: debug auth = 20 2023-10-16T20:03:58.052 INFO:teuthology.orchestra.run.smithi088.stdout: mon reweight min pgs per osd = 4 2023-10-16T20:03:58.052 INFO:teuthology.orchestra.run.smithi088.stdout: mon reweight min bytes per osd = 10 2023-10-16T20:03:58.053 INFO:teuthology.orchestra.run.smithi088.stdout: mgr/telemetry/nag = false 2023-10-16T20:03:58.053 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:58.053 INFO:teuthology.orchestra.run.smithi088.stdout:[mon] 2023-10-16T20:03:58.053 INFO:teuthology.orchestra.run.smithi088.stdout: debug ms = 1 2023-10-16T20:03:58.053 INFO:teuthology.orchestra.run.smithi088.stdout: debug mon = 20 2023-10-16T20:03:58.053 INFO:teuthology.orchestra.run.smithi088.stdout: debug paxos = 20 2023-10-16T20:03:58.054 INFO:teuthology.orchestra.run.smithi088.stdout: debug auth = 20 2023-10-16T20:03:58.054 INFO:teuthology.orchestra.run.smithi088.stdout: mon data avail warn = 5 2023-10-16T20:03:58.054 INFO:teuthology.orchestra.run.smithi088.stdout: mon mgr mkfs grace = 240 2023-10-16T20:03:58.054 INFO:teuthology.orchestra.run.smithi088.stdout: mon reweight min pgs per osd = 4 2023-10-16T20:03:58.054 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd reporter subtree level = osd 2023-10-16T20:03:58.055 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd prime pg temp = true 2023-10-16T20:03:58.055 INFO:teuthology.orchestra.run.smithi088.stdout: mon reweight min bytes per osd = 10 2023-10-16T20:03:58.055 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:58.055 INFO:teuthology.orchestra.run.smithi088.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-16T20:03:58.055 INFO:teuthology.orchestra.run.smithi088.stdout: auth mon ticket ttl = 660 # 11m 2023-10-16T20:03:58.055 INFO:teuthology.orchestra.run.smithi088.stdout: auth service ticket ttl = 240 # 4m 2023-10-16T20:03:58.056 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:58.056 INFO:teuthology.orchestra.run.smithi088.stdout: # don't complain about insecure global_id in the test suite 2023-10-16T20:03:58.056 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-16T20:03:58.056 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-16T20:03:58.056 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:58.057 INFO:teuthology.orchestra.run.smithi088.stdout: # 1m isn't quite enough 2023-10-16T20:03:58.057 INFO:teuthology.orchestra.run.smithi088.stdout: mon_down_mkfs_grace = 2m 2023-10-16T20:03:58.057 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:58.057 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_filestore_osds = false 2023-10-16T20:03:58.057 INFO:teuthology.orchestra.run.smithi088.stdout: mon op complaint time = 120 2023-10-16T20:03:58.058 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:03:58.058 INFO:teuthology.orchestra.run.smithi088.stdout:[client] 2023-10-16T20:03:58.058 INFO:teuthology.orchestra.run.smithi088.stdout: rgw cache enabled = true 2023-10-16T20:03:58.058 INFO:teuthology.orchestra.run.smithi088.stdout: rgw enable ops log = true 2023-10-16T20:03:58.058 INFO:teuthology.orchestra.run.smithi088.stdout: rgw enable usage log = true 2023-10-16T20:03:58.059 INFO:teuthology.orchestra.run.smithi088.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-16T20:03:58.059 INFO:teuthology.orchestra.run.smithi088.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-16T20:03:58.059 INFO:teuthology.orchestra.run.smithi088.stdout: client mount timeout = 600 2023-10-16T20:03:58.059 INFO:teuthology.orchestra.run.smithi088.stdout: debug client = 20 2023-10-16T20:03:58.059 INFO:teuthology.orchestra.run.smithi088.stdout: debug ms = 1 2023-10-16T20:03:58.060 INFO:teuthology.orchestra.run.smithi088.stdout: pid file = "" 2023-10-16T20:03:58.060 INFO:teuthology.orchestra.run.smithi088.stdout: rados mon op timeout = 900 2023-10-16T20:03:58.060 INFO:teuthology.orchestra.run.smithi088.stdout: rados osd op timeout = 900 2023-10-16T20:03:58.060 INFO:teuthology.orchestra.run.smithi088.stdout:[mon.a] 2023-10-16T20:03:58.060 INFO:teuthology.orchestra.run.smithi088.stdout:[mon.b] 2023-10-16T20:03:58.061 INFO:teuthology.orchestra.run.smithi088.stdout:[mon.c] 2023-10-16T20:03:58.061 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.a] 2023-10-16T20:03:58.061 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.c] 2023-10-16T20:03:58.061 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.b] 2023-10-16T20:03:58.061 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.d] 2023-10-16T20:03:58.061 INFO:teuthology.orchestra.run.smithi088.stdout:[mds] 2023-10-16T20:03:58.062 INFO:teuthology.orchestra.run.smithi088.stdout: debug mds = 20 2023-10-16T20:03:58.062 INFO:teuthology.orchestra.run.smithi088.stdout: debug mds balancer = 20 2023-10-16T20:03:58.062 INFO:teuthology.orchestra.run.smithi088.stdout: debug ms = 1 2023-10-16T20:03:58.062 INFO:teuthology.orchestra.run.smithi088.stdout: mds debug frag = True 2023-10-16T20:03:58.062 INFO:teuthology.orchestra.run.smithi088.stdout: mds debug scatterstat = True 2023-10-16T20:03:58.063 INFO:teuthology.orchestra.run.smithi088.stdout: mds op complaint time = 180 2023-10-16T20:03:58.063 INFO:teuthology.orchestra.run.smithi088.stdout: mds verify scatter = True 2023-10-16T20:03:58.063 INFO:teuthology.orchestra.run.smithi088.stdout: osd op complaint time = 180 2023-10-16T20:03:58.063 INFO:teuthology.orchestra.run.smithi088.stdout: rados mon op timeout = 900 2023-10-16T20:03:58.063 INFO:teuthology.orchestra.run.smithi088.stdout: rados osd op timeout = 900 2023-10-16T20:03:58.065 INFO:teuthology.orchestra.run.smithi094.stdout:[global] 2023-10-16T20:03:58.065 INFO:teuthology.orchestra.run.smithi094.stdout: chdir = "" 2023-10-16T20:03:58.065 INFO:teuthology.orchestra.run.smithi094.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-16T20:03:58.065 INFO:teuthology.orchestra.run.smithi094.stdout: auth supported = cephx 2023-10-16T20:03:58.066 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:58.066 INFO:teuthology.orchestra.run.smithi094.stdout: filestore xattr use omap = true 2023-10-16T20:03:58.066 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:58.066 INFO:teuthology.orchestra.run.smithi094.stdout: mon clock drift allowed = 1.000 2023-10-16T20:03:58.066 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:58.067 INFO:teuthology.orchestra.run.smithi094.stdout: osd crush chooseleaf type = 0 2023-10-16T20:03:58.067 INFO:teuthology.orchestra.run.smithi094.stdout: auth debug = true 2023-10-16T20:03:58.067 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:58.067 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on old message = true 2023-10-16T20:03:58.067 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on bug = true 2023-10-16T20:03:58.068 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:58.068 INFO:teuthology.orchestra.run.smithi094.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-16T20:03:58.068 INFO:teuthology.orchestra.run.smithi094.stdout: mon pg warn max object skew = 0 2023-10-16T20:03:58.068 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:58.068 INFO:teuthology.orchestra.run.smithi094.stdout: # disable pg_autoscaler by default for new pools 2023-10-16T20:03:58.069 INFO:teuthology.orchestra.run.smithi094.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-16T20:03:58.069 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:58.069 INFO:teuthology.orchestra.run.smithi094.stdout: osd pool default size = 2 2023-10-16T20:03:58.069 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:58.069 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd allow primary affinity = true 2023-10-16T20:03:58.070 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd allow pg remap = true 2023-10-16T20:03:58.070 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on legacy crush tunables = false 2023-10-16T20:03:58.070 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on crush straw calc version zero = false 2023-10-16T20:03:58.070 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on no sortbitwise = false 2023-10-16T20:03:58.070 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on osd down out interval zero = false 2023-10-16T20:03:58.070 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on too few osds = false 2023-10-16T20:03:58.071 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-16T20:03:58.071 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-16T20:03:58.071 INFO:teuthology.orchestra.run.smithi094.stdout: mon_allow_pool_size_one = true 2023-10-16T20:03:58.071 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:58.071 INFO:teuthology.orchestra.run.smithi094.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-16T20:03:58.072 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:58.072 INFO:teuthology.orchestra.run.smithi094.stdout: osd default data pool replay window = 5 2023-10-16T20:03:58.072 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:58.072 INFO:teuthology.orchestra.run.smithi094.stdout: mon allow pool delete = true 2023-10-16T20:03:58.072 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:58.073 INFO:teuthology.orchestra.run.smithi094.stdout: mon cluster log file level = debug 2023-10-16T20:03:58.073 INFO:teuthology.orchestra.run.smithi094.stdout: debug asserts on shutdown = true 2023-10-16T20:03:58.073 INFO:teuthology.orchestra.run.smithi094.stdout: mon health detail to clog = false 2023-10-16T20:03:58.073 INFO:teuthology.orchestra.run.smithi094.stdout: mon host = "172.21.15.88,172.21.15.94,[v2:172.21.15.94:3301,v1:172.21.15.94:6790]" 2023-10-16T20:03:58.073 INFO:teuthology.orchestra.run.smithi094.stdout: fsid = d11187c0-3dd8-4c92-8d13-6ec536230748 2023-10-16T20:03:58.073 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:58.074 INFO:teuthology.orchestra.run.smithi094.stdout:[osd] 2023-10-16T20:03:58.074 INFO:teuthology.orchestra.run.smithi094.stdout: osd journal size = 100 2023-10-16T20:03:58.074 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:58.074 INFO:teuthology.orchestra.run.smithi094.stdout: osd scrub load threshold = 5.0 2023-10-16T20:03:58.074 INFO:teuthology.orchestra.run.smithi094.stdout: osd scrub max interval = 600 2023-10-16T20:03:58.074 INFO:teuthology.orchestra.run.smithi094.stdout: osd mclock profile = high_recovery_ops 2023-10-16T20:03:58.075 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:58.075 INFO:teuthology.orchestra.run.smithi094.stdout: osd recover clone overlap = true 2023-10-16T20:03:58.075 INFO:teuthology.orchestra.run.smithi094.stdout: osd recovery max chunk = 1048576 2023-10-16T20:03:58.075 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:58.075 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug shutdown = true 2023-10-16T20:03:58.075 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug op order = true 2023-10-16T20:03:58.076 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug verify stray on activate = true 2023-10-16T20:03:58.076 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:58.076 INFO:teuthology.orchestra.run.smithi094.stdout: osd open classes on start = true 2023-10-16T20:03:58.076 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug pg log writeout = true 2023-10-16T20:03:58.076 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:58.077 INFO:teuthology.orchestra.run.smithi094.stdout: osd deep scrub update digest min age = 30 2023-10-16T20:03:58.077 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:58.077 INFO:teuthology.orchestra.run.smithi094.stdout: osd map max advance = 10 2023-10-16T20:03:58.077 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:58.077 INFO:teuthology.orchestra.run.smithi094.stdout: journal zero on create = true 2023-10-16T20:03:58.077 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:58.078 INFO:teuthology.orchestra.run.smithi094.stdout: filestore ondisk finisher threads = 3 2023-10-16T20:03:58.078 INFO:teuthology.orchestra.run.smithi094.stdout: filestore apply finisher threads = 3 2023-10-16T20:03:58.078 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:58.078 INFO:teuthology.orchestra.run.smithi094.stdout: bdev debug aio = true 2023-10-16T20:03:58.078 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug misdirected ops = true 2023-10-16T20:03:58.079 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore block size = 96636764160 2023-10-16T20:03:58.079 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore fsck on mount = True 2023-10-16T20:03:58.079 INFO:teuthology.orchestra.run.smithi094.stdout: debug bluefs = 20 2023-10-16T20:03:58.079 INFO:teuthology.orchestra.run.smithi094.stdout: debug bluestore = 20 2023-10-16T20:03:58.079 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2023-10-16T20:03:58.080 INFO:teuthology.orchestra.run.smithi094.stdout: debug osd = 20 2023-10-16T20:03:58.080 INFO:teuthology.orchestra.run.smithi094.stdout: debug rocksdb = 10 2023-10-16T20:03:58.080 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd backfillfull_ratio = 0.85 2023-10-16T20:03:58.080 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd full ratio = 0.9 2023-10-16T20:03:58.080 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd nearfull ratio = 0.8 2023-10-16T20:03:58.080 INFO:teuthology.orchestra.run.smithi094.stdout: osd failsafe full ratio = 0.95 2023-10-16T20:03:58.081 INFO:teuthology.orchestra.run.smithi094.stdout: osd objectstore = bluestore 2023-10-16T20:03:58.081 INFO:teuthology.orchestra.run.smithi094.stdout: osd op complaint time = 180 2023-10-16T20:03:58.081 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:58.081 INFO:teuthology.orchestra.run.smithi094.stdout:[mgr] 2023-10-16T20:03:58.081 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2023-10-16T20:03:58.081 INFO:teuthology.orchestra.run.smithi094.stdout: debug mgr = 20 2023-10-16T20:03:58.082 INFO:teuthology.orchestra.run.smithi094.stdout: debug mon = 20 2023-10-16T20:03:58.082 INFO:teuthology.orchestra.run.smithi094.stdout: debug auth = 20 2023-10-16T20:03:58.082 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min pgs per osd = 4 2023-10-16T20:03:58.082 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min bytes per osd = 10 2023-10-16T20:03:58.082 INFO:teuthology.orchestra.run.smithi094.stdout: mgr/telemetry/nag = false 2023-10-16T20:03:58.083 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:58.083 INFO:teuthology.orchestra.run.smithi094.stdout:[mon] 2023-10-16T20:03:58.083 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2023-10-16T20:03:58.083 INFO:teuthology.orchestra.run.smithi094.stdout: debug mon = 20 2023-10-16T20:03:58.083 INFO:teuthology.orchestra.run.smithi094.stdout: debug paxos = 20 2023-10-16T20:03:58.083 INFO:teuthology.orchestra.run.smithi094.stdout: debug auth = 20 2023-10-16T20:03:58.084 INFO:teuthology.orchestra.run.smithi094.stdout: mon data avail warn = 5 2023-10-16T20:03:58.084 INFO:teuthology.orchestra.run.smithi094.stdout: mon mgr mkfs grace = 240 2023-10-16T20:03:58.084 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min pgs per osd = 4 2023-10-16T20:03:58.084 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd reporter subtree level = osd 2023-10-16T20:03:58.084 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd prime pg temp = true 2023-10-16T20:03:58.084 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min bytes per osd = 10 2023-10-16T20:03:58.085 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:58.085 INFO:teuthology.orchestra.run.smithi094.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-16T20:03:58.085 INFO:teuthology.orchestra.run.smithi094.stdout: auth mon ticket ttl = 660 # 11m 2023-10-16T20:03:58.085 INFO:teuthology.orchestra.run.smithi094.stdout: auth service ticket ttl = 240 # 4m 2023-10-16T20:03:58.085 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:58.085 INFO:teuthology.orchestra.run.smithi094.stdout: # don't complain about insecure global_id in the test suite 2023-10-16T20:03:58.086 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-16T20:03:58.086 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-16T20:03:58.086 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:58.086 INFO:teuthology.orchestra.run.smithi094.stdout: # 1m isn't quite enough 2023-10-16T20:03:58.086 INFO:teuthology.orchestra.run.smithi094.stdout: mon_down_mkfs_grace = 2m 2023-10-16T20:03:58.087 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:58.087 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_filestore_osds = false 2023-10-16T20:03:58.087 INFO:teuthology.orchestra.run.smithi094.stdout: mon op complaint time = 120 2023-10-16T20:03:58.087 INFO:teuthology.orchestra.run.smithi094.stdout: 2023-10-16T20:03:58.088 INFO:teuthology.orchestra.run.smithi094.stdout:[client] 2023-10-16T20:03:58.088 INFO:teuthology.orchestra.run.smithi094.stdout: rgw cache enabled = true 2023-10-16T20:03:58.088 INFO:teuthology.orchestra.run.smithi094.stdout: rgw enable ops log = true 2023-10-16T20:03:58.088 INFO:teuthology.orchestra.run.smithi094.stdout: rgw enable usage log = true 2023-10-16T20:03:58.088 INFO:teuthology.orchestra.run.smithi094.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-16T20:03:58.088 INFO:teuthology.orchestra.run.smithi094.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-16T20:03:58.089 INFO:teuthology.orchestra.run.smithi094.stdout: client mount timeout = 600 2023-10-16T20:03:58.089 INFO:teuthology.orchestra.run.smithi094.stdout: debug client = 20 2023-10-16T20:03:58.089 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2023-10-16T20:03:58.089 INFO:teuthology.orchestra.run.smithi094.stdout: pid file = "" 2023-10-16T20:03:58.089 INFO:teuthology.orchestra.run.smithi094.stdout: rados mon op timeout = 900 2023-10-16T20:03:58.090 INFO:teuthology.orchestra.run.smithi094.stdout: rados osd op timeout = 900 2023-10-16T20:03:58.090 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.a] 2023-10-16T20:03:58.090 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.b] 2023-10-16T20:03:58.090 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.c] 2023-10-16T20:03:58.090 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.a] 2023-10-16T20:03:58.091 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.c] 2023-10-16T20:03:58.091 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.b] 2023-10-16T20:03:58.091 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.d] 2023-10-16T20:03:58.091 INFO:teuthology.orchestra.run.smithi094.stdout:[mds] 2023-10-16T20:03:58.091 INFO:teuthology.orchestra.run.smithi094.stdout: debug mds = 20 2023-10-16T20:03:58.091 INFO:teuthology.orchestra.run.smithi094.stdout: debug mds balancer = 20 2023-10-16T20:03:58.092 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2023-10-16T20:03:58.092 INFO:teuthology.orchestra.run.smithi094.stdout: mds debug frag = True 2023-10-16T20:03:58.092 INFO:teuthology.orchestra.run.smithi094.stdout: mds debug scatterstat = True 2023-10-16T20:03:58.092 INFO:teuthology.orchestra.run.smithi094.stdout: mds op complaint time = 180 2023-10-16T20:03:58.092 INFO:teuthology.orchestra.run.smithi094.stdout: mds verify scatter = True 2023-10-16T20:03:58.093 INFO:teuthology.orchestra.run.smithi094.stdout: osd op complaint time = 180 2023-10-16T20:03:58.093 INFO:teuthology.orchestra.run.smithi094.stdout: rados mon op timeout = 900 2023-10-16T20:03:58.093 INFO:teuthology.orchestra.run.smithi094.stdout: rados osd op timeout = 900 2023-10-16T20:03:58.093 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-16T20:03:58.094 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-16T20:03:58.165 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-16T20:03:58.166 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:03:58.166 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-16T20:03:58.175 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:03:58.175 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-16T20:03:58.224 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi088.front.sepia.ceph.com 2023-10-16T20:03:58.224 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:03:58.224 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-16T20:03:58.225 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-16T20:03:58.304 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:03:58.304 DEBUG:teuthology.orchestra.run.smithi088:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-16T20:03:58.360 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi094.front.sepia.ceph.com 2023-10-16T20:03:58.360 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:03:58.360 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-16T20:03:58.361 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-16T20:03:58.397 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:03:58.397 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-16T20:03:58.449 INFO:tasks.ceph:Setting up mon nodes... 2023-10-16T20:03:58.450 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-16T20:03:58.450 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-16T20:03:58.505 INFO:teuthology.orchestra.run.smithi088.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-16T20:03:58.511 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-16T20:03:58.571 INFO:teuthology.orchestra.run.smithi094.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-16T20:03:58.590 INFO:tasks.ceph:Setting up mds nodes... 2023-10-16T20:03:58.590 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-16T20:03:58.646 INFO:teuthology.orchestra.run.smithi088.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-16T20:03:58.650 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-16T20:03:58.671 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-16T20:03:58.766 INFO:teuthology.orchestra.run.smithi088.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-16T20:03:58.770 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-16T20:03:58.791 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-16T20:03:58.849 INFO:teuthology.orchestra.run.smithi094.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-16T20:03:58.854 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-16T20:03:58.875 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-16T20:03:58.972 INFO:teuthology.orchestra.run.smithi094.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-16T20:03:58.977 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-16T20:03:58.997 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-16T20:03:58.997 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-16T20:03:59.040 INFO:teuthology.orchestra.run.smithi088.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-16T20:03:59.056 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-16T20:03:59.145 INFO:teuthology.orchestra.run.smithi088.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-16T20:03:59.162 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-16T20:03:59.206 INFO:teuthology.orchestra.run.smithi094.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-16T20:03:59.223 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-16T20:03:59.307 INFO:teuthology.orchestra.run.smithi094.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-16T20:03:59.324 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-16T20:03:59.324 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi088.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi094.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-16T20:03:59.324 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-16T20:03:59.344 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-16T20:03:59.345 INFO:tasks.ceph:role: osd.0 2023-10-16T20:03:59.345 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi088.front.sepia.ceph.com 2023-10-16T20:03:59.345 DEBUG:teuthology.orchestra.run.smithi088:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-16T20:03:59.414 INFO:teuthology.orchestra.run.smithi088.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T20:03:59.414 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T20:03:59.415 INFO:teuthology.orchestra.run.smithi088.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T20:03:59.415 INFO:teuthology.orchestra.run.smithi088.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-16T20:03:59.415 INFO:teuthology.orchestra.run.smithi088.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T20:03:59.415 INFO:teuthology.orchestra.run.smithi088.stdout: = sunit=0 swidth=0 blks 2023-10-16T20:03:59.415 INFO:teuthology.orchestra.run.smithi088.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T20:03:59.416 INFO:teuthology.orchestra.run.smithi088.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-16T20:03:59.416 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T20:03:59.416 INFO:teuthology.orchestra.run.smithi088.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T20:03:59.883 INFO:teuthology.orchestra.run.smithi088.stdout:Discarding blocks...Done. 2023-10-16T20:03:59.886 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi088.front.sepia.ceph.com -o noatime 2023-10-16T20:03:59.887 DEBUG:teuthology.orchestra.run.smithi088:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-16T20:04:00.033 DEBUG:teuthology.orchestra.run.smithi088:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-16T20:04:00.050 INFO:teuthology.orchestra.run.smithi088.stderr:sudo: /sbin/restorecon: command not found 2023-10-16T20:04:00.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T20:04:00.051 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-16T20:04:00.115 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-16T20:04:00.115 INFO:tasks.ceph:role: osd.1 2023-10-16T20:04:00.116 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi088.front.sepia.ceph.com 2023-10-16T20:04:00.116 DEBUG:teuthology.orchestra.run.smithi088:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-16T20:04:00.187 INFO:teuthology.orchestra.run.smithi088.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T20:04:00.188 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T20:04:00.188 INFO:teuthology.orchestra.run.smithi088.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T20:04:00.188 INFO:teuthology.orchestra.run.smithi088.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-16T20:04:00.188 INFO:teuthology.orchestra.run.smithi088.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T20:04:00.189 INFO:teuthology.orchestra.run.smithi088.stdout: = sunit=0 swidth=0 blks 2023-10-16T20:04:00.189 INFO:teuthology.orchestra.run.smithi088.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T20:04:00.189 INFO:teuthology.orchestra.run.smithi088.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-16T20:04:00.189 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T20:04:00.189 INFO:teuthology.orchestra.run.smithi088.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T20:04:00.625 INFO:teuthology.orchestra.run.smithi088.stdout:Discarding blocks...Done. 2023-10-16T20:04:00.628 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi088.front.sepia.ceph.com -o noatime 2023-10-16T20:04:00.629 DEBUG:teuthology.orchestra.run.smithi088:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-16T20:04:00.723 DEBUG:teuthology.orchestra.run.smithi088:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-16T20:04:00.739 INFO:teuthology.orchestra.run.smithi088.stderr:sudo: /sbin/restorecon: command not found 2023-10-16T20:04:00.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T20:04:00.741 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-16T20:04:00.803 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-16T20:04:00.803 INFO:tasks.ceph:role: osd.2 2023-10-16T20:04:00.804 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi088.front.sepia.ceph.com 2023-10-16T20:04:00.804 DEBUG:teuthology.orchestra.run.smithi088:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-16T20:04:00.872 INFO:teuthology.orchestra.run.smithi088.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T20:04:00.872 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T20:04:00.872 INFO:teuthology.orchestra.run.smithi088.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T20:04:00.873 INFO:teuthology.orchestra.run.smithi088.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-16T20:04:00.873 INFO:teuthology.orchestra.run.smithi088.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T20:04:00.873 INFO:teuthology.orchestra.run.smithi088.stdout: = sunit=0 swidth=0 blks 2023-10-16T20:04:00.873 INFO:teuthology.orchestra.run.smithi088.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T20:04:00.873 INFO:teuthology.orchestra.run.smithi088.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-16T20:04:00.873 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T20:04:00.874 INFO:teuthology.orchestra.run.smithi088.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T20:04:01.338 INFO:teuthology.orchestra.run.smithi088.stdout:Discarding blocks...Done. 2023-10-16T20:04:01.341 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi088.front.sepia.ceph.com -o noatime 2023-10-16T20:04:01.342 DEBUG:teuthology.orchestra.run.smithi088:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-16T20:04:01.485 DEBUG:teuthology.orchestra.run.smithi088:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-16T20:04:01.502 INFO:teuthology.orchestra.run.smithi088.stderr:sudo: /sbin/restorecon: command not found 2023-10-16T20:04:01.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T20:04:01.503 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-16T20:04:01.568 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-16T20:04:01.660 INFO:tasks.ceph:role: osd.3 2023-10-16T20:04:01.661 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi088.front.sepia.ceph.com 2023-10-16T20:04:01.661 DEBUG:teuthology.orchestra.run.smithi088:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-16T20:04:01.691 INFO:teuthology.orchestra.run.smithi088.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T20:04:01.691 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T20:04:01.691 INFO:teuthology.orchestra.run.smithi088.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T20:04:01.692 INFO:teuthology.orchestra.run.smithi088.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-16T20:04:01.692 INFO:teuthology.orchestra.run.smithi088.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T20:04:01.692 INFO:teuthology.orchestra.run.smithi088.stdout: = sunit=0 swidth=0 blks 2023-10-16T20:04:01.692 INFO:teuthology.orchestra.run.smithi088.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T20:04:01.692 INFO:teuthology.orchestra.run.smithi088.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-16T20:04:01.693 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T20:04:01.693 INFO:teuthology.orchestra.run.smithi088.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T20:04:02.167 INFO:teuthology.orchestra.run.smithi088.stdout:Discarding blocks...Done. 2023-10-16T20:04:02.170 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi088.front.sepia.ceph.com -o noatime 2023-10-16T20:04:02.171 DEBUG:teuthology.orchestra.run.smithi088:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-16T20:04:02.300 DEBUG:teuthology.orchestra.run.smithi088:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-16T20:04:02.318 INFO:teuthology.orchestra.run.smithi088.stderr:sudo: /sbin/restorecon: command not found 2023-10-16T20:04:02.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T20:04:02.320 DEBUG:teuthology.orchestra.run.smithi088:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T20:04:02.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:02.445+0000 7fc950f88640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-16T20:04:02.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:02.449+0000 7fc950f88640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-16T20:04:02.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:02.449+0000 7fc950f88640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-16T20:04:02.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:02.449+0000 7fc950f88640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-16T20:04:02.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:02.449+0000 7fc950f88640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-16T20:04:02.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:02.449+0000 7fc950f88640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-16T20:04:05.431 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-16T20:04:05.453 DEBUG:teuthology.orchestra.run.smithi088:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T20:04:05.553 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:05.553+0000 7f7f4ed20640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-16T20:04:05.554 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:05.553+0000 7f7f4ed20640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-16T20:04:05.554 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:05.553+0000 7f7f4ed20640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-16T20:04:05.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:05.553+0000 7f7f4ed20640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-16T20:04:05.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:05.553+0000 7f7f4ed20640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-16T20:04:05.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:05.553+0000 7f7f4ed20640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-16T20:04:08.551 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-16T20:04:08.572 DEBUG:teuthology.orchestra.run.smithi088:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T20:04:08.674 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:08.673+0000 7f64a4738640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-16T20:04:08.675 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:08.673+0000 7f64a4738640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-16T20:04:08.675 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:08.673+0000 7f64a4738640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-16T20:04:08.675 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:08.673+0000 7f64a4738640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-16T20:04:08.675 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:08.673+0000 7f64a4738640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-16T20:04:08.676 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:08.677+0000 7f64a4738640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-16T20:04:11.662 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-16T20:04:11.683 DEBUG:teuthology.orchestra.run.smithi088:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T20:04:11.781 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:11.781+0000 7f952b893640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-16T20:04:11.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:11.781+0000 7f952b893640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-16T20:04:11.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:11.781+0000 7f952b893640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-16T20:04:11.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:11.781+0000 7f952b893640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-16T20:04:11.783 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:11.781+0000 7f952b893640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-16T20:04:11.783 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:11.781+0000 7f952b893640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-16T20:04:14.771 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-16T20:04:14.792 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-16T20:04:14.815 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-16T20:04:14.815 INFO:tasks.ceph:role: osd.4 2023-10-16T20:04:14.815 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi094.front.sepia.ceph.com 2023-10-16T20:04:14.815 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-16T20:04:14.879 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T20:04:14.879 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T20:04:14.879 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T20:04:14.879 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-16T20:04:14.879 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T20:04:14.880 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2023-10-16T20:04:14.880 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T20:04:14.880 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-16T20:04:14.880 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T20:04:14.880 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T20:04:14.903 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2023-10-16T20:04:14.906 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2023-10-16T20:04:14.906 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-16T20:04:14.979 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-16T20:04:15.037 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: /sbin/restorecon: command not found 2023-10-16T20:04:15.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T20:04:15.039 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-16T20:04:15.101 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-16T20:04:15.101 INFO:tasks.ceph:role: osd.5 2023-10-16T20:04:15.102 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi094.front.sepia.ceph.com 2023-10-16T20:04:15.103 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-16T20:04:15.165 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T20:04:15.166 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T20:04:15.166 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T20:04:15.166 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-16T20:04:15.166 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T20:04:15.166 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2023-10-16T20:04:15.167 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T20:04:15.167 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-16T20:04:15.167 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T20:04:15.167 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T20:04:15.189 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2023-10-16T20:04:15.193 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2023-10-16T20:04:15.193 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-16T20:04:15.262 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-16T20:04:15.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T20:04:15.322 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: /sbin/restorecon: command not found 2023-10-16T20:04:15.322 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-16T20:04:15.384 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-16T20:04:15.384 INFO:tasks.ceph:role: osd.6 2023-10-16T20:04:15.384 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi094.front.sepia.ceph.com 2023-10-16T20:04:15.385 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-16T20:04:15.443 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T20:04:15.444 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T20:04:15.444 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T20:04:15.444 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-16T20:04:15.444 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T20:04:15.444 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2023-10-16T20:04:15.445 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T20:04:15.445 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-16T20:04:15.445 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T20:04:15.445 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T20:04:15.467 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2023-10-16T20:04:15.470 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2023-10-16T20:04:15.470 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-16T20:04:15.537 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-16T20:04:15.596 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: /sbin/restorecon: command not found 2023-10-16T20:04:15.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T20:04:15.598 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-16T20:04:15.663 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-16T20:04:15.664 INFO:tasks.ceph:role: osd.7 2023-10-16T20:04:15.664 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi094.front.sepia.ceph.com 2023-10-16T20:04:15.664 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-16T20:04:15.729 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T20:04:15.729 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T20:04:15.730 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T20:04:15.730 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-16T20:04:15.730 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T20:04:15.730 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2023-10-16T20:04:15.730 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T20:04:15.731 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-16T20:04:15.731 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T20:04:15.731 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T20:04:15.752 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2023-10-16T20:04:15.754 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2023-10-16T20:04:15.754 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-16T20:04:15.826 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-16T20:04:15.885 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: /sbin/restorecon: command not found 2023-10-16T20:04:15.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T20:04:15.886 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T20:04:16.014 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:16.010+0000 7fe726fb6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-16T20:04:16.015 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:16.010+0000 7fe726fb6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-16T20:04:16.015 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:16.010+0000 7fe726fb6640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-16T20:04:16.015 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:16.010+0000 7fe726fb6640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-16T20:04:16.015 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:16.010+0000 7fe726fb6640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-16T20:04:16.016 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:16.010+0000 7fe726fb6640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-16T20:04:19.014 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-16T20:04:19.036 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T20:04:19.132 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:19.126+0000 7f2c2bad0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-16T20:04:19.132 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:19.126+0000 7f2c2bad0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-16T20:04:19.132 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:19.126+0000 7f2c2bad0640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-16T20:04:19.133 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:19.126+0000 7f2c2bad0640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-16T20:04:19.133 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:19.126+0000 7f2c2bad0640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-16T20:04:19.133 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:19.130+0000 7f2c2bad0640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-16T20:04:22.120 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-16T20:04:22.142 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T20:04:22.241 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:22.238+0000 7fb982aba640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-16T20:04:22.242 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:22.238+0000 7fb982aba640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-16T20:04:22.242 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:22.238+0000 7fb982aba640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-16T20:04:22.243 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:22.238+0000 7fb982aba640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-16T20:04:22.243 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:22.238+0000 7fb982aba640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-16T20:04:22.243 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:22.238+0000 7fb982aba640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-16T20:04:25.227 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-16T20:04:25.248 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T20:04:25.345 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:25.342+0000 7ff688c95640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-16T20:04:25.346 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:25.342+0000 7ff688c95640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-16T20:04:25.346 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:25.342+0000 7ff688c95640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-16T20:04:25.347 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:25.342+0000 7ff688c95640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-16T20:04:25.347 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:25.342+0000 7ff688c95640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-16T20:04:25.347 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:25.342+0000 7ff688c95640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-16T20:04:26.063 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T20:04:26.068 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T20:04:28.320 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-16T20:04:28.341 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-16T20:04:28.341 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:04:28.342 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-16T20:04:28.364 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:04:28.364 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-16T20:04:28.433 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:04:28.433 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-16T20:04:28.497 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:04:28.497 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-16T20:04:28.561 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:04:28.561 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-16T20:04:28.624 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:04:28.625 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-16T20:04:28.689 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:04:28.689 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-16T20:04:28.752 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:04:28.753 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-16T20:04:28.774 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:04:28.774 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-16T20:04:28.838 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:04:28.838 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-16T20:04:28.901 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:04:28.902 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-16T20:04:28.965 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:04:28.966 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-16T20:04:29.033 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:04:29.034 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-16T20:04:29.098 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:04:29.098 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-16T20:04:29.162 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:04:29.162 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-16T20:04:29.172 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:04:29.172 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-16T20:04:29.224 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:04:29.224 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-16T20:04:29.233 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:04:29.233 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-16T20:04:29.285 INFO:tasks.ceph:Adding keys to all mons... 2023-10-16T20:04:29.286 DEBUG:teuthology.orchestra.run.smithi088:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-16T20:04:29.290 DEBUG:teuthology.orchestra.run.smithi094:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-16T20:04:29.333 INFO:teuthology.orchestra.run.smithi088.stdout:[mgr.y] 2023-10-16T20:04:29.333 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQAuly1lDn1VHhAApRTbdVW1sInFHR4ftXIgLA== 2023-10-16T20:04:29.333 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.a] 2023-10-16T20:04:29.333 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQAuly1lCHa1JhAAhNsz0rC3KpOhywCHwmjWyg== 2023-10-16T20:04:29.334 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.c] 2023-10-16T20:04:29.334 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQAuly1ltUzfLRAAJWxUKQWdDYVtMTHHYipcZQ== 2023-10-16T20:04:29.334 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.0] 2023-10-16T20:04:29.334 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQAyly1l1tzTGhAANOg63+nIoOxhLwHYO2iqQA== 2023-10-16T20:04:29.334 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.1] 2023-10-16T20:04:29.335 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQA1ly1lgigrIRAA0Mzb88vvzSzGqEzZVsRUtQ== 2023-10-16T20:04:29.335 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.2] 2023-10-16T20:04:29.335 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQA4ly1lN0lgKBAAALodK7N7sDf5GdZrccritg== 2023-10-16T20:04:29.335 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.3] 2023-10-16T20:04:29.335 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQA7ly1lrVjALhAAOc9pmNFdvOZ1gT6VMXOgzA== 2023-10-16T20:04:29.336 INFO:teuthology.orchestra.run.smithi088.stdout:[mgr.x] 2023-10-16T20:04:29.336 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQAuly1l9hztIRAAGJWqLd1Aa6sZAzEHc+mXZQ== 2023-10-16T20:04:29.336 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.b] 2023-10-16T20:04:29.336 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQAuly1lxz58MhAA69WwY8VLqgyX4KQ9pGS5QQ== 2023-10-16T20:04:29.336 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.d] 2023-10-16T20:04:29.337 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQAuly1l+TXWORAAJv6W5uoM9XLv1keyH1iJGg== 2023-10-16T20:04:29.337 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.4] 2023-10-16T20:04:29.337 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQBAly1lVlqzABAAIg5Q4X0rytK7dTFXzhulLw== 2023-10-16T20:04:29.337 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.5] 2023-10-16T20:04:29.337 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQBDly1l9gS2BxAA+VSkv60PdAkqsolXnZSLUg== 2023-10-16T20:04:29.338 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.6] 2023-10-16T20:04:29.338 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQBGly1lADNDDhAA/1Wahdkwdt708rxp8XnTRg== 2023-10-16T20:04:29.338 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.7] 2023-10-16T20:04:29.338 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQBJly1lwLN1FBAABUh5iYOJQHAcceNeGuxQKA== 2023-10-16T20:04:29.338 INFO:teuthology.orchestra.run.smithi088.stdout:[client.0] 2023-10-16T20:04:29.339 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQAvly1looeRAhAAeLshHOVIhQqoTtlPehorIg== 2023-10-16T20:04:29.339 INFO:teuthology.orchestra.run.smithi088.stdout:[client.2] 2023-10-16T20:04:29.339 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQAvly1lhdfcCBAAJU/W9X3cX1ukGJOXsW3U7A== 2023-10-16T20:04:29.339 INFO:teuthology.orchestra.run.smithi088.stdout:[client.1] 2023-10-16T20:04:29.339 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQAvly1lvy8qDBAAc789x2xSuuM7SHbL5XbRwg== 2023-10-16T20:04:29.339 INFO:teuthology.orchestra.run.smithi088.stdout:[client.3] 2023-10-16T20:04:29.340 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQAvly1lqPsnEhAAzgEkt/iWPI1jIPCKw0rCUg== 2023-10-16T20:04:29.348 INFO:teuthology.orchestra.run.smithi094.stdout:[mgr.y] 2023-10-16T20:04:29.348 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQAuly1lDn1VHhAApRTbdVW1sInFHR4ftXIgLA== 2023-10-16T20:04:29.348 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.a] 2023-10-16T20:04:29.348 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQAuly1lCHa1JhAAhNsz0rC3KpOhywCHwmjWyg== 2023-10-16T20:04:29.349 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.c] 2023-10-16T20:04:29.349 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQAuly1ltUzfLRAAJWxUKQWdDYVtMTHHYipcZQ== 2023-10-16T20:04:29.349 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.0] 2023-10-16T20:04:29.349 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQAyly1l1tzTGhAANOg63+nIoOxhLwHYO2iqQA== 2023-10-16T20:04:29.349 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.1] 2023-10-16T20:04:29.349 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQA1ly1lgigrIRAA0Mzb88vvzSzGqEzZVsRUtQ== 2023-10-16T20:04:29.350 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.2] 2023-10-16T20:04:29.350 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQA4ly1lN0lgKBAAALodK7N7sDf5GdZrccritg== 2023-10-16T20:04:29.350 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.3] 2023-10-16T20:04:29.350 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQA7ly1lrVjALhAAOc9pmNFdvOZ1gT6VMXOgzA== 2023-10-16T20:04:29.350 INFO:teuthology.orchestra.run.smithi094.stdout:[mgr.x] 2023-10-16T20:04:29.351 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQAuly1l9hztIRAAGJWqLd1Aa6sZAzEHc+mXZQ== 2023-10-16T20:04:29.351 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.b] 2023-10-16T20:04:29.351 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQAuly1lxz58MhAA69WwY8VLqgyX4KQ9pGS5QQ== 2023-10-16T20:04:29.351 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.d] 2023-10-16T20:04:29.351 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQAuly1l+TXWORAAJv6W5uoM9XLv1keyH1iJGg== 2023-10-16T20:04:29.351 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.4] 2023-10-16T20:04:29.352 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBAly1lVlqzABAAIg5Q4X0rytK7dTFXzhulLw== 2023-10-16T20:04:29.352 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.5] 2023-10-16T20:04:29.352 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBDly1l9gS2BxAA+VSkv60PdAkqsolXnZSLUg== 2023-10-16T20:04:29.352 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.6] 2023-10-16T20:04:29.352 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBGly1lADNDDhAA/1Wahdkwdt708rxp8XnTRg== 2023-10-16T20:04:29.352 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.7] 2023-10-16T20:04:29.353 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBJly1lwLN1FBAABUh5iYOJQHAcceNeGuxQKA== 2023-10-16T20:04:29.353 INFO:teuthology.orchestra.run.smithi094.stdout:[client.0] 2023-10-16T20:04:29.353 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQAvly1looeRAhAAeLshHOVIhQqoTtlPehorIg== 2023-10-16T20:04:29.353 INFO:teuthology.orchestra.run.smithi094.stdout:[client.2] 2023-10-16T20:04:29.353 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQAvly1lhdfcCBAAJU/W9X3cX1ukGJOXsW3U7A== 2023-10-16T20:04:29.353 INFO:teuthology.orchestra.run.smithi094.stdout:[client.1] 2023-10-16T20:04:29.354 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQAvly1lvy8qDBAAc789x2xSuuM7SHbL5XbRwg== 2023-10-16T20:04:29.354 INFO:teuthology.orchestra.run.smithi094.stdout:[client.3] 2023-10-16T20:04:29.354 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQAvly1lqPsnEhAAzgEkt/iWPI1jIPCKw0rCUg== 2023-10-16T20:04:29.355 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-16T20:04:29.386 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-16T20:04:29.456 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T20:04:29.460 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T20:04:29.518 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T20:04:29.523 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T20:04:29.580 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T20:04:29.584 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T20:04:29.638 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T20:04:29.642 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T20:04:29.697 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T20:04:29.701 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T20:04:29.754 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T20:04:29.758 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T20:04:29.812 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-16T20:04:29.816 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-16T20:04:29.869 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T20:04:29.873 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T20:04:29.929 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T20:04:29.933 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T20:04:29.990 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T20:04:29.994 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T20:04:30.049 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T20:04:30.053 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T20:04:30.107 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T20:04:30.110 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T20:04:30.164 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T20:04:30.168 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T20:04:30.223 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T20:04:30.226 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T20:04:30.284 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T20:04:30.287 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T20:04:30.345 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T20:04:30.349 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T20:04:30.406 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T20:04:30.409 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T20:04:30.467 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-16T20:04:30.467 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-16T20:04:30.488 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-16T20:04:30.604 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-16T20:04:30.625 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-16T20:04:30.645 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-16T20:04:30.763 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-16T20:04:30.783 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-16T20:04:30.844 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-16T20:04:30.956 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-16T20:04:30.976 DEBUG:teuthology.orchestra.run.smithi088:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-16T20:04:30.980 DEBUG:teuthology.orchestra.run.smithi094:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-16T20:04:31.029 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-16T20:04:31.029 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-16T20:04:31.030 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-16T20:04:31.033 INFO:tasks.ceph.mon.a:Started 2023-10-16T20:04:31.033 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-16T20:04:31.034 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-16T20:04:31.072 INFO:tasks.ceph.mon.b:Started 2023-10-16T20:04:31.072 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-16T20:04:31.072 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-16T20:04:31.076 INFO:tasks.ceph.mon.c:Started 2023-10-16T20:04:31.076 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-16T20:04:31.077 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-16T20:04:31.077 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-16T20:04:31.081 INFO:tasks.ceph.mgr.y:Started 2023-10-16T20:04:31.082 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-16T20:04:31.082 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-16T20:04:31.086 INFO:tasks.ceph.mgr.x:Started 2023-10-16T20:04:31.086 INFO:tasks.ceph:Setting crush tunables to default 2023-10-16T20:04:31.086 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph osd crush tunables default 2023-10-16T20:04:31.229 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.228+0000 7fd0b8646640 1 Processor -- start 2023-10-16T20:04:31.230 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.228+0000 7fd0b8646640 1 -- start start 2023-10-16T20:04:31.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.228+0000 7fd0b8646640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd0b015cb70 0x7fd0b00b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:31.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.228+0000 7fd0b8646640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0b00b7d20 con 0x7fd0b00b6a80 2023-10-16T20:04:31.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.228+0000 7fd0b8646640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0b00b6eb0 con 0x7fd0b00b4530 2023-10-16T20:04:31.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.228+0000 7fd0b8646640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0b00b7110 con 0x7fd0b015cb70 2023-10-16T20:04:31.232 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.228+0000 7fd0b63e2640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd0b015cb70 0x7fd0b00b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:31.232 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.228+0000 7fd0b63e2640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd0b015cb70 0x7fd0b00b83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:55298/0 (socket says 172.21.15.88:55298) 2023-10-16T20:04:31.232 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.228+0000 7fd0b63e2640 1 -- 172.21.15.88:0/1759979667 learned_addr learned my addr 172.21.15.88:0/1759979667 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:04:31.232 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.228+0000 7fd0b63e2640 1 -- 172.21.15.88:0/1759979667 >> v2:172.21.15.94:3300/0 conn(0x7fd0b015cb70 msgr2=0x7fd0b00b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-16T20:04:31.233 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.228+0000 7fd0b63e2640 1 -- 172.21.15.88:0/1759979667 >> v2:172.21.15.94:3300/0 conn(0x7fd0b015cb70 msgr2=0x7fd0b00b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-16T20:04:31.233 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.228+0000 7fd0b63e2640 1 --2- 172.21.15.88:0/1759979667 >> v2:172.21.15.94:3300/0 conn(0x7fd0b015cb70 0x7fd0b00b83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-16T20:04:31.233 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.228+0000 7fd0b63e2640 1 --2- 172.21.15.88:0/1759979667 >> v2:172.21.15.94:3300/0 conn(0x7fd0b015cb70 0x7fd0b00b83d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T20:04:31.394 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.392+0000 7fd0b53e0640 1 -- 172.21.15.88:0/1759979667 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3268710278 0 0) 0x7fd0b00b7d20 con 0x7fd0b00b6a80 2023-10-16T20:04:31.394 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.392+0000 7fd0b53e0640 1 -- 172.21.15.88:0/1759979667 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd09c003530 con 0x7fd0b00b6a80 2023-10-16T20:04:31.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.392+0000 7fd0b53e0640 1 -- 172.21.15.88:0/1759979667 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1715707630 0 0) 0x7fd0b00b6eb0 con 0x7fd0b00b4530 2023-10-16T20:04:31.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.392+0000 7fd0b53e0640 1 -- 172.21.15.88:0/1759979667 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0b00b7d20 con 0x7fd0b00b4530 2023-10-16T20:04:31.396 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.392+0000 7fd0b53e0640 1 -- 172.21.15.88:0/1759979667 <== mon.1 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 610243734 0 0) 0x7fd0b00b7d20 con 0x7fd0b00b4530 2023-10-16T20:04:31.396 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.392+0000 7fd0b53e0640 1 -- 172.21.15.88:0/1759979667 >> v2:172.21.15.94:3300/0 conn(0x7fd0b015cb70 msgr2=0x7fd0b00b83d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T20:04:31.396 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.392+0000 7fd0b53e0640 1 --2- 172.21.15.88:0/1759979667 >> v2:172.21.15.94:3300/0 conn(0x7fd0b015cb70 0x7fd0b00b83d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:31.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.392+0000 7fd0b53e0640 1 -- 172.21.15.88:0/1759979667 >> v1:172.21.15.88:6789/0 conn(0x7fd0b00b6a80 legacy=0x7fd0b00d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:31.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.392+0000 7fd0b53e0640 1 -- 172.21.15.88:0/1759979667 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b00b73f0 con 0x7fd0b00b4530 2023-10-16T20:04:31.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.392+0000 7fd0b53e0640 1 -- 172.21.15.88:0/1759979667 <== mon.1 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7fd0a0001680 con 0x7fd0b00b4530 2023-10-16T20:04:31.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.392+0000 7fd0b53e0640 1 -- 172.21.15.88:0/1759979667 >> v1:172.21.15.94:6789/0 conn(0x7fd0b00b4530 legacy=0x7fd0b00d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:31.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.392+0000 7fd0b53e0640 1 --2- 172.21.15.88:0/1759979667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd09c003b10 0x7fd09c003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:31.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.392+0000 7fd0b53e0640 1 --2- 172.21.15.88:0/1759979667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd09c0047d0 0x7fd09c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:31.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.392+0000 7fd0b53e0640 1 --2- 172.21.15.88:0/1759979667 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd09c005190 0x7fd09c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:31.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.392+0000 7fd0b53e0640 1 -- 172.21.15.88:0/1759979667 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b00b7110 con 0x7fd09c003b10 2023-10-16T20:04:31.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.392+0000 7fd0b53e0640 1 -- 172.21.15.88:0/1759979667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c00c010 con 0x7fd09c0047d0 2023-10-16T20:04:31.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.392+0000 7fd0b53e0640 1 -- 172.21.15.88:0/1759979667 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c00c1d0 con 0x7fd09c005190 2023-10-16T20:04:31.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.392+0000 7fd0b63e2640 1 --2- 172.21.15.88:0/1759979667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd09c003b10 0x7fd09c003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:31.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.392+0000 7fd0b5be1640 1 --2- 172.21.15.88:0/1759979667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd09c0047d0 0x7fd09c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:31.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.392+0000 7fd0b6be3640 1 --2- 172.21.15.88:0/1759979667 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd09c005190 0x7fd09c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:31.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.392+0000 7fd0b63e2640 1 -- 172.21.15.88:0/1759979667 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd09c005190 msgr2=0x7fd09c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:04:31.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.392+0000 7fd0b63e2640 1 --2- 172.21.15.88:0/1759979667 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd09c005190 0x7fd09c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:31.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.392+0000 7fd0b63e2640 1 -- 172.21.15.88:0/1759979667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd09c0047d0 msgr2=0x7fd09c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:04:31.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.392+0000 7fd0b63e2640 1 --2- 172.21.15.88:0/1759979667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd09c0047d0 0x7fd09c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:31.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.392+0000 7fd0b63e2640 1 -- 172.21.15.88:0/1759979667 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b00b7d20 con 0x7fd09c003b10 2023-10-16T20:04:31.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.392+0000 7fd0b6be3640 1 -- 172.21.15.88:0/1759979667 reap_dead start 2023-10-16T20:04:31.401 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.400+0000 7fd0b63e2640 1 --2- 172.21.15.88:0/1759979667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd09c003b10 0x7fd09c003f10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac009890 tx=0x7fd0ac01c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6322bc01ee40cadd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:31.401 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.400+0000 7fd0b53e0640 1 -- 172.21.15.88:0/1759979667 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac0a8660 con 0x7fd09c003b10 2023-10-16T20:04:31.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.400+0000 7fd0b53e0640 1 -- 172.21.15.88:0/1759979667 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0ac0a87c0 con 0x7fd09c003b10 2023-10-16T20:04:31.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.400+0000 7fd0b53e0640 1 -- 172.21.15.88:0/1759979667 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac0a8b70 con 0x7fd09c003b10 2023-10-16T20:04:31.403 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.400+0000 7fd0b8646640 1 -- 172.21.15.88:0/1759979667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd09c003b10 msgr2=0x7fd09c003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:31.403 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.400+0000 7fd0b8646640 1 --2- 172.21.15.88:0/1759979667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd09c003b10 0x7fd09c003f10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac009890 tx=0x7fd0ac01c5f0 comp rx=0 tx=0).stop 2023-10-16T20:04:31.403 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.400+0000 7fd0b8646640 1 -- 172.21.15.88:0/1759979667 shutdown_connections 2023-10-16T20:04:31.403 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.400+0000 7fd0b8646640 1 --2- 172.21.15.88:0/1759979667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd09c003b10 0x7fd09c003f10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:31.403 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.400+0000 7fd0b8646640 1 -- 172.21.15.88:0/1759979667 >> 172.21.15.88:0/1759979667 conn(0x7fd0b006b6e0 msgr2=0x7fd0b006baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:31.404 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.400+0000 7fd0b8646640 1 -- 172.21.15.88:0/1759979667 shutdown_connections 2023-10-16T20:04:31.404 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.400+0000 7fd0b8646640 1 -- 172.21.15.88:0/1759979667 wait complete. 2023-10-16T20:04:31.404 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.404+0000 7fd0b8646640 1 Processor -- start 2023-10-16T20:04:31.405 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.404+0000 7fd0b8646640 1 -- start start 2023-10-16T20:04:31.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.404+0000 7fd0b8646640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd09c003b10 0x7fd0b0150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:31.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.404+0000 7fd0b8646640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0b015e3a0 0x7fd0b01f6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:31.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.404+0000 7fd0b8646640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0b01f2460 0x7fd0b0155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:31.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.404+0000 7fd0b8646640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b01f3370 con 0x7fd0b01f2460 2023-10-16T20:04:31.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.404+0000 7fd0b8646640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b01f5ef0 con 0x7fd09c003b10 2023-10-16T20:04:31.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.404+0000 7fd0b8646640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b0151430 con 0x7fd0b015e3a0 2023-10-16T20:04:31.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.404+0000 7fd0b6be3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0b01f2460 0x7fd0b0155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:31.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.404+0000 7fd0b63e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd09c003b10 0x7fd0b0150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:31.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.404+0000 7fd0b63e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd09c003b10 0x7fd0b0150ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:55352/0 (socket says 172.21.15.88:55352) 2023-10-16T20:04:31.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.404+0000 7fd0b63e2640 1 -- 172.21.15.88:0/1684033211 learned_addr learned my addr 172.21.15.88:0/1684033211 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:04:31.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.404+0000 7fd0b6be3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0b01f2460 0x7fd0b0155c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35698/0 (socket says 172.21.15.88:35698) 2023-10-16T20:04:31.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.404+0000 7fd0b5be1640 1 --2- 172.21.15.88:0/1684033211 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0b015e3a0 0x7fd0b01f6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:31.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.404+0000 7fd0b6be3640 1 -- 172.21.15.88:0/1684033211 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0b015e3a0 msgr2=0x7fd0b01f6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:31.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.404+0000 7fd0b6be3640 1 --2- 172.21.15.88:0/1684033211 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0b015e3a0 0x7fd0b01f6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:31.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.404+0000 7fd0b6be3640 1 -- 172.21.15.88:0/1684033211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd09c003b10 msgr2=0x7fd0b0150ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:31.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.404+0000 7fd0b6be3640 1 --2- 172.21.15.88:0/1684033211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd09c003b10 0x7fd0b0150ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:31.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.404+0000 7fd0b6be3640 1 -- 172.21.15.88:0/1684033211 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b01f2db0 con 0x7fd0b01f2460 2023-10-16T20:04:31.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.404+0000 7fd0b6be3640 1 --2- 172.21.15.88:0/1684033211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0b01f2460 0x7fd0b0155c90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd0b008bb40 tx=0x7fd0a800ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:31.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.404+0000 7fd0a6ffd640 1 -- 172.21.15.88:0/1684033211 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8003b90 con 0x7fd0b01f2460 2023-10-16T20:04:31.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.404+0000 7fd0a6ffd640 1 -- 172.21.15.88:0/1684033211 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a8003cf0 con 0x7fd0b01f2460 2023-10-16T20:04:31.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.404+0000 7fd0b8646640 1 -- 172.21.15.88:0/1684033211 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0b01f3090 con 0x7fd0b01f2460 2023-10-16T20:04:31.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.404+0000 7fd0a6ffd640 1 -- 172.21.15.88:0/1684033211 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8005310 con 0x7fd0b01f2460 2023-10-16T20:04:31.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.404+0000 7fd0b8646640 1 -- 172.21.15.88:0/1684033211 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0b01f68c0 con 0x7fd0b01f2460 2023-10-16T20:04:31.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.404+0000 7fd0a6ffd640 1 -- 172.21.15.88:0/1684033211 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd0a801d460 con 0x7fd0b01f2460 2023-10-16T20:04:31.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.408+0000 7fd0a6ffd640 1 -- 172.21.15.88:0/1684033211 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd0a8005770 con 0x7fd0b01f2460 2023-10-16T20:04:31.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.408+0000 7fd0b8646640 1 -- 172.21.15.88:0/1684033211 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0b00cef30 con 0x7fd0b01f2460 2023-10-16T20:04:31.426 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.424+0000 7fd0a6ffd640 1 -- 172.21.15.88:0/1684033211 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fd0a801d850 con 0x7fd0b01f2460 2023-10-16T20:04:31.486 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.484+0000 7fd0b8646640 1 -- 172.21.15.88:0/1684033211 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fd0b01f6060 con 0x7fd0b01f2460 2023-10-16T20:04:31.489 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.488+0000 7fd0a6ffd640 1 -- 172.21.15.88:0/1684033211 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fd0a802b090 con 0x7fd0b01f2460 2023-10-16T20:04:31.489 INFO:teuthology.orchestra.run.smithi088.stderr:adjusted tunables profile to default 2023-10-16T20:04:31.490 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.488+0000 7fd0b8646640 1 -- 172.21.15.88:0/1684033211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0b01f2460 msgr2=0x7fd0b0155c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:31.490 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.488+0000 7fd0b8646640 1 --2- 172.21.15.88:0/1684033211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0b01f2460 0x7fd0b0155c90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd0b008bb40 tx=0x7fd0a800ead0 comp rx=0 tx=0).stop 2023-10-16T20:04:31.490 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.488+0000 7fd0b8646640 1 -- 172.21.15.88:0/1684033211 shutdown_connections 2023-10-16T20:04:31.491 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.488+0000 7fd0b8646640 1 --2- 172.21.15.88:0/1684033211 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0b015e3a0 0x7fd0b01f6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:31.491 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.488+0000 7fd0b8646640 1 --2- 172.21.15.88:0/1684033211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0b01f2460 0x7fd0b0155c90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:31.491 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.488+0000 7fd0b8646640 1 --2- 172.21.15.88:0/1684033211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd09c003b10 0x7fd0b0150ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:31.491 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.488+0000 7fd0b8646640 1 -- 172.21.15.88:0/1684033211 >> 172.21.15.88:0/1684033211 conn(0x7fd0b006b6e0 msgr2=0x7fd0b0157700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:31.491 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.488+0000 7fd0b8646640 1 -- 172.21.15.88:0/1684033211 shutdown_connections 2023-10-16T20:04:31.492 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:31.488+0000 7fd0b8646640 1 -- 172.21.15.88:0/1684033211 wait complete. 2023-10-16T20:04:31.509 INFO:tasks.ceph:check_enable_crimson: False 2023-10-16T20:04:31.509 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-16T20:04:31.509 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:04:31.510 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-16T20:04:31.530 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:04:31.531 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-16T20:04:31.595 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:04:31.596 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-16T20:04:31.620 INFO:tasks.ceph.mgr.x.smithi094.stderr:2023-10-16T20:04:31.614+0000 7fa4a1e0c180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T20:04:31.622 INFO:tasks.ceph.mgr.y.smithi088.stderr:2023-10-16T20:04:31.620+0000 7f65fc144180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T20:04:31.642 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:04:31.642 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-16T20:04:31.703 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:04:31.704 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-16T20:04:31.724 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:04:31.724 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-16T20:04:31.784 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:04:31.785 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-16T20:04:31.845 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:04:31.845 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-16T20:04:31.908 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph osd new 41293e0f-eacc-49a7-a09b-5e06fc182edb 0 2023-10-16T20:04:32.085 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.078+0000 7f494e068640 1 Processor -- start 2023-10-16T20:04:32.085 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.078+0000 7f494e068640 1 -- start start 2023-10-16T20:04:32.086 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.082+0000 7f494e068640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f49480b4c40 0x7f49480b6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:32.086 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.082+0000 7f494e068640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49480b7d20 con 0x7f494815cb70 2023-10-16T20:04:32.087 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.082+0000 7f494e068640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49480b7110 con 0x7f49480d1be0 2023-10-16T20:04:32.087 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.082+0000 7f494e068640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49480b73f0 con 0x7f49480b4c40 2023-10-16T20:04:32.087 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.082+0000 7f4947fff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f49480b4c40 0x7f49480b6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:32.087 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.082+0000 7f4946ffd640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f49480d1be0 0x7f49480b4530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:46590/0 (socket says 172.21.15.94:46590) 2023-10-16T20:04:32.087 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.082+0000 7f4947fff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f49480b4c40 0x7f49480b6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46060/0 (socket says 172.21.15.94:46060) 2023-10-16T20:04:32.088 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.082+0000 7f4946ffd640 1 -- 172.21.15.94:0/3369834777 learned_addr learned my addr 172.21.15.94:0/3369834777 (peer_addr_for_me v1:172.21.15.94:0/0) 2023-10-16T20:04:32.088 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.082+0000 7f4947fff640 1 -- 172.21.15.94:0/3369834777 >> v1:172.21.15.94:6789/0 conn(0x7f49480d1be0 legacy=0x7f49480b4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:32.088 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.082+0000 7f4947fff640 1 -- 172.21.15.94:0/3369834777 >> v1:172.21.15.88:6789/0 conn(0x7f494815cb70 legacy=0x7f49480b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:04:32.089 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.082+0000 7f4947fff640 1 -- 172.21.15.94:0/3369834777 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49480b76d0 con 0x7f49480b4c40 2023-10-16T20:04:32.090 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.086+0000 7f4947fff640 1 --2- 172.21.15.94:0/3369834777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49480b4c40 0x7f49480b6a80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4940009d90 tx=0x7f494001f100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=706bdb44b0632adc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:32.090 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.086+0000 7f49467fc640 1 -- 172.21.15.94:0/3369834777 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49400af600 con 0x7f49480b4c40 2023-10-16T20:04:32.090 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.086+0000 7f49467fc640 1 -- 172.21.15.94:0/3369834777 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49400ab050 con 0x7f49480b4c40 2023-10-16T20:04:32.090 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.086+0000 7f49467fc640 1 -- 172.21.15.94:0/3369834777 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49400af980 con 0x7f49480b4c40 2023-10-16T20:04:32.091 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.086+0000 7f494e068640 1 -- 172.21.15.94:0/3369834777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49480b4c40 msgr2=0x7f49480b6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:32.091 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.086+0000 7f494e068640 1 --2- 172.21.15.94:0/3369834777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49480b4c40 0x7f49480b6a80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4940009d90 tx=0x7f494001f100 comp rx=0 tx=0).stop 2023-10-16T20:04:32.092 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.086+0000 7f494e068640 1 -- 172.21.15.94:0/3369834777 shutdown_connections 2023-10-16T20:04:32.092 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.086+0000 7f494e068640 1 --2- 172.21.15.94:0/3369834777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49480b4c40 0x7f49480b6a80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:32.092 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.086+0000 7f494e068640 1 -- 172.21.15.94:0/3369834777 >> 172.21.15.94:0/3369834777 conn(0x7f494806b6e0 msgr2=0x7f494806baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:32.092 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.086+0000 7f494e068640 1 -- 172.21.15.94:0/3369834777 shutdown_connections 2023-10-16T20:04:32.093 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.086+0000 7f494e068640 1 -- 172.21.15.94:0/3369834777 wait complete. 2023-10-16T20:04:32.094 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.086+0000 7f494e068640 1 Processor -- start 2023-10-16T20:04:32.095 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.086+0000 7f494e068640 1 -- start start 2023-10-16T20:04:32.095 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.090+0000 7f494e068640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49480b4c40 0x7f494815d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:32.095 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.090+0000 7f494e068640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49480d1be0 0x7f494815f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:32.095 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.090+0000 7f494e068640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f494815cb70 0x7f494814cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:32.095 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.090+0000 7f494e068640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494815dd10 con 0x7f49480b4c40 2023-10-16T20:04:32.096 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.090+0000 7f494e068640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494815dff0 con 0x7f49480d1be0 2023-10-16T20:04:32.096 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.090+0000 7f494e068640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f494815e2d0 con 0x7f494815cb70 2023-10-16T20:04:32.096 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.090+0000 7f4946ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49480d1be0 0x7f494815f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:32.096 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.090+0000 7f4946ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49480d1be0 0x7f494815f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46062/0 (socket says 172.21.15.94:46062) 2023-10-16T20:04:32.096 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.090+0000 7f4946ffd640 1 -- 172.21.15.94:0/2854924442 learned_addr learned my addr 172.21.15.94:0/2854924442 (peer_addr_for_me v2:172.21.15.94:0/0) 2023-10-16T20:04:32.097 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.090+0000 7f4947fff640 1 --2- 172.21.15.94:0/2854924442 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f494815cb70 0x7f494814cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:32.097 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.090+0000 7f49477fe640 1 --2- 172.21.15.94:0/2854924442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49480b4c40 0x7f494815d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:32.097 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.090+0000 7f4946ffd640 1 -- 172.21.15.94:0/2854924442 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f494815cb70 msgr2=0x7f494814cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:32.097 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.090+0000 7f4946ffd640 1 --2- 172.21.15.94:0/2854924442 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f494815cb70 0x7f494814cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:32.097 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.090+0000 7f4946ffd640 1 -- 172.21.15.94:0/2854924442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49480b4c40 msgr2=0x7f494815d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:32.097 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.090+0000 7f4946ffd640 1 --2- 172.21.15.94:0/2854924442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49480b4c40 0x7f494815d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:32.098 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.090+0000 7f4946ffd640 1 -- 172.21.15.94:0/2854924442 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4940009a70 con 0x7f49480d1be0 2023-10-16T20:04:32.098 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.090+0000 7f4946ffd640 1 --2- 172.21.15.94:0/2854924442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49480d1be0 0x7f494815f930 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f493c00c500 tx=0x7f493c00c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:32.099 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.090+0000 7f492ffff640 1 -- 172.21.15.94:0/2854924442 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c005cf0 con 0x7f49480d1be0 2023-10-16T20:04:32.099 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.090+0000 7f492ffff640 1 -- 172.21.15.94:0/2854924442 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493c005e50 con 0x7f49480d1be0 2023-10-16T20:04:32.100 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.090+0000 7f494e068640 1 -- 172.21.15.94:0/2854924442 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f494815e5b0 con 0x7f49480d1be0 2023-10-16T20:04:32.100 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.090+0000 7f492ffff640 1 -- 172.21.15.94:0/2854924442 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c0072f0 con 0x7f49480d1be0 2023-10-16T20:04:32.100 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.090+0000 7f494e068640 1 -- 172.21.15.94:0/2854924442 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f494815edb0 con 0x7f49480d1be0 2023-10-16T20:04:32.100 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.090+0000 7f492ffff640 1 -- 172.21.15.94:0/2854924442 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f493c01c440 con 0x7f49480d1be0 2023-10-16T20:04:32.100 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.090+0000 7f492ffff640 1 -- 172.21.15.94:0/2854924442 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f493c0127d0 con 0x7f49480d1be0 2023-10-16T20:04:32.100 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.090+0000 7f494e068640 1 -- 172.21.15.94:0/2854924442 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f494804e9c0 con 0x7f49480d1be0 2023-10-16T20:04:32.101 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.094+0000 7f492ffff640 1 -- 172.21.15.94:0/2854924442 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f493c021060 con 0x7f49480d1be0 2023-10-16T20:04:32.165 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.162+0000 7f494e068640 1 -- 172.21.15.94:0/2854924442 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd new", "uuid": "41293e0f-eacc-49a7-a09b-5e06fc182edb", "id": 0} v 0) v1 -- 0x7f4948157990 con 0x7f49480d1be0 2023-10-16T20:04:32.167 INFO:tasks.ceph.mgr.x.smithi094.stderr:2023-10-16T20:04:32.162+0000 7fa4a1e0c180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T20:04:32.170 INFO:tasks.ceph.mgr.y.smithi088.stderr:2023-10-16T20:04:32.168+0000 7f65fc144180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T20:04:32.171 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.166+0000 7f492ffff640 1 -- 172.21.15.94:0/2854924442 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "41293e0f-eacc-49a7-a09b-5e06fc182edb", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f493c017070 con 0x7f49480d1be0 2023-10-16T20:04:32.171 INFO:teuthology.orchestra.run.smithi094.stdout:0 2023-10-16T20:04:32.172 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.166+0000 7f494e068640 1 -- 172.21.15.94:0/2854924442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49480d1be0 msgr2=0x7f494815f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:32.172 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.166+0000 7f494e068640 1 --2- 172.21.15.94:0/2854924442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49480d1be0 0x7f494815f930 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f493c00c500 tx=0x7f493c00c9c0 comp rx=0 tx=0).stop 2023-10-16T20:04:32.173 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.166+0000 7f494e068640 1 -- 172.21.15.94:0/2854924442 shutdown_connections 2023-10-16T20:04:32.173 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.166+0000 7f494e068640 1 --2- 172.21.15.94:0/2854924442 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f494815cb70 0x7f494814cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:32.173 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.166+0000 7f494e068640 1 --2- 172.21.15.94:0/2854924442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49480d1be0 0x7f494815f930 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:32.173 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.166+0000 7f494e068640 1 --2- 172.21.15.94:0/2854924442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49480b4c40 0x7f494815d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:32.173 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.166+0000 7f494e068640 1 -- 172.21.15.94:0/2854924442 >> 172.21.15.94:0/2854924442 conn(0x7f494806b6e0 msgr2=0x7f49480b56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:32.174 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.166+0000 7f494e068640 1 -- 172.21.15.94:0/2854924442 shutdown_connections 2023-10-16T20:04:32.174 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.166+0000 7f494e068640 1 -- 172.21.15.94:0/2854924442 wait complete. 2023-10-16T20:04:32.190 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph osd new 445f9559-9221-476a-b9dd-2a542e126270 1 2023-10-16T20:04:32.331 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.322+0000 7f0af7bd9640 1 Processor -- start 2023-10-16T20:04:32.331 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.322+0000 7f0af7bd9640 1 -- start start 2023-10-16T20:04:32.331 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.322+0000 7f0af7bd9640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f0af00d1be0 0x7f0af00b4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:32.331 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.326+0000 7f0af7bd9640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af00b7d20 con 0x7f0af00b4a70 2023-10-16T20:04:32.332 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.326+0000 7f0af7bd9640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af00b4ea0 con 0x7f0af015cb70 2023-10-16T20:04:32.332 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.326+0000 7f0af7bd9640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0af00b7110 con 0x7f0af00d1be0 2023-10-16T20:04:32.332 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.326+0000 7f0af5975640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f0af015cb70 0x7f0af00b83d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:46604/0 (socket says 172.21.15.94:46604) 2023-10-16T20:04:32.332 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.326+0000 7f0af5975640 1 -- 172.21.15.94:0/1874047700 learned_addr learned my addr 172.21.15.94:0/1874047700 (peer_addr_for_me v1:172.21.15.94:0/0) 2023-10-16T20:04:32.332 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.326+0000 7f0af5174640 1 --2- 172.21.15.94:0/1874047700 >> v2:172.21.15.88:3300/0 conn(0x7f0af00d1be0 0x7f0af00b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:32.332 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.326+0000 7f0af4973640 1 -- 172.21.15.94:0/1874047700 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1648416955 0 0) 0x7f0af00b4ea0 con 0x7f0af015cb70 2023-10-16T20:04:32.333 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.326+0000 7f0af4973640 1 -- 172.21.15.94:0/1874047700 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0adc003530 con 0x7f0af015cb70 2023-10-16T20:04:32.333 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.326+0000 7f0af5174640 1 -- 172.21.15.94:0/1874047700 >> v1:172.21.15.94:6789/0 conn(0x7f0af015cb70 legacy=0x7f0af00b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:32.333 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.326+0000 7f0af5174640 1 -- 172.21.15.94:0/1874047700 >> v1:172.21.15.88:6789/0 conn(0x7f0af00b4a70 legacy=0x7f0af00b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:32.333 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.326+0000 7f0af4973640 1 -- 172.21.15.94:0/1874047700 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 479782735 0 0) 0x7f0af00b7d20 con 0x7f0af00b4a70 2023-10-16T20:04:32.333 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.326+0000 7f0af5174640 1 -- 172.21.15.94:0/1874047700 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af00b73f0 con 0x7f0af00d1be0 2023-10-16T20:04:32.335 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.326+0000 7f0af5174640 1 --2- 172.21.15.94:0/1874047700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0af00d1be0 0x7f0af00b4530 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0ae00097c0 tx=0x7f0ae001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bfd3b70cc9f7708 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:32.335 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.330+0000 7f0af4973640 1 -- 172.21.15.94:0/1874047700 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae00a8970 con 0x7f0af00d1be0 2023-10-16T20:04:32.335 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.330+0000 7f0af4973640 1 -- 172.21.15.94:0/1874047700 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae00a8ad0 con 0x7f0af00d1be0 2023-10-16T20:04:32.335 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.330+0000 7f0af4973640 1 -- 172.21.15.94:0/1874047700 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae00a8e60 con 0x7f0af00d1be0 2023-10-16T20:04:32.336 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.330+0000 7f0af7bd9640 1 -- 172.21.15.94:0/1874047700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0af00d1be0 msgr2=0x7f0af00b4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:32.336 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.330+0000 7f0af7bd9640 1 --2- 172.21.15.94:0/1874047700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0af00d1be0 0x7f0af00b4530 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0ae00097c0 tx=0x7f0ae001ec70 comp rx=0 tx=0).stop 2023-10-16T20:04:32.336 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.330+0000 7f0af7bd9640 1 -- 172.21.15.94:0/1874047700 shutdown_connections 2023-10-16T20:04:32.336 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.330+0000 7f0af7bd9640 1 --2- 172.21.15.94:0/1874047700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0af00d1be0 0x7f0af00b4530 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:32.336 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.330+0000 7f0af7bd9640 1 -- 172.21.15.94:0/1874047700 >> 172.21.15.94:0/1874047700 conn(0x7f0af006b6e0 msgr2=0x7f0af006baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:32.336 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.330+0000 7f0af7bd9640 1 -- 172.21.15.94:0/1874047700 shutdown_connections 2023-10-16T20:04:32.337 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.330+0000 7f0af7bd9640 1 -- 172.21.15.94:0/1874047700 wait complete. 2023-10-16T20:04:32.339 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.330+0000 7f0af7bd9640 1 Processor -- start 2023-10-16T20:04:32.339 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.330+0000 7f0af7bd9640 1 -- start start 2023-10-16T20:04:32.340 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.330+0000 7f0af7bd9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0af00b4a70 0x7f0af014f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:32.340 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.334+0000 7f0af7bd9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0af00d1be0 0x7f0af015fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:32.340 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.334+0000 7f0af7bd9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af015cb70 0x7f0af015b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:32.340 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.334+0000 7f0af7bd9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af0150cd0 con 0x7f0af00d1be0 2023-10-16T20:04:32.340 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.334+0000 7f0af7bd9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af01601f0 con 0x7f0af015cb70 2023-10-16T20:04:32.341 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.334+0000 7f0af7bd9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af014fc30 con 0x7f0af00b4a70 2023-10-16T20:04:32.341 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.334+0000 7f0af5975640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0af00b4a70 0x7f0af014f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:32.341 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.334+0000 7f0af6176640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af015cb70 0x7f0af015b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:32.341 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.334+0000 7f0af5975640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0af00b4a70 0x7f0af014f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.94:45022/0 (socket says 172.21.15.94:45022) 2023-10-16T20:04:32.341 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.334+0000 7f0af5174640 1 --2- 172.21.15.94:0/1360922750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0af00d1be0 0x7f0af015fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:32.341 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.334+0000 7f0af5975640 1 -- 172.21.15.94:0/1360922750 learned_addr learned my addr 172.21.15.94:0/1360922750 (peer_addr_for_me v2:172.21.15.94:0/0) 2023-10-16T20:04:32.342 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.334+0000 7f0af6176640 1 -- 172.21.15.94:0/1360922750 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0af00b4a70 msgr2=0x7f0af014f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:32.342 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.334+0000 7f0af6176640 1 --2- 172.21.15.94:0/1360922750 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0af00b4a70 0x7f0af014f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:32.342 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.334+0000 7f0af6176640 1 -- 172.21.15.94:0/1360922750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0af00d1be0 msgr2=0x7f0af015fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:32.342 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.334+0000 7f0af6176640 1 --2- 172.21.15.94:0/1360922750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0af00d1be0 0x7f0af015fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:32.342 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.334+0000 7f0af6176640 1 -- 172.21.15.94:0/1360922750 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae0009420 con 0x7f0af015cb70 2023-10-16T20:04:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.334+0000 7f0af5975640 1 --2- 172.21.15.94:0/1360922750 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0af00b4a70 0x7f0af014f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:04:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.334+0000 7f0af6176640 1 --2- 172.21.15.94:0/1360922750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af015cb70 0x7f0af015b960 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800c780 tx=0x7f0ae800cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.334+0000 7f0ae67fc640 1 -- 172.21.15.94:0/1360922750 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8003db0 con 0x7f0af015cb70 2023-10-16T20:04:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.334+0000 7f0ae67fc640 1 -- 172.21.15.94:0/1360922750 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae800ce30 con 0x7f0af015cb70 2023-10-16T20:04:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.334+0000 7f0af7bd9640 1 -- 172.21.15.94:0/1360922750 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af014ff10 con 0x7f0af015cb70 2023-10-16T20:04:32.344 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.334+0000 7f0ae67fc640 1 -- 172.21.15.94:0/1360922750 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae80052b0 con 0x7f0af015cb70 2023-10-16T20:04:32.344 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.334+0000 7f0af7bd9640 1 -- 172.21.15.94:0/1360922750 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af01504d0 con 0x7f0af015cb70 2023-10-16T20:04:32.344 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.334+0000 7f0ae67fc640 1 -- 172.21.15.94:0/1360922750 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0ae8005710 con 0x7f0af015cb70 2023-10-16T20:04:32.344 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.334+0000 7f0ae67fc640 1 -- 172.21.15.94:0/1360922750 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f0ae801c7f0 con 0x7f0af015cb70 2023-10-16T20:04:32.345 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.334+0000 7f0af7bd9640 1 -- 172.21.15.94:0/1360922750 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af004e9c0 con 0x7f0af015cb70 2023-10-16T20:04:32.345 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.338+0000 7f0ae67fc640 1 -- 172.21.15.94:0/1360922750 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f0ae8017070 con 0x7f0af015cb70 2023-10-16T20:04:32.375 INFO:tasks.ceph.mgr.y.smithi088.stderr:2023-10-16T20:04:32.372+0000 7f65fc144180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T20:04:32.394 INFO:tasks.ceph.mgr.x.smithi094.stderr:2023-10-16T20:04:32.390+0000 7fa4a1e0c180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T20:04:32.422 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.418+0000 7f0af7bd9640 1 -- 172.21.15.94:0/1360922750 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd new", "uuid": "445f9559-9221-476a-b9dd-2a542e126270", "id": 1} v 0) v1 -- 0x7f0af00c60e0 con 0x7f0af015cb70 2023-10-16T20:04:32.429 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.422+0000 7f0ae67fc640 1 -- 172.21.15.94:0/1360922750 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "445f9559-9221-476a-b9dd-2a542e126270", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0ae801ca80 con 0x7f0af015cb70 2023-10-16T20:04:32.429 INFO:teuthology.orchestra.run.smithi094.stdout:1 2023-10-16T20:04:32.430 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.426+0000 7f0af7bd9640 1 -- 172.21.15.94:0/1360922750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af015cb70 msgr2=0x7f0af015b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:32.430 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.426+0000 7f0af7bd9640 1 --2- 172.21.15.94:0/1360922750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af015cb70 0x7f0af015b960 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800c780 tx=0x7f0ae800cc40 comp rx=0 tx=0).stop 2023-10-16T20:04:32.430 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.426+0000 7f0af7bd9640 1 -- 172.21.15.94:0/1360922750 shutdown_connections 2023-10-16T20:04:32.430 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.426+0000 7f0af7bd9640 1 --2- 172.21.15.94:0/1360922750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af015cb70 0x7f0af015b960 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:32.431 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.426+0000 7f0af7bd9640 1 --2- 172.21.15.94:0/1360922750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0af00d1be0 0x7f0af015fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:32.431 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.426+0000 7f0af7bd9640 1 --2- 172.21.15.94:0/1360922750 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0af00b4a70 0x7f0af014f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:32.431 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.426+0000 7f0af7bd9640 1 -- 172.21.15.94:0/1360922750 >> 172.21.15.94:0/1360922750 conn(0x7f0af006b6e0 msgr2=0x7f0af00b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:32.431 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.426+0000 7f0af7bd9640 1 -- 172.21.15.94:0/1360922750 shutdown_connections 2023-10-16T20:04:32.431 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.426+0000 7f0af7bd9640 1 -- 172.21.15.94:0/1360922750 wait complete. 2023-10-16T20:04:32.448 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph osd new 31b450a7-9bd8-4501-b4a1-0ced1a05f331 2 2023-10-16T20:04:32.584 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.578+0000 7f1e33bf8640 1 Processor -- start 2023-10-16T20:04:32.585 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.578+0000 7f1e33bf8640 1 -- start start 2023-10-16T20:04:32.586 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e33bf8640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e2c15cb70 0x7f1e2c0b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:32.586 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e33bf8640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1e2c0b6a80 0x7f1e2c0b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:32.586 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e33bf8640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e2c0b4ed0 con 0x7f1e2c0b4530 2023-10-16T20:04:32.587 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e33bf8640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e2c0b6f10 con 0x7f1e2c0b6a80 2023-10-16T20:04:32.587 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e33bf8640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c0b7d20 con 0x7f1e2c15cb70 2023-10-16T20:04:32.587 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e31994640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e2c15cb70 0x7f1e2c0b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:32.587 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e31193640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f1e2c0b4530 0x7f1e2c0d0bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:46606/0 (socket says 172.21.15.94:46606) 2023-10-16T20:04:32.587 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e31193640 1 -- 172.21.15.94:0/561165412 learned_addr learned my addr 172.21.15.94:0/561165412 (peer_addr_for_me v1:172.21.15.94:0/0) 2023-10-16T20:04:32.588 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e32195640 1 --2- 172.21.15.94:0/561165412 >> v2:172.21.15.88:3300/0 conn(0x7f1e2c0b6a80 0x7f1e2c0b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:32.589 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e30992640 1 -- 172.21.15.94:0/561165412 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2567394798 0 0) 0x7f1e2c0b4ed0 con 0x7f1e2c0b4530 2023-10-16T20:04:32.590 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e30992640 1 -- 172.21.15.94:0/561165412 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e18003530 con 0x7f1e2c0b4530 2023-10-16T20:04:32.590 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e30992640 1 -- 172.21.15.94:0/561165412 <== mon.1 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 234343217 0 0) 0x7f1e18003530 con 0x7f1e2c0b4530 2023-10-16T20:04:32.590 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e30992640 1 -- 172.21.15.94:0/561165412 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e2c15cb70 msgr2=0x7f1e2c0b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:04:32.590 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e30992640 1 --2- 172.21.15.94:0/561165412 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e2c15cb70 0x7f1e2c0b83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:32.591 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e30992640 1 -- 172.21.15.94:0/561165412 >> v2:172.21.15.88:3300/0 conn(0x7f1e2c0b6a80 msgr2=0x7f1e2c0b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:04:32.591 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e30992640 1 --2- 172.21.15.94:0/561165412 >> v2:172.21.15.88:3300/0 conn(0x7f1e2c0b6a80 0x7f1e2c0b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:32.591 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e30992640 1 -- 172.21.15.94:0/561165412 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e2c0b7110 con 0x7f1e2c0b4530 2023-10-16T20:04:32.591 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e30992640 1 -- 172.21.15.94:0/561165412 <== mon.1 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f1e1c001680 con 0x7f1e2c0b4530 2023-10-16T20:04:32.592 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e30992640 1 -- 172.21.15.94:0/561165412 >> v1:172.21.15.94:6789/0 conn(0x7f1e2c0b4530 legacy=0x7f1e2c0d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:32.592 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e30992640 1 --2- 172.21.15.94:0/561165412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e18003980 0x7f1e18003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:32.592 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e30992640 1 --2- 172.21.15.94:0/561165412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e180047d0 0x7f1e18004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:32.592 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e30992640 1 --2- 172.21.15.94:0/561165412 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e2c15cb70 0x7f1e18005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:32.592 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e30992640 1 -- 172.21.15.94:0/561165412 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c0b6f10 con 0x7f1e180047d0 2023-10-16T20:04:32.593 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e30992640 1 -- 172.21.15.94:0/561165412 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c0b7d20 con 0x7f1e18003980 2023-10-16T20:04:32.593 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e30992640 1 -- 172.21.15.94:0/561165412 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1800ba40 con 0x7f1e2c15cb70 2023-10-16T20:04:32.593 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e31994640 1 --2- 172.21.15.94:0/561165412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e18003980 0x7f1e18003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:32.593 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e32195640 1 --2- 172.21.15.94:0/561165412 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e2c15cb70 0x7f1e18005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:32.594 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e31193640 1 --2- 172.21.15.94:0/561165412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e180047d0 0x7f1e18004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:32.594 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e31994640 1 -- 172.21.15.94:0/561165412 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e2c15cb70 msgr2=0x7f1e18005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:04:32.594 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e31994640 1 --2- 172.21.15.94:0/561165412 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e2c15cb70 0x7f1e18005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:32.594 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e31994640 1 -- 172.21.15.94:0/561165412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e180047d0 msgr2=0x7f1e18004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:04:32.595 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e31994640 1 --2- 172.21.15.94:0/561165412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e180047d0 0x7f1e18004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:32.595 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.582+0000 7f1e31994640 1 -- 172.21.15.94:0/561165412 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e18003530 con 0x7f1e18003980 2023-10-16T20:04:32.598 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.586+0000 7f1e31994640 1 --2- 172.21.15.94:0/561165412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e18003980 0x7f1e18003da0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009ae0 tx=0x7f1e2801ca50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51e966901cf4b03e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:32.598 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.586+0000 7f1e30992640 1 -- 172.21.15.94:0/561165412 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e280a5560 con 0x7f1e18003980 2023-10-16T20:04:32.599 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.586+0000 7f1e30992640 1 -- 172.21.15.94:0/561165412 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e280a56c0 con 0x7f1e18003980 2023-10-16T20:04:32.599 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.586+0000 7f1e30992640 1 -- 172.21.15.94:0/561165412 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e280a5a70 con 0x7f1e18003980 2023-10-16T20:04:32.599 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.586+0000 7f1e33bf8640 1 -- 172.21.15.94:0/561165412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e18003980 msgr2=0x7f1e18003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:32.599 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.586+0000 7f1e33bf8640 1 --2- 172.21.15.94:0/561165412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e18003980 0x7f1e18003da0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009ae0 tx=0x7f1e2801ca50 comp rx=0 tx=0).stop 2023-10-16T20:04:32.600 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.586+0000 7f1e32195640 1 -- 172.21.15.94:0/561165412 reap_dead start 2023-10-16T20:04:32.600 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.586+0000 7f1e33bf8640 1 -- 172.21.15.94:0/561165412 shutdown_connections 2023-10-16T20:04:32.600 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.586+0000 7f1e33bf8640 1 -- 172.21.15.94:0/561165412 >> 172.21.15.94:0/561165412 conn(0x7f1e2c06b6e0 msgr2=0x7f1e2c06baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:32.600 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.586+0000 7f1e33bf8640 1 -- 172.21.15.94:0/561165412 shutdown_connections 2023-10-16T20:04:32.600 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.586+0000 7f1e33bf8640 1 -- 172.21.15.94:0/561165412 wait complete. 2023-10-16T20:04:32.601 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.590+0000 7f1e33bf8640 1 Processor -- start 2023-10-16T20:04:32.601 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.590+0000 7f1e33bf8640 1 -- start start 2023-10-16T20:04:32.601 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.590+0000 7f1e33bf8640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e18003980 0x7f1e2c1f63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:32.601 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.590+0000 7f1e33bf8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e2c0d1d80 0x7f1e2c1f2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:32.601 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.590+0000 7f1e33bf8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e2c0d21e0 0x7f1e2c1525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:32.602 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.590+0000 7f1e33bf8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c1f5950 con 0x7f1e2c0d1d80 2023-10-16T20:04:32.602 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.590+0000 7f1e33bf8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c1f2e20 con 0x7f1e2c0d21e0 2023-10-16T20:04:32.602 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.590+0000 7f1e33bf8640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c1f3100 con 0x7f1e18003980 2023-10-16T20:04:32.602 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.590+0000 7f1e31994640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e18003980 0x7f1e2c1f63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:32.602 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.590+0000 7f1e32195640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e2c0d21e0 0x7f1e2c1525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:32.603 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.590+0000 7f1e31994640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e18003980 0x7f1e2c1f63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.94:45052/0 (socket says 172.21.15.94:45052) 2023-10-16T20:04:32.603 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.590+0000 7f1e31994640 1 -- 172.21.15.94:0/743662333 learned_addr learned my addr 172.21.15.94:0/743662333 (peer_addr_for_me v2:172.21.15.94:0/0) 2023-10-16T20:04:32.603 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.590+0000 7f1e31193640 1 --2- 172.21.15.94:0/743662333 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e2c0d1d80 0x7f1e2c1f2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:32.603 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.590+0000 7f1e32195640 1 -- 172.21.15.94:0/743662333 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e18003980 msgr2=0x7f1e2c1f63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:32.603 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.590+0000 7f1e32195640 1 --2- 172.21.15.94:0/743662333 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e18003980 0x7f1e2c1f63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:32.604 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.590+0000 7f1e32195640 1 -- 172.21.15.94:0/743662333 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e2c0d1d80 msgr2=0x7f1e2c1f2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:32.604 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.590+0000 7f1e32195640 1 --2- 172.21.15.94:0/743662333 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e2c0d1d80 0x7f1e2c1f2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:32.604 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.590+0000 7f1e32195640 1 -- 172.21.15.94:0/743662333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e2c1f5ba0 con 0x7f1e2c0d21e0 2023-10-16T20:04:32.604 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.590+0000 7f1e31994640 1 --2- 172.21.15.94:0/743662333 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e18003980 0x7f1e2c1f63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:04:32.604 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.590+0000 7f1e32195640 1 --2- 172.21.15.94:0/743662333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e2c0d21e0 0x7f1e2c1525c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1e1800bbd0 tx=0x7f1e24003ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:32.605 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.590+0000 7f1e227fc640 1 -- 172.21.15.94:0/743662333 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e24007ab0 con 0x7f1e2c0d21e0 2023-10-16T20:04:32.605 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.590+0000 7f1e227fc640 1 -- 172.21.15.94:0/743662333 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e24007c10 con 0x7f1e2c0d21e0 2023-10-16T20:04:32.605 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.590+0000 7f1e227fc640 1 -- 172.21.15.94:0/743662333 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e24015420 con 0x7f1e2c0d21e0 2023-10-16T20:04:32.605 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.590+0000 7f1e33bf8640 1 -- 172.21.15.94:0/743662333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e2c1f5e80 con 0x7f1e2c0d21e0 2023-10-16T20:04:32.605 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.590+0000 7f1e33bf8640 1 -- 172.21.15.94:0/743662333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e2c1f28a0 con 0x7f1e2c0d21e0 2023-10-16T20:04:32.606 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.590+0000 7f1e227fc640 1 -- 172.21.15.94:0/743662333 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1e24015880 con 0x7f1e2c0d21e0 2023-10-16T20:04:32.606 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.590+0000 7f1e227fc640 1 -- 172.21.15.94:0/743662333 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f1e2401f460 con 0x7f1e2c0d21e0 2023-10-16T20:04:32.606 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.590+0000 7f1e33bf8640 1 -- 172.21.15.94:0/743662333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e2c1f32c0 con 0x7f1e2c0d21e0 2023-10-16T20:04:32.606 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.594+0000 7f1e227fc640 1 -- 172.21.15.94:0/743662333 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f1e24015b30 con 0x7f1e2c0d21e0 2023-10-16T20:04:32.665 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.658+0000 7f1e33bf8640 1 -- 172.21.15.94:0/743662333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd new", "uuid": "31b450a7-9bd8-4501-b4a1-0ced1a05f331", "id": 2} v 0) v1 -- 0x7f1e2c1f6070 con 0x7f1e2c0d21e0 2023-10-16T20:04:32.671 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.666+0000 7f1e227fc640 1 -- 172.21.15.94:0/743662333 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "31b450a7-9bd8-4501-b4a1-0ced1a05f331", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1e2401fac0 con 0x7f1e2c0d21e0 2023-10-16T20:04:32.671 INFO:teuthology.orchestra.run.smithi094.stdout:2 2023-10-16T20:04:32.672 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.666+0000 7f1e33bf8640 1 -- 172.21.15.94:0/743662333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e2c0d21e0 msgr2=0x7f1e2c1525c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:32.672 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.666+0000 7f1e33bf8640 1 --2- 172.21.15.94:0/743662333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e2c0d21e0 0x7f1e2c1525c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1e1800bbd0 tx=0x7f1e24003ec0 comp rx=0 tx=0).stop 2023-10-16T20:04:32.672 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.666+0000 7f1e33bf8640 1 -- 172.21.15.94:0/743662333 shutdown_connections 2023-10-16T20:04:32.673 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.666+0000 7f1e33bf8640 1 --2- 172.21.15.94:0/743662333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e2c0d21e0 0x7f1e2c1525c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:32.673 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.666+0000 7f1e33bf8640 1 --2- 172.21.15.94:0/743662333 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e2c0d1d80 0x7f1e2c1f2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:32.673 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.666+0000 7f1e33bf8640 1 --2- 172.21.15.94:0/743662333 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e18003980 0x7f1e2c1f63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:32.673 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.666+0000 7f1e33bf8640 1 -- 172.21.15.94:0/743662333 >> 172.21.15.94:0/743662333 conn(0x7f1e2c06b6e0 msgr2=0x7f1e2c157fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:32.673 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.666+0000 7f1e33bf8640 1 -- 172.21.15.94:0/743662333 shutdown_connections 2023-10-16T20:04:32.673 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.666+0000 7f1e33bf8640 1 -- 172.21.15.94:0/743662333 wait complete. 2023-10-16T20:04:32.690 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph osd new eb397ce8-ae27-4102-a5da-e0db2311baa6 3 2023-10-16T20:04:32.827 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.822+0000 7f79fdb3e640 1 Processor -- start 2023-10-16T20:04:32.827 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.822+0000 7f79fdb3e640 1 -- start start 2023-10-16T20:04:32.828 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.822+0000 7f79fdb3e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f79f8157130 0x7f79f815de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:32.828 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.822+0000 7f79fdb3e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f79f80ce0b0 0x7f79f8160860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:32.829 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.822+0000 7f79fdb3e640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f8161470 con 0x7f79f8157510 2023-10-16T20:04:32.829 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.822+0000 7f79fdb3e640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f80b88a0 con 0x7f79f8157130 2023-10-16T20:04:32.829 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.822+0000 7f79fdb3e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f8161750 con 0x7f79f80ce0b0 2023-10-16T20:04:32.829 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.822+0000 7f79f77fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f79f8157130 0x7f79f815de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:32.829 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.822+0000 7f79f7fff640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f79f80ce0b0 0x7f79f8160860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:32.830 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.822+0000 7f79f77fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f79f8157130 0x7f79f815de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46102/0 (socket says 172.21.15.94:46102) 2023-10-16T20:04:32.830 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.822+0000 7f79f77fe640 1 -- 172.21.15.94:0/2492355247 learned_addr learned my addr 172.21.15.94:0/2492355247 (peer_addr_for_me v2:172.21.15.94:0/0) 2023-10-16T20:04:32.830 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.822+0000 7f79f77fe640 1 -- 172.21.15.94:0/2492355247 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f79f80ce0b0 msgr2=0x7f79f8160860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:04:32.831 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.822+0000 7f79f77fe640 1 --2- 172.21.15.94:0/2492355247 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f79f80ce0b0 0x7f79f8160860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:32.831 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.822+0000 7f79f77fe640 1 -- 172.21.15.94:0/2492355247 >> v1:172.21.15.88:6789/0 conn(0x7f79f8157510 legacy=0x7f79f80cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:32.831 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.822+0000 7f79f77fe640 1 -- 172.21.15.94:0/2492355247 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f815d380 con 0x7f79f8157130 2023-10-16T20:04:32.831 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.822+0000 7f79f7fff640 1 --2- 172.21.15.94:0/2492355247 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f79f80ce0b0 0x7f79f8160860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:04:32.833 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.826+0000 7f79f77fe640 1 --2- 172.21.15.94:0/2492355247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79f8157130 0x7f79f815de60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f79e8009870 tx=0x7f79e801c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f5c6ed81d729ef8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:32.833 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.826+0000 7f79f67fc640 1 -- 172.21.15.94:0/2492355247 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e80a95f0 con 0x7f79f8157130 2023-10-16T20:04:32.833 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.826+0000 7f79f67fc640 1 -- 172.21.15.94:0/2492355247 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e80a9750 con 0x7f79f8157130 2023-10-16T20:04:32.833 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.826+0000 7f79f67fc640 1 -- 172.21.15.94:0/2492355247 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e80a9ae0 con 0x7f79f8157130 2023-10-16T20:04:32.833 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.826+0000 7f79fdb3e640 1 -- 172.21.15.94:0/2492355247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79f8157130 msgr2=0x7f79f815de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:32.834 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.826+0000 7f79fdb3e640 1 --2- 172.21.15.94:0/2492355247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79f8157130 0x7f79f815de60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f79e8009870 tx=0x7f79e801c510 comp rx=0 tx=0).stop 2023-10-16T20:04:32.834 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.826+0000 7f79fdb3e640 1 -- 172.21.15.94:0/2492355247 shutdown_connections 2023-10-16T20:04:32.834 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.826+0000 7f79fdb3e640 1 --2- 172.21.15.94:0/2492355247 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f79f80ce0b0 0x7f79f8160860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:32.834 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.826+0000 7f79fdb3e640 1 --2- 172.21.15.94:0/2492355247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79f8157130 0x7f79f815de60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:32.834 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.826+0000 7f79fdb3e640 1 -- 172.21.15.94:0/2492355247 >> 172.21.15.94:0/2492355247 conn(0x7f79f80715c0 msgr2=0x7f79f80719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:32.835 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.826+0000 7f79fdb3e640 1 -- 172.21.15.94:0/2492355247 shutdown_connections 2023-10-16T20:04:32.835 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.826+0000 7f79fdb3e640 1 -- 172.21.15.94:0/2492355247 wait complete. 2023-10-16T20:04:32.836 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.830+0000 7f79fdb3e640 1 Processor -- start 2023-10-16T20:04:32.837 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.830+0000 7f79fdb3e640 1 -- start start 2023-10-16T20:04:32.837 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.830+0000 7f79fdb3e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79f80ce0b0 0x7f79f80c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:32.837 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.830+0000 7f79fdb3e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f79f8157130 0x7f79f8155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:32.837 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.830+0000 7f79fdb3e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79f8157510 0x7f79f8155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:32.838 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.830+0000 7f79fdb3e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f814e590 con 0x7f79f8157510 2023-10-16T20:04:32.838 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.830+0000 7f79fdb3e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f814e870 con 0x7f79f80ce0b0 2023-10-16T20:04:32.838 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.830+0000 7f79fdb3e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f814eb50 con 0x7f79f8157130 2023-10-16T20:04:32.838 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.830+0000 7f79f77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79f80ce0b0 0x7f79f80c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:32.838 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.830+0000 7f79f77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79f80ce0b0 0x7f79f80c60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46112/0 (socket says 172.21.15.94:46112) 2023-10-16T20:04:32.839 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.830+0000 7f79f77fe640 1 -- 172.21.15.94:0/1856594098 learned_addr learned my addr 172.21.15.94:0/1856594098 (peer_addr_for_me v2:172.21.15.94:0/0) 2023-10-16T20:04:32.839 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.830+0000 7f79f6ffd640 1 --2- 172.21.15.94:0/1856594098 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f79f8157130 0x7f79f8155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:32.839 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.830+0000 7f79f7fff640 1 --2- 172.21.15.94:0/1856594098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79f8157510 0x7f79f8155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:32.839 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.830+0000 7f79f77fe640 1 -- 172.21.15.94:0/1856594098 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f79f8157130 msgr2=0x7f79f8155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:32.839 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.830+0000 7f79f77fe640 1 --2- 172.21.15.94:0/1856594098 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f79f8157130 0x7f79f8155200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:32.840 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.830+0000 7f79f77fe640 1 -- 172.21.15.94:0/1856594098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79f8157510 msgr2=0x7f79f8155740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:32.840 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.830+0000 7f79f77fe640 1 --2- 172.21.15.94:0/1856594098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79f8157510 0x7f79f8155740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:32.840 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.830+0000 7f79f77fe640 1 -- 172.21.15.94:0/1856594098 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f0009a70 con 0x7f79f80ce0b0 2023-10-16T20:04:32.840 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.830+0000 7f79f77fe640 1 --2- 172.21.15.94:0/1856594098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79f80ce0b0 0x7f79f80c60a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f79e80ab040 tx=0x7f79e801ad50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:32.841 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.830+0000 7f79dffff640 1 -- 172.21.15.94:0/1856594098 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e80b1800 con 0x7f79f80ce0b0 2023-10-16T20:04:32.842 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.830+0000 7f79dffff640 1 -- 172.21.15.94:0/1856594098 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e80b1960 con 0x7f79f80ce0b0 2023-10-16T20:04:32.842 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.830+0000 7f79dffff640 1 -- 172.21.15.94:0/1856594098 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e80b1d10 con 0x7f79f80ce0b0 2023-10-16T20:04:32.842 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.830+0000 7f79fdb3e640 1 -- 172.21.15.94:0/1856594098 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e80094d0 con 0x7f79f80ce0b0 2023-10-16T20:04:32.842 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.834+0000 7f79fdb3e640 1 -- 172.21.15.94:0/1856594098 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f814f110 con 0x7f79f80ce0b0 2023-10-16T20:04:32.842 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.834+0000 7f79dffff640 1 -- 172.21.15.94:0/1856594098 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f79e80bc450 con 0x7f79f80ce0b0 2023-10-16T20:04:32.843 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.834+0000 7f79dffff640 1 -- 172.21.15.94:0/1856594098 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f79e80a87e0 con 0x7f79f80ce0b0 2023-10-16T20:04:32.843 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.834+0000 7f79fdb3e640 1 -- 172.21.15.94:0/1856594098 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f814ec90 con 0x7f79f80ce0b0 2023-10-16T20:04:32.843 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.834+0000 7f79dffff640 1 -- 172.21.15.94:0/1856594098 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f79f814ec90 con 0x7f79f80ce0b0 2023-10-16T20:04:32.906 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.902+0000 7f79fdb3e640 1 -- 172.21.15.94:0/1856594098 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd new", "uuid": "eb397ce8-ae27-4102-a5da-e0db2311baa6", "id": 3} v 0) v1 -- 0x7f79f814e9b0 con 0x7f79f80ce0b0 2023-10-16T20:04:32.912 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.906+0000 7f79dffff640 1 -- 172.21.15.94:0/1856594098 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "eb397ce8-ae27-4102-a5da-e0db2311baa6", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f79f814e9b0 con 0x7f79f80ce0b0 2023-10-16T20:04:32.912 INFO:teuthology.orchestra.run.smithi094.stdout:3 2023-10-16T20:04:32.913 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.906+0000 7f79fdb3e640 1 -- 172.21.15.94:0/1856594098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79f80ce0b0 msgr2=0x7f79f80c60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:32.913 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.906+0000 7f79fdb3e640 1 --2- 172.21.15.94:0/1856594098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79f80ce0b0 0x7f79f80c60a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f79e80ab040 tx=0x7f79e801ad50 comp rx=0 tx=0).stop 2023-10-16T20:04:32.913 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.906+0000 7f79fdb3e640 1 -- 172.21.15.94:0/1856594098 shutdown_connections 2023-10-16T20:04:32.913 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.906+0000 7f79fdb3e640 1 --2- 172.21.15.94:0/1856594098 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f79f8157130 0x7f79f8155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:32.914 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.906+0000 7f79fdb3e640 1 --2- 172.21.15.94:0/1856594098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79f8157510 0x7f79f8155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:32.914 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.906+0000 7f79fdb3e640 1 --2- 172.21.15.94:0/1856594098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79f80ce0b0 0x7f79f80c60a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:32.914 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.906+0000 7f79fdb3e640 1 -- 172.21.15.94:0/1856594098 >> 172.21.15.94:0/1856594098 conn(0x7f79f80715c0 msgr2=0x7f79f815ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:32.914 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.906+0000 7f79fdb3e640 1 -- 172.21.15.94:0/1856594098 shutdown_connections 2023-10-16T20:04:32.914 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:32.906+0000 7f79fdb3e640 1 -- 172.21.15.94:0/1856594098 wait complete. 2023-10-16T20:04:32.932 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph osd new dcdd1519-0732-4d8c-a3b1-f553e26ffef7 4 2023-10-16T20:04:33.043 INFO:tasks.ceph.mgr.y.smithi088.stderr:2023-10-16T20:04:33.040+0000 7f65fc144180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T20:04:33.065 INFO:tasks.ceph.mgr.x.smithi094.stderr:2023-10-16T20:04:33.058+0000 7fa4a1e0c180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T20:04:33.070 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.066+0000 7f98665c4640 1 Processor -- start 2023-10-16T20:04:33.070 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.066+0000 7f98665c4640 1 -- start start 2023-10-16T20:04:33.072 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.066+0000 7f98665c4640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f9860157130 0x7f986015de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:33.073 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.066+0000 7f98665c4640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f98600ce0b0 0x7f9860160860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:33.073 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.066+0000 7f98665c4640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9860161470 con 0x7f9860157510 2023-10-16T20:04:33.073 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.066+0000 7f98665c4640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98600b88a0 con 0x7f9860157130 2023-10-16T20:04:33.073 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.066+0000 7f98665c4640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9860161750 con 0x7f98600ce0b0 2023-10-16T20:04:33.073 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.066+0000 7f9864b61640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f98600ce0b0 0x7f9860160860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:33.074 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.066+0000 7f9864b61640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f98600ce0b0 0x7f9860160860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.94:45088/0 (socket says 172.21.15.94:45088) 2023-10-16T20:04:33.074 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.066+0000 7f9864b61640 1 -- 172.21.15.94:0/915154785 learned_addr learned my addr 172.21.15.94:0/915154785 (peer_addr_for_me v2:172.21.15.94:0/0) 2023-10-16T20:04:33.074 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.066+0000 7f985ffff640 1 --2- 172.21.15.94:0/915154785 >> v2:172.21.15.88:3300/0 conn(0x7f9860157130 0x7f986015de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:33.074 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.066+0000 7f985effd640 1 -- 172.21.15.94:0/915154785 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3927901117 0 0) 0x7f9860161470 con 0x7f9860157510 2023-10-16T20:04:33.074 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.066+0000 7f985effd640 1 -- 172.21.15.94:0/915154785 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9848003530 con 0x7f9860157510 2023-10-16T20:04:33.075 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.066+0000 7f9864b61640 1 -- 172.21.15.94:0/915154785 >> v2:172.21.15.88:3300/0 conn(0x7f9860157130 msgr2=0x7f986015de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:04:33.075 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.066+0000 7f9864b61640 1 --2- 172.21.15.94:0/915154785 >> v2:172.21.15.88:3300/0 conn(0x7f9860157130 0x7f986015de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.075 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.066+0000 7f9864b61640 1 -- 172.21.15.94:0/915154785 >> v1:172.21.15.88:6789/0 conn(0x7f9860157510 legacy=0x7f98600cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:33.075 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.066+0000 7f9864b61640 1 -- 172.21.15.94:0/915154785 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f986015d380 con 0x7f98600ce0b0 2023-10-16T20:04:33.077 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.070+0000 7f9864b61640 1 --2- 172.21.15.94:0/915154785 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f98600ce0b0 0x7f9860160860 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9858009e50 tx=0x7f985801c9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7f6fddb515021f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:33.077 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.070+0000 7f985effd640 1 -- 172.21.15.94:0/915154785 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98580a58d0 con 0x7f98600ce0b0 2023-10-16T20:04:33.077 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.070+0000 7f985effd640 1 -- 172.21.15.94:0/915154785 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98580a5a30 con 0x7f98600ce0b0 2023-10-16T20:04:33.077 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.070+0000 7f985effd640 1 -- 172.21.15.94:0/915154785 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98580a5dc0 con 0x7f98600ce0b0 2023-10-16T20:04:33.078 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.070+0000 7f98665c4640 1 -- 172.21.15.94:0/915154785 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f98600ce0b0 msgr2=0x7f9860160860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:33.078 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.070+0000 7f98665c4640 1 --2- 172.21.15.94:0/915154785 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f98600ce0b0 0x7f9860160860 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9858009e50 tx=0x7f985801c9e0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.078 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.070+0000 7f98665c4640 1 -- 172.21.15.94:0/915154785 shutdown_connections 2023-10-16T20:04:33.078 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.070+0000 7f98665c4640 1 --2- 172.21.15.94:0/915154785 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f98600ce0b0 0x7f9860160860 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.078 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.070+0000 7f98665c4640 1 --2- 172.21.15.94:0/915154785 >> v2:172.21.15.88:3300/0 conn(0x7f9860157130 0x7f986015de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.079 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.070+0000 7f98665c4640 1 -- 172.21.15.94:0/915154785 >> 172.21.15.94:0/915154785 conn(0x7f98600715c0 msgr2=0x7f98600719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:33.079 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.070+0000 7f98665c4640 1 -- 172.21.15.94:0/915154785 shutdown_connections 2023-10-16T20:04:33.079 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.070+0000 7f98665c4640 1 -- 172.21.15.94:0/915154785 wait complete. 2023-10-16T20:04:33.080 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.074+0000 7f98665c4640 1 Processor -- start 2023-10-16T20:04:33.080 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.074+0000 7f98665c4640 1 -- start start 2023-10-16T20:04:33.081 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.074+0000 7f98665c4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98600ce0b0 0x7f986014d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:33.082 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.074+0000 7f98665c4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9860157130 0x7f98600b2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:33.082 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.074+0000 7f98665c4640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9860157510 0x7f98600b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:33.082 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.074+0000 7f98665c4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98600c6d20 con 0x7f9860157130 2023-10-16T20:04:33.082 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.074+0000 7f98665c4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986014de80 con 0x7f98600ce0b0 2023-10-16T20:04:33.083 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.074+0000 7f98665c4640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98600b5540 con 0x7f9860157510 2023-10-16T20:04:33.083 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.074+0000 7f985ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98600ce0b0 0x7f986014d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:33.083 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.074+0000 7f985ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98600ce0b0 0x7f986014d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46120/0 (socket says 172.21.15.94:46120) 2023-10-16T20:04:33.083 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.074+0000 7f985ffff640 1 -- 172.21.15.94:0/1909143260 learned_addr learned my addr 172.21.15.94:0/1909143260 (peer_addr_for_me v2:172.21.15.94:0/0) 2023-10-16T20:04:33.083 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.074+0000 7f9864b61640 1 --2- 172.21.15.94:0/1909143260 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9860157510 0x7f98600b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:33.084 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.078+0000 7f985f7fe640 1 --2- 172.21.15.94:0/1909143260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9860157130 0x7f98600b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:33.084 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.078+0000 7f985ffff640 1 -- 172.21.15.94:0/1909143260 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9860157510 msgr2=0x7f98600b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:33.084 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.078+0000 7f985ffff640 1 --2- 172.21.15.94:0/1909143260 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9860157510 0x7f98600b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.084 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.078+0000 7f985ffff640 1 -- 172.21.15.94:0/1909143260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9860157130 msgr2=0x7f98600b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:33.084 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.078+0000 7f985ffff640 1 --2- 172.21.15.94:0/1909143260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9860157130 0x7f98600b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.085 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.078+0000 7f985ffff640 1 -- 172.21.15.94:0/1909143260 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9858009b30 con 0x7f98600ce0b0 2023-10-16T20:04:33.085 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.078+0000 7f985ffff640 1 --2- 172.21.15.94:0/1909143260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98600ce0b0 0x7f986014d940 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f985000e960 tx=0x7f985000ee20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:33.086 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.078+0000 7f985cff9640 1 -- 172.21.15.94:0/1909143260 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9850009690 con 0x7f98600ce0b0 2023-10-16T20:04:33.086 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.078+0000 7f985cff9640 1 -- 172.21.15.94:0/1909143260 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985000ca00 con 0x7f98600ce0b0 2023-10-16T20:04:33.086 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.078+0000 7f985cff9640 1 -- 172.21.15.94:0/1909143260 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985000cd70 con 0x7f98600ce0b0 2023-10-16T20:04:33.087 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.078+0000 7f98665c4640 1 -- 172.21.15.94:0/1909143260 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f986014e160 con 0x7f98600ce0b0 2023-10-16T20:04:33.087 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.078+0000 7f98665c4640 1 -- 172.21.15.94:0/1909143260 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f986014e720 con 0x7f98600ce0b0 2023-10-16T20:04:33.087 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.078+0000 7f985cff9640 1 -- 172.21.15.94:0/1909143260 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9850005ca0 con 0x7f98600ce0b0 2023-10-16T20:04:33.087 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.078+0000 7f985cff9640 1 -- 172.21.15.94:0/1909143260 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f9850006530 con 0x7f98600ce0b0 2023-10-16T20:04:33.087 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.078+0000 7f98665c4640 1 -- 172.21.15.94:0/1909143260 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9860150380 con 0x7f98600ce0b0 2023-10-16T20:04:33.088 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.082+0000 7f985cff9640 1 -- 172.21.15.94:0/1909143260 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f9850021060 con 0x7f98600ce0b0 2023-10-16T20:04:33.106 INFO:tasks.ceph.mgr.y.smithi088.stderr:2023-10-16T20:04:33.104+0000 7f65fc144180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T20:04:33.132 INFO:tasks.ceph.mgr.x.smithi094.stderr:2023-10-16T20:04:33.126+0000 7fa4a1e0c180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T20:04:33.152 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.146+0000 7f98665c4640 1 -- 172.21.15.94:0/1909143260 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd new", "uuid": "dcdd1519-0732-4d8c-a3b1-f553e26ffef7", "id": 4} v 0) v1 -- 0x7f986004e9c0 con 0x7f98600ce0b0 2023-10-16T20:04:33.156 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.150+0000 7f985cff9640 1 -- 172.21.15.94:0/1909143260 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dcdd1519-0732-4d8c-a3b1-f553e26ffef7", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9850029020 con 0x7f98600ce0b0 2023-10-16T20:04:33.157 INFO:teuthology.orchestra.run.smithi094.stdout:4 2023-10-16T20:04:33.158 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.154+0000 7f98665c4640 1 -- 172.21.15.94:0/1909143260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98600ce0b0 msgr2=0x7f986014d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:33.158 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.154+0000 7f98665c4640 1 --2- 172.21.15.94:0/1909143260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98600ce0b0 0x7f986014d940 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f985000e960 tx=0x7f985000ee20 comp rx=0 tx=0).stop 2023-10-16T20:04:33.158 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.154+0000 7f98665c4640 1 -- 172.21.15.94:0/1909143260 shutdown_connections 2023-10-16T20:04:33.158 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.154+0000 7f98665c4640 1 --2- 172.21.15.94:0/1909143260 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9860157510 0x7f98600b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.159 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.154+0000 7f98665c4640 1 --2- 172.21.15.94:0/1909143260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9860157130 0x7f98600b2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.159 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.154+0000 7f98665c4640 1 --2- 172.21.15.94:0/1909143260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98600ce0b0 0x7f986014d940 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.159 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.154+0000 7f98665c4640 1 -- 172.21.15.94:0/1909143260 >> 172.21.15.94:0/1909143260 conn(0x7f98600715c0 msgr2=0x7f98600b5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:33.159 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.154+0000 7f98665c4640 1 -- 172.21.15.94:0/1909143260 shutdown_connections 2023-10-16T20:04:33.159 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.154+0000 7f98665c4640 1 -- 172.21.15.94:0/1909143260 wait complete. 2023-10-16T20:04:33.167 INFO:tasks.ceph.mgr.y.smithi088.stderr:2023-10-16T20:04:33.164+0000 7f65fc144180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T20:04:33.178 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph osd new 131b0869-de39-42a9-9fb3-718426d9290f 5 2023-10-16T20:04:33.198 INFO:tasks.ceph.mgr.x.smithi094.stderr:2023-10-16T20:04:33.194+0000 7fa4a1e0c180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T20:04:33.255 INFO:tasks.ceph.mgr.y.smithi088.stderr:2023-10-16T20:04:33.252+0000 7f65fc144180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T20:04:33.289 INFO:tasks.ceph.mgr.x.smithi094.stderr:2023-10-16T20:04:33.282+0000 7fa4a1e0c180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T20:04:33.313 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.310+0000 7fce7c6c9640 1 Processor -- start 2023-10-16T20:04:33.314 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.310+0000 7fce7c6c9640 1 -- start start 2023-10-16T20:04:33.314 INFO:tasks.ceph.mgr.y.smithi088.stderr:2023-10-16T20:04:33.312+0000 7f65fc144180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T20:04:33.315 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.310+0000 7fce7c6c9640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fce740b4c40 0x7fce740b6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:33.315 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.310+0000 7fce7c6c9640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce740b7d20 con 0x7fce7415cb70 2023-10-16T20:04:33.316 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.310+0000 7fce7c6c9640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce740b7110 con 0x7fce740d1be0 2023-10-16T20:04:33.316 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.310+0000 7fce7c6c9640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce740b73f0 con 0x7fce740b4c40 2023-10-16T20:04:33.316 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.310+0000 7fce79c64640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fce740d1be0 0x7fce740b4530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:46622/0 (socket says 172.21.15.94:46622) 2023-10-16T20:04:33.316 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.310+0000 7fce79c64640 1 -- 172.21.15.94:0/842551263 learned_addr learned my addr 172.21.15.94:0/842551263 (peer_addr_for_me v1:172.21.15.94:0/0) 2023-10-16T20:04:33.316 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.310+0000 7fce7ac66640 1 --2- 172.21.15.94:0/842551263 >> v2:172.21.15.88:3300/0 conn(0x7fce740b4c40 0x7fce740b6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:33.317 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.310+0000 7fce79463640 1 -- 172.21.15.94:0/842551263 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3105286100 0 0) 0x7fce740b7110 con 0x7fce740d1be0 2023-10-16T20:04:33.317 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.310+0000 7fce79463640 1 -- 172.21.15.94:0/842551263 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce60003530 con 0x7fce740d1be0 2023-10-16T20:04:33.318 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.310+0000 7fce79463640 1 -- 172.21.15.94:0/842551263 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1410341492 0 0) 0x7fce740b7d20 con 0x7fce7415cb70 2023-10-16T20:04:33.318 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.310+0000 7fce79463640 1 -- 172.21.15.94:0/842551263 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce740b7110 con 0x7fce7415cb70 2023-10-16T20:04:33.318 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.310+0000 7fce79463640 1 -- 172.21.15.94:0/842551263 <== mon.1 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2197023040 0 0) 0x7fce60003530 con 0x7fce740d1be0 2023-10-16T20:04:33.318 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.310+0000 7fce7ac66640 1 -- 172.21.15.94:0/842551263 >> v1:172.21.15.94:6789/0 conn(0x7fce740d1be0 legacy=0x7fce740b4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:33.319 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.310+0000 7fce7ac66640 1 -- 172.21.15.94:0/842551263 >> v1:172.21.15.88:6789/0 conn(0x7fce7415cb70 legacy=0x7fce740b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:33.319 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.310+0000 7fce7ac66640 1 -- 172.21.15.94:0/842551263 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce740b76d0 con 0x7fce740b4c40 2023-10-16T20:04:33.321 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.314+0000 7fce7ac66640 1 --2- 172.21.15.94:0/842551263 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce740b4c40 0x7fce740b6a80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009d90 tx=0x7fce6c01efa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68fcd42cb9a9ce4c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:33.321 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.318+0000 7fce79463640 1 -- 172.21.15.94:0/842551263 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce70001600 con 0x7fce740b4c40 2023-10-16T20:04:33.322 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.318+0000 7fce79463640 1 -- 172.21.15.94:0/842551263 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6c0a8b70 con 0x7fce740b4c40 2023-10-16T20:04:33.322 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.318+0000 7fce79463640 1 -- 172.21.15.94:0/842551263 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce64001680 con 0x7fce740b4c40 2023-10-16T20:04:33.322 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.318+0000 7fce7c6c9640 1 -- 172.21.15.94:0/842551263 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce740b4c40 msgr2=0x7fce740b6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:33.323 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.318+0000 7fce7c6c9640 1 --2- 172.21.15.94:0/842551263 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce740b4c40 0x7fce740b6a80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009d90 tx=0x7fce6c01efa0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.323 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.318+0000 7fce7c6c9640 1 -- 172.21.15.94:0/842551263 shutdown_connections 2023-10-16T20:04:33.323 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.318+0000 7fce7c6c9640 1 --2- 172.21.15.94:0/842551263 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce740b4c40 0x7fce740b6a80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.323 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.318+0000 7fce7c6c9640 1 -- 172.21.15.94:0/842551263 >> 172.21.15.94:0/842551263 conn(0x7fce7406b6e0 msgr2=0x7fce7406baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:33.323 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.318+0000 7fce7c6c9640 1 -- 172.21.15.94:0/842551263 shutdown_connections 2023-10-16T20:04:33.324 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.318+0000 7fce7c6c9640 1 -- 172.21.15.94:0/842551263 wait complete. 2023-10-16T20:04:33.324 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.318+0000 7fce7c6c9640 1 Processor -- start 2023-10-16T20:04:33.325 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.318+0000 7fce7c6c9640 1 -- start start 2023-10-16T20:04:33.327 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.318+0000 7fce7c6c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce740b4c40 0x7fce74157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:33.327 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.318+0000 7fce7c6c9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce740d1be0 0x7fce74158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:33.327 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.318+0000 7fce7c6c9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce7415cb70 0x7fce7415a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:33.327 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.322+0000 7fce7c6c9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce740c7cc0 con 0x7fce7415cb70 2023-10-16T20:04:33.328 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.322+0000 7fce7c6c9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce740c76b0 con 0x7fce740b4c40 2023-10-16T20:04:33.328 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.322+0000 7fce7c6c9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7415ae70 con 0x7fce740d1be0 2023-10-16T20:04:33.328 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.322+0000 7fce7a465640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce740b4c40 0x7fce74157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:33.328 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.322+0000 7fce79c64640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce740d1be0 0x7fce74158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:33.328 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.322+0000 7fce7a465640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce740b4c40 0x7fce74157f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46134/0 (socket says 172.21.15.94:46134) 2023-10-16T20:04:33.329 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.322+0000 7fce7a465640 1 -- 172.21.15.94:0/1695098737 learned_addr learned my addr 172.21.15.94:0/1695098737 (peer_addr_for_me v2:172.21.15.94:0/0) 2023-10-16T20:04:33.329 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.322+0000 7fce7ac66640 1 --2- 172.21.15.94:0/1695098737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce7415cb70 0x7fce7415a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:33.329 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.322+0000 7fce7a465640 1 -- 172.21.15.94:0/1695098737 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce740d1be0 msgr2=0x7fce74158470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:33.329 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.322+0000 7fce7a465640 1 --2- 172.21.15.94:0/1695098737 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce740d1be0 0x7fce74158470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.329 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.322+0000 7fce7a465640 1 -- 172.21.15.94:0/1695098737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce7415cb70 msgr2=0x7fce7415a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:33.330 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.322+0000 7fce7a465640 1 --2- 172.21.15.94:0/1695098737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce7415cb70 0x7fce7415a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.330 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.322+0000 7fce7a465640 1 -- 172.21.15.94:0/1695098737 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6c009a70 con 0x7fce740b4c40 2023-10-16T20:04:33.330 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.322+0000 7fce7a465640 1 --2- 172.21.15.94:0/1695098737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce740b4c40 0x7fce74157f30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fce7000c780 tx=0x7fce7000cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:33.330 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.322+0000 7fce6affd640 1 -- 172.21.15.94:0/1695098737 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce70005270 con 0x7fce740b4c40 2023-10-16T20:04:33.330 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.322+0000 7fce6affd640 1 -- 172.21.15.94:0/1695098737 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce700053d0 con 0x7fce740b4c40 2023-10-16T20:04:33.330 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.322+0000 7fce7c6c9640 1 -- 172.21.15.94:0/1695098737 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce740c7950 con 0x7fce740b4c40 2023-10-16T20:04:33.331 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.322+0000 7fce6affd640 1 -- 172.21.15.94:0/1695098737 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7000a880 con 0x7fce740b4c40 2023-10-16T20:04:33.331 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.322+0000 7fce7c6c9640 1 -- 172.21.15.94:0/1695098737 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce74140ec0 con 0x7fce740b4c40 2023-10-16T20:04:33.331 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.322+0000 7fce6affd640 1 -- 172.21.15.94:0/1695098737 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fce7000ace0 con 0x7fce740b4c40 2023-10-16T20:04:33.331 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.322+0000 7fce6affd640 1 -- 172.21.15.94:0/1695098737 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fce70012400 con 0x7fce740b4c40 2023-10-16T20:04:33.333 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.322+0000 7fce7c6c9640 1 -- 172.21.15.94:0/1695098737 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce7404e9c0 con 0x7fce740b4c40 2023-10-16T20:04:33.333 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.326+0000 7fce6affd640 1 -- 172.21.15.94:0/1695098737 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fce70017070 con 0x7fce740b4c40 2023-10-16T20:04:33.354 INFO:tasks.ceph.mgr.x.smithi094.stderr:2023-10-16T20:04:33.350+0000 7fa4a1e0c180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T20:04:33.396 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.390+0000 7fce7c6c9640 1 -- 172.21.15.94:0/1695098737 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd new", "uuid": "131b0869-de39-42a9-9fb3-718426d9290f", "id": 5} v 0) v1 -- 0x7fce741604e0 con 0x7fce740b4c40 2023-10-16T20:04:33.403 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.398+0000 7fce6affd640 1 -- 172.21.15.94:0/1695098737 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "131b0869-de39-42a9-9fb3-718426d9290f", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fce70003600 con 0x7fce740b4c40 2023-10-16T20:04:33.403 INFO:teuthology.orchestra.run.smithi094.stdout:5 2023-10-16T20:04:33.404 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.398+0000 7fce7c6c9640 1 -- 172.21.15.94:0/1695098737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce740b4c40 msgr2=0x7fce74157f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:33.404 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.398+0000 7fce7c6c9640 1 --2- 172.21.15.94:0/1695098737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce740b4c40 0x7fce74157f30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fce7000c780 tx=0x7fce7000cc40 comp rx=0 tx=0).stop 2023-10-16T20:04:33.404 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.398+0000 7fce7c6c9640 1 -- 172.21.15.94:0/1695098737 shutdown_connections 2023-10-16T20:04:33.405 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.398+0000 7fce7c6c9640 1 --2- 172.21.15.94:0/1695098737 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce740d1be0 0x7fce74158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.405 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.398+0000 7fce7c6c9640 1 --2- 172.21.15.94:0/1695098737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce7415cb70 0x7fce7415a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.405 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.398+0000 7fce7c6c9640 1 --2- 172.21.15.94:0/1695098737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce740b4c40 0x7fce74157f30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.405 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.398+0000 7fce7c6c9640 1 -- 172.21.15.94:0/1695098737 >> 172.21.15.94:0/1695098737 conn(0x7fce7406b6e0 msgr2=0x7fce740b5760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:33.405 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.398+0000 7fce7c6c9640 1 -- 172.21.15.94:0/1695098737 shutdown_connections 2023-10-16T20:04:33.406 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.398+0000 7fce7c6c9640 1 -- 172.21.15.94:0/1695098737 wait complete. 2023-10-16T20:04:33.406 INFO:tasks.ceph.mgr.y.smithi088.stderr:2023-10-16T20:04:33.404+0000 7f65fc144180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T20:04:33.423 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph osd new 5a576a94-dbf3-4dbb-9017-021d0da8ca66 6 2023-10-16T20:04:33.449 INFO:tasks.ceph.mgr.x.smithi094.stderr:2023-10-16T20:04:33.442+0000 7fa4a1e0c180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T20:04:33.465 INFO:tasks.ceph.mgr.y.smithi088.stderr:2023-10-16T20:04:33.464+0000 7f65fc144180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T20:04:33.512 INFO:tasks.ceph.mgr.x.smithi094.stderr:2023-10-16T20:04:33.506+0000 7fa4a1e0c180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T20:04:33.587 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.582+0000 7f661759e640 1 Processor -- start 2023-10-16T20:04:33.587 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.582+0000 7f661759e640 1 -- start start 2023-10-16T20:04:33.588 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.582+0000 7f661759e640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f661815c9a0 0x7f66180cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:33.588 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.582+0000 7f661759e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f66180cf230 0x7f66180cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.582+0000 7f661759e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66180d0c40 con 0x7f66180d3270 2023-10-16T20:04:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.582+0000 7f661759e640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66180ce7e0 con 0x7f661815c9a0 2023-10-16T20:04:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.582+0000 7f661759e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66180d0630 con 0x7f66180cf230 2023-10-16T20:04:33.590 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.582+0000 7f6616d9d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f66180cf230 0x7f66180cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:33.590 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.582+0000 7f6615d9b640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f66180d3270 0x7f66180d10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:46624/0 (socket says 172.21.15.94:46624) 2023-10-16T20:04:33.590 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.582+0000 7f6615d9b640 1 -- 172.21.15.94:0/3403838399 learned_addr learned my addr 172.21.15.94:0/3403838399 (peer_addr_for_me v1:172.21.15.94:0/0) 2023-10-16T20:04:33.590 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.582+0000 7f661659c640 1 --2- 172.21.15.94:0/3403838399 >> v2:172.21.15.88:3300/0 conn(0x7f661815c9a0 0x7f66180cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:33.591 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.582+0000 7f6616d9d640 1 -- 172.21.15.94:0/3403838399 >> v2:172.21.15.88:3300/0 conn(0x7f661815c9a0 msgr2=0x7f66180cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:04:33.591 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.582+0000 7f6616d9d640 1 --2- 172.21.15.94:0/3403838399 >> v2:172.21.15.88:3300/0 conn(0x7f661815c9a0 0x7f66180cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.591 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.582+0000 7f661559a640 1 -- 172.21.15.94:0/3403838399 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4111695026 0 0) 0x7f66180d0c40 con 0x7f66180d3270 2023-10-16T20:04:33.591 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.582+0000 7f6616d9d640 1 -- 172.21.15.94:0/3403838399 >> v1:172.21.15.94:6789/0 conn(0x7f66180d3270 legacy=0x7f66180d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:33.592 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.582+0000 7f6616d9d640 1 -- 172.21.15.94:0/3403838399 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66180ce520 con 0x7f66180cf230 2023-10-16T20:04:33.592 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.582+0000 7f661659c640 1 --2- 172.21.15.94:0/3403838399 >> v2:172.21.15.88:3300/0 conn(0x7f661815c9a0 0x7f66180cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:04:33.593 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.586+0000 7f6616d9d640 1 --2- 172.21.15.94:0/3403838399 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f66180cf230 0x7f66180cf680 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6610009d90 tx=0x7f661001c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7ad0293a05702bb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:33.593 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.586+0000 7f661559a640 1 -- 172.21.15.94:0/3403838399 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66100a5a10 con 0x7f66180cf230 2023-10-16T20:04:33.594 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.586+0000 7f661559a640 1 -- 172.21.15.94:0/3403838399 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66100a5b70 con 0x7f66180cf230 2023-10-16T20:04:33.594 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.586+0000 7f661559a640 1 -- 172.21.15.94:0/3403838399 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66100ad440 con 0x7f66180cf230 2023-10-16T20:04:33.594 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.586+0000 7f661759e640 1 -- 172.21.15.94:0/3403838399 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f66180cf230 msgr2=0x7f66180cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:33.594 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.586+0000 7f661759e640 1 --2- 172.21.15.94:0/3403838399 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f66180cf230 0x7f66180cf680 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6610009d90 tx=0x7f661001c950 comp rx=0 tx=0).stop 2023-10-16T20:04:33.594 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.586+0000 7f661759e640 1 -- 172.21.15.94:0/3403838399 shutdown_connections 2023-10-16T20:04:33.595 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.586+0000 7f661759e640 1 --2- 172.21.15.94:0/3403838399 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f66180cf230 0x7f66180cf680 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.595 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.586+0000 7f661759e640 1 --2- 172.21.15.94:0/3403838399 >> v2:172.21.15.88:3300/0 conn(0x7f661815c9a0 0x7f66180cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.595 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.586+0000 7f661759e640 1 -- 172.21.15.94:0/3403838399 >> 172.21.15.94:0/3403838399 conn(0x7f661809bc30 msgr2=0x7f66180d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:33.595 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.586+0000 7f661759e640 1 -- 172.21.15.94:0/3403838399 shutdown_connections 2023-10-16T20:04:33.595 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.586+0000 7f661759e640 1 -- 172.21.15.94:0/3403838399 wait complete. 2023-10-16T20:04:33.598 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.590+0000 7f661759e640 1 Processor -- start 2023-10-16T20:04:33.598 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.590+0000 7f661759e640 1 -- start start 2023-10-16T20:04:33.598 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.590+0000 7f661759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66180cf230 0x7f66180b1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:33.599 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.590+0000 7f661759e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66180d3270 0x7f6618154c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:33.599 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.590+0000 7f661759e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f661815c9a0 0x7f66180cb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:33.599 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.590+0000 7f661759e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66180b0f00 con 0x7f66180d3270 2023-10-16T20:04:33.599 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.590+0000 7f661759e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66180b0280 con 0x7f66180cf230 2023-10-16T20:04:33.599 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.590+0000 7f661759e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66180b0560 con 0x7f661815c9a0 2023-10-16T20:04:33.600 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.590+0000 7f661659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66180cf230 0x7f66180b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:33.600 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.590+0000 7f6616d9d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f661815c9a0 0x7f66180cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:33.600 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.590+0000 7f661659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66180cf230 0x7f66180b1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46150/0 (socket says 172.21.15.94:46150) 2023-10-16T20:04:33.600 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.590+0000 7f661659c640 1 -- 172.21.15.94:0/1856633892 learned_addr learned my addr 172.21.15.94:0/1856633892 (peer_addr_for_me v2:172.21.15.94:0/0) 2023-10-16T20:04:33.601 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.590+0000 7f6615d9b640 1 --2- 172.21.15.94:0/1856633892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66180d3270 0x7f6618154c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:33.601 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.590+0000 7f661659c640 1 -- 172.21.15.94:0/1856633892 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f661815c9a0 msgr2=0x7f66180cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:33.601 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.590+0000 7f661659c640 1 --2- 172.21.15.94:0/1856633892 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f661815c9a0 0x7f66180cb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.601 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.590+0000 7f661659c640 1 -- 172.21.15.94:0/1856633892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66180d3270 msgr2=0x7f6618154c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:33.601 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.590+0000 7f661659c640 1 --2- 172.21.15.94:0/1856633892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66180d3270 0x7f6618154c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.602 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.590+0000 7f661659c640 1 -- 172.21.15.94:0/1856633892 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6610009a70 con 0x7f66180cf230 2023-10-16T20:04:33.602 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.590+0000 7f6616d9d640 1 --2- 172.21.15.94:0/1856633892 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f661815c9a0 0x7f66180cb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:04:33.602 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.590+0000 7f661659c640 1 --2- 172.21.15.94:0/1856633892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66180cf230 0x7f66180b1320 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f660800d7d0 tx=0x7f660800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:33.602 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.594+0000 7f6606ffd640 1 -- 172.21.15.94:0/1856633892 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66080059a0 con 0x7f66180cf230 2023-10-16T20:04:33.602 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.594+0000 7f6606ffd640 1 -- 172.21.15.94:0/1856633892 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6608005b00 con 0x7f66180cf230 2023-10-16T20:04:33.603 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.594+0000 7f661759e640 1 -- 172.21.15.94:0/1856633892 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66180b0840 con 0x7f66180cf230 2023-10-16T20:04:33.603 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.594+0000 7f6606ffd640 1 -- 172.21.15.94:0/1856633892 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6608005e90 con 0x7f66180cf230 2023-10-16T20:04:33.603 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.594+0000 7f661759e640 1 -- 172.21.15.94:0/1856633892 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6618155180 con 0x7f66180cf230 2023-10-16T20:04:33.603 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.594+0000 7f6606ffd640 1 -- 172.21.15.94:0/1856633892 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6608004330 con 0x7f66180cf230 2023-10-16T20:04:33.603 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.594+0000 7f6606ffd640 1 -- 172.21.15.94:0/1856633892 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f6608009c90 con 0x7f66180cf230 2023-10-16T20:04:33.604 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.594+0000 7f661759e640 1 -- 172.21.15.94:0/1856633892 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66180b1170 con 0x7f66180cf230 2023-10-16T20:04:33.604 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.594+0000 7f6606ffd640 1 -- 172.21.15.94:0/1856633892 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f6608018020 con 0x7f66180cf230 2023-10-16T20:04:33.620 INFO:tasks.ceph.mgr.y.smithi088.stderr:2023-10-16T20:04:33.620+0000 7f65fc144180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T20:04:33.666 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.662+0000 7f661759e640 1 -- 172.21.15.94:0/1856633892 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5a576a94-dbf3-4dbb-9017-021d0da8ca66", "id": 6} v 0) v1 -- 0x7f66180674f0 con 0x7f66180cf230 2023-10-16T20:04:33.670 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.666+0000 7f6606ffd640 1 -- 172.21.15.94:0/1856633892 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5a576a94-dbf3-4dbb-9017-021d0da8ca66", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f660802c0b0 con 0x7f66180cf230 2023-10-16T20:04:33.671 INFO:teuthology.orchestra.run.smithi094.stdout:6 2023-10-16T20:04:33.672 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.666+0000 7f661759e640 1 -- 172.21.15.94:0/1856633892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66180cf230 msgr2=0x7f66180b1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:33.672 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.666+0000 7f661759e640 1 --2- 172.21.15.94:0/1856633892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66180cf230 0x7f66180b1320 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f660800d7d0 tx=0x7f660800dc90 comp rx=0 tx=0).stop 2023-10-16T20:04:33.672 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.666+0000 7f661759e640 1 -- 172.21.15.94:0/1856633892 shutdown_connections 2023-10-16T20:04:33.672 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.666+0000 7f661759e640 1 --2- 172.21.15.94:0/1856633892 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f661815c9a0 0x7f66180cb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.672 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.666+0000 7f661759e640 1 --2- 172.21.15.94:0/1856633892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66180d3270 0x7f6618154c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.673 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.666+0000 7f661759e640 1 --2- 172.21.15.94:0/1856633892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66180cf230 0x7f66180b1320 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.673 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.666+0000 7f661759e640 1 -- 172.21.15.94:0/1856633892 >> 172.21.15.94:0/1856633892 conn(0x7f661809bc30 msgr2=0x7f66180d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:33.673 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.666+0000 7f661759e640 1 -- 172.21.15.94:0/1856633892 shutdown_connections 2023-10-16T20:04:33.673 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.666+0000 7f661759e640 1 -- 172.21.15.94:0/1856633892 wait complete. 2023-10-16T20:04:33.675 INFO:tasks.ceph.mgr.x.smithi094.stderr:2023-10-16T20:04:33.670+0000 7fa4a1e0c180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T20:04:33.689 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph osd new 058c9901-1985-45bd-afdd-893a577c9adf 7 2023-10-16T20:04:33.690 INFO:tasks.ceph.mgr.y.smithi088.stderr:2023-10-16T20:04:33.688+0000 7f65fc144180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T20:04:33.746 INFO:tasks.ceph.mgr.x.smithi094.stderr:2023-10-16T20:04:33.742+0000 7fa4a1e0c180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T20:04:33.828 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.822+0000 7f812b972640 1 Processor -- start 2023-10-16T20:04:33.828 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.822+0000 7f812b972640 1 -- start start 2023-10-16T20:04:33.829 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.826+0000 7f812b972640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8124157130 0x7f812415de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:33.830 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.826+0000 7f812b972640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8124157510 0x7f81240cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:33.830 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.826+0000 7f812b972640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f81240cdee0 0x7f8124160860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:33.830 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.826+0000 7f812b972640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81240b88a0 con 0x7f81240cdee0 2023-10-16T20:04:33.830 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.826+0000 7f812b972640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81240ce370 con 0x7f8124157130 2023-10-16T20:04:33.830 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.826+0000 7f812b972640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8124161470 con 0x7f8124157510 2023-10-16T20:04:33.831 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.826+0000 7f812970e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8124157130 0x7f812415de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:33.831 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.826+0000 7f8128f0d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8124157510 0x7f81240cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:33.832 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.826+0000 7f812970e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8124157130 0x7f812415de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46154/0 (socket says 172.21.15.94:46154) 2023-10-16T20:04:33.832 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.826+0000 7f812970e640 1 -- 172.21.15.94:0/1715342389 learned_addr learned my addr 172.21.15.94:0/1715342389 (peer_addr_for_me v2:172.21.15.94:0/0) 2023-10-16T20:04:33.832 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.826+0000 7f812970e640 1 -- 172.21.15.94:0/1715342389 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8124157510 msgr2=0x7f81240cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:04:33.832 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.826+0000 7f812970e640 1 --2- 172.21.15.94:0/1715342389 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8124157510 0x7f81240cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.833 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.826+0000 7f812970e640 1 -- 172.21.15.94:0/1715342389 >> v2:172.21.15.88:3300/0 conn(0x7f81240cdee0 msgr2=0x7f8124160860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:04:33.833 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.826+0000 7f8129f0f640 1 --2- 172.21.15.94:0/1715342389 >> v2:172.21.15.88:3300/0 conn(0x7f81240cdee0 0x7f8124160860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:33.833 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.826+0000 7f812970e640 1 --2- 172.21.15.94:0/1715342389 >> v2:172.21.15.88:3300/0 conn(0x7f81240cdee0 0x7f8124160860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.833 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.826+0000 7f812970e640 1 -- 172.21.15.94:0/1715342389 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8124161750 con 0x7f8124157130 2023-10-16T20:04:33.833 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.826+0000 7f8128f0d640 1 --2- 172.21.15.94:0/1715342389 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8124157510 0x7f81240cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:04:33.834 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.826+0000 7f8129f0f640 1 --2- 172.21.15.94:0/1715342389 >> v2:172.21.15.88:3300/0 conn(0x7f81240cdee0 0x7f8124160860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:04:33.835 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.826+0000 7f812970e640 1 --2- 172.21.15.94:0/1715342389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8124157130 0x7f812415de60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f81240d0310 tx=0x7f812001c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=805797a1ebed525 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:33.835 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.830+0000 7f811bfff640 1 -- 172.21.15.94:0/1715342389 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81200aa610 con 0x7f8124157130 2023-10-16T20:04:33.835 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.830+0000 7f811bfff640 1 -- 172.21.15.94:0/1715342389 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81200a6040 con 0x7f8124157130 2023-10-16T20:04:33.835 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.830+0000 7f811bfff640 1 -- 172.21.15.94:0/1715342389 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81200aa990 con 0x7f8124157130 2023-10-16T20:04:33.836 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.830+0000 7f812b972640 1 -- 172.21.15.94:0/1715342389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8124157130 msgr2=0x7f812415de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:33.836 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.830+0000 7f812b972640 1 --2- 172.21.15.94:0/1715342389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8124157130 0x7f812415de60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f81240d0310 tx=0x7f812001c4d0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.836 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.830+0000 7f812b972640 1 -- 172.21.15.94:0/1715342389 shutdown_connections 2023-10-16T20:04:33.836 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.830+0000 7f812b972640 1 --2- 172.21.15.94:0/1715342389 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8124157510 0x7f81240cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.836 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.830+0000 7f812b972640 1 --2- 172.21.15.94:0/1715342389 >> v2:172.21.15.88:3300/0 conn(0x7f81240cdee0 0x7f8124160860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.837 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.830+0000 7f812b972640 1 --2- 172.21.15.94:0/1715342389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8124157130 0x7f812415de60 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.837 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.830+0000 7f812b972640 1 -- 172.21.15.94:0/1715342389 >> 172.21.15.94:0/1715342389 conn(0x7f81240715c0 msgr2=0x7f81240719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:33.837 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.830+0000 7f812b972640 1 -- 172.21.15.94:0/1715342389 shutdown_connections 2023-10-16T20:04:33.837 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.830+0000 7f812b972640 1 -- 172.21.15.94:0/1715342389 wait complete. 2023-10-16T20:04:33.838 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.830+0000 7f812b972640 1 Processor -- start 2023-10-16T20:04:33.838 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.830+0000 7f812b972640 1 -- start start 2023-10-16T20:04:33.840 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.834+0000 7f812b972640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81240cdee0 0x7f81240b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:33.840 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.834+0000 7f812b972640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8124157130 0x7f81240c8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:33.841 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.834+0000 7f812b972640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8124157510 0x7f81240b68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:33.841 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.834+0000 7f812b972640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81240c8960 con 0x7f81240cdee0 2023-10-16T20:04:33.841 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.834+0000 7f812b972640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81240b73a0 con 0x7f8124157130 2023-10-16T20:04:33.841 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.834+0000 7f812b972640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81241541b0 con 0x7f8124157510 2023-10-16T20:04:33.842 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.834+0000 7f8128f0d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8124157130 0x7f81240c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:33.842 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.834+0000 7f8129f0f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8124157510 0x7f81240b68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:33.842 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.834+0000 7f8129f0f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8124157510 0x7f81240b68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.94:45144/0 (socket says 172.21.15.94:45144) 2023-10-16T20:04:33.842 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.834+0000 7f812970e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81240cdee0 0x7f81240b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:33.842 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.834+0000 7f8129f0f640 1 -- 172.21.15.94:0/4255262667 learned_addr learned my addr 172.21.15.94:0/4255262667 (peer_addr_for_me v2:172.21.15.94:0/0) 2023-10-16T20:04:33.843 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.834+0000 7f8129f0f640 1 -- 172.21.15.94:0/4255262667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8124157130 msgr2=0x7f81240c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:33.843 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.834+0000 7f8129f0f640 1 --2- 172.21.15.94:0/4255262667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8124157130 0x7f81240c8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.843 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.834+0000 7f8129f0f640 1 -- 172.21.15.94:0/4255262667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81240cdee0 msgr2=0x7f81240b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:33.843 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.834+0000 7f8129f0f640 1 --2- 172.21.15.94:0/4255262667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81240cdee0 0x7f81240b2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.843 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.834+0000 7f8129f0f640 1 -- 172.21.15.94:0/4255262667 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8114009420 con 0x7f8124157510 2023-10-16T20:04:33.844 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.834+0000 7f812970e640 1 --2- 172.21.15.94:0/4255262667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81240cdee0 0x7f81240b2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:04:33.844 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.834+0000 7f8129f0f640 1 --2- 172.21.15.94:0/4255262667 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8124157510 0x7f81240b68e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f811c00cef0 tx=0x7f811c00a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:33.844 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.834+0000 7f8119ffb640 1 -- 172.21.15.94:0/4255262667 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c00d840 con 0x7f8124157510 2023-10-16T20:04:33.844 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.834+0000 7f8119ffb640 1 -- 172.21.15.94:0/4255262667 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f811c00d9a0 con 0x7f8124157510 2023-10-16T20:04:33.844 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.834+0000 7f8119ffb640 1 -- 172.21.15.94:0/4255262667 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c00dd50 con 0x7f8124157510 2023-10-16T20:04:33.845 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.834+0000 7f812b972640 1 -- 172.21.15.94:0/4255262667 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81200094d0 con 0x7f8124157510 2023-10-16T20:04:33.845 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.834+0000 7f8119ffb640 1 -- 172.21.15.94:0/4255262667 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f811c015740 con 0x7f8124157510 2023-10-16T20:04:33.845 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.834+0000 7f812b972640 1 -- 172.21.15.94:0/4255262667 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8124154770 con 0x7f8124157510 2023-10-16T20:04:33.845 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.834+0000 7f8119ffb640 1 -- 172.21.15.94:0/4255262667 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f811c01ec90 con 0x7f8124157510 2023-10-16T20:04:33.846 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.838+0000 7f812b972640 1 -- 172.21.15.94:0/4255262667 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8124071bb0 con 0x7f8124157510 2023-10-16T20:04:33.846 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.838+0000 7f8119ffb640 1 -- 172.21.15.94:0/4255262667 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f811c01a070 con 0x7f8124157510 2023-10-16T20:04:33.909 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.906+0000 7f812b972640 1 -- 172.21.15.94:0/4255262667 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd new", "uuid": "058c9901-1985-45bd-afdd-893a577c9adf", "id": 7} v 0) v1 -- 0x7f812404e9c0 con 0x7f8124157510 2023-10-16T20:04:33.914 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.910+0000 7f8119ffb640 1 -- 172.21.15.94:0/4255262667 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "058c9901-1985-45bd-afdd-893a577c9adf", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f811c01ee50 con 0x7f8124157510 2023-10-16T20:04:33.914 INFO:teuthology.orchestra.run.smithi094.stdout:7 2023-10-16T20:04:33.915 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.910+0000 7f812b972640 1 -- 172.21.15.94:0/4255262667 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8124157510 msgr2=0x7f81240b68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:33.916 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.910+0000 7f812b972640 1 --2- 172.21.15.94:0/4255262667 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8124157510 0x7f81240b68e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f811c00cef0 tx=0x7f811c00a710 comp rx=0 tx=0).stop 2023-10-16T20:04:33.916 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.910+0000 7f812b972640 1 -- 172.21.15.94:0/4255262667 shutdown_connections 2023-10-16T20:04:33.916 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.910+0000 7f812b972640 1 --2- 172.21.15.94:0/4255262667 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8124157510 0x7f81240b68e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.916 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.910+0000 7f812b972640 1 --2- 172.21.15.94:0/4255262667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8124157130 0x7f81240c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.916 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.910+0000 7f812b972640 1 --2- 172.21.15.94:0/4255262667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81240cdee0 0x7f81240b2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:33.917 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.910+0000 7f812b972640 1 -- 172.21.15.94:0/4255262667 >> 172.21.15.94:0/4255262667 conn(0x7f81240715c0 msgr2=0x7f812415de80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:33.917 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.910+0000 7f812b972640 1 -- 172.21.15.94:0/4255262667 shutdown_connections 2023-10-16T20:04:33.917 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:04:33.910+0000 7f812b972640 1 -- 172.21.15.94:0/4255262667 wait complete. 2023-10-16T20:04:33.933 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-16T20:04:33.933 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-16T20:04:33.937 INFO:tasks.ceph.osd.0:Started 2023-10-16T20:04:33.937 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-16T20:04:33.938 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-16T20:04:33.941 INFO:tasks.ceph.osd.1:Started 2023-10-16T20:04:33.941 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-16T20:04:33.942 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-16T20:04:33.945 INFO:tasks.ceph.osd.2:Started 2023-10-16T20:04:33.945 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-16T20:04:33.946 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-16T20:04:33.952 INFO:tasks.ceph.osd.3:Started 2023-10-16T20:04:33.952 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-16T20:04:33.952 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-16T20:04:33.956 INFO:tasks.ceph.osd.4:Started 2023-10-16T20:04:33.956 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-16T20:04:33.957 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-16T20:04:33.960 INFO:tasks.ceph.osd.5:Started 2023-10-16T20:04:33.960 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-16T20:04:33.961 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-16T20:04:33.965 INFO:tasks.ceph.osd.6:Started 2023-10-16T20:04:33.966 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-16T20:04:33.966 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-16T20:04:33.971 INFO:tasks.ceph.osd.7:Started 2023-10-16T20:04:33.971 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:04:34.195 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.192+0000 7f9030194640 1 Processor -- start 2023-10-16T20:04:34.196 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.192+0000 7f9030194640 1 -- start start 2023-10-16T20:04:34.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.196+0000 7f9030194640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f902813f730 0x7f90280cc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:34.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.196+0000 7f9030194640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f902806a5a0 0x7f902815d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:34.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.196+0000 7f9030194640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90280abaa0 con 0x7f902813fb10 2023-10-16T20:04:34.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.196+0000 7f9030194640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90280ab890 con 0x7f902806a5a0 2023-10-16T20:04:34.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.196+0000 7f9030194640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90280abd80 con 0x7f902813f730 2023-10-16T20:04:34.198 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.196+0000 7f902e731640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f902806a5a0 0x7f902815d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:34.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.196+0000 7f902df30640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f902813f730 0x7f90280cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:34.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.196+0000 7f902e731640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f902806a5a0 0x7f902815d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:56852/0 (socket says 172.21.15.88:56852) 2023-10-16T20:04:34.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.196+0000 7f902e731640 1 -- 172.21.15.88:0/2417124326 learned_addr learned my addr 172.21.15.88:0/2417124326 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:04:34.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.196+0000 7f902d72f640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f902813fb10 0x7f9028069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:42160/0 (socket says 172.21.15.88:42160) 2023-10-16T20:04:34.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.196+0000 7f902df30640 1 -- 172.21.15.88:0/2417124326 >> v2:172.21.15.94:3300/0 conn(0x7f902806a5a0 msgr2=0x7f902815d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:04:34.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.196+0000 7f902df30640 1 --2- 172.21.15.88:0/2417124326 >> v2:172.21.15.94:3300/0 conn(0x7f902806a5a0 0x7f902815d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:34.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.196+0000 7f902df30640 1 -- 172.21.15.88:0/2417124326 >> v1:172.21.15.94:6789/0 conn(0x7f902813fb10 legacy=0x7f9028069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:34.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.196+0000 7f902df30640 1 -- 172.21.15.88:0/2417124326 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90280ac060 con 0x7f902813f730 2023-10-16T20:04:34.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.196+0000 7f902cf2e640 1 -- 172.21.15.88:0/2417124326 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1423600628 0 0) 0x7f90280abaa0 con 0x7f902813fb10 2023-10-16T20:04:34.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.196+0000 7f902e731640 1 --2- 172.21.15.88:0/2417124326 >> v2:172.21.15.94:3300/0 conn(0x7f902806a5a0 0x7f902815d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:04:34.202 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.200+0000 7f902df30640 1 --2- 172.21.15.88:0/2417124326 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f902813f730 0x7f90280cc120 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f90240097c0 tx=0x7f902401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38f96ad519c49e4f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:34.202 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.200+0000 7f902cf2e640 1 -- 172.21.15.88:0/2417124326 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90240ab670 con 0x7f902813f730 2023-10-16T20:04:34.202 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.200+0000 7f902cf2e640 1 -- 172.21.15.88:0/2417124326 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90240ab7d0 con 0x7f902813f730 2023-10-16T20:04:34.202 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.200+0000 7f902cf2e640 1 -- 172.21.15.88:0/2417124326 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90240abb60 con 0x7f902813f730 2023-10-16T20:04:34.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.200+0000 7f9030194640 1 -- 172.21.15.88:0/2417124326 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f902813f730 msgr2=0x7f90280cc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:34.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.200+0000 7f9030194640 1 --2- 172.21.15.88:0/2417124326 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f902813f730 0x7f90280cc120 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f90240097c0 tx=0x7f902401c450 comp rx=0 tx=0).stop 2023-10-16T20:04:34.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.200+0000 7f9030194640 1 -- 172.21.15.88:0/2417124326 shutdown_connections 2023-10-16T20:04:34.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.200+0000 7f9030194640 1 --2- 172.21.15.88:0/2417124326 >> v2:172.21.15.94:3300/0 conn(0x7f902806a5a0 0x7f902815d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:34.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.200+0000 7f9030194640 1 --2- 172.21.15.88:0/2417124326 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f902813f730 0x7f90280cc120 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:34.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.200+0000 7f9030194640 1 -- 172.21.15.88:0/2417124326 >> 172.21.15.88:0/2417124326 conn(0x7f90280ce5e0 msgr2=0x7f90280ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:34.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.200+0000 7f9030194640 1 -- 172.21.15.88:0/2417124326 shutdown_connections 2023-10-16T20:04:34.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.200+0000 7f9030194640 1 -- 172.21.15.88:0/2417124326 wait complete. 2023-10-16T20:04:34.206 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.200+0000 7f9030194640 1 Processor -- start 2023-10-16T20:04:34.206 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.200+0000 7f9030194640 1 -- start start 2023-10-16T20:04:34.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.204+0000 7f9030194640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f902806a5a0 0x7f90281543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:34.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.204+0000 7f9030194640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f902813f730 0x7f902815ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:34.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.204+0000 7f9030194640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f902813fb10 0x7f90280b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:34.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.204+0000 7f9030194640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9028154900 con 0x7f902813f730 2023-10-16T20:04:34.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.204+0000 7f9030194640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9028140ba0 con 0x7f902813fb10 2023-10-16T20:04:34.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.204+0000 7f9030194640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f902813ffd0 con 0x7f902806a5a0 2023-10-16T20:04:34.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.204+0000 7f902d72f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f902813f730 0x7f902815ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:34.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.204+0000 7f902d72f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f902813f730 0x7f902815ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43228/0 (socket says 172.21.15.88:43228) 2023-10-16T20:04:34.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.204+0000 7f902d72f640 1 -- 172.21.15.88:0/1286451365 learned_addr learned my addr 172.21.15.88:0/1286451365 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:04:34.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.204+0000 7f902e731640 1 --2- 172.21.15.88:0/1286451365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f902813fb10 0x7f90280b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:34.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.204+0000 7f902df30640 1 --2- 172.21.15.88:0/1286451365 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f902806a5a0 0x7f90281543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:34.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.204+0000 7f902d72f640 1 -- 172.21.15.88:0/1286451365 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f902806a5a0 msgr2=0x7f90281543c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:34.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.204+0000 7f902d72f640 1 --2- 172.21.15.88:0/1286451365 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f902806a5a0 0x7f90281543c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:34.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.204+0000 7f902d72f640 1 -- 172.21.15.88:0/1286451365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f902813fb10 msgr2=0x7f90280b2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:34.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.204+0000 7f902d72f640 1 --2- 172.21.15.88:0/1286451365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f902813fb10 0x7f90280b2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:34.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.204+0000 7f902d72f640 1 -- 172.21.15.88:0/1286451365 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9020009b30 con 0x7f902813f730 2023-10-16T20:04:34.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.204+0000 7f902d72f640 1 --2- 172.21.15.88:0/1286451365 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f902813f730 0x7f902815ec80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f90180011f0 tx=0x7f901800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:34.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.204+0000 7f901e7fc640 1 -- 172.21.15.88:0/1286451365 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90180074e0 con 0x7f902813f730 2023-10-16T20:04:34.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.204+0000 7f901e7fc640 1 -- 172.21.15.88:0/1286451365 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9018007640 con 0x7f902813f730 2023-10-16T20:04:34.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.204+0000 7f9030194640 1 -- 172.21.15.88:0/1286451365 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9024009420 con 0x7f902813f730 2023-10-16T20:04:34.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.204+0000 7f901e7fc640 1 -- 172.21.15.88:0/1286451365 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9018004a10 con 0x7f902813f730 2023-10-16T20:04:34.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.204+0000 7f9030194640 1 -- 172.21.15.88:0/1286451365 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9028140320 con 0x7f902813f730 2023-10-16T20:04:34.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.204+0000 7f901e7fc640 1 -- 172.21.15.88:0/1286451365 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f901801d440 con 0x7f902813f730 2023-10-16T20:04:34.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.204+0000 7f901e7fc640 1 -- 172.21.15.88:0/1286451365 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f9018012400 con 0x7f902813f730 2023-10-16T20:04:34.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.204+0000 7f9030194640 1 -- 172.21.15.88:0/1286451365 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f902804e9c0 con 0x7f902813f730 2023-10-16T20:04:34.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.208+0000 7f901e7fc640 1 -- 172.21.15.88:0/1286451365 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f9018018070 con 0x7f902813f730 2023-10-16T20:04:34.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.264+0000 7f9030194640 1 -- 172.21.15.88:0/1286451365 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90281408f0 con 0x7f902813f730 2023-10-16T20:04:34.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.264+0000 7f901e7fc640 1 -- 172.21.15.88:0/1286451365 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f901801d6f0 con 0x7f902813f730 2023-10-16T20:04:34.265 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:04:34.266 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":10,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:04:33.913430+0000","last_up_change":"0.000000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:04:34.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.264+0000 7f9030194640 1 -- 172.21.15.88:0/1286451365 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f902813f730 msgr2=0x7f902815ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:34.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.264+0000 7f9030194640 1 --2- 172.21.15.88:0/1286451365 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f902813f730 0x7f902815ec80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f90180011f0 tx=0x7f901800c9b0 comp rx=0 tx=0).stop 2023-10-16T20:04:34.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.264+0000 7f9030194640 1 -- 172.21.15.88:0/1286451365 shutdown_connections 2023-10-16T20:04:34.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.264+0000 7f9030194640 1 --2- 172.21.15.88:0/1286451365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f902813fb10 0x7f90280b2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:34.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.264+0000 7f9030194640 1 --2- 172.21.15.88:0/1286451365 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f902813f730 0x7f902815ec80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:34.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.264+0000 7f9030194640 1 --2- 172.21.15.88:0/1286451365 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f902806a5a0 0x7f90281543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:34.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.264+0000 7f9030194640 1 -- 172.21.15.88:0/1286451365 >> 172.21.15.88:0/1286451365 conn(0x7f90280ce5e0 msgr2=0x7f9028070130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:34.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.264+0000 7f9030194640 1 -- 172.21.15.88:0/1286451365 shutdown_connections 2023-10-16T20:04:34.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:34.264+0000 7f9030194640 1 -- 172.21.15.88:0/1286451365 wait complete. 2023-10-16T20:04:34.286 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-16T20:04:34.286 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-16T20:04:34.376 INFO:tasks.ceph.mgr.y.smithi088.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-10-16T20:04:34.377 INFO:tasks.ceph.mgr.y.smithi088.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-10-16T20:04:34.377 INFO:tasks.ceph.mgr.y.smithi088.stderr: from numpy import show_config as show_numpy_config 2023-10-16T20:04:34.397 INFO:tasks.ceph.osd.0.smithi088.stderr:2023-10-16T20:04:34.396+0000 7ff8f177e640 -1 Falling back to public interface 2023-10-16T20:04:34.417 INFO:tasks.ceph.mgr.x.smithi094.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-10-16T20:04:34.417 INFO:tasks.ceph.mgr.x.smithi094.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-10-16T20:04:34.417 INFO:tasks.ceph.mgr.x.smithi094.stderr: from numpy import show_config as show_numpy_config 2023-10-16T20:04:34.425 INFO:tasks.ceph.mgr.y.smithi088.stderr:2023-10-16T20:04:34.424+0000 7f65fc144180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T20:04:34.460 INFO:tasks.ceph.mgr.x.smithi094.stderr:2023-10-16T20:04:34.454+0000 7fa4a1e0c180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T20:04:34.561 INFO:tasks.ceph.mgr.y.smithi088.stderr:2023-10-16T20:04:34.556+0000 7f65fc144180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T20:04:34.595 INFO:tasks.ceph.mgr.x.smithi094.stderr:2023-10-16T20:04:34.590+0000 7fa4a1e0c180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T20:04:34.614 INFO:tasks.ceph.osd.5.smithi094.stderr:2023-10-16T20:04:34.610+0000 7f1a663e1640 -1 Falling back to public interface 2023-10-16T20:04:34.618 INFO:tasks.ceph.osd.4.smithi094.stderr:2023-10-16T20:04:34.614+0000 7f8d95f50640 -1 Falling back to public interface 2023-10-16T20:04:34.621 INFO:tasks.ceph.osd.1.smithi088.stderr:2023-10-16T20:04:34.620+0000 7f8d4e4f3640 -1 Falling back to public interface 2023-10-16T20:04:34.626 INFO:tasks.ceph.osd.6.smithi094.stderr:2023-10-16T20:04:34.622+0000 7fb44922c640 -1 Falling back to public interface 2023-10-16T20:04:34.646 INFO:tasks.ceph.osd.7.smithi094.stderr:2023-10-16T20:04:34.642+0000 7f7c1d457640 -1 Falling back to public interface 2023-10-16T20:04:34.661 INFO:tasks.ceph.osd.2.smithi088.stderr:2023-10-16T20:04:34.660+0000 7f6f081bd640 -1 Falling back to public interface 2023-10-16T20:04:34.709 INFO:tasks.ceph.osd.3.smithi088.stderr:2023-10-16T20:04:34.708+0000 7f0f9caa0640 -1 Falling back to public interface 2023-10-16T20:04:34.743 INFO:tasks.ceph.mgr.y.smithi088.stderr:2023-10-16T20:04:34.740+0000 7f65fc144180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T20:04:34.777 INFO:tasks.ceph.mgr.x.smithi094.stderr:2023-10-16T20:04:34.770+0000 7fa4a1e0c180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T20:04:34.816 INFO:tasks.ceph.mgr.y.smithi088.stderr:2023-10-16T20:04:34.812+0000 7f65fc144180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T20:04:34.849 INFO:tasks.ceph.mgr.x.smithi094.stderr:2023-10-16T20:04:34.846+0000 7fa4a1e0c180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T20:04:34.954 INFO:tasks.ceph.mgr.y.smithi088.stderr:2023-10-16T20:04:34.952+0000 7f65fc144180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T20:04:34.989 INFO:tasks.ceph.mgr.x.smithi094.stderr:2023-10-16T20:04:34.982+0000 7fa4a1e0c180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T20:04:35.092 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-16T20:04:35.221 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.220+0000 7f4cee89a640 1 Processor -- start 2023-10-16T20:04:35.221 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.220+0000 7f4cee89a640 1 -- start start 2023-10-16T20:04:35.223 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.220+0000 7f4cee89a640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4ce815a810 0x7f4ce8070320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:35.223 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.220+0000 7f4cee89a640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4ce80c9560 0x7f4ce8071b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:35.223 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.220+0000 7f4cee89a640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ce8070a00 con 0x7f4ce8072040 2023-10-16T20:04:35.223 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.220+0000 7f4cee89a640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ce80724d0 con 0x7f4ce80c9560 2023-10-16T20:04:35.223 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.220+0000 7f4cee89a640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce8070ce0 con 0x7f4ce815a810 2023-10-16T20:04:35.224 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.220+0000 7f4ce77fe640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4ce80c9560 0x7f4ce8071b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:35.224 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.220+0000 7f4ce77fe640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4ce80c9560 0x7f4ce8071b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43240/0 (socket says 172.21.15.88:43240) 2023-10-16T20:04:35.224 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.220+0000 7f4ce77fe640 1 -- 172.21.15.88:0/3265275703 learned_addr learned my addr 172.21.15.88:0/3265275703 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:04:35.224 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.220+0000 7f4ce7fff640 1 --2- 172.21.15.88:0/3265275703 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4ce815a810 0x7f4ce8070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:35.225 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.220+0000 7f4ce77fe640 1 -- 172.21.15.88:0/3265275703 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4ce815a810 msgr2=0x7f4ce8070320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:04:35.225 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.220+0000 7f4ce77fe640 1 --2- 172.21.15.88:0/3265275703 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4ce815a810 0x7f4ce8070320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:35.225 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.220+0000 7f4ce77fe640 1 -- 172.21.15.88:0/3265275703 >> v1:172.21.15.94:6789/0 conn(0x7f4ce8072040 legacy=0x7f4ce815e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:35.226 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.220+0000 7f4ce77fe640 1 -- 172.21.15.88:0/3265275703 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce8070fc0 con 0x7f4ce80c9560 2023-10-16T20:04:35.226 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.224+0000 7f4ce77fe640 1 --2- 172.21.15.88:0/3265275703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ce80c9560 0x7f4ce8071b00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc009870 tx=0x7f4cdc01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ab1f7e3e953f62 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:35.227 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.224+0000 7f4ce6ffd640 1 -- 172.21.15.88:0/3265275703 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc0aa670 con 0x7f4ce80c9560 2023-10-16T20:04:35.227 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.224+0000 7f4ce6ffd640 1 -- 172.21.15.88:0/3265275703 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cdc0aa7d0 con 0x7f4ce80c9560 2023-10-16T20:04:35.227 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.224+0000 7f4ce6ffd640 1 -- 172.21.15.88:0/3265275703 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc0aab60 con 0x7f4ce80c9560 2023-10-16T20:04:35.228 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.224+0000 7f4cee89a640 1 -- 172.21.15.88:0/3265275703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ce80c9560 msgr2=0x7f4ce8071b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:35.228 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.224+0000 7f4cee89a640 1 --2- 172.21.15.88:0/3265275703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ce80c9560 0x7f4ce8071b00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc009870 tx=0x7f4cdc01c510 comp rx=0 tx=0).stop 2023-10-16T20:04:35.228 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.224+0000 7f4cee89a640 1 -- 172.21.15.88:0/3265275703 shutdown_connections 2023-10-16T20:04:35.228 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.224+0000 7f4cee89a640 1 --2- 172.21.15.88:0/3265275703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ce80c9560 0x7f4ce8071b00 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:35.229 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.224+0000 7f4cee89a640 1 --2- 172.21.15.88:0/3265275703 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4ce815a810 0x7f4ce8070320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:35.229 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.224+0000 7f4cee89a640 1 -- 172.21.15.88:0/3265275703 >> 172.21.15.88:0/3265275703 conn(0x7f4ce8069e60 msgr2=0x7f4ce806a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:35.229 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.224+0000 7f4cee89a640 1 -- 172.21.15.88:0/3265275703 shutdown_connections 2023-10-16T20:04:35.229 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.224+0000 7f4cee89a640 1 -- 172.21.15.88:0/3265275703 wait complete. 2023-10-16T20:04:35.231 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.228+0000 7f4cee89a640 1 Processor -- start 2023-10-16T20:04:35.231 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.228+0000 7f4cee89a640 1 -- start start 2023-10-16T20:04:35.231 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.228+0000 7f4cee89a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ce8072040 0x7f4ce80c46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:35.232 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.228+0000 7f4cee89a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ce80c9560 0x7f4ce8164bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:35.232 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.228+0000 7f4cee89a640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4ce815a810 0x7f4ce814bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:35.232 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.228+0000 7f4cee89a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce8144a50 con 0x7f4ce80c9560 2023-10-16T20:04:35.232 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.228+0000 7f4cee89a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce80c52e0 con 0x7f4ce8072040 2023-10-16T20:04:35.233 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.228+0000 7f4cee89a640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce8147ca0 con 0x7f4ce815a810 2023-10-16T20:04:35.233 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.228+0000 7f4ce77fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ce80c9560 0x7f4ce8164bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:35.233 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.228+0000 7f4ce77fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ce80c9560 0x7f4ce8164bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43256/0 (socket says 172.21.15.88:43256) 2023-10-16T20:04:35.233 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.228+0000 7f4ce77fe640 1 -- 172.21.15.88:0/4181189517 learned_addr learned my addr 172.21.15.88:0/4181189517 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:04:35.233 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.228+0000 7f4ce7fff640 1 --2- 172.21.15.88:0/4181189517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ce8072040 0x7f4ce80c46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:35.234 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.228+0000 7f4cece37640 1 --2- 172.21.15.88:0/4181189517 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4ce815a810 0x7f4ce814bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:35.234 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.228+0000 7f4ce77fe640 1 -- 172.21.15.88:0/4181189517 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4ce815a810 msgr2=0x7f4ce814bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:35.234 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.228+0000 7f4ce77fe640 1 --2- 172.21.15.88:0/4181189517 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4ce815a810 0x7f4ce814bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:35.234 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.228+0000 7f4ce77fe640 1 -- 172.21.15.88:0/4181189517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ce8072040 msgr2=0x7f4ce80c46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:35.234 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.228+0000 7f4ce77fe640 1 --2- 172.21.15.88:0/4181189517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ce8072040 0x7f4ce80c46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:35.235 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.228+0000 7f4ce77fe640 1 -- 172.21.15.88:0/4181189517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cdc0094d0 con 0x7f4ce80c9560 2023-10-16T20:04:35.235 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.228+0000 7f4ce77fe640 1 --2- 172.21.15.88:0/4181189517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ce80c9560 0x7f4ce8164bf0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc0ac040 tx=0x7f4cdc01ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:35.236 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.228+0000 7f4ce4ff9640 1 -- 172.21.15.88:0/4181189517 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc0b3800 con 0x7f4ce80c9560 2023-10-16T20:04:35.236 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.228+0000 7f4ce4ff9640 1 -- 172.21.15.88:0/4181189517 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cdc0b3960 con 0x7f4ce80c9560 2023-10-16T20:04:35.236 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.228+0000 7f4cee89a640 1 -- 172.21.15.88:0/4181189517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce80c4d20 con 0x7f4ce80c9560 2023-10-16T20:04:35.236 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.228+0000 7f4ce4ff9640 1 -- 172.21.15.88:0/4181189517 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc0b3d10 con 0x7f4ce80c9560 2023-10-16T20:04:35.237 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.228+0000 7f4cee89a640 1 -- 172.21.15.88:0/4181189517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce81646d0 con 0x7f4ce80c9560 2023-10-16T20:04:35.237 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.228+0000 7f4ce4ff9640 1 -- 172.21.15.88:0/4181189517 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4cdc0bd6f0 con 0x7f4ce80c9560 2023-10-16T20:04:35.237 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.232+0000 7f4ce4ff9640 1 -- 172.21.15.88:0/4181189517 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f4cdc0b2af0 con 0x7f4ce80c9560 2023-10-16T20:04:35.237 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.232+0000 7f4cee89a640 1 -- 172.21.15.88:0/4181189517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ce8073f30 con 0x7f4ce80c9560 2023-10-16T20:04:35.237 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.232+0000 7f4ce4ff9640 1 -- 172.21.15.88:0/4181189517 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f4cdc0c3060 con 0x7f4ce80c9560 2023-10-16T20:04:35.292 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.292+0000 7f4cee89a640 1 -- 172.21.15.88:0/4181189517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ce80cdee0 con 0x7f4ce80c9560 2023-10-16T20:04:35.293 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.292+0000 7f4ce4ff9640 1 -- 172.21.15.88:0/4181189517 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f4cdc0b2cd0 con 0x7f4ce80c9560 2023-10-16T20:04:35.293 INFO:teuthology.misc.health.smithi088.stdout: 2023-10-16T20:04:35.294 INFO:teuthology.misc.health.smithi088.stdout:{"epoch":10,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:04:33.913430+0000","last_up_change":"0.000000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:04:35.295 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.292+0000 7f4cee89a640 1 -- 172.21.15.88:0/4181189517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ce80c9560 msgr2=0x7f4ce8164bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:35.295 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.292+0000 7f4cee89a640 1 --2- 172.21.15.88:0/4181189517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ce80c9560 0x7f4ce8164bf0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc0ac040 tx=0x7f4cdc01ad50 comp rx=0 tx=0).stop 2023-10-16T20:04:35.295 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.292+0000 7f4cee89a640 1 -- 172.21.15.88:0/4181189517 shutdown_connections 2023-10-16T20:04:35.295 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.292+0000 7f4cee89a640 1 --2- 172.21.15.88:0/4181189517 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4ce815a810 0x7f4ce814bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:35.296 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.292+0000 7f4cee89a640 1 --2- 172.21.15.88:0/4181189517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ce80c9560 0x7f4ce8164bf0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:35.296 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.292+0000 7f4cee89a640 1 --2- 172.21.15.88:0/4181189517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ce8072040 0x7f4ce80c46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:35.296 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.292+0000 7f4cee89a640 1 -- 172.21.15.88:0/4181189517 >> 172.21.15.88:0/4181189517 conn(0x7f4ce8069e60 msgr2=0x7f4ce8150f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:35.296 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.292+0000 7f4cee89a640 1 -- 172.21.15.88:0/4181189517 shutdown_connections 2023-10-16T20:04:35.296 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:35.292+0000 7f4cee89a640 1 -- 172.21.15.88:0/4181189517 wait complete. 2023-10-16T20:04:35.311 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-16T20:04:35.664 INFO:tasks.ceph.mgr.x.smithi094.stderr:2023-10-16T20:04:35.658+0000 7fa4a1e0c180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T20:04:35.665 INFO:tasks.ceph.mgr.y.smithi088.stderr:2023-10-16T20:04:35.664+0000 7f65fc144180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T20:04:35.751 INFO:tasks.ceph.mgr.x.smithi094.stderr:2023-10-16T20:04:35.746+0000 7fa4a1e0c180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T20:04:35.751 INFO:tasks.ceph.mgr.y.smithi088.stderr:2023-10-16T20:04:35.748+0000 7f65fc144180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T20:04:35.942 INFO:tasks.ceph.mgr.y.smithi088.stderr:2023-10-16T20:04:35.940+0000 7f65fc144180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T20:04:35.943 INFO:tasks.ceph.mgr.x.smithi094.stderr:2023-10-16T20:04:35.938+0000 7fa4a1e0c180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T20:04:36.026 INFO:tasks.ceph.mgr.x.smithi094.stderr:2023-10-16T20:04:36.022+0000 7fa4a1e0c180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T20:04:36.027 INFO:tasks.ceph.mgr.y.smithi088.stderr:2023-10-16T20:04:36.024+0000 7f65fc144180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T20:04:36.355 INFO:tasks.ceph.mgr.y.smithi088.stderr:2023-10-16T20:04:36.352+0000 7f65fc144180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T20:04:36.358 INFO:tasks.ceph.mgr.x.smithi094.stderr:2023-10-16T20:04:36.354+0000 7fa4a1e0c180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T20:04:36.424 INFO:tasks.ceph.mgr.y.smithi088.stderr:2023-10-16T20:04:36.424+0000 7f65fc144180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T20:04:36.427 INFO:tasks.ceph.mgr.x.smithi094.stderr:2023-10-16T20:04:36.422+0000 7fa4a1e0c180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T20:04:37.446 INFO:tasks.ceph.mgr.y.smithi088.stderr:2023-10-16T20:04:37.444+0000 7f65a39a6640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-16T20:04:37.556 INFO:tasks.ceph.mgr.y.smithi088.stderr:2023-10-16T20:04:37.444+0000 7f65a39a6640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-16T20:04:38.327 INFO:tasks.ceph.osd.0.smithi088.stderr:2023-10-16T20:04:38.324+0000 7ff8f177e640 -1 osd.0 0 log_to_monitors true 2023-10-16T20:04:38.463 INFO:tasks.ceph.osd.1.smithi088.stderr:2023-10-16T20:04:38.460+0000 7f8d4e4f3640 -1 osd.1 0 log_to_monitors true 2023-10-16T20:04:38.478 INFO:tasks.ceph.osd.5.smithi094.stderr:2023-10-16T20:04:38.474+0000 7f1a663e1640 -1 osd.5 0 log_to_monitors true 2023-10-16T20:04:38.538 INFO:tasks.ceph.osd.2.smithi088.stderr:2023-10-16T20:04:38.536+0000 7f6f081bd640 -1 osd.2 0 log_to_monitors true 2023-10-16T20:04:38.613 INFO:tasks.ceph.osd.4.smithi094.stderr:2023-10-16T20:04:38.606+0000 7f8d95f50640 -1 osd.4 0 log_to_monitors true 2023-10-16T20:04:38.645 INFO:tasks.ceph.osd.6.smithi094.stderr:2023-10-16T20:04:38.642+0000 7fb44922c640 -1 osd.6 0 log_to_monitors true 2023-10-16T20:04:38.662 INFO:tasks.ceph.osd.3.smithi088.stderr:2023-10-16T20:04:38.660+0000 7f0f9caa0640 -1 osd.3 0 log_to_monitors true 2023-10-16T20:04:38.672 INFO:tasks.ceph.osd.7.smithi094.stderr:2023-10-16T20:04:38.666+0000 7f7c1d457640 -1 osd.7 0 log_to_monitors true 2023-10-16T20:04:40.360 INFO:tasks.ceph.osd.0.smithi088.stderr:2023-10-16T20:04:40.355+0000 7ff8edf3a640 -1 osd.0 0 waiting for initial osdmap 2023-10-16T20:04:40.385 INFO:tasks.ceph.osd.0.smithi088.stderr:2023-10-16T20:04:40.383+0000 7ff8e555b640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T20:04:42.118 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-16T20:04:42.257 INFO:tasks.ceph.osd.6.smithi094.stderr:2023-10-16T20:04:42.254+0000 7fb444820640 -1 osd.6 0 waiting for initial osdmap 2023-10-16T20:04:42.270 INFO:tasks.ceph.osd.7.smithi094.stderr:2023-10-16T20:04:42.266+0000 7f7c19c13640 -1 osd.7 0 waiting for initial osdmap 2023-10-16T20:04:42.284 INFO:tasks.ceph.osd.5.smithi094.stderr:2023-10-16T20:04:42.278+0000 7f1a62389640 -1 osd.5 0 waiting for initial osdmap 2023-10-16T20:04:42.297 INFO:tasks.ceph.osd.4.smithi094.stderr:2023-10-16T20:04:42.294+0000 7f8d9270c640 -1 osd.4 0 waiting for initial osdmap 2023-10-16T20:04:42.297 INFO:tasks.ceph.osd.6.smithi094.stderr:2023-10-16T20:04:42.294+0000 7fb43c655640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T20:04:42.317 INFO:tasks.ceph.osd.7.smithi094.stderr:2023-10-16T20:04:42.314+0000 7f7c11234640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T20:04:42.323 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.319+0000 7fef9e1d9640 1 Processor -- start 2023-10-16T20:04:42.324 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.319+0000 7fef9e1d9640 1 -- start start 2023-10-16T20:04:42.324 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.319+0000 7fef9e1d9640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fef9815cb60 0x7fef980cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:42.325 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.323+0000 7fef9e1d9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef980cf550 0x7fef980cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:42.325 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.323+0000 7fef9e1d9640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fef980b4600 0x7fef980b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:42.325 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.323+0000 7fef9e1d9640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef981606d0 con 0x7fef980b4600 2023-10-16T20:04:42.325 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.323+0000 7fef9e1d9640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef980cfed0 con 0x7fef9815cb60 2023-10-16T20:04:42.325 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.323+0000 7fef9e1d9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9815cf40 con 0x7fef980cf550 2023-10-16T20:04:42.326 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.323+0000 7fef977fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fef9815cb60 0x7fef980cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:42.327 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.323+0000 7fef96ffd640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef980cf550 0x7fef980cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:42.327 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.323+0000 7fef977fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fef9815cb60 0x7fef980cda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:54756/0 (socket says 172.21.15.88:54756) 2023-10-16T20:04:42.327 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.323+0000 7fef977fe640 1 -- 172.21.15.88:0/4142931439 learned_addr learned my addr 172.21.15.88:0/4142931439 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:04:42.327 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.323+0000 7fef97fff640 1 --2- 172.21.15.88:0/4142931439 >> v2:172.21.15.88:3300/0 conn(0x7fef980b4600 0x7fef980b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:42.327 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.323+0000 7fef96ffd640 1 -- 172.21.15.88:0/4142931439 >> v2:172.21.15.94:3300/0 conn(0x7fef9815cb60 msgr2=0x7fef980cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:04:42.328 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.323+0000 7fef96ffd640 1 --2- 172.21.15.88:0/4142931439 >> v2:172.21.15.94:3300/0 conn(0x7fef9815cb60 0x7fef980cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:42.328 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.323+0000 7fef96ffd640 1 -- 172.21.15.88:0/4142931439 >> v2:172.21.15.88:3300/0 conn(0x7fef980b4600 msgr2=0x7fef980b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:04:42.328 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.323+0000 7fef96ffd640 1 --2- 172.21.15.88:0/4142931439 >> v2:172.21.15.88:3300/0 conn(0x7fef980b4600 0x7fef980b4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:42.328 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.323+0000 7fef96ffd640 1 -- 172.21.15.88:0/4142931439 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef98160920 con 0x7fef980cf550 2023-10-16T20:04:42.328 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.323+0000 7fef97fff640 1 --2- 172.21.15.88:0/4142931439 >> v2:172.21.15.88:3300/0 conn(0x7fef980b4600 0x7fef980b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:04:42.329 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.327+0000 7fef96ffd640 1 --2- 172.21.15.88:0/4142931439 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef980cf550 0x7fef980cf990 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fef980b5710 tx=0x7fef8c01c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=170e74730fc4db8b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:42.329 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.327+0000 7fef967fc640 1 -- 172.21.15.88:0/4142931439 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c0ab5f0 con 0x7fef980cf550 2023-10-16T20:04:42.330 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.327+0000 7fef967fc640 1 -- 172.21.15.88:0/4142931439 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8c0ab750 con 0x7fef980cf550 2023-10-16T20:04:42.330 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.327+0000 7fef967fc640 1 -- 172.21.15.88:0/4142931439 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c0abae0 con 0x7fef980cf550 2023-10-16T20:04:42.330 INFO:tasks.ceph.osd.5.smithi094.stderr:2023-10-16T20:04:42.326+0000 7f1a5a1be640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T20:04:42.331 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.327+0000 7fef9e1d9640 1 -- 172.21.15.88:0/4142931439 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef980cf550 msgr2=0x7fef980cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:42.331 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.327+0000 7fef9e1d9640 1 --2- 172.21.15.88:0/4142931439 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef980cf550 0x7fef980cf990 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fef980b5710 tx=0x7fef8c01c430 comp rx=0 tx=0).stop 2023-10-16T20:04:42.331 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.327+0000 7fef9e1d9640 1 -- 172.21.15.88:0/4142931439 shutdown_connections 2023-10-16T20:04:42.332 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.327+0000 7fef9e1d9640 1 --2- 172.21.15.88:0/4142931439 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef980cf550 0x7fef980cf990 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:42.332 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.327+0000 7fef9e1d9640 1 --2- 172.21.15.88:0/4142931439 >> v2:172.21.15.88:3300/0 conn(0x7fef980b4600 0x7fef980b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:42.332 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.327+0000 7fef9e1d9640 1 --2- 172.21.15.88:0/4142931439 >> v2:172.21.15.94:3300/0 conn(0x7fef9815cb60 0x7fef980cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:42.332 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.327+0000 7fef9e1d9640 1 -- 172.21.15.88:0/4142931439 >> 172.21.15.88:0/4142931439 conn(0x7fef98071590 msgr2=0x7fef980719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:42.332 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.327+0000 7fef9e1d9640 1 -- 172.21.15.88:0/4142931439 shutdown_connections 2023-10-16T20:04:42.333 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.327+0000 7fef9e1d9640 1 -- 172.21.15.88:0/4142931439 wait complete. 2023-10-16T20:04:42.333 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.331+0000 7fef9e1d9640 1 Processor -- start 2023-10-16T20:04:42.333 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.331+0000 7fef9e1d9640 1 -- start start 2023-10-16T20:04:42.334 INFO:tasks.ceph.osd.4.smithi094.stderr:2023-10-16T20:04:42.330+0000 7f8d89d2d640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T20:04:42.335 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.331+0000 7fef9e1d9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef980b4600 0x7fef9814fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:42.335 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.331+0000 7fef9e1d9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef980cf550 0x7fef980c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:42.335 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.331+0000 7fef9e1d9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef9815cb60 0x7fef980ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:42.336 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.331+0000 7fef9e1d9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef980b20e0 con 0x7fef9815cb60 2023-10-16T20:04:42.336 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.331+0000 7fef9e1d9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef980b2330 con 0x7fef980b4600 2023-10-16T20:04:42.336 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.331+0000 7fef9e1d9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef980b2610 con 0x7fef980cf550 2023-10-16T20:04:42.337 INFO:tasks.ceph.osd.3.smithi088.stderr:2023-10-16T20:04:42.331+0000 7f0f98a48640 -1 osd.3 0 waiting for initial osdmap 2023-10-16T20:04:42.338 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.331+0000 7fef96ffd640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef980cf550 0x7fef980c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:42.338 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.331+0000 7fef96ffd640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef980cf550 0x7fef980c9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:42294/0 (socket says 172.21.15.88:42294) 2023-10-16T20:04:42.338 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.331+0000 7fef96ffd640 1 -- 172.21.15.88:0/3812011883 learned_addr learned my addr 172.21.15.88:0/3812011883 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:04:42.338 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.331+0000 7fef977fe640 1 --2- 172.21.15.88:0/3812011883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef980b4600 0x7fef9814fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:42.338 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.331+0000 7fef96ffd640 1 -- 172.21.15.88:0/3812011883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef980b4600 msgr2=0x7fef9814fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:42.339 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.331+0000 7fef96ffd640 1 --2- 172.21.15.88:0/3812011883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef980b4600 0x7fef9814fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:42.339 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.331+0000 7fef97fff640 1 --2- 172.21.15.88:0/3812011883 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef9815cb60 0x7fef980ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:42.339 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.331+0000 7fef96ffd640 1 -- 172.21.15.88:0/3812011883 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef9815cb60 msgr2=0x7fef980ca190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:42.339 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.331+0000 7fef96ffd640 1 --2- 172.21.15.88:0/3812011883 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef9815cb60 0x7fef980ca190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:42.339 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.331+0000 7fef96ffd640 1 -- 172.21.15.88:0/3812011883 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8c009420 con 0x7fef980cf550 2023-10-16T20:04:42.339 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.331+0000 7fef97fff640 1 --2- 172.21.15.88:0/3812011883 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef9815cb60 0x7fef980ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:04:42.340 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.335+0000 7fef96ffd640 1 --2- 172.21.15.88:0/3812011883 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef980cf550 0x7fef980c9c50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fef8c01aae0 tx=0x7fef8c01ac70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:42.340 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.335+0000 7fef7ffff640 1 -- 172.21.15.88:0/3812011883 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c0bf610 con 0x7fef980cf550 2023-10-16T20:04:42.340 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.335+0000 7fef7ffff640 1 -- 172.21.15.88:0/3812011883 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8c0bf790 con 0x7fef980cf550 2023-10-16T20:04:42.340 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.335+0000 7fef7ffff640 1 -- 172.21.15.88:0/3812011883 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c0bfb40 con 0x7fef980cf550 2023-10-16T20:04:42.340 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.335+0000 7fef9e1d9640 1 -- 172.21.15.88:0/3812011883 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef980b28f0 con 0x7fef980cf550 2023-10-16T20:04:42.341 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.335+0000 7fef9e1d9640 1 -- 172.21.15.88:0/3812011883 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef98150160 con 0x7fef980cf550 2023-10-16T20:04:42.341 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.335+0000 7fef9e1d9640 1 -- 172.21.15.88:0/3812011883 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef98073f30 con 0x7fef980cf550 2023-10-16T20:04:42.342 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.335+0000 7fef7ffff640 1 -- 172.21.15.88:0/3812011883 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 4) v1 ==== 85794+0+0 (secure 0 0 0) 0x7fef8c0bfca0 con 0x7fef980cf550 2023-10-16T20:04:42.342 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.339+0000 7fef7ffff640 1 --2- 172.21.15.88:0/3812011883 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fef84066410 0x7fef840688d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:42.342 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.339+0000 7fef7ffff640 1 -- 172.21.15.88:0/3812011883 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2511+0+0 (secure 0 0 0) 0x7fef8c0a8050 con 0x7fef980cf550 2023-10-16T20:04:42.342 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.339+0000 7fef977fe640 1 --2- 172.21.15.88:0/3812011883 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fef84066410 0x7fef840688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:42.343 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.339+0000 7fef977fe640 1 --2- 172.21.15.88:0/3812011883 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fef84066410 0x7fef840688d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fef980b7620 tx=0x7fef88008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:42.346 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.343+0000 7fef7ffff640 1 -- 172.21.15.88:0/3812011883 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fef8c0ebb00 con 0x7fef980cf550 2023-10-16T20:04:42.350 INFO:tasks.ceph.osd.2.smithi088.stderr:2023-10-16T20:04:42.347+0000 7f6f04165640 -1 osd.2 0 waiting for initial osdmap 2023-10-16T20:04:42.373 INFO:tasks.ceph.osd.3.smithi088.stderr:2023-10-16T20:04:42.371+0000 7f0f9087d640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T20:04:42.390 INFO:tasks.ceph.osd.1.smithi088.stderr:2023-10-16T20:04:42.387+0000 7f8d4a49b640 -1 osd.1 0 waiting for initial osdmap 2023-10-16T20:04:42.405 INFO:tasks.ceph.osd.2.smithi088.stderr:2023-10-16T20:04:42.403+0000 7f6efbf9a640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T20:04:42.429 INFO:tasks.ceph.osd.1.smithi088.stderr:2023-10-16T20:04:42.427+0000 7f8d422d0640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T20:04:42.514 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.511+0000 7fef9e1d9640 1 -- 172.21.15.88:0/3812011883 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef980b2da0 con 0x7fef980cf550 2023-10-16T20:04:42.515 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.511+0000 7fef7ffff640 1 -- 172.21.15.88:0/3812011883 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+7565 (secure 0 0 0) 0x7fef8c0ef120 con 0x7fef980cf550 2023-10-16T20:04:42.515 INFO:teuthology.misc.health.smithi088.stdout: 2023-10-16T20:04:42.515 INFO:teuthology.misc.health.smithi088.stdout:{"epoch":14,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:04:42.223691+0000","last_up_change":"2023-10-16T20:04:40.443326+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:04:42.517 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.515+0000 7fef9e1d9640 1 -- 172.21.15.88:0/3812011883 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fef84066410 msgr2=0x7fef840688d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:42.517 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.515+0000 7fef9e1d9640 1 --2- 172.21.15.88:0/3812011883 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fef84066410 0x7fef840688d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fef980b7620 tx=0x7fef88008040 comp rx=0 tx=0).stop 2023-10-16T20:04:42.517 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.515+0000 7fef9e1d9640 1 -- 172.21.15.88:0/3812011883 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef980cf550 msgr2=0x7fef980c9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:42.517 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.515+0000 7fef9e1d9640 1 --2- 172.21.15.88:0/3812011883 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef980cf550 0x7fef980c9c50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fef8c01aae0 tx=0x7fef8c01ac70 comp rx=0 tx=0).stop 2023-10-16T20:04:42.518 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.515+0000 7fef9e1d9640 1 -- 172.21.15.88:0/3812011883 shutdown_connections 2023-10-16T20:04:42.518 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.515+0000 7fef9e1d9640 1 --2- 172.21.15.88:0/3812011883 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fef84066410 0x7fef840688d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:42.518 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.515+0000 7fef9e1d9640 1 --2- 172.21.15.88:0/3812011883 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef980cf550 0x7fef980c9c50 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:42.518 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.515+0000 7fef9e1d9640 1 --2- 172.21.15.88:0/3812011883 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef9815cb60 0x7fef980ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:42.518 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.515+0000 7fef9e1d9640 1 --2- 172.21.15.88:0/3812011883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef980b4600 0x7fef9814fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:42.518 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.515+0000 7fef9e1d9640 1 -- 172.21.15.88:0/3812011883 >> 172.21.15.88:0/3812011883 conn(0x7fef98071590 msgr2=0x7fef980d2120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:42.519 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.515+0000 7fef9e1d9640 1 -- 172.21.15.88:0/3812011883 shutdown_connections 2023-10-16T20:04:42.519 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:42.515+0000 7fef9e1d9640 1 -- 172.21.15.88:0/3812011883 wait complete. 2023-10-16T20:04:42.534 DEBUG:teuthology.misc:1 of 8 OSDs are up 2023-10-16T20:04:49.340 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-16T20:04:49.478 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.475+0000 7f2372e9e640 1 Processor -- start 2023-10-16T20:04:49.478 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.475+0000 7f2372e9e640 1 -- start start 2023-10-16T20:04:49.479 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.475+0000 7f2372e9e640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f236c15cb60 0x7f236c0cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:49.479 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.475+0000 7f2372e9e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f236c0d19a0 0x7f236c0cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:49.479 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.475+0000 7f2372e9e640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f236c1606d0 con 0x7f236c0cf550 2023-10-16T20:04:49.480 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.475+0000 7f2372e9e640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f236c0cff00 con 0x7f236c15cb60 2023-10-16T20:04:49.480 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.475+0000 7f2372e9e640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f236c15cf40 con 0x7f236c0d19a0 2023-10-16T20:04:49.480 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.475+0000 7f2370c3a640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f236c15cb60 0x7f236c0cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:49.480 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.475+0000 7f2363fff640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f236c0cf550 0x7f236c0b4600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:35760/0 (socket says 172.21.15.88:35760) 2023-10-16T20:04:49.480 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.475+0000 7f2363fff640 1 -- 172.21.15.88:0/3599611244 learned_addr learned my addr 172.21.15.88:0/3599611244 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:04:49.481 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.475+0000 7f2370c3a640 1 -- 172.21.15.88:0/3599611244 >> v2:172.21.15.94:3300/0 conn(0x7f236c0d19a0 msgr2=0x7f236c0cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:04:49.481 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.475+0000 7f2370c3a640 1 --2- 172.21.15.88:0/3599611244 >> v2:172.21.15.94:3300/0 conn(0x7f236c0d19a0 0x7f236c0cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:49.481 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.475+0000 7f2370c3a640 1 -- 172.21.15.88:0/3599611244 >> v1:172.21.15.88:6789/0 conn(0x7f236c0cf550 legacy=0x7f236c0b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:49.482 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.475+0000 7f2370c3a640 1 -- 172.21.15.88:0/3599611244 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c160920 con 0x7f236c15cb60 2023-10-16T20:04:49.482 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.475+0000 7f23637fe640 1 -- 172.21.15.88:0/3599611244 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3548716548 0 0) 0x7f236c1606d0 con 0x7f236c0cf550 2023-10-16T20:04:49.482 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.479+0000 7f2370c3a640 1 --2- 172.21.15.88:0/3599611244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f236c15cb60 0x7f236c0cda80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f236c0b88b0 tx=0x7f236801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0cf92bfb41b1bff server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:49.483 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.479+0000 7f23637fe640 1 -- 172.21.15.88:0/3599611244 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2368004230 con 0x7f236c15cb60 2023-10-16T20:04:49.483 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.479+0000 7f23637fe640 1 -- 172.21.15.88:0/3599611244 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23680a7050 con 0x7f236c15cb60 2023-10-16T20:04:49.483 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.479+0000 7f23637fe640 1 -- 172.21.15.88:0/3599611244 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2368003620 con 0x7f236c15cb60 2023-10-16T20:04:49.484 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.479+0000 7f2372e9e640 1 -- 172.21.15.88:0/3599611244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f236c15cb60 msgr2=0x7f236c0cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:49.484 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.479+0000 7f2372e9e640 1 --2- 172.21.15.88:0/3599611244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f236c15cb60 0x7f236c0cda80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f236c0b88b0 tx=0x7f236801c430 comp rx=0 tx=0).stop 2023-10-16T20:04:49.484 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.479+0000 7f2372e9e640 1 -- 172.21.15.88:0/3599611244 shutdown_connections 2023-10-16T20:04:49.484 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.479+0000 7f2372e9e640 1 --2- 172.21.15.88:0/3599611244 >> v2:172.21.15.94:3300/0 conn(0x7f236c0d19a0 0x7f236c0cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:49.485 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.479+0000 7f2372e9e640 1 --2- 172.21.15.88:0/3599611244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f236c15cb60 0x7f236c0cda80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:49.485 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.479+0000 7f2372e9e640 1 -- 172.21.15.88:0/3599611244 >> 172.21.15.88:0/3599611244 conn(0x7f236c071590 msgr2=0x7f236c0719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:49.485 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.479+0000 7f2372e9e640 1 -- 172.21.15.88:0/3599611244 shutdown_connections 2023-10-16T20:04:49.485 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.479+0000 7f2372e9e640 1 -- 172.21.15.88:0/3599611244 wait complete. 2023-10-16T20:04:49.486 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.483+0000 7f2372e9e640 1 Processor -- start 2023-10-16T20:04:49.486 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.483+0000 7f2372e9e640 1 -- start start 2023-10-16T20:04:49.488 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.483+0000 7f2372e9e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f236c0cf550 0x7f236c0cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:49.488 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.483+0000 7f2372e9e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f236c0d19a0 0x7f236c14f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:49.489 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.483+0000 7f2372e9e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f236c15cb60 0x7f236c0c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:49.489 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.483+0000 7f2372e9e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c156890 con 0x7f236c0d19a0 2023-10-16T20:04:49.489 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.483+0000 7f2372e9e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c155f80 con 0x7f236c15cb60 2023-10-16T20:04:49.489 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.483+0000 7f2372e9e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236c15a8c0 con 0x7f236c0cf550 2023-10-16T20:04:49.490 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.483+0000 7f2363fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f236c0d19a0 0x7f236c14f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:49.490 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.483+0000 7f2363fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f236c0d19a0 0x7f236c14f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43436/0 (socket says 172.21.15.88:43436) 2023-10-16T20:04:49.490 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.483+0000 7f2363fff640 1 -- 172.21.15.88:0/780013700 learned_addr learned my addr 172.21.15.88:0/780013700 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:04:49.490 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.483+0000 7f237143b640 1 --2- 172.21.15.88:0/780013700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f236c15cb60 0x7f236c0c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:49.490 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.483+0000 7f2370c3a640 1 --2- 172.21.15.88:0/780013700 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f236c0cf550 0x7f236c0cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:49.491 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.483+0000 7f2363fff640 1 -- 172.21.15.88:0/780013700 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f236c0cf550 msgr2=0x7f236c0cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:49.491 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.483+0000 7f2363fff640 1 --2- 172.21.15.88:0/780013700 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f236c0cf550 0x7f236c0cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:49.491 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.483+0000 7f2363fff640 1 -- 172.21.15.88:0/780013700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f236c15cb60 msgr2=0x7f236c0c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:49.491 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.483+0000 7f2363fff640 1 --2- 172.21.15.88:0/780013700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f236c15cb60 0x7f236c0c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:49.491 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.483+0000 7f2363fff640 1 -- 172.21.15.88:0/780013700 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2368009420 con 0x7f236c0d19a0 2023-10-16T20:04:49.491 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.483+0000 7f2363fff640 1 --2- 172.21.15.88:0/780013700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f236c0d19a0 0x7f236c14f620 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f235c0011f0 tx=0x7f235c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:49.492 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.483+0000 7f23617fa640 1 -- 172.21.15.88:0/780013700 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c0074e0 con 0x7f236c0d19a0 2023-10-16T20:04:49.492 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.483+0000 7f23617fa640 1 -- 172.21.15.88:0/780013700 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235c007640 con 0x7f236c0d19a0 2023-10-16T20:04:49.492 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.483+0000 7f2372e9e640 1 -- 172.21.15.88:0/780013700 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f236c156240 con 0x7f236c0d19a0 2023-10-16T20:04:49.492 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.483+0000 7f23617fa640 1 -- 172.21.15.88:0/780013700 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c004a10 con 0x7f236c0d19a0 2023-10-16T20:04:49.492 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.483+0000 7f2372e9e640 1 -- 172.21.15.88:0/780013700 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f236c14fb60 con 0x7f236c0d19a0 2023-10-16T20:04:49.493 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.487+0000 7f2372e9e640 1 -- 172.21.15.88:0/780013700 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f236c156a40 con 0x7f236c0d19a0 2023-10-16T20:04:49.493 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.487+0000 7f23617fa640 1 -- 172.21.15.88:0/780013700 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f235c004b70 con 0x7f236c0d19a0 2023-10-16T20:04:49.494 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.487+0000 7f23617fa640 1 --2- 172.21.15.88:0/780013700 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f23580664a0 0x7f2358068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:49.494 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.487+0000 7f23617fa640 1 -- 172.21.15.88:0/780013700 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f235c083050 con 0x7f236c0d19a0 2023-10-16T20:04:49.494 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.487+0000 7f2370c3a640 1 --2- 172.21.15.88:0/780013700 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f23580664a0 0x7f2358068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:49.494 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.487+0000 7f2370c3a640 1 --2- 172.21.15.88:0/780013700 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f23580664a0 0x7f2358068960 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f23680ad000 tx=0x7f2368008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:49.495 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.491+0000 7f23617fa640 1 -- 172.21.15.88:0/780013700 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f236c156a40 con 0x7f236c0d19a0 2023-10-16T20:04:49.646 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.643+0000 7f2372e9e640 1 -- 172.21.15.88:0/780013700 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f236c073f30 con 0x7f236c0d19a0 2023-10-16T20:04:49.646 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.643+0000 7f23617fa640 1 -- 172.21.15.88:0/780013700 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+12778 (secure 0 0 0) 0x7f235c018090 con 0x7f236c0d19a0 2023-10-16T20:04:49.647 INFO:teuthology.misc.health.smithi088.stdout: 2023-10-16T20:04:49.647 INFO:teuthology.misc.health.smithi088.stdout:{"epoch":19,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:04:47.326789+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:04:49.649 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.643+0000 7f2372e9e640 1 -- 172.21.15.88:0/780013700 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f23580664a0 msgr2=0x7f2358068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:49.649 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.643+0000 7f2372e9e640 1 --2- 172.21.15.88:0/780013700 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f23580664a0 0x7f2358068960 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f23680ad000 tx=0x7f2368008040 comp rx=0 tx=0).stop 2023-10-16T20:04:49.649 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.643+0000 7f2372e9e640 1 -- 172.21.15.88:0/780013700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f236c0d19a0 msgr2=0x7f236c14f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:49.649 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.643+0000 7f2372e9e640 1 --2- 172.21.15.88:0/780013700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f236c0d19a0 0x7f236c14f620 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f235c0011f0 tx=0x7f235c00c9b0 comp rx=0 tx=0).stop 2023-10-16T20:04:49.650 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.643+0000 7f2372e9e640 1 -- 172.21.15.88:0/780013700 shutdown_connections 2023-10-16T20:04:49.650 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.643+0000 7f2372e9e640 1 --2- 172.21.15.88:0/780013700 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f23580664a0 0x7f2358068960 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:49.650 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.643+0000 7f2372e9e640 1 --2- 172.21.15.88:0/780013700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f236c15cb60 0x7f236c0c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:49.650 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.643+0000 7f2372e9e640 1 --2- 172.21.15.88:0/780013700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f236c0d19a0 0x7f236c14f620 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:49.650 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.643+0000 7f2372e9e640 1 --2- 172.21.15.88:0/780013700 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f236c0cf550 0x7f236c0cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:49.651 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.643+0000 7f2372e9e640 1 -- 172.21.15.88:0/780013700 >> 172.21.15.88:0/780013700 conn(0x7f236c071590 msgr2=0x7f236c15e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:49.651 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.647+0000 7f2372e9e640 1 -- 172.21.15.88:0/780013700 shutdown_connections 2023-10-16T20:04:49.651 INFO:teuthology.misc.health.smithi088.stderr:2023-10-16T20:04:49.647+0000 7f2372e9e640 1 -- 172.21.15.88:0/780013700 wait complete. 2023-10-16T20:04:49.666 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-16T20:04:49.666 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-16T20:04:49.666 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T20:04:49.667 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T20:04:49.671 INFO:tasks.ceph.mds.a:Started 2023-10-16T20:04:49.671 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T20:04:49.671 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T20:04:49.675 INFO:tasks.ceph.mds.c:Started 2023-10-16T20:04:49.675 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T20:04:49.675 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T20:04:49.679 INFO:tasks.ceph.mds.b:Started 2023-10-16T20:04:49.680 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T20:04:49.680 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T20:04:49.684 INFO:tasks.ceph.mds.d:Started 2023-10-16T20:04:49.684 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-16T20:04:49.684 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:04:49.773 INFO:tasks.ceph.mds.a.smithi088.stdout:starting mds.a at 2023-10-16T20:04:49.773 INFO:tasks.ceph.mds.c.smithi088.stdout:starting mds.c at 2023-10-16T20:04:49.785 INFO:tasks.ceph.mds.d.smithi094.stdout:starting mds.d at 2023-10-16T20:04:49.786 INFO:tasks.ceph.mds.b.smithi094.stdout:starting mds.b at 2023-10-16T20:04:49.832 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.827+0000 7f933d246640 1 Processor -- start 2023-10-16T20:04:49.832 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.827+0000 7f933d246640 1 -- start start 2023-10-16T20:04:49.833 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.831+0000 7f933d246640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f933815cb70 0x7f9338161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:49.834 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.831+0000 7f933d246640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f93380cdad0 0x7f93380cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:49.834 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.831+0000 7f933d246640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93380b2190 con 0x7f93380cff40 2023-10-16T20:04:49.834 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.831+0000 7f933d246640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93381615a0 con 0x7f93380cdad0 2023-10-16T20:04:49.834 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.831+0000 7f933d246640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f933815df60 con 0x7f933815cb70 2023-10-16T20:04:49.834 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.831+0000 7f932e59c640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f93380cdad0 0x7f93380cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:49.835 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.831+0000 7f932e59c640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f93380cdad0 0x7f93380cfa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43470/0 (socket says 172.21.15.88:43470) 2023-10-16T20:04:49.835 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.831+0000 7f932e59c640 1 -- 172.21.15.88:0/3483605468 learned_addr learned my addr 172.21.15.88:0/3483605468 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:04:49.835 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.831+0000 7f9336d9d640 1 --2- 172.21.15.88:0/3483605468 >> v2:172.21.15.94:3300/0 conn(0x7f933815cb70 0x7f9338161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:49.835 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.831+0000 7f932e59c640 1 -- 172.21.15.88:0/3483605468 >> v2:172.21.15.94:3300/0 conn(0x7f933815cb70 msgr2=0x7f9338161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:04:49.836 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.831+0000 7f932e59c640 1 --2- 172.21.15.88:0/3483605468 >> v2:172.21.15.94:3300/0 conn(0x7f933815cb70 0x7f9338161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:49.836 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.831+0000 7f932e59c640 1 -- 172.21.15.88:0/3483605468 >> v1:172.21.15.94:6789/0 conn(0x7f93380cff40 legacy=0x7f93380b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:49.836 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.831+0000 7f932e59c640 1 -- 172.21.15.88:0/3483605468 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93380d0370 con 0x7f93380cdad0 2023-10-16T20:04:49.837 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.835+0000 7f932e59c640 1 --2- 172.21.15.88:0/3483605468 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93380cdad0 0x7f93380cfa00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f93381616e0 tx=0x7f932001c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc04ab17fd1dfc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:49.837 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.835+0000 7f933659c640 1 -- 172.21.15.88:0/3483605468 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93200a5780 con 0x7f93380cdad0 2023-10-16T20:04:49.837 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.835+0000 7f933659c640 1 -- 172.21.15.88:0/3483605468 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93200a58e0 con 0x7f93380cdad0 2023-10-16T20:04:49.837 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.835+0000 7f933659c640 1 -- 172.21.15.88:0/3483605468 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93200a5c70 con 0x7f93380cdad0 2023-10-16T20:04:49.838 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.835+0000 7f933d246640 1 -- 172.21.15.88:0/3483605468 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93380cdad0 msgr2=0x7f93380cfa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:49.839 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.835+0000 7f933d246640 1 --2- 172.21.15.88:0/3483605468 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93380cdad0 0x7f93380cfa00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f93381616e0 tx=0x7f932001c9d0 comp rx=0 tx=0).stop 2023-10-16T20:04:49.839 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.835+0000 7f933d246640 1 -- 172.21.15.88:0/3483605468 shutdown_connections 2023-10-16T20:04:49.839 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.835+0000 7f933d246640 1 --2- 172.21.15.88:0/3483605468 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93380cdad0 0x7f93380cfa00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:49.839 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.835+0000 7f933d246640 1 --2- 172.21.15.88:0/3483605468 >> v2:172.21.15.94:3300/0 conn(0x7f933815cb70 0x7f9338161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:49.839 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.835+0000 7f933d246640 1 -- 172.21.15.88:0/3483605468 >> 172.21.15.88:0/3483605468 conn(0x7f93380719c0 msgr2=0x7f9338071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:49.840 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.835+0000 7f933d246640 1 -- 172.21.15.88:0/3483605468 shutdown_connections 2023-10-16T20:04:49.840 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.835+0000 7f933d246640 1 -- 172.21.15.88:0/3483605468 wait complete. 2023-10-16T20:04:49.841 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.835+0000 7f933d246640 1 Processor -- start 2023-10-16T20:04:49.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.835+0000 7f933d246640 1 -- start start 2023-10-16T20:04:49.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.839+0000 7f933d246640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f93380cdad0 0x7f9338156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:49.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.839+0000 7f933d246640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93380cff40 0x7f93381581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:49.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.839+0000 7f933d246640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f933815cb70 0x7f93380c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:49.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.839+0000 7f933d246640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93380cafa0 con 0x7f93380cff40 2023-10-16T20:04:49.843 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.839+0000 7f933d246640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93380c7920 con 0x7f933815cb70 2023-10-16T20:04:49.843 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.839+0000 7f933d246640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93380c7c00 con 0x7f93380cdad0 2023-10-16T20:04:49.843 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.839+0000 7f932e59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93380cff40 0x7f93381581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:49.843 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.839+0000 7f932e59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93380cff40 0x7f93381581a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43486/0 (socket says 172.21.15.88:43486) 2023-10-16T20:04:49.843 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.839+0000 7f932e59c640 1 -- 172.21.15.88:0/2107230469 learned_addr learned my addr 172.21.15.88:0/2107230469 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:04:49.843 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.839+0000 7f9336d9d640 1 --2- 172.21.15.88:0/2107230469 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f93380cdad0 0x7f9338156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:49.844 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.839+0000 7f933759e640 1 --2- 172.21.15.88:0/2107230469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f933815cb70 0x7f93380c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:49.844 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.839+0000 7f932e59c640 1 -- 172.21.15.88:0/2107230469 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f93380cdad0 msgr2=0x7f9338156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:49.844 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.839+0000 7f932e59c640 1 --2- 172.21.15.88:0/2107230469 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f93380cdad0 0x7f9338156b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:49.844 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.839+0000 7f932e59c640 1 -- 172.21.15.88:0/2107230469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f933815cb70 msgr2=0x7f93380c65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:49.844 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.839+0000 7f932e59c640 1 --2- 172.21.15.88:0/2107230469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f933815cb70 0x7f93380c65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:49.845 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.839+0000 7f932e59c640 1 -- 172.21.15.88:0/2107230469 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9320009b30 con 0x7f93380cff40 2023-10-16T20:04:49.845 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.839+0000 7f932e59c640 1 --2- 172.21.15.88:0/2107230469 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93380cff40 0x7f93381581a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f93200195e0 tx=0x7f9320019770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:49.846 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.839+0000 7f932ffff640 1 -- 172.21.15.88:0/2107230469 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93200b8670 con 0x7f93380cff40 2023-10-16T20:04:49.846 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.839+0000 7f932ffff640 1 -- 172.21.15.88:0/2107230469 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93200b87d0 con 0x7f93380cff40 2023-10-16T20:04:49.847 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.839+0000 7f932ffff640 1 -- 172.21.15.88:0/2107230469 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93200b8b80 con 0x7f93380cff40 2023-10-16T20:04:49.847 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.839+0000 7f933d246640 1 -- 172.21.15.88:0/2107230469 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93380c7ee0 con 0x7f93380cff40 2023-10-16T20:04:49.847 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.839+0000 7f933d246640 1 -- 172.21.15.88:0/2107230469 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93380cb280 con 0x7f93380cff40 2023-10-16T20:04:49.847 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.839+0000 7f933d246640 1 -- 172.21.15.88:0/2107230469 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93380c80d0 con 0x7f93380cff40 2023-10-16T20:04:49.847 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.839+0000 7f932ffff640 1 -- 172.21.15.88:0/2107230469 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f93200b8ce0 con 0x7f93380cff40 2023-10-16T20:04:49.848 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.843+0000 7f932ffff640 1 --2- 172.21.15.88:0/2107230469 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f93180664a0 0x7f9318068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:49.848 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.843+0000 7f932ffff640 1 -- 172.21.15.88:0/2107230469 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f932011ef10 con 0x7f93380cff40 2023-10-16T20:04:49.848 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.843+0000 7f9336d9d640 1 --2- 172.21.15.88:0/2107230469 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f93180664a0 0x7f9318068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:49.848 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.843+0000 7f9336d9d640 1 --2- 172.21.15.88:0/2107230469 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f93180664a0 0x7f9318068960 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f93380b82e0 tx=0x7f9328009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:49.849 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.847+0000 7f932ffff640 1 -- 172.21.15.88:0/2107230469 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f93200ec940 con 0x7f93380cff40 2023-10-16T20:04:49.999 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.995+0000 7f933d246640 1 -- 172.21.15.88:0/2107230469 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f933804e9c0 con 0x7f93380cff40 2023-10-16T20:04:50.000 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.995+0000 7f932ffff640 1 -- 172.21.15.88:0/2107230469 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+12778 (secure 0 0 0) 0x7f93200eff60 con 0x7f93380cff40 2023-10-16T20:04:50.001 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:04:50.001 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":19,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:04:47.326789+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:04:50.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.999+0000 7f933d246640 1 -- 172.21.15.88:0/2107230469 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f93180664a0 msgr2=0x7f9318068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:50.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.999+0000 7f933d246640 1 --2- 172.21.15.88:0/2107230469 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f93180664a0 0x7f9318068960 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f93380b82e0 tx=0x7f9328009190 comp rx=0 tx=0).stop 2023-10-16T20:04:50.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.999+0000 7f933d246640 1 -- 172.21.15.88:0/2107230469 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93380cff40 msgr2=0x7f93381581a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:50.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.999+0000 7f933d246640 1 --2- 172.21.15.88:0/2107230469 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93380cff40 0x7f93381581a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f93200195e0 tx=0x7f9320019770 comp rx=0 tx=0).stop 2023-10-16T20:04:50.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.999+0000 7f933d246640 1 -- 172.21.15.88:0/2107230469 shutdown_connections 2023-10-16T20:04:50.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.999+0000 7f933d246640 1 --2- 172.21.15.88:0/2107230469 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f93180664a0 0x7f9318068960 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:50.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.999+0000 7f933d246640 1 --2- 172.21.15.88:0/2107230469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f933815cb70 0x7f93380c65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:50.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.999+0000 7f933d246640 1 --2- 172.21.15.88:0/2107230469 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93380cff40 0x7f93381581a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:50.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.999+0000 7f933d246640 1 --2- 172.21.15.88:0/2107230469 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f93380cdad0 0x7f9338156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:50.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.999+0000 7f933d246640 1 -- 172.21.15.88:0/2107230469 >> 172.21.15.88:0/2107230469 conn(0x7f93380719c0 msgr2=0x7f9338071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:50.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.999+0000 7f933d246640 1 -- 172.21.15.88:0/2107230469 shutdown_connections 2023-10-16T20:04:50.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:49.999+0000 7f933d246640 1 -- 172.21.15.88:0/2107230469 wait complete. 2023-10-16T20:04:50.022 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T20:04:50.023 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:04:50.166 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.163+0000 7fe624605640 1 Processor -- start 2023-10-16T20:04:50.166 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.163+0000 7fe624605640 1 -- start start 2023-10-16T20:04:50.168 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.163+0000 7fe624605640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe61c15cb70 0x7fe61c0b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:50.168 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.163+0000 7fe624605640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe61c0b6a80 0x7fe61c0b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:50.169 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.163+0000 7fe624605640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe61c0b4ed0 con 0x7fe61c0b4530 2023-10-16T20:04:50.169 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.163+0000 7fe624605640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe61c0b6f10 con 0x7fe61c0b6a80 2023-10-16T20:04:50.169 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.163+0000 7fe624605640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c0b7d20 con 0x7fe61c15cb70 2023-10-16T20:04:50.169 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.163+0000 7fe621ba0640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fe61c0b4530 0x7fe61c0d0bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:35790/0 (socket says 172.21.15.88:35790) 2023-10-16T20:04:50.169 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.163+0000 7fe621ba0640 1 -- 172.21.15.88:0/593348220 learned_addr learned my addr 172.21.15.88:0/593348220 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:04:50.170 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.163+0000 7fe622ba2640 1 --2- 172.21.15.88:0/593348220 >> v2:172.21.15.94:3300/0 conn(0x7fe61c0b6a80 0x7fe61c0b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:50.170 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.163+0000 7fe6223a1640 1 --2- 172.21.15.88:0/593348220 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe61c15cb70 0x7fe61c0b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:50.170 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.163+0000 7fe62139f640 1 -- 172.21.15.88:0/593348220 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3045067937 0 0) 0x7fe61c0b4ed0 con 0x7fe61c0b4530 2023-10-16T20:04:50.170 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.163+0000 7fe62139f640 1 -- 172.21.15.88:0/593348220 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe608003530 con 0x7fe61c0b4530 2023-10-16T20:04:50.170 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.163+0000 7fe622ba2640 1 -- 172.21.15.88:0/593348220 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe61c15cb70 msgr2=0x7fe61c0b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:04:50.171 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.163+0000 7fe622ba2640 1 --2- 172.21.15.88:0/593348220 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe61c15cb70 0x7fe61c0b83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:50.171 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.163+0000 7fe622ba2640 1 -- 172.21.15.88:0/593348220 >> v1:172.21.15.88:6789/0 conn(0x7fe61c0b4530 legacy=0x7fe61c0d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:50.171 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.163+0000 7fe622ba2640 1 -- 172.21.15.88:0/593348220 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61c0b7110 con 0x7fe61c0b6a80 2023-10-16T20:04:50.172 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.167+0000 7fe622ba2640 1 --2- 172.21.15.88:0/593348220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe61c0b6a80 0x7fe61c0b4990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe614009d90 tx=0x7fe61401f130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc62031f9bf115ad server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:50.173 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.167+0000 7fe62139f640 1 -- 172.21.15.88:0/593348220 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6140af600 con 0x7fe61c0b6a80 2023-10-16T20:04:50.173 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.167+0000 7fe62139f640 1 -- 172.21.15.88:0/593348220 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6140ab050 con 0x7fe61c0b6a80 2023-10-16T20:04:50.173 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.167+0000 7fe62139f640 1 -- 172.21.15.88:0/593348220 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6140af980 con 0x7fe61c0b6a80 2023-10-16T20:04:50.173 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.167+0000 7fe624605640 1 -- 172.21.15.88:0/593348220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe61c0b6a80 msgr2=0x7fe61c0b4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:50.173 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.167+0000 7fe624605640 1 --2- 172.21.15.88:0/593348220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe61c0b6a80 0x7fe61c0b4990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe614009d90 tx=0x7fe61401f130 comp rx=0 tx=0).stop 2023-10-16T20:04:50.174 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.167+0000 7fe624605640 1 -- 172.21.15.88:0/593348220 shutdown_connections 2023-10-16T20:04:50.174 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.167+0000 7fe624605640 1 --2- 172.21.15.88:0/593348220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe61c0b6a80 0x7fe61c0b4990 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:50.174 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.167+0000 7fe624605640 1 --2- 172.21.15.88:0/593348220 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe61c15cb70 0x7fe61c0b83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:50.174 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.167+0000 7fe624605640 1 -- 172.21.15.88:0/593348220 >> 172.21.15.88:0/593348220 conn(0x7fe61c06b6e0 msgr2=0x7fe61c06baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:50.175 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.167+0000 7fe624605640 1 -- 172.21.15.88:0/593348220 shutdown_connections 2023-10-16T20:04:50.175 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.167+0000 7fe624605640 1 -- 172.21.15.88:0/593348220 wait complete. 2023-10-16T20:04:50.177 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.171+0000 7fe624605640 1 Processor -- start 2023-10-16T20:04:50.177 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.171+0000 7fe624605640 1 -- start start 2023-10-16T20:04:50.177 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.171+0000 7fe624605640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe61c0b4530 0x7fe61c15cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:50.178 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.171+0000 7fe624605640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe61c0b6a80 0x7fe61c0c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:50.178 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.171+0000 7fe624605640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe61c15cb70 0x7fe61c15b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:50.178 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.171+0000 7fe624605640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c14fbb0 con 0x7fe61c15cb70 2023-10-16T20:04:50.178 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.171+0000 7fe624605640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c152ab0 con 0x7fe61c0b6a80 2023-10-16T20:04:50.179 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.171+0000 7fe624605640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c14f880 con 0x7fe61c0b4530 2023-10-16T20:04:50.179 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.171+0000 7fe622ba2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe61c15cb70 0x7fe61c15b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:50.179 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.171+0000 7fe622ba2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe61c15cb70 0x7fe61c15b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43498/0 (socket says 172.21.15.88:43498) 2023-10-16T20:04:50.179 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.171+0000 7fe622ba2640 1 -- 172.21.15.88:0/3035076285 learned_addr learned my addr 172.21.15.88:0/3035076285 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:04:50.179 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.171+0000 7fe6223a1640 1 --2- 172.21.15.88:0/3035076285 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe61c0b4530 0x7fe61c15cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:50.180 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.171+0000 7fe621ba0640 1 --2- 172.21.15.88:0/3035076285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe61c0b6a80 0x7fe61c0c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:50.180 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.171+0000 7fe622ba2640 1 -- 172.21.15.88:0/3035076285 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe61c0b4530 msgr2=0x7fe61c15cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:50.180 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.171+0000 7fe622ba2640 1 --2- 172.21.15.88:0/3035076285 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe61c0b4530 0x7fe61c15cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:50.180 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.171+0000 7fe622ba2640 1 -- 172.21.15.88:0/3035076285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe61c0b6a80 msgr2=0x7fe61c0c8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:50.180 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.171+0000 7fe622ba2640 1 --2- 172.21.15.88:0/3035076285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe61c0b6a80 0x7fe61c0c8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:50.181 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.171+0000 7fe622ba2640 1 -- 172.21.15.88:0/3035076285 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe614009a70 con 0x7fe61c15cb70 2023-10-16T20:04:50.181 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.171+0000 7fe622ba2640 1 --2- 172.21.15.88:0/3035076285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe61c15cb70 0x7fe61c15b4b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe6140b2040 tx=0x7fe61401bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:50.181 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.171+0000 7fe612ffd640 1 -- 172.21.15.88:0/3035076285 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe614009450 con 0x7fe61c15cb70 2023-10-16T20:04:50.181 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.171+0000 7fe612ffd640 1 -- 172.21.15.88:0/3035076285 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6140b9410 con 0x7fe61c15cb70 2023-10-16T20:04:50.181 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.171+0000 7fe624605640 1 -- 172.21.15.88:0/3035076285 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe61c15d630 con 0x7fe61c15cb70 2023-10-16T20:04:50.181 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.171+0000 7fe612ffd640 1 -- 172.21.15.88:0/3035076285 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6140b97c0 con 0x7fe61c15cb70 2023-10-16T20:04:50.182 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.171+0000 7fe624605640 1 -- 172.21.15.88:0/3035076285 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe61c15dbf0 con 0x7fe61c15cb70 2023-10-16T20:04:50.182 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.175+0000 7fe624605640 1 -- 172.21.15.88:0/3035076285 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe61c14fd20 con 0x7fe61c15cb70 2023-10-16T20:04:50.183 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.175+0000 7fe612ffd640 1 -- 172.21.15.88:0/3035076285 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe6140b6050 con 0x7fe61c15cb70 2023-10-16T20:04:50.183 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.175+0000 7fe612ffd640 1 --2- 172.21.15.88:0/3035076285 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe6000664d0 0x7fe600068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:50.183 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.175+0000 7fe612ffd640 1 -- 172.21.15.88:0/3035076285 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe614121fd0 con 0x7fe61c15cb70 2023-10-16T20:04:50.184 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.175+0000 7fe6223a1640 1 --2- 172.21.15.88:0/3035076285 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe6000664d0 0x7fe600068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:50.184 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.175+0000 7fe6223a1640 1 --2- 172.21.15.88:0/3035076285 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe6000664d0 0x7fe600068990 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe618009a50 tx=0x7fe618008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:50.184 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.179+0000 7fe612ffd640 1 -- 172.21.15.88:0/3035076285 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe6140ef8e0 con 0x7fe61c15cb70 2023-10-16T20:04:50.351 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.347+0000 7fe624605640 1 -- 172.21.15.88:0/3035076285 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe61c04e9c0 con 0x7fe61c15cb70 2023-10-16T20:04:50.351 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.347+0000 7fe612ffd640 1 -- 172.21.15.88:0/3035076285 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe6140c9020 con 0x7fe61c15cb70 2023-10-16T20:04:50.352 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:04:50.354 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.351+0000 7fe624605640 1 -- 172.21.15.88:0/3035076285 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe6000664d0 msgr2=0x7fe600068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:50.354 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.351+0000 7fe624605640 1 --2- 172.21.15.88:0/3035076285 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe6000664d0 0x7fe600068990 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe618009a50 tx=0x7fe618008040 comp rx=0 tx=0).stop 2023-10-16T20:04:50.354 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.351+0000 7fe624605640 1 -- 172.21.15.88:0/3035076285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe61c15cb70 msgr2=0x7fe61c15b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:50.354 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.351+0000 7fe624605640 1 --2- 172.21.15.88:0/3035076285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe61c15cb70 0x7fe61c15b4b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe6140b2040 tx=0x7fe61401bed0 comp rx=0 tx=0).stop 2023-10-16T20:04:50.355 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.351+0000 7fe624605640 1 -- 172.21.15.88:0/3035076285 shutdown_connections 2023-10-16T20:04:50.355 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.351+0000 7fe624605640 1 --2- 172.21.15.88:0/3035076285 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe6000664d0 0x7fe600068990 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:50.355 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.351+0000 7fe624605640 1 --2- 172.21.15.88:0/3035076285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe61c15cb70 0x7fe61c15b4b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:50.355 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.351+0000 7fe624605640 1 --2- 172.21.15.88:0/3035076285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe61c0b6a80 0x7fe61c0c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:50.355 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.351+0000 7fe624605640 1 --2- 172.21.15.88:0/3035076285 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe61c0b4530 0x7fe61c15cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:50.355 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.351+0000 7fe624605640 1 -- 172.21.15.88:0/3035076285 >> 172.21.15.88:0/3035076285 conn(0x7fe61c06b6e0 msgr2=0x7fe61c0d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:50.356 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.351+0000 7fe624605640 1 -- 172.21.15.88:0/3035076285 shutdown_connections 2023-10-16T20:04:50.356 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.351+0000 7fe624605640 1 -- 172.21.15.88:0/3035076285 wait complete. 2023-10-16T20:04:50.373 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T20:04:50.512 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.507+0000 7f28caaf7640 1 Processor -- start 2023-10-16T20:04:50.512 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.507+0000 7f28caaf7640 1 -- start start 2023-10-16T20:04:50.513 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.511+0000 7f28caaf7640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f28c415d9f0 0x7f28c4161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:50.513 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.511+0000 7f28caaf7640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f28c40d26c0 0x7f28c40cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:50.514 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.511+0000 7f28caaf7640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c40b87e0 con 0x7f28c40d22e0 2023-10-16T20:04:50.514 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.511+0000 7f28caaf7640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c415df10 con 0x7f28c40d26c0 2023-10-16T20:04:50.514 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.511+0000 7f28caaf7640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c415e1f0 con 0x7f28c415d9f0 2023-10-16T20:04:50.514 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.511+0000 7f28c8893640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f28c40d22e0 0x7f28c415d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:35804/0 (socket says 172.21.15.88:35804) 2023-10-16T20:04:50.514 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.511+0000 7f28c8893640 1 -- 172.21.15.88:0/3205770954 learned_addr learned my addr 172.21.15.88:0/3205770954 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:04:50.515 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.511+0000 7f28c9094640 1 --2- 172.21.15.88:0/3205770954 >> v2:172.21.15.94:3300/0 conn(0x7f28c40d26c0 0x7f28c40cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:50.515 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.511+0000 7f28c3fff640 1 --2- 172.21.15.88:0/3205770954 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f28c415d9f0 0x7f28c4161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:50.516 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.511+0000 7f28c37fe640 1 -- 172.21.15.88:0/3205770954 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 487992923 0 0) 0x7f28c40b87e0 con 0x7f28c40d22e0 2023-10-16T20:04:50.516 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.511+0000 7f28c37fe640 1 -- 172.21.15.88:0/3205770954 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28b4003530 con 0x7f28c40d22e0 2023-10-16T20:04:50.516 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.511+0000 7f28c37fe640 1 -- 172.21.15.88:0/3205770954 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2114422273 0 0) 0x7f28b4003530 con 0x7f28c40d22e0 2023-10-16T20:04:50.516 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.511+0000 7f28c9094640 1 -- 172.21.15.88:0/3205770954 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f28c415d9f0 msgr2=0x7f28c4161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:04:50.516 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.511+0000 7f28c9094640 1 --2- 172.21.15.88:0/3205770954 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f28c415d9f0 0x7f28c4161450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:50.517 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.511+0000 7f28c9094640 1 -- 172.21.15.88:0/3205770954 >> v1:172.21.15.88:6789/0 conn(0x7f28c40d22e0 legacy=0x7f28c415d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:50.517 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.511+0000 7f28c9094640 1 -- 172.21.15.88:0/3205770954 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c4160970 con 0x7f28c40d26c0 2023-10-16T20:04:50.518 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.515+0000 7f28c9094640 1 --2- 172.21.15.88:0/3205770954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28c40d26c0 0x7f28c40cf2e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009d90 tx=0x7f28bc01c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=211a58948cf23dae server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:50.518 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.515+0000 7f28c37fe640 1 -- 172.21.15.88:0/3205770954 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b80016a0 con 0x7f28c40d26c0 2023-10-16T20:04:50.519 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.515+0000 7f28c37fe640 1 -- 172.21.15.88:0/3205770954 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28bc0a55b0 con 0x7f28c40d26c0 2023-10-16T20:04:50.519 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.515+0000 7f28c37fe640 1 -- 172.21.15.88:0/3205770954 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc0a5940 con 0x7f28c40d26c0 2023-10-16T20:04:50.519 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.515+0000 7f28caaf7640 1 -- 172.21.15.88:0/3205770954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28c40d26c0 msgr2=0x7f28c40cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:50.519 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.515+0000 7f28caaf7640 1 --2- 172.21.15.88:0/3205770954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28c40d26c0 0x7f28c40cf2e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009d90 tx=0x7f28bc01c7b0 comp rx=0 tx=0).stop 2023-10-16T20:04:50.519 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.515+0000 7f28caaf7640 1 -- 172.21.15.88:0/3205770954 shutdown_connections 2023-10-16T20:04:50.520 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.515+0000 7f28caaf7640 1 --2- 172.21.15.88:0/3205770954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28c40d26c0 0x7f28c40cf2e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:50.520 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.515+0000 7f28caaf7640 1 --2- 172.21.15.88:0/3205770954 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f28c415d9f0 0x7f28c4161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:50.520 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.515+0000 7f28caaf7640 1 -- 172.21.15.88:0/3205770954 >> 172.21.15.88:0/3205770954 conn(0x7f28c40715c0 msgr2=0x7f28c40719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:50.520 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.515+0000 7f28caaf7640 1 -- 172.21.15.88:0/3205770954 shutdown_connections 2023-10-16T20:04:50.520 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.515+0000 7f28caaf7640 1 -- 172.21.15.88:0/3205770954 wait complete. 2023-10-16T20:04:50.523 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.515+0000 7f28caaf7640 1 Processor -- start 2023-10-16T20:04:50.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.515+0000 7f28caaf7640 1 -- start start 2023-10-16T20:04:50.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.519+0000 7f28caaf7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28c40d22e0 0x7f28c4151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:50.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.519+0000 7f28caaf7640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f28c40d26c0 0x7f28c4151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:50.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.519+0000 7f28caaf7640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28c415d9f0 0x7f28c41501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:50.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.519+0000 7f28caaf7640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c41478c0 con 0x7f28c415d9f0 2023-10-16T20:04:50.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.519+0000 7f28caaf7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c4154dc0 con 0x7f28c40d22e0 2023-10-16T20:04:50.527 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.519+0000 7f28caaf7640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c41550a0 con 0x7f28c40d26c0 2023-10-16T20:04:50.527 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.519+0000 7f28c9094640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28c415d9f0 0x7f28c41501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:50.527 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.519+0000 7f28c9094640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28c415d9f0 0x7f28c41501c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43514/0 (socket says 172.21.15.88:43514) 2023-10-16T20:04:50.527 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.519+0000 7f28c9094640 1 -- 172.21.15.88:0/577652703 learned_addr learned my addr 172.21.15.88:0/577652703 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:04:50.527 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.519+0000 7f28c3fff640 1 --2- 172.21.15.88:0/577652703 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f28c40d26c0 0x7f28c4151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:50.527 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.519+0000 7f28c8893640 1 --2- 172.21.15.88:0/577652703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28c40d22e0 0x7f28c4151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:50.528 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.519+0000 7f28c9094640 1 -- 172.21.15.88:0/577652703 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f28c40d26c0 msgr2=0x7f28c4151840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:50.528 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.519+0000 7f28c9094640 1 --2- 172.21.15.88:0/577652703 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f28c40d26c0 0x7f28c4151840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:50.528 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.519+0000 7f28c9094640 1 -- 172.21.15.88:0/577652703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28c40d22e0 msgr2=0x7f28c4151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:50.528 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.519+0000 7f28c9094640 1 --2- 172.21.15.88:0/577652703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28c40d22e0 0x7f28c4151300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:50.528 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.519+0000 7f28c9094640 1 -- 172.21.15.88:0/577652703 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28bc009a70 con 0x7f28c415d9f0 2023-10-16T20:04:50.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.519+0000 7f28c8893640 1 --2- 172.21.15.88:0/577652703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28c40d22e0 0x7f28c4151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:04:50.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.519+0000 7f28c9094640 1 --2- 172.21.15.88:0/577652703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28c415d9f0 0x7f28c41501c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f28bc0b0040 tx=0x7f28bc019550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:50.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.519+0000 7f28c17fa640 1 -- 172.21.15.88:0/577652703 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc0ad640 con 0x7f28c415d9f0 2023-10-16T20:04:50.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.519+0000 7f28c17fa640 1 -- 172.21.15.88:0/577652703 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28bc0ad7a0 con 0x7f28c415d9f0 2023-10-16T20:04:50.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.519+0000 7f28c17fa640 1 -- 172.21.15.88:0/577652703 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc0adb50 con 0x7f28c415d9f0 2023-10-16T20:04:50.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.519+0000 7f28caaf7640 1 -- 172.21.15.88:0/577652703 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c4155380 con 0x7f28c415d9f0 2023-10-16T20:04:50.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.519+0000 7f28caaf7640 1 -- 172.21.15.88:0/577652703 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c4155940 con 0x7f28c415d9f0 2023-10-16T20:04:50.531 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.519+0000 7f28caaf7640 1 -- 172.21.15.88:0/577652703 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c4155b30 con 0x7f28c415d9f0 2023-10-16T20:04:50.531 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.519+0000 7f28c17fa640 1 -- 172.21.15.88:0/577652703 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f28bc0b4050 con 0x7f28c415d9f0 2023-10-16T20:04:50.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.523+0000 7f28c17fa640 1 --2- 172.21.15.88:0/577652703 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f28a80664a0 0x7f28a8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:50.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.523+0000 7f28c17fa640 1 -- 172.21.15.88:0/577652703 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f28bc11eb40 con 0x7f28c415d9f0 2023-10-16T20:04:50.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.523+0000 7f28c8893640 1 --2- 172.21.15.88:0/577652703 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f28a80664a0 0x7f28a8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:50.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.523+0000 7f28c8893640 1 --2- 172.21.15.88:0/577652703 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f28a80664a0 0x7f28a8068960 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f28b8001ec0 tx=0x7f28b8001e50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:50.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.527+0000 7f28c17fa640 1 -- 172.21.15.88:0/577652703 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f28bc0ec3b0 con 0x7f28c415d9f0 2023-10-16T20:04:50.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.707+0000 7f28caaf7640 1 -- 172.21.15.88:0/577652703 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28c404e9c0 con 0x7f28c415d9f0 2023-10-16T20:04:50.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.707+0000 7f28c17fa640 1 -- 172.21.15.88:0/577652703 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7f28bc0c7540 con 0x7f28c415d9f0 2023-10-16T20:04:50.713 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:04:50.713 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/3836772385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3836772385},{"type":"v1","addr":"172.21.15.88:6836","nonce":3836772385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-16T20:04:50.714 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 2 2023-10-16T20:04:50.714 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.711+0000 7f28caaf7640 1 -- 172.21.15.88:0/577652703 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f28a80664a0 msgr2=0x7f28a8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:50.714 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.711+0000 7f28caaf7640 1 --2- 172.21.15.88:0/577652703 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f28a80664a0 0x7f28a8068960 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f28b8001ec0 tx=0x7f28b8001e50 comp rx=0 tx=0).stop 2023-10-16T20:04:50.715 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.711+0000 7f28caaf7640 1 -- 172.21.15.88:0/577652703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28c415d9f0 msgr2=0x7f28c41501c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:50.715 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.711+0000 7f28caaf7640 1 --2- 172.21.15.88:0/577652703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28c415d9f0 0x7f28c41501c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f28bc0b0040 tx=0x7f28bc019550 comp rx=0 tx=0).stop 2023-10-16T20:04:50.715 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.711+0000 7f28caaf7640 1 -- 172.21.15.88:0/577652703 shutdown_connections 2023-10-16T20:04:50.715 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.711+0000 7f28caaf7640 1 --2- 172.21.15.88:0/577652703 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f28a80664a0 0x7f28a8068960 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:50.715 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.711+0000 7f28caaf7640 1 --2- 172.21.15.88:0/577652703 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f28c40d26c0 0x7f28c4151840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:50.716 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.711+0000 7f28caaf7640 1 --2- 172.21.15.88:0/577652703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28c415d9f0 0x7f28c41501c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:50.716 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.711+0000 7f28caaf7640 1 --2- 172.21.15.88:0/577652703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28c40d22e0 0x7f28c4151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:50.716 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.711+0000 7f28caaf7640 1 -- 172.21.15.88:0/577652703 >> 172.21.15.88:0/577652703 conn(0x7f28c40715c0 msgr2=0x7f28c415d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:50.716 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.711+0000 7f28caaf7640 1 -- 172.21.15.88:0/577652703 shutdown_connections 2023-10-16T20:04:50.717 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:50.711+0000 7f28caaf7640 1 -- 172.21.15.88:0/577652703 wait complete. 2023-10-16T20:04:52.735 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T20:04:52.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.879+0000 7f35bca46640 1 Processor -- start 2023-10-16T20:04:52.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.879+0000 7f35bca46640 1 -- start start 2023-10-16T20:04:52.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.879+0000 7f35bca46640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f35b815d9f0 0x7f35b8161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:52.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.879+0000 7f35bca46640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f35b80d26c0 0x7f35b80cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:52.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.879+0000 7f35bca46640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35b80b87e0 con 0x7f35b80d22e0 2023-10-16T20:04:52.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.879+0000 7f35bca46640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b815df10 con 0x7f35b80d26c0 2023-10-16T20:04:52.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.879+0000 7f35bca46640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b815e1f0 con 0x7f35b815d9f0 2023-10-16T20:04:52.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.879+0000 7f35b6d9d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f35b80d26c0 0x7f35b80cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:52.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.879+0000 7f35b6d9d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f35b80d26c0 0x7f35b80cf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41346/0 (socket says 172.21.15.88:41346) 2023-10-16T20:04:52.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.879+0000 7f35b6d9d640 1 -- 172.21.15.88:0/3431385228 learned_addr learned my addr 172.21.15.88:0/3431385228 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:04:52.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.879+0000 7f35add9b640 1 --2- 172.21.15.88:0/3431385228 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f35b815d9f0 0x7f35b8161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:52.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.879+0000 7f35b6d9d640 1 -- 172.21.15.88:0/3431385228 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f35b815d9f0 msgr2=0x7f35b8161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:04:52.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.879+0000 7f35b6d9d640 1 --2- 172.21.15.88:0/3431385228 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f35b815d9f0 0x7f35b8161450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:52.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.879+0000 7f35b6d9d640 1 -- 172.21.15.88:0/3431385228 >> v1:172.21.15.94:6789/0 conn(0x7f35b80d22e0 legacy=0x7f35b815d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:52.887 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.879+0000 7f35b6d9d640 1 -- 172.21.15.88:0/3431385228 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b8160970 con 0x7f35b80d26c0 2023-10-16T20:04:52.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.883+0000 7f35b6d9d640 1 --2- 172.21.15.88:0/3431385228 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f35b80d26c0 0x7f35b80cf2e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f35b0009e50 tx=0x7f35b001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0c7d6213c813a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:52.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.883+0000 7f35b5d9b640 1 -- 172.21.15.88:0/3431385228 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b00a5810 con 0x7f35b80d26c0 2023-10-16T20:04:52.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.883+0000 7f35b5d9b640 1 -- 172.21.15.88:0/3431385228 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b00a5970 con 0x7f35b80d26c0 2023-10-16T20:04:52.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.883+0000 7f35b5d9b640 1 -- 172.21.15.88:0/3431385228 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b00a5d00 con 0x7f35b80d26c0 2023-10-16T20:04:52.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.883+0000 7f35bca46640 1 -- 172.21.15.88:0/3431385228 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f35b80d26c0 msgr2=0x7f35b80cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:52.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.883+0000 7f35bca46640 1 --2- 172.21.15.88:0/3431385228 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f35b80d26c0 0x7f35b80cf2e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f35b0009e50 tx=0x7f35b001c950 comp rx=0 tx=0).stop 2023-10-16T20:04:52.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.883+0000 7f35bca46640 1 -- 172.21.15.88:0/3431385228 shutdown_connections 2023-10-16T20:04:52.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.883+0000 7f35bca46640 1 --2- 172.21.15.88:0/3431385228 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f35b80d26c0 0x7f35b80cf2e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:52.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.883+0000 7f35bca46640 1 --2- 172.21.15.88:0/3431385228 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f35b815d9f0 0x7f35b8161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:52.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.883+0000 7f35bca46640 1 -- 172.21.15.88:0/3431385228 >> 172.21.15.88:0/3431385228 conn(0x7f35b80715c0 msgr2=0x7f35b80719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:52.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.883+0000 7f35bca46640 1 -- 172.21.15.88:0/3431385228 shutdown_connections 2023-10-16T20:04:52.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.883+0000 7f35bca46640 1 -- 172.21.15.88:0/3431385228 wait complete. 2023-10-16T20:04:52.893 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.887+0000 7f35bca46640 1 Processor -- start 2023-10-16T20:04:52.893 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.887+0000 7f35bca46640 1 -- start start 2023-10-16T20:04:52.893 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.887+0000 7f35bca46640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f35b80d22e0 0x7f35b814a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:52.893 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.887+0000 7f35bca46640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35b80d26c0 0x7f35b81f6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:52.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.887+0000 7f35bca46640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f35b815d9f0 0x7f35b814ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:52.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.887+0000 7f35bca46640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b81516c0 con 0x7f35b815d9f0 2023-10-16T20:04:52.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.887+0000 7f35bca46640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b8151960 con 0x7f35b80d26c0 2023-10-16T20:04:52.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.887+0000 7f35bca46640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b8151c40 con 0x7f35b80d22e0 2023-10-16T20:04:52.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.887+0000 7f35b6d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f35b815d9f0 0x7f35b814ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:52.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.887+0000 7f35b6d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f35b815d9f0 0x7f35b814ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41360/0 (socket says 172.21.15.88:41360) 2023-10-16T20:04:52.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.887+0000 7f35b6d9d640 1 -- 172.21.15.88:0/3245710198 learned_addr learned my addr 172.21.15.88:0/3245710198 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:04:52.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.887+0000 7f35add9b640 1 --2- 172.21.15.88:0/3245710198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35b80d26c0 0x7f35b81f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:52.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.887+0000 7f35b659c640 1 --2- 172.21.15.88:0/3245710198 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f35b80d22e0 0x7f35b814a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:52.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.887+0000 7f35b6d9d640 1 -- 172.21.15.88:0/3245710198 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f35b80d22e0 msgr2=0x7f35b814a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:52.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.887+0000 7f35b6d9d640 1 --2- 172.21.15.88:0/3245710198 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f35b80d22e0 0x7f35b814a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:52.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.887+0000 7f35b6d9d640 1 -- 172.21.15.88:0/3245710198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35b80d26c0 msgr2=0x7f35b81f6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:52.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.887+0000 7f35b6d9d640 1 --2- 172.21.15.88:0/3245710198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35b80d26c0 0x7f35b81f6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:52.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.887+0000 7f35b6d9d640 1 -- 172.21.15.88:0/3245710198 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b0009b30 con 0x7f35b815d9f0 2023-10-16T20:04:52.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.887+0000 7f35b659c640 1 --2- 172.21.15.88:0/3245710198 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f35b80d22e0 0x7f35b814a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:04:52.897 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.887+0000 7f35b6d9d640 1 --2- 172.21.15.88:0/3245710198 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f35b815d9f0 0x7f35b814ec30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f35b001a580 tx=0x7f35b00196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:52.897 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.887+0000 7f35af7fe640 1 -- 172.21.15.88:0/3245710198 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b00b8670 con 0x7f35b815d9f0 2023-10-16T20:04:52.897 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.887+0000 7f35af7fe640 1 -- 172.21.15.88:0/3245710198 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b00b87d0 con 0x7f35b815d9f0 2023-10-16T20:04:52.897 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.887+0000 7f35bca46640 1 -- 172.21.15.88:0/3245710198 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b8151f20 con 0x7f35b815d9f0 2023-10-16T20:04:52.897 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.887+0000 7f35af7fe640 1 -- 172.21.15.88:0/3245710198 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b00b8b80 con 0x7f35b815d9f0 2023-10-16T20:04:52.898 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.887+0000 7f35bca46640 1 -- 172.21.15.88:0/3245710198 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b814ff60 con 0x7f35b815d9f0 2023-10-16T20:04:52.899 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.891+0000 7f35bca46640 1 -- 172.21.15.88:0/3245710198 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b804e9c0 con 0x7f35b815d9f0 2023-10-16T20:04:52.899 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.891+0000 7f35af7fe640 1 -- 172.21.15.88:0/3245710198 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f35b00b8ce0 con 0x7f35b815d9f0 2023-10-16T20:04:52.899 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.891+0000 7f35af7fe640 1 --2- 172.21.15.88:0/3245710198 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f35980664a0 0x7f3598068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:52.899 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.891+0000 7f35af7fe640 1 -- 172.21.15.88:0/3245710198 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f35b011eff0 con 0x7f35b815d9f0 2023-10-16T20:04:52.899 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.891+0000 7f35b659c640 1 --2- 172.21.15.88:0/3245710198 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f35980664a0 0x7f3598068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:52.900 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.891+0000 7f35b659c640 1 --2- 172.21.15.88:0/3245710198 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f35980664a0 0x7f3598068960 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f35a8000f70 tx=0x7f35a8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:52.900 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:52.895+0000 7f35af7fe640 1 -- 172.21.15.88:0/3245710198 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f35b0150090 con 0x7f35b815d9f0 2023-10-16T20:04:53.080 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.075+0000 7f35bca46640 1 -- 172.21.15.88:0/3245710198 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35b81523d0 con 0x7f35b815d9f0 2023-10-16T20:04:53.081 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.079+0000 7f35af7fe640 1 -- 172.21.15.88:0/3245710198 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3691 (secure 0 0 0) 0x7f35b00ec8b0 con 0x7f35b815d9f0 2023-10-16T20:04:53.081 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:04:53.082 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3803627113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3803627113},{"type":"v1","addr":"172.21.15.94:6834","nonce":3803627113}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3758093239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":3758093239},{"type":"v1","addr":"172.21.15.88:6837","nonce":3758093239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/3836772385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3836772385},{"type":"v1","addr":"172.21.15.88:6836","nonce":3836772385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/555575750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":555575750},{"type":"v1","addr":"172.21.15.94:6835","nonce":555575750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-10-16T20:04:53.082 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 3 2023-10-16T20:04:53.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.079+0000 7f35bca46640 1 -- 172.21.15.88:0/3245710198 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f35980664a0 msgr2=0x7f3598068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:53.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.079+0000 7f35bca46640 1 --2- 172.21.15.88:0/3245710198 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f35980664a0 0x7f3598068960 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f35a8000f70 tx=0x7f35a8009040 comp rx=0 tx=0).stop 2023-10-16T20:04:53.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.079+0000 7f35bca46640 1 -- 172.21.15.88:0/3245710198 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f35b815d9f0 msgr2=0x7f35b814ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:53.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.079+0000 7f35bca46640 1 --2- 172.21.15.88:0/3245710198 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f35b815d9f0 0x7f35b814ec30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f35b001a580 tx=0x7f35b00196f0 comp rx=0 tx=0).stop 2023-10-16T20:04:53.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.079+0000 7f35bca46640 1 -- 172.21.15.88:0/3245710198 shutdown_connections 2023-10-16T20:04:53.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.079+0000 7f35bca46640 1 --2- 172.21.15.88:0/3245710198 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f35980664a0 0x7f3598068960 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:53.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.079+0000 7f35bca46640 1 --2- 172.21.15.88:0/3245710198 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f35b815d9f0 0x7f35b814ec30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:53.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.079+0000 7f35bca46640 1 --2- 172.21.15.88:0/3245710198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35b80d26c0 0x7f35b81f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:53.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.079+0000 7f35bca46640 1 --2- 172.21.15.88:0/3245710198 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f35b80d22e0 0x7f35b814a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:53.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.079+0000 7f35bca46640 1 -- 172.21.15.88:0/3245710198 >> 172.21.15.88:0/3245710198 conn(0x7f35b80715c0 msgr2=0x7f35b815aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:53.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.079+0000 7f35bca46640 1 -- 172.21.15.88:0/3245710198 shutdown_connections 2023-10-16T20:04:53.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.079+0000 7f35bca46640 1 -- 172.21.15.88:0/3245710198 wait complete. 2023-10-16T20:04:53.102 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:04:53.238 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.235+0000 7f411d654640 1 Processor -- start 2023-10-16T20:04:53.239 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.235+0000 7f411d654640 1 -- start start 2023-10-16T20:04:53.239 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.235+0000 7f411d654640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f41180d0110 0x7f4118161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:53.239 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.235+0000 7f411d654640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41180cdad0 0x7f41180b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:53.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.235+0000 7f411d654640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41180b2190 con 0x7f411815cb70 2023-10-16T20:04:53.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.235+0000 7f411d654640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41181615a0 con 0x7f41180d0110 2023-10-16T20:04:53.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.235+0000 7f411d654640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411815df60 con 0x7f41180cdad0 2023-10-16T20:04:53.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.235+0000 7f4116ffd640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f411815cb70 0x7f41180cfa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:44774/0 (socket says 172.21.15.88:44774) 2023-10-16T20:04:53.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.235+0000 7f4116ffd640 1 -- 172.21.15.88:0/3935308883 learned_addr learned my addr 172.21.15.88:0/3935308883 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:04:53.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.235+0000 7f410e7fc640 1 --2- 172.21.15.88:0/3935308883 >> v2:172.21.15.94:3300/0 conn(0x7f41180d0110 0x7f4118161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:53.242 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.235+0000 7f41177fe640 1 --2- 172.21.15.88:0/3935308883 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41180cdad0 0x7f41180b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:53.242 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.235+0000 7f41167fc640 1 -- 172.21.15.88:0/3935308883 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1239969023 0 0) 0x7f41180b2190 con 0x7f411815cb70 2023-10-16T20:04:53.242 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.235+0000 7f41167fc640 1 -- 172.21.15.88:0/3935308883 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4104003530 con 0x7f411815cb70 2023-10-16T20:04:53.242 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.235+0000 7f41167fc640 1 -- 172.21.15.88:0/3935308883 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 624167097 0 0) 0x7f4104003530 con 0x7f411815cb70 2023-10-16T20:04:53.242 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.235+0000 7f410e7fc640 1 -- 172.21.15.88:0/3935308883 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41180cdad0 msgr2=0x7f41180b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:04:53.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.235+0000 7f410e7fc640 1 --2- 172.21.15.88:0/3935308883 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41180cdad0 0x7f41180b7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:53.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.235+0000 7f410e7fc640 1 -- 172.21.15.88:0/3935308883 >> v1:172.21.15.88:6789/0 conn(0x7f411815cb70 legacy=0x7f41180cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:53.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.235+0000 7f410e7fc640 1 -- 172.21.15.88:0/3935308883 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411815e230 con 0x7f41180d0110 2023-10-16T20:04:53.244 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.239+0000 7f410e7fc640 1 --2- 172.21.15.88:0/3935308883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41180d0110 0x7f4118161060 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f41000097c0 tx=0x7f410001c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c60feb6366110f44 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:53.244 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.239+0000 7f41167fc640 1 -- 172.21.15.88:0/3935308883 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4108001680 con 0x7f41180d0110 2023-10-16T20:04:53.245 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.239+0000 7f41167fc640 1 -- 172.21.15.88:0/3935308883 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41000a7050 con 0x7f41180d0110 2023-10-16T20:04:53.245 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.239+0000 7f41167fc640 1 -- 172.21.15.88:0/3935308883 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41000ab620 con 0x7f41180d0110 2023-10-16T20:04:53.245 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.239+0000 7f411d654640 1 -- 172.21.15.88:0/3935308883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41180d0110 msgr2=0x7f4118161060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:53.245 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.239+0000 7f411d654640 1 --2- 172.21.15.88:0/3935308883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41180d0110 0x7f4118161060 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f41000097c0 tx=0x7f410001c3a0 comp rx=0 tx=0).stop 2023-10-16T20:04:53.245 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.239+0000 7f411d654640 1 -- 172.21.15.88:0/3935308883 shutdown_connections 2023-10-16T20:04:53.246 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.239+0000 7f411d654640 1 --2- 172.21.15.88:0/3935308883 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41180cdad0 0x7f41180b7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:53.246 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.239+0000 7f411d654640 1 --2- 172.21.15.88:0/3935308883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41180d0110 0x7f4118161060 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:53.246 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.239+0000 7f411d654640 1 -- 172.21.15.88:0/3935308883 >> 172.21.15.88:0/3935308883 conn(0x7f41180719c0 msgr2=0x7f4118071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:53.246 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.239+0000 7f411d654640 1 -- 172.21.15.88:0/3935308883 shutdown_connections 2023-10-16T20:04:53.246 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.239+0000 7f411d654640 1 -- 172.21.15.88:0/3935308883 wait complete. 2023-10-16T20:04:53.249 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.243+0000 7f411d654640 1 Processor -- start 2023-10-16T20:04:53.249 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.243+0000 7f411d654640 1 -- start start 2023-10-16T20:04:53.249 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.243+0000 7f411d654640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41180cdad0 0x7f411806b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:53.249 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.243+0000 7f411d654640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41180d0110 0x7f41180c7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:53.249 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.243+0000 7f411d654640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f411815cb70 0x7f41180c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:53.250 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.243+0000 7f411d654640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41180d2260 con 0x7f411815cb70 2023-10-16T20:04:53.250 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.243+0000 7f411d654640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41180c8e70 con 0x7f41180d0110 2023-10-16T20:04:53.250 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.243+0000 7f411d654640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41180c8260 con 0x7f41180cdad0 2023-10-16T20:04:53.250 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.243+0000 7f41177fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f411815cb70 0x7f41180c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:53.250 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.243+0000 7f41177fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f411815cb70 0x7f41180c7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41368/0 (socket says 172.21.15.88:41368) 2023-10-16T20:04:53.251 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.243+0000 7f41177fe640 1 -- 172.21.15.88:0/2977499313 learned_addr learned my addr 172.21.15.88:0/2977499313 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:04:53.251 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.243+0000 7f4116ffd640 1 --2- 172.21.15.88:0/2977499313 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41180cdad0 0x7f411806b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:53.251 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.243+0000 7f410e7fc640 1 --2- 172.21.15.88:0/2977499313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41180d0110 0x7f41180c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:53.251 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.243+0000 7f41177fe640 1 -- 172.21.15.88:0/2977499313 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41180cdad0 msgr2=0x7f411806b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:53.251 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.243+0000 7f41177fe640 1 --2- 172.21.15.88:0/2977499313 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41180cdad0 0x7f411806b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:53.252 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.243+0000 7f41177fe640 1 -- 172.21.15.88:0/2977499313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41180d0110 msgr2=0x7f41180c7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:53.252 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.243+0000 7f41177fe640 1 --2- 172.21.15.88:0/2977499313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41180d0110 0x7f41180c7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:53.252 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.243+0000 7f41177fe640 1 -- 172.21.15.88:0/2977499313 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4100009420 con 0x7f411815cb70 2023-10-16T20:04:53.252 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.243+0000 7f410e7fc640 1 --2- 172.21.15.88:0/2977499313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41180d0110 0x7f41180c7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:04:53.252 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.243+0000 7f41177fe640 1 --2- 172.21.15.88:0/2977499313 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f411815cb70 0x7f41180c7b50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f411000efc0 tx=0x7f411000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:53.253 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.243+0000 7f410ffff640 1 -- 172.21.15.88:0/2977499313 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4110014480 con 0x7f411815cb70 2023-10-16T20:04:53.253 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.243+0000 7f410ffff640 1 -- 172.21.15.88:0/2977499313 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411000f040 con 0x7f411815cb70 2023-10-16T20:04:53.253 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.243+0000 7f410ffff640 1 -- 172.21.15.88:0/2977499313 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4110014820 con 0x7f411815cb70 2023-10-16T20:04:53.253 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.243+0000 7f411d654640 1 -- 172.21.15.88:0/2977499313 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41180c8540 con 0x7f411815cb70 2023-10-16T20:04:53.253 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.243+0000 7f411d654640 1 -- 172.21.15.88:0/2977499313 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41180c8b00 con 0x7f411815cb70 2023-10-16T20:04:53.254 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.247+0000 7f411d654640 1 -- 172.21.15.88:0/2977499313 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41180c8fe0 con 0x7f411815cb70 2023-10-16T20:04:53.254 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.247+0000 7f410ffff640 1 -- 172.21.15.88:0/2977499313 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4110014980 con 0x7f411815cb70 2023-10-16T20:04:53.255 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.247+0000 7f410ffff640 1 --2- 172.21.15.88:0/2977499313 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f40f80664a0 0x7f40f8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:53.255 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.247+0000 7f410ffff640 1 -- 172.21.15.88:0/2977499313 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4110084980 con 0x7f411815cb70 2023-10-16T20:04:53.255 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.247+0000 7f4116ffd640 1 --2- 172.21.15.88:0/2977499313 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f40f80664a0 0x7f40f8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:53.255 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.247+0000 7f4116ffd640 1 --2- 172.21.15.88:0/2977499313 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f40f80664a0 0x7f40f8068960 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4108001b00 tx=0x7f4108009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:53.256 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.251+0000 7f410ffff640 1 -- 172.21.15.88:0/2977499313 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f41100521c0 con 0x7f411815cb70 2023-10-16T20:04:53.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.403+0000 7f411d654640 1 -- 172.21.15.88:0/2977499313 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f411804e9c0 con 0x7f411815cb70 2023-10-16T20:04:53.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.403+0000 7f410ffff640 1 -- 172.21.15.88:0/2977499313 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+12778 (secure 0 0 0) 0x7f41100557e0 con 0x7f411815cb70 2023-10-16T20:04:53.408 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:04:53.408 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":19,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:04:47.326789+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:04:53.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.407+0000 7f411d654640 1 -- 172.21.15.88:0/2977499313 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f40f80664a0 msgr2=0x7f40f8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:53.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.407+0000 7f411d654640 1 --2- 172.21.15.88:0/2977499313 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f40f80664a0 0x7f40f8068960 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4108001b00 tx=0x7f4108009040 comp rx=0 tx=0).stop 2023-10-16T20:04:53.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.407+0000 7f411d654640 1 -- 172.21.15.88:0/2977499313 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f411815cb70 msgr2=0x7f41180c7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:53.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.407+0000 7f411d654640 1 --2- 172.21.15.88:0/2977499313 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f411815cb70 0x7f41180c7b50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f411000efc0 tx=0x7f411000c490 comp rx=0 tx=0).stop 2023-10-16T20:04:53.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.407+0000 7f411d654640 1 -- 172.21.15.88:0/2977499313 shutdown_connections 2023-10-16T20:04:53.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.407+0000 7f411d654640 1 --2- 172.21.15.88:0/2977499313 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f40f80664a0 0x7f40f8068960 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:53.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.407+0000 7f411d654640 1 --2- 172.21.15.88:0/2977499313 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f411815cb70 0x7f41180c7b50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:53.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.407+0000 7f411d654640 1 --2- 172.21.15.88:0/2977499313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41180d0110 0x7f41180c7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:53.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.407+0000 7f411d654640 1 --2- 172.21.15.88:0/2977499313 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41180cdad0 0x7f411806b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:53.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.407+0000 7f411d654640 1 -- 172.21.15.88:0/2977499313 >> 172.21.15.88:0/2977499313 conn(0x7f41180719c0 msgr2=0x7f41180b7230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:53.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.407+0000 7f411d654640 1 -- 172.21.15.88:0/2977499313 shutdown_connections 2023-10-16T20:04:53.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.407+0000 7f411d654640 1 -- 172.21.15.88:0/2977499313 wait complete. 2023-10-16T20:04:53.429 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T20:04:53.430 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:04:53.571 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.567+0000 7f516cfc5640 1 Processor -- start 2023-10-16T20:04:53.571 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.567+0000 7f516cfc5640 1 -- start start 2023-10-16T20:04:53.572 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.567+0000 7f516cfc5640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f516815ac00 0x7f51680b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:53.572 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.567+0000 7f516cfc5640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51680ce4b0 con 0x7f51680b2930 2023-10-16T20:04:53.573 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.567+0000 7f516cfc5640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51680cf8f0 con 0x7f516815a820 2023-10-16T20:04:53.573 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.567+0000 7f516cfc5640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51680b2d40 con 0x7f516815ac00 2023-10-16T20:04:53.573 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.567+0000 7f515dd9b640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f516815ac00 0x7f51680b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:53.573 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.567+0000 7f5166d9d640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f51680b2930 0x7f516806c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:44784/0 (socket says 172.21.15.88:44784) 2023-10-16T20:04:53.573 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.567+0000 7f515dd9b640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f516815ac00 0x7f51680b23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41378/0 (socket says 172.21.15.88:41378) 2023-10-16T20:04:53.573 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.567+0000 7f5166d9d640 1 -- 172.21.15.88:0/2088932156 learned_addr learned my addr 172.21.15.88:0/2088932156 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:04:53.574 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.571+0000 7f515dd9b640 1 -- 172.21.15.88:0/2088932156 >> v1:172.21.15.94:6789/0 conn(0x7f516815a820 legacy=0x7f51680cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:53.574 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.571+0000 7f515dd9b640 1 -- 172.21.15.88:0/2088932156 >> v1:172.21.15.88:6789/0 conn(0x7f51680b2930 legacy=0x7f516806c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:53.574 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.571+0000 7f515dd9b640 1 -- 172.21.15.88:0/2088932156 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f516815e270 con 0x7f516815ac00 2023-10-16T20:04:53.575 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.571+0000 7f515dd9b640 1 --2- 172.21.15.88:0/2088932156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f516815ac00 0x7f51680b23f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f51580097c0 tx=0x7f515801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df7244e7104dd71c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:53.576 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.571+0000 7f5165d9b640 1 -- 172.21.15.88:0/2088932156 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51580ad660 con 0x7f516815ac00 2023-10-16T20:04:53.577 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.571+0000 7f5165d9b640 1 -- 172.21.15.88:0/2088932156 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51580ad7c0 con 0x7f516815ac00 2023-10-16T20:04:53.577 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.571+0000 7f5165d9b640 1 -- 172.21.15.88:0/2088932156 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51580adb50 con 0x7f516815ac00 2023-10-16T20:04:53.577 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.571+0000 7f516cfc5640 1 -- 172.21.15.88:0/2088932156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f516815ac00 msgr2=0x7f51680b23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:53.577 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.571+0000 7f516cfc5640 1 --2- 172.21.15.88:0/2088932156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f516815ac00 0x7f51680b23f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f51580097c0 tx=0x7f515801c450 comp rx=0 tx=0).stop 2023-10-16T20:04:53.577 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.571+0000 7f516cfc5640 1 -- 172.21.15.88:0/2088932156 shutdown_connections 2023-10-16T20:04:53.577 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.571+0000 7f516cfc5640 1 --2- 172.21.15.88:0/2088932156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f516815ac00 0x7f51680b23f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:53.578 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.571+0000 7f516cfc5640 1 -- 172.21.15.88:0/2088932156 >> 172.21.15.88:0/2088932156 conn(0x7f51680cd3f0 msgr2=0x7f51680cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:53.578 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.571+0000 7f516cfc5640 1 -- 172.21.15.88:0/2088932156 shutdown_connections 2023-10-16T20:04:53.578 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.571+0000 7f516cfc5640 1 -- 172.21.15.88:0/2088932156 wait complete. 2023-10-16T20:04:53.579 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.575+0000 7f516cfc5640 1 Processor -- start 2023-10-16T20:04:53.579 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.575+0000 7f516cfc5640 1 -- start start 2023-10-16T20:04:53.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.575+0000 7f516cfc5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51680b2930 0x7f516814e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:53.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.575+0000 7f516cfc5640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f516815a820 0x7f51680b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:53.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.575+0000 7f516cfc5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f516815ac00 0x7f51680b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:53.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.575+0000 7f516cfc5640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5168155a60 con 0x7f516815ac00 2023-10-16T20:04:53.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.575+0000 7f516cfc5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51680b4050 con 0x7f51680b2930 2023-10-16T20:04:53.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.575+0000 7f516cfc5640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51680b4330 con 0x7f516815a820 2023-10-16T20:04:53.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.575+0000 7f5166d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f516815ac00 0x7f51680b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:53.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.575+0000 7f5166d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f516815ac00 0x7f51680b50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41388/0 (socket says 172.21.15.88:41388) 2023-10-16T20:04:53.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.575+0000 7f5166d9d640 1 -- 172.21.15.88:0/524659202 learned_addr learned my addr 172.21.15.88:0/524659202 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:04:53.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.575+0000 7f516659c640 1 --2- 172.21.15.88:0/524659202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51680b2930 0x7f516814e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:53.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.575+0000 7f515dd9b640 1 --2- 172.21.15.88:0/524659202 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f516815a820 0x7f51680b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:53.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.575+0000 7f5166d9d640 1 -- 172.21.15.88:0/524659202 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f516815a820 msgr2=0x7f51680b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:53.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.575+0000 7f5166d9d640 1 --2- 172.21.15.88:0/524659202 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f516815a820 0x7f51680b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:53.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.575+0000 7f5166d9d640 1 -- 172.21.15.88:0/524659202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51680b2930 msgr2=0x7f516814e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:53.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.575+0000 7f5166d9d640 1 --2- 172.21.15.88:0/524659202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51680b2930 0x7f516814e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:53.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.575+0000 7f5166d9d640 1 -- 172.21.15.88:0/524659202 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5158009420 con 0x7f516815ac00 2023-10-16T20:04:53.585 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.579+0000 7f5166d9d640 1 --2- 172.21.15.88:0/524659202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f516815ac00 0x7f51680b50f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f516000cbc0 tx=0x7f516000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:53.585 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.579+0000 7f515f7fe640 1 -- 172.21.15.88:0/524659202 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516000ad40 con 0x7f516815ac00 2023-10-16T20:04:53.585 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.579+0000 7f515f7fe640 1 -- 172.21.15.88:0/524659202 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5160012400 con 0x7f516815ac00 2023-10-16T20:04:53.585 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.579+0000 7f516cfc5640 1 -- 172.21.15.88:0/524659202 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51680b4610 con 0x7f516815ac00 2023-10-16T20:04:53.585 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.579+0000 7f515f7fe640 1 -- 172.21.15.88:0/524659202 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516000aea0 con 0x7f516815ac00 2023-10-16T20:04:53.586 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.579+0000 7f516cfc5640 1 -- 172.21.15.88:0/524659202 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51680b49a0 con 0x7f516815ac00 2023-10-16T20:04:53.586 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.579+0000 7f516cfc5640 1 -- 172.21.15.88:0/524659202 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51680b4e10 con 0x7f516815ac00 2023-10-16T20:04:53.586 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.579+0000 7f515f7fe640 1 -- 172.21.15.88:0/524659202 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5160007070 con 0x7f516815ac00 2023-10-16T20:04:53.586 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.579+0000 7f515f7fe640 1 --2- 172.21.15.88:0/524659202 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f51480664a0 0x7f5148068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:53.586 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.579+0000 7f515f7fe640 1 -- 172.21.15.88:0/524659202 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5160083a60 con 0x7f516815ac00 2023-10-16T20:04:53.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.579+0000 7f516659c640 1 --2- 172.21.15.88:0/524659202 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f51480664a0 0x7f5148068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:53.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.583+0000 7f516659c640 1 --2- 172.21.15.88:0/524659202 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f51480664a0 0x7f5148068960 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5150007fd0 tx=0x7f5150009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:53.588 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.583+0000 7f515f7fe640 1 -- 172.21.15.88:0/524659202 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f51680b4e10 con 0x7f516815ac00 2023-10-16T20:04:53.735 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.731+0000 7f516cfc5640 1 -- 172.21.15.88:0/524659202 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f51680b4e10 con 0x7f516815ac00 2023-10-16T20:04:53.735 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.731+0000 7f515f7fe640 1 -- 172.21.15.88:0/524659202 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7f51680b4e10 con 0x7f516815ac00 2023-10-16T20:04:53.736 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:04:53.738 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.735+0000 7f516cfc5640 1 -- 172.21.15.88:0/524659202 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f51480664a0 msgr2=0x7f5148068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:53.738 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.735+0000 7f516cfc5640 1 --2- 172.21.15.88:0/524659202 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f51480664a0 0x7f5148068960 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5150007fd0 tx=0x7f5150009040 comp rx=0 tx=0).stop 2023-10-16T20:04:53.738 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.735+0000 7f516cfc5640 1 -- 172.21.15.88:0/524659202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f516815ac00 msgr2=0x7f51680b50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:53.738 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.735+0000 7f516cfc5640 1 --2- 172.21.15.88:0/524659202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f516815ac00 0x7f51680b50f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f516000cbc0 tx=0x7f516000a430 comp rx=0 tx=0).stop 2023-10-16T20:04:53.738 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.735+0000 7f516cfc5640 1 -- 172.21.15.88:0/524659202 shutdown_connections 2023-10-16T20:04:53.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.735+0000 7f516cfc5640 1 --2- 172.21.15.88:0/524659202 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f51480664a0 0x7f5148068960 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:53.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.735+0000 7f516cfc5640 1 --2- 172.21.15.88:0/524659202 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f516815a820 0x7f51680b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:53.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.735+0000 7f516cfc5640 1 --2- 172.21.15.88:0/524659202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f516815ac00 0x7f51680b50f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:53.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.735+0000 7f516cfc5640 1 --2- 172.21.15.88:0/524659202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51680b2930 0x7f516814e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:53.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.735+0000 7f516cfc5640 1 -- 172.21.15.88:0/524659202 >> 172.21.15.88:0/524659202 conn(0x7f51680cd3f0 msgr2=0x7f516806ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:53.740 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.735+0000 7f516cfc5640 1 -- 172.21.15.88:0/524659202 shutdown_connections 2023-10-16T20:04:53.740 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.735+0000 7f516cfc5640 1 -- 172.21.15.88:0/524659202 wait complete. 2023-10-16T20:04:53.756 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T20:04:53.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.891+0000 7fe0958ea640 1 Processor -- start 2023-10-16T20:04:53.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.891+0000 7fe0958ea640 1 -- start start 2023-10-16T20:04:53.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.891+0000 7fe0958ea640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fe0900d0110 0x7fe090161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:53.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.891+0000 7fe0958ea640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0900b2190 con 0x7fe09015cb70 2023-10-16T20:04:53.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.891+0000 7fe0958ea640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe09015df60 con 0x7fe0900cdad0 2023-10-16T20:04:53.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.891+0000 7fe0958ea640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0901615a0 con 0x7fe0900d0110 2023-10-16T20:04:53.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.891+0000 7fe08effd640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fe09015cb70 0x7fe0900cfa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:44786/0 (socket says 172.21.15.88:44786) 2023-10-16T20:04:53.897 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.891+0000 7fe08effd640 1 -- 172.21.15.88:0/1498826423 learned_addr learned my addr 172.21.15.88:0/1498826423 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:04:53.897 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.891+0000 7fe0867fc640 1 --2- 172.21.15.88:0/1498826423 >> v2:172.21.15.88:3300/0 conn(0x7fe0900d0110 0x7fe090161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:53.898 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.891+0000 7fe08e7fc640 1 -- 172.21.15.88:0/1498826423 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 834401995 0 0) 0x7fe0900b2190 con 0x7fe09015cb70 2023-10-16T20:04:53.898 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.891+0000 7fe08e7fc640 1 -- 172.21.15.88:0/1498826423 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe07c003530 con 0x7fe09015cb70 2023-10-16T20:04:53.898 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.891+0000 7fe0867fc640 1 -- 172.21.15.88:0/1498826423 >> v1:172.21.15.94:6789/0 conn(0x7fe0900cdad0 legacy=0x7fe0900b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:04:53.898 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.891+0000 7fe0867fc640 1 -- 172.21.15.88:0/1498826423 >> v1:172.21.15.88:6789/0 conn(0x7fe09015cb70 legacy=0x7fe0900cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:53.898 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.891+0000 7fe0867fc640 1 -- 172.21.15.88:0/1498826423 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe09015e230 con 0x7fe0900d0110 2023-10-16T20:04:53.899 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.895+0000 7fe0867fc640 1 --2- 172.21.15.88:0/1498826423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0900d0110 0x7fe090161060 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe0780097c0 tx=0x7fe07801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8feca47db2c1bc3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:53.899 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.895+0000 7fe08e7fc640 1 -- 172.21.15.88:0/1498826423 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0780ab600 con 0x7fe0900d0110 2023-10-16T20:04:53.900 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.895+0000 7fe08e7fc640 1 -- 172.21.15.88:0/1498826423 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0780a7050 con 0x7fe0900d0110 2023-10-16T20:04:53.900 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.895+0000 7fe08e7fc640 1 -- 172.21.15.88:0/1498826423 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0780ab960 con 0x7fe0900d0110 2023-10-16T20:04:53.900 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.895+0000 7fe0958ea640 1 -- 172.21.15.88:0/1498826423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0900d0110 msgr2=0x7fe090161060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:53.901 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.895+0000 7fe0958ea640 1 --2- 172.21.15.88:0/1498826423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0900d0110 0x7fe090161060 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe0780097c0 tx=0x7fe07801c440 comp rx=0 tx=0).stop 2023-10-16T20:04:53.901 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.895+0000 7fe0958ea640 1 -- 172.21.15.88:0/1498826423 shutdown_connections 2023-10-16T20:04:53.901 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.895+0000 7fe0958ea640 1 --2- 172.21.15.88:0/1498826423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0900d0110 0x7fe090161060 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:53.901 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.895+0000 7fe0958ea640 1 -- 172.21.15.88:0/1498826423 >> 172.21.15.88:0/1498826423 conn(0x7fe0900719c0 msgr2=0x7fe090071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:53.902 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.895+0000 7fe0958ea640 1 -- 172.21.15.88:0/1498826423 shutdown_connections 2023-10-16T20:04:53.902 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.895+0000 7fe0958ea640 1 -- 172.21.15.88:0/1498826423 wait complete. 2023-10-16T20:04:53.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.899+0000 7fe0958ea640 1 Processor -- start 2023-10-16T20:04:53.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.899+0000 7fe0958ea640 1 -- start start 2023-10-16T20:04:53.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.899+0000 7fe0958ea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0900cdad0 0x7fe090152d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:53.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.899+0000 7fe0958ea640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0900d0110 0x7fe09014ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:53.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.899+0000 7fe0958ea640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe09015cb70 0x7fe0900b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:53.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.899+0000 7fe0958ea640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe090150820 con 0x7fe0900d0110 2023-10-16T20:04:53.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.899+0000 7fe0958ea640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0901510c0 con 0x7fe0900cdad0 2023-10-16T20:04:53.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.899+0000 7fe0958ea640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0900b5f50 con 0x7fe09015cb70 2023-10-16T20:04:53.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.899+0000 7fe0867fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0900d0110 0x7fe09014ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:53.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.899+0000 7fe0867fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0900d0110 0x7fe09014ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41404/0 (socket says 172.21.15.88:41404) 2023-10-16T20:04:53.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.899+0000 7fe0867fc640 1 -- 172.21.15.88:0/1619489400 learned_addr learned my addr 172.21.15.88:0/1619489400 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:04:53.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.899+0000 7fe08f7fe640 1 --2- 172.21.15.88:0/1619489400 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe09015cb70 0x7fe0900b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:53.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.899+0000 7fe08effd640 1 --2- 172.21.15.88:0/1619489400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0900cdad0 0x7fe090152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:53.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.899+0000 7fe0867fc640 1 -- 172.21.15.88:0/1619489400 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe09015cb70 msgr2=0x7fe0900b24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:53.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.899+0000 7fe0867fc640 1 --2- 172.21.15.88:0/1619489400 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe09015cb70 0x7fe0900b24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:53.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.899+0000 7fe0867fc640 1 -- 172.21.15.88:0/1619489400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0900cdad0 msgr2=0x7fe090152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:53.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.899+0000 7fe0867fc640 1 --2- 172.21.15.88:0/1619489400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0900cdad0 0x7fe090152d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:53.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.899+0000 7fe0867fc640 1 -- 172.21.15.88:0/1619489400 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe078009420 con 0x7fe0900d0110 2023-10-16T20:04:53.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.899+0000 7fe0867fc640 1 --2- 172.21.15.88:0/1619489400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0900d0110 0x7fe09014ffb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe0780097c0 tx=0x7fe0780191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:53.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.899+0000 7fe08cff9640 1 -- 172.21.15.88:0/1619489400 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe078006e20 con 0x7fe0900d0110 2023-10-16T20:04:53.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.899+0000 7fe08cff9640 1 -- 172.21.15.88:0/1619489400 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe078004030 con 0x7fe0900d0110 2023-10-16T20:04:53.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.899+0000 7fe08cff9640 1 -- 172.21.15.88:0/1619489400 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0780043a0 con 0x7fe0900d0110 2023-10-16T20:04:53.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.899+0000 7fe0958ea640 1 -- 172.21.15.88:0/1619489400 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe090150a70 con 0x7fe0900d0110 2023-10-16T20:04:53.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.899+0000 7fe0958ea640 1 -- 172.21.15.88:0/1619489400 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0900b5be0 con 0x7fe0900d0110 2023-10-16T20:04:53.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.903+0000 7fe0958ea640 1 -- 172.21.15.88:0/1619489400 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe09004e9c0 con 0x7fe0900d0110 2023-10-16T20:04:53.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.903+0000 7fe08cff9640 1 -- 172.21.15.88:0/1619489400 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe0780b2040 con 0x7fe0900d0110 2023-10-16T20:04:53.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.903+0000 7fe08cff9640 1 --2- 172.21.15.88:0/1619489400 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe0700664a0 0x7fe070068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:53.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.903+0000 7fe08cff9640 1 -- 172.21.15.88:0/1619489400 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe07811ca80 con 0x7fe0900d0110 2023-10-16T20:04:53.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.903+0000 7fe08effd640 1 --2- 172.21.15.88:0/1619489400 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe0700664a0 0x7fe070068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:53.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.903+0000 7fe08effd640 1 --2- 172.21.15.88:0/1619489400 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe0700664a0 0x7fe070068960 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe080005890 tx=0x7fe080009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:53.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:53.907+0000 7fe08cff9640 1 -- 172.21.15.88:0/1619489400 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe0780ea250 con 0x7fe0900d0110 2023-10-16T20:04:54.061 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.059+0000 7fe0958ea640 1 -- 172.21.15.88:0/1619489400 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe09015df60 con 0x7fe0900d0110 2023-10-16T20:04:54.062 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.059+0000 7fe08cff9640 1 -- 172.21.15.88:0/1619489400 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe09015df60 con 0x7fe0900d0110 2023-10-16T20:04:54.062 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:04:54.063 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2023-10-16T20:04:54.063 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:04:54.063 INFO:teuthology.orchestra.run.smithi088.stdout: "poolnum": 1, 2023-10-16T20:04:54.063 INFO:teuthology.orchestra.run.smithi088.stdout: "poolname": ".mgr" 2023-10-16T20:04:54.063 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:04:54.064 INFO:teuthology.orchestra.run.smithi088.stdout:] 2023-10-16T20:04:54.065 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.059+0000 7fe0958ea640 1 -- 172.21.15.88:0/1619489400 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe0700664a0 msgr2=0x7fe070068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:54.065 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.059+0000 7fe0958ea640 1 --2- 172.21.15.88:0/1619489400 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe0700664a0 0x7fe070068960 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe080005890 tx=0x7fe080009040 comp rx=0 tx=0).stop 2023-10-16T20:04:54.065 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.059+0000 7fe0958ea640 1 -- 172.21.15.88:0/1619489400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0900d0110 msgr2=0x7fe09014ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:54.065 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.059+0000 7fe0958ea640 1 --2- 172.21.15.88:0/1619489400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0900d0110 0x7fe09014ffb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe0780097c0 tx=0x7fe0780191e0 comp rx=0 tx=0).stop 2023-10-16T20:04:54.066 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.059+0000 7fe0958ea640 1 -- 172.21.15.88:0/1619489400 shutdown_connections 2023-10-16T20:04:54.066 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.059+0000 7fe0958ea640 1 --2- 172.21.15.88:0/1619489400 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe0700664a0 0x7fe070068960 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:54.066 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.059+0000 7fe0958ea640 1 --2- 172.21.15.88:0/1619489400 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe09015cb70 0x7fe0900b24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:54.066 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.059+0000 7fe0958ea640 1 --2- 172.21.15.88:0/1619489400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0900d0110 0x7fe09014ffb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:54.066 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.059+0000 7fe0958ea640 1 --2- 172.21.15.88:0/1619489400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0900cdad0 0x7fe090152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:54.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.059+0000 7fe0958ea640 1 -- 172.21.15.88:0/1619489400 >> 172.21.15.88:0/1619489400 conn(0x7fe0900719c0 msgr2=0x7fe0900b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:54.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.059+0000 7fe0958ea640 1 -- 172.21.15.88:0/1619489400 shutdown_connections 2023-10-16T20:04:54.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.059+0000 7fe0958ea640 1 -- 172.21.15.88:0/1619489400 wait complete. 2023-10-16T20:04:54.082 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T20:04:54.083 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T20:04:54.224 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.219+0000 7f6557b94640 1 Processor -- start 2023-10-16T20:04:54.224 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.219+0000 7f6557b94640 1 -- start start 2023-10-16T20:04:54.226 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.223+0000 7f6557b94640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f655015cb80 0x7f65500ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:54.226 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.223+0000 7f6557b94640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f6550161400 0x7f65500cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:54.226 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.223+0000 7f6557b94640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65500cb3b0 con 0x7f65500cdb60 2023-10-16T20:04:54.226 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.223+0000 7f6557b94640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65500cdf90 con 0x7f6550161400 2023-10-16T20:04:54.226 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.223+0000 7f6557b94640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65500cecf0 con 0x7f655015cb80 2023-10-16T20:04:54.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.223+0000 7f655512f640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f6550161400 0x7f65500cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:54.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.223+0000 7f655512f640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f6550161400 0x7f65500cd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41420/0 (socket says 172.21.15.88:41420) 2023-10-16T20:04:54.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.223+0000 7f655512f640 1 -- 172.21.15.88:0/1672304476 learned_addr learned my addr 172.21.15.88:0/1672304476 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:04:54.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.223+0000 7f6556131640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f65500cdb60 0x7f655006e4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:44796/0 (socket says 172.21.15.88:44796) 2023-10-16T20:04:54.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.223+0000 7f6555930640 1 --2- 172.21.15.88:0/1672304476 >> v2:172.21.15.94:3300/0 conn(0x7f655015cb80 0x7f65500ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:54.228 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.223+0000 7f655512f640 1 -- 172.21.15.88:0/1672304476 >> v2:172.21.15.94:3300/0 conn(0x7f655015cb80 msgr2=0x7f65500ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:04:54.228 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.223+0000 7f655512f640 1 --2- 172.21.15.88:0/1672304476 >> v2:172.21.15.94:3300/0 conn(0x7f655015cb80 0x7f65500ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:54.229 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.223+0000 7f655512f640 1 -- 172.21.15.88:0/1672304476 >> v1:172.21.15.88:6789/0 conn(0x7f65500cdb60 legacy=0x7f655006e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:54.229 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.223+0000 7f655512f640 1 -- 172.21.15.88:0/1672304476 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65500cefd0 con 0x7f6550161400 2023-10-16T20:04:54.229 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.223+0000 7f655492e640 1 -- 172.21.15.88:0/1672304476 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1620204571 0 0) 0x7f65500cb3b0 con 0x7f65500cdb60 2023-10-16T20:04:54.230 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.227+0000 7f655512f640 1 --2- 172.21.15.88:0/1672304476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6550161400 0x7f65500cd620 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f65500b2760 tx=0x7f654001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca49b22a2fb4458 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:54.230 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.227+0000 7f655492e640 1 -- 172.21.15.88:0/1672304476 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65400a89f0 con 0x7f6550161400 2023-10-16T20:04:54.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.227+0000 7f655492e640 1 -- 172.21.15.88:0/1672304476 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65400a8b50 con 0x7f6550161400 2023-10-16T20:04:54.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.227+0000 7f655492e640 1 -- 172.21.15.88:0/1672304476 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65400b0440 con 0x7f6550161400 2023-10-16T20:04:54.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.227+0000 7f6557b94640 1 -- 172.21.15.88:0/1672304476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6550161400 msgr2=0x7f65500cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:54.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.227+0000 7f6557b94640 1 --2- 172.21.15.88:0/1672304476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6550161400 0x7f65500cd620 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f65500b2760 tx=0x7f654001ecf0 comp rx=0 tx=0).stop 2023-10-16T20:04:54.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.227+0000 7f6557b94640 1 -- 172.21.15.88:0/1672304476 shutdown_connections 2023-10-16T20:04:54.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.227+0000 7f6557b94640 1 --2- 172.21.15.88:0/1672304476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6550161400 0x7f65500cd620 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:54.232 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.227+0000 7f6557b94640 1 --2- 172.21.15.88:0/1672304476 >> v2:172.21.15.94:3300/0 conn(0x7f655015cb80 0x7f65500ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:54.232 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.227+0000 7f6557b94640 1 -- 172.21.15.88:0/1672304476 >> 172.21.15.88:0/1672304476 conn(0x7f655006a060 msgr2=0x7f655006a440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:54.232 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.227+0000 7f6557b94640 1 -- 172.21.15.88:0/1672304476 shutdown_connections 2023-10-16T20:04:54.232 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.227+0000 7f6557b94640 1 -- 172.21.15.88:0/1672304476 wait complete. 2023-10-16T20:04:54.234 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.227+0000 7f6557b94640 1 Processor -- start 2023-10-16T20:04:54.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.227+0000 7f6557b94640 1 -- start start 2023-10-16T20:04:54.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.231+0000 7f6557b94640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65500cdb60 0x7f65500b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:54.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.231+0000 7f6557b94640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f655015cb80 0x7f65500b5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:54.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.231+0000 7f6557b94640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6550161400 0x7f6550156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:54.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.231+0000 7f6557b94640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65500b4410 con 0x7f6550161400 2023-10-16T20:04:54.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.231+0000 7f6557b94640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65500b5da0 con 0x7f65500cdb60 2023-10-16T20:04:54.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.231+0000 7f6557b94640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65500b4600 con 0x7f655015cb80 2023-10-16T20:04:54.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.231+0000 7f6556131640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6550161400 0x7f6550156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:54.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.231+0000 7f6556131640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6550161400 0x7f6550156dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41422/0 (socket says 172.21.15.88:41422) 2023-10-16T20:04:54.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.231+0000 7f6556131640 1 -- 172.21.15.88:0/181500314 learned_addr learned my addr 172.21.15.88:0/181500314 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:04:54.237 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.231+0000 7f6555930640 1 --2- 172.21.15.88:0/181500314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65500cdb60 0x7f65500b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:54.237 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.231+0000 7f655512f640 1 --2- 172.21.15.88:0/181500314 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f655015cb80 0x7f65500b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:54.237 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.231+0000 7f6556131640 1 -- 172.21.15.88:0/181500314 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f655015cb80 msgr2=0x7f65500b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:54.237 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.231+0000 7f6556131640 1 --2- 172.21.15.88:0/181500314 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f655015cb80 0x7f65500b5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:54.237 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.231+0000 7f6556131640 1 -- 172.21.15.88:0/181500314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65500cdb60 msgr2=0x7f65500b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:54.238 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.231+0000 7f6556131640 1 --2- 172.21.15.88:0/181500314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65500cdb60 0x7f65500b3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:54.238 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.231+0000 7f6556131640 1 -- 172.21.15.88:0/181500314 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65400094d0 con 0x7f6550161400 2023-10-16T20:04:54.238 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.231+0000 7f6556131640 1 --2- 172.21.15.88:0/181500314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6550161400 0x7f6550156dc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6548001570 tx=0x7f654800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:54.238 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.231+0000 7f65467fc640 1 -- 172.21.15.88:0/181500314 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654800cea0 con 0x7f6550161400 2023-10-16T20:04:54.238 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.231+0000 7f65467fc640 1 -- 172.21.15.88:0/181500314 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654800a5e0 con 0x7f6550161400 2023-10-16T20:04:54.239 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.231+0000 7f6557b94640 1 -- 172.21.15.88:0/181500314 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65500b48e0 con 0x7f6550161400 2023-10-16T20:04:54.239 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.231+0000 7f65467fc640 1 -- 172.21.15.88:0/181500314 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654800a990 con 0x7f6550161400 2023-10-16T20:04:54.239 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.231+0000 7f6557b94640 1 -- 172.21.15.88:0/181500314 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65500b4ea0 con 0x7f6550161400 2023-10-16T20:04:54.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.231+0000 7f6557b94640 1 -- 172.21.15.88:0/181500314 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f655004e9c0 con 0x7f6550161400 2023-10-16T20:04:54.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.231+0000 7f65467fc640 1 -- 172.21.15.88:0/181500314 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f654800aaf0 con 0x7f6550161400 2023-10-16T20:04:54.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.235+0000 7f65467fc640 1 --2- 172.21.15.88:0/181500314 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f65340664a0 0x7f6534068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:54.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.235+0000 7f65467fc640 1 -- 172.21.15.88:0/181500314 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6548083290 con 0x7f6550161400 2023-10-16T20:04:54.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.235+0000 7f6555930640 1 --2- 172.21.15.88:0/181500314 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f65340664a0 0x7f6534068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:54.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.235+0000 7f6555930640 1 --2- 172.21.15.88:0/181500314 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f65340664a0 0x7f6534068960 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f65500c4080 tx=0x7f654c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:54.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.239+0000 7f65467fc640 1 -- 172.21.15.88:0/181500314 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6548050a50 con 0x7f6550161400 2023-10-16T20:04:54.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:54.391+0000 7f6557b94640 1 -- 172.21.15.88:0/181500314 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6550070960 con 0x7f6550161400 2023-10-16T20:04:55.373 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.367+0000 7f65467fc640 1 -- 172.21.15.88:0/181500314 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6548054070 con 0x7f6550161400 2023-10-16T20:04:55.373 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' created 2023-10-16T20:04:55.376 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.371+0000 7f6557b94640 1 -- 172.21.15.88:0/181500314 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f65340664a0 msgr2=0x7f6534068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:55.376 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.371+0000 7f6557b94640 1 --2- 172.21.15.88:0/181500314 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f65340664a0 0x7f6534068960 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f65500c4080 tx=0x7f654c008040 comp rx=0 tx=0).stop 2023-10-16T20:04:55.376 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.371+0000 7f6557b94640 1 -- 172.21.15.88:0/181500314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6550161400 msgr2=0x7f6550156dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:55.377 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.371+0000 7f6557b94640 1 --2- 172.21.15.88:0/181500314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6550161400 0x7f6550156dc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6548001570 tx=0x7f654800cc50 comp rx=0 tx=0).stop 2023-10-16T20:04:55.377 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.371+0000 7f6557b94640 1 -- 172.21.15.88:0/181500314 shutdown_connections 2023-10-16T20:04:55.377 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.371+0000 7f6557b94640 1 --2- 172.21.15.88:0/181500314 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f65340664a0 0x7f6534068960 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:55.377 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.371+0000 7f6557b94640 1 --2- 172.21.15.88:0/181500314 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f655015cb80 0x7f65500b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:55.377 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.371+0000 7f6557b94640 1 --2- 172.21.15.88:0/181500314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6550161400 0x7f6550156dc0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:55.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.371+0000 7f6557b94640 1 --2- 172.21.15.88:0/181500314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65500cdb60 0x7f65500b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:55.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.371+0000 7f6557b94640 1 -- 172.21.15.88:0/181500314 >> 172.21.15.88:0/181500314 conn(0x7f655006a060 msgr2=0x7f655006c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:55.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.371+0000 7f6557b94640 1 -- 172.21.15.88:0/181500314 shutdown_connections 2023-10-16T20:04:55.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.371+0000 7f6557b94640 1 -- 172.21.15.88:0/181500314 wait complete. 2023-10-16T20:04:55.407 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-10-16T20:04:55.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.555+0000 7f1896daa640 1 Processor -- start 2023-10-16T20:04:55.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.555+0000 7f1896daa640 1 -- start start 2023-10-16T20:04:55.556 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.555+0000 7f1896daa640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1890070960 0x7f18901677f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:55.556 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.555+0000 7f1896daa640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f189006dab0 con 0x7f18901457f0 2023-10-16T20:04:55.556 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.555+0000 7f1896daa640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f189006c2e0 con 0x7f18900aeb90 2023-10-16T20:04:55.557 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.555+0000 7f1896daa640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1890145bd0 con 0x7f1890070960 2023-10-16T20:04:55.557 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.555+0000 7f1895347640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1890070960 0x7f18901677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:55.557 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.555+0000 7f1895347640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1890070960 0x7f18901677f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41436/0 (socket says 172.21.15.88:41436) 2023-10-16T20:04:55.557 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.555+0000 7f1895347640 1 -- 172.21.15.88:0/1690961273 learned_addr learned my addr 172.21.15.88:0/1690961273 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:04:55.558 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.555+0000 7f1895347640 1 -- 172.21.15.88:0/1690961273 >> v1:172.21.15.94:6789/0 conn(0x7f18900aeb90 legacy=0x7f1890070250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:55.558 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.555+0000 7f1895347640 1 -- 172.21.15.88:0/1690961273 >> v1:172.21.15.88:6789/0 conn(0x7f18901457f0 legacy=0x7f1890160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:55.558 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.555+0000 7f18877fe640 1 -- 172.21.15.88:0/1690961273 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 896083080 0 0) 0x7f189006dab0 con 0x7f18901457f0 2023-10-16T20:04:55.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.555+0000 7f1895347640 1 -- 172.21.15.88:0/1690961273 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1890160ec0 con 0x7f1890070960 2023-10-16T20:04:55.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.559+0000 7f1895347640 1 --2- 172.21.15.88:0/1690961273 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1890070960 0x7f18901677f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1888009e50 tx=0x7f188801c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467d3eb8774dd9bc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:55.560 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.559+0000 7f18877fe640 1 -- 172.21.15.88:0/1690961273 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18880a58a0 con 0x7f1890070960 2023-10-16T20:04:55.560 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.559+0000 7f18877fe640 1 -- 172.21.15.88:0/1690961273 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18880a5a00 con 0x7f1890070960 2023-10-16T20:04:55.560 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.559+0000 7f18877fe640 1 -- 172.21.15.88:0/1690961273 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18880a5d90 con 0x7f1890070960 2023-10-16T20:04:55.561 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.559+0000 7f1896daa640 1 -- 172.21.15.88:0/1690961273 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1890070960 msgr2=0x7f18901677f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:55.561 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.559+0000 7f1896daa640 1 --2- 172.21.15.88:0/1690961273 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1890070960 0x7f18901677f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1888009e50 tx=0x7f188801c9e0 comp rx=0 tx=0).stop 2023-10-16T20:04:55.561 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.559+0000 7f1896daa640 1 -- 172.21.15.88:0/1690961273 shutdown_connections 2023-10-16T20:04:55.562 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.559+0000 7f1896daa640 1 --2- 172.21.15.88:0/1690961273 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1890070960 0x7f18901677f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:55.562 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.559+0000 7f1896daa640 1 -- 172.21.15.88:0/1690961273 >> 172.21.15.88:0/1690961273 conn(0x7f18900cb080 msgr2=0x7f18900cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:55.562 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.559+0000 7f1896daa640 1 -- 172.21.15.88:0/1690961273 shutdown_connections 2023-10-16T20:04:55.562 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.559+0000 7f1896daa640 1 -- 172.21.15.88:0/1690961273 wait complete. 2023-10-16T20:04:55.564 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.563+0000 7f1896daa640 1 Processor -- start 2023-10-16T20:04:55.564 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.563+0000 7f1896daa640 1 -- start start 2023-10-16T20:04:55.565 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.563+0000 7f1896daa640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1890070960 0x7f189013e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:55.565 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.563+0000 7f1896daa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18900aeb90 0x7f189015b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:55.565 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.563+0000 7f1896daa640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f18901457f0 0x7f18900aba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:55.565 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.563+0000 7f1896daa640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1890142ef0 con 0x7f1890070960 2023-10-16T20:04:55.565 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.563+0000 7f1896daa640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189015b960 con 0x7f18900aeb90 2023-10-16T20:04:55.566 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.563+0000 7f1896daa640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f189015a280 con 0x7f18901457f0 2023-10-16T20:04:55.566 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.563+0000 7f1894b46640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1890070960 0x7f189013e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:55.566 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.563+0000 7f1894b46640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1890070960 0x7f189013e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41446/0 (socket says 172.21.15.88:41446) 2023-10-16T20:04:55.566 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.563+0000 7f1894b46640 1 -- 172.21.15.88:0/4280531692 learned_addr learned my addr 172.21.15.88:0/4280531692 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:04:55.567 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.563+0000 7f1894b46640 1 -- 172.21.15.88:0/4280531692 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f18901457f0 msgr2=0x7f18900aba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:55.567 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.563+0000 7f1894b46640 1 --2- 172.21.15.88:0/4280531692 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f18901457f0 0x7f18900aba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:55.567 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.563+0000 7f1894b46640 1 -- 172.21.15.88:0/4280531692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18900aeb90 msgr2=0x7f189015b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:55.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.563+0000 7f1894b46640 1 --2- 172.21.15.88:0/4280531692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18900aeb90 0x7f189015b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:55.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.563+0000 7f1894b46640 1 -- 172.21.15.88:0/4280531692 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1888009b30 con 0x7f1890070960 2023-10-16T20:04:55.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.563+0000 7f1894b46640 1 --2- 172.21.15.88:0/4280531692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1890070960 0x7f189013e7f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f188c00c410 tx=0x7f188c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:55.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.563+0000 7f18857fa640 1 -- 172.21.15.88:0/4280531692 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c0054e0 con 0x7f1890070960 2023-10-16T20:04:55.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.563+0000 7f18857fa640 1 -- 172.21.15.88:0/4280531692 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188c003600 con 0x7f1890070960 2023-10-16T20:04:55.569 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.563+0000 7f1896daa640 1 -- 172.21.15.88:0/4280531692 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f189015a560 con 0x7f1890070960 2023-10-16T20:04:55.569 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.563+0000 7f18857fa640 1 -- 172.21.15.88:0/4280531692 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c0039b0 con 0x7f1890070960 2023-10-16T20:04:55.569 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.563+0000 7f1896daa640 1 -- 172.21.15.88:0/4280531692 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f189015ab20 con 0x7f1890070960 2023-10-16T20:04:55.570 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.567+0000 7f1896daa640 1 -- 172.21.15.88:0/4280531692 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f189004e9c0 con 0x7f1890070960 2023-10-16T20:04:55.570 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.567+0000 7f18857fa640 1 -- 172.21.15.88:0/4280531692 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f188c003b10 con 0x7f1890070960 2023-10-16T20:04:55.571 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.567+0000 7f18857fa640 1 --2- 172.21.15.88:0/4280531692 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f187c0664a0 0x7f187c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:55.571 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.567+0000 7f18857fa640 1 -- 172.21.15.88:0/4280531692 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f188c083210 con 0x7f1890070960 2023-10-16T20:04:55.571 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.567+0000 7f1887fff640 1 --2- 172.21.15.88:0/4280531692 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f187c0664a0 0x7f187c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:55.571 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.567+0000 7f1887fff640 1 --2- 172.21.15.88:0/4280531692 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f187c0664a0 0x7f187c068960 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1880000f70 tx=0x7f18800079c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:55.572 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.571+0000 7f18857fa640 1 -- 172.21.15.88:0/4280531692 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f188c050900 con 0x7f1890070960 2023-10-16T20:04:55.722 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:55.723+0000 7f1896daa640 1 -- 172.21.15.88:0/4280531692 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f1890145fd0 con 0x7f1890070960 2023-10-16T20:04:56.144 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T20:04:56.150 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T20:04:56.376 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.375+0000 7f18857fa640 1 -- 172.21.15.88:0/4280531692 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v21) v1 ==== 171+0+0 (secure 0 0 0) 0x7f188c053f20 con 0x7f1890070960 2023-10-16T20:04:56.376 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' created 2023-10-16T20:04:56.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.379+0000 7f1896daa640 1 -- 172.21.15.88:0/4280531692 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f187c0664a0 msgr2=0x7f187c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:56.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.379+0000 7f1896daa640 1 --2- 172.21.15.88:0/4280531692 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f187c0664a0 0x7f187c068960 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1880000f70 tx=0x7f18800079c0 comp rx=0 tx=0).stop 2023-10-16T20:04:56.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.379+0000 7f1896daa640 1 -- 172.21.15.88:0/4280531692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1890070960 msgr2=0x7f189013e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:56.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.379+0000 7f1896daa640 1 --2- 172.21.15.88:0/4280531692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1890070960 0x7f189013e7f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f188c00c410 tx=0x7f188c00c8d0 comp rx=0 tx=0).stop 2023-10-16T20:04:56.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.379+0000 7f1896daa640 1 -- 172.21.15.88:0/4280531692 shutdown_connections 2023-10-16T20:04:56.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.379+0000 7f1896daa640 1 --2- 172.21.15.88:0/4280531692 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f187c0664a0 0x7f187c068960 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:56.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.379+0000 7f1896daa640 1 --2- 172.21.15.88:0/4280531692 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f18901457f0 0x7f18900aba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:56.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.379+0000 7f1896daa640 1 --2- 172.21.15.88:0/4280531692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18900aeb90 0x7f189015b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:56.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.379+0000 7f1896daa640 1 --2- 172.21.15.88:0/4280531692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1890070960 0x7f189013e7f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:56.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.379+0000 7f1896daa640 1 -- 172.21.15.88:0/4280531692 >> 172.21.15.88:0/4280531692 conn(0x7f18900cb080 msgr2=0x7f18900ccd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:56.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.379+0000 7f1896daa640 1 -- 172.21.15.88:0/4280531692 shutdown_connections 2023-10-16T20:04:56.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.379+0000 7f1896daa640 1 -- 172.21.15.88:0/4280531692 wait complete. 2023-10-16T20:04:56.418 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T20:04:56.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.559+0000 7fca4a6dc640 1 Processor -- start 2023-10-16T20:04:56.560 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.559+0000 7fca4a6dc640 1 -- start start 2023-10-16T20:04:56.561 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.559+0000 7fca4a6dc640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fca4415c850 0x7fca4415cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:56.561 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.559+0000 7fca4a6dc640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fca440c9810 0x7fca440cae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:56.561 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.559+0000 7fca4a6dc640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca440b0420 con 0x7fca440aff90 2023-10-16T20:04:56.561 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.559+0000 7fca4a6dc640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca44160cf0 con 0x7fca4415c850 2023-10-16T20:04:56.562 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.559+0000 7fca4a6dc640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca440b0700 con 0x7fca440c9810 2023-10-16T20:04:56.562 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.563+0000 7fca496da640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fca4415c850 0x7fca4415cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:56.562 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.563+0000 7fca496da640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fca4415c850 0x7fca4415cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41450/0 (socket says 172.21.15.88:41450) 2023-10-16T20:04:56.562 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.563+0000 7fca496da640 1 -- 172.21.15.88:0/1091193533 learned_addr learned my addr 172.21.15.88:0/1091193533 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:04:56.563 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.563+0000 7fca49edb640 1 --2- 172.21.15.88:0/1091193533 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fca440c9810 0x7fca440cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:56.563 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.563+0000 7fca496da640 1 -- 172.21.15.88:0/1091193533 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fca440c9810 msgr2=0x7fca440cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:04:56.563 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.563+0000 7fca496da640 1 --2- 172.21.15.88:0/1091193533 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fca440c9810 0x7fca440cae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:56.564 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.563+0000 7fca496da640 1 -- 172.21.15.88:0/1091193533 >> v1:172.21.15.94:6789/0 conn(0x7fca440aff90 legacy=0x7fca441605e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:04:56.564 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.563+0000 7fca496da640 1 -- 172.21.15.88:0/1091193533 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca440b09e0 con 0x7fca4415c850 2023-10-16T20:04:56.564 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.563+0000 7fca49edb640 1 --2- 172.21.15.88:0/1091193533 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fca440c9810 0x7fca440cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:04:56.565 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.563+0000 7fca496da640 1 --2- 172.21.15.88:0/1091193533 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca4415c850 0x7fca4415cc30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fca40009870 tx=0x7fca4001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84fe9371b70c0745 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:56.565 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.563+0000 7fca3bfff640 1 -- 172.21.15.88:0/1091193533 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca400ab620 con 0x7fca4415c850 2023-10-16T20:04:56.566 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.563+0000 7fca3bfff640 1 -- 172.21.15.88:0/1091193533 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca400ab780 con 0x7fca4415c850 2023-10-16T20:04:56.566 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.563+0000 7fca3bfff640 1 -- 172.21.15.88:0/1091193533 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca400abb10 con 0x7fca4415c850 2023-10-16T20:04:56.566 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.567+0000 7fca4a6dc640 1 -- 172.21.15.88:0/1091193533 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca4415c850 msgr2=0x7fca4415cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:56.566 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.567+0000 7fca4a6dc640 1 --2- 172.21.15.88:0/1091193533 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca4415c850 0x7fca4415cc30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fca40009870 tx=0x7fca4001c440 comp rx=0 tx=0).stop 2023-10-16T20:04:56.566 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.567+0000 7fca4a6dc640 1 -- 172.21.15.88:0/1091193533 shutdown_connections 2023-10-16T20:04:56.567 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.567+0000 7fca4a6dc640 1 --2- 172.21.15.88:0/1091193533 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fca440c9810 0x7fca440cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:56.567 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.567+0000 7fca4a6dc640 1 --2- 172.21.15.88:0/1091193533 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca4415c850 0x7fca4415cc30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:56.567 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.567+0000 7fca4a6dc640 1 -- 172.21.15.88:0/1091193533 >> 172.21.15.88:0/1091193533 conn(0x7fca4406a4c0 msgr2=0x7fca4406c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:56.567 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.567+0000 7fca4a6dc640 1 -- 172.21.15.88:0/1091193533 shutdown_connections 2023-10-16T20:04:56.567 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.567+0000 7fca4a6dc640 1 -- 172.21.15.88:0/1091193533 wait complete. 2023-10-16T20:04:56.569 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.567+0000 7fca4a6dc640 1 Processor -- start 2023-10-16T20:04:56.569 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.567+0000 7fca4a6dc640 1 -- start start 2023-10-16T20:04:56.569 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.567+0000 7fca4a6dc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca440c9810 0x7fca440b3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:56.570 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.567+0000 7fca4a6dc640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fca4415c850 0x7fca44153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:56.570 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.567+0000 7fca4a6dc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca441f9d10 0x7fca44153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:56.570 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.567+0000 7fca4a6dc640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca440c2c80 con 0x7fca440c9810 2023-10-16T20:04:56.570 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.567+0000 7fca4a6dc640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca440c31d0 con 0x7fca441f9d10 2023-10-16T20:04:56.570 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.567+0000 7fca4a6dc640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca440c34b0 con 0x7fca4415c850 2023-10-16T20:04:56.571 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.567+0000 7fca496da640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca440c9810 0x7fca440b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:56.571 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.567+0000 7fca48ed9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fca4415c850 0x7fca44153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:56.571 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.567+0000 7fca496da640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca440c9810 0x7fca440b3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41466/0 (socket says 172.21.15.88:41466) 2023-10-16T20:04:56.571 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.567+0000 7fca496da640 1 -- 172.21.15.88:0/3319944641 learned_addr learned my addr 172.21.15.88:0/3319944641 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:04:56.571 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.567+0000 7fca49edb640 1 --2- 172.21.15.88:0/3319944641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca441f9d10 0x7fca44153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:56.572 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.571+0000 7fca496da640 1 -- 172.21.15.88:0/3319944641 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fca4415c850 msgr2=0x7fca44153ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:56.572 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.571+0000 7fca496da640 1 --2- 172.21.15.88:0/3319944641 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fca4415c850 0x7fca44153ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:56.572 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.571+0000 7fca496da640 1 -- 172.21.15.88:0/3319944641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca441f9d10 msgr2=0x7fca44153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:56.572 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.571+0000 7fca496da640 1 --2- 172.21.15.88:0/3319944641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca441f9d10 0x7fca44153ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:56.572 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.571+0000 7fca496da640 1 -- 172.21.15.88:0/3319944641 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca400094d0 con 0x7fca440c9810 2023-10-16T20:04:56.572 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.571+0000 7fca496da640 1 --2- 172.21.15.88:0/3319944641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca440c9810 0x7fca440b3690 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fca400ad040 tx=0x7fca4001ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:56.574 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.571+0000 7fca39ffb640 1 -- 172.21.15.88:0/3319944641 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca400be5f0 con 0x7fca440c9810 2023-10-16T20:04:56.574 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.571+0000 7fca39ffb640 1 -- 172.21.15.88:0/3319944641 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca400be770 con 0x7fca440c9810 2023-10-16T20:04:56.574 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.571+0000 7fca39ffb640 1 -- 172.21.15.88:0/3319944641 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca400beb20 con 0x7fca440c9810 2023-10-16T20:04:56.574 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.571+0000 7fca4a6dc640 1 -- 172.21.15.88:0/3319944641 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca440c3790 con 0x7fca440c9810 2023-10-16T20:04:56.575 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.571+0000 7fca4a6dc640 1 -- 172.21.15.88:0/3319944641 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca440b1dd0 con 0x7fca440c9810 2023-10-16T20:04:56.575 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.571+0000 7fca4a6dc640 1 -- 172.21.15.88:0/3319944641 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca440674f0 con 0x7fca440c9810 2023-10-16T20:04:56.575 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.571+0000 7fca39ffb640 1 -- 172.21.15.88:0/3319944641 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fca400b43f0 con 0x7fca440c9810 2023-10-16T20:04:56.575 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.575+0000 7fca39ffb640 1 --2- 172.21.15.88:0/3319944641 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fca280664a0 0x7fca28068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:56.575 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.575+0000 7fca39ffb640 1 -- 172.21.15.88:0/3319944641 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fca4011eb30 con 0x7fca440c9810 2023-10-16T20:04:56.576 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.575+0000 7fca48ed9640 1 --2- 172.21.15.88:0/3319944641 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fca280664a0 0x7fca28068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:56.576 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.575+0000 7fca48ed9640 1 --2- 172.21.15.88:0/3319944641 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fca280664a0 0x7fca28068960 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fca440c2860 tx=0x7fca34007e40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:56.577 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.579+0000 7fca39ffb640 1 -- 172.21.15.88:0/3319944641 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fca400ec020 con 0x7fca440c9810 2023-10-16T20:04:56.756 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:56.755+0000 7fca4a6dc640 1 -- 172.21.15.88:0/3319944641 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fca440c35f0 con 0x7fca440c9810 2023-10-16T20:04:57.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.407+0000 7fca39ffb640 1 -- 172.21.15.88:0/3319944641 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fca440c35f0 con 0x7fca440c9810 2023-10-16T20:04:57.409 INFO:teuthology.orchestra.run.smithi088.stderr:new fs with metadata pool 2 and data pool 3 2023-10-16T20:04:57.417 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.419+0000 7fca4a6dc640 1 -- 172.21.15.88:0/3319944641 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fca280664a0 msgr2=0x7fca28068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:57.418 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.419+0000 7fca4a6dc640 1 --2- 172.21.15.88:0/3319944641 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fca280664a0 0x7fca28068960 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fca440c2860 tx=0x7fca34007e40 comp rx=0 tx=0).stop 2023-10-16T20:04:57.418 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.419+0000 7fca4a6dc640 1 -- 172.21.15.88:0/3319944641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca440c9810 msgr2=0x7fca440b3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:57.418 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.419+0000 7fca4a6dc640 1 --2- 172.21.15.88:0/3319944641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca440c9810 0x7fca440b3690 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fca400ad040 tx=0x7fca4001ac80 comp rx=0 tx=0).stop 2023-10-16T20:04:57.418 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.419+0000 7fca4a6dc640 1 -- 172.21.15.88:0/3319944641 shutdown_connections 2023-10-16T20:04:57.419 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.419+0000 7fca4a6dc640 1 --2- 172.21.15.88:0/3319944641 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fca280664a0 0x7fca28068960 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:57.419 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.419+0000 7fca4a6dc640 1 --2- 172.21.15.88:0/3319944641 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fca4415c850 0x7fca44153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:57.419 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.419+0000 7fca4a6dc640 1 --2- 172.21.15.88:0/3319944641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca441f9d10 0x7fca44153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:57.419 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.419+0000 7fca4a6dc640 1 --2- 172.21.15.88:0/3319944641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca440c9810 0x7fca440b3690 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:57.419 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.419+0000 7fca4a6dc640 1 -- 172.21.15.88:0/3319944641 >> 172.21.15.88:0/3319944641 conn(0x7fca4406a4c0 msgr2=0x7fca44071730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:57.420 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.419+0000 7fca4a6dc640 1 -- 172.21.15.88:0/3319944641 shutdown_connections 2023-10-16T20:04:57.420 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.419+0000 7fca4a6dc640 1 -- 172.21.15.88:0/3319944641 wait complete. 2023-10-16T20:04:57.456 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-10-16T20:04:57.457 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-10-16T20:04:57.601 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.603+0000 7fddf4e54640 1 Processor -- start 2023-10-16T20:04:57.602 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.603+0000 7fddf4e54640 1 -- start start 2023-10-16T20:04:57.602 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.603+0000 7fddf4e54640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fddf01570e0 0x7fddf01611a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:57.603 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.603+0000 7fddf4e54640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fddf00cdef0 0x7fddf015d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:57.603 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.603+0000 7fddf4e54640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddf01616e0 con 0x7fddf015d140 2023-10-16T20:04:57.603 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.603+0000 7fddf4e54640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddf0160790 con 0x7fddf01570e0 2023-10-16T20:04:57.603 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.603+0000 7fddf4e54640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf015dcb0 con 0x7fddf00cdef0 2023-10-16T20:04:57.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.603+0000 7fddee59c640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fddf01570e0 0x7fddf01611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:57.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.603+0000 7fddee59c640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fddf01570e0 0x7fddf01611a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41476/0 (socket says 172.21.15.88:41476) 2023-10-16T20:04:57.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.603+0000 7fddee59c640 1 -- 172.21.15.88:0/2429204295 learned_addr learned my addr 172.21.15.88:0/2429204295 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:04:57.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.603+0000 7fddeed9d640 1 --2- 172.21.15.88:0/2429204295 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fddf00cdef0 0x7fddf015d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:57.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.603+0000 7fddee59c640 1 -- 172.21.15.88:0/2429204295 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fddf00cdef0 msgr2=0x7fddf015d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:04:57.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.603+0000 7fddee59c640 1 --2- 172.21.15.88:0/2429204295 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fddf00cdef0 0x7fddf015d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:57.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.603+0000 7fddee59c640 1 -- 172.21.15.88:0/2429204295 >> v1:172.21.15.88:6789/0 conn(0x7fddf015d140 legacy=0x7fddf00cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:57.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.603+0000 7fddee59c640 1 -- 172.21.15.88:0/2429204295 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf015df90 con 0x7fddf01570e0 2023-10-16T20:04:57.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.607+0000 7fddee59c640 1 --2- 172.21.15.88:0/2429204295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddf01570e0 0x7fddf01611a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdde0009870 tx=0x7fdde001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d4b385275a5215 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:57.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.607+0000 7fdded59a640 1 -- 172.21.15.88:0/2429204295 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde00a7070 con 0x7fddf01570e0 2023-10-16T20:04:57.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.607+0000 7fdded59a640 1 -- 172.21.15.88:0/2429204295 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde00ab430 con 0x7fddf01570e0 2023-10-16T20:04:57.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.607+0000 7fdded59a640 1 -- 172.21.15.88:0/2429204295 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde00ab7c0 con 0x7fddf01570e0 2023-10-16T20:04:57.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.607+0000 7fddf4e54640 1 -- 172.21.15.88:0/2429204295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddf01570e0 msgr2=0x7fddf01611a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:57.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.607+0000 7fddf4e54640 1 --2- 172.21.15.88:0/2429204295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddf01570e0 0x7fddf01611a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdde0009870 tx=0x7fdde001c540 comp rx=0 tx=0).stop 2023-10-16T20:04:57.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.607+0000 7fddf4e54640 1 -- 172.21.15.88:0/2429204295 shutdown_connections 2023-10-16T20:04:57.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.607+0000 7fddf4e54640 1 --2- 172.21.15.88:0/2429204295 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fddf00cdef0 0x7fddf015d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:57.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.607+0000 7fddf4e54640 1 --2- 172.21.15.88:0/2429204295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddf01570e0 0x7fddf01611a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:57.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.607+0000 7fddf4e54640 1 -- 172.21.15.88:0/2429204295 >> 172.21.15.88:0/2429204295 conn(0x7fddf00715c0 msgr2=0x7fddf00719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:57.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.607+0000 7fddf4e54640 1 -- 172.21.15.88:0/2429204295 shutdown_connections 2023-10-16T20:04:57.610 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.607+0000 7fddf4e54640 1 -- 172.21.15.88:0/2429204295 wait complete. 2023-10-16T20:04:57.612 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.611+0000 7fddf4e54640 1 Processor -- start 2023-10-16T20:04:57.612 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.611+0000 7fddf4e54640 1 -- start start 2023-10-16T20:04:57.612 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.611+0000 7fddf4e54640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddf00cdef0 0x7fddf00b7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:57.613 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.611+0000 7fddf4e54640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fddf01570e0 0x7fddf014e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:57.613 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.611+0000 7fddf4e54640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddf015d140 0x7fddf014edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:57.613 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.611+0000 7fddf4e54640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf01447f0 con 0x7fddf00cdef0 2023-10-16T20:04:57.613 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.611+0000 7fddf4e54640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf01545b0 con 0x7fddf015d140 2023-10-16T20:04:57.613 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.611+0000 7fddf4e54640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf00b8010 con 0x7fddf01570e0 2023-10-16T20:04:57.614 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.611+0000 7fddee59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddf00cdef0 0x7fddf00b7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:57.614 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.611+0000 7fddee59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddf00cdef0 0x7fddf00b7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41490/0 (socket says 172.21.15.88:41490) 2023-10-16T20:04:57.614 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.611+0000 7fddee59c640 1 -- 172.21.15.88:0/1576720245 learned_addr learned my addr 172.21.15.88:0/1576720245 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:04:57.614 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.611+0000 7fddedd9b640 1 --2- 172.21.15.88:0/1576720245 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fddf01570e0 0x7fddf014e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:57.614 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.611+0000 7fddeed9d640 1 --2- 172.21.15.88:0/1576720245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddf015d140 0x7fddf014edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:57.615 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.611+0000 7fddee59c640 1 -- 172.21.15.88:0/1576720245 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fddf01570e0 msgr2=0x7fddf014e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:57.615 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.611+0000 7fddee59c640 1 --2- 172.21.15.88:0/1576720245 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fddf01570e0 0x7fddf014e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:57.615 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.611+0000 7fddee59c640 1 -- 172.21.15.88:0/1576720245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddf015d140 msgr2=0x7fddf014edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:57.615 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.611+0000 7fddee59c640 1 --2- 172.21.15.88:0/1576720245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddf015d140 0x7fddf014edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:57.615 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.611+0000 7fddee59c640 1 -- 172.21.15.88:0/1576720245 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde00094d0 con 0x7fddf00cdef0 2023-10-16T20:04:57.616 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.611+0000 7fddee59c640 1 --2- 172.21.15.88:0/1576720245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddf00cdef0 0x7fddf00b7a30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdde00ad040 tx=0x7fdde00192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:57.616 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.611+0000 7fdddeffd640 1 -- 172.21.15.88:0/1576720245 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde00bc070 con 0x7fddf00cdef0 2023-10-16T20:04:57.616 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.611+0000 7fdddeffd640 1 -- 172.21.15.88:0/1576720245 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde00abe10 con 0x7fddf00cdef0 2023-10-16T20:04:57.616 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.611+0000 7fddf4e54640 1 -- 172.21.15.88:0/1576720245 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf00b82f0 con 0x7fddf00cdef0 2023-10-16T20:04:57.616 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.611+0000 7fdddeffd640 1 -- 172.21.15.88:0/1576720245 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde00b4870 con 0x7fddf00cdef0 2023-10-16T20:04:57.616 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.611+0000 7fddf4e54640 1 -- 172.21.15.88:0/1576720245 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddf0154240 con 0x7fddf00cdef0 2023-10-16T20:04:57.617 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.611+0000 7fddf4e54640 1 -- 172.21.15.88:0/1576720245 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddf00b86a0 con 0x7fddf00cdef0 2023-10-16T20:04:57.618 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.615+0000 7fdddeffd640 1 -- 172.21.15.88:0/1576720245 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdde00b49d0 con 0x7fddf00cdef0 2023-10-16T20:04:57.618 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.615+0000 7fdddeffd640 1 --2- 172.21.15.88:0/1576720245 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fddd00664a0 0x7fddd0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:57.618 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.615+0000 7fdddeffd640 1 -- 172.21.15.88:0/1576720245 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdde011d7b0 con 0x7fddf00cdef0 2023-10-16T20:04:57.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.615+0000 7fddedd9b640 1 --2- 172.21.15.88:0/1576720245 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fddd00664a0 0x7fddd0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:57.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.615+0000 7fddedd9b640 1 --2- 172.21.15.88:0/1576720245 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fddd00664a0 0x7fddd0068960 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fddf00d1580 tx=0x7fdde4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:57.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.619+0000 7fdddeffd640 1 -- 172.21.15.88:0/1576720245 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdde00eacc0 con 0x7fddf00cdef0 2023-10-16T20:04:57.776 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:57.775+0000 7fddf4e54640 1 -- 172.21.15.88:0/1576720245 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fddf004e9c0 con 0x7fddf00cdef0 2023-10-16T20:04:58.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.411+0000 7fdddeffd640 1 -- 172.21.15.88:0/1576720245 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v23) v1 ==== 157+0+0 (secure 0 0 0) 0x7fdde00ee2e0 con 0x7fddf00cdef0 2023-10-16T20:04:58.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.411+0000 7fddf4e54640 1 -- 172.21.15.88:0/1576720245 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fddd00664a0 msgr2=0x7fddd0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:58.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.411+0000 7fddf4e54640 1 --2- 172.21.15.88:0/1576720245 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fddd00664a0 0x7fddd0068960 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fddf00d1580 tx=0x7fdde4009040 comp rx=0 tx=0).stop 2023-10-16T20:04:58.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.411+0000 7fddf4e54640 1 -- 172.21.15.88:0/1576720245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddf00cdef0 msgr2=0x7fddf00b7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:58.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.411+0000 7fddf4e54640 1 --2- 172.21.15.88:0/1576720245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddf00cdef0 0x7fddf00b7a30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdde00ad040 tx=0x7fdde00192e0 comp rx=0 tx=0).stop 2023-10-16T20:04:58.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.411+0000 7fddf4e54640 1 -- 172.21.15.88:0/1576720245 shutdown_connections 2023-10-16T20:04:58.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.411+0000 7fddf4e54640 1 --2- 172.21.15.88:0/1576720245 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fddd00664a0 0x7fddd0068960 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:58.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.411+0000 7fddf4e54640 1 --2- 172.21.15.88:0/1576720245 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fddf01570e0 0x7fddf014e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:58.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.411+0000 7fddf4e54640 1 --2- 172.21.15.88:0/1576720245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddf015d140 0x7fddf014edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:58.414 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.411+0000 7fddf4e54640 1 --2- 172.21.15.88:0/1576720245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddf00cdef0 0x7fddf00b7a30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:58.414 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.411+0000 7fddf4e54640 1 -- 172.21.15.88:0/1576720245 >> 172.21.15.88:0/1576720245 conn(0x7fddf00715c0 msgr2=0x7fddf015c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:58.414 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.411+0000 7fddf4e54640 1 -- 172.21.15.88:0/1576720245 shutdown_connections 2023-10-16T20:04:58.414 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.411+0000 7fddf4e54640 1 -- 172.21.15.88:0/1576720245 wait complete. 2023-10-16T20:04:58.435 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T20:04:58.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.583+0000 7f3717fa1640 1 Processor -- start 2023-10-16T20:04:58.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.583+0000 7f3717fa1640 1 -- start start 2023-10-16T20:04:58.584 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.583+0000 7f3717fa1640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3710164570 0x7f37100af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:58.584 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.583+0000 7f3717fa1640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f37100afe20 0x7f37100addf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:58.585 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.583+0000 7f3717fa1640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37100ae330 0x7f3710070250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:58.585 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.583+0000 7f3717fa1640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37100ae840 con 0x7f37100afe20 2023-10-16T20:04:58.585 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.583+0000 7f3717fa1640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37100aeb20 con 0x7f3710164570 2023-10-16T20:04:58.585 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.583+0000 7f3717fa1640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37100aee00 con 0x7f37100ae330 2023-10-16T20:04:58.585 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.583+0000 7f371553c640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f37100afe20 0x7f37100addf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:58.586 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.583+0000 7f371553c640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f37100afe20 0x7f37100addf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41494/0 (socket says 172.21.15.88:41494) 2023-10-16T20:04:58.586 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.583+0000 7f371553c640 1 -- 172.21.15.88:0/646181510 learned_addr learned my addr 172.21.15.88:0/646181510 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:04:58.586 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.583+0000 7f3715d3d640 1 --2- 172.21.15.88:0/646181510 >> v2:172.21.15.94:3300/0 conn(0x7f3710164570 0x7f37100af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:58.586 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.583+0000 7f371653e640 1 --2- 172.21.15.88:0/646181510 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37100ae330 0x7f3710070250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:58.586 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.583+0000 7f371553c640 1 -- 172.21.15.88:0/646181510 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37100ae330 msgr2=0x7f3710070250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:04:58.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.583+0000 7f371553c640 1 --2- 172.21.15.88:0/646181510 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37100ae330 0x7f3710070250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:58.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.583+0000 7f371553c640 1 -- 172.21.15.88:0/646181510 >> v2:172.21.15.94:3300/0 conn(0x7f3710164570 msgr2=0x7f37100af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:04:58.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.583+0000 7f371553c640 1 --2- 172.21.15.88:0/646181510 >> v2:172.21.15.94:3300/0 conn(0x7f3710164570 0x7f37100af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:58.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.583+0000 7f371553c640 1 -- 172.21.15.88:0/646181510 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37100af0e0 con 0x7f37100afe20 2023-10-16T20:04:58.588 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.587+0000 7f371553c640 1 --2- 172.21.15.88:0/646181510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37100afe20 0x7f37100addf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f371006e5f0 tx=0x7f370001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed5078e2c72de71 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:58.589 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.587+0000 7f3714d3b640 1 -- 172.21.15.88:0/646181510 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37000a89b0 con 0x7f37100afe20 2023-10-16T20:04:58.589 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.587+0000 7f3714d3b640 1 -- 172.21.15.88:0/646181510 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37000a8b10 con 0x7f37100afe20 2023-10-16T20:04:58.589 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.587+0000 7f3714d3b640 1 -- 172.21.15.88:0/646181510 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37000a8ea0 con 0x7f37100afe20 2023-10-16T20:04:58.589 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.587+0000 7f3717fa1640 1 -- 172.21.15.88:0/646181510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37100afe20 msgr2=0x7f37100addf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:58.589 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.587+0000 7f3717fa1640 1 --2- 172.21.15.88:0/646181510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37100afe20 0x7f37100addf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f371006e5f0 tx=0x7f370001ecf0 comp rx=0 tx=0).stop 2023-10-16T20:04:58.590 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.587+0000 7f3717fa1640 1 -- 172.21.15.88:0/646181510 shutdown_connections 2023-10-16T20:04:58.590 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.587+0000 7f3717fa1640 1 --2- 172.21.15.88:0/646181510 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37100ae330 0x7f3710070250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:58.590 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.587+0000 7f3717fa1640 1 --2- 172.21.15.88:0/646181510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37100afe20 0x7f37100addf0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:58.590 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.587+0000 7f3717fa1640 1 --2- 172.21.15.88:0/646181510 >> v2:172.21.15.94:3300/0 conn(0x7f3710164570 0x7f37100af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:58.590 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.587+0000 7f3717fa1640 1 -- 172.21.15.88:0/646181510 >> 172.21.15.88:0/646181510 conn(0x7f37100cb080 msgr2=0x7f37100cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:04:58.591 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.587+0000 7f3717fa1640 1 -- 172.21.15.88:0/646181510 shutdown_connections 2023-10-16T20:04:58.591 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.587+0000 7f3717fa1640 1 -- 172.21.15.88:0/646181510 wait complete. 2023-10-16T20:04:58.593 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.591+0000 7f3717fa1640 1 Processor -- start 2023-10-16T20:04:58.593 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.591+0000 7f3717fa1640 1 -- start start 2023-10-16T20:04:58.593 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.591+0000 7f3717fa1640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37100ae330 0x7f37101f5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:58.593 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.591+0000 7f3717fa1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37100afe20 0x7f371006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:58.593 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.591+0000 7f3717fa1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3710164570 0x7f371006d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:58.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.591+0000 7f3717fa1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37100ac190 con 0x7f3710164570 2023-10-16T20:04:58.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.591+0000 7f3717fa1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3710146ed0 con 0x7f37100afe20 2023-10-16T20:04:58.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.591+0000 7f3717fa1640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3710146530 con 0x7f37100ae330 2023-10-16T20:04:58.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.591+0000 7f371653e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3710164570 0x7f371006d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:58.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.591+0000 7f371653e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3710164570 0x7f371006d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41510/0 (socket says 172.21.15.88:41510) 2023-10-16T20:04:58.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.591+0000 7f371653e640 1 -- 172.21.15.88:0/3326049400 learned_addr learned my addr 172.21.15.88:0/3326049400 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:04:58.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.591+0000 7f371553c640 1 --2- 172.21.15.88:0/3326049400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37100afe20 0x7f371006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:58.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.591+0000 7f3715d3d640 1 --2- 172.21.15.88:0/3326049400 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37100ae330 0x7f37101f5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:58.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.591+0000 7f371653e640 1 -- 172.21.15.88:0/3326049400 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37100ae330 msgr2=0x7f37101f5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:58.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.591+0000 7f371653e640 1 --2- 172.21.15.88:0/3326049400 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37100ae330 0x7f37101f5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:58.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.591+0000 7f371653e640 1 -- 172.21.15.88:0/3326049400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37100afe20 msgr2=0x7f371006d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:04:58.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.591+0000 7f371653e640 1 --2- 172.21.15.88:0/3326049400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37100afe20 0x7f371006d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:04:58.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.591+0000 7f371653e640 1 -- 172.21.15.88:0/3326049400 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37000094d0 con 0x7f3710164570 2023-10-16T20:04:58.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.591+0000 7f371653e640 1 --2- 172.21.15.88:0/3326049400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3710164570 0x7f371006d680 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3708007e40 tx=0x7f370800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:58.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.591+0000 7f37067fc640 1 -- 172.21.15.88:0/3326049400 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3708014620 con 0x7f3710164570 2023-10-16T20:04:58.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.591+0000 7f37067fc640 1 -- 172.21.15.88:0/3326049400 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f370800ee90 con 0x7f3710164570 2023-10-16T20:04:58.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.591+0000 7f37067fc640 1 -- 172.21.15.88:0/3326049400 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37080149e0 con 0x7f3710164570 2023-10-16T20:04:58.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.591+0000 7f3717fa1640 1 -- 172.21.15.88:0/3326049400 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3710146810 con 0x7f3710164570 2023-10-16T20:04:58.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.591+0000 7f3717fa1640 1 -- 172.21.15.88:0/3326049400 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37100a8b40 con 0x7f3710164570 2023-10-16T20:04:58.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.595+0000 7f3717fa1640 1 -- 172.21.15.88:0/3326049400 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3710147010 con 0x7f3710164570 2023-10-16T20:04:58.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.595+0000 7f37067fc640 1 -- 172.21.15.88:0/3326049400 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3708014b40 con 0x7f3710164570 2023-10-16T20:04:58.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.595+0000 7f37067fc640 1 --2- 172.21.15.88:0/3326049400 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f36f40664a0 0x7f36f4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:04:58.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.595+0000 7f37067fc640 1 -- 172.21.15.88:0/3326049400 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5935+0+0 (secure 0 0 0) 0x7f37080847d0 con 0x7f3710164570 2023-10-16T20:04:58.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.595+0000 7f3715d3d640 1 --2- 172.21.15.88:0/3326049400 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f36f40664a0 0x7f36f4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:04:58.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.595+0000 7f3715d3d640 1 --2- 172.21.15.88:0/3326049400 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f36f40664a0 0x7f36f4068960 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f37100713b0 tx=0x7f370c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:04:58.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.599+0000 7f37067fc640 1 -- 172.21.15.88:0/3326049400 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3710147010 con 0x7f3710164570 2023-10-16T20:04:58.748 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:04:58.747+0000 7f3717fa1640 1 -- 172.21.15.88:0/3326049400 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3710147010 con 0x7f3710164570 2023-10-16T20:05:00.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.431+0000 7f37067fc640 1 -- 172.21.15.88:0/3326049400 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v25) v1 ==== 228+0+0 (secure 0 0 0) 0x7f3710147010 con 0x7f3710164570 2023-10-16T20:05:00.431 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data_ec' created 2023-10-16T20:05:00.433 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.435+0000 7f3717fa1640 1 -- 172.21.15.88:0/3326049400 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f36f40664a0 msgr2=0x7f36f4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:00.433 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.435+0000 7f3717fa1640 1 --2- 172.21.15.88:0/3326049400 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f36f40664a0 0x7f36f4068960 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f37100713b0 tx=0x7f370c009190 comp rx=0 tx=0).stop 2023-10-16T20:05:00.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.435+0000 7f3717fa1640 1 -- 172.21.15.88:0/3326049400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3710164570 msgr2=0x7f371006d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:00.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.435+0000 7f3717fa1640 1 --2- 172.21.15.88:0/3326049400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3710164570 0x7f371006d680 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3708007e40 tx=0x7f370800e4b0 comp rx=0 tx=0).stop 2023-10-16T20:05:00.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.435+0000 7f3717fa1640 1 -- 172.21.15.88:0/3326049400 shutdown_connections 2023-10-16T20:05:00.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.435+0000 7f3717fa1640 1 --2- 172.21.15.88:0/3326049400 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f36f40664a0 0x7f36f4068960 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:00.435 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.435+0000 7f3717fa1640 1 --2- 172.21.15.88:0/3326049400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3710164570 0x7f371006d680 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:00.435 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.435+0000 7f3717fa1640 1 --2- 172.21.15.88:0/3326049400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37100afe20 0x7f371006d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:00.435 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.435+0000 7f3717fa1640 1 --2- 172.21.15.88:0/3326049400 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37100ae330 0x7f37101f5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:00.435 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.435+0000 7f3717fa1640 1 -- 172.21.15.88:0/3326049400 >> 172.21.15.88:0/3326049400 conn(0x7f37100cb080 msgr2=0x7f3710069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:00.435 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.435+0000 7f3717fa1640 1 -- 172.21.15.88:0/3326049400 shutdown_connections 2023-10-16T20:05:00.436 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.435+0000 7f3717fa1640 1 -- 172.21.15.88:0/3326049400 wait complete. 2023-10-16T20:05:00.467 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-10-16T20:05:00.614 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.615+0000 7f34a8209640 1 Processor -- start 2023-10-16T20:05:00.614 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.615+0000 7f34a8209640 1 -- start start 2023-10-16T20:05:00.615 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.615+0000 7f34a8209640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f34a015ab20 0x7f34a00ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:00.616 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.615+0000 7f34a8209640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f34a006a5a0 0x7f34a00b1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:00.616 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.615+0000 7f34a8209640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34a00c9840 con 0x7f34a00c91b0 2023-10-16T20:05:00.616 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.615+0000 7f34a8209640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34a015af00 con 0x7f34a015ab20 2023-10-16T20:05:00.616 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.615+0000 7f34a8209640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34a00c9640 con 0x7f34a006a5a0 2023-10-16T20:05:00.616 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.615+0000 7f34a5fa5640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f34a015ab20 0x7f34a00ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:00.616 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.615+0000 7f34a5fa5640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f34a015ab20 0x7f34a00ca7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41520/0 (socket says 172.21.15.88:41520) 2023-10-16T20:05:00.617 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.615+0000 7f34a5fa5640 1 -- 172.21.15.88:0/986788373 learned_addr learned my addr 172.21.15.88:0/986788373 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:00.617 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.615+0000 7f34a5fa5640 1 -- 172.21.15.88:0/986788373 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f34a006a5a0 msgr2=0x7f34a00b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:00.617 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.615+0000 7f34a5fa5640 1 --2- 172.21.15.88:0/986788373 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f34a006a5a0 0x7f34a00b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:00.617 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.615+0000 7f34a5fa5640 1 -- 172.21.15.88:0/986788373 >> v1:172.21.15.94:6789/0 conn(0x7f34a00c91b0 legacy=0x7f34a0069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:00.617 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.615+0000 7f34a5fa5640 1 -- 172.21.15.88:0/986788373 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a00c9b20 con 0x7f34a015ab20 2023-10-16T20:05:00.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.619+0000 7f34a5fa5640 1 --2- 172.21.15.88:0/986788373 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34a015ab20 0x7f34a00ca7a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f349c009870 tx=0x7f349c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68113404e2994b70 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:00.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.619+0000 7f34a4fa3640 1 -- 172.21.15.88:0/986788373 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349c0ac630 con 0x7f34a015ab20 2023-10-16T20:05:00.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.619+0000 7f34a4fa3640 1 -- 172.21.15.88:0/986788373 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f349c0ac790 con 0x7f34a015ab20 2023-10-16T20:05:00.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.619+0000 7f34a4fa3640 1 -- 172.21.15.88:0/986788373 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349c0acb20 con 0x7f34a015ab20 2023-10-16T20:05:00.620 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.619+0000 7f34a8209640 1 -- 172.21.15.88:0/986788373 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34a015ab20 msgr2=0x7f34a00ca7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:00.620 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.619+0000 7f34a8209640 1 --2- 172.21.15.88:0/986788373 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34a015ab20 0x7f34a00ca7a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f349c009870 tx=0x7f349c01c500 comp rx=0 tx=0).stop 2023-10-16T20:05:00.621 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.619+0000 7f34a8209640 1 -- 172.21.15.88:0/986788373 shutdown_connections 2023-10-16T20:05:00.621 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.619+0000 7f34a8209640 1 --2- 172.21.15.88:0/986788373 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f34a006a5a0 0x7f34a00b1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:00.621 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.619+0000 7f34a8209640 1 --2- 172.21.15.88:0/986788373 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34a015ab20 0x7f34a00ca7a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:00.621 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.619+0000 7f34a8209640 1 -- 172.21.15.88:0/986788373 >> 172.21.15.88:0/986788373 conn(0x7f34a00ce660 msgr2=0x7f34a00cea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:00.621 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.619+0000 7f34a8209640 1 -- 172.21.15.88:0/986788373 shutdown_connections 2023-10-16T20:05:00.621 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.619+0000 7f34a8209640 1 -- 172.21.15.88:0/986788373 wait complete. 2023-10-16T20:05:00.623 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.623+0000 7f34a8209640 1 Processor -- start 2023-10-16T20:05:00.624 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.623+0000 7f34a8209640 1 -- start start 2023-10-16T20:05:00.624 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.623+0000 7f34a8209640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34a006a5a0 0x7f34a0152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:00.624 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.623+0000 7f34a8209640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34a00c91b0 0x7f34a0155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:00.624 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.623+0000 7f34a8209640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f34a015ab20 0x7f34a00cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:00.625 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.623+0000 7f34a8209640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a0154650 con 0x7f34a006a5a0 2023-10-16T20:05:00.625 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.623+0000 7f34a8209640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a0153a40 con 0x7f34a00c91b0 2023-10-16T20:05:00.625 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.623+0000 7f34a8209640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34a0153d20 con 0x7f34a015ab20 2023-10-16T20:05:00.625 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.623+0000 7f34a5fa5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34a006a5a0 0x7f34a0152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:00.625 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.623+0000 7f34a5fa5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34a006a5a0 0x7f34a0152350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41530/0 (socket says 172.21.15.88:41530) 2023-10-16T20:05:00.626 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.623+0000 7f34a5fa5640 1 -- 172.21.15.88:0/4263497419 learned_addr learned my addr 172.21.15.88:0/4263497419 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:00.626 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.623+0000 7f34a5fa5640 1 -- 172.21.15.88:0/4263497419 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f34a015ab20 msgr2=0x7f34a00cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:00.626 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.623+0000 7f34a5fa5640 1 --2- 172.21.15.88:0/4263497419 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f34a015ab20 0x7f34a00cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:00.626 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.623+0000 7f34a5fa5640 1 -- 172.21.15.88:0/4263497419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34a00c91b0 msgr2=0x7f34a0155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:00.626 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.623+0000 7f34a5fa5640 1 --2- 172.21.15.88:0/4263497419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34a00c91b0 0x7f34a0155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:00.627 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.623+0000 7f34a5fa5640 1 -- 172.21.15.88:0/4263497419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f349c0094d0 con 0x7f34a006a5a0 2023-10-16T20:05:00.627 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.623+0000 7f34a5fa5640 1 --2- 172.21.15.88:0/4263497419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34a006a5a0 0x7f34a0152350 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f349c0af040 tx=0x7f349c01ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:00.627 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.623+0000 7f34967fc640 1 -- 172.21.15.88:0/4263497419 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349c0b7830 con 0x7f34a006a5a0 2023-10-16T20:05:00.627 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.623+0000 7f34967fc640 1 -- 172.21.15.88:0/4263497419 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f349c0b7990 con 0x7f34a006a5a0 2023-10-16T20:05:00.627 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.623+0000 7f34967fc640 1 -- 172.21.15.88:0/4263497419 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349c0b7d20 con 0x7f34a006a5a0 2023-10-16T20:05:00.627 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.623+0000 7f34a8209640 1 -- 172.21.15.88:0/4263497419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34a0154000 con 0x7f34a006a5a0 2023-10-16T20:05:00.628 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.623+0000 7f34a8209640 1 -- 172.21.15.88:0/4263497419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34a0152ac0 con 0x7f34a006a5a0 2023-10-16T20:05:00.629 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.623+0000 7f34a8209640 1 -- 172.21.15.88:0/4263497419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34a004e9c0 con 0x7f34a006a5a0 2023-10-16T20:05:00.629 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.627+0000 7f34967fc640 1 -- 172.21.15.88:0/4263497419 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f349c0b3040 con 0x7f34a006a5a0 2023-10-16T20:05:00.629 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.627+0000 7f34967fc640 1 --2- 172.21.15.88:0/4263497419 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f34840664d0 0x7f3484068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:00.629 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.627+0000 7f34967fc640 1 -- 172.21.15.88:0/4263497419 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f349c11e520 con 0x7f34a006a5a0 2023-10-16T20:05:00.629 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.627+0000 7f34a57a4640 1 --2- 172.21.15.88:0/4263497419 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f34840664d0 0x7f3484068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:00.630 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.627+0000 7f34a57a4640 1 --2- 172.21.15.88:0/4263497419 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f34840664d0 0x7f3484068990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3490007fd0 tx=0x7f34900079a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:00.631 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.631+0000 7f34967fc640 1 -- 172.21.15.88:0/4263497419 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f349c0eb760 con 0x7f34a006a5a0 2023-10-16T20:05:00.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:00.783+0000 7f34a8209640 1 -- 172.21.15.88:0/4263497419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f34a0159a70 con 0x7f34a006a5a0 2023-10-16T20:05:00.802 INFO:tasks.ceph.mds.b.smithi094.stderr:2023-10-16T20:05:00.798+0000 7f1da1992640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T20:05:01.436 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.435+0000 7f34967fc640 1 -- 172.21.15.88:0/4263497419 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7f349c0eed80 con 0x7f34a006a5a0 2023-10-16T20:05:01.436 INFO:teuthology.orchestra.run.smithi088.stderr:set pool 4 allow_ec_overwrites to true 2023-10-16T20:05:01.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.439+0000 7f34a8209640 1 -- 172.21.15.88:0/4263497419 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f34840664d0 msgr2=0x7f3484068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:01.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.439+0000 7f34a8209640 1 --2- 172.21.15.88:0/4263497419 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f34840664d0 0x7f3484068990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3490007fd0 tx=0x7f34900079a0 comp rx=0 tx=0).stop 2023-10-16T20:05:01.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.439+0000 7f34a8209640 1 -- 172.21.15.88:0/4263497419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34a006a5a0 msgr2=0x7f34a0152350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:01.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.439+0000 7f34a8209640 1 --2- 172.21.15.88:0/4263497419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34a006a5a0 0x7f34a0152350 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f349c0af040 tx=0x7f349c01ad40 comp rx=0 tx=0).stop 2023-10-16T20:05:01.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.439+0000 7f34a8209640 1 -- 172.21.15.88:0/4263497419 shutdown_connections 2023-10-16T20:05:01.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.439+0000 7f34a8209640 1 --2- 172.21.15.88:0/4263497419 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f34840664d0 0x7f3484068990 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3490007fd0 tx=0x7f34900079a0 comp rx=0 tx=0).stop 2023-10-16T20:05:01.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.439+0000 7f34a8209640 1 --2- 172.21.15.88:0/4263497419 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f34a015ab20 0x7f34a00cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:01.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.439+0000 7f34a8209640 1 --2- 172.21.15.88:0/4263497419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34a00c91b0 0x7f34a0155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:01.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.439+0000 7f34a8209640 1 --2- 172.21.15.88:0/4263497419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34a006a5a0 0x7f34a0152350 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:01.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.439+0000 7f34a8209640 1 -- 172.21.15.88:0/4263497419 >> 172.21.15.88:0/4263497419 conn(0x7f34a00ce660 msgr2=0x7f34a006c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:01.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.443+0000 7f34a8209640 1 -- 172.21.15.88:0/4263497419 shutdown_connections 2023-10-16T20:05:01.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.443+0000 7f34a8209640 1 -- 172.21.15.88:0/4263497419 wait complete. 2023-10-16T20:05:01.482 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-10-16T20:05:01.624 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.623+0000 7f69abda1640 1 Processor -- start 2023-10-16T20:05:01.624 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.623+0000 7f69abda1640 1 -- start start 2023-10-16T20:05:01.625 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.627+0000 7f69abda1640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f69a40b0c10 0x7f69a415aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:01.626 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.627+0000 7f69abda1640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69a415bd20 con 0x7f69a415b530 2023-10-16T20:05:01.626 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.627+0000 7f69abda1640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69a40702f0 con 0x7f69a4154f30 2023-10-16T20:05:01.626 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.627+0000 7f69abda1640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69a4155310 con 0x7f69a40b0c10 2023-10-16T20:05:01.627 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.627+0000 7f69aa33e640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f69a415b530 0x7f69a406c2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:44818/0 (socket says 172.21.15.88:44818) 2023-10-16T20:05:01.627 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.627+0000 7f69aa33e640 1 -- 172.21.15.88:0/4264914454 learned_addr learned my addr 172.21.15.88:0/4264914454 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:01.627 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.627+0000 7f69a933c640 1 --2- 172.21.15.88:0/4264914454 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f69a40b0c10 0x7f69a415aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:01.628 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.627+0000 7f69a8b3b640 1 -- 172.21.15.88:0/4264914454 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3205636757 0 0) 0x7f69a415bd20 con 0x7f69a415b530 2023-10-16T20:05:01.628 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.627+0000 7f69a8b3b640 1 -- 172.21.15.88:0/4264914454 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6990003530 con 0x7f69a415b530 2023-10-16T20:05:01.628 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.627+0000 7f69a8b3b640 1 -- 172.21.15.88:0/4264914454 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2423443598 0 0) 0x7f69a40702f0 con 0x7f69a4154f30 2023-10-16T20:05:01.628 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.627+0000 7f69a8b3b640 1 -- 172.21.15.88:0/4264914454 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69a415bd20 con 0x7f69a4154f30 2023-10-16T20:05:01.628 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.627+0000 7f69a933c640 1 -- 172.21.15.88:0/4264914454 >> v1:172.21.15.94:6789/0 conn(0x7f69a4154f30 legacy=0x7f69a40b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:01.629 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.627+0000 7f69a933c640 1 -- 172.21.15.88:0/4264914454 >> v1:172.21.15.88:6789/0 conn(0x7f69a415b530 legacy=0x7f69a406c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:01.629 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.627+0000 7f69a933c640 1 -- 172.21.15.88:0/4264914454 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a40705d0 con 0x7f69a40b0c10 2023-10-16T20:05:01.630 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.631+0000 7f69a933c640 1 --2- 172.21.15.88:0/4264914454 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f69a40b0c10 0x7f69a415aff0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f69940097f0 tx=0x7f699401c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bb6275c78cd7d61 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:01.630 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.631+0000 7f69a8b3b640 1 -- 172.21.15.88:0/4264914454 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69940aa5f0 con 0x7f69a40b0c10 2023-10-16T20:05:01.630 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.631+0000 7f69a8b3b640 1 -- 172.21.15.88:0/4264914454 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69940aa750 con 0x7f69a40b0c10 2023-10-16T20:05:01.630 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.631+0000 7f69a8b3b640 1 -- 172.21.15.88:0/4264914454 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69940aaae0 con 0x7f69a40b0c10 2023-10-16T20:05:01.631 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.631+0000 7f69abda1640 1 -- 172.21.15.88:0/4264914454 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f69a40b0c10 msgr2=0x7f69a415aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:01.631 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.631+0000 7f69abda1640 1 --2- 172.21.15.88:0/4264914454 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f69a40b0c10 0x7f69a415aff0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f69940097f0 tx=0x7f699401c490 comp rx=0 tx=0).stop 2023-10-16T20:05:01.632 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.631+0000 7f69abda1640 1 -- 172.21.15.88:0/4264914454 shutdown_connections 2023-10-16T20:05:01.632 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.631+0000 7f69abda1640 1 --2- 172.21.15.88:0/4264914454 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f69a40b0c10 0x7f69a415aff0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:01.632 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.631+0000 7f69abda1640 1 -- 172.21.15.88:0/4264914454 >> 172.21.15.88:0/4264914454 conn(0x7f69a40cb080 msgr2=0x7f69a40cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:01.632 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.631+0000 7f69abda1640 1 -- 172.21.15.88:0/4264914454 shutdown_connections 2023-10-16T20:05:01.632 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.631+0000 7f69abda1640 1 -- 172.21.15.88:0/4264914454 wait complete. 2023-10-16T20:05:01.633 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.631+0000 7f69abda1640 1 Processor -- start 2023-10-16T20:05:01.633 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.631+0000 7f69abda1640 1 -- start start 2023-10-16T20:05:01.635 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.635+0000 7f69abda1640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f69a40b0c10 0x7f69a413a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:01.635 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.635+0000 7f69abda1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69a4154f30 0x7f69a413a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:01.635 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.635+0000 7f69abda1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69a415b530 0x7f69a4150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:01.635 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.635+0000 7f69abda1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a4152f80 con 0x7f69a415b530 2023-10-16T20:05:01.636 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.635+0000 7f69abda1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a413ac80 con 0x7f69a4154f30 2023-10-16T20:05:01.636 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.635+0000 7f69abda1640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69a413af60 con 0x7f69a40b0c10 2023-10-16T20:05:01.636 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.635+0000 7f69aa33e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69a415b530 0x7f69a4150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:01.636 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.635+0000 7f69aa33e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69a415b530 0x7f69a4150670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41540/0 (socket says 172.21.15.88:41540) 2023-10-16T20:05:01.636 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.635+0000 7f69aa33e640 1 -- 172.21.15.88:0/1843632654 learned_addr learned my addr 172.21.15.88:0/1843632654 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:01.636 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.635+0000 7f69a9b3d640 1 --2- 172.21.15.88:0/1843632654 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f69a40b0c10 0x7f69a413a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:01.637 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.635+0000 7f69a933c640 1 --2- 172.21.15.88:0/1843632654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69a4154f30 0x7f69a413a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:01.637 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.635+0000 7f69aa33e640 1 -- 172.21.15.88:0/1843632654 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f69a40b0c10 msgr2=0x7f69a413a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:01.637 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.635+0000 7f69aa33e640 1 --2- 172.21.15.88:0/1843632654 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f69a40b0c10 0x7f69a413a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:01.637 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.635+0000 7f69aa33e640 1 -- 172.21.15.88:0/1843632654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69a4154f30 msgr2=0x7f69a413a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:01.637 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.635+0000 7f69aa33e640 1 --2- 172.21.15.88:0/1843632654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69a4154f30 0x7f69a413a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:01.638 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.635+0000 7f69aa33e640 1 -- 172.21.15.88:0/1843632654 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6994009420 con 0x7f69a415b530 2023-10-16T20:05:01.638 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.635+0000 7f69a9b3d640 1 --2- 172.21.15.88:0/1843632654 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f69a40b0c10 0x7f69a413a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:01.638 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.635+0000 7f69aa33e640 1 --2- 172.21.15.88:0/1843632654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69a415b530 0x7f69a4150670 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f699c00caf0 tx=0x7f699c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:01.638 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.635+0000 7f699a7fc640 1 -- 172.21.15.88:0/1843632654 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699c00aca0 con 0x7f69a415b530 2023-10-16T20:05:01.638 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.635+0000 7f699a7fc640 1 -- 172.21.15.88:0/1843632654 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f699c00ae00 con 0x7f69a415b530 2023-10-16T20:05:01.639 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.635+0000 7f69abda1640 1 -- 172.21.15.88:0/1843632654 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69a413b240 con 0x7f69a415b530 2023-10-16T20:05:01.639 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.635+0000 7f699a7fc640 1 -- 172.21.15.88:0/1843632654 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699c012620 con 0x7f69a415b530 2023-10-16T20:05:01.639 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.635+0000 7f69abda1640 1 -- 172.21.15.88:0/1843632654 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69a4150bb0 con 0x7f69a415b530 2023-10-16T20:05:01.640 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.635+0000 7f69abda1640 1 -- 172.21.15.88:0/1843632654 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69a413b480 con 0x7f69a415b530 2023-10-16T20:05:01.640 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.635+0000 7f699a7fc640 1 -- 172.21.15.88:0/1843632654 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f699c007070 con 0x7f69a415b530 2023-10-16T20:05:01.640 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.639+0000 7f699a7fc640 1 --2- 172.21.15.88:0/1843632654 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f69880664d0 0x7f6988068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:01.641 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.639+0000 7f699a7fc640 1 -- 172.21.15.88:0/1843632654 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f699c084070 con 0x7f69a415b530 2023-10-16T20:05:01.641 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.639+0000 7f69a9b3d640 1 --2- 172.21.15.88:0/1843632654 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f69880664d0 0x7f6988068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:01.641 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.639+0000 7f69a9b3d640 1 --2- 172.21.15.88:0/1843632654 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f69880664d0 0x7f6988068990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f69a00011f0 tx=0x7f69a0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:01.642 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.643+0000 7f699a7fc640 1 -- 172.21.15.88:0/1843632654 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f699c0b5090 con 0x7f69a415b530 2023-10-16T20:05:01.833 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:01.835+0000 7f69abda1640 1 -- 172.21.15.88:0/1843632654 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f69a404e9c0 con 0x7f69a415b530 2023-10-16T20:05:02.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.439+0000 7f699a7fc640 1 -- 172.21.15.88:0/1843632654 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f699c0170b0 con 0x7f69a415b530 2023-10-16T20:05:02.438 INFO:teuthology.orchestra.run.smithi088.stderr:added data pool 4 to fsmap 2023-10-16T20:05:02.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.439+0000 7f69abda1640 1 -- 172.21.15.88:0/1843632654 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f69880664d0 msgr2=0x7f6988068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:02.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.439+0000 7f69abda1640 1 --2- 172.21.15.88:0/1843632654 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f69880664d0 0x7f6988068990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f69a00011f0 tx=0x7f69a0009040 comp rx=0 tx=0).stop 2023-10-16T20:05:02.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.439+0000 7f69abda1640 1 -- 172.21.15.88:0/1843632654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69a415b530 msgr2=0x7f69a4150670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:02.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.439+0000 7f69abda1640 1 --2- 172.21.15.88:0/1843632654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69a415b530 0x7f69a4150670 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f699c00caf0 tx=0x7f699c00a060 comp rx=0 tx=0).stop 2023-10-16T20:05:02.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.439+0000 7f69abda1640 1 -- 172.21.15.88:0/1843632654 shutdown_connections 2023-10-16T20:05:02.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.439+0000 7f69abda1640 1 --2- 172.21.15.88:0/1843632654 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f69880664d0 0x7f6988068990 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:02.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.439+0000 7f69abda1640 1 --2- 172.21.15.88:0/1843632654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69a415b530 0x7f69a4150670 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:02.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.439+0000 7f69abda1640 1 --2- 172.21.15.88:0/1843632654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69a4154f30 0x7f69a413a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:02.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.439+0000 7f69abda1640 1 --2- 172.21.15.88:0/1843632654 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f69a40b0c10 0x7f69a413a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:02.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.439+0000 7f69abda1640 1 -- 172.21.15.88:0/1843632654 >> 172.21.15.88:0/1843632654 conn(0x7f69a40cb080 msgr2=0x7f69a40ca800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:02.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.443+0000 7f69abda1640 1 -- 172.21.15.88:0/1843632654 shutdown_connections 2023-10-16T20:05:02.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.443+0000 7f69abda1640 1 -- 172.21.15.88:0/1843632654 wait complete. 2023-10-16T20:05:02.465 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T20:05:02.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.607+0000 7feff666b640 1 Processor -- start 2023-10-16T20:05:02.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.607+0000 7feff666b640 1 -- start start 2023-10-16T20:05:02.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.607+0000 7feff666b640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7feff015d9f0 0x7feff0161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:02.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.607+0000 7feff666b640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff00b87e0 con 0x7feff00d22e0 2023-10-16T20:05:02.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.607+0000 7feff666b640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff015df10 con 0x7feff00d26c0 2023-10-16T20:05:02.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.607+0000 7feff666b640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feff015e1f0 con 0x7feff015d9f0 2023-10-16T20:05:02.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.607+0000 7fefef7fe640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7feff015d9f0 0x7feff0161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:02.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.607+0000 7fefeffff640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7feff00d22e0 0x7feff015d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:48224/0 (socket says 172.21.15.88:48224) 2023-10-16T20:05:02.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.607+0000 7fefeffff640 1 -- 172.21.15.88:0/1763041799 learned_addr learned my addr 172.21.15.88:0/1763041799 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:02.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.607+0000 7fefef7fe640 1 -- 172.21.15.88:0/1763041799 >> v1:172.21.15.94:6789/0 conn(0x7feff00d26c0 legacy=0x7feff00cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:02.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.607+0000 7fefef7fe640 1 -- 172.21.15.88:0/1763041799 >> v1:172.21.15.88:6789/0 conn(0x7feff00d22e0 legacy=0x7feff015d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:02.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.607+0000 7fefef7fe640 1 -- 172.21.15.88:0/1763041799 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff0160970 con 0x7feff015d9f0 2023-10-16T20:05:02.610 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.611+0000 7fefef7fe640 1 --2- 172.21.15.88:0/1763041799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feff015d9f0 0x7feff0161450 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fefe40097c0 tx=0x7fefe401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a6bce020d9e09e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:02.611 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.611+0000 7fefeeffd640 1 -- 172.21.15.88:0/1763041799 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe40ab600 con 0x7feff015d9f0 2023-10-16T20:05:02.611 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.611+0000 7fefeeffd640 1 -- 172.21.15.88:0/1763041799 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe40a7050 con 0x7feff015d9f0 2023-10-16T20:05:02.612 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.611+0000 7fefeeffd640 1 -- 172.21.15.88:0/1763041799 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe40ab980 con 0x7feff015d9f0 2023-10-16T20:05:02.612 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.611+0000 7feff666b640 1 -- 172.21.15.88:0/1763041799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feff015d9f0 msgr2=0x7feff0161450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:02.612 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.611+0000 7feff666b640 1 --2- 172.21.15.88:0/1763041799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feff015d9f0 0x7feff0161450 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fefe40097c0 tx=0x7fefe401c380 comp rx=0 tx=0).stop 2023-10-16T20:05:02.613 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.611+0000 7feff666b640 1 -- 172.21.15.88:0/1763041799 shutdown_connections 2023-10-16T20:05:02.613 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.611+0000 7feff666b640 1 --2- 172.21.15.88:0/1763041799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feff015d9f0 0x7feff0161450 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:02.613 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.611+0000 7feff666b640 1 -- 172.21.15.88:0/1763041799 >> 172.21.15.88:0/1763041799 conn(0x7feff00715c0 msgr2=0x7feff00719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:02.614 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.611+0000 7feff666b640 1 -- 172.21.15.88:0/1763041799 shutdown_connections 2023-10-16T20:05:02.614 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.611+0000 7feff666b640 1 -- 172.21.15.88:0/1763041799 wait complete. 2023-10-16T20:05:02.617 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.615+0000 7feff666b640 1 Processor -- start 2023-10-16T20:05:02.618 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.615+0000 7feff666b640 1 -- start start 2023-10-16T20:05:02.618 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.615+0000 7feff666b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feff00d22e0 0x7feff00c0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:02.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.615+0000 7feff666b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feff00d26c0 0x7feff00b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:02.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.615+0000 7feff666b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feff015d9f0 0x7feff00c5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:02.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.615+0000 7feff666b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff014dd50 con 0x7feff00d22e0 2023-10-16T20:05:02.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.615+0000 7feff666b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff014e8d0 con 0x7feff00d26c0 2023-10-16T20:05:02.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.615+0000 7feff666b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff014dfa0 con 0x7feff015d9f0 2023-10-16T20:05:02.620 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.615+0000 7fefeffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feff00d22e0 0x7feff00c0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:02.620 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.615+0000 7fefeffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feff00d22e0 0x7feff00c0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47932/0 (socket says 172.21.15.88:47932) 2023-10-16T20:05:02.620 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.615+0000 7fefeffff640 1 -- 172.21.15.88:0/617334462 learned_addr learned my addr 172.21.15.88:0/617334462 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:02.620 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.615+0000 7fefef7fe640 1 --2- 172.21.15.88:0/617334462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feff00d26c0 0x7feff00b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:02.620 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.615+0000 7feff4c08640 1 --2- 172.21.15.88:0/617334462 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feff015d9f0 0x7feff00c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:02.621 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.615+0000 7fefeffff640 1 -- 172.21.15.88:0/617334462 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feff015d9f0 msgr2=0x7feff00c5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:02.621 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.615+0000 7fefeffff640 1 --2- 172.21.15.88:0/617334462 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feff015d9f0 0x7feff00c5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:02.621 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.615+0000 7fefeffff640 1 -- 172.21.15.88:0/617334462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feff00d26c0 msgr2=0x7feff00b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:02.621 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.615+0000 7fefeffff640 1 --2- 172.21.15.88:0/617334462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feff00d26c0 0x7feff00b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:02.621 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.615+0000 7fefeffff640 1 -- 172.21.15.88:0/617334462 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe4009420 con 0x7feff00d22e0 2023-10-16T20:05:02.622 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.615+0000 7fefef7fe640 1 --2- 172.21.15.88:0/617334462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feff00d26c0 0x7feff00b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:02.622 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.615+0000 7fefeffff640 1 --2- 172.21.15.88:0/617334462 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feff00d22e0 0x7feff00c0dc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fefe000c500 tx=0x7fefe000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:02.622 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.615+0000 7fefecff9640 1 -- 172.21.15.88:0/617334462 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe0005cf0 con 0x7feff00d22e0 2023-10-16T20:05:02.622 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.615+0000 7fefecff9640 1 -- 172.21.15.88:0/617334462 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe0005e50 con 0x7feff00d22e0 2023-10-16T20:05:02.622 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.615+0000 7feff666b640 1 -- 172.21.15.88:0/617334462 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff014e280 con 0x7feff00d22e0 2023-10-16T20:05:02.623 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.615+0000 7fefecff9640 1 -- 172.21.15.88:0/617334462 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe00072f0 con 0x7feff00d22e0 2023-10-16T20:05:02.623 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.615+0000 7feff666b640 1 -- 172.21.15.88:0/617334462 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff00c1300 con 0x7feff00d22e0 2023-10-16T20:05:02.623 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.615+0000 7feff666b640 1 -- 172.21.15.88:0/617334462 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff004e9c0 con 0x7feff00d22e0 2023-10-16T20:05:02.623 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.619+0000 7fefecff9640 1 -- 172.21.15.88:0/617334462 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fefe0007450 con 0x7feff00d22e0 2023-10-16T20:05:02.624 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.619+0000 7fefecff9640 1 --2- 172.21.15.88:0/617334462 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fefdc0664a0 0x7fefdc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:02.624 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.619+0000 7fefecff9640 1 -- 172.21.15.88:0/617334462 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fefe0083650 con 0x7feff00d22e0 2023-10-16T20:05:02.624 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.619+0000 7fefef7fe640 1 --2- 172.21.15.88:0/617334462 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fefdc0664a0 0x7fefdc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:02.625 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.619+0000 7fefef7fe640 1 --2- 172.21.15.88:0/617334462 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fefdc0664a0 0x7fefdc068960 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fefe40ad000 tx=0x7fefe4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:02.625 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.623+0000 7fefecff9640 1 -- 172.21.15.88:0/617334462 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fefe0050900 con 0x7feff00d22e0 2023-10-16T20:05:02.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.806+0000 7feff666b640 1 -- 172.21.15.88:0/617334462 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feff015b920 con 0x7feff00d22e0 2023-10-16T20:05:02.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.806+0000 7fefecff9640 1 -- 172.21.15.88:0/617334462 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4906 (secure 0 0 0) 0x7fefe0053f20 con 0x7feff00d22e0 2023-10-16T20:05:02.808 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:02.808 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3803627113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3803627113},{"type":"v1","addr":"172.21.15.94:6834","nonce":3803627113}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3758093239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":3758093239},{"type":"v1","addr":"172.21.15.88:6837","nonce":3758093239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/3836772385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3836772385},{"type":"v1","addr":"172.21.15.88:6836","nonce":3836772385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T20:04:56.759719+0000","modified":"2023-10-16T20:05:01.835793+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6835/555575750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":555575750},{"type":"v1","addr":"172.21.15.94:6835","nonce":555575750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-10-16T20:05:02.809 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 7 2023-10-16T20:05:02.810 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.810+0000 7feff666b640 1 -- 172.21.15.88:0/617334462 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fefdc0664a0 msgr2=0x7fefdc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:02.810 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.810+0000 7feff666b640 1 --2- 172.21.15.88:0/617334462 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fefdc0664a0 0x7fefdc068960 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fefe40ad000 tx=0x7fefe4008040 comp rx=0 tx=0).stop 2023-10-16T20:05:02.810 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.810+0000 7feff666b640 1 -- 172.21.15.88:0/617334462 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feff00d22e0 msgr2=0x7feff00c0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:02.810 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.810+0000 7feff666b640 1 --2- 172.21.15.88:0/617334462 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feff00d22e0 0x7feff00c0dc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fefe000c500 tx=0x7fefe000c9c0 comp rx=0 tx=0).stop 2023-10-16T20:05:02.810 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.810+0000 7feff666b640 1 -- 172.21.15.88:0/617334462 shutdown_connections 2023-10-16T20:05:02.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.810+0000 7feff666b640 1 --2- 172.21.15.88:0/617334462 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fefdc0664a0 0x7fefdc068960 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:02.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.810+0000 7feff666b640 1 --2- 172.21.15.88:0/617334462 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feff015d9f0 0x7feff00c5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:02.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.810+0000 7feff666b640 1 --2- 172.21.15.88:0/617334462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feff00d26c0 0x7feff00b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:02.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.810+0000 7feff666b640 1 --2- 172.21.15.88:0/617334462 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feff00d22e0 0x7feff00c0dc0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:02.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.810+0000 7feff666b640 1 -- 172.21.15.88:0/617334462 >> 172.21.15.88:0/617334462 conn(0x7feff00715c0 msgr2=0x7feff015bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:02.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.810+0000 7feff666b640 1 -- 172.21.15.88:0/617334462 shutdown_connections 2023-10-16T20:05:02.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.810+0000 7feff666b640 1 -- 172.21.15.88:0/617334462 wait complete. 2023-10-16T20:05:02.828 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:05:02.966 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.966+0000 7fa30ecea640 1 Processor -- start 2023-10-16T20:05:02.966 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.966+0000 7fa30ecea640 1 -- start start 2023-10-16T20:05:02.968 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.966+0000 7fa30ecea640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fa308153ea0 0x7fa3080d1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:02.968 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.966+0000 7fa30ecea640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fa3080cd970 0x7fa3080cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:02.968 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.966+0000 7fa30ecea640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3080b5510 con 0x7fa3080d2660 2023-10-16T20:05:02.968 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.966+0000 7fa30ecea640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa30815d9a0 con 0x7fa3080cd970 2023-10-16T20:05:02.968 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.966+0000 7fa30ecea640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3080b4990 con 0x7fa308153ea0 2023-10-16T20:05:02.969 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.966+0000 7fa30d287640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fa3080cd970 0x7fa3080cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:02.969 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.966+0000 7fa30d287640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fa3080cd970 0x7fa3080cdd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47934/0 (socket says 172.21.15.88:47934) 2023-10-16T20:05:02.969 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.966+0000 7fa30d287640 1 -- 172.21.15.88:0/2364433985 learned_addr learned my addr 172.21.15.88:0/2364433985 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:02.969 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.966+0000 7fa30ca86640 1 --2- 172.21.15.88:0/2364433985 >> v2:172.21.15.94:3300/0 conn(0x7fa308153ea0 0x7fa3080d1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:02.969 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.966+0000 7fa30d287640 1 -- 172.21.15.88:0/2364433985 >> v2:172.21.15.94:3300/0 conn(0x7fa308153ea0 msgr2=0x7fa3080d1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:02.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.966+0000 7fa30d287640 1 --2- 172.21.15.88:0/2364433985 >> v2:172.21.15.94:3300/0 conn(0x7fa308153ea0 0x7fa3080d1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:02.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.966+0000 7fa30d287640 1 -- 172.21.15.88:0/2364433985 >> v1:172.21.15.94:6789/0 conn(0x7fa3080d2660 legacy=0x7fa30815d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:02.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.966+0000 7fa30d287640 1 -- 172.21.15.88:0/2364433985 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3080b4c70 con 0x7fa3080cd970 2023-10-16T20:05:02.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.970+0000 7fa30d287640 1 --2- 172.21.15.88:0/2364433985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3080cd970 0x7fa3080cdd80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa30806c610 tx=0x7fa30001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f383774caa106d88 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:02.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.970+0000 7fa2ff7fe640 1 -- 172.21.15.88:0/2364433985 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3000a5860 con 0x7fa3080cd970 2023-10-16T20:05:02.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.970+0000 7fa2ff7fe640 1 -- 172.21.15.88:0/2364433985 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3000a59c0 con 0x7fa3080cd970 2023-10-16T20:05:02.972 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.970+0000 7fa2ff7fe640 1 -- 172.21.15.88:0/2364433985 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3000a5d50 con 0x7fa3080cd970 2023-10-16T20:05:02.972 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.970+0000 7fa30ecea640 1 -- 172.21.15.88:0/2364433985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3080cd970 msgr2=0x7fa3080cdd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:02.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.970+0000 7fa30ecea640 1 --2- 172.21.15.88:0/2364433985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3080cd970 0x7fa3080cdd80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa30806c610 tx=0x7fa30001c9a0 comp rx=0 tx=0).stop 2023-10-16T20:05:02.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.970+0000 7fa30ecea640 1 -- 172.21.15.88:0/2364433985 shutdown_connections 2023-10-16T20:05:02.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.970+0000 7fa30ecea640 1 --2- 172.21.15.88:0/2364433985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3080cd970 0x7fa3080cdd80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:02.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.970+0000 7fa30ecea640 1 --2- 172.21.15.88:0/2364433985 >> v2:172.21.15.94:3300/0 conn(0x7fa308153ea0 0x7fa3080d1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:02.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.970+0000 7fa30ecea640 1 -- 172.21.15.88:0/2364433985 >> 172.21.15.88:0/2364433985 conn(0x7fa3080715c0 msgr2=0x7fa3080719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:02.974 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.970+0000 7fa30ecea640 1 -- 172.21.15.88:0/2364433985 shutdown_connections 2023-10-16T20:05:02.974 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.970+0000 7fa30ecea640 1 -- 172.21.15.88:0/2364433985 wait complete. 2023-10-16T20:05:02.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.974+0000 7fa30ecea640 1 Processor -- start 2023-10-16T20:05:02.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.974+0000 7fa30ecea640 1 -- start start 2023-10-16T20:05:02.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.974+0000 7fa30ecea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3080cd970 0x7fa3080c4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:02.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.974+0000 7fa30ecea640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3080d2660 0x7fa30814b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:02.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.974+0000 7fa30ecea640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa308153ea0 0x7fa3080d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:02.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.974+0000 7fa30ecea640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3080b5df0 con 0x7fa3080d2660 2023-10-16T20:05:02.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.974+0000 7fa30ecea640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa308152720 con 0x7fa3080cd970 2023-10-16T20:05:02.978 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.974+0000 7fa30ecea640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa308152a00 con 0x7fa308153ea0 2023-10-16T20:05:02.978 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.974+0000 7fa2fffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3080d2660 0x7fa30814b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:02.978 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.974+0000 7fa2fffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3080d2660 0x7fa30814b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47946/0 (socket says 172.21.15.88:47946) 2023-10-16T20:05:02.978 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.974+0000 7fa2fffff640 1 -- 172.21.15.88:0/1145898819 learned_addr learned my addr 172.21.15.88:0/1145898819 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:02.978 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.974+0000 7fa30ca86640 1 --2- 172.21.15.88:0/1145898819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3080cd970 0x7fa3080c4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:02.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.974+0000 7fa30d287640 1 --2- 172.21.15.88:0/1145898819 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa308153ea0 0x7fa3080d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:02.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.974+0000 7fa2fffff640 1 -- 172.21.15.88:0/1145898819 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa308153ea0 msgr2=0x7fa3080d2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:02.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.974+0000 7fa2fffff640 1 --2- 172.21.15.88:0/1145898819 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa308153ea0 0x7fa3080d2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:02.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.974+0000 7fa2fffff640 1 -- 172.21.15.88:0/1145898819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3080cd970 msgr2=0x7fa3080c4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:02.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.974+0000 7fa2fffff640 1 --2- 172.21.15.88:0/1145898819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3080cd970 0x7fa3080c4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:02.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.974+0000 7fa2fffff640 1 -- 172.21.15.88:0/1145898819 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa300009b30 con 0x7fa3080d2660 2023-10-16T20:05:02.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.974+0000 7fa2fffff640 1 --2- 172.21.15.88:0/1145898819 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3080d2660 0x7fa30814b6f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa2f800c500 tx=0x7fa2f800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:02.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.974+0000 7fa2fd7fa640 1 -- 172.21.15.88:0/1145898819 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8005220 con 0x7fa3080d2660 2023-10-16T20:05:02.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.974+0000 7fa2fd7fa640 1 -- 172.21.15.88:0/1145898819 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f8005380 con 0x7fa3080d2660 2023-10-16T20:05:02.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.974+0000 7fa2fd7fa640 1 -- 172.21.15.88:0/1145898819 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8007270 con 0x7fa3080d2660 2023-10-16T20:05:02.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.974+0000 7fa30ecea640 1 -- 172.21.15.88:0/1145898819 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa308152ce0 con 0x7fa3080d2660 2023-10-16T20:05:02.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.974+0000 7fa30ecea640 1 -- 172.21.15.88:0/1145898819 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa30815f4e0 con 0x7fa3080d2660 2023-10-16T20:05:02.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.974+0000 7fa30ecea640 1 -- 172.21.15.88:0/1145898819 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa30814cca0 con 0x7fa3080d2660 2023-10-16T20:05:02.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.978+0000 7fa2fd7fa640 1 -- 172.21.15.88:0/1145898819 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa2f80073d0 con 0x7fa3080d2660 2023-10-16T20:05:02.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.978+0000 7fa2fd7fa640 1 --2- 172.21.15.88:0/1145898819 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa2f40664d0 0x7fa2f4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:02.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.978+0000 7fa2fd7fa640 1 -- 172.21.15.88:0/1145898819 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa2f80838b0 con 0x7fa3080d2660 2023-10-16T20:05:02.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.978+0000 7fa30ca86640 1 --2- 172.21.15.88:0/1145898819 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa2f40664d0 0x7fa2f4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:02.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.978+0000 7fa30ca86640 1 --2- 172.21.15.88:0/1145898819 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa2f40664d0 0x7fa2f4068990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa3081534d0 tx=0x7fa304008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:02.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:02.982+0000 7fa2fd7fa640 1 -- 172.21.15.88:0/1145898819 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa2f8050b00 con 0x7fa3080d2660 2023-10-16T20:05:03.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.130+0000 7fa30ecea640 1 -- 172.21.15.88:0/1145898819 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa30804e9c0 con 0x7fa3080d2660 2023-10-16T20:05:03.134 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.134+0000 7fa2fd7fa640 1 -- 172.21.15.88:0/1145898819 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18274 (secure 0 0 0) 0x7fa2f8054120 con 0x7fa3080d2660 2023-10-16T20:05:03.134 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:03.135 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:05:01.835630+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T20:04:54.442487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T20:04:55.766641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T20:04:59.475830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:05:03.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.134+0000 7fa30ecea640 1 -- 172.21.15.88:0/1145898819 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa2f40664d0 msgr2=0x7fa2f4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:03.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.134+0000 7fa30ecea640 1 --2- 172.21.15.88:0/1145898819 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa2f40664d0 0x7fa2f4068990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa3081534d0 tx=0x7fa304008040 comp rx=0 tx=0).stop 2023-10-16T20:05:03.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.134+0000 7fa30ecea640 1 -- 172.21.15.88:0/1145898819 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3080d2660 msgr2=0x7fa30814b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:03.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.134+0000 7fa30ecea640 1 --2- 172.21.15.88:0/1145898819 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3080d2660 0x7fa30814b6f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa2f800c500 tx=0x7fa2f800c9c0 comp rx=0 tx=0).stop 2023-10-16T20:05:03.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.134+0000 7fa30ecea640 1 -- 172.21.15.88:0/1145898819 shutdown_connections 2023-10-16T20:05:03.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.134+0000 7fa30ecea640 1 --2- 172.21.15.88:0/1145898819 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa2f40664d0 0x7fa2f4068990 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:03.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.134+0000 7fa30ecea640 1 --2- 172.21.15.88:0/1145898819 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa308153ea0 0x7fa3080d2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:03.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.134+0000 7fa30ecea640 1 --2- 172.21.15.88:0/1145898819 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3080d2660 0x7fa30814b6f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:03.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.134+0000 7fa30ecea640 1 --2- 172.21.15.88:0/1145898819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3080cd970 0x7fa3080c4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:03.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.134+0000 7fa30ecea640 1 -- 172.21.15.88:0/1145898819 >> 172.21.15.88:0/1145898819 conn(0x7fa3080715c0 msgr2=0x7fa30814c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:03.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.134+0000 7fa30ecea640 1 -- 172.21.15.88:0/1145898819 shutdown_connections 2023-10-16T20:05:03.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.134+0000 7fa30ecea640 1 -- 172.21.15.88:0/1145898819 wait complete. 2023-10-16T20:05:03.155 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:05:03.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.290+0000 7f17241d9640 1 Processor -- start 2023-10-16T20:05:03.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.290+0000 7f17241d9640 1 -- start start 2023-10-16T20:05:03.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f17241d9640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f171c15cb80 0x7f171c0b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:03.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f17241d9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f171c0cebd0 0x7f171c06bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:03.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f17241d9640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f171c06b490 con 0x7f171c0b28b0 2023-10-16T20:05:03.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f17241d9640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f171c0b2d20 con 0x7f171c15cb80 2023-10-16T20:05:03.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f17241d9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f171c0cf180 con 0x7f171c0cebd0 2023-10-16T20:05:03.295 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f1721774640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f171c0b28b0 0x7f171c0ce4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:48238/0 (socket says 172.21.15.88:48238) 2023-10-16T20:05:03.295 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f1721774640 1 -- 172.21.15.88:0/2851066659 learned_addr learned my addr 172.21.15.88:0/2851066659 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:03.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f1722776640 1 --2- 172.21.15.88:0/2851066659 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f171c0cebd0 0x7f171c06bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:03.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f1721f75640 1 --2- 172.21.15.88:0/2851066659 >> v2:172.21.15.94:3300/0 conn(0x7f171c15cb80 0x7f171c0b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:03.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f1720f73640 1 -- 172.21.15.88:0/2851066659 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2423836247 0 0) 0x7f171c06b490 con 0x7f171c0b28b0 2023-10-16T20:05:03.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f1720f73640 1 -- 172.21.15.88:0/2851066659 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1708003530 con 0x7f171c0b28b0 2023-10-16T20:05:03.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f1720f73640 1 -- 172.21.15.88:0/2851066659 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4137014677 0 0) 0x7f1708003530 con 0x7f171c0b28b0 2023-10-16T20:05:03.298 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f1720f73640 1 -- 172.21.15.88:0/2851066659 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f171c0cebd0 msgr2=0x7f171c06bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:03.298 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f1720f73640 1 --2- 172.21.15.88:0/2851066659 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f171c0cebd0 0x7f171c06bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:03.298 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f1720f73640 1 -- 172.21.15.88:0/2851066659 >> v2:172.21.15.94:3300/0 conn(0x7f171c15cb80 msgr2=0x7f171c0b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:03.298 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f1720f73640 1 --2- 172.21.15.88:0/2851066659 >> v2:172.21.15.94:3300/0 conn(0x7f171c15cb80 0x7f171c0b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:03.298 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f1720f73640 1 -- 172.21.15.88:0/2851066659 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171c06b700 con 0x7f171c0b28b0 2023-10-16T20:05:03.299 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f1720f73640 1 -- 172.21.15.88:0/2851066659 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f170c0016a0 con 0x7f171c0b28b0 2023-10-16T20:05:03.299 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f1720f73640 1 -- 172.21.15.88:0/2851066659 >> v1:172.21.15.88:6789/0 conn(0x7f171c0b28b0 legacy=0x7f171c0ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:03.299 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f1720f73640 1 --2- 172.21.15.88:0/2851066659 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f171c0cebd0 0x7f1708003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:03.299 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f1720f73640 1 --2- 172.21.15.88:0/2851066659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1708003f00 0x7f17080047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:03.299 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f1720f73640 1 --2- 172.21.15.88:0/2851066659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1708004d10 0x7f1708005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:03.300 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f1720f73640 1 -- 172.21.15.88:0/2851066659 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171c0b2d20 con 0x7f1708003f00 2023-10-16T20:05:03.300 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f1720f73640 1 -- 172.21.15.88:0/2851066659 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171c0cf180 con 0x7f1708004d10 2023-10-16T20:05:03.300 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f1720f73640 1 -- 172.21.15.88:0/2851066659 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170800b990 con 0x7f171c0cebd0 2023-10-16T20:05:03.300 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f1721774640 1 --2- 172.21.15.88:0/2851066659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1708003f00 0x7f17080047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:03.300 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f1721f75640 1 --2- 172.21.15.88:0/2851066659 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f171c0cebd0 0x7f1708003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:03.300 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f1722776640 1 --2- 172.21.15.88:0/2851066659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1708004d10 0x7f1708005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:03.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f1721774640 1 -- 172.21.15.88:0/2851066659 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f171c0cebd0 msgr2=0x7f1708003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:03.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f1721774640 1 --2- 172.21.15.88:0/2851066659 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f171c0cebd0 0x7f1708003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:03.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f1721774640 1 -- 172.21.15.88:0/2851066659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1708004d10 msgr2=0x7f1708005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:03.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f1721774640 1 --2- 172.21.15.88:0/2851066659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1708004d10 0x7f1708005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:03.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.294+0000 7f1721774640 1 -- 172.21.15.88:0/2851066659 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1708003530 con 0x7f1708003f00 2023-10-16T20:05:03.304 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.298+0000 7f1721774640 1 --2- 172.21.15.88:0/2851066659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1708003f00 0x7f17080047d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f170c00c210 tx=0x7f170c01f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774657f079573f9c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:03.305 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.298+0000 7f1720f73640 1 -- 172.21.15.88:0/2851066659 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c005ae0 con 0x7f1708003f00 2023-10-16T20:05:03.305 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.298+0000 7f1720f73640 1 -- 172.21.15.88:0/2851066659 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170c0ab070 con 0x7f1708003f00 2023-10-16T20:05:03.305 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.298+0000 7f1720f73640 1 -- 172.21.15.88:0/2851066659 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c005e80 con 0x7f1708003f00 2023-10-16T20:05:03.305 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.298+0000 7f17241d9640 1 -- 172.21.15.88:0/2851066659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1708003f00 msgr2=0x7f17080047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:03.306 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.298+0000 7f17241d9640 1 --2- 172.21.15.88:0/2851066659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1708003f00 0x7f17080047d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f170c00c210 tx=0x7f170c01f330 comp rx=0 tx=0).stop 2023-10-16T20:05:03.306 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.298+0000 7f1722776640 1 -- 172.21.15.88:0/2851066659 reap_dead start 2023-10-16T20:05:03.306 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.298+0000 7f17241d9640 1 -- 172.21.15.88:0/2851066659 shutdown_connections 2023-10-16T20:05:03.306 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.298+0000 7f17241d9640 1 -- 172.21.15.88:0/2851066659 >> 172.21.15.88:0/2851066659 conn(0x7f171c0cd370 msgr2=0x7f171c0cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:03.306 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.298+0000 7f17241d9640 1 -- 172.21.15.88:0/2851066659 shutdown_connections 2023-10-16T20:05:03.307 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.298+0000 7f17241d9640 1 -- 172.21.15.88:0/2851066659 wait complete. 2023-10-16T20:05:03.307 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.302+0000 7f17241d9640 1 Processor -- start 2023-10-16T20:05:03.307 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.302+0000 7f17241d9640 1 -- start start 2023-10-16T20:05:03.307 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.302+0000 7f17241d9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1708003f00 0x7f171c0b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:03.307 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.302+0000 7f17241d9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f171c0ca250 0x7f171c1f57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:03.307 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.302+0000 7f17241d9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f171c0ca6d0 0x7f171c1f5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:03.308 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.302+0000 7f17241d9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171c0b55d0 con 0x7f171c0ca250 2023-10-16T20:05:03.308 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.302+0000 7f17241d9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171c0b58b0 con 0x7f1708003f00 2023-10-16T20:05:03.308 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.302+0000 7f17241d9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f171c0b5b90 con 0x7f171c0ca6d0 2023-10-16T20:05:03.308 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.302+0000 7f1721774640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f171c0ca250 0x7f171c1f57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:03.308 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.302+0000 7f1721774640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f171c0ca250 0x7f171c1f57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47962/0 (socket says 172.21.15.88:47962) 2023-10-16T20:05:03.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.302+0000 7f1721774640 1 -- 172.21.15.88:0/4117066981 learned_addr learned my addr 172.21.15.88:0/4117066981 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:03.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.302+0000 7f1721f75640 1 --2- 172.21.15.88:0/4117066981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1708003f00 0x7f171c0b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:03.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.302+0000 7f1722776640 1 --2- 172.21.15.88:0/4117066981 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f171c0ca6d0 0x7f171c1f5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:03.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.302+0000 7f1721774640 1 -- 172.21.15.88:0/4117066981 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f171c0ca6d0 msgr2=0x7f171c1f5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:03.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.302+0000 7f1721774640 1 --2- 172.21.15.88:0/4117066981 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f171c0ca6d0 0x7f171c1f5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:03.310 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.302+0000 7f1721774640 1 -- 172.21.15.88:0/4117066981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1708003f00 msgr2=0x7f171c0b4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:03.310 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.302+0000 7f1721774640 1 --2- 172.21.15.88:0/4117066981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1708003f00 0x7f171c0b4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:03.310 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.302+0000 7f1721774640 1 -- 172.21.15.88:0/4117066981 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171c0b5e70 con 0x7f171c0ca250 2023-10-16T20:05:03.310 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.302+0000 7f1721774640 1 --2- 172.21.15.88:0/4117066981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f171c0ca250 0x7f171c1f57d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f170c01f370 tx=0x7f1708005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:03.310 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.302+0000 7f17127fc640 1 -- 172.21.15.88:0/4117066981 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c0b7070 con 0x7f171c0ca250 2023-10-16T20:05:03.311 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.302+0000 7f17127fc640 1 -- 172.21.15.88:0/4117066981 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170c005980 con 0x7f171c0ca250 2023-10-16T20:05:03.311 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.302+0000 7f17127fc640 1 -- 172.21.15.88:0/4117066981 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c005d30 con 0x7f171c0ca250 2023-10-16T20:05:03.311 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.302+0000 7f17241d9640 1 -- 172.21.15.88:0/4117066981 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f171c0b6150 con 0x7f171c0ca250 2023-10-16T20:05:03.311 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.302+0000 7f17241d9640 1 -- 172.21.15.88:0/4117066981 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f171c1f6420 con 0x7f171c0ca250 2023-10-16T20:05:03.311 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.302+0000 7f17241d9640 1 -- 172.21.15.88:0/4117066981 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f171c1f6610 con 0x7f171c0ca250 2023-10-16T20:05:03.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.306+0000 7f17127fc640 1 -- 172.21.15.88:0/4117066981 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f170c0ad020 con 0x7f171c0ca250 2023-10-16T20:05:03.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.306+0000 7f17127fc640 1 --2- 172.21.15.88:0/4117066981 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f17000664a0 0x7f1700068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:03.313 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.306+0000 7f17127fc640 1 -- 172.21.15.88:0/4117066981 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f170c120640 con 0x7f171c0ca250 2023-10-16T20:05:03.313 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.306+0000 7f1721f75640 1 --2- 172.21.15.88:0/4117066981 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f17000664a0 0x7f1700068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:03.313 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.306+0000 7f1721f75640 1 --2- 172.21.15.88:0/4117066981 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f17000664a0 0x7f1700068960 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f171c0c41b0 tx=0x7f171800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:03.313 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.310+0000 7f17127fc640 1 -- 172.21.15.88:0/4117066981 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f170c0ed7d0 con 0x7f171c0ca250 2023-10-16T20:05:03.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.466+0000 7f17241d9640 1 -- 172.21.15.88:0/4117066981 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f171c04e9c0 con 0x7f171c0ca250 2023-10-16T20:05:03.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.470+0000 7f17127fc640 1 -- 172.21.15.88:0/4117066981 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18274 (secure 0 0 0) 0x7f170c0f0df0 con 0x7f171c0ca250 2023-10-16T20:05:03.469 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:03.470 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:05:01.835630+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T20:04:54.442487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T20:04:55.766641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T20:04:59.475830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:05:03.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.470+0000 7f17241d9640 1 -- 172.21.15.88:0/4117066981 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f17000664a0 msgr2=0x7f1700068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:03.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.470+0000 7f17241d9640 1 --2- 172.21.15.88:0/4117066981 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f17000664a0 0x7f1700068960 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f171c0c41b0 tx=0x7f171800b040 comp rx=0 tx=0).stop 2023-10-16T20:05:03.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.470+0000 7f17241d9640 1 -- 172.21.15.88:0/4117066981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f171c0ca250 msgr2=0x7f171c1f57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:03.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.470+0000 7f17241d9640 1 --2- 172.21.15.88:0/4117066981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f171c0ca250 0x7f171c1f57d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f170c01f370 tx=0x7f1708005800 comp rx=0 tx=0).stop 2023-10-16T20:05:03.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.470+0000 7f17241d9640 1 -- 172.21.15.88:0/4117066981 shutdown_connections 2023-10-16T20:05:03.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.470+0000 7f17241d9640 1 --2- 172.21.15.88:0/4117066981 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f17000664a0 0x7f1700068960 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:03.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.470+0000 7f17241d9640 1 --2- 172.21.15.88:0/4117066981 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f171c0ca6d0 0x7f171c1f5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:03.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.470+0000 7f17241d9640 1 --2- 172.21.15.88:0/4117066981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f171c0ca250 0x7f171c1f57d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:03.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.470+0000 7f17241d9640 1 --2- 172.21.15.88:0/4117066981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1708003f00 0x7f171c0b4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:03.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.470+0000 7f17241d9640 1 -- 172.21.15.88:0/4117066981 >> 172.21.15.88:0/4117066981 conn(0x7f171c0cd370 msgr2=0x7f171c0b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:03.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.470+0000 7f17241d9640 1 -- 172.21.15.88:0/4117066981 shutdown_connections 2023-10-16T20:05:03.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.470+0000 7f17241d9640 1 -- 172.21.15.88:0/4117066981 wait complete. 2023-10-16T20:05:03.505 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T20:05:03.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.642+0000 7f3e0d209640 1 Processor -- start 2023-10-16T20:05:03.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.642+0000 7f3e0d209640 1 -- start start 2023-10-16T20:05:03.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.642+0000 7f3e0d209640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3e080d22e0 0x7f3e08161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:03.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.642+0000 7f3e0d209640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3e0815d9f0 0x7f3e080cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:03.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.642+0000 7f3e0d209640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e080b87e0 con 0x7f3e080d26c0 2023-10-16T20:05:03.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.642+0000 7f3e0d209640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e0815df10 con 0x7f3e0815d9f0 2023-10-16T20:05:03.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.642+0000 7f3e0d209640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0815e1f0 con 0x7f3e080d22e0 2023-10-16T20:05:03.646 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.642+0000 7f3e06d9d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3e080d22e0 0x7f3e08161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:03.646 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.642+0000 7f3dfffff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f3e080d26c0 0x7f3e0815d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:50674/0 (socket says 172.21.15.88:50674) 2023-10-16T20:05:03.646 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.642+0000 7f3dfffff640 1 -- 172.21.15.88:0/1195105999 learned_addr learned my addr 172.21.15.88:0/1195105999 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:03.646 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.642+0000 7f3e0759e640 1 --2- 172.21.15.88:0/1195105999 >> v2:172.21.15.94:3300/0 conn(0x7f3e0815d9f0 0x7f3e080cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:03.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.646+0000 7f3e06d9d640 1 -- 172.21.15.88:0/1195105999 >> v2:172.21.15.94:3300/0 conn(0x7f3e0815d9f0 msgr2=0x7f3e080cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:03.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.646+0000 7f3e06d9d640 1 --2- 172.21.15.88:0/1195105999 >> v2:172.21.15.94:3300/0 conn(0x7f3e0815d9f0 0x7f3e080cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:03.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.646+0000 7f3e06d9d640 1 -- 172.21.15.88:0/1195105999 >> v1:172.21.15.94:6789/0 conn(0x7f3e080d26c0 legacy=0x7f3e0815d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:03.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.646+0000 7f3e06d9d640 1 -- 172.21.15.88:0/1195105999 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e08160970 con 0x7f3e080d22e0 2023-10-16T20:05:03.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.646+0000 7f3e0659c640 1 -- 172.21.15.88:0/1195105999 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1421848919 0 0) 0x7f3e080b87e0 con 0x7f3e080d26c0 2023-10-16T20:05:03.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.646+0000 7f3e06d9d640 1 --2- 172.21.15.88:0/1195105999 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3e080d22e0 0x7f3e08161450 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3df80097c0 tx=0x7f3df801c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20ec7fd889c96327 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:03.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.646+0000 7f3e0659c640 1 -- 172.21.15.88:0/1195105999 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df80ac620 con 0x7f3e080d22e0 2023-10-16T20:05:03.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.646+0000 7f3e0659c640 1 -- 172.21.15.88:0/1195105999 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3df80ac780 con 0x7f3e080d22e0 2023-10-16T20:05:03.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.646+0000 7f3e0659c640 1 -- 172.21.15.88:0/1195105999 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df80acb10 con 0x7f3e080d22e0 2023-10-16T20:05:03.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.646+0000 7f3e0d209640 1 -- 172.21.15.88:0/1195105999 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3e080d22e0 msgr2=0x7f3e08161450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:03.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.646+0000 7f3e0d209640 1 --2- 172.21.15.88:0/1195105999 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3e080d22e0 0x7f3e08161450 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3df80097c0 tx=0x7f3df801c3c0 comp rx=0 tx=0).stop 2023-10-16T20:05:03.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.646+0000 7f3e0d209640 1 -- 172.21.15.88:0/1195105999 shutdown_connections 2023-10-16T20:05:03.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.646+0000 7f3e0d209640 1 --2- 172.21.15.88:0/1195105999 >> v2:172.21.15.94:3300/0 conn(0x7f3e0815d9f0 0x7f3e080cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:03.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.646+0000 7f3e0d209640 1 --2- 172.21.15.88:0/1195105999 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3e080d22e0 0x7f3e08161450 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:03.651 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.646+0000 7f3e0d209640 1 -- 172.21.15.88:0/1195105999 >> 172.21.15.88:0/1195105999 conn(0x7f3e080715c0 msgr2=0x7f3e080719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:03.651 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.650+0000 7f3e0d209640 1 -- 172.21.15.88:0/1195105999 shutdown_connections 2023-10-16T20:05:03.651 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.650+0000 7f3e0d209640 1 -- 172.21.15.88:0/1195105999 wait complete. 2023-10-16T20:05:03.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.650+0000 7f3e0d209640 1 Processor -- start 2023-10-16T20:05:03.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.650+0000 7f3e0d209640 1 -- start start 2023-10-16T20:05:03.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.650+0000 7f3e0d209640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e080d22e0 0x7f3e08147900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:03.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.650+0000 7f3e0d209640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e080d26c0 0x7f3e0815e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:03.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.650+0000 7f3e0d209640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3e0815d9f0 0x7f3e080c5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:03.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.650+0000 7f3e0d209640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e080c4d60 con 0x7f3e080d26c0 2023-10-16T20:05:03.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.650+0000 7f3e0d209640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0815fa30 con 0x7f3e080d22e0 2023-10-16T20:05:03.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.650+0000 7f3e0d209640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0815ee20 con 0x7f3e0815d9f0 2023-10-16T20:05:03.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.650+0000 7f3dfffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e080d26c0 0x7f3e0815e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:03.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.650+0000 7f3dfffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e080d26c0 0x7f3e0815e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47976/0 (socket says 172.21.15.88:47976) 2023-10-16T20:05:03.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.650+0000 7f3dfffff640 1 -- 172.21.15.88:0/1355414710 learned_addr learned my addr 172.21.15.88:0/1355414710 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:03.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.650+0000 7f3e06d9d640 1 --2- 172.21.15.88:0/1355414710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e080d22e0 0x7f3e08147900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:03.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.650+0000 7f3e0759e640 1 --2- 172.21.15.88:0/1355414710 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3e0815d9f0 0x7f3e080c5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:03.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.650+0000 7f3dfffff640 1 -- 172.21.15.88:0/1355414710 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3e0815d9f0 msgr2=0x7f3e080c5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:03.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.650+0000 7f3dfffff640 1 --2- 172.21.15.88:0/1355414710 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3e0815d9f0 0x7f3e080c5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:03.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.650+0000 7f3dfffff640 1 -- 172.21.15.88:0/1355414710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e080d22e0 msgr2=0x7f3e08147900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:03.657 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.650+0000 7f3dfffff640 1 --2- 172.21.15.88:0/1355414710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e080d22e0 0x7f3e08147900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:03.657 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.650+0000 7f3dfffff640 1 -- 172.21.15.88:0/1355414710 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df8009420 con 0x7f3e080d26c0 2023-10-16T20:05:03.657 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.650+0000 7f3dfffff640 1 --2- 172.21.15.88:0/1355414710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e080d26c0 0x7f3e0815e360 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3df000c550 tx=0x7f3df000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:03.657 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.654+0000 7f3dff7fe640 1 -- 172.21.15.88:0/1355414710 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df0005cf0 con 0x7f3e080d26c0 2023-10-16T20:05:03.657 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.654+0000 7f3dff7fe640 1 -- 172.21.15.88:0/1355414710 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3df0005e70 con 0x7f3e080d26c0 2023-10-16T20:05:03.657 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.654+0000 7f3dff7fe640 1 -- 172.21.15.88:0/1355414710 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df0007310 con 0x7f3e080d26c0 2023-10-16T20:05:03.658 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.654+0000 7f3e0d209640 1 -- 172.21.15.88:0/1355414710 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e0815f100 con 0x7f3e080d26c0 2023-10-16T20:05:03.658 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.654+0000 7f3e0d209640 1 -- 172.21.15.88:0/1355414710 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e0815f6c0 con 0x7f3e080d26c0 2023-10-16T20:05:03.659 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.654+0000 7f3e0d209640 1 -- 172.21.15.88:0/1355414710 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e0804e9c0 con 0x7f3e080d26c0 2023-10-16T20:05:03.659 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.654+0000 7f3dff7fe640 1 -- 172.21.15.88:0/1355414710 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3df0007470 con 0x7f3e080d26c0 2023-10-16T20:05:03.659 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.654+0000 7f3dff7fe640 1 --2- 172.21.15.88:0/1355414710 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3de80664a0 0x7f3de8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:03.659 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.654+0000 7f3dff7fe640 1 -- 172.21.15.88:0/1355414710 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3df00836b0 con 0x7f3e080d26c0 2023-10-16T20:05:03.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.654+0000 7f3e06d9d640 1 --2- 172.21.15.88:0/1355414710 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3de80664a0 0x7f3de8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:03.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.658+0000 7f3e06d9d640 1 --2- 172.21.15.88:0/1355414710 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3de80664a0 0x7f3de8068960 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3df80ae000 tx=0x7f3df80a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:03.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.658+0000 7f3dff7fe640 1 -- 172.21.15.88:0/1355414710 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3df0050980 con 0x7f3e080d26c0 2023-10-16T20:05:03.840 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.838+0000 7f3e0d209640 1 -- 172.21.15.88:0/1355414710 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e0815fb70 con 0x7f3e080d26c0 2023-10-16T20:05:03.841 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.838+0000 7f3dff7fe640 1 -- 172.21.15.88:0/1355414710 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4906 (secure 0 0 0) 0x7f3df0053fa0 con 0x7f3e080d26c0 2023-10-16T20:05:03.841 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:03.841 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3803627113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3803627113},{"type":"v1","addr":"172.21.15.94:6834","nonce":3803627113}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3758093239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":3758093239},{"type":"v1","addr":"172.21.15.88:6837","nonce":3758093239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/3836772385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3836772385},{"type":"v1","addr":"172.21.15.88:6836","nonce":3836772385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T20:04:56.759719+0000","modified":"2023-10-16T20:05:01.835793+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6835/555575750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":555575750},{"type":"v1","addr":"172.21.15.94:6835","nonce":555575750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-10-16T20:05:03.842 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 7 2023-10-16T20:05:03.843 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.842+0000 7f3e0d209640 1 -- 172.21.15.88:0/1355414710 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3de80664a0 msgr2=0x7f3de8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:03.843 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.842+0000 7f3e0d209640 1 --2- 172.21.15.88:0/1355414710 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3de80664a0 0x7f3de8068960 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3df80ae000 tx=0x7f3df80a7000 comp rx=0 tx=0).stop 2023-10-16T20:05:03.844 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.842+0000 7f3e0d209640 1 -- 172.21.15.88:0/1355414710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e080d26c0 msgr2=0x7f3e0815e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:03.844 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.842+0000 7f3e0d209640 1 --2- 172.21.15.88:0/1355414710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e080d26c0 0x7f3e0815e360 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3df000c550 tx=0x7f3df000ca10 comp rx=0 tx=0).stop 2023-10-16T20:05:03.844 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.842+0000 7f3e0d209640 1 -- 172.21.15.88:0/1355414710 shutdown_connections 2023-10-16T20:05:03.844 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.842+0000 7f3e0d209640 1 --2- 172.21.15.88:0/1355414710 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3de80664a0 0x7f3de8068960 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:03.844 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.842+0000 7f3e0d209640 1 --2- 172.21.15.88:0/1355414710 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3e0815d9f0 0x7f3e080c5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:03.845 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.842+0000 7f3e0d209640 1 --2- 172.21.15.88:0/1355414710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e080d26c0 0x7f3e0815e360 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:03.845 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.842+0000 7f3e0d209640 1 --2- 172.21.15.88:0/1355414710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e080d22e0 0x7f3e08147900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:03.845 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.842+0000 7f3e0d209640 1 -- 172.21.15.88:0/1355414710 >> 172.21.15.88:0/1355414710 conn(0x7f3e080715c0 msgr2=0x7f3e081508c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:03.845 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.842+0000 7f3e0d209640 1 -- 172.21.15.88:0/1355414710 shutdown_connections 2023-10-16T20:05:03.845 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.842+0000 7f3e0d209640 1 -- 172.21.15.88:0/1355414710 wait complete. 2023-10-16T20:05:03.862 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:05:03.998 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.998+0000 7f10b1972640 1 Processor -- start 2023-10-16T20:05:03.999 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.998+0000 7f10b1972640 1 -- start start 2023-10-16T20:05:04.000 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.998+0000 7f10b1972640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f10ac15d9f0 0x7f10ac161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:04.000 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.998+0000 7f10b1972640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f10ac0d26c0 0x7f10ac0cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:04.000 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.998+0000 7f10b1972640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10ac0b87e0 con 0x7f10ac0d22e0 2023-10-16T20:05:04.001 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.998+0000 7f10b1972640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10ac15df10 con 0x7f10ac0d26c0 2023-10-16T20:05:04.001 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.998+0000 7f10b1972640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac15e1f0 con 0x7f10ac15d9f0 2023-10-16T20:05:04.001 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.998+0000 7f10ab7fe640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f10ac0d26c0 0x7f10ac0cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:04.001 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.998+0000 7f10ab7fe640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f10ac0d26c0 0x7f10ac0cf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47990/0 (socket says 172.21.15.88:47990) 2023-10-16T20:05:04.001 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.998+0000 7f10ab7fe640 1 -- 172.21.15.88:0/685233058 learned_addr learned my addr 172.21.15.88:0/685233058 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:04.002 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.998+0000 7f10a27fc640 1 --2- 172.21.15.88:0/685233058 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f10ac15d9f0 0x7f10ac161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:04.002 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.998+0000 7f10ab7fe640 1 -- 172.21.15.88:0/685233058 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f10ac15d9f0 msgr2=0x7f10ac161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:04.002 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.998+0000 7f10ab7fe640 1 --2- 172.21.15.88:0/685233058 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f10ac15d9f0 0x7f10ac161450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.002 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.998+0000 7f10ab7fe640 1 -- 172.21.15.88:0/685233058 >> v1:172.21.15.94:6789/0 conn(0x7f10ac0d22e0 legacy=0x7f10ac15d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:04.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:03.998+0000 7f10ab7fe640 1 -- 172.21.15.88:0/685233058 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10ac160970 con 0x7f10ac0d26c0 2023-10-16T20:05:04.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.002+0000 7f10ab7fe640 1 --2- 172.21.15.88:0/685233058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10ac0d26c0 0x7f10ac0cf2e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f10a4009870 tx=0x7f10a401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c3d1df0efff6678 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:04.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.002+0000 7f10aa7fc640 1 -- 172.21.15.88:0/685233058 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a40ab680 con 0x7f10ac0d26c0 2023-10-16T20:05:04.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.002+0000 7f10aa7fc640 1 -- 172.21.15.88:0/685233058 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a40ab7e0 con 0x7f10ac0d26c0 2023-10-16T20:05:04.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.002+0000 7f10aa7fc640 1 -- 172.21.15.88:0/685233058 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a40abb70 con 0x7f10ac0d26c0 2023-10-16T20:05:04.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.002+0000 7f10b1972640 1 -- 172.21.15.88:0/685233058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10ac0d26c0 msgr2=0x7f10ac0cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:04.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.002+0000 7f10b1972640 1 --2- 172.21.15.88:0/685233058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10ac0d26c0 0x7f10ac0cf2e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f10a4009870 tx=0x7f10a401c480 comp rx=0 tx=0).stop 2023-10-16T20:05:04.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.002+0000 7f10b1972640 1 -- 172.21.15.88:0/685233058 shutdown_connections 2023-10-16T20:05:04.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.002+0000 7f10b1972640 1 --2- 172.21.15.88:0/685233058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10ac0d26c0 0x7f10ac0cf2e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.002+0000 7f10b1972640 1 --2- 172.21.15.88:0/685233058 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f10ac15d9f0 0x7f10ac161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.002+0000 7f10b1972640 1 -- 172.21.15.88:0/685233058 >> 172.21.15.88:0/685233058 conn(0x7f10ac0715c0 msgr2=0x7f10ac0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:04.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.002+0000 7f10b1972640 1 -- 172.21.15.88:0/685233058 shutdown_connections 2023-10-16T20:05:04.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.002+0000 7f10b1972640 1 -- 172.21.15.88:0/685233058 wait complete. 2023-10-16T20:05:04.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.006+0000 7f10b1972640 1 Processor -- start 2023-10-16T20:05:04.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.006+0000 7f10b1972640 1 -- start start 2023-10-16T20:05:04.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.006+0000 7f10b1972640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f10ac0d22e0 0x7f10ac1f8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:04.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.006+0000 7f10b1972640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10ac0d26c0 0x7f10ac0719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:04.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.006+0000 7f10b1972640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10ac15d9f0 0x7f10ac071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:04.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.006+0000 7f10b1972640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac1f8590 con 0x7f10ac15d9f0 2023-10-16T20:05:04.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.006+0000 7f10b1972640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac1602d0 con 0x7f10ac0d26c0 2023-10-16T20:05:04.010 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.006+0000 7f10b1972640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac15fa30 con 0x7f10ac0d22e0 2023-10-16T20:05:04.010 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.006+0000 7f10ab7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10ac15d9f0 0x7f10ac071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:04.010 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.006+0000 7f10ab7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10ac15d9f0 0x7f10ac071ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48006/0 (socket says 172.21.15.88:48006) 2023-10-16T20:05:04.010 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.006+0000 7f10ab7fe640 1 -- 172.21.15.88:0/2651493890 learned_addr learned my addr 172.21.15.88:0/2651493890 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:04.010 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.006+0000 7f10a27fc640 1 --2- 172.21.15.88:0/2651493890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10ac0d26c0 0x7f10ac0719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:04.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.006+0000 7f10aaffd640 1 --2- 172.21.15.88:0/2651493890 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f10ac0d22e0 0x7f10ac1f8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:04.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.006+0000 7f10ab7fe640 1 -- 172.21.15.88:0/2651493890 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f10ac0d22e0 msgr2=0x7f10ac1f8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:04.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.006+0000 7f10ab7fe640 1 --2- 172.21.15.88:0/2651493890 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f10ac0d22e0 0x7f10ac1f8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.006+0000 7f10ab7fe640 1 -- 172.21.15.88:0/2651493890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10ac0d26c0 msgr2=0x7f10ac0719a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:04.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.006+0000 7f10ab7fe640 1 --2- 172.21.15.88:0/2651493890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10ac0d26c0 0x7f10ac0719a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.012 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.006+0000 7f10ab7fe640 1 -- 172.21.15.88:0/2651493890 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a40094d0 con 0x7f10ac15d9f0 2023-10-16T20:05:04.012 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.006+0000 7f10ab7fe640 1 --2- 172.21.15.88:0/2651493890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10ac15d9f0 0x7f10ac071ee0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f10a40ad040 tx=0x7f10a401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:04.012 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.006+0000 7f10a8ff9640 1 -- 172.21.15.88:0/2651493890 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a40b4800 con 0x7f10ac15d9f0 2023-10-16T20:05:04.012 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.006+0000 7f10a8ff9640 1 -- 172.21.15.88:0/2651493890 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a40b4960 con 0x7f10ac15d9f0 2023-10-16T20:05:04.012 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.006+0000 7f10a8ff9640 1 -- 172.21.15.88:0/2651493890 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a40b4d10 con 0x7f10ac15d9f0 2023-10-16T20:05:04.013 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.006+0000 7f10b1972640 1 -- 172.21.15.88:0/2651493890 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10ac15fd10 con 0x7f10ac15d9f0 2023-10-16T20:05:04.013 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.006+0000 7f10b1972640 1 -- 172.21.15.88:0/2651493890 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10ac0c7460 con 0x7f10ac15d9f0 2023-10-16T20:05:04.014 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.010+0000 7f10b1972640 1 -- 172.21.15.88:0/2651493890 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10ac04e9c0 con 0x7f10ac15d9f0 2023-10-16T20:05:04.014 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.010+0000 7f10a8ff9640 1 -- 172.21.15.88:0/2651493890 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f10a40b3380 con 0x7f10ac15d9f0 2023-10-16T20:05:04.014 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.010+0000 7f10a8ff9640 1 --2- 172.21.15.88:0/2651493890 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f108c0664d0 0x7f108c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:04.014 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.010+0000 7f10a8ff9640 1 -- 172.21.15.88:0/2651493890 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f10a411ead0 con 0x7f10ac15d9f0 2023-10-16T20:05:04.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.010+0000 7f10aaffd640 1 --2- 172.21.15.88:0/2651493890 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f108c0664d0 0x7f108c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:04.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.010+0000 7f10aaffd640 1 --2- 172.21.15.88:0/2651493890 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f108c0664d0 0x7f108c068990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f109c007fd0 tx=0x7f109c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:04.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.014+0000 7f10a8ff9640 1 -- 172.21.15.88:0/2651493890 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f10a40ebc70 con 0x7f10ac15d9f0 2023-10-16T20:05:04.164 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.162+0000 7f10b1972640 1 -- 172.21.15.88:0/2651493890 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10ac0c7650 con 0x7f10ac15d9f0 2023-10-16T20:05:04.166 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.166+0000 7f10a8ff9640 1 -- 172.21.15.88:0/2651493890 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18274 (secure 0 0 0) 0x7f10a40ef290 con 0x7f10ac15d9f0 2023-10-16T20:05:04.166 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:04.166 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:05:01.835630+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T20:04:54.442487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T20:04:55.766641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T20:04:59.475830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:05:04.168 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.166+0000 7f10b1972640 1 -- 172.21.15.88:0/2651493890 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f108c0664d0 msgr2=0x7f108c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:04.168 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.166+0000 7f10b1972640 1 --2- 172.21.15.88:0/2651493890 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f108c0664d0 0x7f108c068990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f109c007fd0 tx=0x7f109c009040 comp rx=0 tx=0).stop 2023-10-16T20:05:04.168 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.166+0000 7f10b1972640 1 -- 172.21.15.88:0/2651493890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10ac15d9f0 msgr2=0x7f10ac071ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:04.168 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.166+0000 7f10b1972640 1 --2- 172.21.15.88:0/2651493890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10ac15d9f0 0x7f10ac071ee0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f10a40ad040 tx=0x7f10a401acc0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.169 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.166+0000 7f10b1972640 1 -- 172.21.15.88:0/2651493890 shutdown_connections 2023-10-16T20:05:04.169 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.166+0000 7f10b1972640 1 --2- 172.21.15.88:0/2651493890 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f108c0664d0 0x7f108c068990 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.169 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.166+0000 7f10b1972640 1 --2- 172.21.15.88:0/2651493890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10ac15d9f0 0x7f10ac071ee0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.169 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.166+0000 7f10b1972640 1 --2- 172.21.15.88:0/2651493890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10ac0d26c0 0x7f10ac0719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.169 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.166+0000 7f10b1972640 1 --2- 172.21.15.88:0/2651493890 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f10ac0d22e0 0x7f10ac1f8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.170 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.166+0000 7f10b1972640 1 -- 172.21.15.88:0/2651493890 >> 172.21.15.88:0/2651493890 conn(0x7f10ac0715c0 msgr2=0x7f10ac156f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:04.170 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.166+0000 7f10b1972640 1 -- 172.21.15.88:0/2651493890 shutdown_connections 2023-10-16T20:05:04.170 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.166+0000 7f10b1972640 1 -- 172.21.15.88:0/2651493890 wait complete. 2023-10-16T20:05:04.589 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T20:04:56.759719+0000', 'modified': '2023-10-16T20:05:01.835793+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4272}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4272': {'gid': 4272, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.94:6835/555575750', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6833', 'nonce': 555575750}, {'type': 'v1', 'addr': '172.21.15.94:6835', 'nonce': 555575750}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-10-16T20:05:04.590 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T20:05:04.590 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-16T20:05:04.728 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.726+0000 7f5bf20d6640 1 Processor -- start 2023-10-16T20:05:04.729 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.726+0000 7f5bf20d6640 1 -- start start 2023-10-16T20:05:04.730 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.730+0000 7f5bf20d6640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bec0b2a80 0x7f5bec0ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:04.730 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.730+0000 7f5bf20d6640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5bec0cea00 0x7f5bec06bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:04.730 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.730+0000 7f5bf20d6640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bec06b490 con 0x7f5bec15cb80 2023-10-16T20:05:04.730 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.730+0000 7f5bf20d6640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bec0cf180 con 0x7f5bec0cea00 2023-10-16T20:05:04.731 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.730+0000 7f5bf20d6640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec06b700 con 0x7f5bec0b2a80 2023-10-16T20:05:04.731 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.730+0000 7f5bebfff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5bec0cea00 0x7f5bec06bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:04.731 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.730+0000 7f5bebfff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5bec0cea00 0x7f5bec06bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48018/0 (socket says 172.21.15.88:48018) 2023-10-16T20:05:04.731 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.730+0000 7f5bebfff640 1 -- 172.21.15.88:0/1266470592 learned_addr learned my addr 172.21.15.88:0/1266470592 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:04.731 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.730+0000 7f5be3fff640 1 --2- 172.21.15.88:0/1266470592 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bec0b2a80 0x7f5bec0ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:04.732 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.730+0000 7f5bebfff640 1 -- 172.21.15.88:0/1266470592 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bec0b2a80 msgr2=0x7f5bec0ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:04.732 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.730+0000 7f5bebfff640 1 --2- 172.21.15.88:0/1266470592 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bec0b2a80 0x7f5bec0ce4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.733 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.730+0000 7f5bebfff640 1 -- 172.21.15.88:0/1266470592 >> v1:172.21.15.94:6789/0 conn(0x7f5bec15cb80 legacy=0x7f5bec0b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:04.733 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.730+0000 7f5bebfff640 1 -- 172.21.15.88:0/1266470592 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bec06b9e0 con 0x7f5bec0cea00 2023-10-16T20:05:04.733 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.734+0000 7f5bebfff640 1 --2- 172.21.15.88:0/1266470592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bec0cea00 0x7f5bec06bf00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5be4009e50 tx=0x7f5be401c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4add79fcf9a4cd6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:04.734 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.734+0000 7f5beaffd640 1 -- 172.21.15.88:0/1266470592 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be40a57a0 con 0x7f5bec0cea00 2023-10-16T20:05:04.734 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.734+0000 7f5beaffd640 1 -- 172.21.15.88:0/1266470592 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be40a5900 con 0x7f5bec0cea00 2023-10-16T20:05:04.734 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.734+0000 7f5beaffd640 1 -- 172.21.15.88:0/1266470592 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be40a5c90 con 0x7f5bec0cea00 2023-10-16T20:05:04.735 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.734+0000 7f5bf20d6640 1 -- 172.21.15.88:0/1266470592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bec0cea00 msgr2=0x7f5bec06bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:04.735 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.734+0000 7f5bf20d6640 1 --2- 172.21.15.88:0/1266470592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bec0cea00 0x7f5bec06bf00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5be4009e50 tx=0x7f5be401c9b0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.736 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.734+0000 7f5bf20d6640 1 -- 172.21.15.88:0/1266470592 shutdown_connections 2023-10-16T20:05:04.736 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.734+0000 7f5bf20d6640 1 --2- 172.21.15.88:0/1266470592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bec0cea00 0x7f5bec06bf00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.736 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.734+0000 7f5bf20d6640 1 --2- 172.21.15.88:0/1266470592 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bec0b2a80 0x7f5bec0ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.736 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.734+0000 7f5bf20d6640 1 -- 172.21.15.88:0/1266470592 >> 172.21.15.88:0/1266470592 conn(0x7f5bec0cd370 msgr2=0x7f5bec0cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:04.736 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.734+0000 7f5bf20d6640 1 -- 172.21.15.88:0/1266470592 shutdown_connections 2023-10-16T20:05:04.737 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.734+0000 7f5bf20d6640 1 -- 172.21.15.88:0/1266470592 wait complete. 2023-10-16T20:05:04.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.734+0000 7f5bf20d6640 1 Processor -- start 2023-10-16T20:05:04.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.734+0000 7f5bf20d6640 1 -- start start 2023-10-16T20:05:04.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.738+0000 7f5bf20d6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bec0b2a80 0x7f5bec1556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:04.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.738+0000 7f5bf20d6640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bec0cea00 0x7f5bec0c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:04.740 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.738+0000 7f5bf20d6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bec15cb80 0x7f5bec0c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:04.740 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.738+0000 7f5bf20d6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec155be0 con 0x7f5bec15cb80 2023-10-16T20:05:04.740 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.738+0000 7f5bf20d6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec0c5c10 con 0x7f5bec0b2a80 2023-10-16T20:05:04.740 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.738+0000 7f5bf20d6640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec0c4630 con 0x7f5bec0cea00 2023-10-16T20:05:04.740 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.738+0000 7f5bebfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bec15cb80 0x7f5bec0c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:04.740 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.738+0000 7f5bebfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bec15cb80 0x7f5bec0c9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48034/0 (socket says 172.21.15.88:48034) 2023-10-16T20:05:04.741 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.738+0000 7f5bebfff640 1 -- 172.21.15.88:0/1791482059 learned_addr learned my addr 172.21.15.88:0/1791482059 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:04.741 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.738+0000 7f5beb7fe640 1 --2- 172.21.15.88:0/1791482059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bec0b2a80 0x7f5bec1556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:04.741 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.738+0000 7f5be3fff640 1 --2- 172.21.15.88:0/1791482059 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bec0cea00 0x7f5bec0c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:04.741 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.738+0000 7f5bebfff640 1 -- 172.21.15.88:0/1791482059 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bec0cea00 msgr2=0x7f5bec0c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:04.741 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.738+0000 7f5bebfff640 1 --2- 172.21.15.88:0/1791482059 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bec0cea00 0x7f5bec0c56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.742 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.738+0000 7f5bebfff640 1 -- 172.21.15.88:0/1791482059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bec0b2a80 msgr2=0x7f5bec1556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:04.742 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.738+0000 7f5bebfff640 1 --2- 172.21.15.88:0/1791482059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bec0b2a80 0x7f5bec1556a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.742 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.738+0000 7f5bebfff640 1 -- 172.21.15.88:0/1791482059 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be4009b30 con 0x7f5bec15cb80 2023-10-16T20:05:04.742 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.738+0000 7f5be3fff640 1 --2- 172.21.15.88:0/1791482059 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bec0cea00 0x7f5bec0c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:04.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.738+0000 7f5bebfff640 1 --2- 172.21.15.88:0/1791482059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bec15cb80 0x7f5bec0c9fc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5be401a640 tx=0x7f5be40197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:04.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.738+0000 7f5be8ff9640 1 -- 172.21.15.88:0/1791482059 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be40b8670 con 0x7f5bec15cb80 2023-10-16T20:05:04.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.738+0000 7f5be8ff9640 1 -- 172.21.15.88:0/1791482059 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be40b87f0 con 0x7f5bec15cb80 2023-10-16T20:05:04.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.738+0000 7f5be8ff9640 1 -- 172.21.15.88:0/1791482059 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be40b8ba0 con 0x7f5bec15cb80 2023-10-16T20:05:04.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.738+0000 7f5bf20d6640 1 -- 172.21.15.88:0/1791482059 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bec0c4910 con 0x7f5bec15cb80 2023-10-16T20:05:04.744 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.738+0000 7f5bf20d6640 1 -- 172.21.15.88:0/1791482059 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bec0c4ed0 con 0x7f5bec15cb80 2023-10-16T20:05:04.747 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.738+0000 7f5be8ff9640 1 -- 172.21.15.88:0/1791482059 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5be40b8d00 con 0x7f5bec15cb80 2023-10-16T20:05:04.747 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.742+0000 7f5be8ff9640 1 --2- 172.21.15.88:0/1791482059 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5bcc0664a0 0x7f5bcc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:04.747 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.742+0000 7f5be8ff9640 1 -- 172.21.15.88:0/1791482059 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5be411f7d0 con 0x7f5bec15cb80 2023-10-16T20:05:04.747 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.742+0000 7f5beb7fe640 1 --2- 172.21.15.88:0/1791482059 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5bcc0664a0 0x7f5bcc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:04.748 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.742+0000 7f5bf5451000 1 Processor -- start 2023-10-16T20:05:04.748 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.742+0000 7f5bf5451000 1 -- start start 2023-10-16T20:05:04.748 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.742+0000 7f5beb7fe640 1 --2- 172.21.15.88:0/1791482059 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5bcc0664a0 0x7f5bcc068960 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc007fd0 tx=0x7f5bdc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:04.748 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.742+0000 7f5bf5451000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556430f378a0 0x556430f37c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:04.748 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.742+0000 7f5bf5451000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x556430f381c0 0x556430eed710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:04.749 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.742+0000 7f5bf5451000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556430eedc50 0x556430eee030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:04.749 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.742+0000 7f5bf5451000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x556430ce2cc0 con 0x556430eedc50 2023-10-16T20:05:04.749 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.742+0000 7f5bf5451000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556430d45920 con 0x556430f378a0 2023-10-16T20:05:04.749 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.742+0000 7f5bf5451000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x556430d42e50 con 0x556430f381c0 2023-10-16T20:05:04.749 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.742+0000 7f5bebfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556430eedc50 0x556430eee030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:04.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.742+0000 7f5beb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556430f378a0 0x556430f37c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:04.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.742+0000 7f5bebfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556430eedc50 0x556430eee030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48036/0 (socket says 172.21.15.88:48036) 2023-10-16T20:05:04.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.742+0000 7f5be3fff640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x556430f381c0 0x556430eed710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:04.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.742+0000 7f5bebfff640 1 -- 172.21.15.88:0/3498830859 learned_addr learned my addr 172.21.15.88:0/3498830859 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:04.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.742+0000 7f5bebfff640 1 -- 172.21.15.88:0/3498830859 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x556430f381c0 msgr2=0x556430eed710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:04.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.742+0000 7f5bebfff640 1 --2- 172.21.15.88:0/3498830859 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x556430f381c0 0x556430eed710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.742+0000 7f5bebfff640 1 -- 172.21.15.88:0/3498830859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556430f378a0 msgr2=0x556430f37c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:04.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.742+0000 7f5bebfff640 1 --2- 172.21.15.88:0/3498830859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556430f378a0 0x556430f37c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.742+0000 7f5bebfff640 1 -- 172.21.15.88:0/3498830859 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556430a37890 con 0x556430eedc50 2023-10-16T20:05:04.752 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.742+0000 7f5bebfff640 1 --2- 172.21.15.88:0/3498830859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556430eedc50 0x556430eee030 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5be40f06f0 tx=0x7f5be401c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd69b107165b84f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:04.752 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.742+0000 7f5be17fa640 1 -- 172.21.15.88:0/3498830859 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be40f8a60 con 0x556430eedc50 2023-10-16T20:05:04.752 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.742+0000 7f5be17fa640 1 -- 172.21.15.88:0/3498830859 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be40eeb00 con 0x556430eedc50 2023-10-16T20:05:04.752 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5be17fa640 1 -- 172.21.15.88:0/3498830859 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be40f09c0 con 0x556430eedc50 2023-10-16T20:05:04.752 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5bf5451000 1 -- 172.21.15.88:0/3498830859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556430eedc50 msgr2=0x556430eee030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:04.753 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5bf5451000 1 --2- 172.21.15.88:0/3498830859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556430eedc50 0x556430eee030 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5be40f06f0 tx=0x7f5be401c9b0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.753 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5bf5451000 1 -- 172.21.15.88:0/3498830859 shutdown_connections 2023-10-16T20:05:04.753 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5bf5451000 1 --2- 172.21.15.88:0/3498830859 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x556430f381c0 0x556430eed710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.753 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5bf5451000 1 --2- 172.21.15.88:0/3498830859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556430eedc50 0x556430eee030 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.753 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5bf5451000 1 --2- 172.21.15.88:0/3498830859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556430f378a0 0x556430f37c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.754 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5bf5451000 1 -- 172.21.15.88:0/3498830859 >> 172.21.15.88:0/3498830859 conn(0x556430ee4fc0 msgr2=0x556430eebff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:04.754 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5bf5451000 1 -- 172.21.15.88:0/3498830859 shutdown_connections 2023-10-16T20:05:04.754 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5bf5451000 1 -- 172.21.15.88:0/3498830859 wait complete. 2023-10-16T20:05:04.754 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5bf5451000 1 -- start start 2023-10-16T20:05:04.754 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5bf5451000 1 Processor -- start 2023-10-16T20:05:04.757 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5bf5451000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556430eedc50 0x556430ef8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:04.758 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5bf5451000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556430f378a0 0x556430ef8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:04.758 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5bf5451000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x556430f381c0 0x556430ef92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:04.758 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5bf5451000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x556430d3e740 con 0x556430f378a0 2023-10-16T20:05:04.758 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5bf5451000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556430cfdd10 con 0x556430eedc50 2023-10-16T20:05:04.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5bf5451000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x556430cfd9b0 con 0x556430f381c0 2023-10-16T20:05:04.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5bf5451000 10 client.0 _ll_register_callbacks cb 0x556430ed9390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T20:05:04.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5be3fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556430f378a0 0x556430ef8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:04.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5be3fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556430f378a0 0x556430ef8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48042/0 (socket says 172.21.15.88:48042) 2023-10-16T20:05:04.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5be3fff640 1 -- 172.21.15.88:0/925425271 learned_addr learned my addr 172.21.15.88:0/925425271 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:04.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5beb7fe640 1 --2- 172.21.15.88:0/925425271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556430eedc50 0x556430ef8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:04.760 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5bebfff640 1 --2- 172.21.15.88:0/925425271 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x556430f381c0 0x556430ef92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:04.760 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5be3fff640 1 -- 172.21.15.88:0/925425271 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x556430f381c0 msgr2=0x556430ef92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:04.760 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5be3fff640 1 --2- 172.21.15.88:0/925425271 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x556430f381c0 0x556430ef92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.760 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5be3fff640 1 -- 172.21.15.88:0/925425271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556430eedc50 msgr2=0x556430ef8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:04.760 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5be3fff640 1 --2- 172.21.15.88:0/925425271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556430eedc50 0x556430ef8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.760 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5be3fff640 1 -- 172.21.15.88:0/925425271 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bec0c4ed0 con 0x556430f378a0 2023-10-16T20:05:04.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5beb7fe640 1 --2- 172.21.15.88:0/925425271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556430eedc50 0x556430ef8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:04.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5be3fff640 1 --2- 172.21.15.88:0/925425271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556430f378a0 0x556430ef8da0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40145d0 tx=0x7f5bd4014a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254a99a1f42ef861 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:04.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5bca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2023-10-16T20:05:04.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5bca7fc640 1 -- 172.21.15.88:0/925425271 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4004500 con 0x556430f378a0 2023-10-16T20:05:04.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5bf5451000 1 -- 172.21.15.88:0/925425271 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556430cf2370 con 0x556430f378a0 2023-10-16T20:05:04.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.746+0000 7f5bca7fc640 1 -- 172.21.15.88:0/925425271 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5bd401a030 con 0x556430f378a0 2023-10-16T20:05:04.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.750+0000 7f5bca7fc640 1 -- 172.21.15.88:0/925425271 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bd4003620 con 0x556430f378a0 2023-10-16T20:05:04.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.750+0000 7f5bf5451000 10 client.4413 fetch_fsmap learned FSMap version 7 2023-10-16T20:05:04.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.750+0000 7f5bf5451000 1 -- 172.21.15.88:0/925425271 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x556430d00460 con 0x556430f378a0 2023-10-16T20:05:04.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.750+0000 7f5bca7fc640 1 -- 172.21.15.88:0/925425271 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556430cf2370 con 0x556430f378a0 2023-10-16T20:05:04.763 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.750+0000 7f5bca7fc640 1 -- 172.21.15.88:0/925425271 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f5bd4003780 con 0x556430f378a0 2023-10-16T20:05:04.763 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.750+0000 7f5bf5451000 10 client.4413 fetch_fsmap finished waiting for FSMap version 7 2023-10-16T20:05:04.763 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.750+0000 7f5bf5451000 10 client.4413 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-16T20:05:04.763 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.750+0000 7f5bf5451000 20 client.4413 populate_metadata read hostname 'smithi088' 2023-10-16T20:05:04.763 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.750+0000 7f5bf5451000 1 --2- 172.21.15.88:0/925425271 >> [v2:172.21.15.94:6833/555575750,v1:172.21.15.94:6835/555575750] conn(0x556430f46930 0x556430f48d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:04.764 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.750+0000 7f5bf5451000 4 client.4413 mds_command: new command op to 4272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T20:05:04.764 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.750+0000 7f5bf5451000 1 -- 172.21.15.88:0/925425271 --> [v2:172.21.15.94:6833/555575750,v1:172.21.15.94:6835/555575750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556430cd82d0 con 0x556430f46930 2023-10-16T20:05:04.764 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.750+0000 7f5beb7fe640 1 --2- 172.21.15.88:0/925425271 >> [v2:172.21.15.94:6833/555575750,v1:172.21.15.94:6835/555575750] conn(0x556430f46930 0x556430f48d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:04.764 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.750+0000 7f5beb7fe640 1 --2- 172.21.15.88:0/925425271 >> [v2:172.21.15.94:6833/555575750,v1:172.21.15.94:6835/555575750] conn(0x556430f46930 0x556430f48d20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4f2fbe025243c24 server_cookie=e442007d4157d767 in_seq=0 out_seq=0 2023-10-16T20:05:04.764 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.750+0000 7f5bca7fc640 10 client.4413 ms_handle_connect on v2:172.21.15.94:6833/555575750 2023-10-16T20:05:04.764 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.750+0000 7f5bca7fc640 1 -- 172.21.15.88:0/925425271 <== mds.0 v2:172.21.15.94:6833/555575750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556430cd82d0 con 0x556430f46930 2023-10-16T20:05:04.765 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.750+0000 7f5bca7fc640 10 client.4413 handle_command_reply: tid=0 2023-10-16T20:05:04.765 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.750+0000 7f5bf5451000 1 client.4413 shutdown 2023-10-16T20:05:04.765 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.750+0000 7f5bf5451000 1 -- 172.21.15.88:0/925425271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556430f378a0 msgr2=0x556430ef8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:04.765 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.750+0000 7f5bf5451000 1 --2- 172.21.15.88:0/925425271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556430f378a0 0x556430ef8da0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40145d0 tx=0x7f5bd4014a90 comp rx=0 tx=0).stop 2023-10-16T20:05:04.766 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.750+0000 7f5bf5451000 1 -- 172.21.15.88:0/925425271 shutdown_connections 2023-10-16T20:05:04.766 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.750+0000 7f5bf5451000 1 --2- 172.21.15.88:0/925425271 >> [v2:172.21.15.94:6833/555575750,v1:172.21.15.94:6835/555575750] conn(0x556430f46930 0x556430f48d20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.766 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.750+0000 7f5bf5451000 1 --2- 172.21.15.88:0/925425271 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x556430f381c0 0x556430ef92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.766 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.750+0000 7f5bf5451000 1 --2- 172.21.15.88:0/925425271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556430f378a0 0x556430ef8da0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.766 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.750+0000 7f5bca7fc640 0 client.4413 ms_handle_reset on v2:172.21.15.94:6833/555575750 2023-10-16T20:05:04.767 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.750+0000 7f5bf5451000 1 --2- 172.21.15.88:0/925425271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556430eedc50 0x556430ef8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.767 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.750+0000 7f5bf5451000 1 -- 172.21.15.88:0/925425271 >> 172.21.15.88:0/925425271 conn(0x556430ee4fc0 msgr2=0x556430eec6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:04.767 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.750+0000 7f5bf5451000 1 -- 172.21.15.88:0/925425271 shutdown_connections 2023-10-16T20:05:04.767 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.750+0000 7f5bf5451000 1 -- 172.21.15.88:0/925425271 wait complete. 2023-10-16T20:05:04.767 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.750+0000 7f5bf5451000 20 client.4413 trim_cache size 0 max 16384 2023-10-16T20:05:04.769 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.766+0000 7f5bf5451000 1 Processor -- start 2023-10-16T20:05:04.769 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.766+0000 7f5bf5451000 1 -- start start 2023-10-16T20:05:04.769 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.766+0000 7f5bf5451000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556430eedc50 0x556430f92130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:04.770 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.766+0000 7f5bf5451000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x556430f378a0 0x556430f92670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:04.770 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.766+0000 7f5bf5451000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556430f381c0 0x556430f92bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:04.770 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.766+0000 7f5bf5451000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564309f8940 con 0x556430eedc50 2023-10-16T20:05:04.770 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.766+0000 7f5bf5451000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556430cec6c0 con 0x556430f381c0 2023-10-16T20:05:04.770 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.766+0000 7f5bf5451000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x556430d14a10 con 0x556430f378a0 2023-10-16T20:05:04.771 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.766+0000 7f5beb7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556430eedc50 0x556430f92130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:04.771 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.766+0000 7f5beb7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556430eedc50 0x556430f92130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48056/0 (socket says 172.21.15.88:48056) 2023-10-16T20:05:04.771 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.766+0000 7f5beb7fe640 1 -- 172.21.15.88:0/3184517320 learned_addr learned my addr 172.21.15.88:0/3184517320 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:04.771 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.766+0000 7f5beb7fe640 1 -- 172.21.15.88:0/3184517320 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x556430f378a0 msgr2=0x556430f92670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:04.771 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.766+0000 7f5beb7fe640 1 --2- 172.21.15.88:0/3184517320 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x556430f378a0 0x556430f92670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.772 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.766+0000 7f5beb7fe640 1 -- 172.21.15.88:0/3184517320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556430f381c0 msgr2=0x556430f92bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T20:05:04.772 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.766+0000 7f5beb7fe640 1 --2- 172.21.15.88:0/3184517320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556430f381c0 0x556430f92bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.772 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.766+0000 7f5beb7fe640 1 -- 172.21.15.88:0/3184517320 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556430ce6760 con 0x556430eedc50 2023-10-16T20:05:04.772 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.766+0000 7f5beb7fe640 1 --2- 172.21.15.88:0/3184517320 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556430eedc50 0x556430f92130 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x556430d3e1c0 tx=0x7f5bdc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f597f751c18bcbf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:04.772 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.766+0000 7f5be1ffb640 1 -- 172.21.15.88:0/3184517320 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc028070 con 0x556430eedc50 2023-10-16T20:05:04.773 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5be1ffb640 1 -- 172.21.15.88:0/3184517320 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bdc023960 con 0x556430eedc50 2023-10-16T20:05:04.773 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5be1ffb640 1 -- 172.21.15.88:0/3184517320 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc023cd0 con 0x556430eedc50 2023-10-16T20:05:04.777 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5bf5451000 1 -- 172.21.15.88:0/3184517320 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556430eedc50 msgr2=0x556430f92130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:04.777 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5bf5451000 1 --2- 172.21.15.88:0/3184517320 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556430eedc50 0x556430f92130 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x556430d3e1c0 tx=0x7f5bdc012000 comp rx=0 tx=0).stop 2023-10-16T20:05:04.777 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5bf5451000 1 -- 172.21.15.88:0/3184517320 shutdown_connections 2023-10-16T20:05:04.777 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5bf5451000 1 --2- 172.21.15.88:0/3184517320 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x556430f378a0 0x556430f92670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.778 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5bf5451000 1 --2- 172.21.15.88:0/3184517320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556430f381c0 0x556430f92bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.778 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5bf5451000 1 --2- 172.21.15.88:0/3184517320 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556430eedc50 0x556430f92130 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.778 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5bf5451000 1 -- 172.21.15.88:0/3184517320 >> 172.21.15.88:0/3184517320 conn(0x556430ee4fc0 msgr2=0x556430f49c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:04.778 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5bf5451000 1 -- 172.21.15.88:0/3184517320 shutdown_connections 2023-10-16T20:05:04.779 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5bf5451000 1 -- 172.21.15.88:0/3184517320 wait complete. 2023-10-16T20:05:04.779 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5bf5451000 1 -- start start 2023-10-16T20:05:04.779 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5bf5451000 1 Processor -- start 2023-10-16T20:05:04.779 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5bf5451000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x556430eedc50 0x556430f36840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:04.779 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5bf5451000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556430f378a0 0x556430f520e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:04.779 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5bf5451000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556430f381c0 0x556430f52620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:04.780 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5bf5451000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x556430ef8a90 con 0x556430f378a0 2023-10-16T20:05:04.780 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5bf5451000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556430d38c20 con 0x556430f381c0 2023-10-16T20:05:04.780 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5bf5451000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x556430d3bf50 con 0x556430eedc50 2023-10-16T20:05:04.780 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5bf5451000 10 client.0 _ll_register_callbacks cb 0x7f5be40ecaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T20:05:04.780 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5be3fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556430f378a0 0x556430f520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:04.781 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5be3fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556430f378a0 0x556430f520e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48066/0 (socket says 172.21.15.88:48066) 2023-10-16T20:05:04.781 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5be3fff640 1 -- 172.21.15.88:0/163415567 learned_addr learned my addr 172.21.15.88:0/163415567 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:04.781 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5bebfff640 1 --2- 172.21.15.88:0/163415567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556430f381c0 0x556430f52620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:04.781 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5beb7fe640 1 --2- 172.21.15.88:0/163415567 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x556430eedc50 0x556430f36840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:04.781 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5be3fff640 1 -- 172.21.15.88:0/163415567 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x556430eedc50 msgr2=0x556430f36840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:04.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5be3fff640 1 --2- 172.21.15.88:0/163415567 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x556430eedc50 0x556430f36840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5be3fff640 1 -- 172.21.15.88:0/163415567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556430f381c0 msgr2=0x556430f52620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:04.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5be3fff640 1 --2- 172.21.15.88:0/163415567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556430f381c0 0x556430f52620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5be3fff640 1 -- 172.21.15.88:0/163415567 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bdc019040 con 0x556430f378a0 2023-10-16T20:05:04.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5be3fff640 1 --2- 172.21.15.88:0/163415567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556430f378a0 0x556430f520e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4014ad0 tx=0x7f5bd4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f19262f5bcfb9c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:04.783 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5bca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2023-10-16T20:05:04.783 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5bca7fc640 1 -- 172.21.15.88:0/163415567 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4003820 con 0x556430f378a0 2023-10-16T20:05:04.783 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5bca7fc640 1 -- 172.21.15.88:0/163415567 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5bd401a070 con 0x556430f378a0 2023-10-16T20:05:04.783 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5bf5451000 1 -- 172.21.15.88:0/163415567 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556430ef15a0 con 0x556430f378a0 2023-10-16T20:05:04.783 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5bca7fc640 1 -- 172.21.15.88:0/163415567 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bd4004510 con 0x556430f378a0 2023-10-16T20:05:04.784 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5bf5451000 10 client.4419 fetch_fsmap learned FSMap version 7 2023-10-16T20:05:04.784 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.770+0000 7f5bf5451000 1 -- 172.21.15.88:0/163415567 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x556430a37cc0 con 0x556430f378a0 2023-10-16T20:05:04.784 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bca7fc640 1 -- 172.21.15.88:0/163415567 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556430ef15a0 con 0x556430f378a0 2023-10-16T20:05:04.784 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bca7fc640 1 -- 172.21.15.88:0/163415567 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f5bd4032350 con 0x556430f378a0 2023-10-16T20:05:04.784 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bf5451000 10 client.4419 fetch_fsmap finished waiting for FSMap version 7 2023-10-16T20:05:04.785 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bf5451000 10 client.4419 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-16T20:05:04.785 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bf5451000 20 client.4419 populate_metadata read hostname 'smithi088' 2023-10-16T20:05:04.785 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bf5451000 1 --2- 172.21.15.88:0/163415567 >> [v2:172.21.15.94:6833/555575750,v1:172.21.15.94:6835/555575750] conn(0x556430f963f0 0x556430f967d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:04.785 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bf5451000 4 client.4419 mds_command: new command op to 4272 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T20:05:04.785 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bf5451000 1 -- 172.21.15.88:0/163415567 --> [v2:172.21.15.94:6833/555575750,v1:172.21.15.94:6835/555575750] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556430cd66f0 con 0x556430f963f0 2023-10-16T20:05:04.786 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5beb7fe640 1 --2- 172.21.15.88:0/163415567 >> [v2:172.21.15.94:6833/555575750,v1:172.21.15.94:6835/555575750] conn(0x556430f963f0 0x556430f967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:04.786 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bca7fc640 10 client.4419 ms_handle_connect on v2:172.21.15.94:6833/555575750 2023-10-16T20:05:04.786 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5beb7fe640 1 --2- 172.21.15.88:0/163415567 >> [v2:172.21.15.94:6833/555575750,v1:172.21.15.94:6835/555575750] conn(0x556430f963f0 0x556430f967d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d21430850ead7d6c server_cookie=e0ca799896c4eef in_seq=0 out_seq=0 2023-10-16T20:05:04.786 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bca7fc640 1 -- 172.21.15.88:0/163415567 <== mds.0 v2:172.21.15.94:6833/555575750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x556430cd66f0 con 0x556430f963f0 2023-10-16T20:05:04.786 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bca7fc640 10 client.4419 handle_command_reply: tid=0 2023-10-16T20:05:04.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bf5451000 1 client.4419 shutdown 2023-10-16T20:05:04.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bf5451000 1 -- 172.21.15.88:0/163415567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556430f378a0 msgr2=0x556430f520e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:04.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bf5451000 1 --2- 172.21.15.88:0/163415567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556430f378a0 0x556430f520e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4014ad0 tx=0x7f5bd4004740 comp rx=0 tx=0).stop 2023-10-16T20:05:04.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bf5451000 1 -- 172.21.15.88:0/163415567 shutdown_connections 2023-10-16T20:05:04.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bf5451000 1 --2- 172.21.15.88:0/163415567 >> [v2:172.21.15.94:6833/555575750,v1:172.21.15.94:6835/555575750] conn(0x556430f963f0 0x556430f967d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.788 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bf5451000 1 --2- 172.21.15.88:0/163415567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556430f381c0 0x556430f52620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.788 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bf5451000 1 --2- 172.21.15.88:0/163415567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556430f378a0 0x556430f520e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.788 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bca7fc640 0 client.4419 ms_handle_reset on v2:172.21.15.94:6833/555575750 2023-10-16T20:05:04.788 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bf5451000 1 --2- 172.21.15.88:0/163415567 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x556430eedc50 0x556430f36840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.788 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bf5451000 1 -- 172.21.15.88:0/163415567 >> 172.21.15.88:0/163415567 conn(0x556430ee4fc0 msgr2=0x556430f4a330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:04.789 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bf5451000 1 -- 172.21.15.88:0/163415567 shutdown_connections 2023-10-16T20:05:04.789 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bf5451000 1 -- 172.21.15.88:0/163415567 wait complete. 2023-10-16T20:05:04.789 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bf5451000 20 client.4419 trim_cache size 0 max 16384 2023-10-16T20:05:04.789 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2023-10-16T20:05:04.790 INFO:teuthology.orchestra.run.smithi088.stdout: "cluster_fsid": "d11187c0-3dd8-4c92-8d13-6ec536230748", 2023-10-16T20:05:04.790 INFO:teuthology.orchestra.run.smithi088.stdout: "whoami": 0, 2023-10-16T20:05:04.790 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 4272, 2023-10-16T20:05:04.790 INFO:teuthology.orchestra.run.smithi088.stdout: "want_state": "up:active", 2023-10-16T20:05:04.790 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "up:active", 2023-10-16T20:05:04.791 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "cephfs", 2023-10-16T20:05:04.791 INFO:teuthology.orchestra.run.smithi088.stdout: "rank_uptime": 7.3661608879999996, 2023-10-16T20:05:04.791 INFO:teuthology.orchestra.run.smithi088.stdout: "mdsmap_epoch": 7, 2023-10-16T20:05:04.791 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch": 27, 2023-10-16T20:05:04.791 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch_barrier": 26, 2023-10-16T20:05:04.792 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 14.985389676 2023-10-16T20:05:04.792 INFO:teuthology.orchestra.run.smithi088.stdout:} 2023-10-16T20:05:04.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bf20d6640 1 -- 172.21.15.88:0/1791482059 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5bcc0664a0 msgr2=0x7f5bcc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:04.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bf20d6640 1 --2- 172.21.15.88:0/1791482059 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5bcc0664a0 0x7f5bcc068960 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc007fd0 tx=0x7f5bdc009040 comp rx=0 tx=0).stop 2023-10-16T20:05:04.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bf20d6640 1 -- 172.21.15.88:0/1791482059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bec15cb80 msgr2=0x7f5bec0c9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:04.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bf20d6640 1 --2- 172.21.15.88:0/1791482059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bec15cb80 0x7f5bec0c9fc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5be401a640 tx=0x7f5be40197b0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bf20d6640 1 -- 172.21.15.88:0/1791482059 shutdown_connections 2023-10-16T20:05:04.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bf20d6640 1 --2- 172.21.15.88:0/1791482059 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5bcc0664a0 0x7f5bcc068960 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bf20d6640 1 --2- 172.21.15.88:0/1791482059 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bec0cea00 0x7f5bec0c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bf20d6640 1 --2- 172.21.15.88:0/1791482059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bec15cb80 0x7f5bec0c9fc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bf20d6640 1 --2- 172.21.15.88:0/1791482059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bec0b2a80 0x7f5bec1556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bf20d6640 1 -- 172.21.15.88:0/1791482059 >> 172.21.15.88:0/1791482059 conn(0x7f5bec0cd370 msgr2=0x7f5bec0b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:04.795 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bf20d6640 1 -- 172.21.15.88:0/1791482059 shutdown_connections 2023-10-16T20:05:04.795 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.774+0000 7f5bf20d6640 1 -- 172.21.15.88:0/1791482059 wait complete. 2023-10-16T20:05:04.796 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-16T20:05:04.797 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-10-16T20:05:04.797 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T20:05:04.935 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.934+0000 7fb19ec79640 1 Processor -- start 2023-10-16T20:05:04.935 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.934+0000 7fb19ec79640 1 -- start start 2023-10-16T20:05:04.937 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.934+0000 7fb19ec79640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb19815cb80 0x7fb1980b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:04.937 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.934+0000 7fb19ec79640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb1980b28b0 0x7fb1980ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:04.937 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.934+0000 7fb19ec79640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb19806b490 con 0x7fb1980cea00 2023-10-16T20:05:04.937 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.934+0000 7fb19ec79640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1980b2d20 con 0x7fb19815cb80 2023-10-16T20:05:04.938 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.934+0000 7fb19ec79640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1980cf180 con 0x7fb1980b28b0 2023-10-16T20:05:04.938 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.934+0000 7fb19ca15640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb19815cb80 0x7fb1980b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:04.938 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.934+0000 7fb19ca15640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb19815cb80 0x7fb1980b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48082/0 (socket says 172.21.15.88:48082) 2023-10-16T20:05:04.938 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.934+0000 7fb19ca15640 1 -- 172.21.15.88:0/139206317 learned_addr learned my addr 172.21.15.88:0/139206317 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:04.938 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.934+0000 7fb197fff640 1 --2- 172.21.15.88:0/139206317 >> v2:172.21.15.94:3300/0 conn(0x7fb1980b28b0 0x7fb1980ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:04.939 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.938+0000 7fb19ca15640 1 -- 172.21.15.88:0/139206317 >> v2:172.21.15.94:3300/0 conn(0x7fb1980b28b0 msgr2=0x7fb1980ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:04.939 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.938+0000 7fb19ca15640 1 --2- 172.21.15.88:0/139206317 >> v2:172.21.15.94:3300/0 conn(0x7fb1980b28b0 0x7fb1980ce4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.939 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.938+0000 7fb19ca15640 1 -- 172.21.15.88:0/139206317 >> v1:172.21.15.88:6789/0 conn(0x7fb1980cea00 legacy=0x7fb19806ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:04.939 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.938+0000 7fb19ca15640 1 -- 172.21.15.88:0/139206317 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb19806b700 con 0x7fb19815cb80 2023-10-16T20:05:04.939 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.938+0000 7fb1977fe640 1 -- 172.21.15.88:0/139206317 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3934854401 0 0) 0x7fb19806b490 con 0x7fb1980cea00 2023-10-16T20:05:04.940 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.938+0000 7fb19ca15640 1 --2- 172.21.15.88:0/139206317 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb19815cb80 0x7fb1980b2370 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb1980c4190 tx=0x7fb18c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c77af0df92d22e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:04.941 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.938+0000 7fb1977fe640 1 -- 172.21.15.88:0/139206317 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c0ad680 con 0x7fb19815cb80 2023-10-16T20:05:04.941 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.938+0000 7fb1977fe640 1 -- 172.21.15.88:0/139206317 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb18c0ad7e0 con 0x7fb19815cb80 2023-10-16T20:05:04.942 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.938+0000 7fb1977fe640 1 -- 172.21.15.88:0/139206317 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c0adb70 con 0x7fb19815cb80 2023-10-16T20:05:04.942 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.942+0000 7fb19ec79640 1 -- 172.21.15.88:0/139206317 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb19815cb80 msgr2=0x7fb1980b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:04.942 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.942+0000 7fb19ec79640 1 --2- 172.21.15.88:0/139206317 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb19815cb80 0x7fb1980b2370 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb1980c4190 tx=0x7fb18c01c4d0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.942 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.942+0000 7fb19ec79640 1 -- 172.21.15.88:0/139206317 shutdown_connections 2023-10-16T20:05:04.942 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.942+0000 7fb19ec79640 1 --2- 172.21.15.88:0/139206317 >> v2:172.21.15.94:3300/0 conn(0x7fb1980b28b0 0x7fb1980ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.943 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.942+0000 7fb19ec79640 1 --2- 172.21.15.88:0/139206317 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb19815cb80 0x7fb1980b2370 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.943 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.942+0000 7fb19ec79640 1 -- 172.21.15.88:0/139206317 >> 172.21.15.88:0/139206317 conn(0x7fb1980cd370 msgr2=0x7fb1980cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:04.943 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.942+0000 7fb19ec79640 1 -- 172.21.15.88:0/139206317 shutdown_connections 2023-10-16T20:05:04.943 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.942+0000 7fb19ec79640 1 -- 172.21.15.88:0/139206317 wait complete. 2023-10-16T20:05:04.945 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.942+0000 7fb19ec79640 1 Processor -- start 2023-10-16T20:05:04.945 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.942+0000 7fb19ec79640 1 -- start start 2023-10-16T20:05:04.946 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.942+0000 7fb19ec79640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1980b28b0 0x7fb19814ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:04.946 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.942+0000 7fb19ec79640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1980cea00 0x7fb19814ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:04.946 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.942+0000 7fb19ec79640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb19815cb80 0x7fb19814df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:04.946 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.942+0000 7fb19ec79640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19815e630 con 0x7fb1980cea00 2023-10-16T20:05:04.947 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.942+0000 7fb19ec79640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19814e610 con 0x7fb19815cb80 2023-10-16T20:05:04.947 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.942+0000 7fb19ec79640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1981598d0 con 0x7fb1980b28b0 2023-10-16T20:05:04.947 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.942+0000 7fb197fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1980cea00 0x7fb19814ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:04.947 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.942+0000 7fb197fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1980cea00 0x7fb19814ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48090/0 (socket says 172.21.15.88:48090) 2023-10-16T20:05:04.947 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.942+0000 7fb197fff640 1 -- 172.21.15.88:0/3166973590 learned_addr learned my addr 172.21.15.88:0/3166973590 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:04.948 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.942+0000 7fb19ca15640 1 --2- 172.21.15.88:0/3166973590 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1980b28b0 0x7fb19814ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:04.948 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.942+0000 7fb19d216640 1 --2- 172.21.15.88:0/3166973590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb19815cb80 0x7fb19814df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:04.948 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.942+0000 7fb197fff640 1 -- 172.21.15.88:0/3166973590 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1980b28b0 msgr2=0x7fb19814ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:04.948 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.942+0000 7fb197fff640 1 --2- 172.21.15.88:0/3166973590 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1980b28b0 0x7fb19814ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.948 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.942+0000 7fb197fff640 1 -- 172.21.15.88:0/3166973590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb19815cb80 msgr2=0x7fb19814df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:04.949 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.942+0000 7fb197fff640 1 --2- 172.21.15.88:0/3166973590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb19815cb80 0x7fb19814df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:04.949 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.942+0000 7fb197fff640 1 -- 172.21.15.88:0/3166973590 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18c0094d0 con 0x7fb1980cea00 2023-10-16T20:05:04.949 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.942+0000 7fb19ca15640 1 --2- 172.21.15.88:0/3166973590 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1980b28b0 0x7fb19814ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:04.949 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.946+0000 7fb197fff640 1 --2- 172.21.15.88:0/3166973590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1980cea00 0x7fb19814ced0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb18400d800 tx=0x7fb18400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:04.949 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.946+0000 7fb1957fa640 1 -- 172.21.15.88:0/3166973590 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18400b6c0 con 0x7fb1980cea00 2023-10-16T20:05:04.950 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.946+0000 7fb1957fa640 1 -- 172.21.15.88:0/3166973590 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb18400b820 con 0x7fb1980cea00 2023-10-16T20:05:04.950 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.946+0000 7fb1957fa640 1 -- 172.21.15.88:0/3166973590 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18400bb90 con 0x7fb1980cea00 2023-10-16T20:05:04.950 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.946+0000 7fb19ec79640 1 -- 172.21.15.88:0/3166973590 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb198159bb0 con 0x7fb1980cea00 2023-10-16T20:05:04.950 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.946+0000 7fb19ec79640 1 -- 172.21.15.88:0/3166973590 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb198153c80 con 0x7fb1980cea00 2023-10-16T20:05:04.951 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.946+0000 7fb19ec79640 1 -- 172.21.15.88:0/3166973590 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb198159730 con 0x7fb1980cea00 2023-10-16T20:05:04.951 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.946+0000 7fb1957fa640 1 -- 172.21.15.88:0/3166973590 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb18400bcf0 con 0x7fb1980cea00 2023-10-16T20:05:04.952 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.946+0000 7fb1957fa640 1 --2- 172.21.15.88:0/3166973590 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb17c0664a0 0x7fb17c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:04.952 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.946+0000 7fb1957fa640 1 -- 172.21.15.88:0/3166973590 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb1840846c0 con 0x7fb1980cea00 2023-10-16T20:05:04.952 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.946+0000 7fb19ca15640 1 --2- 172.21.15.88:0/3166973590 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb17c0664a0 0x7fb17c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:04.952 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.950+0000 7fb19ca15640 1 --2- 172.21.15.88:0/3166973590 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb17c0664a0 0x7fb17c068960 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb18c0af000 tx=0x7fb18c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:04.953 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:04.950+0000 7fb1957fa640 1 -- 172.21.15.88:0/3166973590 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb198159730 con 0x7fb1980cea00 2023-10-16T20:05:05.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:05.134+0000 7fb19ec79640 1 -- 172.21.15.88:0/3166973590 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb198159730 con 0x7fb1980cea00 2023-10-16T20:05:05.134 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:05.134+0000 7fb1957fa640 1 -- 172.21.15.88:0/3166973590 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7fb198159730 con 0x7fb1980cea00 2023-10-16T20:05:05.134 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-16T20:05:05.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:05.134+0000 7fb19ec79640 1 -- 172.21.15.88:0/3166973590 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb17c0664a0 msgr2=0x7fb17c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:05.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:05.134+0000 7fb19ec79640 1 --2- 172.21.15.88:0/3166973590 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb17c0664a0 0x7fb17c068960 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb18c0af000 tx=0x7fb18c008040 comp rx=0 tx=0).stop 2023-10-16T20:05:05.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:05.134+0000 7fb19ec79640 1 -- 172.21.15.88:0/3166973590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1980cea00 msgr2=0x7fb19814ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:05.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:05.134+0000 7fb19ec79640 1 --2- 172.21.15.88:0/3166973590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1980cea00 0x7fb19814ced0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb18400d800 tx=0x7fb18400dcc0 comp rx=0 tx=0).stop 2023-10-16T20:05:05.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:05.134+0000 7fb19ec79640 1 -- 172.21.15.88:0/3166973590 shutdown_connections 2023-10-16T20:05:05.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:05.134+0000 7fb19ec79640 1 --2- 172.21.15.88:0/3166973590 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb17c0664a0 0x7fb17c068960 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:05.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:05.134+0000 7fb19ec79640 1 --2- 172.21.15.88:0/3166973590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb19815cb80 0x7fb19814df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:05.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:05.134+0000 7fb19ec79640 1 --2- 172.21.15.88:0/3166973590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1980cea00 0x7fb19814ced0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:05.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:05.134+0000 7fb19ec79640 1 --2- 172.21.15.88:0/3166973590 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1980b28b0 0x7fb19814ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:05.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:05.134+0000 7fb19ec79640 1 -- 172.21.15.88:0/3166973590 >> 172.21.15.88:0/3166973590 conn(0x7fb1980cd370 msgr2=0x7fb198158600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:05.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:05.134+0000 7fb19ec79640 1 -- 172.21.15.88:0/3166973590 shutdown_connections 2023-10-16T20:05:05.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:05.134+0000 7fb19ec79640 1 -- 172.21.15.88:0/3166973590 wait complete. 2023-10-16T20:05:05.154 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T20:05:05.155 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T20:05:05.155 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi088.front.sepia.ceph.com 2023-10-16T20:05:05.155 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-10-16T20:05:05.155 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-10-16T20:05:05.156 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T20:05:05.156 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T20:05:05.156 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T20:05:05.157 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-10-16T20:05:05.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:05.158 DEBUG:teuthology.orchestra.run.smithi088:> ip addr 2023-10-16T20:05:05.167 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T20:05:05.167 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T20:05:05.167 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T20:05:05.168 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:05.168 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2023-10-16T20:05:05.168 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:05.168 INFO:teuthology.orchestra.run.smithi088.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:05:05.168 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f0 brd ff:ff:ff:ff:ff:ff 2023-10-16T20:05:05.169 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f0 2023-10-16T20:05:05.169 INFO:teuthology.orchestra.run.smithi088.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:05:05.169 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f1 brd ff:ff:ff:ff:ff:ff 2023-10-16T20:05:05.169 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f1 2023-10-16T20:05:05.169 INFO:teuthology.orchestra.run.smithi088.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:05:05.170 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2c brd ff:ff:ff:ff:ff:ff 2023-10-16T20:05:05.170 INFO:teuthology.orchestra.run.smithi088.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T20:05:05.170 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2d brd ff:ff:ff:ff:ff:ff 2023-10-16T20:05:05.170 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.21.15.88/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-16T20:05:05.170 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft 42427sec preferred_lft 42427sec 2023-10-16T20:05:05.171 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link 2023-10-16T20:05:05.171 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:05.171 INFO:teuthology.orchestra.run.smithi088.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-16T20:05:05.171 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 02:42:d9:3b:fb:39 brd ff:ff:ff:ff:ff:ff 2023-10-16T20:05:05.171 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-16T20:05:05.171 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:05.173 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-16T20:05:05.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:05.173 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec bash -c ' 2023-10-16T20:05:05.174 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2023-10-16T20:05:05.174 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link add name ceph-brx type bridge 2023-10-16T20:05:05.174 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip addr flush dev ceph-brx 2023-10-16T20:05:05.175 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set ceph-brx up 2023-10-16T20:05:05.175 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-16T20:05:05.176 DEBUG:teuthology.orchestra.run.smithi088:> ') 2023-10-16T20:05:05.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:05.291 DEBUG:teuthology.orchestra.run.smithi088:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T20:05:05.310 INFO:teuthology.orchestra.run.smithi088.stdout:1 2023-10-16T20:05:05.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:05.313 DEBUG:teuthology.orchestra.run.smithi088:> route 2023-10-16T20:05:05.397 INFO:teuthology.orchestra.run.smithi088.stdout:Kernel IP routing table 2023-10-16T20:05:05.397 INFO:teuthology.orchestra.run.smithi088.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T20:05:05.398 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-16T20:05:05.398 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T20:05:05.398 INFO:teuthology.orchestra.run.smithi088.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-16T20:05:05.398 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T20:05:05.398 INFO:teuthology.orchestra.run.smithi088.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:05:05.399 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:05:05.399 INFO:teuthology.orchestra.run.smithi088.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:05:05.399 INFO:teuthology.orchestra.run.smithi088.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T20:05:05.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:05.400 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec bash -c ' 2023-10-16T20:05:05.400 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2023-10-16T20:05:05.401 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T20:05:05.401 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T20:05:05.401 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T20:05:05.402 DEBUG:teuthology.orchestra.run.smithi088:> ') 2023-10-16T20:05:05.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:05.497 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2023-10-16T20:05:05.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:05.508 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list-id 2023-10-16T20:05:05.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:05.558 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec bash -c ' 2023-10-16T20:05:05.558 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2023-10-16T20:05:05.559 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-16T20:05:05.559 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-10-16T20:05:05.559 DEBUG:teuthology.orchestra.run.smithi088:> ') 2023-10-16T20:05:05.640 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-10-16T20:05:05.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:05.641 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec bash -c ' 2023-10-16T20:05:05.642 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2023-10-16T20:05:05.642 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-10-16T20:05:05.642 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-16T20:05:05.643 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-10-16T20:05:05.643 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-10-16T20:05:05.643 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-10-16T20:05:05.644 DEBUG:teuthology.orchestra.run.smithi088:> ') 2023-10-16T20:05:05.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:05.805 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec bash -c ' 2023-10-16T20:05:05.805 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2023-10-16T20:05:05.806 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set brx.0 up 2023-10-16T20:05:05.806 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set dev brx.0 master ceph-brx 2023-10-16T20:05:05.806 DEBUG:teuthology.orchestra.run.smithi088:> ') 2023-10-16T20:05:05.847 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-10-16T20:05:05.847 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T20:05:05.848 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-10-16T20:05:05.897 INFO:teuthology.orchestra.run.smithi088.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-10-16T20:05:05.897 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T20:05:05.898 DEBUG:teuthology.orchestra.run.smithi088:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-10-16T20:05:05.953 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2023-10-16T20:05:06.017 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T20:05:06.070 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2023-10-16T20:05:06.071 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2023-10-16T20:05:06.071 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2023-10-16T20:05:06.071 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2023-10-16T20:05:06.071 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2023-10-16T20:05:06.072 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2023-10-16T20:05:06.072 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2023-10-16T20:05:06.072 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2023-10-16T20:05:06.072 INFO:teuthology.orchestra.run.smithi088.stdout:/run/lock 2023-10-16T20:05:06.073 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2023-10-16T20:05:06.073 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2023-10-16T20:05:06.073 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2023-10-16T20:05:06.073 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T20:05:06.073 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2023-10-16T20:05:06.074 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2023-10-16T20:05:06.074 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2023-10-16T20:05:06.074 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2023-10-16T20:05:06.074 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/fs/nfsd 2023-10-16T20:05:06.074 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2023-10-16T20:05:06.074 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2023-10-16T20:05:06.075 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T20:05:06.075 INFO:teuthology.orchestra.run.smithi088.stdout:/run/rpc_pipefs 2023-10-16T20:05:06.075 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2023-10-16T20:05:06.075 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2023-10-16T20:05:06.075 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T20:05:06.076 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T20:05:06.076 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T20:05:06.076 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T20:05:06.076 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2023-10-16T20:05:06.076 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-16T20:05:06.077 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-16T20:05:06.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:06.078 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2023-10-16T20:05:06.129 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-16T20:05:06.129 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-10-16T20:05:06.175 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2023-10-16T20:05:06.197 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T20:05:06.250 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2023-10-16T20:05:06.251 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2023-10-16T20:05:06.251 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2023-10-16T20:05:06.251 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2023-10-16T20:05:06.251 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2023-10-16T20:05:06.251 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2023-10-16T20:05:06.252 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2023-10-16T20:05:06.252 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2023-10-16T20:05:06.252 INFO:teuthology.orchestra.run.smithi088.stdout:/run/lock 2023-10-16T20:05:06.252 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2023-10-16T20:05:06.252 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2023-10-16T20:05:06.252 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2023-10-16T20:05:06.253 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T20:05:06.253 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2023-10-16T20:05:06.253 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2023-10-16T20:05:06.253 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2023-10-16T20:05:06.253 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2023-10-16T20:05:06.253 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/fs/nfsd 2023-10-16T20:05:06.254 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2023-10-16T20:05:06.254 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2023-10-16T20:05:06.254 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T20:05:06.254 INFO:teuthology.orchestra.run.smithi088.stdout:/run/rpc_pipefs 2023-10-16T20:05:06.254 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2023-10-16T20:05:06.255 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2023-10-16T20:05:06.255 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T20:05:06.255 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T20:05:06.255 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T20:05:06.255 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T20:05:06.255 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2023-10-16T20:05:06.256 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-16T20:05:06.256 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-16T20:05:06.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:06.257 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2023-10-16T20:05:06.288 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi088.stderr:2023-10-16T20:05:06.286+0000 7fe20ed05580 -1 init, newargv = 0x5574a1255430 newargc=15 2023-10-16T20:05:06.289 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi088.stderr:ceph-fuse[43995]: starting ceph client 2023-10-16T20:05:06.299 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi088.stderr:ceph-fuse[43995]: starting fuse 2023-10-16T20:05:07.398 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2023-10-16T20:05:07.423 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T20:05:07.475 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2023-10-16T20:05:07.476 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2023-10-16T20:05:07.476 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2023-10-16T20:05:07.476 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2023-10-16T20:05:07.476 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2023-10-16T20:05:07.476 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2023-10-16T20:05:07.477 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2023-10-16T20:05:07.477 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2023-10-16T20:05:07.477 INFO:teuthology.orchestra.run.smithi088.stdout:/run/lock 2023-10-16T20:05:07.477 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2023-10-16T20:05:07.477 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2023-10-16T20:05:07.478 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2023-10-16T20:05:07.478 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T20:05:07.478 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2023-10-16T20:05:07.478 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2023-10-16T20:05:07.478 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2023-10-16T20:05:07.479 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2023-10-16T20:05:07.479 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/fs/nfsd 2023-10-16T20:05:07.479 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2023-10-16T20:05:07.479 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2023-10-16T20:05:07.479 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T20:05:07.480 INFO:teuthology.orchestra.run.smithi088.stdout:/run/rpc_pipefs 2023-10-16T20:05:07.480 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2023-10-16T20:05:07.480 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2023-10-16T20:05:07.480 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T20:05:07.480 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T20:05:07.481 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T20:05:07.481 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T20:05:07.481 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2023-10-16T20:05:07.481 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-16T20:05:07.481 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-16T20:05:07.481 INFO:teuthology.orchestra.run.smithi088.stdout:/home/ubuntu/cephtest/mnt.admin 2023-10-16T20:05:07.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:07.483 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2023-10-16T20:05:07.533 INFO:teuthology.orchestra.run.smithi088.stdout:43 2023-10-16T20:05:07.534 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-10-16T20:05:07.534 DEBUG:teuthology.orchestra.run.smithi088:> sudo stdin-killer -- python3 -c ' 2023-10-16T20:05:07.535 DEBUG:teuthology.orchestra.run.smithi088:> import glob 2023-10-16T20:05:07.535 DEBUG:teuthology.orchestra.run.smithi088:> import re 2023-10-16T20:05:07.535 DEBUG:teuthology.orchestra.run.smithi088:> import os 2023-10-16T20:05:07.536 DEBUG:teuthology.orchestra.run.smithi088:> import subprocess 2023-10-16T20:05:07.536 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:05:07.536 DEBUG:teuthology.orchestra.run.smithi088:> def _find_admin_socket(client_name): 2023-10-16T20:05:07.537 DEBUG:teuthology.orchestra.run.smithi088:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-10-16T20:05:07.537 DEBUG:teuthology.orchestra.run.smithi088:> files = glob.glob(asok_path) 2023-10-16T20:05:07.537 DEBUG:teuthology.orchestra.run.smithi088:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-10-16T20:05:07.538 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:05:07.538 DEBUG:teuthology.orchestra.run.smithi088:> # Given a non-glob path, it better be there 2023-10-16T20:05:07.539 DEBUG:teuthology.orchestra.run.smithi088:> if "*" not in asok_path: 2023-10-16T20:05:07.539 DEBUG:teuthology.orchestra.run.smithi088:> assert(len(files) == 1) 2023-10-16T20:05:07.539 DEBUG:teuthology.orchestra.run.smithi088:> return files[0] 2023-10-16T20:05:07.540 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:05:07.540 DEBUG:teuthology.orchestra.run.smithi088:> for f in files: 2023-10-16T20:05:07.540 DEBUG:teuthology.orchestra.run.smithi088:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-16T20:05:07.541 DEBUG:teuthology.orchestra.run.smithi088:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-16T20:05:07.541 DEBUG:teuthology.orchestra.run.smithi088:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-16T20:05:07.541 DEBUG:teuthology.orchestra.run.smithi088:> contents = proc_f.read() 2023-10-16T20:05:07.542 DEBUG:teuthology.orchestra.run.smithi088:> if mountpoint in contents: 2023-10-16T20:05:07.542 DEBUG:teuthology.orchestra.run.smithi088:> return f 2023-10-16T20:05:07.543 DEBUG:teuthology.orchestra.run.smithi088:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-16T20:05:07.543 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:05:07.543 DEBUG:teuthology.orchestra.run.smithi088:> print(_find_admin_socket("client.admin")) 2023-10-16T20:05:07.544 DEBUG:teuthology.orchestra.run.smithi088:> ' 2023-10-16T20:05:07.665 INFO:teuthology.orchestra.run.smithi088.stdout:/var/run/ceph/ceph-client.admin.43995.asok 2023-10-16T20:05:07.669 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-16T20:05:07.677 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.43995.asok 2023-10-16T20:05:07.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:07.678 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.43995.asok status 2023-10-16T20:05:07.854 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2023-10-16T20:05:07.854 INFO:teuthology.orchestra.run.smithi088.stdout: "metadata": { 2023-10-16T20:05:07.855 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_sha1": "4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0", 2023-10-16T20:05:07.855 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_version": "ceph version 18.0.0-6730-g4ba8a190 (4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0) reef (dev)", 2023-10-16T20:05:07.855 INFO:teuthology.orchestra.run.smithi088.stdout: "entity_id": "admin", 2023-10-16T20:05:07.855 INFO:teuthology.orchestra.run.smithi088.stdout: "hostname": "smithi088", 2023-10-16T20:05:07.855 INFO:teuthology.orchestra.run.smithi088.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-10-16T20:05:07.856 INFO:teuthology.orchestra.run.smithi088.stdout: "pid": "43995", 2023-10-16T20:05:07.856 INFO:teuthology.orchestra.run.smithi088.stdout: "root": "/" 2023-10-16T20:05:07.856 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:05:07.856 INFO:teuthology.orchestra.run.smithi088.stdout: "dentry_count": 0, 2023-10-16T20:05:07.856 INFO:teuthology.orchestra.run.smithi088.stdout: "dentry_pinned_count": 0, 2023-10-16T20:05:07.857 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 4434, 2023-10-16T20:05:07.857 INFO:teuthology.orchestra.run.smithi088.stdout: "inst": { 2023-10-16T20:05:07.857 INFO:teuthology.orchestra.run.smithi088.stdout: "name": { 2023-10-16T20:05:07.857 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "client", 2023-10-16T20:05:07.857 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 4434 2023-10-16T20:05:07.858 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:05:07.858 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": { 2023-10-16T20:05:07.858 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2023-10-16T20:05:07.858 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "192.168.0.1:0", 2023-10-16T20:05:07.858 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 3675508068 2023-10-16T20:05:07.859 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:05:07.859 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:05:07.859 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": { 2023-10-16T20:05:07.859 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2023-10-16T20:05:07.859 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "192.168.0.1:0", 2023-10-16T20:05:07.859 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 3675508068 2023-10-16T20:05:07.860 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:05:07.860 INFO:teuthology.orchestra.run.smithi088.stdout: "inst_str": "client.4434 192.168.0.1:0/3675508068", 2023-10-16T20:05:07.860 INFO:teuthology.orchestra.run.smithi088.stdout: "addr_str": "192.168.0.1:0/3675508068", 2023-10-16T20:05:07.860 INFO:teuthology.orchestra.run.smithi088.stdout: "inode_count": 1, 2023-10-16T20:05:07.860 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_epoch": 7, 2023-10-16T20:05:07.860 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_epoch": 27, 2023-10-16T20:05:07.861 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_epoch_barrier": 0, 2023-10-16T20:05:07.861 INFO:teuthology.orchestra.run.smithi088.stdout: "blocklisted": false, 2023-10-16T20:05:07.861 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "cephfs" 2023-10-16T20:05:07.861 INFO:teuthology.orchestra.run.smithi088.stdout:} 2023-10-16T20:05:07.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:07.869 DEBUG:teuthology.orchestra.run.smithi088:> stat --file-system '--printf=%T 2023-10-16T20:05:07.869 DEBUG:teuthology.orchestra.run.smithi088:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-10-16T20:05:07.880 INFO:teuthology.orchestra.run.smithi088.stdout:fuseblk 2023-10-16T20:05:07.880 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-10-16T20:05:07.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:07.881 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-10-16T20:05:07.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:07.942 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-10-16T20:05:08.027 INFO:teuthology.orchestra.run.smithi088.stdout:# file: . 2023-10-16T20:05:08.027 INFO:teuthology.orchestra.run.smithi088.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-10-16T20:05:08.028 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:08.029 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:05:08.029 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:05:08.081 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:05:08.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.262+0000 7fd9073c4640 1 Processor -- start 2023-10-16T20:05:08.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.262+0000 7fd9073c4640 1 -- start start 2023-10-16T20:05:08.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.262+0000 7fd9073c4640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd90015cb80 0x7fd9000b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:08.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.262+0000 7fd9073c4640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd9000b28b0 0x7fd9000ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:08.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.262+0000 7fd9073c4640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9000b2cf0 con 0x7fd9000cea00 2023-10-16T20:05:08.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.262+0000 7fd9073c4640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd90006b490 con 0x7fd90015cb80 2023-10-16T20:05:08.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.262+0000 7fd9073c4640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9000cf180 con 0x7fd9000b28b0 2023-10-16T20:05:08.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.262+0000 7fd905160640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd90015cb80 0x7fd9000b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:08.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.262+0000 7fd905160640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd90015cb80 0x7fd9000b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48094/0 (socket says 172.21.15.88:48094) 2023-10-16T20:05:08.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.262+0000 7fd905160640 1 -- 172.21.15.88:0/1007888687 learned_addr learned my addr 172.21.15.88:0/1007888687 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:08.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.262+0000 7fd90495f640 1 --2- 172.21.15.88:0/1007888687 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd9000b28b0 0x7fd9000ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:08.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.262+0000 7fd905160640 1 -- 172.21.15.88:0/1007888687 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd9000b28b0 msgr2=0x7fd9000ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:08.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.262+0000 7fd905160640 1 --2- 172.21.15.88:0/1007888687 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd9000b28b0 0x7fd9000ce4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:08.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.262+0000 7fd905160640 1 -- 172.21.15.88:0/1007888687 >> v1:172.21.15.88:6789/0 conn(0x7fd9000cea00 legacy=0x7fd90006ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:08.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.262+0000 7fd905160640 1 -- 172.21.15.88:0/1007888687 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd90006b700 con 0x7fd90015cb80 2023-10-16T20:05:08.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.262+0000 7fd8f7fff640 1 -- 172.21.15.88:0/1007888687 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 780575545 0 0) 0x7fd9000b2cf0 con 0x7fd9000cea00 2023-10-16T20:05:08.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.262+0000 7fd90495f640 1 --2- 172.21.15.88:0/1007888687 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd9000b28b0 0x7fd9000ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:08.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.266+0000 7fd905160640 1 --2- 172.21.15.88:0/1007888687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd90015cb80 0x7fd9000b2370 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc009870 tx=0x7fd8fc01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f886f950e909d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:08.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.266+0000 7fd8f7fff640 1 -- 172.21.15.88:0/1007888687 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc0ac620 con 0x7fd90015cb80 2023-10-16T20:05:08.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.266+0000 7fd8f7fff640 1 -- 172.21.15.88:0/1007888687 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8fc0ac780 con 0x7fd90015cb80 2023-10-16T20:05:08.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.266+0000 7fd8f7fff640 1 -- 172.21.15.88:0/1007888687 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc0acb10 con 0x7fd90015cb80 2023-10-16T20:05:08.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.266+0000 7fd9073c4640 1 -- 172.21.15.88:0/1007888687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd90015cb80 msgr2=0x7fd9000b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:08.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.266+0000 7fd9073c4640 1 --2- 172.21.15.88:0/1007888687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd90015cb80 0x7fd9000b2370 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc009870 tx=0x7fd8fc01c510 comp rx=0 tx=0).stop 2023-10-16T20:05:08.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.266+0000 7fd9073c4640 1 -- 172.21.15.88:0/1007888687 shutdown_connections 2023-10-16T20:05:08.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.266+0000 7fd9073c4640 1 --2- 172.21.15.88:0/1007888687 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd9000b28b0 0x7fd9000ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:08.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.266+0000 7fd9073c4640 1 --2- 172.21.15.88:0/1007888687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd90015cb80 0x7fd9000b2370 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:08.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.266+0000 7fd9073c4640 1 -- 172.21.15.88:0/1007888687 >> 172.21.15.88:0/1007888687 conn(0x7fd9000cd370 msgr2=0x7fd9000cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:08.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.266+0000 7fd9073c4640 1 -- 172.21.15.88:0/1007888687 shutdown_connections 2023-10-16T20:05:08.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.266+0000 7fd9073c4640 1 -- 172.21.15.88:0/1007888687 wait complete. 2023-10-16T20:05:08.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.270+0000 7fd9073c4640 1 Processor -- start 2023-10-16T20:05:08.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.270+0000 7fd9073c4640 1 -- start start 2023-10-16T20:05:08.273 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.270+0000 7fd9073c4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9000b28b0 0x7fd90015cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:08.273 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.270+0000 7fd9073c4640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd9000cea00 0x7fd9000b1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:08.273 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.270+0000 7fd9073c4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd90015cb80 0x7fd9000b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:08.273 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.270+0000 7fd9073c4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90014efc0 con 0x7fd90015cb80 2023-10-16T20:05:08.273 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.270+0000 7fd9073c4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90015a760 con 0x7fd9000b28b0 2023-10-16T20:05:08.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.270+0000 7fd9073c4640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd90015aa40 con 0x7fd9000cea00 2023-10-16T20:05:08.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.270+0000 7fd905961640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd90015cb80 0x7fd9000b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:08.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.270+0000 7fd905961640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd90015cb80 0x7fd9000b18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48102/0 (socket says 172.21.15.88:48102) 2023-10-16T20:05:08.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.270+0000 7fd905961640 1 -- 172.21.15.88:0/1071357024 learned_addr learned my addr 172.21.15.88:0/1071357024 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:08.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.270+0000 7fd905160640 1 --2- 172.21.15.88:0/1071357024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9000b28b0 0x7fd90015cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:08.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.270+0000 7fd90495f640 1 --2- 172.21.15.88:0/1071357024 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd9000cea00 0x7fd9000b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:08.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.270+0000 7fd905961640 1 -- 172.21.15.88:0/1071357024 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd9000cea00 msgr2=0x7fd9000b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:08.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.270+0000 7fd905961640 1 --2- 172.21.15.88:0/1071357024 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd9000cea00 0x7fd9000b1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:08.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.270+0000 7fd905961640 1 -- 172.21.15.88:0/1071357024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9000b28b0 msgr2=0x7fd90015cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:08.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.270+0000 7fd905961640 1 --2- 172.21.15.88:0/1071357024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9000b28b0 0x7fd90015cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:08.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.270+0000 7fd905961640 1 -- 172.21.15.88:0/1071357024 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8fc0094d0 con 0x7fd90015cb80 2023-10-16T20:05:08.276 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.270+0000 7fd90495f640 1 --2- 172.21.15.88:0/1071357024 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd9000cea00 0x7fd9000b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:08.276 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.270+0000 7fd905961640 1 --2- 172.21.15.88:0/1071357024 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd90015cb80 0x7fd9000b18d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd8f800c890 tx=0x7fd8f800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:08.276 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.270+0000 7fd8f5ffb640 1 -- 172.21.15.88:0/1071357024 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f8003db0 con 0x7fd90015cb80 2023-10-16T20:05:08.276 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.270+0000 7fd8f5ffb640 1 -- 172.21.15.88:0/1071357024 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f8005070 con 0x7fd90015cb80 2023-10-16T20:05:08.276 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.270+0000 7fd8f5ffb640 1 -- 172.21.15.88:0/1071357024 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f8005420 con 0x7fd90015cb80 2023-10-16T20:05:08.277 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.270+0000 7fd9073c4640 1 -- 172.21.15.88:0/1071357024 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9000c6ee0 con 0x7fd90015cb80 2023-10-16T20:05:08.277 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.270+0000 7fd9073c4640 1 -- 172.21.15.88:0/1071357024 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9000c57c0 con 0x7fd90015cb80 2023-10-16T20:05:08.278 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.270+0000 7fd9073c4640 1 -- 172.21.15.88:0/1071357024 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd90004e9c0 con 0x7fd90015cb80 2023-10-16T20:05:08.278 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.274+0000 7fd8f5ffb640 1 -- 172.21.15.88:0/1071357024 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd8f8005580 con 0x7fd90015cb80 2023-10-16T20:05:08.278 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.274+0000 7fd8f5ffb640 1 --2- 172.21.15.88:0/1071357024 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd8e40664d0 0x7fd8e4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:08.278 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.274+0000 7fd8f5ffb640 1 -- 172.21.15.88:0/1071357024 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd8f8083980 con 0x7fd90015cb80 2023-10-16T20:05:08.279 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.274+0000 7fd905160640 1 --2- 172.21.15.88:0/1071357024 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd8e40664d0 0x7fd8e4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:08.279 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.274+0000 7fd905160640 1 --2- 172.21.15.88:0/1071357024 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd8e40664d0 0x7fd8e4068990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc0ae000 tx=0x7fd8fc0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:08.279 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.278+0000 7fd8f5ffb640 1 -- 172.21.15.88:0/1071357024 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd8f8050c70 con 0x7fd90015cb80 2023-10-16T20:05:08.429 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.426+0000 7fd9073c4640 1 -- 172.21.15.88:0/1071357024 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd900160020 con 0x7fd90015cb80 2023-10-16T20:05:08.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.430+0000 7fd8f5ffb640 1 -- 172.21.15.88:0/1071357024 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18274 (secure 0 0 0) 0x7fd8f8017020 con 0x7fd90015cb80 2023-10-16T20:05:08.430 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:08.430 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:05:01.835630+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T20:04:54.442487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T20:04:55.766641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T20:04:59.475830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:05:08.432 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.430+0000 7fd9073c4640 1 -- 172.21.15.88:0/1071357024 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd8e40664d0 msgr2=0x7fd8e4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:08.432 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.430+0000 7fd9073c4640 1 --2- 172.21.15.88:0/1071357024 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd8e40664d0 0x7fd8e4068990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc0ae000 tx=0x7fd8fc0a7000 comp rx=0 tx=0).stop 2023-10-16T20:05:08.432 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.430+0000 7fd9073c4640 1 -- 172.21.15.88:0/1071357024 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd90015cb80 msgr2=0x7fd9000b18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:08.433 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.430+0000 7fd9073c4640 1 --2- 172.21.15.88:0/1071357024 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd90015cb80 0x7fd9000b18d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd8f800c890 tx=0x7fd8f800cd50 comp rx=0 tx=0).stop 2023-10-16T20:05:08.433 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.430+0000 7fd9073c4640 1 -- 172.21.15.88:0/1071357024 shutdown_connections 2023-10-16T20:05:08.433 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.430+0000 7fd9073c4640 1 --2- 172.21.15.88:0/1071357024 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd8e40664d0 0x7fd8e4068990 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:08.433 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.430+0000 7fd9073c4640 1 --2- 172.21.15.88:0/1071357024 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd9000cea00 0x7fd9000b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:08.433 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.430+0000 7fd9073c4640 1 --2- 172.21.15.88:0/1071357024 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd90015cb80 0x7fd9000b18d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:08.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.430+0000 7fd9073c4640 1 --2- 172.21.15.88:0/1071357024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9000b28b0 0x7fd90015cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:08.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.430+0000 7fd9073c4640 1 -- 172.21.15.88:0/1071357024 >> 172.21.15.88:0/1071357024 conn(0x7fd9000cd370 msgr2=0x7fd9000cc150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:08.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.430+0000 7fd9073c4640 1 -- 172.21.15.88:0/1071357024 shutdown_connections 2023-10-16T20:05:08.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.430+0000 7fd9073c4640 1 -- 172.21.15.88:0/1071357024 wait complete. 2023-10-16T20:05:08.451 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T20:04:54.442487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T20:04:55.766641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T20:04:59.475830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T20:05:08.452 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:05:08.593 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.590+0000 7f39c1046640 1 Processor -- start 2023-10-16T20:05:08.593 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.590+0000 7f39c1046640 1 -- start start 2023-10-16T20:05:08.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.594+0000 7f39c1046640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f39bc1500a0 0x7f39bc159f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:08.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.594+0000 7f39c1046640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f39bc06c2e0 0x7f39bc070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:08.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.594+0000 7f39c1046640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39bc15aa10 con 0x7f39bc071020 2023-10-16T20:05:08.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.594+0000 7f39c1046640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39bc150480 con 0x7f39bc1500a0 2023-10-16T20:05:08.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.594+0000 7f39c1046640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc15a490 con 0x7f39bc06c2e0 2023-10-16T20:05:08.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.594+0000 7f39b3fff640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f39bc06c2e0 0x7f39bc070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:08.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.594+0000 7f39b3fff640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f39bc06c2e0 0x7f39bc070ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:54882/0 (socket says 172.21.15.88:54882) 2023-10-16T20:05:08.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.594+0000 7f39b3fff640 1 -- 172.21.15.88:0/3579280687 learned_addr learned my addr 172.21.15.88:0/3579280687 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:08.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.594+0000 7f39bad9d640 1 --2- 172.21.15.88:0/3579280687 >> v2:172.21.15.94:3300/0 conn(0x7f39bc1500a0 0x7f39bc159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:08.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.594+0000 7f39b3fff640 1 -- 172.21.15.88:0/3579280687 >> v2:172.21.15.94:3300/0 conn(0x7f39bc1500a0 msgr2=0x7f39bc159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:08.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.594+0000 7f39b3fff640 1 --2- 172.21.15.88:0/3579280687 >> v2:172.21.15.94:3300/0 conn(0x7f39bc1500a0 0x7f39bc159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:08.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.594+0000 7f39b3fff640 1 -- 172.21.15.88:0/3579280687 >> v1:172.21.15.94:6789/0 conn(0x7f39bc071020 legacy=0x7f39bc15ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:08.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.594+0000 7f39b3fff640 1 -- 172.21.15.88:0/3579280687 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39bc06ca10 con 0x7f39bc06c2e0 2023-10-16T20:05:08.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.598+0000 7f39b3fff640 1 --2- 172.21.15.88:0/3579280687 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f39bc06c2e0 0x7f39bc070ae0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f39a4009870 tx=0x7f39a401d650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c94a751dc97751fd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:08.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.598+0000 7f39ba59c640 1 -- 172.21.15.88:0/3579280687 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a40ad640 con 0x7f39bc06c2e0 2023-10-16T20:05:08.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.598+0000 7f39ba59c640 1 -- 172.21.15.88:0/3579280687 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a40ad7a0 con 0x7f39bc06c2e0 2023-10-16T20:05:08.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.598+0000 7f39ba59c640 1 -- 172.21.15.88:0/3579280687 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a40adb30 con 0x7f39bc06c2e0 2023-10-16T20:05:08.600 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.598+0000 7f39c1046640 1 -- 172.21.15.88:0/3579280687 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f39bc06c2e0 msgr2=0x7f39bc070ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:08.600 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.598+0000 7f39c1046640 1 --2- 172.21.15.88:0/3579280687 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f39bc06c2e0 0x7f39bc070ae0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f39a4009870 tx=0x7f39a401d650 comp rx=0 tx=0).stop 2023-10-16T20:05:08.600 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.598+0000 7f39c1046640 1 -- 172.21.15.88:0/3579280687 shutdown_connections 2023-10-16T20:05:08.600 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.598+0000 7f39c1046640 1 --2- 172.21.15.88:0/3579280687 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f39bc06c2e0 0x7f39bc070ae0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:08.600 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.598+0000 7f39c1046640 1 --2- 172.21.15.88:0/3579280687 >> v2:172.21.15.94:3300/0 conn(0x7f39bc1500a0 0x7f39bc159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:08.601 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.598+0000 7f39c1046640 1 -- 172.21.15.88:0/3579280687 >> 172.21.15.88:0/3579280687 conn(0x7f39bc0cb080 msgr2=0x7f39bc0cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:08.601 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.598+0000 7f39c1046640 1 -- 172.21.15.88:0/3579280687 shutdown_connections 2023-10-16T20:05:08.601 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.598+0000 7f39c1046640 1 -- 172.21.15.88:0/3579280687 wait complete. 2023-10-16T20:05:08.603 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.598+0000 7f39c1046640 1 Processor -- start 2023-10-16T20:05:08.603 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.602+0000 7f39c1046640 1 -- start start 2023-10-16T20:05:08.603 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.602+0000 7f39c1046640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39bc06c2e0 0x7f39bc1f5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:08.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.602+0000 7f39c1046640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f39bc071020 0x7f39bc1589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:08.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.602+0000 7f39c1046640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39bc1500a0 0x7f39bc0cc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:08.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.602+0000 7f39c1046640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc147960 con 0x7f39bc06c2e0 2023-10-16T20:05:08.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.602+0000 7f39c1046640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc1f64c0 con 0x7f39bc1500a0 2023-10-16T20:05:08.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.602+0000 7f39c1046640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc158f00 con 0x7f39bc071020 2023-10-16T20:05:08.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.602+0000 7f39bad9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39bc06c2e0 0x7f39bc1f5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:08.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.602+0000 7f39bad9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39bc06c2e0 0x7f39bc1f5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48106/0 (socket says 172.21.15.88:48106) 2023-10-16T20:05:08.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.602+0000 7f39bad9d640 1 -- 172.21.15.88:0/1513275015 learned_addr learned my addr 172.21.15.88:0/1513275015 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:08.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.602+0000 7f39b3fff640 1 --2- 172.21.15.88:0/1513275015 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f39bc071020 0x7f39bc1589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:08.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.602+0000 7f39bb59e640 1 --2- 172.21.15.88:0/1513275015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39bc1500a0 0x7f39bc0cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:08.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.602+0000 7f39bad9d640 1 -- 172.21.15.88:0/1513275015 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f39bc071020 msgr2=0x7f39bc1589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:08.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.602+0000 7f39bad9d640 1 --2- 172.21.15.88:0/1513275015 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f39bc071020 0x7f39bc1589c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:08.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.602+0000 7f39bad9d640 1 -- 172.21.15.88:0/1513275015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39bc1500a0 msgr2=0x7f39bc0cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:08.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.602+0000 7f39bad9d640 1 --2- 172.21.15.88:0/1513275015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39bc1500a0 0x7f39bc0cc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:08.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.602+0000 7f39bad9d640 1 -- 172.21.15.88:0/1513275015 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a40094d0 con 0x7f39bc06c2e0 2023-10-16T20:05:08.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.602+0000 7f39b3fff640 1 --2- 172.21.15.88:0/1513275015 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f39bc071020 0x7f39bc1589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:08.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.602+0000 7f39bb59e640 1 --2- 172.21.15.88:0/1513275015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39bc1500a0 0x7f39bc0cc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:08.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.602+0000 7f39bad9d640 1 --2- 172.21.15.88:0/1513275015 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39bc06c2e0 0x7f39bc1f5f80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00e8a0 tx=0x7f39ac00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:08.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.602+0000 7f39b37fe640 1 -- 172.21.15.88:0/1513275015 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac009510 con 0x7f39bc06c2e0 2023-10-16T20:05:08.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.602+0000 7f39b37fe640 1 -- 172.21.15.88:0/1513275015 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ac00c940 con 0x7f39bc06c2e0 2023-10-16T20:05:08.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.602+0000 7f39c1046640 1 -- 172.21.15.88:0/1513275015 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39bc163190 con 0x7f39bc06c2e0 2023-10-16T20:05:08.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.602+0000 7f39b37fe640 1 -- 172.21.15.88:0/1513275015 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac00ccb0 con 0x7f39bc06c2e0 2023-10-16T20:05:08.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.602+0000 7f39c1046640 1 -- 172.21.15.88:0/1513275015 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39bc163750 con 0x7f39bc06c2e0 2023-10-16T20:05:08.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.602+0000 7f39c1046640 1 -- 172.21.15.88:0/1513275015 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39bc04e9c0 con 0x7f39bc06c2e0 2023-10-16T20:05:08.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.606+0000 7f39b37fe640 1 -- 172.21.15.88:0/1513275015 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f39ac013470 con 0x7f39bc06c2e0 2023-10-16T20:05:08.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.606+0000 7f39b37fe640 1 --2- 172.21.15.88:0/1513275015 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f399c0664a0 0x7f399c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:08.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.606+0000 7f39b37fe640 1 -- 172.21.15.88:0/1513275015 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f39ac084250 con 0x7f39bc06c2e0 2023-10-16T20:05:08.610 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.606+0000 7f39b3fff640 1 --2- 172.21.15.88:0/1513275015 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f399c0664a0 0x7f399c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:08.610 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.606+0000 7f39b3fff640 1 --2- 172.21.15.88:0/1513275015 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f399c0664a0 0x7f399c068960 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f39a40af000 tx=0x7f39a40a8000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:08.610 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.610+0000 7f39b37fe640 1 -- 172.21.15.88:0/1513275015 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f39ac0513d0 con 0x7f39bc06c2e0 2023-10-16T20:05:08.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.758+0000 7f39c1046640 1 -- 172.21.15.88:0/1513275015 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f39bc06d430 con 0x7f39bc06c2e0 2023-10-16T20:05:08.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.758+0000 7f39b37fe640 1 -- 172.21.15.88:0/1513275015 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f39ac0549f0 con 0x7f39bc06c2e0 2023-10-16T20:05:08.760 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:05:08.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.762+0000 7f39c1046640 1 -- 172.21.15.88:0/1513275015 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f399c0664a0 msgr2=0x7f399c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:08.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.762+0000 7f39c1046640 1 --2- 172.21.15.88:0/1513275015 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f399c0664a0 0x7f399c068960 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f39a40af000 tx=0x7f39a40a8000 comp rx=0 tx=0).stop 2023-10-16T20:05:08.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.762+0000 7f39c1046640 1 -- 172.21.15.88:0/1513275015 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39bc06c2e0 msgr2=0x7f39bc1f5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:08.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.762+0000 7f39c1046640 1 --2- 172.21.15.88:0/1513275015 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39bc06c2e0 0x7f39bc1f5f80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00e8a0 tx=0x7f39ac00eca0 comp rx=0 tx=0).stop 2023-10-16T20:05:08.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.762+0000 7f39c1046640 1 -- 172.21.15.88:0/1513275015 shutdown_connections 2023-10-16T20:05:08.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.762+0000 7f39c1046640 1 --2- 172.21.15.88:0/1513275015 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f399c0664a0 0x7f399c068960 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:08.763 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.762+0000 7f39c1046640 1 --2- 172.21.15.88:0/1513275015 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f39bc071020 0x7f39bc1589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:08.763 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.762+0000 7f39c1046640 1 --2- 172.21.15.88:0/1513275015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39bc1500a0 0x7f39bc0cc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:08.763 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.762+0000 7f39c1046640 1 --2- 172.21.15.88:0/1513275015 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39bc06c2e0 0x7f39bc1f5f80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:08.763 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.762+0000 7f39c1046640 1 -- 172.21.15.88:0/1513275015 >> 172.21.15.88:0/1513275015 conn(0x7f39bc0cb080 msgr2=0x7f39bc14df60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:08.763 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.762+0000 7f39c1046640 1 -- 172.21.15.88:0/1513275015 shutdown_connections 2023-10-16T20:05:08.764 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.762+0000 7f39c1046640 1 -- 172.21.15.88:0/1513275015 wait complete. 2023-10-16T20:05:08.780 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T20:05:08.925 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.922+0000 7ffa9a209640 1 Processor -- start 2023-10-16T20:05:08.925 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.922+0000 7ffa9a209640 1 -- start start 2023-10-16T20:05:08.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.926+0000 7ffa9a209640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ffa940cd050 0x7ffa940cd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:08.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.926+0000 7ffa9a209640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa940b2fb0 con 0x7ffa9415c8d0 2023-10-16T20:05:08.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.926+0000 7ffa9a209640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa9415d510 con 0x7ffa940b2b20 2023-10-16T20:05:08.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.926+0000 7ffa9a209640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa9406cde0 con 0x7ffa940cd050 2023-10-16T20:05:08.929 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.926+0000 7ffa99207640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7ffa9415c8d0 0x7ffa9415ccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:48268/0 (socket says 172.21.15.88:48268) 2023-10-16T20:05:08.929 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.926+0000 7ffa99207640 1 -- 172.21.15.88:0/2690647412 learned_addr learned my addr 172.21.15.88:0/2690647412 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:08.929 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.926+0000 7ffa99a08640 1 --2- 172.21.15.88:0/2690647412 >> v2:172.21.15.94:3300/0 conn(0x7ffa940cd050 0x7ffa940cd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:08.929 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.926+0000 7ffa8bfff640 1 -- 172.21.15.88:0/2690647412 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 704305277 0 0) 0x7ffa940b2fb0 con 0x7ffa9415c8d0 2023-10-16T20:05:08.929 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.926+0000 7ffa8bfff640 1 -- 172.21.15.88:0/2690647412 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa80003530 con 0x7ffa9415c8d0 2023-10-16T20:05:08.930 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.926+0000 7ffa8bfff640 1 -- 172.21.15.88:0/2690647412 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1485786259 0 0) 0x7ffa80003530 con 0x7ffa9415c8d0 2023-10-16T20:05:08.930 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.926+0000 7ffa8bfff640 1 -- 172.21.15.88:0/2690647412 >> v2:172.21.15.94:3300/0 conn(0x7ffa940cd050 msgr2=0x7ffa940cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:08.930 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.926+0000 7ffa8bfff640 1 --2- 172.21.15.88:0/2690647412 >> v2:172.21.15.94:3300/0 conn(0x7ffa940cd050 0x7ffa940cd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:08.930 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.926+0000 7ffa8bfff640 1 -- 172.21.15.88:0/2690647412 >> v1:172.21.15.94:6789/0 conn(0x7ffa940b2b20 legacy=0x7ffa940d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:08.930 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.926+0000 7ffa8bfff640 1 -- 172.21.15.88:0/2690647412 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9415d7f0 con 0x7ffa9415c8d0 2023-10-16T20:05:08.931 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.926+0000 7ffa8bfff640 1 -- 172.21.15.88:0/2690647412 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7ffa90001680 con 0x7ffa9415c8d0 2023-10-16T20:05:08.931 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.926+0000 7ffa8bfff640 1 -- 172.21.15.88:0/2690647412 >> v1:172.21.15.88:6789/0 conn(0x7ffa9415c8d0 legacy=0x7ffa9415ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:08.931 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.926+0000 7ffa8bfff640 1 --2- 172.21.15.88:0/2690647412 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa80003980 0x7ffa80003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:08.931 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.926+0000 7ffa8bfff640 1 --2- 172.21.15.88:0/2690647412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa80004380 0x7ffa80004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:08.931 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.926+0000 7ffa8bfff640 1 --2- 172.21.15.88:0/2690647412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa80005050 0x7ffa8000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:08.932 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.926+0000 7ffa8bfff640 1 -- 172.21.15.88:0/2690647412 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8000bf80 con 0x7ffa80004380 2023-10-16T20:05:08.932 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.926+0000 7ffa8bfff640 1 -- 172.21.15.88:0/2690647412 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8000c0f0 con 0x7ffa80005050 2023-10-16T20:05:08.932 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.926+0000 7ffa8bfff640 1 -- 172.21.15.88:0/2690647412 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8000c2b0 con 0x7ffa80003980 2023-10-16T20:05:08.932 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.926+0000 7ffa98a06640 1 --2- 172.21.15.88:0/2690647412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa80004380 0x7ffa80004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:08.932 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.926+0000 7ffa99207640 1 --2- 172.21.15.88:0/2690647412 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa80003980 0x7ffa80003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:08.932 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.926+0000 7ffa99a08640 1 --2- 172.21.15.88:0/2690647412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa80005050 0x7ffa8000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:08.933 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.926+0000 7ffa98a06640 1 -- 172.21.15.88:0/2690647412 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa80003980 msgr2=0x7ffa80003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:08.933 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.926+0000 7ffa98a06640 1 --2- 172.21.15.88:0/2690647412 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa80003980 0x7ffa80003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:08.933 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.926+0000 7ffa98a06640 1 -- 172.21.15.88:0/2690647412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa80005050 msgr2=0x7ffa8000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:08.933 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.926+0000 7ffa98a06640 1 --2- 172.21.15.88:0/2690647412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa80005050 0x7ffa8000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:08.933 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.926+0000 7ffa98a06640 1 -- 172.21.15.88:0/2690647412 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa80003530 con 0x7ffa80004380 2023-10-16T20:05:08.934 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.926+0000 7ffa99a08640 1 --2- 172.21.15.88:0/2690647412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa80005050 0x7ffa8000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:08.934 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.926+0000 7ffa99a08640 1 -- 172.21.15.88:0/2690647412 reap_dead start 2023-10-16T20:05:08.936 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.930+0000 7ffa98a06640 1 --2- 172.21.15.88:0/2690647412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa80004380 0x7ffa80004850 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ffa84001910 tx=0x7ffa8401ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65aacaf3c93af56a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:08.936 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.930+0000 7ffa8bfff640 1 -- 172.21.15.88:0/2690647412 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa840a88b0 con 0x7ffa80004380 2023-10-16T20:05:08.937 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.930+0000 7ffa8bfff640 1 -- 172.21.15.88:0/2690647412 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa840a8a30 con 0x7ffa80004380 2023-10-16T20:05:08.937 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.930+0000 7ffa8bfff640 1 -- 172.21.15.88:0/2690647412 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa840a8de0 con 0x7ffa80004380 2023-10-16T20:05:08.937 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.930+0000 7ffa9a209640 1 -- 172.21.15.88:0/2690647412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa80004380 msgr2=0x7ffa80004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:08.937 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.930+0000 7ffa9a209640 1 --2- 172.21.15.88:0/2690647412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa80004380 0x7ffa80004850 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ffa84001910 tx=0x7ffa8401ec90 comp rx=0 tx=0).stop 2023-10-16T20:05:08.937 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.930+0000 7ffa9a209640 1 -- 172.21.15.88:0/2690647412 shutdown_connections 2023-10-16T20:05:08.937 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.930+0000 7ffa9a209640 1 --2- 172.21.15.88:0/2690647412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa80004380 0x7ffa80004850 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:08.938 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.930+0000 7ffa9a209640 1 -- 172.21.15.88:0/2690647412 >> 172.21.15.88:0/2690647412 conn(0x7ffa94069e90 msgr2=0x7ffa9406a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:08.938 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.930+0000 7ffa9a209640 1 -- 172.21.15.88:0/2690647412 shutdown_connections 2023-10-16T20:05:08.938 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.930+0000 7ffa9a209640 1 -- 172.21.15.88:0/2690647412 wait complete. 2023-10-16T20:05:08.938 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.934+0000 7ffa9a209640 1 Processor -- start 2023-10-16T20:05:08.939 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.934+0000 7ffa9a209640 1 -- start start 2023-10-16T20:05:08.939 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.934+0000 7ffa9a209640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa80004380 0x7ffa941f22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:08.939 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.934+0000 7ffa9a209640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa940acc70 0x7ffa940ad810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:08.939 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.934+0000 7ffa9a209640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa940b3260 0x7ffa940b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:08.939 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.934+0000 7ffa9a209640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa940ac7e0 con 0x7ffa940b3260 2023-10-16T20:05:08.940 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.934+0000 7ffa9a209640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa941f2830 con 0x7ffa940acc70 2023-10-16T20:05:08.940 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.934+0000 7ffa9a209640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa940abbd0 con 0x7ffa80004380 2023-10-16T20:05:08.940 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.934+0000 7ffa99a08640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa940b3260 0x7ffa940b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:08.940 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.934+0000 7ffa99a08640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa940b3260 0x7ffa940b36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48132/0 (socket says 172.21.15.88:48132) 2023-10-16T20:05:08.940 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.934+0000 7ffa99a08640 1 -- 172.21.15.88:0/2890156015 learned_addr learned my addr 172.21.15.88:0/2890156015 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:08.940 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.934+0000 7ffa98a06640 1 --2- 172.21.15.88:0/2890156015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa940acc70 0x7ffa940ad810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:08.941 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.934+0000 7ffa99207640 1 --2- 172.21.15.88:0/2890156015 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa80004380 0x7ffa941f22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:08.941 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.934+0000 7ffa99a08640 1 -- 172.21.15.88:0/2890156015 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa80004380 msgr2=0x7ffa941f22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:08.941 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.934+0000 7ffa99a08640 1 --2- 172.21.15.88:0/2890156015 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa80004380 0x7ffa941f22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:08.941 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.934+0000 7ffa99a08640 1 -- 172.21.15.88:0/2890156015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa940acc70 msgr2=0x7ffa940ad810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:08.941 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.934+0000 7ffa99a08640 1 --2- 172.21.15.88:0/2890156015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa940acc70 0x7ffa940ad810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:08.941 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.934+0000 7ffa99a08640 1 -- 172.21.15.88:0/2890156015 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa940abeb0 con 0x7ffa940b3260 2023-10-16T20:05:08.942 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.934+0000 7ffa99a08640 1 --2- 172.21.15.88:0/2890156015 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa940b3260 0x7ffa940b36e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ffa940b0030 tx=0x7ffa8c010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:08.942 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.934+0000 7ffa89ffb640 1 -- 172.21.15.88:0/2890156015 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c01b070 con 0x7ffa940b3260 2023-10-16T20:05:08.942 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.934+0000 7ffa89ffb640 1 -- 172.21.15.88:0/2890156015 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8c00f040 con 0x7ffa940b3260 2023-10-16T20:05:08.942 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.934+0000 7ffa89ffb640 1 -- 172.21.15.88:0/2890156015 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c00e050 con 0x7ffa940b3260 2023-10-16T20:05:08.942 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.934+0000 7ffa9a209640 1 -- 172.21.15.88:0/2890156015 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa940ac190 con 0x7ffa940b3260 2023-10-16T20:05:08.943 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.934+0000 7ffa9a209640 1 -- 172.21.15.88:0/2890156015 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa940ab8a0 con 0x7ffa940b3260 2023-10-16T20:05:08.944 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.934+0000 7ffa9a209640 1 -- 172.21.15.88:0/2890156015 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa940674f0 con 0x7ffa940b3260 2023-10-16T20:05:08.944 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.938+0000 7ffa89ffb640 1 -- 172.21.15.88:0/2890156015 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ffa8c015440 con 0x7ffa940b3260 2023-10-16T20:05:08.944 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.938+0000 7ffa89ffb640 1 --2- 172.21.15.88:0/2890156015 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ffa780664d0 0x7ffa78068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:08.944 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.938+0000 7ffa89ffb640 1 -- 172.21.15.88:0/2890156015 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ffa8c0844f0 con 0x7ffa940b3260 2023-10-16T20:05:08.944 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.938+0000 7ffa99207640 1 --2- 172.21.15.88:0/2890156015 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ffa780664d0 0x7ffa78068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:08.945 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.938+0000 7ffa99207640 1 --2- 172.21.15.88:0/2890156015 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ffa780664d0 0x7ffa78068990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ffa90001060 tx=0x7ffa90009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:08.945 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:08.942+0000 7ffa89ffb640 1 -- 172.21.15.88:0/2890156015 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ffa8c0515e0 con 0x7ffa940b3260 2023-10-16T20:05:09.090 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.090+0000 7ffa9a209640 1 -- 172.21.15.88:0/2890156015 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffa940b65c0 con 0x7ffa940b3260 2023-10-16T20:05:09.091 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.090+0000 7ffa89ffb640 1 -- 172.21.15.88:0/2890156015 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffa8c054c00 con 0x7ffa940b3260 2023-10-16T20:05:09.091 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:05:09.093 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.090+0000 7ffa9a209640 1 -- 172.21.15.88:0/2890156015 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ffa780664d0 msgr2=0x7ffa78068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:09.093 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.090+0000 7ffa9a209640 1 --2- 172.21.15.88:0/2890156015 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ffa780664d0 0x7ffa78068990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ffa90001060 tx=0x7ffa90009040 comp rx=0 tx=0).stop 2023-10-16T20:05:09.093 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.090+0000 7ffa9a209640 1 -- 172.21.15.88:0/2890156015 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa940b3260 msgr2=0x7ffa940b36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:09.094 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.090+0000 7ffa9a209640 1 --2- 172.21.15.88:0/2890156015 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa940b3260 0x7ffa940b36e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ffa940b0030 tx=0x7ffa8c010590 comp rx=0 tx=0).stop 2023-10-16T20:05:09.094 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.094+0000 7ffa9a209640 1 -- 172.21.15.88:0/2890156015 shutdown_connections 2023-10-16T20:05:09.094 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.094+0000 7ffa9a209640 1 --2- 172.21.15.88:0/2890156015 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ffa780664d0 0x7ffa78068990 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:09.094 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.094+0000 7ffa9a209640 1 --2- 172.21.15.88:0/2890156015 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa940b3260 0x7ffa940b36e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:09.094 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.094+0000 7ffa9a209640 1 --2- 172.21.15.88:0/2890156015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa940acc70 0x7ffa940ad810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:09.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.094+0000 7ffa9a209640 1 --2- 172.21.15.88:0/2890156015 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa80004380 0x7ffa941f22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:09.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.094+0000 7ffa9a209640 1 -- 172.21.15.88:0/2890156015 >> 172.21.15.88:0/2890156015 conn(0x7ffa94069e90 msgr2=0x7ffa940d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:09.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.094+0000 7ffa9a209640 1 -- 172.21.15.88:0/2890156015 shutdown_connections 2023-10-16T20:05:09.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.094+0000 7ffa9a209640 1 -- 172.21.15.88:0/2890156015 wait complete. 2023-10-16T20:05:09.112 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T20:05:09.256 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.254+0000 7f7c31b96640 1 Processor -- start 2023-10-16T20:05:09.256 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.254+0000 7f7c31b96640 1 -- start start 2023-10-16T20:05:09.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.258+0000 7f7c31b96640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f7c2c0cde80 0x7f7c2c0d3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:09.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.258+0000 7f7c31b96640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c2c0d0c40 con 0x7f7c2c15c9a0 2023-10-16T20:05:09.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.258+0000 7f7c31b96640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c2c0ce2d0 con 0x7f7c2c0cf230 2023-10-16T20:05:09.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.258+0000 7f7c31b96640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c2c0d37b0 con 0x7f7c2c0cde80 2023-10-16T20:05:09.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.258+0000 7f7c23fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f7c2c0cde80 0x7f7c2c0d3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:09.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.258+0000 7f7c23fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f7c2c0cde80 0x7f7c2c0d3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48144/0 (socket says 172.21.15.88:48144) 2023-10-16T20:05:09.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.258+0000 7f7c23fff640 1 -- 172.21.15.88:0/2991884290 learned_addr learned my addr 172.21.15.88:0/2991884290 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:09.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.258+0000 7f7c30b94640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f7c2c15c9a0 0x7f7c2c0d10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:48284/0 (socket says 172.21.15.88:48284) 2023-10-16T20:05:09.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.258+0000 7f7c23fff640 1 -- 172.21.15.88:0/2991884290 >> v1:172.21.15.94:6789/0 conn(0x7f7c2c0cf230 legacy=0x7f7c2c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:09.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.258+0000 7f7c23fff640 1 -- 172.21.15.88:0/2991884290 >> v1:172.21.15.88:6789/0 conn(0x7f7c2c15c9a0 legacy=0x7f7c2c0d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:09.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.258+0000 7f7c23fff640 1 -- 172.21.15.88:0/2991884290 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c2c0d0630 con 0x7f7c2c0cde80 2023-10-16T20:05:09.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.258+0000 7f7c237fe640 1 -- 172.21.15.88:0/2991884290 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 126068400 0 0) 0x7f7c2c0d0c40 con 0x7f7c2c15c9a0 2023-10-16T20:05:09.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.258+0000 7f7c23fff640 1 --2- 172.21.15.88:0/2991884290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c2c0cde80 0x7f7c2c0d3270 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c009870 tx=0x7f7c1c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15c8c4fe88d78510 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:09.261 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.262+0000 7f7c237fe640 1 -- 172.21.15.88:0/2991884290 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c0ad660 con 0x7f7c2c0cde80 2023-10-16T20:05:09.261 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.262+0000 7f7c237fe640 1 -- 172.21.15.88:0/2991884290 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1c0ad7c0 con 0x7f7c2c0cde80 2023-10-16T20:05:09.261 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.262+0000 7f7c237fe640 1 -- 172.21.15.88:0/2991884290 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c0adb50 con 0x7f7c2c0cde80 2023-10-16T20:05:09.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.262+0000 7f7c31b96640 1 -- 172.21.15.88:0/2991884290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c2c0cde80 msgr2=0x7f7c2c0d3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:09.263 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.262+0000 7f7c31b96640 1 --2- 172.21.15.88:0/2991884290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c2c0cde80 0x7f7c2c0d3270 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c009870 tx=0x7f7c1c01c470 comp rx=0 tx=0).stop 2023-10-16T20:05:09.263 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.262+0000 7f7c31b96640 1 -- 172.21.15.88:0/2991884290 shutdown_connections 2023-10-16T20:05:09.263 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.262+0000 7f7c31b96640 1 --2- 172.21.15.88:0/2991884290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c2c0cde80 0x7f7c2c0d3270 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:09.263 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.262+0000 7f7c31b96640 1 -- 172.21.15.88:0/2991884290 >> 172.21.15.88:0/2991884290 conn(0x7f7c2c09bc30 msgr2=0x7f7c2c0d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:09.263 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.262+0000 7f7c31b96640 1 -- 172.21.15.88:0/2991884290 shutdown_connections 2023-10-16T20:05:09.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.262+0000 7f7c31b96640 1 -- 172.21.15.88:0/2991884290 wait complete. 2023-10-16T20:05:09.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.262+0000 7f7c31b96640 1 Processor -- start 2023-10-16T20:05:09.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.262+0000 7f7c31b96640 1 -- start start 2023-10-16T20:05:09.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.262+0000 7f7c31b96640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c2c0cde80 0x7f7c2c15b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:09.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.262+0000 7f7c31b96640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c2c0cf230 0x7f7c2c0b80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:09.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.262+0000 7f7c31b96640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7c2c15c9a0 0x7f7c2c159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:09.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.262+0000 7f7c31b96640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c159f80 con 0x7f7c2c0cde80 2023-10-16T20:05:09.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.262+0000 7f7c31b96640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c15a260 con 0x7f7c2c0cf230 2023-10-16T20:05:09.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.262+0000 7f7c31b96640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c15a540 con 0x7f7c2c15c9a0 2023-10-16T20:05:09.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.262+0000 7f7c30b94640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c2c0cde80 0x7f7c2c15b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:09.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.262+0000 7f7c30b94640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c2c0cde80 0x7f7c2c15b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48150/0 (socket says 172.21.15.88:48150) 2023-10-16T20:05:09.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.262+0000 7f7c30b94640 1 -- 172.21.15.88:0/1829415545 learned_addr learned my addr 172.21.15.88:0/1829415545 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:09.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.262+0000 7f7c23fff640 1 --2- 172.21.15.88:0/1829415545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c2c0cf230 0x7f7c2c0b80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:09.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.266+0000 7f7c31395640 1 --2- 172.21.15.88:0/1829415545 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7c2c15c9a0 0x7f7c2c159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:09.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.266+0000 7f7c30b94640 1 -- 172.21.15.88:0/1829415545 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7c2c15c9a0 msgr2=0x7f7c2c159750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:09.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.266+0000 7f7c30b94640 1 --2- 172.21.15.88:0/1829415545 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7c2c15c9a0 0x7f7c2c159750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:09.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.266+0000 7f7c30b94640 1 -- 172.21.15.88:0/1829415545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c2c0cf230 msgr2=0x7f7c2c0b80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:09.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.266+0000 7f7c30b94640 1 --2- 172.21.15.88:0/1829415545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c2c0cf230 0x7f7c2c0b80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:09.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.266+0000 7f7c30b94640 1 -- 172.21.15.88:0/1829415545 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c1c0094d0 con 0x7f7c2c0cde80 2023-10-16T20:05:09.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.266+0000 7f7c30b94640 1 --2- 172.21.15.88:0/1829415545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c2c0cde80 0x7f7c2c15b020 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7c28001ec0 tx=0x7f7c2800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:09.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.266+0000 7f7c217fa640 1 -- 172.21.15.88:0/1829415545 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c280074e0 con 0x7f7c2c0cde80 2023-10-16T20:05:09.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.266+0000 7f7c217fa640 1 -- 172.21.15.88:0/1829415545 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c28007640 con 0x7f7c2c0cde80 2023-10-16T20:05:09.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.266+0000 7f7c31b96640 1 -- 172.21.15.88:0/1829415545 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c2c15cf00 con 0x7f7c2c0cde80 2023-10-16T20:05:09.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.266+0000 7f7c217fa640 1 -- 172.21.15.88:0/1829415545 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c280059d0 con 0x7f7c2c0cde80 2023-10-16T20:05:09.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.266+0000 7f7c31b96640 1 -- 172.21.15.88:0/1829415545 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c2c15ab00 con 0x7f7c2c0cde80 2023-10-16T20:05:09.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.266+0000 7f7c31b96640 1 -- 172.21.15.88:0/1829415545 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c2c0d17e0 con 0x7f7c2c0cde80 2023-10-16T20:05:09.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.266+0000 7f7c217fa640 1 -- 172.21.15.88:0/1829415545 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7c28005b30 con 0x7f7c2c0cde80 2023-10-16T20:05:09.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.266+0000 7f7c217fa640 1 --2- 172.21.15.88:0/1829415545 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7c180664a0 0x7f7c18068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:09.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.266+0000 7f7c217fa640 1 -- 172.21.15.88:0/1829415545 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7c28083620 con 0x7f7c2c0cde80 2023-10-16T20:05:09.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.266+0000 7f7c23fff640 1 --2- 172.21.15.88:0/1829415545 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7c180664a0 0x7f7c18068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:09.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.270+0000 7f7c23fff640 1 --2- 172.21.15.88:0/1829415545 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7c180664a0 0x7f7c18068960 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0af000 tx=0x7f7c1c0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:09.273 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.270+0000 7f7c217fa640 1 -- 172.21.15.88:0/1829415545 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7c280508f0 con 0x7f7c2c0cde80 2023-10-16T20:05:09.421 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.422+0000 7f7c31b96640 1 -- 172.21.15.88:0/1829415545 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7c2c15acf0 con 0x7f7c2c0cde80 2023-10-16T20:05:09.422 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.422+0000 7f7c217fa640 1 -- 172.21.15.88:0/1829415545 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7c280170a0 con 0x7f7c2c0cde80 2023-10-16T20:05:09.422 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2023-10-16T20:05:09.424 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.422+0000 7f7c31b96640 1 -- 172.21.15.88:0/1829415545 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7c180664a0 msgr2=0x7f7c18068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:09.424 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.422+0000 7f7c31b96640 1 --2- 172.21.15.88:0/1829415545 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7c180664a0 0x7f7c18068960 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0af000 tx=0x7f7c1c0a7000 comp rx=0 tx=0).stop 2023-10-16T20:05:09.424 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.422+0000 7f7c31b96640 1 -- 172.21.15.88:0/1829415545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c2c0cde80 msgr2=0x7f7c2c15b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:09.424 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.422+0000 7f7c31b96640 1 --2- 172.21.15.88:0/1829415545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c2c0cde80 0x7f7c2c15b020 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7c28001ec0 tx=0x7f7c2800c8e0 comp rx=0 tx=0).stop 2023-10-16T20:05:09.424 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.422+0000 7f7c31b96640 1 -- 172.21.15.88:0/1829415545 shutdown_connections 2023-10-16T20:05:09.425 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.422+0000 7f7c31b96640 1 --2- 172.21.15.88:0/1829415545 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7c180664a0 0x7f7c18068960 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:09.425 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.422+0000 7f7c31b96640 1 --2- 172.21.15.88:0/1829415545 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7c2c15c9a0 0x7f7c2c159750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:09.425 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.422+0000 7f7c31b96640 1 --2- 172.21.15.88:0/1829415545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c2c0cf230 0x7f7c2c0b80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:09.425 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.422+0000 7f7c31b96640 1 --2- 172.21.15.88:0/1829415545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c2c0cde80 0x7f7c2c15b020 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:09.425 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.422+0000 7f7c31b96640 1 -- 172.21.15.88:0/1829415545 >> 172.21.15.88:0/1829415545 conn(0x7f7c2c09bc30 msgr2=0x7f7c2c0b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:09.426 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.422+0000 7f7c31b96640 1 -- 172.21.15.88:0/1829415545 shutdown_connections 2023-10-16T20:05:09.426 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.422+0000 7f7c31b96640 1 -- 172.21.15.88:0/1829415545 wait complete. 2023-10-16T20:05:09.442 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T20:05:09.585 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.582+0000 7f27ef394640 1 Processor -- start 2023-10-16T20:05:09.585 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.582+0000 7f27ef394640 1 -- start start 2023-10-16T20:05:09.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.586+0000 7f27ef394640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f27e80addf0 0x7f27e80ae230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:09.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.586+0000 7f27ef394640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f27e8070250 0x7f27e8070700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:09.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.586+0000 7f27ef394640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27e80afff0 con 0x7f27e8164570 2023-10-16T20:05:09.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.586+0000 7f27ef394640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e8070c40 con 0x7f27e8070250 2023-10-16T20:05:09.588 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.586+0000 7f27ef394640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e80ae840 con 0x7f27e80addf0 2023-10-16T20:05:09.588 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.586+0000 7f27ed931640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f27e8070250 0x7f27e8070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:09.588 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.586+0000 7f27ed931640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f27e8070250 0x7f27e8070700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48164/0 (socket says 172.21.15.88:48164) 2023-10-16T20:05:09.588 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.586+0000 7f27ed931640 1 -- 172.21.15.88:0/3767033745 learned_addr learned my addr 172.21.15.88:0/3767033745 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:09.588 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.586+0000 7f27ec92f640 1 --2- 172.21.15.88:0/3767033745 >> v2:172.21.15.94:3300/0 conn(0x7f27e80addf0 0x7f27e80ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:09.589 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.586+0000 7f27ed931640 1 -- 172.21.15.88:0/3767033745 >> v2:172.21.15.94:3300/0 conn(0x7f27e80addf0 msgr2=0x7f27e80ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:09.589 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.586+0000 7f27ed931640 1 --2- 172.21.15.88:0/3767033745 >> v2:172.21.15.94:3300/0 conn(0x7f27e80addf0 0x7f27e80ae230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:09.589 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.586+0000 7f27ed931640 1 -- 172.21.15.88:0/3767033745 >> v1:172.21.15.94:6789/0 conn(0x7f27e8164570 legacy=0x7f27e80af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:09.589 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.586+0000 7f27ed931640 1 -- 172.21.15.88:0/3767033745 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e80aeb20 con 0x7f27e8070250 2023-10-16T20:05:09.590 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.590+0000 7f27ed931640 1 --2- 172.21.15.88:0/3767033745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27e8070250 0x7f27e8070700 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f27e80b13d0 tx=0x7f27e001f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=129eb5e218f38e14 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:09.590 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.590+0000 7f27dffff640 1 -- 172.21.15.88:0/3767033745 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e00af400 con 0x7f27e8070250 2023-10-16T20:05:09.590 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.590+0000 7f27dffff640 1 -- 172.21.15.88:0/3767033745 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e00ab050 con 0x7f27e8070250 2023-10-16T20:05:09.591 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.590+0000 7f27dffff640 1 -- 172.21.15.88:0/3767033745 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e00af780 con 0x7f27e8070250 2023-10-16T20:05:09.591 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.590+0000 7f27ef394640 1 -- 172.21.15.88:0/3767033745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27e8070250 msgr2=0x7f27e8070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:09.592 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.590+0000 7f27ef394640 1 --2- 172.21.15.88:0/3767033745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27e8070250 0x7f27e8070700 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f27e80b13d0 tx=0x7f27e001f1c0 comp rx=0 tx=0).stop 2023-10-16T20:05:09.592 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.590+0000 7f27ef394640 1 -- 172.21.15.88:0/3767033745 shutdown_connections 2023-10-16T20:05:09.592 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.590+0000 7f27ef394640 1 --2- 172.21.15.88:0/3767033745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27e8070250 0x7f27e8070700 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:09.592 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.590+0000 7f27ef394640 1 --2- 172.21.15.88:0/3767033745 >> v2:172.21.15.94:3300/0 conn(0x7f27e80addf0 0x7f27e80ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:09.592 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.590+0000 7f27ef394640 1 -- 172.21.15.88:0/3767033745 >> 172.21.15.88:0/3767033745 conn(0x7f27e80cb080 msgr2=0x7f27e80cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:09.592 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.590+0000 7f27ef394640 1 -- 172.21.15.88:0/3767033745 shutdown_connections 2023-10-16T20:05:09.593 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.590+0000 7f27ef394640 1 -- 172.21.15.88:0/3767033745 wait complete. 2023-10-16T20:05:09.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.590+0000 7f27ef394640 1 Processor -- start 2023-10-16T20:05:09.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.590+0000 7f27ef394640 1 -- start start 2023-10-16T20:05:09.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.594+0000 7f27ef394640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f27e8070250 0x7f27e8158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:09.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.594+0000 7f27ef394640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27e80addf0 0x7f27e81f66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:09.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.594+0000 7f27ef394640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27e8164570 0x7f27e8156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:09.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.594+0000 7f27ef394640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e81f7740 con 0x7f27e80addf0 2023-10-16T20:05:09.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.594+0000 7f27ef394640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e81f6c20 con 0x7f27e8164570 2023-10-16T20:05:09.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.594+0000 7f27ef394640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e8156b70 con 0x7f27e8070250 2023-10-16T20:05:09.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.594+0000 7f27ec92f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27e80addf0 0x7f27e81f66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:09.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.594+0000 7f27ec92f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27e80addf0 0x7f27e81f66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48166/0 (socket says 172.21.15.88:48166) 2023-10-16T20:05:09.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.594+0000 7f27ec92f640 1 -- 172.21.15.88:0/1788203775 learned_addr learned my addr 172.21.15.88:0/1788203775 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:09.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.594+0000 7f27ed931640 1 --2- 172.21.15.88:0/1788203775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27e8164570 0x7f27e8156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:09.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.594+0000 7f27ed130640 1 --2- 172.21.15.88:0/1788203775 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f27e8070250 0x7f27e8158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:09.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.594+0000 7f27ec92f640 1 -- 172.21.15.88:0/1788203775 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f27e8070250 msgr2=0x7f27e8158730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:09.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.594+0000 7f27ec92f640 1 --2- 172.21.15.88:0/1788203775 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f27e8070250 0x7f27e8158730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:09.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.594+0000 7f27ec92f640 1 -- 172.21.15.88:0/1788203775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27e8164570 msgr2=0x7f27e8156630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:09.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.594+0000 7f27ec92f640 1 --2- 172.21.15.88:0/1788203775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27e8164570 0x7f27e8156630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:09.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.594+0000 7f27ec92f640 1 -- 172.21.15.88:0/1788203775 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e0009b30 con 0x7f27e80addf0 2023-10-16T20:05:09.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.594+0000 7f27ed130640 1 --2- 172.21.15.88:0/1788203775 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f27e8070250 0x7f27e8158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:09.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.594+0000 7f27ec92f640 1 --2- 172.21.15.88:0/1788203775 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27e80addf0 0x7f27e81f66e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f27d800e840 tx=0x7f27d800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:09.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.594+0000 7f27ddffb640 1 -- 172.21.15.88:0/1788203775 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d8009510 con 0x7f27e80addf0 2023-10-16T20:05:09.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.594+0000 7f27ddffb640 1 -- 172.21.15.88:0/1788203775 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d800ee90 con 0x7f27e80addf0 2023-10-16T20:05:09.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.594+0000 7f27ddffb640 1 -- 172.21.15.88:0/1788203775 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d800cb60 con 0x7f27e80addf0 2023-10-16T20:05:09.600 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.594+0000 7f27ef394640 1 -- 172.21.15.88:0/1788203775 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e81f6ea0 con 0x7f27e80addf0 2023-10-16T20:05:09.600 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.594+0000 7f27ef394640 1 -- 172.21.15.88:0/1788203775 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e81f7460 con 0x7f27e80addf0 2023-10-16T20:05:09.601 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.594+0000 7f27ef394640 1 -- 172.21.15.88:0/1788203775 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e80cb6e0 con 0x7f27e80addf0 2023-10-16T20:05:09.601 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.594+0000 7f27ddffb640 1 -- 172.21.15.88:0/1788203775 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f27d800ccc0 con 0x7f27e80addf0 2023-10-16T20:05:09.601 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.598+0000 7f27ddffb640 1 --2- 172.21.15.88:0/1788203775 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f27cc0664d0 0x7f27cc068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:09.601 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.598+0000 7f27ddffb640 1 -- 172.21.15.88:0/1788203775 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f27d8084150 con 0x7f27e80addf0 2023-10-16T20:05:09.602 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.598+0000 7f27ed130640 1 --2- 172.21.15.88:0/1788203775 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f27cc0664d0 0x7f27cc068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:09.602 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.598+0000 7f27ed130640 1 --2- 172.21.15.88:0/1788203775 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f27cc0664d0 0x7f27cc068990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f27e8160df0 tx=0x7f27e4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:09.602 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.602+0000 7f27ddffb640 1 -- 172.21.15.88:0/1788203775 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f27d8051380 con 0x7f27e80addf0 2023-10-16T20:05:09.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.750+0000 7f27ef394640 1 -- 172.21.15.88:0/1788203775 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f27e804e9c0 con 0x7f27e80addf0 2023-10-16T20:05:09.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.750+0000 7f27ddffb640 1 -- 172.21.15.88:0/1788203775 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f27d80549a0 con 0x7f27e80addf0 2023-10-16T20:05:09.751 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:05:09.752 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.750+0000 7f27ef394640 1 -- 172.21.15.88:0/1788203775 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f27cc0664d0 msgr2=0x7f27cc068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:09.753 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.750+0000 7f27ef394640 1 --2- 172.21.15.88:0/1788203775 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f27cc0664d0 0x7f27cc068990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f27e8160df0 tx=0x7f27e4009040 comp rx=0 tx=0).stop 2023-10-16T20:05:09.753 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.750+0000 7f27ef394640 1 -- 172.21.15.88:0/1788203775 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27e80addf0 msgr2=0x7f27e81f66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:09.753 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.750+0000 7f27ef394640 1 --2- 172.21.15.88:0/1788203775 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27e80addf0 0x7f27e81f66e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f27d800e840 tx=0x7f27d800ec40 comp rx=0 tx=0).stop 2023-10-16T20:05:09.753 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.750+0000 7f27ef394640 1 -- 172.21.15.88:0/1788203775 shutdown_connections 2023-10-16T20:05:09.753 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.750+0000 7f27ef394640 1 --2- 172.21.15.88:0/1788203775 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f27cc0664d0 0x7f27cc068990 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:09.754 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.750+0000 7f27ef394640 1 --2- 172.21.15.88:0/1788203775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27e8164570 0x7f27e8156630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:09.754 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.750+0000 7f27ef394640 1 --2- 172.21.15.88:0/1788203775 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27e80addf0 0x7f27e81f66e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:09.754 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.750+0000 7f27ef394640 1 --2- 172.21.15.88:0/1788203775 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f27e8070250 0x7f27e8158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:09.754 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.750+0000 7f27ef394640 1 -- 172.21.15.88:0/1788203775 >> 172.21.15.88:0/1788203775 conn(0x7f27e80cb080 msgr2=0x7f27e806d170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:09.754 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.750+0000 7f27ef394640 1 -- 172.21.15.88:0/1788203775 shutdown_connections 2023-10-16T20:05:09.755 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.750+0000 7f27ef394640 1 -- 172.21.15.88:0/1788203775 wait complete. 2023-10-16T20:05:09.773 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T20:05:09.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.914+0000 7f24a40ea640 1 Processor -- start 2023-10-16T20:05:09.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.914+0000 7f24a40ea640 1 -- start start 2023-10-16T20:05:09.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.914+0000 7f24a40ea640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f249c0afdf0 0x7f249c0aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:09.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.914+0000 7f24a40ea640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f249c162580 con 0x7f249c069e90 2023-10-16T20:05:09.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.914+0000 7f24a40ea640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f249c0ae830 con 0x7f249c0cc830 2023-10-16T20:05:09.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.914+0000 7f24a40ea640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f249c0adf00 con 0x7f249c0afdf0 2023-10-16T20:05:09.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.914+0000 7f24a1e86640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f249c0afdf0 0x7f249c0aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:09.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.914+0000 7f24a1e86640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f249c0afdf0 0x7f249c0aecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48174/0 (socket says 172.21.15.88:48174) 2023-10-16T20:05:09.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.914+0000 7f24a1e86640 1 -- 172.21.15.88:0/1393003143 learned_addr learned my addr 172.21.15.88:0/1393003143 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:09.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.914+0000 7f24a1685640 1 --1- 172.21.15.88:0/1393003143 >> v1:172.21.15.88:6789/0 conn(0x7f249c069e90 0x7f249c0cc120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:48298/0 (socket says 172.21.15.88:48298) 2023-10-16T20:05:09.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.914+0000 7f24a1e86640 1 -- 172.21.15.88:0/1393003143 >> v1:172.21.15.94:6789/0 conn(0x7f249c0cc830 legacy=0x7f249c161810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:09.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.914+0000 7f24a1e86640 1 -- 172.21.15.88:0/1393003143 >> v1:172.21.15.88:6789/0 conn(0x7f249c069e90 legacy=0x7f249c0cc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:09.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.914+0000 7f24a1e86640 1 -- 172.21.15.88:0/1393003143 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f249c0ae1e0 con 0x7f249c0afdf0 2023-10-16T20:05:09.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.914+0000 7f24a0e84640 1 -- 172.21.15.88:0/1393003143 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 318245817 0 0) 0x7f249c162580 con 0x7f249c069e90 2023-10-16T20:05:09.919 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.918+0000 7f24a1e86640 1 --2- 172.21.15.88:0/1393003143 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f249c0afdf0 0x7f249c0aecc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2498009870 tx=0x7f249801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae070d7969f1d152 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:09.919 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.918+0000 7f24a0e84640 1 -- 172.21.15.88:0/1393003143 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24980ab600 con 0x7f249c0afdf0 2023-10-16T20:05:09.919 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.918+0000 7f24a0e84640 1 -- 172.21.15.88:0/1393003143 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24980a7050 con 0x7f249c0afdf0 2023-10-16T20:05:09.919 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.918+0000 7f24a0e84640 1 -- 172.21.15.88:0/1393003143 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24980ab980 con 0x7f249c0afdf0 2023-10-16T20:05:09.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.918+0000 7f24a40ea640 1 -- 172.21.15.88:0/1393003143 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f249c0afdf0 msgr2=0x7f249c0aecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:09.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.918+0000 7f24a40ea640 1 --2- 172.21.15.88:0/1393003143 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f249c0afdf0 0x7f249c0aecc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2498009870 tx=0x7f249801c420 comp rx=0 tx=0).stop 2023-10-16T20:05:09.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.918+0000 7f24a40ea640 1 -- 172.21.15.88:0/1393003143 shutdown_connections 2023-10-16T20:05:09.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.918+0000 7f24a40ea640 1 --2- 172.21.15.88:0/1393003143 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f249c0afdf0 0x7f249c0aecc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:09.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.918+0000 7f24a40ea640 1 -- 172.21.15.88:0/1393003143 >> 172.21.15.88:0/1393003143 conn(0x7f249c0ce5e0 msgr2=0x7f249c0ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:09.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.918+0000 7f24a40ea640 1 -- 172.21.15.88:0/1393003143 shutdown_connections 2023-10-16T20:05:09.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.918+0000 7f24a40ea640 1 -- 172.21.15.88:0/1393003143 wait complete. 2023-10-16T20:05:09.922 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.922+0000 7f24a40ea640 1 Processor -- start 2023-10-16T20:05:09.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.922+0000 7f24a40ea640 1 -- start start 2023-10-16T20:05:09.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.922+0000 7f24a40ea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f249c069e90 0x7f249c154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:09.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.922+0000 7f24a40ea640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f249c0afdf0 0x7f249c15bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:09.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.922+0000 7f24a40ea640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f249c0cc830 0x7f249c15c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:09.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.922+0000 7f24a40ea640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249c15d130 con 0x7f249c0cc830 2023-10-16T20:05:09.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.922+0000 7f24a40ea640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249c15d410 con 0x7f249c069e90 2023-10-16T20:05:09.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.922+0000 7f24a40ea640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f249c159dc0 con 0x7f249c0afdf0 2023-10-16T20:05:09.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.922+0000 7f24a2687640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f249c0cc830 0x7f249c15c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:09.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.922+0000 7f24a2687640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f249c0cc830 0x7f249c15c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48190/0 (socket says 172.21.15.88:48190) 2023-10-16T20:05:09.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.922+0000 7f24a2687640 1 -- 172.21.15.88:0/3045975737 learned_addr learned my addr 172.21.15.88:0/3045975737 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:09.926 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.922+0000 7f24a1e86640 1 --2- 172.21.15.88:0/3045975737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f249c069e90 0x7f249c154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:09.926 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.922+0000 7f24a1685640 1 --2- 172.21.15.88:0/3045975737 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f249c0afdf0 0x7f249c15bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:09.926 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.922+0000 7f24a2687640 1 -- 172.21.15.88:0/3045975737 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f249c0afdf0 msgr2=0x7f249c15bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:09.926 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.922+0000 7f24a2687640 1 --2- 172.21.15.88:0/3045975737 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f249c0afdf0 0x7f249c15bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:09.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.922+0000 7f24a2687640 1 -- 172.21.15.88:0/3045975737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f249c069e90 msgr2=0x7f249c154720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:09.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.922+0000 7f24a2687640 1 --2- 172.21.15.88:0/3045975737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f249c069e90 0x7f249c154720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:09.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.922+0000 7f24a2687640 1 -- 172.21.15.88:0/3045975737 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24980094d0 con 0x7f249c0cc830 2023-10-16T20:05:09.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.922+0000 7f24a2687640 1 --2- 172.21.15.88:0/3045975737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f249c0cc830 0x7f249c15c510 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f249400c730 tx=0x7f249400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:09.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.922+0000 7f24927fc640 1 -- 172.21.15.88:0/3045975737 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2494003b80 con 0x7f249c0cc830 2023-10-16T20:05:09.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.922+0000 7f24927fc640 1 -- 172.21.15.88:0/3045975737 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2494003ce0 con 0x7f249c0cc830 2023-10-16T20:05:09.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.922+0000 7f24a40ea640 1 -- 172.21.15.88:0/3045975737 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f249c159510 con 0x7f249c0cc830 2023-10-16T20:05:09.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.922+0000 7f24927fc640 1 -- 172.21.15.88:0/3045975737 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249400aa30 con 0x7f249c0cc830 2023-10-16T20:05:09.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.922+0000 7f24a40ea640 1 -- 172.21.15.88:0/3045975737 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f249c15cd50 con 0x7f249c0cc830 2023-10-16T20:05:09.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.922+0000 7f24a40ea640 1 -- 172.21.15.88:0/3045975737 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f249c15d270 con 0x7f249c0cc830 2023-10-16T20:05:09.929 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.926+0000 7f24927fc640 1 -- 172.21.15.88:0/3045975737 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f249400ab90 con 0x7f249c0cc830 2023-10-16T20:05:09.929 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.926+0000 7f24927fc640 1 --2- 172.21.15.88:0/3045975737 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f24800664a0 0x7f2480068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:09.930 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.926+0000 7f24927fc640 1 -- 172.21.15.88:0/3045975737 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2494083f40 con 0x7f249c0cc830 2023-10-16T20:05:09.930 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.926+0000 7f24a1e86640 1 --2- 172.21.15.88:0/3045975737 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f24800664a0 0x7f2480068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:09.930 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.926+0000 7f24a1e86640 1 --2- 172.21.15.88:0/3045975737 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f24800664a0 0x7f2480068960 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f24980ad000 tx=0x7f2498008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:09.931 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:09.930+0000 7f24927fc640 1 -- 172.21.15.88:0/3045975737 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f249c15d270 con 0x7f249c0cc830 2023-10-16T20:05:10.092 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.090+0000 7f24a40ea640 1 -- 172.21.15.88:0/3045975737 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f249c15d270 con 0x7f249c0cc830 2023-10-16T20:05:10.093 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.090+0000 7f24927fc640 1 -- 172.21.15.88:0/3045975737 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f249c15d270 con 0x7f249c0cc830 2023-10-16T20:05:10.093 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2023-10-16T20:05:10.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.094+0000 7f24a40ea640 1 -- 172.21.15.88:0/3045975737 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f24800664a0 msgr2=0x7f2480068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:10.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.094+0000 7f24a40ea640 1 --2- 172.21.15.88:0/3045975737 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f24800664a0 0x7f2480068960 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f24980ad000 tx=0x7f2498008040 comp rx=0 tx=0).stop 2023-10-16T20:05:10.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.094+0000 7f24a40ea640 1 -- 172.21.15.88:0/3045975737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f249c0cc830 msgr2=0x7f249c15c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:10.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.094+0000 7f24a40ea640 1 --2- 172.21.15.88:0/3045975737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f249c0cc830 0x7f249c15c510 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f249400c730 tx=0x7f249400cbf0 comp rx=0 tx=0).stop 2023-10-16T20:05:10.096 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.094+0000 7f24a40ea640 1 -- 172.21.15.88:0/3045975737 shutdown_connections 2023-10-16T20:05:10.096 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.094+0000 7f24a40ea640 1 --2- 172.21.15.88:0/3045975737 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f24800664a0 0x7f2480068960 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:10.096 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.094+0000 7f24a40ea640 1 --2- 172.21.15.88:0/3045975737 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f249c0afdf0 0x7f249c15bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:10.096 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.094+0000 7f24a40ea640 1 --2- 172.21.15.88:0/3045975737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f249c0cc830 0x7f249c15c510 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:10.096 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.094+0000 7f24a40ea640 1 --2- 172.21.15.88:0/3045975737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f249c069e90 0x7f249c154720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:10.096 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.094+0000 7f24a40ea640 1 -- 172.21.15.88:0/3045975737 >> 172.21.15.88:0/3045975737 conn(0x7f249c0ce5e0 msgr2=0x7f249c0cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:10.097 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.094+0000 7f24a40ea640 1 -- 172.21.15.88:0/3045975737 shutdown_connections 2023-10-16T20:05:10.097 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.094+0000 7f24a40ea640 1 -- 172.21.15.88:0/3045975737 wait complete. 2023-10-16T20:05:10.113 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:05:10.114 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:05:10.123 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:05:10.304 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.302+0000 7ff1fb7d9640 1 Processor -- start 2023-10-16T20:05:10.305 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.302+0000 7ff1fb7d9640 1 -- start start 2023-10-16T20:05:10.306 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.306+0000 7ff1fb7d9640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff1f41610a0 0x7ff1f40ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:10.306 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.306+0000 7ff1fb7d9640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff1f40ae710 0x7ff1f40cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:10.307 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.306+0000 7ff1fb7d9640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1f40cc570 con 0x7ff1f4160cc0 2023-10-16T20:05:10.307 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.306+0000 7ff1fb7d9640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1f40cc850 con 0x7ff1f41610a0 2023-10-16T20:05:10.307 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.306+0000 7ff1fb7d9640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1f40ccb30 con 0x7ff1f40ae710 2023-10-16T20:05:10.307 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.306+0000 7ff1f8d74640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff1f41610a0 0x7ff1f40ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:10.308 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.306+0000 7ff1f9575640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7ff1f4160cc0 0x7ff1f40ca800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:48310/0 (socket says 172.21.15.88:48310) 2023-10-16T20:05:10.308 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.306+0000 7ff1f9575640 1 -- 172.21.15.88:0/3360045432 learned_addr learned my addr 172.21.15.88:0/3360045432 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:10.308 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.306+0000 7ff1f9d76640 1 --2- 172.21.15.88:0/3360045432 >> v2:172.21.15.94:3300/0 conn(0x7ff1f40ae710 0x7ff1f40cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:10.308 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.306+0000 7ff1f8d74640 1 -- 172.21.15.88:0/3360045432 >> v2:172.21.15.94:3300/0 conn(0x7ff1f40ae710 msgr2=0x7ff1f40cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:10.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.306+0000 7ff1f8d74640 1 --2- 172.21.15.88:0/3360045432 >> v2:172.21.15.94:3300/0 conn(0x7ff1f40ae710 0x7ff1f40cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:10.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.306+0000 7ff1f8d74640 1 -- 172.21.15.88:0/3360045432 >> v1:172.21.15.88:6789/0 conn(0x7ff1f4160cc0 legacy=0x7ff1f40ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:10.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.306+0000 7ff1f8d74640 1 -- 172.21.15.88:0/3360045432 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f40cce10 con 0x7ff1f41610a0 2023-10-16T20:05:10.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.306+0000 7ff1ebfff640 1 -- 172.21.15.88:0/3360045432 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1023131359 0 0) 0x7ff1f40cc570 con 0x7ff1f4160cc0 2023-10-16T20:05:10.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.306+0000 7ff1f9d76640 1 --2- 172.21.15.88:0/3360045432 >> v2:172.21.15.94:3300/0 conn(0x7ff1f40ae710 0x7ff1f40cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:10.311 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.310+0000 7ff1f8d74640 1 --2- 172.21.15.88:0/3360045432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1f41610a0 0x7ff1f40ae1d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff1f40b0e10 tx=0x7ff1e401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee13d595780b040f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:10.311 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.310+0000 7ff1ebfff640 1 -- 172.21.15.88:0/3360045432 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e40ab600 con 0x7ff1f41610a0 2023-10-16T20:05:10.311 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.310+0000 7ff1ebfff640 1 -- 172.21.15.88:0/3360045432 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e40a7050 con 0x7ff1f41610a0 2023-10-16T20:05:10.311 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.310+0000 7ff1ebfff640 1 -- 172.21.15.88:0/3360045432 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e40ab980 con 0x7ff1f41610a0 2023-10-16T20:05:10.311 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.310+0000 7ff1fb7d9640 1 -- 172.21.15.88:0/3360045432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1f41610a0 msgr2=0x7ff1f40ae1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:10.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.310+0000 7ff1fb7d9640 1 --2- 172.21.15.88:0/3360045432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1f41610a0 0x7ff1f40ae1d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff1f40b0e10 tx=0x7ff1e401c3d0 comp rx=0 tx=0).stop 2023-10-16T20:05:10.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.310+0000 7ff1fb7d9640 1 -- 172.21.15.88:0/3360045432 shutdown_connections 2023-10-16T20:05:10.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.310+0000 7ff1fb7d9640 1 --2- 172.21.15.88:0/3360045432 >> v2:172.21.15.94:3300/0 conn(0x7ff1f40ae710 0x7ff1f40cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:10.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.310+0000 7ff1fb7d9640 1 --2- 172.21.15.88:0/3360045432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1f41610a0 0x7ff1f40ae1d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:10.313 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.310+0000 7ff1fb7d9640 1 -- 172.21.15.88:0/3360045432 >> 172.21.15.88:0/3360045432 conn(0x7ff1f40ce5e0 msgr2=0x7ff1f40ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:10.313 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.310+0000 7ff1fb7d9640 1 -- 172.21.15.88:0/3360045432 shutdown_connections 2023-10-16T20:05:10.313 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.310+0000 7ff1fb7d9640 1 -- 172.21.15.88:0/3360045432 wait complete. 2023-10-16T20:05:10.315 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.310+0000 7ff1fb7d9640 1 Processor -- start 2023-10-16T20:05:10.315 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.310+0000 7ff1fb7d9640 1 -- start start 2023-10-16T20:05:10.316 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.314+0000 7ff1fb7d9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1f40ae710 0x7ff1f4070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:10.316 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.314+0000 7ff1fb7d9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1f4160cc0 0x7ff1f41507b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:10.316 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.314+0000 7ff1fb7d9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f41610a0 0x7ff1f4150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:10.316 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.314+0000 7ff1fb7d9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f4151230 con 0x7ff1f40ae710 2023-10-16T20:05:10.316 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.314+0000 7ff1fb7d9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f40b1ec0 con 0x7ff1f41610a0 2023-10-16T20:05:10.317 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.314+0000 7ff1fb7d9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f40b21a0 con 0x7ff1f4160cc0 2023-10-16T20:05:10.317 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.314+0000 7ff1f9575640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1f40ae710 0x7ff1f4070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:10.317 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.314+0000 7ff1f9575640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1f40ae710 0x7ff1f4070770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48216/0 (socket says 172.21.15.88:48216) 2023-10-16T20:05:10.317 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.314+0000 7ff1f9575640 1 -- 172.21.15.88:0/363022968 learned_addr learned my addr 172.21.15.88:0/363022968 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:10.317 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.314+0000 7ff1f9d76640 1 --2- 172.21.15.88:0/363022968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f41610a0 0x7ff1f4150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:10.318 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.314+0000 7ff1f8d74640 1 --2- 172.21.15.88:0/363022968 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1f4160cc0 0x7ff1f41507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:10.318 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.314+0000 7ff1f9575640 1 -- 172.21.15.88:0/363022968 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1f4160cc0 msgr2=0x7ff1f41507b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:10.318 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.314+0000 7ff1f9575640 1 --2- 172.21.15.88:0/363022968 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1f4160cc0 0x7ff1f41507b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:10.318 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.314+0000 7ff1f9575640 1 -- 172.21.15.88:0/363022968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f41610a0 msgr2=0x7ff1f4150cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:10.318 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.314+0000 7ff1f9575640 1 --2- 172.21.15.88:0/363022968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f41610a0 0x7ff1f4150cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:10.319 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.314+0000 7ff1f9575640 1 -- 172.21.15.88:0/363022968 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e4009420 con 0x7ff1f40ae710 2023-10-16T20:05:10.319 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.314+0000 7ff1f9575640 1 --2- 172.21.15.88:0/363022968 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1f40ae710 0x7ff1f4070770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff1f00011f0 tx=0x7ff1f000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:10.319 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.314+0000 7ff1e9ffb640 1 -- 172.21.15.88:0/363022968 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f00074e0 con 0x7ff1f40ae710 2023-10-16T20:05:10.319 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.314+0000 7ff1e9ffb640 1 -- 172.21.15.88:0/363022968 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f0007640 con 0x7ff1f40ae710 2023-10-16T20:05:10.319 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.314+0000 7ff1e9ffb640 1 -- 172.21.15.88:0/363022968 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0004a10 con 0x7ff1f40ae710 2023-10-16T20:05:10.320 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.314+0000 7ff1fb7d9640 1 -- 172.21.15.88:0/363022968 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1f40b2480 con 0x7ff1f40ae710 2023-10-16T20:05:10.320 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.314+0000 7ff1fb7d9640 1 -- 172.21.15.88:0/363022968 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f40b2a40 con 0x7ff1f40ae710 2023-10-16T20:05:10.320 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.314+0000 7ff1fb7d9640 1 -- 172.21.15.88:0/363022968 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1f404e9c0 con 0x7ff1f40ae710 2023-10-16T20:05:10.321 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.314+0000 7ff1e9ffb640 1 -- 172.21.15.88:0/363022968 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff1f0004b70 con 0x7ff1f40ae710 2023-10-16T20:05:10.321 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.318+0000 7ff1e9ffb640 1 --2- 172.21.15.88:0/363022968 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff1d80664a0 0x7ff1d8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:10.321 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.318+0000 7ff1e9ffb640 1 -- 172.21.15.88:0/363022968 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff1f0083470 con 0x7ff1f40ae710 2023-10-16T20:05:10.321 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.318+0000 7ff1f8d74640 1 --2- 172.21.15.88:0/363022968 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff1d80664a0 0x7ff1d8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:10.321 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.318+0000 7ff1f8d74640 1 --2- 172.21.15.88:0/363022968 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff1d80664a0 0x7ff1d8068960 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff1e40ad000 tx=0x7ff1e40a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:10.322 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.322+0000 7ff1e9ffb640 1 -- 172.21.15.88:0/363022968 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff1f0050600 con 0x7ff1f40ae710 2023-10-16T20:05:10.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.474+0000 7ff1fb7d9640 1 -- 172.21.15.88:0/363022968 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1f406ad70 con 0x7ff1f40ae710 2023-10-16T20:05:10.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.474+0000 7ff1e9ffb640 1 -- 172.21.15.88:0/363022968 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18274 (secure 0 0 0) 0x7ff1f0018090 con 0x7ff1f40ae710 2023-10-16T20:05:10.475 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:10.475 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:05:01.835630+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T20:04:54.442487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T20:04:55.766641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T20:04:59.475830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:05:10.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.474+0000 7ff1fb7d9640 1 -- 172.21.15.88:0/363022968 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff1d80664a0 msgr2=0x7ff1d8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:10.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.474+0000 7ff1fb7d9640 1 --2- 172.21.15.88:0/363022968 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff1d80664a0 0x7ff1d8068960 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff1e40ad000 tx=0x7ff1e40a6000 comp rx=0 tx=0).stop 2023-10-16T20:05:10.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.474+0000 7ff1fb7d9640 1 -- 172.21.15.88:0/363022968 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1f40ae710 msgr2=0x7ff1f4070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:10.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.474+0000 7ff1fb7d9640 1 --2- 172.21.15.88:0/363022968 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1f40ae710 0x7ff1f4070770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff1f00011f0 tx=0x7ff1f000c9b0 comp rx=0 tx=0).stop 2023-10-16T20:05:10.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.474+0000 7ff1fb7d9640 1 -- 172.21.15.88:0/363022968 shutdown_connections 2023-10-16T20:05:10.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.474+0000 7ff1fb7d9640 1 --2- 172.21.15.88:0/363022968 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff1d80664a0 0x7ff1d8068960 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:10.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.474+0000 7ff1fb7d9640 1 --2- 172.21.15.88:0/363022968 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1f4160cc0 0x7ff1f41507b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:10.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.474+0000 7ff1fb7d9640 1 --2- 172.21.15.88:0/363022968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f41610a0 0x7ff1f4150cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:10.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.474+0000 7ff1fb7d9640 1 --2- 172.21.15.88:0/363022968 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1f40ae710 0x7ff1f4070770 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:10.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.474+0000 7ff1fb7d9640 1 -- 172.21.15.88:0/363022968 >> 172.21.15.88:0/363022968 conn(0x7ff1f40ce5e0 msgr2=0x7ff1f4162f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:10.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.474+0000 7ff1fb7d9640 1 -- 172.21.15.88:0/363022968 shutdown_connections 2023-10-16T20:05:10.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.474+0000 7ff1fb7d9640 1 -- 172.21.15.88:0/363022968 wait complete. 2023-10-16T20:05:10.496 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T20:04:54.442487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T20:04:55.766641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T20:04:59.475830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T20:05:10.497 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:05:10.637 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.634+0000 7f94a9f9d640 1 Processor -- start 2023-10-16T20:05:10.637 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.634+0000 7f94a9f9d640 1 -- start start 2023-10-16T20:05:10.639 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.638+0000 7f94a9f9d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f94a40b4260 0x7f94a415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:10.639 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.638+0000 7f94a9f9d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f94a40b0da0 0x7f94a4159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:10.639 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.638+0000 7f94a9f9d640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a415ad30 con 0x7f94a415a1f0 2023-10-16T20:05:10.639 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.638+0000 7f94a9f9d640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a40b4640 con 0x7f94a40b0da0 2023-10-16T20:05:10.639 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.638+0000 7f94a9f9d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a415a770 con 0x7f94a40b4260 2023-10-16T20:05:10.640 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.638+0000 7f949bfff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f94a40b0da0 0x7f94a4159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:10.640 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.638+0000 7f949bfff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f94a40b0da0 0x7f94a4159cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48230/0 (socket says 172.21.15.88:48230) 2023-10-16T20:05:10.640 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.638+0000 7f949bfff640 1 -- 172.21.15.88:0/1709522764 learned_addr learned my addr 172.21.15.88:0/1709522764 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:10.640 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.638+0000 7f94a37fe640 1 --2- 172.21.15.88:0/1709522764 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f94a40b4260 0x7f94a415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:10.640 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.638+0000 7f949bfff640 1 -- 172.21.15.88:0/1709522764 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f94a40b4260 msgr2=0x7f94a415dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:10.640 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.638+0000 7f949bfff640 1 --2- 172.21.15.88:0/1709522764 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f94a40b4260 0x7f94a415dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:10.641 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.638+0000 7f949bfff640 1 -- 172.21.15.88:0/1709522764 >> v1:172.21.15.94:6789/0 conn(0x7f94a415a1f0 legacy=0x7f94a406d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:10.641 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.638+0000 7f949bfff640 1 -- 172.21.15.88:0/1709522764 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a415aa50 con 0x7f94a40b0da0 2023-10-16T20:05:10.641 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.638+0000 7f94a37fe640 1 --2- 172.21.15.88:0/1709522764 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f94a40b4260 0x7f94a415dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:10.642 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.642+0000 7f949bfff640 1 --2- 172.21.15.88:0/1709522764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a40b0da0 0x7f94a4159cb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f948c00bb10 tx=0x7f948c01eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c46ac94f6f115f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:10.642 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.642+0000 7f94a2ffd640 1 -- 172.21.15.88:0/1709522764 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c0a7c20 con 0x7f94a40b0da0 2023-10-16T20:05:10.642 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.642+0000 7f94a2ffd640 1 -- 172.21.15.88:0/1709522764 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948c0a7d80 con 0x7f94a40b0da0 2023-10-16T20:05:10.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.642+0000 7f94a2ffd640 1 -- 172.21.15.88:0/1709522764 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c0af660 con 0x7f94a40b0da0 2023-10-16T20:05:10.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.642+0000 7f94a9f9d640 1 -- 172.21.15.88:0/1709522764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a40b0da0 msgr2=0x7f94a4159cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:10.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.642+0000 7f94a9f9d640 1 --2- 172.21.15.88:0/1709522764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a40b0da0 0x7f94a4159cb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f948c00bb10 tx=0x7f948c01eb50 comp rx=0 tx=0).stop 2023-10-16T20:05:10.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.642+0000 7f94a9f9d640 1 -- 172.21.15.88:0/1709522764 shutdown_connections 2023-10-16T20:05:10.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.642+0000 7f94a9f9d640 1 --2- 172.21.15.88:0/1709522764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a40b0da0 0x7f94a4159cb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:10.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.642+0000 7f94a9f9d640 1 --2- 172.21.15.88:0/1709522764 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f94a40b4260 0x7f94a415dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:10.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.642+0000 7f94a9f9d640 1 -- 172.21.15.88:0/1709522764 >> 172.21.15.88:0/1709522764 conn(0x7f94a40cb080 msgr2=0x7f94a40cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:10.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.642+0000 7f94a9f9d640 1 -- 172.21.15.88:0/1709522764 shutdown_connections 2023-10-16T20:05:10.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.642+0000 7f94a9f9d640 1 -- 172.21.15.88:0/1709522764 wait complete. 2023-10-16T20:05:10.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.642+0000 7f94a9f9d640 1 Processor -- start 2023-10-16T20:05:10.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.642+0000 7f94a9f9d640 1 -- start start 2023-10-16T20:05:10.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.646+0000 7f94a9f9d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f94a40b0da0 0x7f94a4150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:10.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.646+0000 7f94a9f9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94a40b4260 0x7f94a4148690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:10.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.646+0000 7f94a9f9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a415a1f0 0x7f94a4148e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:10.648 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.646+0000 7f94a9f9d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a415c310 con 0x7f94a415a1f0 2023-10-16T20:05:10.648 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.646+0000 7f94a9f9d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a415cbb0 con 0x7f94a40b4260 2023-10-16T20:05:10.648 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.646+0000 7f94a9f9d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a4148bd0 con 0x7f94a40b0da0 2023-10-16T20:05:10.648 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.646+0000 7f94a3fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a415a1f0 0x7f94a4148e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:10.648 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.646+0000 7f94a3fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a415a1f0 0x7f94a4148e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48240/0 (socket says 172.21.15.88:48240) 2023-10-16T20:05:10.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.646+0000 7f94a3fff640 1 -- 172.21.15.88:0/2916995374 learned_addr learned my addr 172.21.15.88:0/2916995374 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:10.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.646+0000 7f949bfff640 1 --2- 172.21.15.88:0/2916995374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94a40b4260 0x7f94a4148690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:10.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.646+0000 7f94a37fe640 1 --2- 172.21.15.88:0/2916995374 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f94a40b0da0 0x7f94a4150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:10.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.646+0000 7f94a3fff640 1 -- 172.21.15.88:0/2916995374 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f94a40b0da0 msgr2=0x7f94a4150bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:10.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.646+0000 7f94a3fff640 1 --2- 172.21.15.88:0/2916995374 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f94a40b0da0 0x7f94a4150bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:10.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.646+0000 7f94a3fff640 1 -- 172.21.15.88:0/2916995374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94a40b4260 msgr2=0x7f94a4148690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:10.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.646+0000 7f94a3fff640 1 --2- 172.21.15.88:0/2916995374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94a40b4260 0x7f94a4148690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:10.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.646+0000 7f94a3fff640 1 -- 172.21.15.88:0/2916995374 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948c00b770 con 0x7f94a415a1f0 2023-10-16T20:05:10.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.646+0000 7f94a3fff640 1 --2- 172.21.15.88:0/2916995374 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a415a1f0 0x7f94a4148e30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f949c00c730 tx=0x7f949c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:10.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.646+0000 7f94a0ff9640 1 -- 172.21.15.88:0/2916995374 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c003b80 con 0x7f94a415a1f0 2023-10-16T20:05:10.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.646+0000 7f94a0ff9640 1 -- 172.21.15.88:0/2916995374 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949c003ce0 con 0x7f94a415a1f0 2023-10-16T20:05:10.651 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.646+0000 7f94a0ff9640 1 -- 172.21.15.88:0/2916995374 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c00aa30 con 0x7f94a415a1f0 2023-10-16T20:05:10.651 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.646+0000 7f94a9f9d640 1 -- 172.21.15.88:0/2916995374 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a4151200 con 0x7f94a415a1f0 2023-10-16T20:05:10.651 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.646+0000 7f94a9f9d640 1 -- 172.21.15.88:0/2916995374 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a415c840 con 0x7f94a415a1f0 2023-10-16T20:05:10.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.646+0000 7f94a9f9d640 1 -- 172.21.15.88:0/2916995374 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a406c380 con 0x7f94a415a1f0 2023-10-16T20:05:10.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.646+0000 7f94a0ff9640 1 -- 172.21.15.88:0/2916995374 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f949c00ab90 con 0x7f94a415a1f0 2023-10-16T20:05:10.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.650+0000 7f94a0ff9640 1 --2- 172.21.15.88:0/2916995374 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f94840664a0 0x7f9484068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:10.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.650+0000 7f94a0ff9640 1 -- 172.21.15.88:0/2916995374 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f949c083e80 con 0x7f94a415a1f0 2023-10-16T20:05:10.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.650+0000 7f94a37fe640 1 --2- 172.21.15.88:0/2916995374 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f94840664a0 0x7f9484068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:10.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.650+0000 7f94a37fe640 1 --2- 172.21.15.88:0/2916995374 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f94840664a0 0x7f9484068960 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9494009950 tx=0x7f9494008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:10.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.654+0000 7f94a0ff9640 1 -- 172.21.15.88:0/2916995374 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f949c051020 con 0x7f94a415a1f0 2023-10-16T20:05:10.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.802+0000 7f94a9f9d640 1 -- 172.21.15.88:0/2916995374 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f94a415cd20 con 0x7f94a415a1f0 2023-10-16T20:05:10.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.802+0000 7f94a0ff9640 1 -- 172.21.15.88:0/2916995374 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f949c054640 con 0x7f94a415a1f0 2023-10-16T20:05:10.802 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:05:10.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.802+0000 7f94a9f9d640 1 -- 172.21.15.88:0/2916995374 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f94840664a0 msgr2=0x7f9484068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:10.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.802+0000 7f94a9f9d640 1 --2- 172.21.15.88:0/2916995374 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f94840664a0 0x7f9484068960 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9494009950 tx=0x7f9494008040 comp rx=0 tx=0).stop 2023-10-16T20:05:10.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.802+0000 7f94a9f9d640 1 -- 172.21.15.88:0/2916995374 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a415a1f0 msgr2=0x7f94a4148e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:10.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.802+0000 7f94a9f9d640 1 --2- 172.21.15.88:0/2916995374 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a415a1f0 0x7f94a4148e30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f949c00c730 tx=0x7f949c00cbf0 comp rx=0 tx=0).stop 2023-10-16T20:05:10.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.802+0000 7f94a9f9d640 1 -- 172.21.15.88:0/2916995374 shutdown_connections 2023-10-16T20:05:10.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.802+0000 7f94a9f9d640 1 --2- 172.21.15.88:0/2916995374 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f94840664a0 0x7f9484068960 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:10.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.802+0000 7f94a9f9d640 1 --2- 172.21.15.88:0/2916995374 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a415a1f0 0x7f94a4148e30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:10.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.802+0000 7f94a9f9d640 1 --2- 172.21.15.88:0/2916995374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94a40b4260 0x7f94a4148690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:10.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.802+0000 7f94a9f9d640 1 --2- 172.21.15.88:0/2916995374 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f94a40b0da0 0x7f94a4150bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:10.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.802+0000 7f94a9f9d640 1 -- 172.21.15.88:0/2916995374 >> 172.21.15.88:0/2916995374 conn(0x7f94a40cb080 msgr2=0x7f94a41552b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:10.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.802+0000 7f94a9f9d640 1 -- 172.21.15.88:0/2916995374 shutdown_connections 2023-10-16T20:05:10.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.802+0000 7f94a9f9d640 1 -- 172.21.15.88:0/2916995374 wait complete. 2023-10-16T20:05:10.823 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T20:05:10.963 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.962+0000 7f3a3046b640 1 Processor -- start 2023-10-16T20:05:10.963 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.962+0000 7f3a3046b640 1 -- start start 2023-10-16T20:05:10.964 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.962+0000 7f3a3046b640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f3a2815cae0 0x7f3a28160730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:10.965 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.962+0000 7f3a3046b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a280caec0 0x7f3a280cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:10.965 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.962+0000 7f3a3046b640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a2806b370 con 0x7f3a280b22b0 2023-10-16T20:05:10.965 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.962+0000 7f3a3046b640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a28161660 con 0x7f3a2815cae0 2023-10-16T20:05:10.965 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.962+0000 7f3a3046b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a28160dc0 con 0x7f3a280caec0 2023-10-16T20:05:10.965 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.962+0000 7f3a2e207640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f3a2815cae0 0x7f3a28160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:10.966 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.962+0000 7f3a2e207640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f3a2815cae0 0x7f3a28160730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48256/0 (socket says 172.21.15.88:48256) 2023-10-16T20:05:10.966 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.962+0000 7f3a2e207640 1 -- 172.21.15.88:0/1644510839 learned_addr learned my addr 172.21.15.88:0/1644510839 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:10.966 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.966+0000 7f3a2da06640 1 --2- 172.21.15.88:0/1644510839 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a280caec0 0x7f3a280cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:10.967 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.966+0000 7f3a2e207640 1 -- 172.21.15.88:0/1644510839 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a280caec0 msgr2=0x7f3a280cedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:10.967 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.966+0000 7f3a2e207640 1 --2- 172.21.15.88:0/1644510839 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a280caec0 0x7f3a280cedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:10.967 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.966+0000 7f3a2e207640 1 -- 172.21.15.88:0/1644510839 >> v1:172.21.15.94:6789/0 conn(0x7f3a280b22b0 legacy=0x7f3a2806b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:10.967 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.966+0000 7f3a2e207640 1 -- 172.21.15.88:0/1644510839 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a281610a0 con 0x7f3a2815cae0 2023-10-16T20:05:10.967 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.966+0000 7f3a2da06640 1 --2- 172.21.15.88:0/1644510839 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a280caec0 0x7f3a280cedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:10.968 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.966+0000 7f3a2e207640 1 --2- 172.21.15.88:0/1644510839 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a2815cae0 0x7f3a28160730 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3a24009870 tx=0x7f3a2401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1068241c88955838 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:10.968 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.966+0000 7f3a2d205640 1 -- 172.21.15.88:0/1644510839 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a240ab670 con 0x7f3a2815cae0 2023-10-16T20:05:10.969 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.966+0000 7f3a2d205640 1 -- 172.21.15.88:0/1644510839 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a240ab7d0 con 0x7f3a2815cae0 2023-10-16T20:05:10.969 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.966+0000 7f3a2d205640 1 -- 172.21.15.88:0/1644510839 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a240abb60 con 0x7f3a2815cae0 2023-10-16T20:05:10.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.966+0000 7f3a3046b640 1 -- 172.21.15.88:0/1644510839 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a2815cae0 msgr2=0x7f3a28160730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:10.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.966+0000 7f3a3046b640 1 --2- 172.21.15.88:0/1644510839 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a2815cae0 0x7f3a28160730 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3a24009870 tx=0x7f3a2401c4d0 comp rx=0 tx=0).stop 2023-10-16T20:05:10.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.970+0000 7f3a3046b640 1 -- 172.21.15.88:0/1644510839 shutdown_connections 2023-10-16T20:05:10.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.970+0000 7f3a3046b640 1 --2- 172.21.15.88:0/1644510839 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a280caec0 0x7f3a280cedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:10.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.970+0000 7f3a3046b640 1 --2- 172.21.15.88:0/1644510839 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a2815cae0 0x7f3a28160730 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:10.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.970+0000 7f3a3046b640 1 -- 172.21.15.88:0/1644510839 >> 172.21.15.88:0/1644510839 conn(0x7f3a280cd2b0 msgr2=0x7f3a280cd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:10.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.970+0000 7f3a3046b640 1 -- 172.21.15.88:0/1644510839 shutdown_connections 2023-10-16T20:05:10.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.970+0000 7f3a3046b640 1 -- 172.21.15.88:0/1644510839 wait complete. 2023-10-16T20:05:10.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.970+0000 7f3a3046b640 1 Processor -- start 2023-10-16T20:05:10.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.970+0000 7f3a3046b640 1 -- start start 2023-10-16T20:05:10.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.970+0000 7f3a3046b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a280b22b0 0x7f3a2815d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:10.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.970+0000 7f3a3046b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a280caec0 0x7f3a2815f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:10.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.970+0000 7f3a3046b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a2815cae0 0x7f3a2815f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:10.974 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.970+0000 7f3a3046b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2815b520 con 0x7f3a280caec0 2023-10-16T20:05:10.974 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.970+0000 7f3a3046b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2815fe40 con 0x7f3a2815cae0 2023-10-16T20:05:10.974 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.970+0000 7f3a3046b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2815ac80 con 0x7f3a280b22b0 2023-10-16T20:05:10.974 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.970+0000 7f3a2da06640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a280caec0 0x7f3a2815f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:10.974 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.970+0000 7f3a2da06640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a280caec0 0x7f3a2815f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48258/0 (socket says 172.21.15.88:48258) 2023-10-16T20:05:10.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.970+0000 7f3a2da06640 1 -- 172.21.15.88:0/2194169057 learned_addr learned my addr 172.21.15.88:0/2194169057 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:10.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.970+0000 7f3a2ea08640 1 --2- 172.21.15.88:0/2194169057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a2815cae0 0x7f3a2815f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:10.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.970+0000 7f3a2e207640 1 --2- 172.21.15.88:0/2194169057 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a280b22b0 0x7f3a2815d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:10.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.970+0000 7f3a2da06640 1 -- 172.21.15.88:0/2194169057 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a280b22b0 msgr2=0x7f3a2815d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:10.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.970+0000 7f3a2da06640 1 --2- 172.21.15.88:0/2194169057 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a280b22b0 0x7f3a2815d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:10.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.970+0000 7f3a2da06640 1 -- 172.21.15.88:0/2194169057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a2815cae0 msgr2=0x7f3a2815f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:10.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.970+0000 7f3a2da06640 1 --2- 172.21.15.88:0/2194169057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a2815cae0 0x7f3a2815f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:10.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.970+0000 7f3a2da06640 1 -- 172.21.15.88:0/2194169057 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a240094d0 con 0x7f3a280caec0 2023-10-16T20:05:10.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.970+0000 7f3a2da06640 1 --2- 172.21.15.88:0/2194169057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a280caec0 0x7f3a2815f3c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800c7c0 tx=0x7f3a1800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:10.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.970+0000 7f3a1effd640 1 -- 172.21.15.88:0/2194169057 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a18007a30 con 0x7f3a280caec0 2023-10-16T20:05:10.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.970+0000 7f3a1effd640 1 -- 172.21.15.88:0/2194169057 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a18007b90 con 0x7f3a280caec0 2023-10-16T20:05:10.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.970+0000 7f3a1effd640 1 -- 172.21.15.88:0/2194169057 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1800ce70 con 0x7f3a280caec0 2023-10-16T20:05:10.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.970+0000 7f3a3046b640 1 -- 172.21.15.88:0/2194169057 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a2815af60 con 0x7f3a280caec0 2023-10-16T20:05:10.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.974+0000 7f3a3046b640 1 -- 172.21.15.88:0/2194169057 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a280b35f0 con 0x7f3a280caec0 2023-10-16T20:05:10.978 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.974+0000 7f3a3046b640 1 -- 172.21.15.88:0/2194169057 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a280b3040 con 0x7f3a280caec0 2023-10-16T20:05:10.978 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.974+0000 7f3a1effd640 1 -- 172.21.15.88:0/2194169057 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3a180113f0 con 0x7f3a280caec0 2023-10-16T20:05:10.978 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.974+0000 7f3a1effd640 1 --2- 172.21.15.88:0/2194169057 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3a0c0664a0 0x7f3a0c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:10.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.974+0000 7f3a1effd640 1 -- 172.21.15.88:0/2194169057 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3a180845d0 con 0x7f3a280caec0 2023-10-16T20:05:10.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.974+0000 7f3a2e207640 1 --2- 172.21.15.88:0/2194169057 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3a0c0664a0 0x7f3a0c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:10.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.974+0000 7f3a2e207640 1 --2- 172.21.15.88:0/2194169057 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3a0c0664a0 0x7f3a0c068960 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3a240af000 tx=0x7f3a240b6040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:10.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:10.978+0000 7f3a1effd640 1 -- 172.21.15.88:0/2194169057 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3a18051800 con 0x7f3a280caec0 2023-10-16T20:05:11.129 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.126+0000 7f3a3046b640 1 -- 172.21.15.88:0/2194169057 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3a2804e9c0 con 0x7f3a280caec0 2023-10-16T20:05:11.129 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.130+0000 7f3a1effd640 1 -- 172.21.15.88:0/2194169057 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3a18054e20 con 0x7f3a280caec0 2023-10-16T20:05:11.130 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:05:11.131 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.130+0000 7f3a3046b640 1 -- 172.21.15.88:0/2194169057 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3a0c0664a0 msgr2=0x7f3a0c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:11.131 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.130+0000 7f3a3046b640 1 --2- 172.21.15.88:0/2194169057 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3a0c0664a0 0x7f3a0c068960 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3a240af000 tx=0x7f3a240b6040 comp rx=0 tx=0).stop 2023-10-16T20:05:11.132 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.130+0000 7f3a3046b640 1 -- 172.21.15.88:0/2194169057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a280caec0 msgr2=0x7f3a2815f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:11.132 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.130+0000 7f3a3046b640 1 --2- 172.21.15.88:0/2194169057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a280caec0 0x7f3a2815f3c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800c7c0 tx=0x7f3a1800cc80 comp rx=0 tx=0).stop 2023-10-16T20:05:11.132 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.130+0000 7f3a3046b640 1 -- 172.21.15.88:0/2194169057 shutdown_connections 2023-10-16T20:05:11.132 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.130+0000 7f3a3046b640 1 --2- 172.21.15.88:0/2194169057 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3a0c0664a0 0x7f3a0c068960 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:11.132 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.130+0000 7f3a3046b640 1 --2- 172.21.15.88:0/2194169057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a2815cae0 0x7f3a2815f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:11.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.130+0000 7f3a3046b640 1 --2- 172.21.15.88:0/2194169057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a280caec0 0x7f3a2815f3c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:11.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.130+0000 7f3a3046b640 1 --2- 172.21.15.88:0/2194169057 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a280b22b0 0x7f3a2815d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:11.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.130+0000 7f3a3046b640 1 -- 172.21.15.88:0/2194169057 >> 172.21.15.88:0/2194169057 conn(0x7f3a280cd2b0 msgr2=0x7f3a280cb2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:11.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.130+0000 7f3a3046b640 1 -- 172.21.15.88:0/2194169057 shutdown_connections 2023-10-16T20:05:11.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.130+0000 7f3a3046b640 1 -- 172.21.15.88:0/2194169057 wait complete. 2023-10-16T20:05:11.150 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T20:05:11.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.290+0000 7f324c5d0640 1 Processor -- start 2023-10-16T20:05:11.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.290+0000 7f324c5d0640 1 -- start start 2023-10-16T20:05:11.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.294+0000 7f324c5d0640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32440b4c40 0x7f32440b6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:11.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.294+0000 7f324c5d0640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32440b7d20 con 0x7f324415cb70 2023-10-16T20:05:11.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.294+0000 7f324c5d0640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32440b7110 con 0x7f32440d1be0 2023-10-16T20:05:11.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.294+0000 7f324c5d0640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32440b73f0 con 0x7f32440b4c40 2023-10-16T20:05:11.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.294+0000 7f324a36c640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f324415cb70 0x7f32440b83d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:48318/0 (socket says 172.21.15.88:48318) 2023-10-16T20:05:11.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.294+0000 7f324a36c640 1 -- 172.21.15.88:0/1121797420 learned_addr learned my addr 172.21.15.88:0/1121797420 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:11.295 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.294+0000 7f324ab6d640 1 --2- 172.21.15.88:0/1121797420 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32440b4c40 0x7f32440b6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:11.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.294+0000 7f324936a640 1 -- 172.21.15.88:0/1121797420 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 585101200 0 0) 0x7f32440b7d20 con 0x7f324415cb70 2023-10-16T20:05:11.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.294+0000 7f324936a640 1 -- 172.21.15.88:0/1121797420 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3230003530 con 0x7f324415cb70 2023-10-16T20:05:11.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.294+0000 7f324936a640 1 -- 172.21.15.88:0/1121797420 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3634209772 0 0) 0x7f32440b7110 con 0x7f32440d1be0 2023-10-16T20:05:11.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.294+0000 7f324936a640 1 -- 172.21.15.88:0/1121797420 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32440b7d20 con 0x7f32440d1be0 2023-10-16T20:05:11.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.294+0000 7f324ab6d640 1 -- 172.21.15.88:0/1121797420 >> v1:172.21.15.94:6789/0 conn(0x7f32440d1be0 legacy=0x7f32440b4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:11.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.294+0000 7f324ab6d640 1 -- 172.21.15.88:0/1121797420 >> v1:172.21.15.88:6789/0 conn(0x7f324415cb70 legacy=0x7f32440b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:11.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.294+0000 7f324ab6d640 1 -- 172.21.15.88:0/1121797420 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32440b76d0 con 0x7f32440b4c40 2023-10-16T20:05:11.298 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.298+0000 7f324ab6d640 1 --2- 172.21.15.88:0/1121797420 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32440b4c40 0x7f32440b6a80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f323c009dc0 tx=0x7f323c01f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25796a9676a47cbe server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:11.298 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.298+0000 7f324936a640 1 -- 172.21.15.88:0/1121797420 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c0af600 con 0x7f32440b4c40 2023-10-16T20:05:11.298 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.298+0000 7f324936a640 1 -- 172.21.15.88:0/1121797420 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323c0ab050 con 0x7f32440b4c40 2023-10-16T20:05:11.298 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.298+0000 7f324936a640 1 -- 172.21.15.88:0/1121797420 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c0af980 con 0x7f32440b4c40 2023-10-16T20:05:11.299 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.298+0000 7f324c5d0640 1 -- 172.21.15.88:0/1121797420 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32440b4c40 msgr2=0x7f32440b6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:11.299 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.298+0000 7f324c5d0640 1 --2- 172.21.15.88:0/1121797420 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32440b4c40 0x7f32440b6a80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f323c009dc0 tx=0x7f323c01f150 comp rx=0 tx=0).stop 2023-10-16T20:05:11.299 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.298+0000 7f324c5d0640 1 -- 172.21.15.88:0/1121797420 shutdown_connections 2023-10-16T20:05:11.299 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.298+0000 7f324c5d0640 1 --2- 172.21.15.88:0/1121797420 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32440b4c40 0x7f32440b6a80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:11.299 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.298+0000 7f324c5d0640 1 -- 172.21.15.88:0/1121797420 >> 172.21.15.88:0/1121797420 conn(0x7f324406b6e0 msgr2=0x7f324406baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:11.300 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.298+0000 7f324c5d0640 1 -- 172.21.15.88:0/1121797420 shutdown_connections 2023-10-16T20:05:11.300 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.298+0000 7f324c5d0640 1 -- 172.21.15.88:0/1121797420 wait complete. 2023-10-16T20:05:11.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.298+0000 7f324c5d0640 1 Processor -- start 2023-10-16T20:05:11.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.298+0000 7f324c5d0640 1 -- start start 2023-10-16T20:05:11.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.298+0000 7f324c5d0640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32440b4c40 0x7f3244157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:11.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.298+0000 7f324c5d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32440d1be0 0x7f324414cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:11.302 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.298+0000 7f324c5d0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f324415cb70 0x7f32440c6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:11.302 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.298+0000 7f324c5d0640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324414d260 con 0x7f324415cb70 2023-10-16T20:05:11.302 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.298+0000 7f324c5d0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324415d9d0 con 0x7f32440d1be0 2023-10-16T20:05:11.302 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.298+0000 7f324c5d0640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f324415ae40 con 0x7f32440b4c40 2023-10-16T20:05:11.302 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.302+0000 7f324ab6d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f324415cb70 0x7f32440c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:11.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.302+0000 7f324ab6d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f324415cb70 0x7f32440c6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48274/0 (socket says 172.21.15.88:48274) 2023-10-16T20:05:11.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.302+0000 7f324ab6d640 1 -- 172.21.15.88:0/855090418 learned_addr learned my addr 172.21.15.88:0/855090418 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:11.304 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.302+0000 7f324a36c640 1 --2- 172.21.15.88:0/855090418 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32440b4c40 0x7f3244157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:11.304 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.302+0000 7f3249b6b640 1 --2- 172.21.15.88:0/855090418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32440d1be0 0x7f324414cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:11.304 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.302+0000 7f324ab6d640 1 -- 172.21.15.88:0/855090418 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32440b4c40 msgr2=0x7f3244157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:11.305 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.302+0000 7f324ab6d640 1 --2- 172.21.15.88:0/855090418 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32440b4c40 0x7f3244157fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:11.305 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.302+0000 7f324ab6d640 1 -- 172.21.15.88:0/855090418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32440d1be0 msgr2=0x7f324414cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:11.305 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.302+0000 7f324ab6d640 1 --2- 172.21.15.88:0/855090418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32440d1be0 0x7f324414cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:11.305 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.302+0000 7f324ab6d640 1 -- 172.21.15.88:0/855090418 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323c009a70 con 0x7f324415cb70 2023-10-16T20:05:11.305 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.302+0000 7f324a36c640 1 --2- 172.21.15.88:0/855090418 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32440b4c40 0x7f3244157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:11.306 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.302+0000 7f324ab6d640 1 --2- 172.21.15.88:0/855090418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f324415cb70 0x7f32440c6890 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f323c0b2040 tx=0x7f323c01bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:11.306 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.302+0000 7f323affd640 1 -- 172.21.15.88:0/855090418 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c0afe90 con 0x7f324415cb70 2023-10-16T20:05:11.306 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.302+0000 7f323affd640 1 -- 172.21.15.88:0/855090418 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323c009450 con 0x7f324415cb70 2023-10-16T20:05:11.306 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.302+0000 7f324c5d0640 1 -- 172.21.15.88:0/855090418 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f324415ab60 con 0x7f324415cb70 2023-10-16T20:05:11.306 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.302+0000 7f323affd640 1 -- 172.21.15.88:0/855090418 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c0b9630 con 0x7f324415cb70 2023-10-16T20:05:11.306 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.302+0000 7f324c5d0640 1 -- 172.21.15.88:0/855090418 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f324415d660 con 0x7f324415cb70 2023-10-16T20:05:11.307 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.302+0000 7f324c5d0640 1 -- 172.21.15.88:0/855090418 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f324415db60 con 0x7f324415cb70 2023-10-16T20:05:11.307 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.302+0000 7f323affd640 1 -- 172.21.15.88:0/855090418 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f323c0b6050 con 0x7f324415cb70 2023-10-16T20:05:11.308 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.302+0000 7f323affd640 1 --2- 172.21.15.88:0/855090418 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f32280664d0 0x7f3228068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:11.308 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.302+0000 7f323affd640 1 -- 172.21.15.88:0/855090418 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f323c122300 con 0x7f324415cb70 2023-10-16T20:05:11.308 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.306+0000 7f324a36c640 1 --2- 172.21.15.88:0/855090418 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f32280664d0 0x7f3228068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:11.308 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.306+0000 7f324a36c640 1 --2- 172.21.15.88:0/855090418 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f32280664d0 0x7f3228068990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3240000f70 tx=0x7f3240009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:11.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.306+0000 7f323affd640 1 -- 172.21.15.88:0/855090418 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f323c0f2fd0 con 0x7f324415cb70 2023-10-16T20:05:11.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.458+0000 7f324c5d0640 1 -- 172.21.15.88:0/855090418 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f324404e9c0 con 0x7f324415cb70 2023-10-16T20:05:11.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.458+0000 7f323affd640 1 -- 172.21.15.88:0/855090418 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f323c0f2b60 con 0x7f324415cb70 2023-10-16T20:05:11.461 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2023-10-16T20:05:11.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.462+0000 7f324c5d0640 1 -- 172.21.15.88:0/855090418 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f32280664d0 msgr2=0x7f3228068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:11.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.462+0000 7f324c5d0640 1 --2- 172.21.15.88:0/855090418 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f32280664d0 0x7f3228068990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3240000f70 tx=0x7f3240009040 comp rx=0 tx=0).stop 2023-10-16T20:05:11.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.462+0000 7f324c5d0640 1 -- 172.21.15.88:0/855090418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f324415cb70 msgr2=0x7f32440c6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:11.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.462+0000 7f324c5d0640 1 --2- 172.21.15.88:0/855090418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f324415cb70 0x7f32440c6890 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f323c0b2040 tx=0x7f323c01bef0 comp rx=0 tx=0).stop 2023-10-16T20:05:11.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.462+0000 7f324c5d0640 1 -- 172.21.15.88:0/855090418 shutdown_connections 2023-10-16T20:05:11.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.462+0000 7f324c5d0640 1 --2- 172.21.15.88:0/855090418 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f32280664d0 0x7f3228068990 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:11.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.462+0000 7f324c5d0640 1 --2- 172.21.15.88:0/855090418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f324415cb70 0x7f32440c6890 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:11.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.462+0000 7f324c5d0640 1 --2- 172.21.15.88:0/855090418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32440d1be0 0x7f324414cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:11.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.462+0000 7f324c5d0640 1 --2- 172.21.15.88:0/855090418 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32440b4c40 0x7f3244157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:11.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.462+0000 7f324c5d0640 1 -- 172.21.15.88:0/855090418 >> 172.21.15.88:0/855090418 conn(0x7f324406b6e0 msgr2=0x7f32440d29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:11.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.462+0000 7f324c5d0640 1 -- 172.21.15.88:0/855090418 shutdown_connections 2023-10-16T20:05:11.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.462+0000 7f324c5d0640 1 -- 172.21.15.88:0/855090418 wait complete. 2023-10-16T20:05:11.481 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T20:05:11.621 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.622+0000 7efde9232640 1 Processor -- start 2023-10-16T20:05:11.622 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.622+0000 7efde9232640 1 -- start start 2023-10-16T20:05:11.623 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.622+0000 7efde9232640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7efde415a820 0x7efde40cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:11.623 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.622+0000 7efde9232640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7efde40b2b00 0x7efde406c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:11.624 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.622+0000 7efde9232640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efde40ce4b0 con 0x7efde415ac00 2023-10-16T20:05:11.624 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.622+0000 7efde9232640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efde415e270 con 0x7efde40b2b00 2023-10-16T20:05:11.624 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.622+0000 7efde9232640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efde40ce700 con 0x7efde415a820 2023-10-16T20:05:11.624 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.622+0000 7efde359e640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7efde40b2b00 0x7efde406c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:11.624 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.622+0000 7efde259c640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7efde415ac00 0x7efde40b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:48322/0 (socket says 172.21.15.88:48322) 2023-10-16T20:05:11.624 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.622+0000 7efde259c640 1 -- 172.21.15.88:0/3535393405 learned_addr learned my addr 172.21.15.88:0/3535393405 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:11.625 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.622+0000 7efde359e640 1 --2- 172.21.15.88:0/3535393405 >> v2:172.21.15.88:3300/0 conn(0x7efde40b2b00 0x7efde406c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48280/0 (socket says 172.21.15.88:48280) 2023-10-16T20:05:11.625 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.622+0000 7efde2d9d640 1 --2- 172.21.15.88:0/3535393405 >> v2:172.21.15.94:3300/0 conn(0x7efde415a820 0x7efde40cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:11.625 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.622+0000 7efde359e640 1 -- 172.21.15.88:0/3535393405 >> v2:172.21.15.94:3300/0 conn(0x7efde415a820 msgr2=0x7efde40cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:11.625 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.622+0000 7efde359e640 1 --2- 172.21.15.88:0/3535393405 >> v2:172.21.15.94:3300/0 conn(0x7efde415a820 0x7efde40cf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:11.625 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.622+0000 7efde359e640 1 -- 172.21.15.88:0/3535393405 >> v1:172.21.15.88:6789/0 conn(0x7efde415ac00 legacy=0x7efde40b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:11.626 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.622+0000 7efde359e640 1 -- 172.21.15.88:0/3535393405 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde40ce9e0 con 0x7efde40b2b00 2023-10-16T20:05:11.626 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.622+0000 7efde1d9b640 1 -- 172.21.15.88:0/3535393405 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3131418796 0 0) 0x7efde40ce4b0 con 0x7efde415ac00 2023-10-16T20:05:11.626 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.622+0000 7efde2d9d640 1 --2- 172.21.15.88:0/3535393405 >> v2:172.21.15.94:3300/0 conn(0x7efde415a820 0x7efde40cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:11.627 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.626+0000 7efde359e640 1 --2- 172.21.15.88:0/3535393405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efde40b2b00 0x7efde406c340 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efde40ce460 tx=0x7efddc01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7058ee93fbcc6e44 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:11.627 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.626+0000 7efde1d9b640 1 -- 172.21.15.88:0/3535393405 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde406cea0 con 0x7efde40b2b00 2023-10-16T20:05:11.628 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.626+0000 7efde1d9b640 1 -- 172.21.15.88:0/3535393405 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efddc0a59c0 con 0x7efde40b2b00 2023-10-16T20:05:11.628 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.626+0000 7efde1d9b640 1 -- 172.21.15.88:0/3535393405 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc0a5d50 con 0x7efde40b2b00 2023-10-16T20:05:11.628 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.626+0000 7efde9232640 1 -- 172.21.15.88:0/3535393405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efde40b2b00 msgr2=0x7efde406c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:11.628 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.626+0000 7efde9232640 1 --2- 172.21.15.88:0/3535393405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efde40b2b00 0x7efde406c340 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efde40ce460 tx=0x7efddc01c8e0 comp rx=0 tx=0).stop 2023-10-16T20:05:11.628 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.626+0000 7efde9232640 1 -- 172.21.15.88:0/3535393405 shutdown_connections 2023-10-16T20:05:11.629 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.626+0000 7efde9232640 1 --2- 172.21.15.88:0/3535393405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efde40b2b00 0x7efde406c340 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:11.629 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.626+0000 7efde9232640 1 --2- 172.21.15.88:0/3535393405 >> v2:172.21.15.94:3300/0 conn(0x7efde415a820 0x7efde40cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:11.629 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.626+0000 7efde9232640 1 -- 172.21.15.88:0/3535393405 >> 172.21.15.88:0/3535393405 conn(0x7efde40cd3f0 msgr2=0x7efde40cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:11.629 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.626+0000 7efde9232640 1 -- 172.21.15.88:0/3535393405 shutdown_connections 2023-10-16T20:05:11.630 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.626+0000 7efde9232640 1 -- 172.21.15.88:0/3535393405 wait complete. 2023-10-16T20:05:11.632 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.630+0000 7efde9232640 1 Processor -- start 2023-10-16T20:05:11.632 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.630+0000 7efde9232640 1 -- start start 2023-10-16T20:05:11.632 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.630+0000 7efde9232640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efde40b2b00 0x7efde415c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:11.632 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.630+0000 7efde9232640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efde415a820 0x7efde4159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:11.633 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.630+0000 7efde9232640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efde415ac00 0x7efde4159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:11.633 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.630+0000 7efde9232640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde40b1190 con 0x7efde415ac00 2023-10-16T20:05:11.633 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.630+0000 7efde9232640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde40b1470 con 0x7efde415a820 2023-10-16T20:05:11.633 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.630+0000 7efde9232640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde40b1750 con 0x7efde40b2b00 2023-10-16T20:05:11.633 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.630+0000 7efde359e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efde415ac00 0x7efde4159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:11.634 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.630+0000 7efde359e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efde415ac00 0x7efde4159790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48288/0 (socket says 172.21.15.88:48288) 2023-10-16T20:05:11.634 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.630+0000 7efde359e640 1 -- 172.21.15.88:0/1201650361 learned_addr learned my addr 172.21.15.88:0/1201650361 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:11.634 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.630+0000 7efde259c640 1 --2- 172.21.15.88:0/1201650361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efde415a820 0x7efde4159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:11.634 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.630+0000 7efde2d9d640 1 --2- 172.21.15.88:0/1201650361 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efde40b2b00 0x7efde415c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:11.634 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.630+0000 7efde359e640 1 -- 172.21.15.88:0/1201650361 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efde40b2b00 msgr2=0x7efde415c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:11.635 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.630+0000 7efde359e640 1 --2- 172.21.15.88:0/1201650361 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efde40b2b00 0x7efde415c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:11.635 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.630+0000 7efde359e640 1 -- 172.21.15.88:0/1201650361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efde415a820 msgr2=0x7efde4159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:11.635 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.630+0000 7efde359e640 1 --2- 172.21.15.88:0/1201650361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efde415a820 0x7efde4159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:11.635 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.630+0000 7efde359e640 1 -- 172.21.15.88:0/1201650361 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efddc009a70 con 0x7efde415ac00 2023-10-16T20:05:11.635 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.630+0000 7efde359e640 1 --2- 172.21.15.88:0/1201650361 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efde415ac00 0x7efde4159790 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efddc0194f0 tx=0x7efddc019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:11.636 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.630+0000 7efdcb7fe640 1 -- 172.21.15.88:0/1201650361 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc0ad9d0 con 0x7efde415ac00 2023-10-16T20:05:11.636 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.630+0000 7efdcb7fe640 1 -- 172.21.15.88:0/1201650361 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efddc0adb30 con 0x7efde415ac00 2023-10-16T20:05:11.636 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.630+0000 7efde9232640 1 -- 172.21.15.88:0/1201650361 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efde40b1a30 con 0x7efde415ac00 2023-10-16T20:05:11.636 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.630+0000 7efdcb7fe640 1 -- 172.21.15.88:0/1201650361 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc0b9450 con 0x7efde415ac00 2023-10-16T20:05:11.636 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.630+0000 7efde9232640 1 -- 172.21.15.88:0/1201650361 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efde40b0db0 con 0x7efde415ac00 2023-10-16T20:05:11.637 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.630+0000 7efde9232640 1 -- 172.21.15.88:0/1201650361 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efde404e9c0 con 0x7efde415ac00 2023-10-16T20:05:11.637 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.634+0000 7efdcb7fe640 1 -- 172.21.15.88:0/1201650361 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7efddc0b95b0 con 0x7efde415ac00 2023-10-16T20:05:11.638 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.634+0000 7efdcb7fe640 1 --2- 172.21.15.88:0/1201650361 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7efdd00664a0 0x7efdd0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:11.638 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.634+0000 7efdcb7fe640 1 -- 172.21.15.88:0/1201650361 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7efddc11f820 con 0x7efde415ac00 2023-10-16T20:05:11.638 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.634+0000 7efde2d9d640 1 --2- 172.21.15.88:0/1201650361 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7efdd00664a0 0x7efdd0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:11.638 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.634+0000 7efde2d9d640 1 --2- 172.21.15.88:0/1201650361 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7efdd00664a0 0x7efdd0068960 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efdd400a9a0 tx=0x7efdd4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:11.639 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.638+0000 7efdcb7fe640 1 -- 172.21.15.88:0/1201650361 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7efddc0ec910 con 0x7efde415ac00 2023-10-16T20:05:11.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.786+0000 7efde9232640 1 -- 172.21.15.88:0/1201650361 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7efde40b1ee0 con 0x7efde415ac00 2023-10-16T20:05:11.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.786+0000 7efdcb7fe640 1 -- 172.21.15.88:0/1201650361 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7efddc0eff30 con 0x7efde415ac00 2023-10-16T20:05:11.788 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:05:11.789 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.790+0000 7efde9232640 1 -- 172.21.15.88:0/1201650361 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7efdd00664a0 msgr2=0x7efdd0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:11.790 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.790+0000 7efde9232640 1 --2- 172.21.15.88:0/1201650361 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7efdd00664a0 0x7efdd0068960 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efdd400a9a0 tx=0x7efdd4008040 comp rx=0 tx=0).stop 2023-10-16T20:05:11.790 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.790+0000 7efde9232640 1 -- 172.21.15.88:0/1201650361 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efde415ac00 msgr2=0x7efde4159790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:11.790 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.790+0000 7efde9232640 1 --2- 172.21.15.88:0/1201650361 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efde415ac00 0x7efde4159790 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efddc0194f0 tx=0x7efddc019680 comp rx=0 tx=0).stop 2023-10-16T20:05:11.790 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.790+0000 7efde9232640 1 -- 172.21.15.88:0/1201650361 shutdown_connections 2023-10-16T20:05:11.790 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.790+0000 7efde9232640 1 --2- 172.21.15.88:0/1201650361 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7efdd00664a0 0x7efdd0068960 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:11.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.790+0000 7efde9232640 1 --2- 172.21.15.88:0/1201650361 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efde415ac00 0x7efde4159790 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:11.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.790+0000 7efde9232640 1 --2- 172.21.15.88:0/1201650361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efde415a820 0x7efde4159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:11.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.790+0000 7efde9232640 1 --2- 172.21.15.88:0/1201650361 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efde40b2b00 0x7efde415c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:11.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.790+0000 7efde9232640 1 -- 172.21.15.88:0/1201650361 >> 172.21.15.88:0/1201650361 conn(0x7efde40cd3f0 msgr2=0x7efde406ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:11.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.790+0000 7efde9232640 1 -- 172.21.15.88:0/1201650361 shutdown_connections 2023-10-16T20:05:11.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.790+0000 7efde9232640 1 -- 172.21.15.88:0/1201650361 wait complete. 2023-10-16T20:05:11.808 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T20:05:11.946 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.946+0000 7f67e953e640 1 Processor -- start 2023-10-16T20:05:11.946 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.946+0000 7f67e953e640 1 -- start start 2023-10-16T20:05:11.947 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.946+0000 7f67e953e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f67e40d2660 0x7f67e40d1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:11.947 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.946+0000 7f67e953e640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f67e40cd970 0x7f67e40cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:11.947 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.946+0000 7f67e953e640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e40b5510 con 0x7f67e4153ea0 2023-10-16T20:05:11.948 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.946+0000 7f67e953e640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67e415d9a0 con 0x7f67e40cd970 2023-10-16T20:05:11.948 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.946+0000 7f67e953e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e40ce320 con 0x7f67e40d2660 2023-10-16T20:05:11.948 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.946+0000 7f67e2ffd640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f67e4153ea0 0x7f67e415d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:48326/0 (socket says 172.21.15.88:48326) 2023-10-16T20:05:11.948 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.946+0000 7f67e2ffd640 1 -- 172.21.15.88:0/1987795164 learned_addr learned my addr 172.21.15.88:0/1987795164 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:11.948 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.946+0000 7f67e37fe640 1 --2- 172.21.15.88:0/1987795164 >> v2:172.21.15.88:3300/0 conn(0x7f67e40cd970 0x7f67e40cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:11.949 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.946+0000 7f67e27fc640 1 --2- 172.21.15.88:0/1987795164 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f67e40d2660 0x7f67e40d1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:11.949 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.946+0000 7f67e37fe640 1 -- 172.21.15.88:0/1987795164 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f67e40d2660 msgr2=0x7f67e40d1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:11.949 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.946+0000 7f67e37fe640 1 --2- 172.21.15.88:0/1987795164 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f67e40d2660 0x7f67e40d1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:11.950 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.946+0000 7f67e37fe640 1 -- 172.21.15.88:0/1987795164 >> v1:172.21.15.88:6789/0 conn(0x7f67e4153ea0 legacy=0x7f67e415d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:11.950 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.946+0000 7f67e1ffb640 1 -- 172.21.15.88:0/1987795164 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1188728573 0 0) 0x7f67e40b5510 con 0x7f67e4153ea0 2023-10-16T20:05:11.950 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.946+0000 7f67e37fe640 1 -- 172.21.15.88:0/1987795164 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e40b4990 con 0x7f67e40cd970 2023-10-16T20:05:11.951 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.950+0000 7f67e37fe640 1 --2- 172.21.15.88:0/1987795164 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67e40cd970 0x7f67e40cdde0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f67dc009d90 tx=0x7f67dc01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98a58decd7a46e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:11.951 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.950+0000 7f67e1ffb640 1 -- 172.21.15.88:0/1987795164 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc0a59e0 con 0x7f67e40cd970 2023-10-16T20:05:11.951 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.950+0000 7f67e1ffb640 1 -- 172.21.15.88:0/1987795164 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67dc0a5b40 con 0x7f67e40cd970 2023-10-16T20:05:11.951 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.950+0000 7f67e1ffb640 1 -- 172.21.15.88:0/1987795164 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc0ad440 con 0x7f67e40cd970 2023-10-16T20:05:11.952 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.950+0000 7f67e953e640 1 -- 172.21.15.88:0/1987795164 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67e40cd970 msgr2=0x7f67e40cdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:11.952 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.950+0000 7f67e953e640 1 --2- 172.21.15.88:0/1987795164 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67e40cd970 0x7f67e40cdde0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f67dc009d90 tx=0x7f67dc01c920 comp rx=0 tx=0).stop 2023-10-16T20:05:11.953 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.950+0000 7f67e953e640 1 -- 172.21.15.88:0/1987795164 shutdown_connections 2023-10-16T20:05:11.953 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.950+0000 7f67e953e640 1 --2- 172.21.15.88:0/1987795164 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67e40cd970 0x7f67e40cdde0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:11.953 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.950+0000 7f67e953e640 1 --2- 172.21.15.88:0/1987795164 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f67e40d2660 0x7f67e40d1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:11.953 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.950+0000 7f67e953e640 1 -- 172.21.15.88:0/1987795164 >> 172.21.15.88:0/1987795164 conn(0x7f67e40715c0 msgr2=0x7f67e40719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:11.953 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.950+0000 7f67e953e640 1 -- 172.21.15.88:0/1987795164 shutdown_connections 2023-10-16T20:05:11.954 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.950+0000 7f67e953e640 1 -- 172.21.15.88:0/1987795164 wait complete. 2023-10-16T20:05:11.955 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.954+0000 7f67e953e640 1 Processor -- start 2023-10-16T20:05:11.955 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.954+0000 7f67e953e640 1 -- start start 2023-10-16T20:05:11.956 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.954+0000 7f67e953e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f67e40cd970 0x7f67e4148f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:11.956 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.954+0000 7f67e953e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67e40d2660 0x7f67e414b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:11.956 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.954+0000 7f67e953e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67e4153ea0 0x7f67e40d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:11.956 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.954+0000 7f67e953e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e41479e0 con 0x7f67e4153ea0 2023-10-16T20:05:11.956 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.954+0000 7f67e953e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e4151560 con 0x7f67e40d2660 2023-10-16T20:05:11.956 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.954+0000 7f67e953e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e4151820 con 0x7f67e40cd970 2023-10-16T20:05:11.957 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.954+0000 7f67e37fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67e4153ea0 0x7f67e40d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:11.957 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.954+0000 7f67e37fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67e4153ea0 0x7f67e40d0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48312/0 (socket says 172.21.15.88:48312) 2023-10-16T20:05:11.957 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.954+0000 7f67e37fe640 1 -- 172.21.15.88:0/3808507295 learned_addr learned my addr 172.21.15.88:0/3808507295 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:11.957 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.954+0000 7f67e2ffd640 1 --2- 172.21.15.88:0/3808507295 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f67e40cd970 0x7f67e4148f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:11.957 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.954+0000 7f67e27fc640 1 --2- 172.21.15.88:0/3808507295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67e40d2660 0x7f67e414b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:11.958 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.954+0000 7f67e37fe640 1 -- 172.21.15.88:0/3808507295 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f67e40cd970 msgr2=0x7f67e4148f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:11.958 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.954+0000 7f67e37fe640 1 --2- 172.21.15.88:0/3808507295 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f67e40cd970 0x7f67e4148f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:11.958 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.954+0000 7f67e37fe640 1 -- 172.21.15.88:0/3808507295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67e40d2660 msgr2=0x7f67e414b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:11.958 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.954+0000 7f67e37fe640 1 --2- 172.21.15.88:0/3808507295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67e40d2660 0x7f67e414b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:11.958 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.954+0000 7f67e37fe640 1 -- 172.21.15.88:0/3808507295 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67dc009a70 con 0x7f67e4153ea0 2023-10-16T20:05:11.959 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.954+0000 7f67e2ffd640 1 --2- 172.21.15.88:0/3808507295 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f67e40cd970 0x7f67e4148f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:11.959 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.954+0000 7f67e27fc640 1 --2- 172.21.15.88:0/3808507295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67e40d2660 0x7f67e414b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:11.959 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.954+0000 7f67e37fe640 1 --2- 172.21.15.88:0/3808507295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67e4153ea0 0x7f67e40d0c90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0b0040 tx=0x7f67dc0196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:11.960 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.954+0000 7f67cb7fe640 1 -- 172.21.15.88:0/3808507295 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc0ade40 con 0x7f67e4153ea0 2023-10-16T20:05:11.961 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.954+0000 7f67cb7fe640 1 -- 172.21.15.88:0/3808507295 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67dc009450 con 0x7f67e4153ea0 2023-10-16T20:05:11.961 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.954+0000 7f67e953e640 1 -- 172.21.15.88:0/3808507295 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e4151b00 con 0x7f67e4153ea0 2023-10-16T20:05:11.961 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.954+0000 7f67cb7fe640 1 -- 172.21.15.88:0/3808507295 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc0b9690 con 0x7f67e4153ea0 2023-10-16T20:05:11.961 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.954+0000 7f67e953e640 1 -- 172.21.15.88:0/3808507295 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e41520c0 con 0x7f67e4153ea0 2023-10-16T20:05:11.961 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.954+0000 7f67e953e640 1 -- 172.21.15.88:0/3808507295 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67e404e9c0 con 0x7f67e4153ea0 2023-10-16T20:05:11.961 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.958+0000 7f67cb7fe640 1 -- 172.21.15.88:0/3808507295 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f67dc0b97f0 con 0x7f67e4153ea0 2023-10-16T20:05:11.962 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.958+0000 7f67cb7fe640 1 --2- 172.21.15.88:0/3808507295 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f67d00664a0 0x7f67d0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:11.962 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.958+0000 7f67cb7fe640 1 -- 172.21.15.88:0/3808507295 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f67dc11fa00 con 0x7f67e4153ea0 2023-10-16T20:05:11.962 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.958+0000 7f67e2ffd640 1 --2- 172.21.15.88:0/3808507295 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f67d00664a0 0x7f67d0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:11.962 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.958+0000 7f67e2ffd640 1 --2- 172.21.15.88:0/3808507295 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f67d00664a0 0x7f67d0068960 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f67d4000f70 tx=0x7f67d4007cb0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:11.963 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:11.962+0000 7f67cb7fe640 1 -- 172.21.15.88:0/3808507295 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f67dc0ecbc0 con 0x7f67e4153ea0 2023-10-16T20:05:12.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.118+0000 7f67e953e640 1 -- 172.21.15.88:0/3808507295 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f67e4167a20 con 0x7f67e4153ea0 2023-10-16T20:05:12.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.122+0000 7f67cb7fe640 1 -- 172.21.15.88:0/3808507295 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f67dc0f01e0 con 0x7f67e4153ea0 2023-10-16T20:05:12.121 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2023-10-16T20:05:12.123 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.122+0000 7f67e953e640 1 -- 172.21.15.88:0/3808507295 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f67d00664a0 msgr2=0x7f67d0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:12.123 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.122+0000 7f67e953e640 1 --2- 172.21.15.88:0/3808507295 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f67d00664a0 0x7f67d0068960 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f67d4000f70 tx=0x7f67d4007cb0 comp rx=0 tx=0).stop 2023-10-16T20:05:12.123 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.122+0000 7f67e953e640 1 -- 172.21.15.88:0/3808507295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67e4153ea0 msgr2=0x7f67e40d0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:12.124 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.122+0000 7f67e953e640 1 --2- 172.21.15.88:0/3808507295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67e4153ea0 0x7f67e40d0c90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0b0040 tx=0x7f67dc0196c0 comp rx=0 tx=0).stop 2023-10-16T20:05:12.124 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.122+0000 7f67e953e640 1 -- 172.21.15.88:0/3808507295 shutdown_connections 2023-10-16T20:05:12.124 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.122+0000 7f67e953e640 1 --2- 172.21.15.88:0/3808507295 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f67d00664a0 0x7f67d0068960 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:12.124 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.122+0000 7f67e953e640 1 --2- 172.21.15.88:0/3808507295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67e4153ea0 0x7f67e40d0c90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:12.124 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.122+0000 7f67e953e640 1 --2- 172.21.15.88:0/3808507295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67e40d2660 0x7f67e414b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:12.125 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.122+0000 7f67e953e640 1 --2- 172.21.15.88:0/3808507295 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f67e40cd970 0x7f67e4148f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:12.125 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.122+0000 7f67e953e640 1 -- 172.21.15.88:0/3808507295 >> 172.21.15.88:0/3808507295 conn(0x7f67e40715c0 msgr2=0x7f67e415ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:12.125 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.122+0000 7f67e953e640 1 -- 172.21.15.88:0/3808507295 shutdown_connections 2023-10-16T20:05:12.125 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.122+0000 7f67e953e640 1 -- 172.21.15.88:0/3808507295 wait complete. 2023-10-16T20:05:12.141 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi088.front.sepia.ceph.com... 2023-10-16T20:05:12.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:12.142 DEBUG:teuthology.orchestra.run.smithi088:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-10-16T20:05:12.184 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi088.stderr:ceph-fuse[43995]: fuse finished with error 0 and tester_r 0 2023-10-16T20:05:12.187 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:05:12.277 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2023-10-16T20:05:12.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:12.277 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-10-16T20:05:12.291 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:05:12.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.470+0000 7f43f45d0640 1 Processor -- start 2023-10-16T20:05:12.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.470+0000 7f43f45d0640 1 -- start start 2023-10-16T20:05:12.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.474+0000 7f43f45d0640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f43ec1599f0 0x7f43ec0b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:12.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.474+0000 7f43f45d0640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f43ec069e90 0x7f43ec06a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:12.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.474+0000 7f43f45d0640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f43ec0b05a0 0x7f43ec0b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:12.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.474+0000 7f43f45d0640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43ec06a830 con 0x7f43ec069e90 2023-10-16T20:05:12.475 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.474+0000 7f43f45d0640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43ec0b1260 con 0x7f43ec1599f0 2023-10-16T20:05:12.475 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.474+0000 7f43f45d0640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec0ae960 con 0x7f43ec0b05a0 2023-10-16T20:05:12.475 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.474+0000 7f43f1b6b640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f43ec069e90 0x7f43ec06a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:12.475 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.474+0000 7f43f1b6b640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f43ec069e90 0x7f43ec06a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40040/0 (socket says 172.21.15.88:40040) 2023-10-16T20:05:12.475 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.474+0000 7f43f1b6b640 1 -- 172.21.15.88:0/3898181053 learned_addr learned my addr 172.21.15.88:0/3898181053 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:12.476 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.474+0000 7f43f236c640 1 --2- 172.21.15.88:0/3898181053 >> v2:172.21.15.94:3300/0 conn(0x7f43ec1599f0 0x7f43ec0b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:12.476 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.474+0000 7f43f2b6d640 1 --2- 172.21.15.88:0/3898181053 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f43ec0b05a0 0x7f43ec0b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:12.476 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.474+0000 7f43f1b6b640 1 -- 172.21.15.88:0/3898181053 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f43ec0b05a0 msgr2=0x7f43ec0b0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:12.476 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.474+0000 7f43f1b6b640 1 --2- 172.21.15.88:0/3898181053 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f43ec0b05a0 0x7f43ec0b0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:12.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.474+0000 7f43f1b6b640 1 -- 172.21.15.88:0/3898181053 >> v2:172.21.15.94:3300/0 conn(0x7f43ec1599f0 msgr2=0x7f43ec0b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:12.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.474+0000 7f43f1b6b640 1 --2- 172.21.15.88:0/3898181053 >> v2:172.21.15.94:3300/0 conn(0x7f43ec1599f0 0x7f43ec0b2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:12.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.474+0000 7f43f1b6b640 1 -- 172.21.15.88:0/3898181053 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ec0aec40 con 0x7f43ec069e90 2023-10-16T20:05:12.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.478+0000 7f43f1b6b640 1 --2- 172.21.15.88:0/3898181053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43ec069e90 0x7f43ec06a2f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f43ec06d650 tx=0x7f43dc01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cbf889cd2463386 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:12.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.478+0000 7f43f136a640 1 -- 172.21.15.88:0/3898181053 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc0aa5f0 con 0x7f43ec069e90 2023-10-16T20:05:12.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.478+0000 7f43f136a640 1 -- 172.21.15.88:0/3898181053 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43dc0aa750 con 0x7f43ec069e90 2023-10-16T20:05:12.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.478+0000 7f43f136a640 1 -- 172.21.15.88:0/3898181053 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc0aaae0 con 0x7f43ec069e90 2023-10-16T20:05:12.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.478+0000 7f43f45d0640 1 -- 172.21.15.88:0/3898181053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43ec069e90 msgr2=0x7f43ec06a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:12.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.478+0000 7f43f45d0640 1 --2- 172.21.15.88:0/3898181053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43ec069e90 0x7f43ec06a2f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f43ec06d650 tx=0x7f43dc01c4d0 comp rx=0 tx=0).stop 2023-10-16T20:05:12.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.478+0000 7f43f45d0640 1 -- 172.21.15.88:0/3898181053 shutdown_connections 2023-10-16T20:05:12.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.478+0000 7f43f45d0640 1 --2- 172.21.15.88:0/3898181053 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f43ec0b05a0 0x7f43ec0b0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:12.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.478+0000 7f43f45d0640 1 --2- 172.21.15.88:0/3898181053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43ec069e90 0x7f43ec06a2f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:12.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.478+0000 7f43f45d0640 1 --2- 172.21.15.88:0/3898181053 >> v2:172.21.15.94:3300/0 conn(0x7f43ec1599f0 0x7f43ec0b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:12.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.478+0000 7f43f45d0640 1 -- 172.21.15.88:0/3898181053 >> 172.21.15.88:0/3898181053 conn(0x7f43ec0ce5e0 msgr2=0x7f43ec0ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:12.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.478+0000 7f43f45d0640 1 -- 172.21.15.88:0/3898181053 shutdown_connections 2023-10-16T20:05:12.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.478+0000 7f43f45d0640 1 -- 172.21.15.88:0/3898181053 wait complete. 2023-10-16T20:05:12.482 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.478+0000 7f43f45d0640 1 Processor -- start 2023-10-16T20:05:12.482 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.478+0000 7f43f45d0640 1 -- start start 2023-10-16T20:05:12.482 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.478+0000 7f43f45d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43ec069e90 0x7f43ec147040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:12.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.478+0000 7f43f45d0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43ec0b05a0 0x7f43ec157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:12.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.478+0000 7f43f45d0640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f43ec1599f0 0x7f43ec1499f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:12.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.478+0000 7f43f45d0640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec0ab890 con 0x7f43ec0b05a0 2023-10-16T20:05:12.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.478+0000 7f43f45d0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec157cc0 con 0x7f43ec069e90 2023-10-16T20:05:12.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.478+0000 7f43f45d0640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec157fa0 con 0x7f43ec1599f0 2023-10-16T20:05:12.484 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.482+0000 7f43f1b6b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43ec0b05a0 0x7f43ec157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:12.484 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.482+0000 7f43f1b6b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43ec0b05a0 0x7f43ec157200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40044/0 (socket says 172.21.15.88:40044) 2023-10-16T20:05:12.484 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.482+0000 7f43f1b6b640 1 -- 172.21.15.88:0/1457778906 learned_addr learned my addr 172.21.15.88:0/1457778906 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:12.484 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.482+0000 7f43f236c640 1 --2- 172.21.15.88:0/1457778906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43ec069e90 0x7f43ec147040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:12.484 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.482+0000 7f43f2b6d640 1 --2- 172.21.15.88:0/1457778906 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f43ec1599f0 0x7f43ec1499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:12.484 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.482+0000 7f43f1b6b640 1 -- 172.21.15.88:0/1457778906 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f43ec1599f0 msgr2=0x7f43ec1499f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:12.485 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.482+0000 7f43f1b6b640 1 --2- 172.21.15.88:0/1457778906 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f43ec1599f0 0x7f43ec1499f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:12.485 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.482+0000 7f43f1b6b640 1 -- 172.21.15.88:0/1457778906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43ec069e90 msgr2=0x7f43ec147040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:12.485 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.482+0000 7f43f1b6b640 1 --2- 172.21.15.88:0/1457778906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43ec069e90 0x7f43ec147040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:12.485 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.482+0000 7f43f1b6b640 1 -- 172.21.15.88:0/1457778906 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43dc0094d0 con 0x7f43ec0b05a0 2023-10-16T20:05:12.485 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.482+0000 7f43f1b6b640 1 --2- 172.21.15.88:0/1457778906 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43ec0b05a0 0x7f43ec157200 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f43dc01ab80 tx=0x7f43dc01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:12.486 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.482+0000 7f43e2ffd640 1 -- 172.21.15.88:0/1457778906 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc0b3800 con 0x7f43ec0b05a0 2023-10-16T20:05:12.486 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.482+0000 7f43e2ffd640 1 -- 172.21.15.88:0/1457778906 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43dc0b3980 con 0x7f43ec0b05a0 2023-10-16T20:05:12.486 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.482+0000 7f43f45d0640 1 -- 172.21.15.88:0/1457778906 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43ec158280 con 0x7f43ec0b05a0 2023-10-16T20:05:12.486 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.482+0000 7f43e2ffd640 1 -- 172.21.15.88:0/1457778906 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc0b3d30 con 0x7f43ec0b05a0 2023-10-16T20:05:12.486 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.482+0000 7f43f45d0640 1 -- 172.21.15.88:0/1457778906 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43ec14b6a0 con 0x7f43ec0b05a0 2023-10-16T20:05:12.487 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.482+0000 7f43f45d0640 1 -- 172.21.15.88:0/1457778906 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43ec14bbe0 con 0x7f43ec0b05a0 2023-10-16T20:05:12.488 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.482+0000 7f43e2ffd640 1 -- 172.21.15.88:0/1457778906 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f43dc0c6440 con 0x7f43ec0b05a0 2023-10-16T20:05:12.488 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.482+0000 7f43e2ffd640 1 --2- 172.21.15.88:0/1457778906 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f43d00664a0 0x7f43d0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:12.488 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.482+0000 7f43e2ffd640 1 -- 172.21.15.88:0/1457778906 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f43dc0a7070 con 0x7f43ec0b05a0 2023-10-16T20:05:12.488 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.486+0000 7f43f236c640 1 --2- 172.21.15.88:0/1457778906 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f43d00664a0 0x7f43d0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:12.488 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.486+0000 7f43f236c640 1 --2- 172.21.15.88:0/1457778906 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f43d00664a0 0x7f43d0068960 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f43ec1f8450 tx=0x7f43e8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:12.489 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.486+0000 7f43e2ffd640 1 -- 172.21.15.88:0/1457778906 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f43ec14bbe0 con 0x7f43ec0b05a0 2023-10-16T20:05:12.639 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.638+0000 7f43f45d0640 1 -- 172.21.15.88:0/1457778906 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43ec14bbe0 con 0x7f43ec0b05a0 2023-10-16T20:05:12.640 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.638+0000 7f43e2ffd640 1 -- 172.21.15.88:0/1457778906 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18274 (secure 0 0 0) 0x7f43ec14bbe0 con 0x7f43ec0b05a0 2023-10-16T20:05:12.641 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:12.641 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:05:01.835630+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T20:04:54.442487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T20:04:55.766641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T20:04:59.475830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:05:12.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.642+0000 7f43f45d0640 1 -- 172.21.15.88:0/1457778906 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f43d00664a0 msgr2=0x7f43d0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:12.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.642+0000 7f43f45d0640 1 --2- 172.21.15.88:0/1457778906 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f43d00664a0 0x7f43d0068960 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f43ec1f8450 tx=0x7f43e8008040 comp rx=0 tx=0).stop 2023-10-16T20:05:12.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.642+0000 7f43f45d0640 1 -- 172.21.15.88:0/1457778906 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43ec0b05a0 msgr2=0x7f43ec157200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:12.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.642+0000 7f43f45d0640 1 --2- 172.21.15.88:0/1457778906 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43ec0b05a0 0x7f43ec157200 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f43dc01ab80 tx=0x7f43dc01ad10 comp rx=0 tx=0).stop 2023-10-16T20:05:12.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.642+0000 7f43f45d0640 1 -- 172.21.15.88:0/1457778906 shutdown_connections 2023-10-16T20:05:12.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.642+0000 7f43f45d0640 1 --2- 172.21.15.88:0/1457778906 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f43d00664a0 0x7f43d0068960 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:12.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.642+0000 7f43f45d0640 1 --2- 172.21.15.88:0/1457778906 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f43ec1599f0 0x7f43ec1499f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:12.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.642+0000 7f43f45d0640 1 --2- 172.21.15.88:0/1457778906 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43ec0b05a0 0x7f43ec157200 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:12.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.642+0000 7f43f45d0640 1 --2- 172.21.15.88:0/1457778906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43ec069e90 0x7f43ec147040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:12.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.642+0000 7f43f45d0640 1 -- 172.21.15.88:0/1457778906 >> 172.21.15.88:0/1457778906 conn(0x7f43ec0ce5e0 msgr2=0x7f43ec0cb060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:12.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.642+0000 7f43f45d0640 1 -- 172.21.15.88:0/1457778906 shutdown_connections 2023-10-16T20:05:12.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.642+0000 7f43f45d0640 1 -- 172.21.15.88:0/1457778906 wait complete. 2023-10-16T20:05:12.662 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:05:12.800 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.798+0000 7f0b299e4640 1 Processor -- start 2023-10-16T20:05:12.800 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.798+0000 7f0b299e4640 1 -- start start 2023-10-16T20:05:12.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.802+0000 7f0b299e4640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0b2415c9a0 0x7f0b240cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:12.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.802+0000 7f0b299e4640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f0b240d3270 0x7f0b240d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:12.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.802+0000 7f0b299e4640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0b240d1610 0x7f0b240cf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:12.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.802+0000 7f0b299e4640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b240ce7e0 con 0x7f0b240d3270 2023-10-16T20:05:12.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.802+0000 7f0b299e4640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b240d0c40 con 0x7f0b240d1610 2023-10-16T20:05:12.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.802+0000 7f0b299e4640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b240d0630 con 0x7f0b2415c9a0 2023-10-16T20:05:12.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.802+0000 7f0b23fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f0b240d3270 0x7f0b240d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:12.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.802+0000 7f0b23fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f0b240d3270 0x7f0b240d10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40060/0 (socket says 172.21.15.88:40060) 2023-10-16T20:05:12.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.802+0000 7f0b23fff640 1 -- 172.21.15.88:0/1646427114 learned_addr learned my addr 172.21.15.88:0/1646427114 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:12.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.802+0000 7f0b291e3640 1 --2- 172.21.15.88:0/1646427114 >> v2:172.21.15.94:3300/0 conn(0x7f0b240d1610 0x7f0b240cf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:12.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.802+0000 7f0b289e2640 1 --2- 172.21.15.88:0/1646427114 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0b2415c9a0 0x7f0b240cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:12.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.802+0000 7f0b23fff640 1 -- 172.21.15.88:0/1646427114 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0b2415c9a0 msgr2=0x7f0b240cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:12.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.802+0000 7f0b23fff640 1 --2- 172.21.15.88:0/1646427114 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0b2415c9a0 0x7f0b240cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:12.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.802+0000 7f0b23fff640 1 -- 172.21.15.88:0/1646427114 >> v2:172.21.15.94:3300/0 conn(0x7f0b240d1610 msgr2=0x7f0b240cf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:12.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.802+0000 7f0b23fff640 1 --2- 172.21.15.88:0/1646427114 >> v2:172.21.15.94:3300/0 conn(0x7f0b240d1610 0x7f0b240cf230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:12.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.802+0000 7f0b23fff640 1 -- 172.21.15.88:0/1646427114 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b240ce520 con 0x7f0b240d3270 2023-10-16T20:05:12.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.802+0000 7f0b23fff640 1 --2- 172.21.15.88:0/1646427114 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b240d3270 0x7f0b240d10d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0b2408d090 tx=0x7f0b1001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4ef6ed63314291 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:12.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.802+0000 7f0b237fe640 1 -- 172.21.15.88:0/1646427114 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b100aa610 con 0x7f0b240d3270 2023-10-16T20:05:12.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.802+0000 7f0b237fe640 1 -- 172.21.15.88:0/1646427114 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b100a6040 con 0x7f0b240d3270 2023-10-16T20:05:12.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.806+0000 7f0b237fe640 1 -- 172.21.15.88:0/1646427114 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b100aa990 con 0x7f0b240d3270 2023-10-16T20:05:12.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.806+0000 7f0b299e4640 1 -- 172.21.15.88:0/1646427114 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b240d3270 msgr2=0x7f0b240d10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:12.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.806+0000 7f0b299e4640 1 --2- 172.21.15.88:0/1646427114 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b240d3270 0x7f0b240d10d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0b2408d090 tx=0x7f0b1001c4d0 comp rx=0 tx=0).stop 2023-10-16T20:05:12.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.806+0000 7f0b299e4640 1 -- 172.21.15.88:0/1646427114 shutdown_connections 2023-10-16T20:05:12.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.806+0000 7f0b299e4640 1 --2- 172.21.15.88:0/1646427114 >> v2:172.21.15.94:3300/0 conn(0x7f0b240d1610 0x7f0b240cf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:12.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.806+0000 7f0b299e4640 1 --2- 172.21.15.88:0/1646427114 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b240d3270 0x7f0b240d10d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:12.808 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.806+0000 7f0b299e4640 1 --2- 172.21.15.88:0/1646427114 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0b2415c9a0 0x7f0b240cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:12.808 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.806+0000 7f0b299e4640 1 -- 172.21.15.88:0/1646427114 >> 172.21.15.88:0/1646427114 conn(0x7f0b2409bc30 msgr2=0x7f0b240d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:12.808 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.806+0000 7f0b299e4640 1 -- 172.21.15.88:0/1646427114 shutdown_connections 2023-10-16T20:05:12.808 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.806+0000 7f0b299e4640 1 -- 172.21.15.88:0/1646427114 wait complete. 2023-10-16T20:05:12.810 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.806+0000 7f0b299e4640 1 Processor -- start 2023-10-16T20:05:12.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.806+0000 7f0b299e4640 1 -- start start 2023-10-16T20:05:12.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.806+0000 7f0b299e4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b240d1610 0x7f0b2413ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:12.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.806+0000 7f0b299e4640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0b240d3270 0x7f0b240b6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:12.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.806+0000 7f0b299e4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b2415c9a0 0x7f0b240b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:12.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.806+0000 7f0b299e4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2414f8e0 con 0x7f0b2415c9a0 2023-10-16T20:05:12.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.806+0000 7f0b299e4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2414fbc0 con 0x7f0b240d1610 2023-10-16T20:05:12.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.806+0000 7f0b299e4640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2414fea0 con 0x7f0b240d3270 2023-10-16T20:05:12.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.806+0000 7f0b291e3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b2415c9a0 0x7f0b240b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:12.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.806+0000 7f0b291e3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b2415c9a0 0x7f0b240b2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40072/0 (socket says 172.21.15.88:40072) 2023-10-16T20:05:12.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.806+0000 7f0b291e3640 1 -- 172.21.15.88:0/3281288199 learned_addr learned my addr 172.21.15.88:0/3281288199 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:12.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.810+0000 7f0b289e2640 1 --2- 172.21.15.88:0/3281288199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b240d1610 0x7f0b2413ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:12.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.810+0000 7f0b23fff640 1 --2- 172.21.15.88:0/3281288199 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0b240d3270 0x7f0b240b6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:12.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.810+0000 7f0b291e3640 1 -- 172.21.15.88:0/3281288199 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0b240d3270 msgr2=0x7f0b240b6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:12.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.810+0000 7f0b291e3640 1 --2- 172.21.15.88:0/3281288199 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0b240d3270 0x7f0b240b6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:12.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.810+0000 7f0b291e3640 1 -- 172.21.15.88:0/3281288199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b240d1610 msgr2=0x7f0b2413ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:12.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.810+0000 7f0b291e3640 1 --2- 172.21.15.88:0/3281288199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b240d1610 0x7f0b2413ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:12.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.810+0000 7f0b291e3640 1 -- 172.21.15.88:0/3281288199 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b100094d0 con 0x7f0b2415c9a0 2023-10-16T20:05:12.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.810+0000 7f0b289e2640 1 --2- 172.21.15.88:0/3281288199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b240d1610 0x7f0b2413ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:12.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.810+0000 7f0b23fff640 1 --2- 172.21.15.88:0/3281288199 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0b240d3270 0x7f0b240b6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:12.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.810+0000 7f0b291e3640 1 --2- 172.21.15.88:0/3281288199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b2415c9a0 0x7f0b240b2a70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c007cc0 tx=0x7f0b1c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:12.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.810+0000 7f0b217fa640 1 -- 172.21.15.88:0/3281288199 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c014420 con 0x7f0b2415c9a0 2023-10-16T20:05:12.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.810+0000 7f0b217fa640 1 -- 172.21.15.88:0/3281288199 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1c014580 con 0x7f0b2415c9a0 2023-10-16T20:05:12.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.810+0000 7f0b299e4640 1 -- 172.21.15.88:0/3281288199 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b2415cf00 con 0x7f0b2415c9a0 2023-10-16T20:05:12.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.810+0000 7f0b217fa640 1 -- 172.21.15.88:0/3281288199 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c014930 con 0x7f0b2415c9a0 2023-10-16T20:05:12.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.810+0000 7f0b299e4640 1 -- 172.21.15.88:0/3281288199 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b24150460 con 0x7f0b2415c9a0 2023-10-16T20:05:12.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.810+0000 7f0b299e4640 1 -- 172.21.15.88:0/3281288199 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b24150650 con 0x7f0b2415c9a0 2023-10-16T20:05:12.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.810+0000 7f0b217fa640 1 -- 172.21.15.88:0/3281288199 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0b1c014a90 con 0x7f0b2415c9a0 2023-10-16T20:05:12.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.810+0000 7f0b217fa640 1 --2- 172.21.15.88:0/3281288199 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0b080664a0 0x7f0b08068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:12.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.810+0000 7f0b217fa640 1 -- 172.21.15.88:0/3281288199 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0b1c084930 con 0x7f0b2415c9a0 2023-10-16T20:05:12.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.810+0000 7f0b289e2640 1 --2- 172.21.15.88:0/3281288199 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0b080664a0 0x7f0b08068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:12.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.814+0000 7f0b289e2640 1 --2- 172.21.15.88:0/3281288199 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0b080664a0 0x7f0b08068960 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0b18006fd0 tx=0x7f0b18008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:12.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.814+0000 7f0b217fa640 1 -- 172.21.15.88:0/3281288199 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0b1c051b50 con 0x7f0b2415c9a0 2023-10-16T20:05:12.968 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.966+0000 7f0b299e4640 1 -- 172.21.15.88:0/3281288199 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b240674f0 con 0x7f0b2415c9a0 2023-10-16T20:05:12.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.970+0000 7f0b217fa640 1 -- 172.21.15.88:0/3281288199 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18274 (secure 0 0 0) 0x7f0b1c055170 con 0x7f0b2415c9a0 2023-10-16T20:05:12.970 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:12.970 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:05:01.835630+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T20:04:54.442487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T20:04:55.766641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T20:04:59.475830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:05:12.972 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.970+0000 7f0b299e4640 1 -- 172.21.15.88:0/3281288199 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0b080664a0 msgr2=0x7f0b08068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:12.972 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.970+0000 7f0b299e4640 1 --2- 172.21.15.88:0/3281288199 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0b080664a0 0x7f0b08068960 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0b18006fd0 tx=0x7f0b18008040 comp rx=0 tx=0).stop 2023-10-16T20:05:12.972 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.970+0000 7f0b299e4640 1 -- 172.21.15.88:0/3281288199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b2415c9a0 msgr2=0x7f0b240b2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:12.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.970+0000 7f0b299e4640 1 --2- 172.21.15.88:0/3281288199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b2415c9a0 0x7f0b240b2a70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c007cc0 tx=0x7f0b1c00e4b0 comp rx=0 tx=0).stop 2023-10-16T20:05:12.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.970+0000 7f0b299e4640 1 -- 172.21.15.88:0/3281288199 shutdown_connections 2023-10-16T20:05:12.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.970+0000 7f0b299e4640 1 --2- 172.21.15.88:0/3281288199 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0b080664a0 0x7f0b08068960 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:12.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.970+0000 7f0b299e4640 1 --2- 172.21.15.88:0/3281288199 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0b240d3270 0x7f0b240b6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:12.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.970+0000 7f0b299e4640 1 --2- 172.21.15.88:0/3281288199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b2415c9a0 0x7f0b240b2a70 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:12.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.970+0000 7f0b299e4640 1 --2- 172.21.15.88:0/3281288199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b240d1610 0x7f0b2413ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:12.974 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.970+0000 7f0b299e4640 1 -- 172.21.15.88:0/3281288199 >> 172.21.15.88:0/3281288199 conn(0x7f0b2409bc30 msgr2=0x7f0b240d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:12.974 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.970+0000 7f0b299e4640 1 -- 172.21.15.88:0/3281288199 shutdown_connections 2023-10-16T20:05:12.974 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:12.970+0000 7f0b299e4640 1 -- 172.21.15.88:0/3281288199 wait complete. 2023-10-16T20:05:12.991 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T20:05:13.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c1c209640 1 Processor -- start 2023-10-16T20:05:13.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c1c209640 1 -- start start 2023-10-16T20:05:13.134 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c1c209640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3c1415d2c0 0x7f3c140cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:13.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c1c209640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c140b5050 0x7f3c140b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:13.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c1c209640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c140cbfd0 con 0x7f3c140ce9e0 2023-10-16T20:05:13.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c1c209640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c140cacc0 con 0x7f3c1415d2c0 2023-10-16T20:05:13.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c1c209640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1415d6a0 con 0x7f3c140b5050 2023-10-16T20:05:13.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c1a7a6640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f3c140ce9e0 0x7f3c14071920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:39094/0 (socket says 172.21.15.88:39094) 2023-10-16T20:05:13.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c1a7a6640 1 -- 172.21.15.88:0/2958117927 learned_addr learned my addr 172.21.15.88:0/2958117927 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:13.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c197a4640 1 --2- 172.21.15.88:0/2958117927 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c140b5050 0x7f3c140b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:13.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c19fa5640 1 --2- 172.21.15.88:0/2958117927 >> v2:172.21.15.94:3300/0 conn(0x7f3c1415d2c0 0x7f3c140cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:13.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c18fa3640 1 -- 172.21.15.88:0/2958117927 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 783711278 0 0) 0x7f3c140cbfd0 con 0x7f3c140ce9e0 2023-10-16T20:05:13.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c18fa3640 1 -- 172.21.15.88:0/2958117927 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c00003530 con 0x7f3c140ce9e0 2023-10-16T20:05:13.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c18fa3640 1 -- 172.21.15.88:0/2958117927 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 678994475 0 0) 0x7f3c00003530 con 0x7f3c140ce9e0 2023-10-16T20:05:13.139 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c18fa3640 1 -- 172.21.15.88:0/2958117927 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c140b5050 msgr2=0x7f3c140b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:13.139 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c18fa3640 1 --2- 172.21.15.88:0/2958117927 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c140b5050 0x7f3c140b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:13.139 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c18fa3640 1 -- 172.21.15.88:0/2958117927 >> v2:172.21.15.94:3300/0 conn(0x7f3c1415d2c0 msgr2=0x7f3c140cd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:13.139 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c18fa3640 1 --2- 172.21.15.88:0/2958117927 >> v2:172.21.15.94:3300/0 conn(0x7f3c1415d2c0 0x7f3c140cd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:13.139 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c18fa3640 1 -- 172.21.15.88:0/2958117927 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c14071490 con 0x7f3c140ce9e0 2023-10-16T20:05:13.139 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c18fa3640 1 -- 172.21.15.88:0/2958117927 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f3c0c001a00 con 0x7f3c140ce9e0 2023-10-16T20:05:13.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c18fa3640 1 -- 172.21.15.88:0/2958117927 >> v1:172.21.15.88:6789/0 conn(0x7f3c140ce9e0 legacy=0x7f3c14071920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:13.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c18fa3640 1 --2- 172.21.15.88:0/2958117927 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c140b5050 0x7f3c00003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:13.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c18fa3640 1 --2- 172.21.15.88:0/2958117927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c00003f00 0x7f3c000047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:13.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c18fa3640 1 --2- 172.21.15.88:0/2958117927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c00004d10 0x7f3c00005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:13.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c18fa3640 1 -- 172.21.15.88:0/2958117927 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1415d6a0 con 0x7f3c00004d10 2023-10-16T20:05:13.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c18fa3640 1 -- 172.21.15.88:0/2958117927 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0000b990 con 0x7f3c00003f00 2023-10-16T20:05:13.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c18fa3640 1 -- 172.21.15.88:0/2958117927 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0000bad0 con 0x7f3c140b5050 2023-10-16T20:05:13.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c1a7a6640 1 --2- 172.21.15.88:0/2958117927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c00004d10 0x7f3c00005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:13.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c197a4640 1 --2- 172.21.15.88:0/2958117927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c00003f00 0x7f3c000047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:13.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c19fa5640 1 --2- 172.21.15.88:0/2958117927 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c140b5050 0x7f3c00003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:13.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c197a4640 1 -- 172.21.15.88:0/2958117927 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c140b5050 msgr2=0x7f3c00003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:13.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c197a4640 1 --2- 172.21.15.88:0/2958117927 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c140b5050 0x7f3c00003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:13.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c197a4640 1 -- 172.21.15.88:0/2958117927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c00004d10 msgr2=0x7f3c00005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:13.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c197a4640 1 --2- 172.21.15.88:0/2958117927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c00004d10 0x7f3c00005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:13.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.134+0000 7f3c197a4640 1 -- 172.21.15.88:0/2958117927 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c00003530 con 0x7f3c00003f00 2023-10-16T20:05:13.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.138+0000 7f3c197a4640 1 --2- 172.21.15.88:0/2958117927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c00003f00 0x7f3c000047d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3c04009670 tx=0x7f3c0401ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ab1e6e940969908 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:13.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.138+0000 7f3c18fa3640 1 -- 172.21.15.88:0/2958117927 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c040a7880 con 0x7f3c00003f00 2023-10-16T20:05:13.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.138+0000 7f3c18fa3640 1 -- 172.21.15.88:0/2958117927 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c040a79e0 con 0x7f3c00003f00 2023-10-16T20:05:13.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.138+0000 7f3c18fa3640 1 -- 172.21.15.88:0/2958117927 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c040a7d90 con 0x7f3c00003f00 2023-10-16T20:05:13.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.138+0000 7f3c1c209640 1 -- 172.21.15.88:0/2958117927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c00003f00 msgr2=0x7f3c000047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:13.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.138+0000 7f3c1c209640 1 --2- 172.21.15.88:0/2958117927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c00003f00 0x7f3c000047d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3c04009670 tx=0x7f3c0401ec20 comp rx=0 tx=0).stop 2023-10-16T20:05:13.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.138+0000 7f3c1a7a6640 1 -- 172.21.15.88:0/2958117927 reap_dead start 2023-10-16T20:05:13.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.138+0000 7f3c1c209640 1 -- 172.21.15.88:0/2958117927 shutdown_connections 2023-10-16T20:05:13.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.138+0000 7f3c1c209640 1 -- 172.21.15.88:0/2958117927 >> 172.21.15.88:0/2958117927 conn(0x7f3c140cf710 msgr2=0x7f3c140cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:13.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.138+0000 7f3c1c209640 1 -- 172.21.15.88:0/2958117927 shutdown_connections 2023-10-16T20:05:13.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.138+0000 7f3c1c209640 1 -- 172.21.15.88:0/2958117927 wait complete. 2023-10-16T20:05:13.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.142+0000 7f3c1c209640 1 Processor -- start 2023-10-16T20:05:13.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.142+0000 7f3c1c209640 1 -- start start 2023-10-16T20:05:13.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.142+0000 7f3c1c209640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c00003f00 0x7f3c141f1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:13.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.142+0000 7f3c1c209640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c140ade10 0x7f3c140b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:13.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.142+0000 7f3c1c209640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c140b4920 0x7f3c141474d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:13.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.142+0000 7f3c1c209640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c140ad980 con 0x7f3c00003f00 2023-10-16T20:05:13.149 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.142+0000 7f3c1c209640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c141f2010 con 0x7f3c140b4920 2023-10-16T20:05:13.149 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.142+0000 7f3c1c209640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c140b4da0 con 0x7f3c140ade10 2023-10-16T20:05:13.149 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.142+0000 7f3c19fa5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c00003f00 0x7f3c141f1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:13.149 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.142+0000 7f3c19fa5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c00003f00 0x7f3c141f1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40102/0 (socket says 172.21.15.88:40102) 2023-10-16T20:05:13.149 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.142+0000 7f3c19fa5640 1 -- 172.21.15.88:0/2265429501 learned_addr learned my addr 172.21.15.88:0/2265429501 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:13.149 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.142+0000 7f3c1a7a6640 1 --2- 172.21.15.88:0/2265429501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c140b4920 0x7f3c141474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:13.150 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.142+0000 7f3c197a4640 1 --2- 172.21.15.88:0/2265429501 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c140ade10 0x7f3c140b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:13.150 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.142+0000 7f3c19fa5640 1 -- 172.21.15.88:0/2265429501 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c140ade10 msgr2=0x7f3c140b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:13.150 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.142+0000 7f3c19fa5640 1 --2- 172.21.15.88:0/2265429501 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c140ade10 0x7f3c140b43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:13.150 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.142+0000 7f3c19fa5640 1 -- 172.21.15.88:0/2265429501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c140b4920 msgr2=0x7f3c141474d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:13.150 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.142+0000 7f3c19fa5640 1 --2- 172.21.15.88:0/2265429501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c140b4920 0x7f3c141474d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:13.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.142+0000 7f3c19fa5640 1 -- 172.21.15.88:0/2265429501 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c14148270 con 0x7f3c00003f00 2023-10-16T20:05:13.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.142+0000 7f3c19fa5640 1 --2- 172.21.15.88:0/2265429501 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c00003f00 0x7f3c141f1ad0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3c10011ad0 tx=0x7f3c10011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:13.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.142+0000 7f3c0a7fc640 1 -- 172.21.15.88:0/2265429501 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1000fb50 con 0x7f3c00003f00 2023-10-16T20:05:13.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.142+0000 7f3c0a7fc640 1 -- 172.21.15.88:0/2265429501 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c1000fcb0 con 0x7f3c00003f00 2023-10-16T20:05:13.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.142+0000 7f3c1c209640 1 -- 172.21.15.88:0/2265429501 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c140ad050 con 0x7f3c00003f00 2023-10-16T20:05:13.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.142+0000 7f3c0a7fc640 1 -- 172.21.15.88:0/2265429501 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c10014440 con 0x7f3c00003f00 2023-10-16T20:05:13.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.142+0000 7f3c1c209640 1 -- 172.21.15.88:0/2265429501 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c140ad610 con 0x7f3c00003f00 2023-10-16T20:05:13.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.146+0000 7f3c1c209640 1 -- 172.21.15.88:0/2265429501 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c1404e9c0 con 0x7f3c00003f00 2023-10-16T20:05:13.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.146+0000 7f3c0a7fc640 1 -- 172.21.15.88:0/2265429501 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3c100145c0 con 0x7f3c00003f00 2023-10-16T20:05:13.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.146+0000 7f3c0a7fc640 1 --2- 172.21.15.88:0/2265429501 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3bf80664d0 0x7f3bf8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:13.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.146+0000 7f3c0a7fc640 1 -- 172.21.15.88:0/2265429501 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3c10084b10 con 0x7f3c00003f00 2023-10-16T20:05:13.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.146+0000 7f3c197a4640 1 --2- 172.21.15.88:0/2265429501 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3bf80664d0 0x7f3bf8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:13.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.146+0000 7f3c197a4640 1 --2- 172.21.15.88:0/2265429501 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3bf80664d0 0x7f3bf8068990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3c040b1000 tx=0x7f3c04010040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:13.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.150+0000 7f3c0a7fc640 1 -- 172.21.15.88:0/2265429501 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3c10051c00 con 0x7f3c00003f00 2023-10-16T20:05:13.340 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:13.338+0000 7f3c1c209640 1 -- 172.21.15.88:0/2265429501 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3c14158d90 con 0x7f3c00003f00 2023-10-16T20:05:14.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.202+0000 7f3c0a7fc640 1 -- 172.21.15.88:0/2265429501 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3c10055220 con 0x7f3c00003f00 2023-10-16T20:05:14.206 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.206+0000 7f3c1c209640 1 -- 172.21.15.88:0/2265429501 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3bf80664d0 msgr2=0x7f3bf8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:14.206 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.206+0000 7f3c1c209640 1 --2- 172.21.15.88:0/2265429501 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3bf80664d0 0x7f3bf8068990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3c040b1000 tx=0x7f3c04010040 comp rx=0 tx=0).stop 2023-10-16T20:05:14.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.206+0000 7f3c1c209640 1 -- 172.21.15.88:0/2265429501 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c00003f00 msgr2=0x7f3c141f1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:14.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.206+0000 7f3c1c209640 1 --2- 172.21.15.88:0/2265429501 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c00003f00 0x7f3c141f1ad0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3c10011ad0 tx=0x7f3c10011f90 comp rx=0 tx=0).stop 2023-10-16T20:05:14.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.206+0000 7f3c1c209640 1 -- 172.21.15.88:0/2265429501 shutdown_connections 2023-10-16T20:05:14.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.206+0000 7f3c1c209640 1 --2- 172.21.15.88:0/2265429501 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3bf80664d0 0x7f3bf8068990 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:14.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.206+0000 7f3c1c209640 1 --2- 172.21.15.88:0/2265429501 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3c140ade10 0x7f3c140b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:14.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.206+0000 7f3c1c209640 1 --2- 172.21.15.88:0/2265429501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c140b4920 0x7f3c141474d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:14.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.206+0000 7f3c1c209640 1 --2- 172.21.15.88:0/2265429501 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c00003f00 0x7f3c141f1ad0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:14.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.206+0000 7f3c1c209640 1 -- 172.21.15.88:0/2265429501 >> 172.21.15.88:0/2265429501 conn(0x7f3c140cf710 msgr2=0x7f3c14072790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:14.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.206+0000 7f3c1c209640 1 -- 172.21.15.88:0/2265429501 shutdown_connections 2023-10-16T20:05:14.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.206+0000 7f3c1c209640 1 -- 172.21.15.88:0/2265429501 wait complete. 2023-10-16T20:05:14.226 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2023-10-16T20:05:14.227 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T20:05:14.368 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.366+0000 7f97367f8640 1 Processor -- start 2023-10-16T20:05:14.368 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.366+0000 7f97367f8640 1 -- start start 2023-10-16T20:05:14.369 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.366+0000 7f97367f8640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f97301602f0 0x7f97300cd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:14.370 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.366+0000 7f97367f8640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f97300d25e0 0x7f97300cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:14.370 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.366+0000 7f97367f8640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97300d1540 con 0x7f973015ff10 2023-10-16T20:05:14.370 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.366+0000 7f97367f8640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97300d1820 con 0x7f97301602f0 2023-10-16T20:05:14.370 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.366+0000 7f97367f8640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97300d1b00 con 0x7f97300d25e0 2023-10-16T20:05:14.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.370+0000 7f972ffff640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f973015ff10 0x7f97300d0c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:39110/0 (socket says 172.21.15.88:39110) 2023-10-16T20:05:14.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.370+0000 7f972ffff640 1 -- 172.21.15.88:0/1219617839 learned_addr learned my addr 172.21.15.88:0/1219617839 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:14.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.370+0000 7f972f7fe640 1 --2- 172.21.15.88:0/1219617839 >> v2:172.21.15.88:3300/0 conn(0x7f97301602f0 0x7f97300cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:14.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.370+0000 7f9734d95640 1 --2- 172.21.15.88:0/1219617839 >> v2:172.21.15.94:3300/0 conn(0x7f97300d25e0 0x7f97300cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:14.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.370+0000 7f972f7fe640 1 -- 172.21.15.88:0/1219617839 >> v2:172.21.15.94:3300/0 conn(0x7f97300d25e0 msgr2=0x7f97300cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:14.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.370+0000 7f972f7fe640 1 --2- 172.21.15.88:0/1219617839 >> v2:172.21.15.94:3300/0 conn(0x7f97300d25e0 0x7f97300cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:14.372 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.370+0000 7f972f7fe640 1 -- 172.21.15.88:0/1219617839 >> v1:172.21.15.88:6789/0 conn(0x7f973015ff10 legacy=0x7f97300d0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:14.372 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.370+0000 7f972f7fe640 1 -- 172.21.15.88:0/1219617839 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97300d1de0 con 0x7f97301602f0 2023-10-16T20:05:14.372 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.370+0000 7f972effd640 1 -- 172.21.15.88:0/1219617839 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1085959910 0 0) 0x7f97300d1540 con 0x7f973015ff10 2023-10-16T20:05:14.373 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.370+0000 7f972f7fe640 1 --2- 172.21.15.88:0/1219617839 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97301602f0 0x7f97300cd940 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f97300b50a0 tx=0x7f972401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b261e9fbab1cc49 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:14.373 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.370+0000 7f972effd640 1 -- 172.21.15.88:0/1219617839 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97240ab600 con 0x7f97301602f0 2023-10-16T20:05:14.374 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.370+0000 7f972effd640 1 -- 172.21.15.88:0/1219617839 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97240a7050 con 0x7f97301602f0 2023-10-16T20:05:14.374 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.370+0000 7f972effd640 1 -- 172.21.15.88:0/1219617839 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97240ab980 con 0x7f97301602f0 2023-10-16T20:05:14.374 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.374+0000 7f97367f8640 1 -- 172.21.15.88:0/1219617839 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97301602f0 msgr2=0x7f97300cd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:14.374 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.374+0000 7f97367f8640 1 --2- 172.21.15.88:0/1219617839 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97301602f0 0x7f97300cd940 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f97300b50a0 tx=0x7f972401c460 comp rx=0 tx=0).stop 2023-10-16T20:05:14.374 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.374+0000 7f97367f8640 1 -- 172.21.15.88:0/1219617839 shutdown_connections 2023-10-16T20:05:14.375 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.374+0000 7f97367f8640 1 --2- 172.21.15.88:0/1219617839 >> v2:172.21.15.94:3300/0 conn(0x7f97300d25e0 0x7f97300cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:14.375 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.374+0000 7f97367f8640 1 --2- 172.21.15.88:0/1219617839 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97301602f0 0x7f97300cd940 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:14.375 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.374+0000 7f97367f8640 1 -- 172.21.15.88:0/1219617839 >> 172.21.15.88:0/1219617839 conn(0x7f973006b6b0 msgr2=0x7f973006bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:14.375 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.374+0000 7f97367f8640 1 -- 172.21.15.88:0/1219617839 shutdown_connections 2023-10-16T20:05:14.375 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.374+0000 7f97367f8640 1 -- 172.21.15.88:0/1219617839 wait complete. 2023-10-16T20:05:14.377 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.374+0000 7f97367f8640 1 Processor -- start 2023-10-16T20:05:14.377 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.374+0000 7f97367f8640 1 -- start start 2023-10-16T20:05:14.377 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.374+0000 7f97367f8640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f97300d25e0 0x7f97300c5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:14.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.374+0000 7f97367f8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f973015ff10 0x7f9730154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:14.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.374+0000 7f97367f8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97301602f0 0x7f9730155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:14.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.374+0000 7f97367f8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97301575f0 con 0x7f97301602f0 2023-10-16T20:05:14.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.374+0000 7f97367f8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97300c5c10 con 0x7f973015ff10 2023-10-16T20:05:14.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.374+0000 7f97367f8640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97300c5ed0 con 0x7f97300d25e0 2023-10-16T20:05:14.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.374+0000 7f9734d95640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97301602f0 0x7f9730155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:14.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.374+0000 7f9734d95640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97301602f0 0x7f9730155530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40130/0 (socket says 172.21.15.88:40130) 2023-10-16T20:05:14.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.374+0000 7f9734d95640 1 -- 172.21.15.88:0/3562188397 learned_addr learned my addr 172.21.15.88:0/3562188397 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:14.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.374+0000 7f972f7fe640 1 --2- 172.21.15.88:0/3562188397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f973015ff10 0x7f9730154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:14.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.374+0000 7f972ffff640 1 --2- 172.21.15.88:0/3562188397 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f97300d25e0 0x7f97300c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:14.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.374+0000 7f9734d95640 1 -- 172.21.15.88:0/3562188397 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f97300d25e0 msgr2=0x7f97300c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:14.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.374+0000 7f9734d95640 1 --2- 172.21.15.88:0/3562188397 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f97300d25e0 0x7f97300c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:14.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.374+0000 7f9734d95640 1 -- 172.21.15.88:0/3562188397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f973015ff10 msgr2=0x7f9730154ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:14.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.374+0000 7f9734d95640 1 --2- 172.21.15.88:0/3562188397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f973015ff10 0x7f9730154ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:14.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.374+0000 7f9734d95640 1 -- 172.21.15.88:0/3562188397 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9724009420 con 0x7f97301602f0 2023-10-16T20:05:14.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.378+0000 7f9734d95640 1 --2- 172.21.15.88:0/3562188397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97301602f0 0x7f9730155530 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9728007e40 tx=0x7f972800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:14.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.378+0000 7f972cff9640 1 -- 172.21.15.88:0/3562188397 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9728014620 con 0x7f97301602f0 2023-10-16T20:05:14.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.378+0000 7f972cff9640 1 -- 172.21.15.88:0/3562188397 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f972800ee90 con 0x7f97301602f0 2023-10-16T20:05:14.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.378+0000 7f972cff9640 1 -- 172.21.15.88:0/3562188397 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97280149e0 con 0x7f97301602f0 2023-10-16T20:05:14.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.378+0000 7f97367f8640 1 -- 172.21.15.88:0/3562188397 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9730156cc0 con 0x7f97301602f0 2023-10-16T20:05:14.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.378+0000 7f97367f8640 1 -- 172.21.15.88:0/3562188397 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9730157280 con 0x7f97301602f0 2023-10-16T20:05:14.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.378+0000 7f97367f8640 1 -- 172.21.15.88:0/3562188397 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9730157730 con 0x7f97301602f0 2023-10-16T20:05:14.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.378+0000 7f972cff9640 1 -- 172.21.15.88:0/3562188397 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9728014b40 con 0x7f97301602f0 2023-10-16T20:05:14.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.378+0000 7f972cff9640 1 --2- 172.21.15.88:0/3562188397 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f971c0664a0 0x7f971c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:14.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.378+0000 7f972cff9640 1 -- 172.21.15.88:0/3562188397 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f97280849e0 con 0x7f97301602f0 2023-10-16T20:05:14.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.378+0000 7f972ffff640 1 --2- 172.21.15.88:0/3562188397 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f971c0664a0 0x7f971c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:14.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.382+0000 7f972ffff640 1 --2- 172.21.15.88:0/3562188397 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f971c0664a0 0x7f971c068960 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9720007fd0 tx=0x7f9720007d60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:14.384 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.382+0000 7f972cff9640 1 -- 172.21.15.88:0/3562188397 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9730157730 con 0x7f97301602f0 2023-10-16T20:05:14.567 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:14.566+0000 7f97367f8640 1 -- 172.21.15.88:0/3562188397 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9730157730 con 0x7f97301602f0 2023-10-16T20:05:15.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.210+0000 7f972cff9640 1 -- 172.21.15.88:0/3562188397 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9730157730 con 0x7f97301602f0 2023-10-16T20:05:15.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.210+0000 7f97367f8640 1 -- 172.21.15.88:0/3562188397 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f971c0664a0 msgr2=0x7f971c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:15.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.210+0000 7f97367f8640 1 --2- 172.21.15.88:0/3562188397 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f971c0664a0 0x7f971c068960 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9720007fd0 tx=0x7f9720007d60 comp rx=0 tx=0).stop 2023-10-16T20:05:15.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.210+0000 7f97367f8640 1 -- 172.21.15.88:0/3562188397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97301602f0 msgr2=0x7f9730155530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:15.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.210+0000 7f97367f8640 1 --2- 172.21.15.88:0/3562188397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97301602f0 0x7f9730155530 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9728007e40 tx=0x7f972800e4b0 comp rx=0 tx=0).stop 2023-10-16T20:05:15.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.210+0000 7f97367f8640 1 -- 172.21.15.88:0/3562188397 shutdown_connections 2023-10-16T20:05:15.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.210+0000 7f97367f8640 1 --2- 172.21.15.88:0/3562188397 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f971c0664a0 0x7f971c068960 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:15.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.210+0000 7f97367f8640 1 --2- 172.21.15.88:0/3562188397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97301602f0 0x7f9730155530 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:15.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.210+0000 7f97367f8640 1 --2- 172.21.15.88:0/3562188397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f973015ff10 0x7f9730154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:15.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.210+0000 7f97367f8640 1 --2- 172.21.15.88:0/3562188397 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f97300d25e0 0x7f97300c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:15.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.210+0000 7f97367f8640 1 -- 172.21.15.88:0/3562188397 >> 172.21.15.88:0/3562188397 conn(0x7f973006b6b0 msgr2=0x7f97300cd070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:15.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.210+0000 7f97367f8640 1 -- 172.21.15.88:0/3562188397 shutdown_connections 2023-10-16T20:05:15.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.210+0000 7f97367f8640 1 -- 172.21.15.88:0/3562188397 wait complete. 2023-10-16T20:05:15.235 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T20:05:15.235 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T20:05:15.376 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.374+0000 7ff1f3d3e640 1 Processor -- start 2023-10-16T20:05:15.376 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.374+0000 7ff1f3d3e640 1 -- start start 2023-10-16T20:05:15.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.378+0000 7ff1f3d3e640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff1ec160040 0x7ff1ec071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:15.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.378+0000 7ff1f3d3e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1ec160420 0x7ff1ec069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:15.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.378+0000 7ff1f3d3e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1ec0cd7d0 con 0x7ff1ec06a3d0 2023-10-16T20:05:15.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.378+0000 7ff1f3d3e640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1ec06a860 con 0x7ff1ec160040 2023-10-16T20:05:15.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.378+0000 7ff1f3d3e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec0cdab0 con 0x7ff1ec160420 2023-10-16T20:05:15.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.378+0000 7ff1f1ada640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff1ec160040 0x7ff1ec071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:15.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.378+0000 7ff1f1ada640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff1ec160040 0x7ff1ec071f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40138/0 (socket says 172.21.15.88:40138) 2023-10-16T20:05:15.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.378+0000 7ff1f1ada640 1 -- 172.21.15.88:0/1914110243 learned_addr learned my addr 172.21.15.88:0/1914110243 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:15.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.378+0000 7ff1f12d9640 1 --2- 172.21.15.88:0/1914110243 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1ec160420 0x7ff1ec069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:15.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.378+0000 7ff1f1ada640 1 -- 172.21.15.88:0/1914110243 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1ec160420 msgr2=0x7ff1ec069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:15.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.378+0000 7ff1f1ada640 1 --2- 172.21.15.88:0/1914110243 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1ec160420 0x7ff1ec069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:15.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.378+0000 7ff1f1ada640 1 -- 172.21.15.88:0/1914110243 >> v1:172.21.15.94:6789/0 conn(0x7ff1ec06a3d0 legacy=0x7ff1ec1608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:15.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.378+0000 7ff1f1ada640 1 -- 172.21.15.88:0/1914110243 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ec0cdd90 con 0x7ff1ec160040 2023-10-16T20:05:15.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.378+0000 7ff1f12d9640 1 --2- 172.21.15.88:0/1914110243 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1ec160420 0x7ff1ec069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:15.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.378+0000 7ff1f1ada640 1 --2- 172.21.15.88:0/1914110243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1ec160040 0x7ff1ec071f10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009870 tx=0x7ff1e801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90687f8f377d6c01 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:15.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.382+0000 7ff1f0ad8640 1 -- 172.21.15.88:0/1914110243 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e80aa430 con 0x7ff1ec160040 2023-10-16T20:05:15.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.382+0000 7ff1f0ad8640 1 -- 172.21.15.88:0/1914110243 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e80aa590 con 0x7ff1ec160040 2023-10-16T20:05:15.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.382+0000 7ff1f0ad8640 1 -- 172.21.15.88:0/1914110243 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e80aa920 con 0x7ff1ec160040 2023-10-16T20:05:15.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.382+0000 7ff1f3d3e640 1 -- 172.21.15.88:0/1914110243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1ec160040 msgr2=0x7ff1ec071f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:15.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.382+0000 7ff1f3d3e640 1 --2- 172.21.15.88:0/1914110243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1ec160040 0x7ff1ec071f10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009870 tx=0x7ff1e801c510 comp rx=0 tx=0).stop 2023-10-16T20:05:15.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.382+0000 7ff1f3d3e640 1 -- 172.21.15.88:0/1914110243 shutdown_connections 2023-10-16T20:05:15.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.382+0000 7ff1f3d3e640 1 --2- 172.21.15.88:0/1914110243 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1ec160420 0x7ff1ec069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:15.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.382+0000 7ff1f3d3e640 1 --2- 172.21.15.88:0/1914110243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1ec160040 0x7ff1ec071f10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:15.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.382+0000 7ff1f3d3e640 1 -- 172.21.15.88:0/1914110243 >> 172.21.15.88:0/1914110243 conn(0x7ff1ec06d340 msgr2=0x7ff1ec06d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:15.384 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.382+0000 7ff1f3d3e640 1 -- 172.21.15.88:0/1914110243 shutdown_connections 2023-10-16T20:05:15.384 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.382+0000 7ff1f3d3e640 1 -- 172.21.15.88:0/1914110243 wait complete. 2023-10-16T20:05:15.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.382+0000 7ff1f3d3e640 1 Processor -- start 2023-10-16T20:05:15.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.382+0000 7ff1f3d3e640 1 -- start start 2023-10-16T20:05:15.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.382+0000 7ff1f3d3e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1ec06a3d0 0x7ff1ec149f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:15.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.382+0000 7ff1f3d3e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1ec160040 0x7ff1ec154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:15.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.382+0000 7ff1f3d3e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1ec160420 0x7ff1ec1546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:15.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.382+0000 7ff1f3d3e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec14a4b0 con 0x7ff1ec160040 2023-10-16T20:05:15.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.382+0000 7ff1f3d3e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec0b2450 con 0x7ff1ec06a3d0 2023-10-16T20:05:15.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.382+0000 7ff1f3d3e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec0b0450 con 0x7ff1ec160420 2023-10-16T20:05:15.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.382+0000 7ff1f12d9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1ec160040 0x7ff1ec154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:15.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.382+0000 7ff1f12d9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1ec160040 0x7ff1ec154180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40144/0 (socket says 172.21.15.88:40144) 2023-10-16T20:05:15.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.382+0000 7ff1f12d9640 1 -- 172.21.15.88:0/3017228082 learned_addr learned my addr 172.21.15.88:0/3017228082 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:15.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.386+0000 7ff1f1ada640 1 --2- 172.21.15.88:0/3017228082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1ec06a3d0 0x7ff1ec149f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:15.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.386+0000 7ff1f22db640 1 --2- 172.21.15.88:0/3017228082 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1ec160420 0x7ff1ec1546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:15.388 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.386+0000 7ff1f12d9640 1 -- 172.21.15.88:0/3017228082 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1ec160420 msgr2=0x7ff1ec1546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:15.388 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.386+0000 7ff1f12d9640 1 --2- 172.21.15.88:0/3017228082 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1ec160420 0x7ff1ec1546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:15.388 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.386+0000 7ff1f12d9640 1 -- 172.21.15.88:0/3017228082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1ec06a3d0 msgr2=0x7ff1ec149f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:15.388 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.386+0000 7ff1f12d9640 1 --2- 172.21.15.88:0/3017228082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1ec06a3d0 0x7ff1ec149f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:15.388 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.386+0000 7ff1f12d9640 1 -- 172.21.15.88:0/3017228082 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e80094d0 con 0x7ff1ec160040 2023-10-16T20:05:15.389 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.386+0000 7ff1f12d9640 1 --2- 172.21.15.88:0/3017228082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1ec160040 0x7ff1ec154180 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00e810 tx=0x7ff1dc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:15.390 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.386+0000 7ff1e27fc640 1 -- 172.21.15.88:0/3017228082 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc00c8d0 con 0x7ff1ec160040 2023-10-16T20:05:15.390 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.386+0000 7ff1e27fc640 1 -- 172.21.15.88:0/3017228082 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1dc00ca30 con 0x7ff1ec160040 2023-10-16T20:05:15.390 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.386+0000 7ff1e27fc640 1 -- 172.21.15.88:0/3017228082 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc00cdc0 con 0x7ff1ec160040 2023-10-16T20:05:15.390 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.386+0000 7ff1f3d3e640 1 -- 172.21.15.88:0/3017228082 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1ec0b26c0 con 0x7ff1ec160040 2023-10-16T20:05:15.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.386+0000 7ff1f3d3e640 1 -- 172.21.15.88:0/3017228082 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1ec0b0730 con 0x7ff1ec160040 2023-10-16T20:05:15.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.386+0000 7ff1f3d3e640 1 -- 172.21.15.88:0/3017228082 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1ec04e9c0 con 0x7ff1ec160040 2023-10-16T20:05:15.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.386+0000 7ff1e27fc640 1 -- 172.21.15.88:0/3017228082 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff1dc013410 con 0x7ff1ec160040 2023-10-16T20:05:15.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.386+0000 7ff1e27fc640 1 --2- 172.21.15.88:0/3017228082 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff1d00664a0 0x7ff1d0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:15.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.386+0000 7ff1e27fc640 1 -- 172.21.15.88:0/3017228082 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff1dc083f30 con 0x7ff1ec160040 2023-10-16T20:05:15.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.386+0000 7ff1f1ada640 1 --2- 172.21.15.88:0/3017228082 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff1d00664a0 0x7ff1d0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:15.392 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.390+0000 7ff1f1ada640 1 --2- 172.21.15.88:0/3017228082 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff1d00664a0 0x7ff1d0068960 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80ae000 tx=0x7ff1e80b5040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:15.392 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.390+0000 7ff1e27fc640 1 -- 172.21.15.88:0/3017228082 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff1dc0510d0 con 0x7ff1ec160040 2023-10-16T20:05:15.574 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.574+0000 7ff1f3d3e640 1 -- 172.21.15.88:0/3017228082 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1ec0b5710 con 0x7ff1ec160040 2023-10-16T20:05:15.575 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.574+0000 7ff1e27fc640 1 -- 172.21.15.88:0/3017228082 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4906 (secure 0 0 0) 0x7ff1dc0546f0 con 0x7ff1ec160040 2023-10-16T20:05:15.576 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:15.576 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3803627113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3803627113},{"type":"v1","addr":"172.21.15.94:6834","nonce":3803627113}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3758093239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":3758093239},{"type":"v1","addr":"172.21.15.88:6837","nonce":3758093239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/3836772385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3836772385},{"type":"v1","addr":"172.21.15.88:6836","nonce":3836772385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T20:04:56.759719+0000","modified":"2023-10-16T20:05:14.569618+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6835/555575750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":555575750},{"type":"v1","addr":"172.21.15.94:6835","nonce":555575750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T20:05:15.578 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 9 2023-10-16T20:05:15.578 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.574+0000 7ff1f3d3e640 1 -- 172.21.15.88:0/3017228082 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff1d00664a0 msgr2=0x7ff1d0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:15.579 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.574+0000 7ff1f3d3e640 1 --2- 172.21.15.88:0/3017228082 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff1d00664a0 0x7ff1d0068960 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80ae000 tx=0x7ff1e80b5040 comp rx=0 tx=0).stop 2023-10-16T20:05:15.579 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.574+0000 7ff1f3d3e640 1 -- 172.21.15.88:0/3017228082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1ec160040 msgr2=0x7ff1ec154180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:15.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.574+0000 7ff1f3d3e640 1 --2- 172.21.15.88:0/3017228082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1ec160040 0x7ff1ec154180 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00e810 tx=0x7ff1dc00ecd0 comp rx=0 tx=0).stop 2023-10-16T20:05:15.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.574+0000 7ff1f3d3e640 1 -- 172.21.15.88:0/3017228082 shutdown_connections 2023-10-16T20:05:15.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.574+0000 7ff1f3d3e640 1 --2- 172.21.15.88:0/3017228082 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff1d00664a0 0x7ff1d0068960 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:15.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.574+0000 7ff1f3d3e640 1 --2- 172.21.15.88:0/3017228082 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1ec160420 0x7ff1ec1546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:15.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.574+0000 7ff1f3d3e640 1 --2- 172.21.15.88:0/3017228082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1ec160040 0x7ff1ec154180 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:15.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.574+0000 7ff1f3d3e640 1 --2- 172.21.15.88:0/3017228082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1ec06a3d0 0x7ff1ec149f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:15.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.574+0000 7ff1f3d3e640 1 -- 172.21.15.88:0/3017228082 >> 172.21.15.88:0/3017228082 conn(0x7ff1ec06d340 msgr2=0x7ff1ec155bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:15.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.574+0000 7ff1f3d3e640 1 -- 172.21.15.88:0/3017228082 shutdown_connections 2023-10-16T20:05:15.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.574+0000 7ff1f3d3e640 1 -- 172.21.15.88:0/3017228082 wait complete. 2023-10-16T20:05:15.595 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:05:15.732 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.730+0000 7f6892b3e640 1 Processor -- start 2023-10-16T20:05:15.732 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.730+0000 7f6892b3e640 1 -- start start 2023-10-16T20:05:15.733 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.734+0000 7f6892b3e640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f688c15ae40 0x7f688c0cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:15.734 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.734+0000 7f6892b3e640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f688c0cba30 con 0x7f688c0af730 2023-10-16T20:05:15.734 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.734+0000 7f6892b3e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f688c0cbd10 con 0x7f688c15aa60 2023-10-16T20:05:15.734 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.734+0000 7f6892b3e640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f688c0ce9c0 con 0x7f688c15ae40 2023-10-16T20:05:15.734 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.734+0000 7f6883fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f688c15ae40 0x7f688c0cf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:15.734 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.734+0000 7f6883fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f688c15ae40 0x7f688c0cf4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40154/0 (socket says 172.21.15.88:40154) 2023-10-16T20:05:15.735 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.734+0000 7f6883fff640 1 -- 172.21.15.88:0/4009481193 learned_addr learned my addr 172.21.15.88:0/4009481193 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:15.735 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.734+0000 7f6883fff640 1 -- 172.21.15.88:0/4009481193 >> v1:172.21.15.94:6789/0 conn(0x7f688c15aa60 legacy=0x7f688c0cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:15.735 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.734+0000 7f6883fff640 1 -- 172.21.15.88:0/4009481193 >> v1:172.21.15.88:6789/0 conn(0x7f688c0af730 legacy=0x7f688c0b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:15.736 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.734+0000 7f6883fff640 1 -- 172.21.15.88:0/4009481193 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f688c0ceca0 con 0x7f688c15ae40 2023-10-16T20:05:15.736 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.734+0000 7f68837fe640 1 -- 172.21.15.88:0/4009481193 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4135325810 0 0) 0x7f688c0cba30 con 0x7f688c0af730 2023-10-16T20:05:15.737 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.734+0000 7f6883fff640 1 --2- 172.21.15.88:0/4009481193 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f688c15ae40 0x7f688c0cf4a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f687c009870 tx=0x7f687c01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82eb1f44292aa5b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:15.737 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.738+0000 7f68837fe640 1 -- 172.21.15.88:0/4009481193 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c0ab650 con 0x7f688c15ae40 2023-10-16T20:05:15.737 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.738+0000 7f68837fe640 1 -- 172.21.15.88:0/4009481193 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f687c0ab7b0 con 0x7f688c15ae40 2023-10-16T20:05:15.738 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.738+0000 7f68837fe640 1 -- 172.21.15.88:0/4009481193 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c0abb40 con 0x7f688c15ae40 2023-10-16T20:05:15.738 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.738+0000 7f6892b3e640 1 -- 172.21.15.88:0/4009481193 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f688c15ae40 msgr2=0x7f688c0cf4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:15.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.738+0000 7f6892b3e640 1 --2- 172.21.15.88:0/4009481193 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f688c15ae40 0x7f688c0cf4a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f687c009870 tx=0x7f687c01c540 comp rx=0 tx=0).stop 2023-10-16T20:05:15.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.738+0000 7f6892b3e640 1 -- 172.21.15.88:0/4009481193 shutdown_connections 2023-10-16T20:05:15.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.738+0000 7f6892b3e640 1 --2- 172.21.15.88:0/4009481193 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f688c15ae40 0x7f688c0cf4a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:15.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.738+0000 7f6892b3e640 1 -- 172.21.15.88:0/4009481193 >> 172.21.15.88:0/4009481193 conn(0x7f688c0cd370 msgr2=0x7f688c0cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:15.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.738+0000 7f6892b3e640 1 -- 172.21.15.88:0/4009481193 shutdown_connections 2023-10-16T20:05:15.740 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.738+0000 7f6892b3e640 1 -- 172.21.15.88:0/4009481193 wait complete. 2023-10-16T20:05:15.742 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.738+0000 7f6892b3e640 1 Processor -- start 2023-10-16T20:05:15.742 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.738+0000 7f6892b3e640 1 -- start start 2023-10-16T20:05:15.742 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.738+0000 7f6892b3e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f688c0af730 0x7f688c156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:15.742 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.738+0000 7f6892b3e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f688c15aa60 0x7f688c14b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:15.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.738+0000 7f6892b3e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f688c15ae40 0x7f688c155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:15.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.738+0000 7f6892b3e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688c157e50 con 0x7f688c0af730 2023-10-16T20:05:15.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.738+0000 7f6892b3e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688c1572d0 con 0x7f688c15ae40 2023-10-16T20:05:15.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.738+0000 7f6892b3e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f688c1575b0 con 0x7f688c15aa60 2023-10-16T20:05:15.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.738+0000 7f68908da640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f688c0af730 0x7f688c156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:15.744 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.738+0000 7f68908da640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f688c0af730 0x7f688c156ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40158/0 (socket says 172.21.15.88:40158) 2023-10-16T20:05:15.744 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.738+0000 7f68908da640 1 -- 172.21.15.88:0/1394296524 learned_addr learned my addr 172.21.15.88:0/1394296524 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:15.744 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.742+0000 7f6883fff640 1 --2- 172.21.15.88:0/1394296524 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f688c15aa60 0x7f688c14b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:15.744 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.742+0000 7f68910db640 1 --2- 172.21.15.88:0/1394296524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f688c15ae40 0x7f688c155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:15.744 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.742+0000 7f68908da640 1 -- 172.21.15.88:0/1394296524 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f688c15aa60 msgr2=0x7f688c14b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:15.745 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.742+0000 7f68908da640 1 --2- 172.21.15.88:0/1394296524 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f688c15aa60 0x7f688c14b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:15.745 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.742+0000 7f68908da640 1 -- 172.21.15.88:0/1394296524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f688c15ae40 msgr2=0x7f688c155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:15.745 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.742+0000 7f68908da640 1 --2- 172.21.15.88:0/1394296524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f688c15ae40 0x7f688c155fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:15.745 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.742+0000 7f68908da640 1 -- 172.21.15.88:0/1394296524 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687c0094d0 con 0x7f688c0af730 2023-10-16T20:05:15.745 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.742+0000 7f6883fff640 1 --2- 172.21.15.88:0/1394296524 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f688c15aa60 0x7f688c14b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:15.745 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.742+0000 7f68908da640 1 --2- 172.21.15.88:0/1394296524 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f688c0af730 0x7f688c156ad0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f688800c500 tx=0x7f688800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:15.746 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.742+0000 7f68817fa640 1 -- 172.21.15.88:0/1394296524 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6888005220 con 0x7f688c0af730 2023-10-16T20:05:15.746 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.742+0000 7f68817fa640 1 -- 172.21.15.88:0/1394296524 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6888005380 con 0x7f688c0af730 2023-10-16T20:05:15.746 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.742+0000 7f6892b3e640 1 -- 172.21.15.88:0/1394296524 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f688c157890 con 0x7f688c0af730 2023-10-16T20:05:15.746 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.742+0000 7f68817fa640 1 -- 172.21.15.88:0/1394296524 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6888007270 con 0x7f688c0af730 2023-10-16T20:05:15.746 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.742+0000 7f6892b3e640 1 -- 172.21.15.88:0/1394296524 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f688c14c490 con 0x7f688c0af730 2023-10-16T20:05:15.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.742+0000 7f6892b3e640 1 -- 172.21.15.88:0/1394296524 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f688c15d3f0 con 0x7f688c0af730 2023-10-16T20:05:15.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.742+0000 7f68817fa640 1 -- 172.21.15.88:0/1394296524 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f68880073d0 con 0x7f688c0af730 2023-10-16T20:05:15.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.742+0000 7f68817fa640 1 --2- 172.21.15.88:0/1394296524 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f68780664a0 0x7f6878068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:15.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.742+0000 7f68817fa640 1 -- 172.21.15.88:0/1394296524 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6888083ad0 con 0x7f688c0af730 2023-10-16T20:05:15.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.742+0000 7f6883fff640 1 --2- 172.21.15.88:0/1394296524 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f68780664a0 0x7f6878068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:15.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.746+0000 7f6883fff640 1 --2- 172.21.15.88:0/1394296524 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f68780664a0 0x7f6878068960 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f687c0af000 tx=0x7f687c0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:15.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.746+0000 7f68817fa640 1 -- 172.21.15.88:0/1394296524 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6888050c70 con 0x7f688c0af730 2023-10-16T20:05:15.899 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.898+0000 7f6892b3e640 1 -- 172.21.15.88:0/1394296524 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f688c04e9c0 con 0x7f688c0af730 2023-10-16T20:05:15.900 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.898+0000 7f68817fa640 1 -- 172.21.15.88:0/1394296524 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18274 (secure 0 0 0) 0x7f6888054290 con 0x7f688c0af730 2023-10-16T20:05:15.900 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:15.901 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:05:01.835630+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T20:04:54.442487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T20:04:55.766641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T20:04:59.475830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:05:15.902 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.902+0000 7f6892b3e640 1 -- 172.21.15.88:0/1394296524 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f68780664a0 msgr2=0x7f6878068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:15.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.902+0000 7f6892b3e640 1 --2- 172.21.15.88:0/1394296524 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f68780664a0 0x7f6878068960 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f687c0af000 tx=0x7f687c0a7000 comp rx=0 tx=0).stop 2023-10-16T20:05:15.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.902+0000 7f6892b3e640 1 -- 172.21.15.88:0/1394296524 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f688c0af730 msgr2=0x7f688c156ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:15.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.902+0000 7f6892b3e640 1 --2- 172.21.15.88:0/1394296524 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f688c0af730 0x7f688c156ad0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f688800c500 tx=0x7f688800c9c0 comp rx=0 tx=0).stop 2023-10-16T20:05:15.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.902+0000 7f6892b3e640 1 -- 172.21.15.88:0/1394296524 shutdown_connections 2023-10-16T20:05:15.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.902+0000 7f6892b3e640 1 --2- 172.21.15.88:0/1394296524 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f68780664a0 0x7f6878068960 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:15.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.902+0000 7f6892b3e640 1 --2- 172.21.15.88:0/1394296524 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f688c15aa60 0x7f688c14b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:15.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.902+0000 7f6892b3e640 1 --2- 172.21.15.88:0/1394296524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f688c15ae40 0x7f688c155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:15.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.902+0000 7f6892b3e640 1 --2- 172.21.15.88:0/1394296524 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f688c0af730 0x7f688c156ad0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:15.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.902+0000 7f6892b3e640 1 -- 172.21.15.88:0/1394296524 >> 172.21.15.88:0/1394296524 conn(0x7f688c0cd370 msgr2=0x7f688c0b1670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:15.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.902+0000 7f6892b3e640 1 -- 172.21.15.88:0/1394296524 shutdown_connections 2023-10-16T20:05:15.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:15.902+0000 7f6892b3e640 1 -- 172.21.15.88:0/1394296524 wait complete. 2023-10-16T20:05:15.922 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T20:05:15.922 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T20:05:16.059 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.058+0000 7ff064f94640 1 Processor -- start 2023-10-16T20:05:16.060 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.058+0000 7ff064f94640 1 -- start start 2023-10-16T20:05:16.060 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.058+0000 7ff064f94640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff0600b0c10 0x7ff0600b1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:16.061 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.058+0000 7ff064f94640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0600b2570 con 0x7ff0600ada40 2023-10-16T20:05:16.061 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.058+0000 7ff064f94640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0600cca50 con 0x7ff0600ade20 2023-10-16T20:05:16.061 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.058+0000 7ff064f94640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0600b1fb0 con 0x7ff0600b0c10 2023-10-16T20:05:16.061 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.058+0000 7ff05e59c640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7ff0600ada40 0x7ff06015adb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:39134/0 (socket says 172.21.15.88:39134) 2023-10-16T20:05:16.062 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.058+0000 7ff05e59c640 1 -- 172.21.15.88:0/1201283579 learned_addr learned my addr 172.21.15.88:0/1201283579 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:16.063 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.062+0000 7ff05ed9d640 1 --2- 172.21.15.88:0/1201283579 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff0600b0c10 0x7ff0600b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:16.063 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.062+0000 7ff05d59a640 1 -- 172.21.15.88:0/1201283579 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2491726147 0 0) 0x7ff0600b2570 con 0x7ff0600ada40 2023-10-16T20:05:16.063 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.062+0000 7ff05d59a640 1 -- 172.21.15.88:0/1201283579 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff048003530 con 0x7ff0600ada40 2023-10-16T20:05:16.063 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.062+0000 7ff05d59a640 1 -- 172.21.15.88:0/1201283579 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3472671494 0 0) 0x7ff0600cca50 con 0x7ff0600ade20 2023-10-16T20:05:16.063 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.062+0000 7ff05d59a640 1 -- 172.21.15.88:0/1201283579 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0600b2570 con 0x7ff0600ade20 2023-10-16T20:05:16.064 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.062+0000 7ff05ed9d640 1 -- 172.21.15.88:0/1201283579 >> v1:172.21.15.94:6789/0 conn(0x7ff0600ade20 legacy=0x7ff0600b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:16.064 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.062+0000 7ff05ed9d640 1 -- 172.21.15.88:0/1201283579 >> v1:172.21.15.88:6789/0 conn(0x7ff0600ada40 legacy=0x7ff06015adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:16.064 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.062+0000 7ff05ed9d640 1 -- 172.21.15.88:0/1201283579 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0600b2290 con 0x7ff0600b0c10 2023-10-16T20:05:16.065 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.062+0000 7ff05ed9d640 1 --2- 172.21.15.88:0/1201283579 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff0600b0c10 0x7ff0600b1920 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff058009dc0 tx=0x7ff05801c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3d5b4e5b75a7015 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:16.065 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.062+0000 7ff05d59a640 1 -- 172.21.15.88:0/1201283579 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0580a5990 con 0x7ff0600b0c10 2023-10-16T20:05:16.065 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.062+0000 7ff05d59a640 1 -- 172.21.15.88:0/1201283579 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0580a5af0 con 0x7ff0600b0c10 2023-10-16T20:05:16.065 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.062+0000 7ff05d59a640 1 -- 172.21.15.88:0/1201283579 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0580a5e60 con 0x7ff0600b0c10 2023-10-16T20:05:16.066 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.066+0000 7ff064f94640 1 -- 172.21.15.88:0/1201283579 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff0600b0c10 msgr2=0x7ff0600b1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:16.066 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.066+0000 7ff064f94640 1 --2- 172.21.15.88:0/1201283579 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff0600b0c10 0x7ff0600b1920 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff058009dc0 tx=0x7ff05801c8b0 comp rx=0 tx=0).stop 2023-10-16T20:05:16.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.066+0000 7ff064f94640 1 -- 172.21.15.88:0/1201283579 shutdown_connections 2023-10-16T20:05:16.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.066+0000 7ff064f94640 1 --2- 172.21.15.88:0/1201283579 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff0600b0c10 0x7ff0600b1920 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:16.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.066+0000 7ff064f94640 1 -- 172.21.15.88:0/1201283579 >> 172.21.15.88:0/1201283579 conn(0x7ff0600ce5e0 msgr2=0x7ff0600ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:16.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.066+0000 7ff064f94640 1 -- 172.21.15.88:0/1201283579 shutdown_connections 2023-10-16T20:05:16.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.066+0000 7ff064f94640 1 -- 172.21.15.88:0/1201283579 wait complete. 2023-10-16T20:05:16.069 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.066+0000 7ff064f94640 1 Processor -- start 2023-10-16T20:05:16.069 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.066+0000 7ff064f94640 1 -- start start 2023-10-16T20:05:16.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.066+0000 7ff064f94640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0600ada40 0x7ff06015ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:16.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.066+0000 7ff064f94640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff0600ade20 0x7ff060139ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:16.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.066+0000 7ff064f94640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0600b0c10 0x7ff0600af5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:16.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.066+0000 7ff064f94640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06015b380 con 0x7ff0600ada40 2023-10-16T20:05:16.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.066+0000 7ff064f94640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0600ab890 con 0x7ff0600b0c10 2023-10-16T20:05:16.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.066+0000 7ff064f94640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff06013a940 con 0x7ff0600ade20 2023-10-16T20:05:16.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.066+0000 7ff05e59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0600ada40 0x7ff06015ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:16.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.066+0000 7ff05e59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0600ada40 0x7ff06015ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40162/0 (socket says 172.21.15.88:40162) 2023-10-16T20:05:16.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.066+0000 7ff05e59c640 1 -- 172.21.15.88:0/2532791909 learned_addr learned my addr 172.21.15.88:0/2532791909 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:16.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.066+0000 7ff05ed9d640 1 --2- 172.21.15.88:0/2532791909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0600b0c10 0x7ff0600af5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:16.072 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.066+0000 7ff05dd9b640 1 --2- 172.21.15.88:0/2532791909 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff0600ade20 0x7ff060139ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:16.072 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.070+0000 7ff05e59c640 1 -- 172.21.15.88:0/2532791909 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff0600ade20 msgr2=0x7ff060139ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:16.072 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.070+0000 7ff05e59c640 1 --2- 172.21.15.88:0/2532791909 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff0600ade20 0x7ff060139ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:16.072 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.070+0000 7ff05e59c640 1 -- 172.21.15.88:0/2532791909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0600b0c10 msgr2=0x7ff0600af5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:16.073 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.070+0000 7ff05e59c640 1 --2- 172.21.15.88:0/2532791909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0600b0c10 0x7ff0600af5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:16.073 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.070+0000 7ff05e59c640 1 -- 172.21.15.88:0/2532791909 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff058009a70 con 0x7ff0600ada40 2023-10-16T20:05:16.073 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.070+0000 7ff05e59c640 1 --2- 172.21.15.88:0/2532791909 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0600ada40 0x7ff06015ae40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff05000c420 tx=0x7ff05000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:16.075 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.070+0000 7ff04effd640 1 -- 172.21.15.88:0/2532791909 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff050005cf0 con 0x7ff0600ada40 2023-10-16T20:05:16.075 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.070+0000 7ff04effd640 1 -- 172.21.15.88:0/2532791909 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff050005e50 con 0x7ff0600ada40 2023-10-16T20:05:16.075 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.070+0000 7ff04effd640 1 -- 172.21.15.88:0/2532791909 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff050007310 con 0x7ff0600ada40 2023-10-16T20:05:16.075 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.070+0000 7ff064f94640 1 -- 172.21.15.88:0/2532791909 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff06013ac20 con 0x7ff0600ada40 2023-10-16T20:05:16.075 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.070+0000 7ff064f94640 1 -- 172.21.15.88:0/2532791909 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff06013b1e0 con 0x7ff0600ada40 2023-10-16T20:05:16.076 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.070+0000 7ff064f94640 1 -- 172.21.15.88:0/2532791909 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff06013b400 con 0x7ff0600ada40 2023-10-16T20:05:16.076 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.070+0000 7ff04effd640 1 -- 172.21.15.88:0/2532791909 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff050007470 con 0x7ff0600ada40 2023-10-16T20:05:16.076 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.070+0000 7ff04effd640 1 --2- 172.21.15.88:0/2532791909 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff0400664a0 0x7ff040068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:16.076 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.070+0000 7ff04effd640 1 -- 172.21.15.88:0/2532791909 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff050083610 con 0x7ff0600ada40 2023-10-16T20:05:16.076 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.070+0000 7ff05dd9b640 1 --2- 172.21.15.88:0/2532791909 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff0400664a0 0x7ff040068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:16.077 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.074+0000 7ff05dd9b640 1 --2- 172.21.15.88:0/2532791909 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff0400664a0 0x7ff040068960 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff06013c450 tx=0x7ff054009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:16.077 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.074+0000 7ff04effd640 1 -- 172.21.15.88:0/2532791909 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff0500508e0 con 0x7ff0600ada40 2023-10-16T20:05:16.224 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.222+0000 7ff064f94640 1 -- 172.21.15.88:0/2532791909 --> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff06004e9c0 con 0x7ff0400664a0 2023-10-16T20:05:16.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.234+0000 7ff04effd640 1 -- 172.21.15.88:0/2532791909 <== mgr.4110 v2:172.21.15.88:6832/4078483333 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345271 (secure 0 0 0) 0x7ff06004e9c0 con 0x7ff0400664a0 2023-10-16T20:05:16.239 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:16.241 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2023-10-16T20:05:16.242 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.238+0000 7ff064f94640 1 -- 172.21.15.88:0/2532791909 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff0400664a0 msgr2=0x7ff040068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:16.242 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.238+0000 7ff064f94640 1 --2- 172.21.15.88:0/2532791909 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff0400664a0 0x7ff040068960 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff06013c450 tx=0x7ff054009040 comp rx=0 tx=0).stop 2023-10-16T20:05:16.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.238+0000 7ff064f94640 1 -- 172.21.15.88:0/2532791909 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0600ada40 msgr2=0x7ff06015ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:16.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.238+0000 7ff064f94640 1 --2- 172.21.15.88:0/2532791909 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0600ada40 0x7ff06015ae40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff05000c420 tx=0x7ff05000c8e0 comp rx=0 tx=0).stop 2023-10-16T20:05:16.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.238+0000 7ff064f94640 1 -- 172.21.15.88:0/2532791909 shutdown_connections 2023-10-16T20:05:16.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.238+0000 7ff064f94640 1 --2- 172.21.15.88:0/2532791909 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff0400664a0 0x7ff040068960 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:16.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.238+0000 7ff064f94640 1 --2- 172.21.15.88:0/2532791909 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff0600ade20 0x7ff060139ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:16.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.238+0000 7ff064f94640 1 --2- 172.21.15.88:0/2532791909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0600b0c10 0x7ff0600af5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:16.244 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.238+0000 7ff064f94640 1 --2- 172.21.15.88:0/2532791909 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0600ada40 0x7ff06015ae40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:16.244 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.238+0000 7ff064f94640 1 -- 172.21.15.88:0/2532791909 >> 172.21.15.88:0/2532791909 conn(0x7ff0600ce5e0 msgr2=0x7ff06015b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:16.244 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.238+0000 7ff064f94640 1 -- 172.21.15.88:0/2532791909 shutdown_connections 2023-10-16T20:05:16.244 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.238+0000 7ff064f94640 1 -- 172.21.15.88:0/2532791909 wait complete. 2023-10-16T20:05:16.259 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2023-10-16T20:05:14.447296+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219628,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755092,"statfs":{"total":773094113280,"available":772869214208,"internally_reserved":0,"allocated":3514368,"data_stored":2070598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":221236793},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2229,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002744"},"pg_stats":[{"pgid":"2.1f","version":"22'4","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.465897+0000","last_change":"2023-10-16T20:04:56.406335+0000","last_active":"2023-10-16T20:05:01.465897+0000","last_peered":"2023-10-16T20:05:01.465897+0000","last_clean":"2023-10-16T20:05:01.465897+0000","last_became_active":"2023-10-16T20:04:56.406060+0000","last_became_peered":"2023-10-16T20:04:56.406060+0000","last_unstale":"2023-10-16T20:05:01.465897+0000","last_undegraded":"2023-10-16T20:05:01.465897+0000","last_fullsized":"2023-10-16T20:05:01.465897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:15:58.928603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442061+0000","last_change":"2023-10-16T20:04:57.416039+0000","last_active":"2023-10-16T20:05:01.442061+0000","last_peered":"2023-10-16T20:05:01.442061+0000","last_clean":"2023-10-16T20:05:01.442061+0000","last_became_active":"2023-10-16T20:04:57.415746+0000","last_became_peered":"2023-10-16T20:04:57.415746+0000","last_unstale":"2023-10-16T20:05:01.442061+0000","last_undegraded":"2023-10-16T20:05:01.442061+0000","last_fullsized":"2023-10-16T20:05:01.442061+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:27:55.972790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.685606+0000","last_change":"2023-10-16T20:05:01.770262+0000","last_active":"2023-10-16T20:05:02.685606+0000","last_peered":"2023-10-16T20:05:02.685606+0000","last_clean":"2023-10-16T20:05:02.685606+0000","last_became_active":"2023-10-16T20:05:01.769688+0000","last_became_peered":"2023-10-16T20:05:01.769688+0000","last_unstale":"2023-10-16T20:05:02.685606+0000","last_undegraded":"2023-10-16T20:05:02.685606+0000","last_fullsized":"2023-10-16T20:05:02.685606+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:51:12.210160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442225+0000","last_change":"2023-10-16T20:04:56.409621+0000","last_active":"2023-10-16T20:05:01.442225+0000","last_peered":"2023-10-16T20:05:01.442225+0000","last_clean":"2023-10-16T20:05:01.442225+0000","last_became_active":"2023-10-16T20:04:56.409150+0000","last_became_peered":"2023-10-16T20:04:56.409150+0000","last_unstale":"2023-10-16T20:05:01.442225+0000","last_undegraded":"2023-10-16T20:05:01.442225+0000","last_fullsized":"2023-10-16T20:05:01.442225+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:24:16.965233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.465724+0000","last_change":"2023-10-16T20:04:57.430587+0000","last_active":"2023-10-16T20:05:01.465724+0000","last_peered":"2023-10-16T20:05:01.465724+0000","last_clean":"2023-10-16T20:05:01.465724+0000","last_became_active":"2023-10-16T20:04:57.430224+0000","last_became_peered":"2023-10-16T20:04:57.430224+0000","last_unstale":"2023-10-16T20:05:01.465724+0000","last_undegraded":"2023-10-16T20:05:01.465724+0000","last_fullsized":"2023-10-16T20:05:01.465724+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:49:52.662102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.682460+0000","last_change":"2023-10-16T20:05:01.478221+0000","last_active":"2023-10-16T20:05:02.682460+0000","last_peered":"2023-10-16T20:05:02.682460+0000","last_clean":"2023-10-16T20:05:02.682460+0000","last_became_active":"2023-10-16T20:05:01.477692+0000","last_became_peered":"2023-10-16T20:05:01.477692+0000","last_unstale":"2023-10-16T20:05:02.682460+0000","last_undegraded":"2023-10-16T20:05:02.682460+0000","last_fullsized":"2023-10-16T20:05:02.682460+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:58:37.917893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.758057+0000","last_change":"2023-10-16T20:04:56.409779+0000","last_active":"2023-10-16T20:05:01.758057+0000","last_peered":"2023-10-16T20:05:01.758057+0000","last_clean":"2023-10-16T20:05:01.758057+0000","last_became_active":"2023-10-16T20:04:56.409515+0000","last_became_peered":"2023-10-16T20:04:56.409515+0000","last_unstale":"2023-10-16T20:05:01.758057+0000","last_undegraded":"2023-10-16T20:05:01.758057+0000","last_fullsized":"2023-10-16T20:05:01.758057+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:17:23.198236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438236+0000","last_change":"2023-10-16T20:04:57.414284+0000","last_active":"2023-10-16T20:05:01.438236+0000","last_peered":"2023-10-16T20:05:01.438236+0000","last_clean":"2023-10-16T20:05:01.438236+0000","last_became_active":"2023-10-16T20:04:57.413997+0000","last_became_peered":"2023-10-16T20:04:57.413997+0000","last_unstale":"2023-10-16T20:05:01.438236+0000","last_undegraded":"2023-10-16T20:05:01.438236+0000","last_fullsized":"2023-10-16T20:05:01.438236+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:28:00.096002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.682751+0000","last_change":"2023-10-16T20:05:01.482429+0000","last_active":"2023-10-16T20:05:02.682751+0000","last_peered":"2023-10-16T20:05:02.682751+0000","last_clean":"2023-10-16T20:05:02.682751+0000","last_became_active":"2023-10-16T20:05:01.481993+0000","last_became_peered":"2023-10-16T20:05:01.481993+0000","last_unstale":"2023-10-16T20:05:02.682751+0000","last_undegraded":"2023-10-16T20:05:02.682751+0000","last_fullsized":"2023-10-16T20:05:02.682751+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:54:13.091947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757866+0000","last_change":"2023-10-16T20:04:56.412981+0000","last_active":"2023-10-16T20:05:01.757866+0000","last_peered":"2023-10-16T20:05:01.757866+0000","last_clean":"2023-10-16T20:05:01.757866+0000","last_became_active":"2023-10-16T20:04:56.412613+0000","last_became_peered":"2023-10-16T20:04:56.412613+0000","last_unstale":"2023-10-16T20:05:01.757866+0000","last_undegraded":"2023-10-16T20:05:01.757866+0000","last_fullsized":"2023-10-16T20:05:01.757866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:14:27.634593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.440267+0000","last_change":"2023-10-16T20:04:57.411866+0000","last_active":"2023-10-16T20:05:01.440267+0000","last_peered":"2023-10-16T20:05:01.440267+0000","last_clean":"2023-10-16T20:05:01.440267+0000","last_became_active":"2023-10-16T20:04:57.411523+0000","last_became_peered":"2023-10-16T20:04:57.411523+0000","last_unstale":"2023-10-16T20:05:01.440267+0000","last_undegraded":"2023-10-16T20:05:01.440267+0000","last_fullsized":"2023-10-16T20:05:01.440267+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:01:38.242317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.683398+0000","last_change":"2023-10-16T20:05:01.460546+0000","last_active":"2023-10-16T20:05:02.683398+0000","last_peered":"2023-10-16T20:05:02.683398+0000","last_clean":"2023-10-16T20:05:02.683398+0000","last_became_active":"2023-10-16T20:05:01.460120+0000","last_became_peered":"2023-10-16T20:05:01.460120+0000","last_unstale":"2023-10-16T20:05:02.683398+0000","last_undegraded":"2023-10-16T20:05:02.683398+0000","last_fullsized":"2023-10-16T20:05:02.683398+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:31:38.939408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438705+0000","last_change":"2023-10-16T20:04:57.409882+0000","last_active":"2023-10-16T20:05:01.438705+0000","last_peered":"2023-10-16T20:05:01.438705+0000","last_clean":"2023-10-16T20:05:01.438705+0000","last_became_active":"2023-10-16T20:04:57.409534+0000","last_became_peered":"2023-10-16T20:04:57.409534+0000","last_unstale":"2023-10-16T20:05:01.438705+0000","last_undegraded":"2023-10-16T20:05:01.438705+0000","last_fullsized":"2023-10-16T20:05:01.438705+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:23:01.141545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757875+0000","last_change":"2023-10-16T20:04:56.406078+0000","last_active":"2023-10-16T20:05:01.757875+0000","last_peered":"2023-10-16T20:05:01.757875+0000","last_clean":"2023-10-16T20:05:01.757875+0000","last_became_active":"2023-10-16T20:04:56.405784+0000","last_became_peered":"2023-10-16T20:04:56.405784+0000","last_unstale":"2023-10-16T20:05:01.757875+0000","last_undegraded":"2023-10-16T20:05:01.757875+0000","last_fullsized":"2023-10-16T20:05:01.757875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:13:40.552947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.683535+0000","last_change":"2023-10-16T20:05:01.477288+0000","last_active":"2023-10-16T20:05:02.683535+0000","last_peered":"2023-10-16T20:05:02.683535+0000","last_clean":"2023-10-16T20:05:02.683535+0000","last_became_active":"2023-10-16T20:05:01.476269+0000","last_became_peered":"2023-10-16T20:05:01.476269+0000","last_unstale":"2023-10-16T20:05:02.683535+0000","last_undegraded":"2023-10-16T20:05:02.683535+0000","last_fullsized":"2023-10-16T20:05:02.683535+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:28:04.151249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.441837+0000","last_change":"2023-10-16T20:04:57.426154+0000","last_active":"2023-10-16T20:05:01.441837+0000","last_peered":"2023-10-16T20:05:01.441837+0000","last_clean":"2023-10-16T20:05:01.441837+0000","last_became_active":"2023-10-16T20:04:57.425681+0000","last_became_peered":"2023-10-16T20:04:57.425681+0000","last_unstale":"2023-10-16T20:05:01.441837+0000","last_undegraded":"2023-10-16T20:05:01.441837+0000","last_fullsized":"2023-10-16T20:05:01.441837+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:54:09.036700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471304+0000","last_change":"2023-10-16T20:04:56.406283+0000","last_active":"2023-10-16T20:05:01.471304+0000","last_peered":"2023-10-16T20:05:01.471304+0000","last_clean":"2023-10-16T20:05:01.471304+0000","last_became_active":"2023-10-16T20:04:56.405952+0000","last_became_peered":"2023-10-16T20:04:56.405952+0000","last_unstale":"2023-10-16T20:05:01.471304+0000","last_undegraded":"2023-10-16T20:05:01.471304+0000","last_fullsized":"2023-10-16T20:05:01.471304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:14:27.634593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.682647+0000","last_change":"2023-10-16T20:05:01.482495+0000","last_active":"2023-10-16T20:05:02.682647+0000","last_peered":"2023-10-16T20:05:02.682647+0000","last_clean":"2023-10-16T20:05:02.682647+0000","last_became_active":"2023-10-16T20:05:01.482072+0000","last_became_peered":"2023-10-16T20:05:01.482072+0000","last_unstale":"2023-10-16T20:05:02.682647+0000","last_undegraded":"2023-10-16T20:05:02.682647+0000","last_fullsized":"2023-10-16T20:05:02.682647+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:28:00.028037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442089+0000","last_change":"2023-10-16T20:04:57.425668+0000","last_active":"2023-10-16T20:05:01.442089+0000","last_peered":"2023-10-16T20:05:01.442089+0000","last_clean":"2023-10-16T20:05:01.442089+0000","last_became_active":"2023-10-16T20:04:57.425420+0000","last_became_peered":"2023-10-16T20:04:57.425420+0000","last_unstale":"2023-10-16T20:05:01.442089+0000","last_undegraded":"2023-10-16T20:05:01.442089+0000","last_fullsized":"2023-10-16T20:05:01.442089+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:33:36.402743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471190+0000","last_change":"2023-10-16T20:04:56.403231+0000","last_active":"2023-10-16T20:05:01.471190+0000","last_peered":"2023-10-16T20:05:01.471190+0000","last_clean":"2023-10-16T20:05:01.471190+0000","last_became_active":"2023-10-16T20:04:56.402769+0000","last_became_peered":"2023-10-16T20:04:56.402769+0000","last_unstale":"2023-10-16T20:05:01.471190+0000","last_undegraded":"2023-10-16T20:05:01.471190+0000","last_fullsized":"2023-10-16T20:05:01.471190+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:55:47.529420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.445092+0000","last_change":"2023-10-16T20:05:01.768596+0000","last_active":"2023-10-16T20:05:02.445092+0000","last_peered":"2023-10-16T20:05:02.445092+0000","last_clean":"2023-10-16T20:05:02.445092+0000","last_became_active":"2023-10-16T20:05:01.768140+0000","last_became_peered":"2023-10-16T20:05:01.768140+0000","last_unstale":"2023-10-16T20:05:02.445092+0000","last_undegraded":"2023-10-16T20:05:02.445092+0000","last_fullsized":"2023-10-16T20:05:02.445092+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:01:42.297564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.447386+0000","last_change":"2023-10-16T20:04:57.407922+0000","last_active":"2023-10-16T20:05:01.447386+0000","last_peered":"2023-10-16T20:05:01.447386+0000","last_clean":"2023-10-16T20:05:01.447386+0000","last_became_active":"2023-10-16T20:04:57.407580+0000","last_became_peered":"2023-10-16T20:04:57.407580+0000","last_unstale":"2023-10-16T20:05:01.447386+0000","last_undegraded":"2023-10-16T20:05:01.447386+0000","last_fullsized":"2023-10-16T20:05:01.447386+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:24:17.972220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757524+0000","last_change":"2023-10-16T20:04:56.408300+0000","last_active":"2023-10-16T20:05:01.757524+0000","last_peered":"2023-10-16T20:05:01.757524+0000","last_clean":"2023-10-16T20:05:01.757524+0000","last_became_active":"2023-10-16T20:04:56.407908+0000","last_became_peered":"2023-10-16T20:04:56.407908+0000","last_unstale":"2023-10-16T20:05:01.757524+0000","last_undegraded":"2023-10-16T20:05:01.757524+0000","last_fullsized":"2023-10-16T20:05:01.757524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:53:19.915183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.683126+0000","last_change":"2023-10-16T20:05:01.476964+0000","last_active":"2023-10-16T20:05:02.683126+0000","last_peered":"2023-10-16T20:05:02.683126+0000","last_clean":"2023-10-16T20:05:02.683126+0000","last_became_active":"2023-10-16T20:05:01.476584+0000","last_became_peered":"2023-10-16T20:05:01.476584+0000","last_unstale":"2023-10-16T20:05:02.683126+0000","last_undegraded":"2023-10-16T20:05:02.683126+0000","last_fullsized":"2023-10-16T20:05:02.683126+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:22:16.748229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.465558+0000","last_change":"2023-10-16T20:04:57.424772+0000","last_active":"2023-10-16T20:05:01.465558+0000","last_peered":"2023-10-16T20:05:01.465558+0000","last_clean":"2023-10-16T20:05:01.465558+0000","last_became_active":"2023-10-16T20:04:57.424340+0000","last_became_peered":"2023-10-16T20:04:57.424340+0000","last_unstale":"2023-10-16T20:05:01.465558+0000","last_undegraded":"2023-10-16T20:05:01.465558+0000","last_fullsized":"2023-10-16T20:05:01.465558+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:10:08.760019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.439134+0000","last_change":"2023-10-16T20:04:56.401336+0000","last_active":"2023-10-16T20:05:01.439134+0000","last_peered":"2023-10-16T20:05:01.439134+0000","last_clean":"2023-10-16T20:05:01.439134+0000","last_became_active":"2023-10-16T20:04:56.401019+0000","last_became_peered":"2023-10-16T20:04:56.401019+0000","last_unstale":"2023-10-16T20:05:01.439134+0000","last_undegraded":"2023-10-16T20:05:01.439134+0000","last_fullsized":"2023-10-16T20:05:01.439134+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:58:19.945263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.682734+0000","last_change":"2023-10-16T20:05:01.760505+0000","last_active":"2023-10-16T20:05:02.682734+0000","last_peered":"2023-10-16T20:05:02.682734+0000","last_clean":"2023-10-16T20:05:02.682734+0000","last_became_active":"2023-10-16T20:05:01.760043+0000","last_became_peered":"2023-10-16T20:05:01.760043+0000","last_unstale":"2023-10-16T20:05:02.682734+0000","last_undegraded":"2023-10-16T20:05:02.682734+0000","last_fullsized":"2023-10-16T20:05:02.682734+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:11:32.906879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442279+0000","last_change":"2023-10-16T20:04:57.430184+0000","last_active":"2023-10-16T20:05:01.442279+0000","last_peered":"2023-10-16T20:05:01.442279+0000","last_clean":"2023-10-16T20:05:01.442279+0000","last_became_active":"2023-10-16T20:04:57.429845+0000","last_became_peered":"2023-10-16T20:04:57.429845+0000","last_unstale":"2023-10-16T20:05:01.442279+0000","last_undegraded":"2023-10-16T20:05:01.442279+0000","last_fullsized":"2023-10-16T20:05:01.442279+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:11:28.851632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471731+0000","last_change":"2023-10-16T20:04:56.403148+0000","last_active":"2023-10-16T20:05:01.471731+0000","last_peered":"2023-10-16T20:05:01.471731+0000","last_clean":"2023-10-16T20:05:01.471731+0000","last_became_active":"2023-10-16T20:04:56.402707+0000","last_became_peered":"2023-10-16T20:04:56.402707+0000","last_unstale":"2023-10-16T20:05:01.471731+0000","last_undegraded":"2023-10-16T20:05:01.471731+0000","last_fullsized":"2023-10-16T20:05:01.471731+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:53:19.915183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.685503+0000","last_change":"2023-10-16T20:05:01.764474+0000","last_active":"2023-10-16T20:05:02.685503+0000","last_peered":"2023-10-16T20:05:02.685503+0000","last_clean":"2023-10-16T20:05:02.685503+0000","last_became_active":"2023-10-16T20:05:01.763840+0000","last_became_peered":"2023-10-16T20:05:01.763840+0000","last_unstale":"2023-10-16T20:05:02.685503+0000","last_undegraded":"2023-10-16T20:05:02.685503+0000","last_fullsized":"2023-10-16T20:05:02.685503+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:57:32.014259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471448+0000","last_change":"2023-10-16T20:04:57.423634+0000","last_active":"2023-10-16T20:05:01.471448+0000","last_peered":"2023-10-16T20:05:01.471448+0000","last_clean":"2023-10-16T20:05:01.471448+0000","last_became_active":"2023-10-16T20:04:57.423321+0000","last_became_peered":"2023-10-16T20:04:57.423321+0000","last_unstale":"2023-10-16T20:05:01.471448+0000","last_undegraded":"2023-10-16T20:05:01.471448+0000","last_fullsized":"2023-10-16T20:05:01.471448+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:23:01.141545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.758143+0000","last_change":"2023-10-16T20:04:56.408656+0000","last_active":"2023-10-16T20:05:01.758143+0000","last_peered":"2023-10-16T20:05:01.758143+0000","last_clean":"2023-10-16T20:05:01.758143+0000","last_became_active":"2023-10-16T20:04:56.408192+0000","last_became_peered":"2023-10-16T20:04:56.408192+0000","last_unstale":"2023-10-16T20:05:01.758143+0000","last_undegraded":"2023-10-16T20:05:01.758143+0000","last_fullsized":"2023-10-16T20:05:01.758143+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:54:08.029713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.688512+0000","last_change":"2023-10-16T20:05:01.767238+0000","last_active":"2023-10-16T20:05:02.688512+0000","last_peered":"2023-10-16T20:05:02.688512+0000","last_clean":"2023-10-16T20:05:02.688512+0000","last_became_active":"2023-10-16T20:05:01.765827+0000","last_became_peered":"2023-10-16T20:05:01.765827+0000","last_unstale":"2023-10-16T20:05:02.688512+0000","last_undegraded":"2023-10-16T20:05:02.688512+0000","last_fullsized":"2023-10-16T20:05:02.688512+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:14:36.535545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.440300+0000","last_change":"2023-10-16T20:04:57.430055+0000","last_active":"2023-10-16T20:05:01.440300+0000","last_peered":"2023-10-16T20:05:01.440300+0000","last_clean":"2023-10-16T20:05:01.440300+0000","last_became_active":"2023-10-16T20:04:57.428753+0000","last_became_peered":"2023-10-16T20:04:57.428753+0000","last_unstale":"2023-10-16T20:05:01.440300+0000","last_undegraded":"2023-10-16T20:05:01.440300+0000","last_fullsized":"2023-10-16T20:05:01.440300+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:14:29.143380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471458+0000","last_change":"2023-10-16T20:04:56.411374+0000","last_active":"2023-10-16T20:05:01.471458+0000","last_peered":"2023-10-16T20:05:01.471458+0000","last_clean":"2023-10-16T20:05:01.471458+0000","last_became_active":"2023-10-16T20:04:56.411098+0000","last_became_peered":"2023-10-16T20:04:56.411098+0000","last_unstale":"2023-10-16T20:05:01.471458+0000","last_undegraded":"2023-10-16T20:05:01.471458+0000","last_fullsized":"2023-10-16T20:05:01.471458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:04:32.948232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.444207+0000","last_change":"2023-10-16T20:05:01.473814+0000","last_active":"2023-10-16T20:05:02.444207+0000","last_peered":"2023-10-16T20:05:02.444207+0000","last_clean":"2023-10-16T20:05:02.444207+0000","last_became_active":"2023-10-16T20:05:01.472021+0000","last_became_peered":"2023-10-16T20:05:01.472021+0000","last_unstale":"2023-10-16T20:05:02.444207+0000","last_undegraded":"2023-10-16T20:05:02.444207+0000","last_fullsized":"2023-10-16T20:05:02.444207+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:12:00.889299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.447473+0000","last_change":"2023-10-16T20:04:57.419253+0000","last_active":"2023-10-16T20:05:01.447473+0000","last_peered":"2023-10-16T20:05:01.447473+0000","last_clean":"2023-10-16T20:05:01.447473+0000","last_became_active":"2023-10-16T20:04:57.418196+0000","last_became_peered":"2023-10-16T20:04:57.418196+0000","last_unstale":"2023-10-16T20:05:01.447473+0000","last_undegraded":"2023-10-16T20:05:01.447473+0000","last_fullsized":"2023-10-16T20:05:01.447473+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:58:20.952250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.437911+0000","last_change":"2023-10-16T20:04:56.405694+0000","last_active":"2023-10-16T20:05:01.437911+0000","last_peered":"2023-10-16T20:05:01.437911+0000","last_clean":"2023-10-16T20:05:01.437911+0000","last_became_active":"2023-10-16T20:04:56.405369+0000","last_became_peered":"2023-10-16T20:04:56.405369+0000","last_unstale":"2023-10-16T20:05:01.437911+0000","last_undegraded":"2023-10-16T20:05:01.437911+0000","last_fullsized":"2023-10-16T20:05:01.437911+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:58:32.855659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.444695+0000","last_change":"2023-10-16T20:05:01.478161+0000","last_active":"2023-10-16T20:05:02.444695+0000","last_peered":"2023-10-16T20:05:02.444695+0000","last_clean":"2023-10-16T20:05:02.444695+0000","last_became_active":"2023-10-16T20:05:01.477735+0000","last_became_peered":"2023-10-16T20:05:01.477735+0000","last_unstale":"2023-10-16T20:05:02.444695+0000","last_undegraded":"2023-10-16T20:05:02.444695+0000","last_fullsized":"2023-10-16T20:05:02.444695+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:14:33.198627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471054+0000","last_change":"2023-10-16T20:04:57.427768+0000","last_active":"2023-10-16T20:05:01.471054+0000","last_peered":"2023-10-16T20:05:01.471054+0000","last_clean":"2023-10-16T20:05:01.471054+0000","last_became_active":"2023-10-16T20:04:57.422018+0000","last_became_peered":"2023-10-16T20:04:57.422018+0000","last_unstale":"2023-10-16T20:05:01.471054+0000","last_undegraded":"2023-10-16T20:05:01.471054+0000","last_fullsized":"2023-10-16T20:05:01.471054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:49:52.409593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.448426+0000","last_change":"2023-10-16T20:04:56.402767+0000","last_active":"2023-10-16T20:05:01.448426+0000","last_peered":"2023-10-16T20:05:01.448426+0000","last_clean":"2023-10-16T20:05:01.448426+0000","last_became_active":"2023-10-16T20:04:56.402122+0000","last_became_peered":"2023-10-16T20:04:56.402122+0000","last_unstale":"2023-10-16T20:05:01.448426+0000","last_undegraded":"2023-10-16T20:05:01.448426+0000","last_fullsized":"2023-10-16T20:05:01.448426+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:54:36.775882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.445579+0000","last_change":"2023-10-16T20:05:01.478710+0000","last_active":"2023-10-16T20:05:02.445579+0000","last_peered":"2023-10-16T20:05:02.445579+0000","last_clean":"2023-10-16T20:05:02.445579+0000","last_became_active":"2023-10-16T20:05:01.478350+0000","last_became_peered":"2023-10-16T20:05:01.478350+0000","last_unstale":"2023-10-16T20:05:02.445579+0000","last_undegraded":"2023-10-16T20:05:02.445579+0000","last_fullsized":"2023-10-16T20:05:02.445579+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:57:32.014259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.459302+0000","last_change":"2023-10-16T20:04:57.420638+0000","last_active":"2023-10-16T20:05:01.459302+0000","last_peered":"2023-10-16T20:05:01.459302+0000","last_clean":"2023-10-16T20:05:01.459302+0000","last_became_active":"2023-10-16T20:04:57.420348+0000","last_became_peered":"2023-10-16T20:04:57.420348+0000","last_unstale":"2023-10-16T20:05:01.459302+0000","last_undegraded":"2023-10-16T20:05:01.459302+0000","last_fullsized":"2023-10-16T20:05:01.459302+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:14:18.231180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757558+0000","last_change":"2023-10-16T20:04:56.403488+0000","last_active":"2023-10-16T20:05:01.757558+0000","last_peered":"2023-10-16T20:05:01.757558+0000","last_clean":"2023-10-16T20:05:01.757558+0000","last_became_active":"2023-10-16T20:04:56.403156+0000","last_became_peered":"2023-10-16T20:04:56.403156+0000","last_unstale":"2023-10-16T20:05:01.757558+0000","last_undegraded":"2023-10-16T20:05:01.757558+0000","last_fullsized":"2023-10-16T20:05:01.757558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:33:35.395756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.688883+0000","last_change":"2023-10-16T20:05:01.765108+0000","last_active":"2023-10-16T20:05:02.688883+0000","last_peered":"2023-10-16T20:05:02.688883+0000","last_clean":"2023-10-16T20:05:02.688883+0000","last_became_active":"2023-10-16T20:05:01.764436+0000","last_became_peered":"2023-10-16T20:05:01.764436+0000","last_unstale":"2023-10-16T20:05:02.688883+0000","last_undegraded":"2023-10-16T20:05:02.688883+0000","last_fullsized":"2023-10-16T20:05:02.688883+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:39:29.554873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442360+0000","last_change":"2023-10-16T20:04:57.429060+0000","last_active":"2023-10-16T20:05:01.442360+0000","last_peered":"2023-10-16T20:05:01.442360+0000","last_clean":"2023-10-16T20:05:01.442360+0000","last_became_active":"2023-10-16T20:04:57.428756+0000","last_became_peered":"2023-10-16T20:04:57.428756+0000","last_unstale":"2023-10-16T20:05:01.442360+0000","last_undegraded":"2023-10-16T20:05:01.442360+0000","last_fullsized":"2023-10-16T20:05:01.442360+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:21:09.387890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.437753+0000","last_change":"2023-10-16T20:04:56.392274+0000","last_active":"2023-10-16T20:05:01.437753+0000","last_peered":"2023-10-16T20:05:01.437753+0000","last_clean":"2023-10-16T20:05:01.437753+0000","last_became_active":"2023-10-16T20:04:56.391958+0000","last_became_peered":"2023-10-16T20:04:56.391958+0000","last_unstale":"2023-10-16T20:05:01.437753+0000","last_undegraded":"2023-10-16T20:05:01.437753+0000","last_fullsized":"2023-10-16T20:05:01.437753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:58:19.945263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":169,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757620+0000","last_change":"2023-10-16T20:04:46.335822+0000","last_active":"2023-10-16T20:05:01.757620+0000","last_peered":"2023-10-16T20:05:01.757620+0000","last_clean":"2023-10-16T20:05:01.757620+0000","last_became_active":"2023-10-16T20:04:46.334617+0000","last_became_peered":"2023-10-16T20:04:46.334617+0000","last_unstale":"2023-10-16T20:05:01.757620+0000","last_undegraded":"2023-10-16T20:05:01.757620+0000","last_fullsized":"2023-10-16T20:05:01.757620+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:45.306240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:45.306240+0000","last_clean_scrub_stamp":"2023-10-16T20:04:45.306240+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:42:11.153721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.682739+0000","last_change":"2023-10-16T20:05:01.760533+0000","last_active":"2023-10-16T20:05:02.682739+0000","last_peered":"2023-10-16T20:05:02.682739+0000","last_clean":"2023-10-16T20:05:02.682739+0000","last_became_active":"2023-10-16T20:05:01.760091+0000","last_became_peered":"2023-10-16T20:05:01.760091+0000","last_unstale":"2023-10-16T20:05:02.682739+0000","last_undegraded":"2023-10-16T20:05:02.682739+0000","last_fullsized":"2023-10-16T20:05:02.682739+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:22:16.748229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757962+0000","last_change":"2023-10-16T20:04:57.435770+0000","last_active":"2023-10-16T20:05:01.757962+0000","last_peered":"2023-10-16T20:05:01.757962+0000","last_clean":"2023-10-16T20:05:01.757962+0000","last_became_active":"2023-10-16T20:04:57.434914+0000","last_became_peered":"2023-10-16T20:04:57.434914+0000","last_unstale":"2023-10-16T20:05:01.757962+0000","last_undegraded":"2023-10-16T20:05:01.757962+0000","last_fullsized":"2023-10-16T20:05:01.757962+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:42:35.216191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.449802+0000","last_change":"2023-10-16T20:04:56.414794+0000","last_active":"2023-10-16T20:05:01.449802+0000","last_peered":"2023-10-16T20:05:01.449802+0000","last_clean":"2023-10-16T20:05:01.449802+0000","last_became_active":"2023-10-16T20:04:56.414400+0000","last_became_peered":"2023-10-16T20:04:56.414400+0000","last_unstale":"2023-10-16T20:05:01.449802+0000","last_undegraded":"2023-10-16T20:05:01.449802+0000","last_fullsized":"2023-10-16T20:05:01.449802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:19:48.110438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.682395+0000","last_change":"2023-10-16T20:05:01.760979+0000","last_active":"2023-10-16T20:05:02.682395+0000","last_peered":"2023-10-16T20:05:02.682395+0000","last_clean":"2023-10-16T20:05:02.682395+0000","last_became_active":"2023-10-16T20:05:01.760141+0000","last_became_peered":"2023-10-16T20:05:01.760141+0000","last_unstale":"2023-10-16T20:05:02.682395+0000","last_undegraded":"2023-10-16T20:05:02.682395+0000","last_fullsized":"2023-10-16T20:05:02.682395+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:21:13.443137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438336+0000","last_change":"2023-10-16T20:04:57.429698+0000","last_active":"2023-10-16T20:05:01.438336+0000","last_peered":"2023-10-16T20:05:01.438336+0000","last_clean":"2023-10-16T20:05:01.438336+0000","last_became_active":"2023-10-16T20:04:57.428050+0000","last_became_peered":"2023-10-16T20:04:57.428050+0000","last_unstale":"2023-10-16T20:05:01.438336+0000","last_undegraded":"2023-10-16T20:05:01.438336+0000","last_fullsized":"2023-10-16T20:05:01.438336+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:42:35.216191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471736+0000","last_change":"2023-10-16T20:04:56.409587+0000","last_active":"2023-10-16T20:05:01.471736+0000","last_peered":"2023-10-16T20:05:01.471736+0000","last_clean":"2023-10-16T20:05:01.471736+0000","last_became_active":"2023-10-16T20:04:56.408150+0000","last_became_peered":"2023-10-16T20:04:56.408150+0000","last_unstale":"2023-10-16T20:05:01.471736+0000","last_undegraded":"2023-10-16T20:05:01.471736+0000","last_fullsized":"2023-10-16T20:05:01.471736+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:13:40.552947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.688655+0000","last_change":"2023-10-16T20:05:01.476280+0000","last_active":"2023-10-16T20:05:02.688655+0000","last_peered":"2023-10-16T20:05:02.688655+0000","last_clean":"2023-10-16T20:05:02.688655+0000","last_became_active":"2023-10-16T20:05:01.475885+0000","last_became_peered":"2023-10-16T20:05:01.475885+0000","last_unstale":"2023-10-16T20:05:02.688655+0000","last_undegraded":"2023-10-16T20:05:02.688655+0000","last_fullsized":"2023-10-16T20:05:02.688655+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:01:16.289925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.758343+0000","last_change":"2023-10-16T20:04:57.442292+0000","last_active":"2023-10-16T20:05:01.758343+0000","last_peered":"2023-10-16T20:05:01.758343+0000","last_clean":"2023-10-16T20:05:01.758343+0000","last_became_active":"2023-10-16T20:04:57.439075+0000","last_became_peered":"2023-10-16T20:04:57.439075+0000","last_unstale":"2023-10-16T20:05:01.758343+0000","last_undegraded":"2023-10-16T20:05:01.758343+0000","last_fullsized":"2023-10-16T20:05:01.758343+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:25:46.207669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.448189+0000","last_change":"2023-10-16T20:04:56.398007+0000","last_active":"2023-10-16T20:05:01.448189+0000","last_peered":"2023-10-16T20:05:01.448189+0000","last_clean":"2023-10-16T20:05:01.448189+0000","last_became_active":"2023-10-16T20:04:56.397108+0000","last_became_peered":"2023-10-16T20:04:56.397108+0000","last_unstale":"2023-10-16T20:05:01.448189+0000","last_undegraded":"2023-10-16T20:05:01.448189+0000","last_fullsized":"2023-10-16T20:05:01.448189+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:24:16.965233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.684215+0000","last_change":"2023-10-16T20:05:01.763390+0000","last_active":"2023-10-16T20:05:02.684215+0000","last_peered":"2023-10-16T20:05:02.684215+0000","last_clean":"2023-10-16T20:05:02.684215+0000","last_became_active":"2023-10-16T20:05:01.760077+0000","last_became_peered":"2023-10-16T20:05:01.760077+0000","last_unstale":"2023-10-16T20:05:02.684215+0000","last_undegraded":"2023-10-16T20:05:02.684215+0000","last_fullsized":"2023-10-16T20:05:02.684215+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:10:33.807776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.758160+0000","last_change":"2023-10-16T20:04:57.437968+0000","last_active":"2023-10-16T20:05:01.758160+0000","last_peered":"2023-10-16T20:05:01.758160+0000","last_clean":"2023-10-16T20:05:01.758160+0000","last_became_active":"2023-10-16T20:04:57.437674+0000","last_became_peered":"2023-10-16T20:04:57.437674+0000","last_unstale":"2023-10-16T20:05:01.758160+0000","last_undegraded":"2023-10-16T20:05:01.758160+0000","last_fullsized":"2023-10-16T20:05:01.758160+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:01:38.242317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.447078+0000","last_change":"2023-10-16T20:04:56.408227+0000","last_active":"2023-10-16T20:05:01.447078+0000","last_peered":"2023-10-16T20:05:01.447078+0000","last_clean":"2023-10-16T20:05:01.447078+0000","last_became_active":"2023-10-16T20:04:56.407279+0000","last_became_peered":"2023-10-16T20:04:56.407279+0000","last_unstale":"2023-10-16T20:05:01.447078+0000","last_undegraded":"2023-10-16T20:05:01.447078+0000","last_fullsized":"2023-10-16T20:05:01.447078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:43:11.864176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.682415+0000","last_change":"2023-10-16T20:05:01.477611+0000","last_active":"2023-10-16T20:05:02.682415+0000","last_peered":"2023-10-16T20:05:02.682415+0000","last_clean":"2023-10-16T20:05:02.682415+0000","last_became_active":"2023-10-16T20:05:01.477172+0000","last_became_peered":"2023-10-16T20:05:01.477172+0000","last_unstale":"2023-10-16T20:05:02.682415+0000","last_undegraded":"2023-10-16T20:05:02.682415+0000","last_fullsized":"2023-10-16T20:05:02.682415+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:23:05.196792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757700+0000","last_change":"2023-10-16T20:04:57.438300+0000","last_active":"2023-10-16T20:05:01.757700+0000","last_peered":"2023-10-16T20:05:01.757700+0000","last_clean":"2023-10-16T20:05:01.757700+0000","last_became_active":"2023-10-16T20:04:57.437951+0000","last_became_peered":"2023-10-16T20:04:57.437951+0000","last_unstale":"2023-10-16T20:05:01.757700+0000","last_undegraded":"2023-10-16T20:05:01.757700+0000","last_fullsized":"2023-10-16T20:05:01.757700+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:31:34.884161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.448376+0000","last_change":"2023-10-16T20:04:56.401341+0000","last_active":"2023-10-16T20:05:01.448376+0000","last_peered":"2023-10-16T20:05:01.448376+0000","last_clean":"2023-10-16T20:05:01.448376+0000","last_became_active":"2023-10-16T20:04:56.401058+0000","last_became_peered":"2023-10-16T20:04:56.401058+0000","last_unstale":"2023-10-16T20:05:01.448376+0000","last_undegraded":"2023-10-16T20:05:01.448376+0000","last_fullsized":"2023-10-16T20:05:01.448376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:41:34.379170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.688808+0000","last_change":"2023-10-16T20:05:01.463304+0000","last_active":"2023-10-16T20:05:02.688808+0000","last_peered":"2023-10-16T20:05:02.688808+0000","last_clean":"2023-10-16T20:05:02.688808+0000","last_became_active":"2023-10-16T20:05:01.462044+0000","last_became_peered":"2023-10-16T20:05:01.462044+0000","last_unstale":"2023-10-16T20:05:02.688808+0000","last_undegraded":"2023-10-16T20:05:02.688808+0000","last_fullsized":"2023-10-16T20:05:02.688808+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:41:12.320369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.440492+0000","last_change":"2023-10-16T20:04:57.409823+0000","last_active":"2023-10-16T20:05:01.440492+0000","last_peered":"2023-10-16T20:05:01.440492+0000","last_clean":"2023-10-16T20:05:01.440492+0000","last_became_active":"2023-10-16T20:04:57.409457+0000","last_became_peered":"2023-10-16T20:04:57.409457+0000","last_unstale":"2023-10-16T20:05:01.440492+0000","last_undegraded":"2023-10-16T20:05:01.440492+0000","last_fullsized":"2023-10-16T20:05:01.440492+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:53:20.922170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.440306+0000","last_change":"2023-10-16T20:04:56.404854+0000","last_active":"2023-10-16T20:05:01.440306+0000","last_peered":"2023-10-16T20:05:01.440306+0000","last_clean":"2023-10-16T20:05:01.440306+0000","last_became_active":"2023-10-16T20:04:56.404457+0000","last_became_peered":"2023-10-16T20:04:56.404457+0000","last_unstale":"2023-10-16T20:05:01.440306+0000","last_undegraded":"2023-10-16T20:05:01.440306+0000","last_fullsized":"2023-10-16T20:05:01.440306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:41:00.246498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.686341+0000","last_change":"2023-10-16T20:05:01.769521+0000","last_active":"2023-10-16T20:05:02.686341+0000","last_peered":"2023-10-16T20:05:02.686341+0000","last_clean":"2023-10-16T20:05:02.686341+0000","last_became_active":"2023-10-16T20:05:01.768728+0000","last_became_peered":"2023-10-16T20:05:01.768728+0000","last_unstale":"2023-10-16T20:05:02.686341+0000","last_undegraded":"2023-10-16T20:05:02.686341+0000","last_fullsized":"2023-10-16T20:05:02.686341+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:01:55.141797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757851+0000","last_change":"2023-10-16T20:04:57.438646+0000","last_active":"2023-10-16T20:05:01.757851+0000","last_peered":"2023-10-16T20:05:01.757851+0000","last_clean":"2023-10-16T20:05:01.757851+0000","last_became_active":"2023-10-16T20:04:57.438375+0000","last_became_peered":"2023-10-16T20:04:57.438375+0000","last_unstale":"2023-10-16T20:05:01.757851+0000","last_undegraded":"2023-10-16T20:05:01.757851+0000","last_fullsized":"2023-10-16T20:05:01.757851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:14:29.143380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.437845+0000","last_change":"2023-10-16T20:04:56.407281+0000","last_active":"2023-10-16T20:05:01.437845+0000","last_peered":"2023-10-16T20:05:01.437845+0000","last_clean":"2023-10-16T20:05:01.437845+0000","last_became_active":"2023-10-16T20:04:56.406824+0000","last_became_peered":"2023-10-16T20:04:56.406824+0000","last_unstale":"2023-10-16T20:05:01.437845+0000","last_undegraded":"2023-10-16T20:05:01.437845+0000","last_fullsized":"2023-10-16T20:05:01.437845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:23:00.134558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.686002+0000","last_change":"2023-10-16T20:05:01.764372+0000","last_active":"2023-10-16T20:05:02.686002+0000","last_peered":"2023-10-16T20:05:02.686002+0000","last_clean":"2023-10-16T20:05:02.686002+0000","last_became_active":"2023-10-16T20:05:01.763918+0000","last_became_peered":"2023-10-16T20:05:01.763918+0000","last_unstale":"2023-10-16T20:05:02.686002+0000","last_undegraded":"2023-10-16T20:05:02.686002+0000","last_fullsized":"2023-10-16T20:05:02.686002+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:10:33.807776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.463201+0000","last_change":"2023-10-16T20:04:57.425920+0000","last_active":"2023-10-16T20:05:01.463201+0000","last_peered":"2023-10-16T20:05:01.463201+0000","last_clean":"2023-10-16T20:05:01.463201+0000","last_became_active":"2023-10-16T20:04:57.425614+0000","last_became_peered":"2023-10-16T20:04:57.425614+0000","last_unstale":"2023-10-16T20:05:01.463201+0000","last_undegraded":"2023-10-16T20:05:01.463201+0000","last_fullsized":"2023-10-16T20:05:01.463201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:28:14.992492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.464594+0000","last_change":"2023-10-16T20:04:56.407990+0000","last_active":"2023-10-16T20:05:01.464594+0000","last_peered":"2023-10-16T20:05:01.464594+0000","last_clean":"2023-10-16T20:05:01.464594+0000","last_became_active":"2023-10-16T20:04:56.407682+0000","last_became_peered":"2023-10-16T20:04:56.407682+0000","last_unstale":"2023-10-16T20:05:01.464594+0000","last_undegraded":"2023-10-16T20:05:01.464594+0000","last_fullsized":"2023-10-16T20:05:01.464594+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:25:27.337575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.685081+0000","last_change":"2023-10-16T20:05:01.761327+0000","last_active":"2023-10-16T20:05:02.685081+0000","last_peered":"2023-10-16T20:05:02.685081+0000","last_clean":"2023-10-16T20:05:02.685081+0000","last_became_active":"2023-10-16T20:05:01.760040+0000","last_became_peered":"2023-10-16T20:05:01.760040+0000","last_unstale":"2023-10-16T20:05:02.685081+0000","last_undegraded":"2023-10-16T20:05:02.685081+0000","last_fullsized":"2023-10-16T20:05:02.685081+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:33:23.371729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757841+0000","last_change":"2023-10-16T20:04:57.437602+0000","last_active":"2023-10-16T20:05:01.757841+0000","last_peered":"2023-10-16T20:05:01.757841+0000","last_clean":"2023-10-16T20:05:01.757841+0000","last_became_active":"2023-10-16T20:04:57.436601+0000","last_became_peered":"2023-10-16T20:04:57.436601+0000","last_unstale":"2023-10-16T20:05:01.757841+0000","last_undegraded":"2023-10-16T20:05:01.757841+0000","last_fullsized":"2023-10-16T20:05:01.757841+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:28:00.096002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'9","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:08.020486+0000","last_change":"2023-10-16T20:04:56.398743+0000","last_active":"2023-10-16T20:05:08.020486+0000","last_peered":"2023-10-16T20:05:08.020486+0000","last_clean":"2023-10-16T20:05:08.020486+0000","last_became_active":"2023-10-16T20:04:56.398410+0000","last_became_peered":"2023-10-16T20:04:56.398410+0000","last_unstale":"2023-10-16T20:05:08.020486+0000","last_undegraded":"2023-10-16T20:05:08.020486+0000","last_fullsized":"2023-10-16T20:05:08.020486+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:21:13.046934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.444394+0000","last_change":"2023-10-16T20:05:01.472414+0000","last_active":"2023-10-16T20:05:02.444394+0000","last_peered":"2023-10-16T20:05:02.444394+0000","last_clean":"2023-10-16T20:05:02.444394+0000","last_became_active":"2023-10-16T20:05:01.471943+0000","last_became_peered":"2023-10-16T20:05:01.471943+0000","last_unstale":"2023-10-16T20:05:02.444394+0000","last_undegraded":"2023-10-16T20:05:02.444394+0000","last_fullsized":"2023-10-16T20:05:02.444394+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:42:39.271438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.447164+0000","last_change":"2023-10-16T20:04:57.413204+0000","last_active":"2023-10-16T20:05:01.447164+0000","last_peered":"2023-10-16T20:05:01.447164+0000","last_clean":"2023-10-16T20:05:01.447164+0000","last_became_active":"2023-10-16T20:04:57.412836+0000","last_became_peered":"2023-10-16T20:04:57.412836+0000","last_unstale":"2023-10-16T20:05:01.447164+0000","last_undegraded":"2023-10-16T20:05:01.447164+0000","last_fullsized":"2023-10-16T20:05:01.447164+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:23:06.051449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471656+0000","last_change":"2023-10-16T20:04:56.414671+0000","last_active":"2023-10-16T20:05:01.471656+0000","last_peered":"2023-10-16T20:05:01.471656+0000","last_clean":"2023-10-16T20:05:01.471656+0000","last_became_active":"2023-10-16T20:04:56.414339+0000","last_became_peered":"2023-10-16T20:04:56.414339+0000","last_unstale":"2023-10-16T20:05:01.471656+0000","last_undegraded":"2023-10-16T20:05:01.471656+0000","last_fullsized":"2023-10-16T20:05:01.471656+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:21:13.046934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.683018+0000","last_change":"2023-10-16T20:05:01.482674+0000","last_active":"2023-10-16T20:05:02.683018+0000","last_peered":"2023-10-16T20:05:02.683018+0000","last_clean":"2023-10-16T20:05:02.683018+0000","last_became_active":"2023-10-16T20:05:01.482048+0000","last_became_peered":"2023-10-16T20:05:01.482048+0000","last_unstale":"2023-10-16T20:05:02.683018+0000","last_undegraded":"2023-10-16T20:05:02.683018+0000","last_fullsized":"2023-10-16T20:05:02.683018+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:21:13.443137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.465807+0000","last_change":"2023-10-16T20:04:57.430429+0000","last_active":"2023-10-16T20:05:01.465807+0000","last_peered":"2023-10-16T20:05:01.465807+0000","last_clean":"2023-10-16T20:05:01.465807+0000","last_became_active":"2023-10-16T20:04:57.429872+0000","last_became_peered":"2023-10-16T20:04:57.429872+0000","last_unstale":"2023-10-16T20:05:01.465807+0000","last_undegraded":"2023-10-16T20:05:01.465807+0000","last_fullsized":"2023-10-16T20:05:01.465807+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:47:33.537999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438017+0000","last_change":"2023-10-16T20:04:56.398061+0000","last_active":"2023-10-16T20:05:01.438017+0000","last_peered":"2023-10-16T20:05:01.438017+0000","last_clean":"2023-10-16T20:05:01.438017+0000","last_became_active":"2023-10-16T20:04:56.397712+0000","last_became_peered":"2023-10-16T20:04:56.397712+0000","last_unstale":"2023-10-16T20:05:01.438017+0000","last_undegraded":"2023-10-16T20:05:01.438017+0000","last_fullsized":"2023-10-16T20:05:01.438017+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:49:51.402606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.685658+0000","last_change":"2023-10-16T20:05:01.476635+0000","last_active":"2023-10-16T20:05:02.685658+0000","last_peered":"2023-10-16T20:05:02.685658+0000","last_clean":"2023-10-16T20:05:02.685658+0000","last_became_active":"2023-10-16T20:05:01.476053+0000","last_became_peered":"2023-10-16T20:05:01.476053+0000","last_unstale":"2023-10-16T20:05:02.685658+0000","last_undegraded":"2023-10-16T20:05:02.685658+0000","last_fullsized":"2023-10-16T20:05:02.685658+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:25:50.262916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.437931+0000","last_change":"2023-10-16T20:04:57.421476+0000","last_active":"2023-10-16T20:05:01.437931+0000","last_peered":"2023-10-16T20:05:01.437931+0000","last_clean":"2023-10-16T20:05:01.437931+0000","last_became_active":"2023-10-16T20:04:57.421206+0000","last_became_peered":"2023-10-16T20:04:57.421206+0000","last_unstale":"2023-10-16T20:05:01.437931+0000","last_undegraded":"2023-10-16T20:05:01.437931+0000","last_fullsized":"2023-10-16T20:05:01.437931+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:11:56.834052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.439019+0000","last_change":"2023-10-16T20:04:56.411232+0000","last_active":"2023-10-16T20:05:01.439019+0000","last_peered":"2023-10-16T20:05:01.439019+0000","last_clean":"2023-10-16T20:05:01.439019+0000","last_became_active":"2023-10-16T20:04:56.410793+0000","last_became_peered":"2023-10-16T20:04:56.410793+0000","last_unstale":"2023-10-16T20:05:01.439019+0000","last_undegraded":"2023-10-16T20:05:01.439019+0000","last_fullsized":"2023-10-16T20:05:01.439019+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:19:48.110438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.685647+0000","last_change":"2023-10-16T20:05:01.764968+0000","last_active":"2023-10-16T20:05:02.685647+0000","last_peered":"2023-10-16T20:05:02.685647+0000","last_clean":"2023-10-16T20:05:02.685647+0000","last_became_active":"2023-10-16T20:05:01.763830+0000","last_became_peered":"2023-10-16T20:05:01.763830+0000","last_unstale":"2023-10-16T20:05:02.685647+0000","last_undegraded":"2023-10-16T20:05:02.685647+0000","last_fullsized":"2023-10-16T20:05:02.685647+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:39:58.794103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.688809+0000","last_change":"2023-10-16T20:05:01.764846+0000","last_active":"2023-10-16T20:05:02.688809+0000","last_peered":"2023-10-16T20:05:02.688809+0000","last_clean":"2023-10-16T20:05:02.688809+0000","last_became_active":"2023-10-16T20:05:01.763756+0000","last_became_peered":"2023-10-16T20:05:01.763756+0000","last_unstale":"2023-10-16T20:05:02.688809+0000","last_undegraded":"2023-10-16T20:05:02.688809+0000","last_fullsized":"2023-10-16T20:05:02.688809+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:10:31.962156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471094+0000","last_change":"2023-10-16T20:04:57.428882+0000","last_active":"2023-10-16T20:05:01.471094+0000","last_peered":"2023-10-16T20:05:01.471094+0000","last_clean":"2023-10-16T20:05:01.471094+0000","last_became_active":"2023-10-16T20:04:57.428591+0000","last_became_peered":"2023-10-16T20:04:57.428591+0000","last_unstale":"2023-10-16T20:05:01.471094+0000","last_undegraded":"2023-10-16T20:05:01.471094+0000","last_fullsized":"2023-10-16T20:05:01.471094+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:33:36.402743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438343+0000","last_change":"2023-10-16T20:04:56.396769+0000","last_active":"2023-10-16T20:05:01.438343+0000","last_peered":"2023-10-16T20:05:01.438343+0000","last_clean":"2023-10-16T20:05:01.438343+0000","last_became_active":"2023-10-16T20:04:56.396273+0000","last_became_peered":"2023-10-16T20:04:56.396273+0000","last_unstale":"2023-10-16T20:05:01.438343+0000","last_undegraded":"2023-10-16T20:05:01.438343+0000","last_fullsized":"2023-10-16T20:05:01.438343+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:04:32.948232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.683891+0000","last_change":"2023-10-16T20:05:01.475065+0000","last_active":"2023-10-16T20:05:02.683891+0000","last_peered":"2023-10-16T20:05:02.683891+0000","last_clean":"2023-10-16T20:05:02.683891+0000","last_became_active":"2023-10-16T20:05:01.474571+0000","last_became_peered":"2023-10-16T20:05:01.474571+0000","last_unstale":"2023-10-16T20:05:02.683891+0000","last_undegraded":"2023-10-16T20:05:02.683891+0000","last_fullsized":"2023-10-16T20:05:02.683891+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:31:38.939408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442005+0000","last_change":"2023-10-16T20:04:57.423943+0000","last_active":"2023-10-16T20:05:01.442005+0000","last_peered":"2023-10-16T20:05:01.442005+0000","last_clean":"2023-10-16T20:05:01.442005+0000","last_became_active":"2023-10-16T20:04:57.423325+0000","last_became_peered":"2023-10-16T20:04:57.423325+0000","last_unstale":"2023-10-16T20:05:01.442005+0000","last_undegraded":"2023-10-16T20:05:01.442005+0000","last_fullsized":"2023-10-16T20:05:01.442005+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:56:28.993951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442942+0000","last_change":"2023-10-16T20:04:56.397549+0000","last_active":"2023-10-16T20:05:01.442942+0000","last_peered":"2023-10-16T20:05:01.442942+0000","last_clean":"2023-10-16T20:05:01.442942+0000","last_became_active":"2023-10-16T20:04:56.396648+0000","last_became_peered":"2023-10-16T20:04:56.396648+0000","last_unstale":"2023-10-16T20:05:01.442942+0000","last_undegraded":"2023-10-16T20:05:01.442942+0000","last_fullsized":"2023-10-16T20:05:01.442942+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:58:19.945263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438803+0000","last_change":"2023-10-16T20:04:56.407155+0000","last_active":"2023-10-16T20:05:01.438803+0000","last_peered":"2023-10-16T20:05:01.438803+0000","last_clean":"2023-10-16T20:05:01.438803+0000","last_became_active":"2023-10-16T20:04:56.406828+0000","last_became_peered":"2023-10-16T20:04:56.406828+0000","last_unstale":"2023-10-16T20:05:01.438803+0000","last_undegraded":"2023-10-16T20:05:01.438803+0000","last_fullsized":"2023-10-16T20:05:01.438803+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:23:05.044462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.464536+0000","last_change":"2023-10-16T20:04:57.416841+0000","last_active":"2023-10-16T20:05:01.464536+0000","last_peered":"2023-10-16T20:05:01.464536+0000","last_clean":"2023-10-16T20:05:01.464536+0000","last_became_active":"2023-10-16T20:04:57.416338+0000","last_became_peered":"2023-10-16T20:04:57.416338+0000","last_unstale":"2023-10-16T20:05:01.464536+0000","last_undegraded":"2023-10-16T20:05:01.464536+0000","last_fullsized":"2023-10-16T20:05:01.464536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:01.396735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.445821+0000","last_change":"2023-10-16T20:05:01.481039+0000","last_active":"2023-10-16T20:05:02.445821+0000","last_peered":"2023-10-16T20:05:02.445821+0000","last_clean":"2023-10-16T20:05:02.445821+0000","last_became_active":"2023-10-16T20:05:01.480626+0000","last_became_peered":"2023-10-16T20:05:01.480626+0000","last_unstale":"2023-10-16T20:05:02.445821+0000","last_undegraded":"2023-10-16T20:05:02.445821+0000","last_fullsized":"2023-10-16T20:05:02.445821+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:21:18.109168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.441938+0000","last_change":"2023-10-16T20:04:56.401545+0000","last_active":"2023-10-16T20:05:01.441938+0000","last_peered":"2023-10-16T20:05:01.441938+0000","last_clean":"2023-10-16T20:05:01.441938+0000","last_became_active":"2023-10-16T20:04:56.401276+0000","last_became_peered":"2023-10-16T20:04:56.401276+0000","last_unstale":"2023-10-16T20:05:01.441938+0000","last_undegraded":"2023-10-16T20:05:01.441938+0000","last_fullsized":"2023-10-16T20:05:01.441938+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:04:32.948232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.447134+0000","last_change":"2023-10-16T20:04:57.425600+0000","last_active":"2023-10-16T20:05:01.447134+0000","last_peered":"2023-10-16T20:05:01.447134+0000","last_clean":"2023-10-16T20:05:01.447134+0000","last_became_active":"2023-10-16T20:04:57.425213+0000","last_became_peered":"2023-10-16T20:04:57.425213+0000","last_unstale":"2023-10-16T20:05:01.447134+0000","last_undegraded":"2023-10-16T20:05:01.447134+0000","last_fullsized":"2023-10-16T20:05:01.447134+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:33:14.623281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.686142+0000","last_change":"2023-10-16T20:05:01.479226+0000","last_active":"2023-10-16T20:05:02.686142+0000","last_peered":"2023-10-16T20:05:02.686142+0000","last_clean":"2023-10-16T20:05:02.686142+0000","last_became_active":"2023-10-16T20:05:01.478808+0000","last_became_peered":"2023-10-16T20:05:01.478808+0000","last_unstale":"2023-10-16T20:05:02.686142+0000","last_undegraded":"2023-10-16T20:05:02.686142+0000","last_fullsized":"2023-10-16T20:05:02.686142+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:33:23.371729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.440406+0000","last_change":"2023-10-16T20:04:57.409772+0000","last_active":"2023-10-16T20:05:01.440406+0000","last_peered":"2023-10-16T20:05:01.440406+0000","last_clean":"2023-10-16T20:05:01.440406+0000","last_became_active":"2023-10-16T20:04:57.409449+0000","last_became_peered":"2023-10-16T20:04:57.409449+0000","last_unstale":"2023-10-16T20:05:01.440406+0000","last_undegraded":"2023-10-16T20:05:01.440406+0000","last_fullsized":"2023-10-16T20:05:01.440406+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:13:41.559934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.448190+0000","last_change":"2023-10-16T20:04:57.418433+0000","last_active":"2023-10-16T20:05:01.448190+0000","last_peered":"2023-10-16T20:05:01.448190+0000","last_clean":"2023-10-16T20:05:01.448190+0000","last_became_active":"2023-10-16T20:04:57.418140+0000","last_became_peered":"2023-10-16T20:04:57.418140+0000","last_unstale":"2023-10-16T20:05:01.448190+0000","last_undegraded":"2023-10-16T20:05:01.448190+0000","last_fullsized":"2023-10-16T20:05:01.448190+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:41:01.253485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.437811+0000","last_change":"2023-10-16T20:04:57.430273+0000","last_active":"2023-10-16T20:05:01.437811+0000","last_peered":"2023-10-16T20:05:01.437811+0000","last_clean":"2023-10-16T20:05:01.437811+0000","last_became_active":"2023-10-16T20:04:57.427616+0000","last_became_peered":"2023-10-16T20:04:57.427616+0000","last_unstale":"2023-10-16T20:05:01.437811+0000","last_undegraded":"2023-10-16T20:05:01.437811+0000","last_fullsized":"2023-10-16T20:05:01.437811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:22:12.692981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.464484+0000","last_change":"2023-10-16T20:04:57.434154+0000","last_active":"2023-10-16T20:05:01.464484+0000","last_peered":"2023-10-16T20:05:01.464484+0000","last_clean":"2023-10-16T20:05:01.464484+0000","last_became_active":"2023-10-16T20:04:57.433876+0000","last_became_peered":"2023-10-16T20:04:57.433876+0000","last_unstale":"2023-10-16T20:05:01.464484+0000","last_undegraded":"2023-10-16T20:05:01.464484+0000","last_fullsized":"2023-10-16T20:05:01.464484+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:45:41.399175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.443384+0000","last_change":"2023-10-16T20:04:57.437575+0000","last_active":"2023-10-16T20:05:01.443384+0000","last_peered":"2023-10-16T20:05:01.443384+0000","last_clean":"2023-10-16T20:05:01.443384+0000","last_became_active":"2023-10-16T20:04:57.437261+0000","last_became_peered":"2023-10-16T20:04:57.437261+0000","last_unstale":"2023-10-16T20:05:01.443384+0000","last_undegraded":"2023-10-16T20:05:01.443384+0000","last_fullsized":"2023-10-16T20:05:01.443384+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:58:33.862646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471548+0000","last_change":"2023-10-16T20:04:57.434048+0000","last_active":"2023-10-16T20:05:01.471548+0000","last_peered":"2023-10-16T20:05:01.471548+0000","last_clean":"2023-10-16T20:05:01.471548+0000","last_became_active":"2023-10-16T20:04:57.433771+0000","last_became_peered":"2023-10-16T20:04:57.433771+0000","last_unstale":"2023-10-16T20:05:01.471548+0000","last_undegraded":"2023-10-16T20:05:01.471548+0000","last_fullsized":"2023-10-16T20:05:01.471548+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:58:33.862646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.440121+0000","last_change":"2023-10-16T20:04:57.439787+0000","last_active":"2023-10-16T20:05:01.440121+0000","last_peered":"2023-10-16T20:05:01.440121+0000","last_clean":"2023-10-16T20:05:01.440121+0000","last_became_active":"2023-10-16T20:04:57.439468+0000","last_became_peered":"2023-10-16T20:04:57.439468+0000","last_unstale":"2023-10-16T20:05:01.440121+0000","last_undegraded":"2023-10-16T20:05:01.440121+0000","last_fullsized":"2023-10-16T20:05:01.440121+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:49:52.409593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442167+0000","last_change":"2023-10-16T20:04:57.421442+0000","last_active":"2023-10-16T20:05:01.442167+0000","last_peered":"2023-10-16T20:05:01.442167+0000","last_clean":"2023-10-16T20:05:01.442167+0000","last_became_active":"2023-10-16T20:04:57.421123+0000","last_became_peered":"2023-10-16T20:04:57.421123+0000","last_unstale":"2023-10-16T20:05:01.442167+0000","last_undegraded":"2023-10-16T20:05:01.442167+0000","last_fullsized":"2023-10-16T20:05:01.442167+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:17:24.205223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438259+0000","last_change":"2023-10-16T20:04:57.412843+0000","last_active":"2023-10-16T20:05:01.438259+0000","last_peered":"2023-10-16T20:05:01.438259+0000","last_clean":"2023-10-16T20:05:01.438259+0000","last_became_active":"2023-10-16T20:04:57.412559+0000","last_became_peered":"2023-10-16T20:04:57.412559+0000","last_unstale":"2023-10-16T20:05:01.438259+0000","last_undegraded":"2023-10-16T20:05:01.438259+0000","last_fullsized":"2023-10-16T20:05:01.438259+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:11:28.851632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442830+0000","last_change":"2023-10-16T20:04:57.421462+0000","last_active":"2023-10-16T20:05:01.442830+0000","last_peered":"2023-10-16T20:05:01.442830+0000","last_clean":"2023-10-16T20:05:01.442830+0000","last_became_active":"2023-10-16T20:04:57.421107+0000","last_became_peered":"2023-10-16T20:04:57.421107+0000","last_unstale":"2023-10-16T20:05:01.442830+0000","last_undegraded":"2023-10-16T20:05:01.442830+0000","last_fullsized":"2023-10-16T20:05:01.442830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:53:20.922170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.437759+0000","last_change":"2023-10-16T20:04:57.422120+0000","last_active":"2023-10-16T20:05:01.437759+0000","last_peered":"2023-10-16T20:05:01.437759+0000","last_clean":"2023-10-16T20:05:01.437759+0000","last_became_active":"2023-10-16T20:04:57.421682+0000","last_became_peered":"2023-10-16T20:04:57.421682+0000","last_unstale":"2023-10-16T20:05:01.437759+0000","last_undegraded":"2023-10-16T20:05:01.437759+0000","last_fullsized":"2023-10-16T20:05:01.437759+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:21:09.387890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.464439+0000","last_change":"2023-10-16T20:04:57.432633+0000","last_active":"2023-10-16T20:05:01.464439+0000","last_peered":"2023-10-16T20:05:01.464439+0000","last_clean":"2023-10-16T20:05:01.464439+0000","last_became_active":"2023-10-16T20:04:57.432324+0000","last_became_peered":"2023-10-16T20:04:57.432324+0000","last_unstale":"2023-10-16T20:05:01.464439+0000","last_undegraded":"2023-10-16T20:05:01.464439+0000","last_fullsized":"2023-10-16T20:05:01.464439+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:23:06.128119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438742+0000","last_change":"2023-10-16T20:04:57.410439+0000","last_active":"2023-10-16T20:05:01.438742+0000","last_peered":"2023-10-16T20:05:01.438742+0000","last_clean":"2023-10-16T20:05:01.438742+0000","last_became_active":"2023-10-16T20:04:57.410178+0000","last_became_peered":"2023-10-16T20:04:57.410178+0000","last_unstale":"2023-10-16T20:05:01.438742+0000","last_undegraded":"2023-10-16T20:05:01.438742+0000","last_fullsized":"2023-10-16T20:05:01.438742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:33:36.402743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.758054+0000","last_change":"2023-10-16T20:04:57.436579+0000","last_active":"2023-10-16T20:05:01.758054+0000","last_peered":"2023-10-16T20:05:01.758054+0000","last_clean":"2023-10-16T20:05:01.758054+0000","last_became_active":"2023-10-16T20:04:57.436033+0000","last_became_peered":"2023-10-16T20:04:57.436033+0000","last_unstale":"2023-10-16T20:05:01.758054+0000","last_undegraded":"2023-10-16T20:05:01.758054+0000","last_fullsized":"2023-10-16T20:05:01.758054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:57:27.959012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.448091+0000","last_change":"2023-10-16T20:04:57.415573+0000","last_active":"2023-10-16T20:05:01.448091+0000","last_peered":"2023-10-16T20:05:01.448091+0000","last_clean":"2023-10-16T20:05:01.448091+0000","last_became_active":"2023-10-16T20:04:57.414461+0000","last_became_peered":"2023-10-16T20:04:57.414461+0000","last_unstale":"2023-10-16T20:05:01.448091+0000","last_undegraded":"2023-10-16T20:05:01.448091+0000","last_fullsized":"2023-10-16T20:05:01.448091+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:23:06.051449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438402+0000","last_change":"2023-10-16T20:04:57.432424+0000","last_active":"2023-10-16T20:05:01.438402+0000","last_peered":"2023-10-16T20:05:01.438402+0000","last_clean":"2023-10-16T20:05:01.438402+0000","last_became_active":"2023-10-16T20:04:57.431858+0000","last_became_peered":"2023-10-16T20:04:57.431858+0000","last_unstale":"2023-10-16T20:05:01.438402+0000","last_undegraded":"2023-10-16T20:05:01.438402+0000","last_fullsized":"2023-10-16T20:05:01.438402+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:57:27.959012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438175+0000","last_change":"2023-10-16T20:04:57.417470+0000","last_active":"2023-10-16T20:05:01.438175+0000","last_peered":"2023-10-16T20:05:01.438175+0000","last_clean":"2023-10-16T20:05:01.438175+0000","last_became_active":"2023-10-16T20:04:57.417181+0000","last_became_peered":"2023-10-16T20:04:57.417181+0000","last_unstale":"2023-10-16T20:05:01.438175+0000","last_undegraded":"2023-10-16T20:05:01.438175+0000","last_fullsized":"2023-10-16T20:05:01.438175+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:27:55.972790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471102+0000","last_change":"2023-10-16T20:04:57.431307+0000","last_active":"2023-10-16T20:05:01.471102+0000","last_peered":"2023-10-16T20:05:01.471102+0000","last_clean":"2023-10-16T20:05:01.471102+0000","last_became_active":"2023-10-16T20:04:57.430943+0000","last_became_peered":"2023-10-16T20:04:57.430943+0000","last_unstale":"2023-10-16T20:05:01.471102+0000","last_undegraded":"2023-10-16T20:05:01.471102+0000","last_fullsized":"2023-10-16T20:05:01.471102+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:17:24.205223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.463204+0000","last_change":"2023-10-16T20:04:57.428233+0000","last_active":"2023-10-16T20:05:01.463204+0000","last_peered":"2023-10-16T20:05:01.463204+0000","last_clean":"2023-10-16T20:05:01.463204+0000","last_became_active":"2023-10-16T20:04:57.427952+0000","last_became_peered":"2023-10-16T20:04:57.427952+0000","last_unstale":"2023-10-16T20:05:01.463204+0000","last_undegraded":"2023-10-16T20:05:01.463204+0000","last_fullsized":"2023-10-16T20:05:01.463204+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:57:35.297293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.441740+0000","last_change":"2023-10-16T20:04:57.425988+0000","last_active":"2023-10-16T20:05:01.441740+0000","last_peered":"2023-10-16T20:05:01.441740+0000","last_clean":"2023-10-16T20:05:01.441740+0000","last_became_active":"2023-10-16T20:04:57.425383+0000","last_became_peered":"2023-10-16T20:04:57.425383+0000","last_unstale":"2023-10-16T20:05:01.441740+0000","last_undegraded":"2023-10-16T20:05:01.441740+0000","last_fullsized":"2023-10-16T20:05:01.441740+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:23:01.141545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.447072+0000","last_change":"2023-10-16T20:04:57.415017+0000","last_active":"2023-10-16T20:05:01.447072+0000","last_peered":"2023-10-16T20:05:01.447072+0000","last_clean":"2023-10-16T20:05:01.447072+0000","last_became_active":"2023-10-16T20:04:57.414629+0000","last_became_peered":"2023-10-16T20:04:57.414629+0000","last_unstale":"2023-10-16T20:05:01.447072+0000","last_undegraded":"2023-10-16T20:05:01.447072+0000","last_fullsized":"2023-10-16T20:05:01.447072+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:41:01.253485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.439630+0000","last_change":"2023-10-16T20:04:57.421682+0000","last_active":"2023-10-16T20:05:01.439630+0000","last_peered":"2023-10-16T20:05:01.439630+0000","last_clean":"2023-10-16T20:05:01.439630+0000","last_became_active":"2023-10-16T20:04:57.421340+0000","last_became_peered":"2023-10-16T20:04:57.421340+0000","last_unstale":"2023-10-16T20:05:01.439630+0000","last_undegraded":"2023-10-16T20:05:01.439630+0000","last_fullsized":"2023-10-16T20:05:01.439630+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:56:28.993951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.440035+0000","last_change":"2023-10-16T20:04:57.415019+0000","last_active":"2023-10-16T20:05:01.440035+0000","last_peered":"2023-10-16T20:05:01.440035+0000","last_clean":"2023-10-16T20:05:01.440035+0000","last_became_active":"2023-10-16T20:04:57.414700+0000","last_became_peered":"2023-10-16T20:04:57.414700+0000","last_unstale":"2023-10-16T20:05:01.440035+0000","last_undegraded":"2023-10-16T20:05:01.440035+0000","last_fullsized":"2023-10-16T20:05:01.440035+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:17:24.205223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.758287+0000","last_change":"2023-10-16T20:04:57.436170+0000","last_active":"2023-10-16T20:05:01.758287+0000","last_peered":"2023-10-16T20:05:01.758287+0000","last_clean":"2023-10-16T20:05:01.758287+0000","last_became_active":"2023-10-16T20:04:57.435430+0000","last_became_peered":"2023-10-16T20:04:57.435430+0000","last_unstale":"2023-10-16T20:05:01.758287+0000","last_undegraded":"2023-10-16T20:05:01.758287+0000","last_fullsized":"2023-10-16T20:05:01.758287+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:39:54.738856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.441360+0000","last_change":"2023-10-16T20:04:57.429885+0000","last_active":"2023-10-16T20:05:01.441360+0000","last_peered":"2023-10-16T20:05:01.441360+0000","last_clean":"2023-10-16T20:05:01.441360+0000","last_became_active":"2023-10-16T20:04:57.429595+0000","last_became_peered":"2023-10-16T20:04:57.429595+0000","last_unstale":"2023-10-16T20:05:01.441360+0000","last_undegraded":"2023-10-16T20:05:01.441360+0000","last_fullsized":"2023-10-16T20:05:01.441360+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:49:52.409593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757637+0000","last_change":"2023-10-16T20:04:57.434436+0000","last_active":"2023-10-16T20:05:01.757637+0000","last_peered":"2023-10-16T20:05:01.757637+0000","last_clean":"2023-10-16T20:05:01.757637+0000","last_became_active":"2023-10-16T20:04:57.433525+0000","last_became_peered":"2023-10-16T20:04:57.433525+0000","last_unstale":"2023-10-16T20:05:01.757637+0000","last_undegraded":"2023-10-16T20:05:01.757637+0000","last_fullsized":"2023-10-16T20:05:01.757637+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:22:12.692981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.437681+0000","last_change":"2023-10-16T20:04:57.421933+0000","last_active":"2023-10-16T20:05:01.437681+0000","last_peered":"2023-10-16T20:05:01.437681+0000","last_clean":"2023-10-16T20:05:01.437681+0000","last_became_active":"2023-10-16T20:04:57.421342+0000","last_became_peered":"2023-10-16T20:04:57.421342+0000","last_unstale":"2023-10-16T20:05:01.437681+0000","last_undegraded":"2023-10-16T20:05:01.437681+0000","last_fullsized":"2023-10-16T20:05:01.437681+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:54:09.036700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757757+0000","last_change":"2023-10-16T20:04:57.434235+0000","last_active":"2023-10-16T20:05:01.757757+0000","last_peered":"2023-10-16T20:05:01.757757+0000","last_clean":"2023-10-16T20:05:01.757757+0000","last_became_active":"2023-10-16T20:04:57.433720+0000","last_became_peered":"2023-10-16T20:04:57.433720+0000","last_unstale":"2023-10-16T20:05:01.757757+0000","last_undegraded":"2023-10-16T20:05:01.757757+0000","last_fullsized":"2023-10-16T20:05:01.757757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:11:56.834052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438139+0000","last_change":"2023-10-16T20:04:57.431571+0000","last_active":"2023-10-16T20:05:01.438139+0000","last_peered":"2023-10-16T20:05:01.438139+0000","last_clean":"2023-10-16T20:05:01.438139+0000","last_became_active":"2023-10-16T20:04:57.429514+0000","last_became_peered":"2023-10-16T20:04:57.429514+0000","last_unstale":"2023-10-16T20:05:01.438139+0000","last_undegraded":"2023-10-16T20:05:01.438139+0000","last_fullsized":"2023-10-16T20:05:01.438139+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:31:34.884161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.464508+0000","last_change":"2023-10-16T20:04:57.412857+0000","last_active":"2023-10-16T20:05:01.464508+0000","last_peered":"2023-10-16T20:05:01.464508+0000","last_clean":"2023-10-16T20:05:01.464508+0000","last_became_active":"2023-10-16T20:04:57.412587+0000","last_became_peered":"2023-10-16T20:04:57.412587+0000","last_unstale":"2023-10-16T20:05:01.464508+0000","last_undegraded":"2023-10-16T20:05:01.464508+0000","last_fullsized":"2023-10-16T20:05:01.464508+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:42:47.128122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471650+0000","last_change":"2023-10-16T20:04:57.423730+0000","last_active":"2023-10-16T20:05:01.471650+0000","last_peered":"2023-10-16T20:05:01.471650+0000","last_clean":"2023-10-16T20:05:01.471650+0000","last_became_active":"2023-10-16T20:04:57.423300+0000","last_became_peered":"2023-10-16T20:04:57.423300+0000","last_unstale":"2023-10-16T20:05:01.471650+0000","last_undegraded":"2023-10-16T20:05:01.471650+0000","last_fullsized":"2023-10-16T20:05:01.471650+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:27:55.972790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.758256+0000","last_change":"2023-10-16T20:04:57.442158+0000","last_active":"2023-10-16T20:05:01.758256+0000","last_peered":"2023-10-16T20:05:01.758256+0000","last_clean":"2023-10-16T20:05:01.758256+0000","last_became_active":"2023-10-16T20:04:57.438707+0000","last_became_peered":"2023-10-16T20:04:57.438707+0000","last_unstale":"2023-10-16T20:05:01.758256+0000","last_undegraded":"2023-10-16T20:05:01.758256+0000","last_fullsized":"2023-10-16T20:05:01.758256+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:51:08.154913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":880640,"data_stored":701023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]}]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":294912,"data_stored":113914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]}]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":110163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":294912,"data_stored":110199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]}]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":294912,"data_stored":113914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]}]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":110073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":290816,"data_stored":110233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":888832,"data_stored":701079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T20:05:16.266 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T20:05:16.436 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.434+0000 7f5ff07c4640 1 Processor -- start 2023-10-16T20:05:16.437 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.434+0000 7f5ff07c4640 1 -- start start 2023-10-16T20:05:16.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.438+0000 7f5ff07c4640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5fe80d2660 0x7f5fe80d1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:16.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.438+0000 7f5ff07c4640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fe80b5510 con 0x7f5fe80cd970 2023-10-16T20:05:16.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.438+0000 7f5ff07c4640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fe815d9a0 con 0x7f5fe8153ea0 2023-10-16T20:05:16.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.438+0000 7f5ff07c4640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe80b4990 con 0x7f5fe80d2660 2023-10-16T20:05:16.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.438+0000 7f5feed61640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f5fe80cd970 0x7f5fe80cdd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:39146/0 (socket says 172.21.15.88:39146) 2023-10-16T20:05:16.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.438+0000 7f5feed61640 1 -- 172.21.15.88:0/30302964 learned_addr learned my addr 172.21.15.88:0/30302964 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:16.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.438+0000 7f5fed55e640 1 -- 172.21.15.88:0/30302964 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 194075085 0 0) 0x7f5fe80b5510 con 0x7f5fe80cd970 2023-10-16T20:05:16.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.438+0000 7f5fed55e640 1 -- 172.21.15.88:0/30302964 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fd4003530 con 0x7f5fe80cd970 2023-10-16T20:05:16.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.438+0000 7f5fedd5f640 1 --2- 172.21.15.88:0/30302964 >> v2:172.21.15.94:3300/0 conn(0x7f5fe80d2660 0x7f5fe80d1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:16.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.438+0000 7f5fed55e640 1 -- 172.21.15.88:0/30302964 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 489787098 0 0) 0x7f5fd4003530 con 0x7f5fe80cd970 2023-10-16T20:05:16.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.438+0000 7f5fed55e640 1 -- 172.21.15.88:0/30302964 >> v2:172.21.15.94:3300/0 conn(0x7f5fe80d2660 msgr2=0x7f5fe80d1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:16.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.438+0000 7f5fed55e640 1 --2- 172.21.15.88:0/30302964 >> v2:172.21.15.94:3300/0 conn(0x7f5fe80d2660 0x7f5fe80d1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:16.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.438+0000 7f5fed55e640 1 -- 172.21.15.88:0/30302964 >> v1:172.21.15.94:6789/0 conn(0x7f5fe8153ea0 legacy=0x7f5fe815d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:16.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.438+0000 7f5fed55e640 1 -- 172.21.15.88:0/30302964 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe80b4c70 con 0x7f5fe80cd970 2023-10-16T20:05:16.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.438+0000 7f5fed55e640 1 -- 172.21.15.88:0/30302964 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f5fe0001a00 con 0x7f5fe80cd970 2023-10-16T20:05:16.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.438+0000 7f5fed55e640 1 -- 172.21.15.88:0/30302964 >> v1:172.21.15.88:6789/0 conn(0x7f5fe80cd970 legacy=0x7f5fe80cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:16.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.438+0000 7f5fed55e640 1 --2- 172.21.15.88:0/30302964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fd4003980 0x7f5fd4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:16.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.438+0000 7f5fed55e640 1 --2- 172.21.15.88:0/30302964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fd40047d0 0x7f5fd4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:16.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.438+0000 7f5fed55e640 1 --2- 172.21.15.88:0/30302964 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5fd4005190 0x7f5fd400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:16.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.438+0000 7f5fed55e640 1 -- 172.21.15.88:0/30302964 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe80b4990 con 0x7f5fd40047d0 2023-10-16T20:05:16.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.438+0000 7f5fed55e640 1 -- 172.21.15.88:0/30302964 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd400c030 con 0x7f5fd4003980 2023-10-16T20:05:16.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.438+0000 7f5fed55e640 1 -- 172.21.15.88:0/30302964 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd400c1f0 con 0x7f5fd4005190 2023-10-16T20:05:16.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.438+0000 7f5fedd5f640 1 --2- 172.21.15.88:0/30302964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fd40047d0 0x7f5fd4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:16.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.438+0000 7f5fee560640 1 --2- 172.21.15.88:0/30302964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fd4003980 0x7f5fd4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:16.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.438+0000 7f5feed61640 1 --2- 172.21.15.88:0/30302964 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5fd4005190 0x7f5fd400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:16.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.438+0000 7f5fedd5f640 1 -- 172.21.15.88:0/30302964 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5fd4005190 msgr2=0x7f5fd400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:16.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.438+0000 7f5fedd5f640 1 --2- 172.21.15.88:0/30302964 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5fd4005190 0x7f5fd400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:16.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.438+0000 7f5fedd5f640 1 -- 172.21.15.88:0/30302964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fd4003980 msgr2=0x7f5fd4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:16.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.438+0000 7f5fedd5f640 1 --2- 172.21.15.88:0/30302964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fd4003980 0x7f5fd4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:16.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.438+0000 7f5fedd5f640 1 -- 172.21.15.88:0/30302964 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd4003530 con 0x7f5fd40047d0 2023-10-16T20:05:16.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.438+0000 7f5feed61640 1 -- 172.21.15.88:0/30302964 reap_dead start 2023-10-16T20:05:16.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.442+0000 7f5fedd5f640 1 --2- 172.21.15.88:0/30302964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fd40047d0 0x7f5fd4004c50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8009ae0 tx=0x7f5fd801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc995ba97e7d191 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:16.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.442+0000 7f5fed55e640 1 -- 172.21.15.88:0/30302964 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd80a5530 con 0x7f5fd40047d0 2023-10-16T20:05:16.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.442+0000 7f5fed55e640 1 -- 172.21.15.88:0/30302964 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd80a5690 con 0x7f5fd40047d0 2023-10-16T20:05:16.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.442+0000 7f5fed55e640 1 -- 172.21.15.88:0/30302964 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd80a5a40 con 0x7f5fd40047d0 2023-10-16T20:05:16.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.442+0000 7f5ff07c4640 1 -- 172.21.15.88:0/30302964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fd40047d0 msgr2=0x7f5fd4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:16.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.442+0000 7f5ff07c4640 1 --2- 172.21.15.88:0/30302964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fd40047d0 0x7f5fd4004c50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8009ae0 tx=0x7f5fd801ca50 comp rx=0 tx=0).stop 2023-10-16T20:05:16.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.442+0000 7f5ff07c4640 1 -- 172.21.15.88:0/30302964 shutdown_connections 2023-10-16T20:05:16.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.442+0000 7f5ff07c4640 1 --2- 172.21.15.88:0/30302964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fd40047d0 0x7f5fd4004c50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:16.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.442+0000 7f5ff07c4640 1 -- 172.21.15.88:0/30302964 >> 172.21.15.88:0/30302964 conn(0x7f5fe80715c0 msgr2=0x7f5fe80719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:16.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.442+0000 7f5ff07c4640 1 -- 172.21.15.88:0/30302964 shutdown_connections 2023-10-16T20:05:16.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.442+0000 7f5ff07c4640 1 -- 172.21.15.88:0/30302964 wait complete. 2023-10-16T20:05:16.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.446+0000 7f5ff07c4640 1 Processor -- start 2023-10-16T20:05:16.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.446+0000 7f5ff07c4640 1 -- start start 2023-10-16T20:05:16.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.446+0000 7f5ff07c4640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5fd40047d0 0x7f5fe81f1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:16.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.446+0000 7f5ff07c4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fe81f2270 0x7f5fe80cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:16.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.446+0000 7f5ff07c4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fe81f6820 0x7f5fe80cc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:16.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.446+0000 7f5ff07c4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe81f6390 con 0x7f5fe81f6820 2023-10-16T20:05:16.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.446+0000 7f5ff07c4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe81483a0 con 0x7f5fe81f2270 2023-10-16T20:05:16.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.446+0000 7f5ff07c4640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe80cc8c0 con 0x7f5fd40047d0 2023-10-16T20:05:16.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.446+0000 7f5feed61640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fe81f6820 0x7f5fe80cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:16.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.446+0000 7f5fee560640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5fd40047d0 0x7f5fe81f1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:16.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.446+0000 7f5fee560640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5fd40047d0 0x7f5fe81f1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:32850/0 (socket says 172.21.15.88:32850) 2023-10-16T20:05:16.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.446+0000 7f5feed61640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fe81f6820 0x7f5fe80cc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40184/0 (socket says 172.21.15.88:40184) 2023-10-16T20:05:16.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.446+0000 7f5fee560640 1 -- 172.21.15.88:0/2359349845 learned_addr learned my addr 172.21.15.88:0/2359349845 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:16.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.446+0000 7f5fedd5f640 1 --2- 172.21.15.88:0/2359349845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fe81f2270 0x7f5fe80cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:16.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.446+0000 7f5feed61640 1 -- 172.21.15.88:0/2359349845 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5fd40047d0 msgr2=0x7f5fe81f1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:16.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.446+0000 7f5feed61640 1 --2- 172.21.15.88:0/2359349845 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5fd40047d0 0x7f5fe81f1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:16.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.446+0000 7f5feed61640 1 -- 172.21.15.88:0/2359349845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fe81f2270 msgr2=0x7f5fe80cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:16.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.446+0000 7f5feed61640 1 --2- 172.21.15.88:0/2359349845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fe81f2270 0x7f5fe80cbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:16.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.446+0000 7f5feed61640 1 -- 172.21.15.88:0/2359349845 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe81f5d40 con 0x7f5fe81f6820 2023-10-16T20:05:16.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.446+0000 7f5feed61640 1 --2- 172.21.15.88:0/2359349845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fe81f6820 0x7f5fe80cc290 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5fe808ba50 tx=0x7f5fe0011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:16.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.446+0000 7f5fdeffd640 1 -- 172.21.15.88:0/2359349845 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0013860 con 0x7f5fe81f6820 2023-10-16T20:05:16.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.446+0000 7f5fdeffd640 1 -- 172.21.15.88:0/2359349845 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe00139e0 con 0x7f5fe81f6820 2023-10-16T20:05:16.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.446+0000 7f5fdeffd640 1 -- 172.21.15.88:0/2359349845 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0013d90 con 0x7f5fe81f6820 2023-10-16T20:05:16.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.446+0000 7f5ff07c4640 1 -- 172.21.15.88:0/2359349845 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe81f6020 con 0x7f5fe81f6820 2023-10-16T20:05:16.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.446+0000 7f5ff07c4640 1 -- 172.21.15.88:0/2359349845 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe80d0ca0 con 0x7f5fe81f6820 2023-10-16T20:05:16.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.446+0000 7f5ff07c4640 1 -- 172.21.15.88:0/2359349845 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe81f5ba0 con 0x7f5fe81f6820 2023-10-16T20:05:16.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.450+0000 7f5fdeffd640 1 -- 172.21.15.88:0/2359349845 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5fe001f460 con 0x7f5fe81f6820 2023-10-16T20:05:16.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.450+0000 7f5fdeffd640 1 --2- 172.21.15.88:0/2359349845 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5fcc0664a0 0x7f5fcc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:16.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.450+0000 7f5fdeffd640 1 -- 172.21.15.88:0/2359349845 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5fe0086ea0 con 0x7f5fe81f6820 2023-10-16T20:05:16.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.450+0000 7f5fee560640 1 --2- 172.21.15.88:0/2359349845 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5fcc0664a0 0x7f5fcc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:16.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.450+0000 7f5fee560640 1 --2- 172.21.15.88:0/2359349845 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5fcc0664a0 0x7f5fcc068960 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4007820 tx=0x7f5fe4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:16.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.454+0000 7f5fdeffd640 1 -- 172.21.15.88:0/2359349845 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5fe81f5ba0 con 0x7f5fe81f6820 2023-10-16T20:05:16.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.602+0000 7f5ff07c4640 1 -- 172.21.15.88:0/2359349845 --> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5fe80b5230 con 0x7f5fcc0664a0 2023-10-16T20:05:16.612 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.610+0000 7f5fdeffd640 1 -- 172.21.15.88:0/2359349845 <== mgr.4110 v2:172.21.15.88:6832/4078483333 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345285 (secure 0 0 0) 0x7f5fe4001cb0 con 0x7f5fcc0664a0 2023-10-16T20:05:16.616 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:16.618 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2023-10-16T20:05:16.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.614+0000 7f5ff07c4640 1 -- 172.21.15.88:0/2359349845 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5fcc0664a0 msgr2=0x7f5fcc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:16.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.614+0000 7f5ff07c4640 1 --2- 172.21.15.88:0/2359349845 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5fcc0664a0 0x7f5fcc068960 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4007820 tx=0x7f5fe4009040 comp rx=0 tx=0).stop 2023-10-16T20:05:16.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.614+0000 7f5ff07c4640 1 -- 172.21.15.88:0/2359349845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fe81f6820 msgr2=0x7f5fe80cc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:16.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.614+0000 7f5ff07c4640 1 --2- 172.21.15.88:0/2359349845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fe81f6820 0x7f5fe80cc290 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5fe808ba50 tx=0x7f5fe0011a10 comp rx=0 tx=0).stop 2023-10-16T20:05:16.620 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.614+0000 7f5ff07c4640 1 -- 172.21.15.88:0/2359349845 shutdown_connections 2023-10-16T20:05:16.620 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.614+0000 7f5ff07c4640 1 --2- 172.21.15.88:0/2359349845 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5fcc0664a0 0x7f5fcc068960 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:16.620 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.614+0000 7f5ff07c4640 1 --2- 172.21.15.88:0/2359349845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fe81f6820 0x7f5fe80cc290 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:16.620 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.614+0000 7f5ff07c4640 1 --2- 172.21.15.88:0/2359349845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fe81f2270 0x7f5fe80cbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:16.620 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.614+0000 7f5ff07c4640 1 --2- 172.21.15.88:0/2359349845 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5fd40047d0 0x7f5fe81f1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:16.621 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.614+0000 7f5ff07c4640 1 -- 172.21.15.88:0/2359349845 >> 172.21.15.88:0/2359349845 conn(0x7f5fe80715c0 msgr2=0x7f5fe815b050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:16.621 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.614+0000 7f5ff07c4640 1 -- 172.21.15.88:0/2359349845 shutdown_connections 2023-10-16T20:05:16.621 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.614+0000 7f5ff07c4640 1 -- 172.21.15.88:0/2359349845 wait complete. 2023-10-16T20:05:16.635 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2023-10-16T20:05:16.447779+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219628,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755092,"statfs":{"total":773094113280,"available":772869214208,"internally_reserved":0,"allocated":3514368,"data_stored":2070598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":221236793},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2229,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002593"},"pg_stats":[{"pgid":"2.1f","version":"22'4","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.465897+0000","last_change":"2023-10-16T20:04:56.406335+0000","last_active":"2023-10-16T20:05:01.465897+0000","last_peered":"2023-10-16T20:05:01.465897+0000","last_clean":"2023-10-16T20:05:01.465897+0000","last_became_active":"2023-10-16T20:04:56.406060+0000","last_became_peered":"2023-10-16T20:04:56.406060+0000","last_unstale":"2023-10-16T20:05:01.465897+0000","last_undegraded":"2023-10-16T20:05:01.465897+0000","last_fullsized":"2023-10-16T20:05:01.465897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:15:58.928603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442061+0000","last_change":"2023-10-16T20:04:57.416039+0000","last_active":"2023-10-16T20:05:01.442061+0000","last_peered":"2023-10-16T20:05:01.442061+0000","last_clean":"2023-10-16T20:05:01.442061+0000","last_became_active":"2023-10-16T20:04:57.415746+0000","last_became_peered":"2023-10-16T20:04:57.415746+0000","last_unstale":"2023-10-16T20:05:01.442061+0000","last_undegraded":"2023-10-16T20:05:01.442061+0000","last_fullsized":"2023-10-16T20:05:01.442061+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:27:55.972790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.685606+0000","last_change":"2023-10-16T20:05:01.770262+0000","last_active":"2023-10-16T20:05:02.685606+0000","last_peered":"2023-10-16T20:05:02.685606+0000","last_clean":"2023-10-16T20:05:02.685606+0000","last_became_active":"2023-10-16T20:05:01.769688+0000","last_became_peered":"2023-10-16T20:05:01.769688+0000","last_unstale":"2023-10-16T20:05:02.685606+0000","last_undegraded":"2023-10-16T20:05:02.685606+0000","last_fullsized":"2023-10-16T20:05:02.685606+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:51:12.210160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442225+0000","last_change":"2023-10-16T20:04:56.409621+0000","last_active":"2023-10-16T20:05:01.442225+0000","last_peered":"2023-10-16T20:05:01.442225+0000","last_clean":"2023-10-16T20:05:01.442225+0000","last_became_active":"2023-10-16T20:04:56.409150+0000","last_became_peered":"2023-10-16T20:04:56.409150+0000","last_unstale":"2023-10-16T20:05:01.442225+0000","last_undegraded":"2023-10-16T20:05:01.442225+0000","last_fullsized":"2023-10-16T20:05:01.442225+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:24:16.965233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.465724+0000","last_change":"2023-10-16T20:04:57.430587+0000","last_active":"2023-10-16T20:05:01.465724+0000","last_peered":"2023-10-16T20:05:01.465724+0000","last_clean":"2023-10-16T20:05:01.465724+0000","last_became_active":"2023-10-16T20:04:57.430224+0000","last_became_peered":"2023-10-16T20:04:57.430224+0000","last_unstale":"2023-10-16T20:05:01.465724+0000","last_undegraded":"2023-10-16T20:05:01.465724+0000","last_fullsized":"2023-10-16T20:05:01.465724+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:49:52.662102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.682460+0000","last_change":"2023-10-16T20:05:01.478221+0000","last_active":"2023-10-16T20:05:02.682460+0000","last_peered":"2023-10-16T20:05:02.682460+0000","last_clean":"2023-10-16T20:05:02.682460+0000","last_became_active":"2023-10-16T20:05:01.477692+0000","last_became_peered":"2023-10-16T20:05:01.477692+0000","last_unstale":"2023-10-16T20:05:02.682460+0000","last_undegraded":"2023-10-16T20:05:02.682460+0000","last_fullsized":"2023-10-16T20:05:02.682460+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:58:37.917893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.758057+0000","last_change":"2023-10-16T20:04:56.409779+0000","last_active":"2023-10-16T20:05:01.758057+0000","last_peered":"2023-10-16T20:05:01.758057+0000","last_clean":"2023-10-16T20:05:01.758057+0000","last_became_active":"2023-10-16T20:04:56.409515+0000","last_became_peered":"2023-10-16T20:04:56.409515+0000","last_unstale":"2023-10-16T20:05:01.758057+0000","last_undegraded":"2023-10-16T20:05:01.758057+0000","last_fullsized":"2023-10-16T20:05:01.758057+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:17:23.198236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438236+0000","last_change":"2023-10-16T20:04:57.414284+0000","last_active":"2023-10-16T20:05:01.438236+0000","last_peered":"2023-10-16T20:05:01.438236+0000","last_clean":"2023-10-16T20:05:01.438236+0000","last_became_active":"2023-10-16T20:04:57.413997+0000","last_became_peered":"2023-10-16T20:04:57.413997+0000","last_unstale":"2023-10-16T20:05:01.438236+0000","last_undegraded":"2023-10-16T20:05:01.438236+0000","last_fullsized":"2023-10-16T20:05:01.438236+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:28:00.096002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.682751+0000","last_change":"2023-10-16T20:05:01.482429+0000","last_active":"2023-10-16T20:05:02.682751+0000","last_peered":"2023-10-16T20:05:02.682751+0000","last_clean":"2023-10-16T20:05:02.682751+0000","last_became_active":"2023-10-16T20:05:01.481993+0000","last_became_peered":"2023-10-16T20:05:01.481993+0000","last_unstale":"2023-10-16T20:05:02.682751+0000","last_undegraded":"2023-10-16T20:05:02.682751+0000","last_fullsized":"2023-10-16T20:05:02.682751+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:54:13.091947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757866+0000","last_change":"2023-10-16T20:04:56.412981+0000","last_active":"2023-10-16T20:05:01.757866+0000","last_peered":"2023-10-16T20:05:01.757866+0000","last_clean":"2023-10-16T20:05:01.757866+0000","last_became_active":"2023-10-16T20:04:56.412613+0000","last_became_peered":"2023-10-16T20:04:56.412613+0000","last_unstale":"2023-10-16T20:05:01.757866+0000","last_undegraded":"2023-10-16T20:05:01.757866+0000","last_fullsized":"2023-10-16T20:05:01.757866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:14:27.634593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.440267+0000","last_change":"2023-10-16T20:04:57.411866+0000","last_active":"2023-10-16T20:05:01.440267+0000","last_peered":"2023-10-16T20:05:01.440267+0000","last_clean":"2023-10-16T20:05:01.440267+0000","last_became_active":"2023-10-16T20:04:57.411523+0000","last_became_peered":"2023-10-16T20:04:57.411523+0000","last_unstale":"2023-10-16T20:05:01.440267+0000","last_undegraded":"2023-10-16T20:05:01.440267+0000","last_fullsized":"2023-10-16T20:05:01.440267+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:01:38.242317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.683398+0000","last_change":"2023-10-16T20:05:01.460546+0000","last_active":"2023-10-16T20:05:02.683398+0000","last_peered":"2023-10-16T20:05:02.683398+0000","last_clean":"2023-10-16T20:05:02.683398+0000","last_became_active":"2023-10-16T20:05:01.460120+0000","last_became_peered":"2023-10-16T20:05:01.460120+0000","last_unstale":"2023-10-16T20:05:02.683398+0000","last_undegraded":"2023-10-16T20:05:02.683398+0000","last_fullsized":"2023-10-16T20:05:02.683398+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:31:38.939408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438705+0000","last_change":"2023-10-16T20:04:57.409882+0000","last_active":"2023-10-16T20:05:01.438705+0000","last_peered":"2023-10-16T20:05:01.438705+0000","last_clean":"2023-10-16T20:05:01.438705+0000","last_became_active":"2023-10-16T20:04:57.409534+0000","last_became_peered":"2023-10-16T20:04:57.409534+0000","last_unstale":"2023-10-16T20:05:01.438705+0000","last_undegraded":"2023-10-16T20:05:01.438705+0000","last_fullsized":"2023-10-16T20:05:01.438705+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:23:01.141545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757875+0000","last_change":"2023-10-16T20:04:56.406078+0000","last_active":"2023-10-16T20:05:01.757875+0000","last_peered":"2023-10-16T20:05:01.757875+0000","last_clean":"2023-10-16T20:05:01.757875+0000","last_became_active":"2023-10-16T20:04:56.405784+0000","last_became_peered":"2023-10-16T20:04:56.405784+0000","last_unstale":"2023-10-16T20:05:01.757875+0000","last_undegraded":"2023-10-16T20:05:01.757875+0000","last_fullsized":"2023-10-16T20:05:01.757875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:13:40.552947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.683535+0000","last_change":"2023-10-16T20:05:01.477288+0000","last_active":"2023-10-16T20:05:02.683535+0000","last_peered":"2023-10-16T20:05:02.683535+0000","last_clean":"2023-10-16T20:05:02.683535+0000","last_became_active":"2023-10-16T20:05:01.476269+0000","last_became_peered":"2023-10-16T20:05:01.476269+0000","last_unstale":"2023-10-16T20:05:02.683535+0000","last_undegraded":"2023-10-16T20:05:02.683535+0000","last_fullsized":"2023-10-16T20:05:02.683535+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:28:04.151249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.441837+0000","last_change":"2023-10-16T20:04:57.426154+0000","last_active":"2023-10-16T20:05:01.441837+0000","last_peered":"2023-10-16T20:05:01.441837+0000","last_clean":"2023-10-16T20:05:01.441837+0000","last_became_active":"2023-10-16T20:04:57.425681+0000","last_became_peered":"2023-10-16T20:04:57.425681+0000","last_unstale":"2023-10-16T20:05:01.441837+0000","last_undegraded":"2023-10-16T20:05:01.441837+0000","last_fullsized":"2023-10-16T20:05:01.441837+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:54:09.036700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471304+0000","last_change":"2023-10-16T20:04:56.406283+0000","last_active":"2023-10-16T20:05:01.471304+0000","last_peered":"2023-10-16T20:05:01.471304+0000","last_clean":"2023-10-16T20:05:01.471304+0000","last_became_active":"2023-10-16T20:04:56.405952+0000","last_became_peered":"2023-10-16T20:04:56.405952+0000","last_unstale":"2023-10-16T20:05:01.471304+0000","last_undegraded":"2023-10-16T20:05:01.471304+0000","last_fullsized":"2023-10-16T20:05:01.471304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:14:27.634593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.682647+0000","last_change":"2023-10-16T20:05:01.482495+0000","last_active":"2023-10-16T20:05:02.682647+0000","last_peered":"2023-10-16T20:05:02.682647+0000","last_clean":"2023-10-16T20:05:02.682647+0000","last_became_active":"2023-10-16T20:05:01.482072+0000","last_became_peered":"2023-10-16T20:05:01.482072+0000","last_unstale":"2023-10-16T20:05:02.682647+0000","last_undegraded":"2023-10-16T20:05:02.682647+0000","last_fullsized":"2023-10-16T20:05:02.682647+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:28:00.028037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442089+0000","last_change":"2023-10-16T20:04:57.425668+0000","last_active":"2023-10-16T20:05:01.442089+0000","last_peered":"2023-10-16T20:05:01.442089+0000","last_clean":"2023-10-16T20:05:01.442089+0000","last_became_active":"2023-10-16T20:04:57.425420+0000","last_became_peered":"2023-10-16T20:04:57.425420+0000","last_unstale":"2023-10-16T20:05:01.442089+0000","last_undegraded":"2023-10-16T20:05:01.442089+0000","last_fullsized":"2023-10-16T20:05:01.442089+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:33:36.402743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471190+0000","last_change":"2023-10-16T20:04:56.403231+0000","last_active":"2023-10-16T20:05:01.471190+0000","last_peered":"2023-10-16T20:05:01.471190+0000","last_clean":"2023-10-16T20:05:01.471190+0000","last_became_active":"2023-10-16T20:04:56.402769+0000","last_became_peered":"2023-10-16T20:04:56.402769+0000","last_unstale":"2023-10-16T20:05:01.471190+0000","last_undegraded":"2023-10-16T20:05:01.471190+0000","last_fullsized":"2023-10-16T20:05:01.471190+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:55:47.529420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.445092+0000","last_change":"2023-10-16T20:05:01.768596+0000","last_active":"2023-10-16T20:05:02.445092+0000","last_peered":"2023-10-16T20:05:02.445092+0000","last_clean":"2023-10-16T20:05:02.445092+0000","last_became_active":"2023-10-16T20:05:01.768140+0000","last_became_peered":"2023-10-16T20:05:01.768140+0000","last_unstale":"2023-10-16T20:05:02.445092+0000","last_undegraded":"2023-10-16T20:05:02.445092+0000","last_fullsized":"2023-10-16T20:05:02.445092+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:01:42.297564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.447386+0000","last_change":"2023-10-16T20:04:57.407922+0000","last_active":"2023-10-16T20:05:01.447386+0000","last_peered":"2023-10-16T20:05:01.447386+0000","last_clean":"2023-10-16T20:05:01.447386+0000","last_became_active":"2023-10-16T20:04:57.407580+0000","last_became_peered":"2023-10-16T20:04:57.407580+0000","last_unstale":"2023-10-16T20:05:01.447386+0000","last_undegraded":"2023-10-16T20:05:01.447386+0000","last_fullsized":"2023-10-16T20:05:01.447386+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:24:17.972220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757524+0000","last_change":"2023-10-16T20:04:56.408300+0000","last_active":"2023-10-16T20:05:01.757524+0000","last_peered":"2023-10-16T20:05:01.757524+0000","last_clean":"2023-10-16T20:05:01.757524+0000","last_became_active":"2023-10-16T20:04:56.407908+0000","last_became_peered":"2023-10-16T20:04:56.407908+0000","last_unstale":"2023-10-16T20:05:01.757524+0000","last_undegraded":"2023-10-16T20:05:01.757524+0000","last_fullsized":"2023-10-16T20:05:01.757524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:53:19.915183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.683126+0000","last_change":"2023-10-16T20:05:01.476964+0000","last_active":"2023-10-16T20:05:02.683126+0000","last_peered":"2023-10-16T20:05:02.683126+0000","last_clean":"2023-10-16T20:05:02.683126+0000","last_became_active":"2023-10-16T20:05:01.476584+0000","last_became_peered":"2023-10-16T20:05:01.476584+0000","last_unstale":"2023-10-16T20:05:02.683126+0000","last_undegraded":"2023-10-16T20:05:02.683126+0000","last_fullsized":"2023-10-16T20:05:02.683126+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:22:16.748229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.465558+0000","last_change":"2023-10-16T20:04:57.424772+0000","last_active":"2023-10-16T20:05:01.465558+0000","last_peered":"2023-10-16T20:05:01.465558+0000","last_clean":"2023-10-16T20:05:01.465558+0000","last_became_active":"2023-10-16T20:04:57.424340+0000","last_became_peered":"2023-10-16T20:04:57.424340+0000","last_unstale":"2023-10-16T20:05:01.465558+0000","last_undegraded":"2023-10-16T20:05:01.465558+0000","last_fullsized":"2023-10-16T20:05:01.465558+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:10:08.760019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.439134+0000","last_change":"2023-10-16T20:04:56.401336+0000","last_active":"2023-10-16T20:05:01.439134+0000","last_peered":"2023-10-16T20:05:01.439134+0000","last_clean":"2023-10-16T20:05:01.439134+0000","last_became_active":"2023-10-16T20:04:56.401019+0000","last_became_peered":"2023-10-16T20:04:56.401019+0000","last_unstale":"2023-10-16T20:05:01.439134+0000","last_undegraded":"2023-10-16T20:05:01.439134+0000","last_fullsized":"2023-10-16T20:05:01.439134+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:58:19.945263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.682734+0000","last_change":"2023-10-16T20:05:01.760505+0000","last_active":"2023-10-16T20:05:02.682734+0000","last_peered":"2023-10-16T20:05:02.682734+0000","last_clean":"2023-10-16T20:05:02.682734+0000","last_became_active":"2023-10-16T20:05:01.760043+0000","last_became_peered":"2023-10-16T20:05:01.760043+0000","last_unstale":"2023-10-16T20:05:02.682734+0000","last_undegraded":"2023-10-16T20:05:02.682734+0000","last_fullsized":"2023-10-16T20:05:02.682734+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:11:32.906879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442279+0000","last_change":"2023-10-16T20:04:57.430184+0000","last_active":"2023-10-16T20:05:01.442279+0000","last_peered":"2023-10-16T20:05:01.442279+0000","last_clean":"2023-10-16T20:05:01.442279+0000","last_became_active":"2023-10-16T20:04:57.429845+0000","last_became_peered":"2023-10-16T20:04:57.429845+0000","last_unstale":"2023-10-16T20:05:01.442279+0000","last_undegraded":"2023-10-16T20:05:01.442279+0000","last_fullsized":"2023-10-16T20:05:01.442279+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:11:28.851632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471731+0000","last_change":"2023-10-16T20:04:56.403148+0000","last_active":"2023-10-16T20:05:01.471731+0000","last_peered":"2023-10-16T20:05:01.471731+0000","last_clean":"2023-10-16T20:05:01.471731+0000","last_became_active":"2023-10-16T20:04:56.402707+0000","last_became_peered":"2023-10-16T20:04:56.402707+0000","last_unstale":"2023-10-16T20:05:01.471731+0000","last_undegraded":"2023-10-16T20:05:01.471731+0000","last_fullsized":"2023-10-16T20:05:01.471731+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:53:19.915183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.685503+0000","last_change":"2023-10-16T20:05:01.764474+0000","last_active":"2023-10-16T20:05:02.685503+0000","last_peered":"2023-10-16T20:05:02.685503+0000","last_clean":"2023-10-16T20:05:02.685503+0000","last_became_active":"2023-10-16T20:05:01.763840+0000","last_became_peered":"2023-10-16T20:05:01.763840+0000","last_unstale":"2023-10-16T20:05:02.685503+0000","last_undegraded":"2023-10-16T20:05:02.685503+0000","last_fullsized":"2023-10-16T20:05:02.685503+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:57:32.014259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471448+0000","last_change":"2023-10-16T20:04:57.423634+0000","last_active":"2023-10-16T20:05:01.471448+0000","last_peered":"2023-10-16T20:05:01.471448+0000","last_clean":"2023-10-16T20:05:01.471448+0000","last_became_active":"2023-10-16T20:04:57.423321+0000","last_became_peered":"2023-10-16T20:04:57.423321+0000","last_unstale":"2023-10-16T20:05:01.471448+0000","last_undegraded":"2023-10-16T20:05:01.471448+0000","last_fullsized":"2023-10-16T20:05:01.471448+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:23:01.141545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.758143+0000","last_change":"2023-10-16T20:04:56.408656+0000","last_active":"2023-10-16T20:05:01.758143+0000","last_peered":"2023-10-16T20:05:01.758143+0000","last_clean":"2023-10-16T20:05:01.758143+0000","last_became_active":"2023-10-16T20:04:56.408192+0000","last_became_peered":"2023-10-16T20:04:56.408192+0000","last_unstale":"2023-10-16T20:05:01.758143+0000","last_undegraded":"2023-10-16T20:05:01.758143+0000","last_fullsized":"2023-10-16T20:05:01.758143+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:54:08.029713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.688512+0000","last_change":"2023-10-16T20:05:01.767238+0000","last_active":"2023-10-16T20:05:02.688512+0000","last_peered":"2023-10-16T20:05:02.688512+0000","last_clean":"2023-10-16T20:05:02.688512+0000","last_became_active":"2023-10-16T20:05:01.765827+0000","last_became_peered":"2023-10-16T20:05:01.765827+0000","last_unstale":"2023-10-16T20:05:02.688512+0000","last_undegraded":"2023-10-16T20:05:02.688512+0000","last_fullsized":"2023-10-16T20:05:02.688512+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:14:36.535545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.440300+0000","last_change":"2023-10-16T20:04:57.430055+0000","last_active":"2023-10-16T20:05:01.440300+0000","last_peered":"2023-10-16T20:05:01.440300+0000","last_clean":"2023-10-16T20:05:01.440300+0000","last_became_active":"2023-10-16T20:04:57.428753+0000","last_became_peered":"2023-10-16T20:04:57.428753+0000","last_unstale":"2023-10-16T20:05:01.440300+0000","last_undegraded":"2023-10-16T20:05:01.440300+0000","last_fullsized":"2023-10-16T20:05:01.440300+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:14:29.143380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471458+0000","last_change":"2023-10-16T20:04:56.411374+0000","last_active":"2023-10-16T20:05:01.471458+0000","last_peered":"2023-10-16T20:05:01.471458+0000","last_clean":"2023-10-16T20:05:01.471458+0000","last_became_active":"2023-10-16T20:04:56.411098+0000","last_became_peered":"2023-10-16T20:04:56.411098+0000","last_unstale":"2023-10-16T20:05:01.471458+0000","last_undegraded":"2023-10-16T20:05:01.471458+0000","last_fullsized":"2023-10-16T20:05:01.471458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:04:32.948232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.444207+0000","last_change":"2023-10-16T20:05:01.473814+0000","last_active":"2023-10-16T20:05:02.444207+0000","last_peered":"2023-10-16T20:05:02.444207+0000","last_clean":"2023-10-16T20:05:02.444207+0000","last_became_active":"2023-10-16T20:05:01.472021+0000","last_became_peered":"2023-10-16T20:05:01.472021+0000","last_unstale":"2023-10-16T20:05:02.444207+0000","last_undegraded":"2023-10-16T20:05:02.444207+0000","last_fullsized":"2023-10-16T20:05:02.444207+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:12:00.889299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.447473+0000","last_change":"2023-10-16T20:04:57.419253+0000","last_active":"2023-10-16T20:05:01.447473+0000","last_peered":"2023-10-16T20:05:01.447473+0000","last_clean":"2023-10-16T20:05:01.447473+0000","last_became_active":"2023-10-16T20:04:57.418196+0000","last_became_peered":"2023-10-16T20:04:57.418196+0000","last_unstale":"2023-10-16T20:05:01.447473+0000","last_undegraded":"2023-10-16T20:05:01.447473+0000","last_fullsized":"2023-10-16T20:05:01.447473+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:58:20.952250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.437911+0000","last_change":"2023-10-16T20:04:56.405694+0000","last_active":"2023-10-16T20:05:01.437911+0000","last_peered":"2023-10-16T20:05:01.437911+0000","last_clean":"2023-10-16T20:05:01.437911+0000","last_became_active":"2023-10-16T20:04:56.405369+0000","last_became_peered":"2023-10-16T20:04:56.405369+0000","last_unstale":"2023-10-16T20:05:01.437911+0000","last_undegraded":"2023-10-16T20:05:01.437911+0000","last_fullsized":"2023-10-16T20:05:01.437911+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:58:32.855659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.444695+0000","last_change":"2023-10-16T20:05:01.478161+0000","last_active":"2023-10-16T20:05:02.444695+0000","last_peered":"2023-10-16T20:05:02.444695+0000","last_clean":"2023-10-16T20:05:02.444695+0000","last_became_active":"2023-10-16T20:05:01.477735+0000","last_became_peered":"2023-10-16T20:05:01.477735+0000","last_unstale":"2023-10-16T20:05:02.444695+0000","last_undegraded":"2023-10-16T20:05:02.444695+0000","last_fullsized":"2023-10-16T20:05:02.444695+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:14:33.198627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471054+0000","last_change":"2023-10-16T20:04:57.427768+0000","last_active":"2023-10-16T20:05:01.471054+0000","last_peered":"2023-10-16T20:05:01.471054+0000","last_clean":"2023-10-16T20:05:01.471054+0000","last_became_active":"2023-10-16T20:04:57.422018+0000","last_became_peered":"2023-10-16T20:04:57.422018+0000","last_unstale":"2023-10-16T20:05:01.471054+0000","last_undegraded":"2023-10-16T20:05:01.471054+0000","last_fullsized":"2023-10-16T20:05:01.471054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:49:52.409593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.448426+0000","last_change":"2023-10-16T20:04:56.402767+0000","last_active":"2023-10-16T20:05:01.448426+0000","last_peered":"2023-10-16T20:05:01.448426+0000","last_clean":"2023-10-16T20:05:01.448426+0000","last_became_active":"2023-10-16T20:04:56.402122+0000","last_became_peered":"2023-10-16T20:04:56.402122+0000","last_unstale":"2023-10-16T20:05:01.448426+0000","last_undegraded":"2023-10-16T20:05:01.448426+0000","last_fullsized":"2023-10-16T20:05:01.448426+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:54:36.775882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.445579+0000","last_change":"2023-10-16T20:05:01.478710+0000","last_active":"2023-10-16T20:05:02.445579+0000","last_peered":"2023-10-16T20:05:02.445579+0000","last_clean":"2023-10-16T20:05:02.445579+0000","last_became_active":"2023-10-16T20:05:01.478350+0000","last_became_peered":"2023-10-16T20:05:01.478350+0000","last_unstale":"2023-10-16T20:05:02.445579+0000","last_undegraded":"2023-10-16T20:05:02.445579+0000","last_fullsized":"2023-10-16T20:05:02.445579+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:57:32.014259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.459302+0000","last_change":"2023-10-16T20:04:57.420638+0000","last_active":"2023-10-16T20:05:01.459302+0000","last_peered":"2023-10-16T20:05:01.459302+0000","last_clean":"2023-10-16T20:05:01.459302+0000","last_became_active":"2023-10-16T20:04:57.420348+0000","last_became_peered":"2023-10-16T20:04:57.420348+0000","last_unstale":"2023-10-16T20:05:01.459302+0000","last_undegraded":"2023-10-16T20:05:01.459302+0000","last_fullsized":"2023-10-16T20:05:01.459302+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:14:18.231180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757558+0000","last_change":"2023-10-16T20:04:56.403488+0000","last_active":"2023-10-16T20:05:01.757558+0000","last_peered":"2023-10-16T20:05:01.757558+0000","last_clean":"2023-10-16T20:05:01.757558+0000","last_became_active":"2023-10-16T20:04:56.403156+0000","last_became_peered":"2023-10-16T20:04:56.403156+0000","last_unstale":"2023-10-16T20:05:01.757558+0000","last_undegraded":"2023-10-16T20:05:01.757558+0000","last_fullsized":"2023-10-16T20:05:01.757558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:33:35.395756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.688883+0000","last_change":"2023-10-16T20:05:01.765108+0000","last_active":"2023-10-16T20:05:02.688883+0000","last_peered":"2023-10-16T20:05:02.688883+0000","last_clean":"2023-10-16T20:05:02.688883+0000","last_became_active":"2023-10-16T20:05:01.764436+0000","last_became_peered":"2023-10-16T20:05:01.764436+0000","last_unstale":"2023-10-16T20:05:02.688883+0000","last_undegraded":"2023-10-16T20:05:02.688883+0000","last_fullsized":"2023-10-16T20:05:02.688883+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:39:29.554873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442360+0000","last_change":"2023-10-16T20:04:57.429060+0000","last_active":"2023-10-16T20:05:01.442360+0000","last_peered":"2023-10-16T20:05:01.442360+0000","last_clean":"2023-10-16T20:05:01.442360+0000","last_became_active":"2023-10-16T20:04:57.428756+0000","last_became_peered":"2023-10-16T20:04:57.428756+0000","last_unstale":"2023-10-16T20:05:01.442360+0000","last_undegraded":"2023-10-16T20:05:01.442360+0000","last_fullsized":"2023-10-16T20:05:01.442360+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:21:09.387890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.437753+0000","last_change":"2023-10-16T20:04:56.392274+0000","last_active":"2023-10-16T20:05:01.437753+0000","last_peered":"2023-10-16T20:05:01.437753+0000","last_clean":"2023-10-16T20:05:01.437753+0000","last_became_active":"2023-10-16T20:04:56.391958+0000","last_became_peered":"2023-10-16T20:04:56.391958+0000","last_unstale":"2023-10-16T20:05:01.437753+0000","last_undegraded":"2023-10-16T20:05:01.437753+0000","last_fullsized":"2023-10-16T20:05:01.437753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:58:19.945263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":169,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757620+0000","last_change":"2023-10-16T20:04:46.335822+0000","last_active":"2023-10-16T20:05:01.757620+0000","last_peered":"2023-10-16T20:05:01.757620+0000","last_clean":"2023-10-16T20:05:01.757620+0000","last_became_active":"2023-10-16T20:04:46.334617+0000","last_became_peered":"2023-10-16T20:04:46.334617+0000","last_unstale":"2023-10-16T20:05:01.757620+0000","last_undegraded":"2023-10-16T20:05:01.757620+0000","last_fullsized":"2023-10-16T20:05:01.757620+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:45.306240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:45.306240+0000","last_clean_scrub_stamp":"2023-10-16T20:04:45.306240+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:42:11.153721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.682739+0000","last_change":"2023-10-16T20:05:01.760533+0000","last_active":"2023-10-16T20:05:02.682739+0000","last_peered":"2023-10-16T20:05:02.682739+0000","last_clean":"2023-10-16T20:05:02.682739+0000","last_became_active":"2023-10-16T20:05:01.760091+0000","last_became_peered":"2023-10-16T20:05:01.760091+0000","last_unstale":"2023-10-16T20:05:02.682739+0000","last_undegraded":"2023-10-16T20:05:02.682739+0000","last_fullsized":"2023-10-16T20:05:02.682739+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:22:16.748229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757962+0000","last_change":"2023-10-16T20:04:57.435770+0000","last_active":"2023-10-16T20:05:01.757962+0000","last_peered":"2023-10-16T20:05:01.757962+0000","last_clean":"2023-10-16T20:05:01.757962+0000","last_became_active":"2023-10-16T20:04:57.434914+0000","last_became_peered":"2023-10-16T20:04:57.434914+0000","last_unstale":"2023-10-16T20:05:01.757962+0000","last_undegraded":"2023-10-16T20:05:01.757962+0000","last_fullsized":"2023-10-16T20:05:01.757962+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:42:35.216191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.449802+0000","last_change":"2023-10-16T20:04:56.414794+0000","last_active":"2023-10-16T20:05:01.449802+0000","last_peered":"2023-10-16T20:05:01.449802+0000","last_clean":"2023-10-16T20:05:01.449802+0000","last_became_active":"2023-10-16T20:04:56.414400+0000","last_became_peered":"2023-10-16T20:04:56.414400+0000","last_unstale":"2023-10-16T20:05:01.449802+0000","last_undegraded":"2023-10-16T20:05:01.449802+0000","last_fullsized":"2023-10-16T20:05:01.449802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:19:48.110438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.682395+0000","last_change":"2023-10-16T20:05:01.760979+0000","last_active":"2023-10-16T20:05:02.682395+0000","last_peered":"2023-10-16T20:05:02.682395+0000","last_clean":"2023-10-16T20:05:02.682395+0000","last_became_active":"2023-10-16T20:05:01.760141+0000","last_became_peered":"2023-10-16T20:05:01.760141+0000","last_unstale":"2023-10-16T20:05:02.682395+0000","last_undegraded":"2023-10-16T20:05:02.682395+0000","last_fullsized":"2023-10-16T20:05:02.682395+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:21:13.443137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438336+0000","last_change":"2023-10-16T20:04:57.429698+0000","last_active":"2023-10-16T20:05:01.438336+0000","last_peered":"2023-10-16T20:05:01.438336+0000","last_clean":"2023-10-16T20:05:01.438336+0000","last_became_active":"2023-10-16T20:04:57.428050+0000","last_became_peered":"2023-10-16T20:04:57.428050+0000","last_unstale":"2023-10-16T20:05:01.438336+0000","last_undegraded":"2023-10-16T20:05:01.438336+0000","last_fullsized":"2023-10-16T20:05:01.438336+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:42:35.216191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471736+0000","last_change":"2023-10-16T20:04:56.409587+0000","last_active":"2023-10-16T20:05:01.471736+0000","last_peered":"2023-10-16T20:05:01.471736+0000","last_clean":"2023-10-16T20:05:01.471736+0000","last_became_active":"2023-10-16T20:04:56.408150+0000","last_became_peered":"2023-10-16T20:04:56.408150+0000","last_unstale":"2023-10-16T20:05:01.471736+0000","last_undegraded":"2023-10-16T20:05:01.471736+0000","last_fullsized":"2023-10-16T20:05:01.471736+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:13:40.552947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.688655+0000","last_change":"2023-10-16T20:05:01.476280+0000","last_active":"2023-10-16T20:05:02.688655+0000","last_peered":"2023-10-16T20:05:02.688655+0000","last_clean":"2023-10-16T20:05:02.688655+0000","last_became_active":"2023-10-16T20:05:01.475885+0000","last_became_peered":"2023-10-16T20:05:01.475885+0000","last_unstale":"2023-10-16T20:05:02.688655+0000","last_undegraded":"2023-10-16T20:05:02.688655+0000","last_fullsized":"2023-10-16T20:05:02.688655+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:01:16.289925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.758343+0000","last_change":"2023-10-16T20:04:57.442292+0000","last_active":"2023-10-16T20:05:01.758343+0000","last_peered":"2023-10-16T20:05:01.758343+0000","last_clean":"2023-10-16T20:05:01.758343+0000","last_became_active":"2023-10-16T20:04:57.439075+0000","last_became_peered":"2023-10-16T20:04:57.439075+0000","last_unstale":"2023-10-16T20:05:01.758343+0000","last_undegraded":"2023-10-16T20:05:01.758343+0000","last_fullsized":"2023-10-16T20:05:01.758343+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:25:46.207669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.448189+0000","last_change":"2023-10-16T20:04:56.398007+0000","last_active":"2023-10-16T20:05:01.448189+0000","last_peered":"2023-10-16T20:05:01.448189+0000","last_clean":"2023-10-16T20:05:01.448189+0000","last_became_active":"2023-10-16T20:04:56.397108+0000","last_became_peered":"2023-10-16T20:04:56.397108+0000","last_unstale":"2023-10-16T20:05:01.448189+0000","last_undegraded":"2023-10-16T20:05:01.448189+0000","last_fullsized":"2023-10-16T20:05:01.448189+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:24:16.965233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.684215+0000","last_change":"2023-10-16T20:05:01.763390+0000","last_active":"2023-10-16T20:05:02.684215+0000","last_peered":"2023-10-16T20:05:02.684215+0000","last_clean":"2023-10-16T20:05:02.684215+0000","last_became_active":"2023-10-16T20:05:01.760077+0000","last_became_peered":"2023-10-16T20:05:01.760077+0000","last_unstale":"2023-10-16T20:05:02.684215+0000","last_undegraded":"2023-10-16T20:05:02.684215+0000","last_fullsized":"2023-10-16T20:05:02.684215+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:10:33.807776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.758160+0000","last_change":"2023-10-16T20:04:57.437968+0000","last_active":"2023-10-16T20:05:01.758160+0000","last_peered":"2023-10-16T20:05:01.758160+0000","last_clean":"2023-10-16T20:05:01.758160+0000","last_became_active":"2023-10-16T20:04:57.437674+0000","last_became_peered":"2023-10-16T20:04:57.437674+0000","last_unstale":"2023-10-16T20:05:01.758160+0000","last_undegraded":"2023-10-16T20:05:01.758160+0000","last_fullsized":"2023-10-16T20:05:01.758160+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:01:38.242317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.447078+0000","last_change":"2023-10-16T20:04:56.408227+0000","last_active":"2023-10-16T20:05:01.447078+0000","last_peered":"2023-10-16T20:05:01.447078+0000","last_clean":"2023-10-16T20:05:01.447078+0000","last_became_active":"2023-10-16T20:04:56.407279+0000","last_became_peered":"2023-10-16T20:04:56.407279+0000","last_unstale":"2023-10-16T20:05:01.447078+0000","last_undegraded":"2023-10-16T20:05:01.447078+0000","last_fullsized":"2023-10-16T20:05:01.447078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:43:11.864176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.682415+0000","last_change":"2023-10-16T20:05:01.477611+0000","last_active":"2023-10-16T20:05:02.682415+0000","last_peered":"2023-10-16T20:05:02.682415+0000","last_clean":"2023-10-16T20:05:02.682415+0000","last_became_active":"2023-10-16T20:05:01.477172+0000","last_became_peered":"2023-10-16T20:05:01.477172+0000","last_unstale":"2023-10-16T20:05:02.682415+0000","last_undegraded":"2023-10-16T20:05:02.682415+0000","last_fullsized":"2023-10-16T20:05:02.682415+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:23:05.196792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757700+0000","last_change":"2023-10-16T20:04:57.438300+0000","last_active":"2023-10-16T20:05:01.757700+0000","last_peered":"2023-10-16T20:05:01.757700+0000","last_clean":"2023-10-16T20:05:01.757700+0000","last_became_active":"2023-10-16T20:04:57.437951+0000","last_became_peered":"2023-10-16T20:04:57.437951+0000","last_unstale":"2023-10-16T20:05:01.757700+0000","last_undegraded":"2023-10-16T20:05:01.757700+0000","last_fullsized":"2023-10-16T20:05:01.757700+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:31:34.884161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.448376+0000","last_change":"2023-10-16T20:04:56.401341+0000","last_active":"2023-10-16T20:05:01.448376+0000","last_peered":"2023-10-16T20:05:01.448376+0000","last_clean":"2023-10-16T20:05:01.448376+0000","last_became_active":"2023-10-16T20:04:56.401058+0000","last_became_peered":"2023-10-16T20:04:56.401058+0000","last_unstale":"2023-10-16T20:05:01.448376+0000","last_undegraded":"2023-10-16T20:05:01.448376+0000","last_fullsized":"2023-10-16T20:05:01.448376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:41:34.379170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.688808+0000","last_change":"2023-10-16T20:05:01.463304+0000","last_active":"2023-10-16T20:05:02.688808+0000","last_peered":"2023-10-16T20:05:02.688808+0000","last_clean":"2023-10-16T20:05:02.688808+0000","last_became_active":"2023-10-16T20:05:01.462044+0000","last_became_peered":"2023-10-16T20:05:01.462044+0000","last_unstale":"2023-10-16T20:05:02.688808+0000","last_undegraded":"2023-10-16T20:05:02.688808+0000","last_fullsized":"2023-10-16T20:05:02.688808+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:41:12.320369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.440492+0000","last_change":"2023-10-16T20:04:57.409823+0000","last_active":"2023-10-16T20:05:01.440492+0000","last_peered":"2023-10-16T20:05:01.440492+0000","last_clean":"2023-10-16T20:05:01.440492+0000","last_became_active":"2023-10-16T20:04:57.409457+0000","last_became_peered":"2023-10-16T20:04:57.409457+0000","last_unstale":"2023-10-16T20:05:01.440492+0000","last_undegraded":"2023-10-16T20:05:01.440492+0000","last_fullsized":"2023-10-16T20:05:01.440492+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:53:20.922170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.440306+0000","last_change":"2023-10-16T20:04:56.404854+0000","last_active":"2023-10-16T20:05:01.440306+0000","last_peered":"2023-10-16T20:05:01.440306+0000","last_clean":"2023-10-16T20:05:01.440306+0000","last_became_active":"2023-10-16T20:04:56.404457+0000","last_became_peered":"2023-10-16T20:04:56.404457+0000","last_unstale":"2023-10-16T20:05:01.440306+0000","last_undegraded":"2023-10-16T20:05:01.440306+0000","last_fullsized":"2023-10-16T20:05:01.440306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:41:00.246498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.686341+0000","last_change":"2023-10-16T20:05:01.769521+0000","last_active":"2023-10-16T20:05:02.686341+0000","last_peered":"2023-10-16T20:05:02.686341+0000","last_clean":"2023-10-16T20:05:02.686341+0000","last_became_active":"2023-10-16T20:05:01.768728+0000","last_became_peered":"2023-10-16T20:05:01.768728+0000","last_unstale":"2023-10-16T20:05:02.686341+0000","last_undegraded":"2023-10-16T20:05:02.686341+0000","last_fullsized":"2023-10-16T20:05:02.686341+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:01:55.141797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757851+0000","last_change":"2023-10-16T20:04:57.438646+0000","last_active":"2023-10-16T20:05:01.757851+0000","last_peered":"2023-10-16T20:05:01.757851+0000","last_clean":"2023-10-16T20:05:01.757851+0000","last_became_active":"2023-10-16T20:04:57.438375+0000","last_became_peered":"2023-10-16T20:04:57.438375+0000","last_unstale":"2023-10-16T20:05:01.757851+0000","last_undegraded":"2023-10-16T20:05:01.757851+0000","last_fullsized":"2023-10-16T20:05:01.757851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:14:29.143380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.437845+0000","last_change":"2023-10-16T20:04:56.407281+0000","last_active":"2023-10-16T20:05:01.437845+0000","last_peered":"2023-10-16T20:05:01.437845+0000","last_clean":"2023-10-16T20:05:01.437845+0000","last_became_active":"2023-10-16T20:04:56.406824+0000","last_became_peered":"2023-10-16T20:04:56.406824+0000","last_unstale":"2023-10-16T20:05:01.437845+0000","last_undegraded":"2023-10-16T20:05:01.437845+0000","last_fullsized":"2023-10-16T20:05:01.437845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:23:00.134558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.686002+0000","last_change":"2023-10-16T20:05:01.764372+0000","last_active":"2023-10-16T20:05:02.686002+0000","last_peered":"2023-10-16T20:05:02.686002+0000","last_clean":"2023-10-16T20:05:02.686002+0000","last_became_active":"2023-10-16T20:05:01.763918+0000","last_became_peered":"2023-10-16T20:05:01.763918+0000","last_unstale":"2023-10-16T20:05:02.686002+0000","last_undegraded":"2023-10-16T20:05:02.686002+0000","last_fullsized":"2023-10-16T20:05:02.686002+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:10:33.807776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.463201+0000","last_change":"2023-10-16T20:04:57.425920+0000","last_active":"2023-10-16T20:05:01.463201+0000","last_peered":"2023-10-16T20:05:01.463201+0000","last_clean":"2023-10-16T20:05:01.463201+0000","last_became_active":"2023-10-16T20:04:57.425614+0000","last_became_peered":"2023-10-16T20:04:57.425614+0000","last_unstale":"2023-10-16T20:05:01.463201+0000","last_undegraded":"2023-10-16T20:05:01.463201+0000","last_fullsized":"2023-10-16T20:05:01.463201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:28:14.992492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.464594+0000","last_change":"2023-10-16T20:04:56.407990+0000","last_active":"2023-10-16T20:05:01.464594+0000","last_peered":"2023-10-16T20:05:01.464594+0000","last_clean":"2023-10-16T20:05:01.464594+0000","last_became_active":"2023-10-16T20:04:56.407682+0000","last_became_peered":"2023-10-16T20:04:56.407682+0000","last_unstale":"2023-10-16T20:05:01.464594+0000","last_undegraded":"2023-10-16T20:05:01.464594+0000","last_fullsized":"2023-10-16T20:05:01.464594+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:25:27.337575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.685081+0000","last_change":"2023-10-16T20:05:01.761327+0000","last_active":"2023-10-16T20:05:02.685081+0000","last_peered":"2023-10-16T20:05:02.685081+0000","last_clean":"2023-10-16T20:05:02.685081+0000","last_became_active":"2023-10-16T20:05:01.760040+0000","last_became_peered":"2023-10-16T20:05:01.760040+0000","last_unstale":"2023-10-16T20:05:02.685081+0000","last_undegraded":"2023-10-16T20:05:02.685081+0000","last_fullsized":"2023-10-16T20:05:02.685081+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:33:23.371729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757841+0000","last_change":"2023-10-16T20:04:57.437602+0000","last_active":"2023-10-16T20:05:01.757841+0000","last_peered":"2023-10-16T20:05:01.757841+0000","last_clean":"2023-10-16T20:05:01.757841+0000","last_became_active":"2023-10-16T20:04:57.436601+0000","last_became_peered":"2023-10-16T20:04:57.436601+0000","last_unstale":"2023-10-16T20:05:01.757841+0000","last_undegraded":"2023-10-16T20:05:01.757841+0000","last_fullsized":"2023-10-16T20:05:01.757841+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:28:00.096002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'9","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:08.020486+0000","last_change":"2023-10-16T20:04:56.398743+0000","last_active":"2023-10-16T20:05:08.020486+0000","last_peered":"2023-10-16T20:05:08.020486+0000","last_clean":"2023-10-16T20:05:08.020486+0000","last_became_active":"2023-10-16T20:04:56.398410+0000","last_became_peered":"2023-10-16T20:04:56.398410+0000","last_unstale":"2023-10-16T20:05:08.020486+0000","last_undegraded":"2023-10-16T20:05:08.020486+0000","last_fullsized":"2023-10-16T20:05:08.020486+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:21:13.046934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.444394+0000","last_change":"2023-10-16T20:05:01.472414+0000","last_active":"2023-10-16T20:05:02.444394+0000","last_peered":"2023-10-16T20:05:02.444394+0000","last_clean":"2023-10-16T20:05:02.444394+0000","last_became_active":"2023-10-16T20:05:01.471943+0000","last_became_peered":"2023-10-16T20:05:01.471943+0000","last_unstale":"2023-10-16T20:05:02.444394+0000","last_undegraded":"2023-10-16T20:05:02.444394+0000","last_fullsized":"2023-10-16T20:05:02.444394+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:42:39.271438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.447164+0000","last_change":"2023-10-16T20:04:57.413204+0000","last_active":"2023-10-16T20:05:01.447164+0000","last_peered":"2023-10-16T20:05:01.447164+0000","last_clean":"2023-10-16T20:05:01.447164+0000","last_became_active":"2023-10-16T20:04:57.412836+0000","last_became_peered":"2023-10-16T20:04:57.412836+0000","last_unstale":"2023-10-16T20:05:01.447164+0000","last_undegraded":"2023-10-16T20:05:01.447164+0000","last_fullsized":"2023-10-16T20:05:01.447164+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:23:06.051449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471656+0000","last_change":"2023-10-16T20:04:56.414671+0000","last_active":"2023-10-16T20:05:01.471656+0000","last_peered":"2023-10-16T20:05:01.471656+0000","last_clean":"2023-10-16T20:05:01.471656+0000","last_became_active":"2023-10-16T20:04:56.414339+0000","last_became_peered":"2023-10-16T20:04:56.414339+0000","last_unstale":"2023-10-16T20:05:01.471656+0000","last_undegraded":"2023-10-16T20:05:01.471656+0000","last_fullsized":"2023-10-16T20:05:01.471656+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:21:13.046934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.683018+0000","last_change":"2023-10-16T20:05:01.482674+0000","last_active":"2023-10-16T20:05:02.683018+0000","last_peered":"2023-10-16T20:05:02.683018+0000","last_clean":"2023-10-16T20:05:02.683018+0000","last_became_active":"2023-10-16T20:05:01.482048+0000","last_became_peered":"2023-10-16T20:05:01.482048+0000","last_unstale":"2023-10-16T20:05:02.683018+0000","last_undegraded":"2023-10-16T20:05:02.683018+0000","last_fullsized":"2023-10-16T20:05:02.683018+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:21:13.443137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.465807+0000","last_change":"2023-10-16T20:04:57.430429+0000","last_active":"2023-10-16T20:05:01.465807+0000","last_peered":"2023-10-16T20:05:01.465807+0000","last_clean":"2023-10-16T20:05:01.465807+0000","last_became_active":"2023-10-16T20:04:57.429872+0000","last_became_peered":"2023-10-16T20:04:57.429872+0000","last_unstale":"2023-10-16T20:05:01.465807+0000","last_undegraded":"2023-10-16T20:05:01.465807+0000","last_fullsized":"2023-10-16T20:05:01.465807+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:47:33.537999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438017+0000","last_change":"2023-10-16T20:04:56.398061+0000","last_active":"2023-10-16T20:05:01.438017+0000","last_peered":"2023-10-16T20:05:01.438017+0000","last_clean":"2023-10-16T20:05:01.438017+0000","last_became_active":"2023-10-16T20:04:56.397712+0000","last_became_peered":"2023-10-16T20:04:56.397712+0000","last_unstale":"2023-10-16T20:05:01.438017+0000","last_undegraded":"2023-10-16T20:05:01.438017+0000","last_fullsized":"2023-10-16T20:05:01.438017+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:49:51.402606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.685658+0000","last_change":"2023-10-16T20:05:01.476635+0000","last_active":"2023-10-16T20:05:02.685658+0000","last_peered":"2023-10-16T20:05:02.685658+0000","last_clean":"2023-10-16T20:05:02.685658+0000","last_became_active":"2023-10-16T20:05:01.476053+0000","last_became_peered":"2023-10-16T20:05:01.476053+0000","last_unstale":"2023-10-16T20:05:02.685658+0000","last_undegraded":"2023-10-16T20:05:02.685658+0000","last_fullsized":"2023-10-16T20:05:02.685658+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:25:50.262916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.437931+0000","last_change":"2023-10-16T20:04:57.421476+0000","last_active":"2023-10-16T20:05:01.437931+0000","last_peered":"2023-10-16T20:05:01.437931+0000","last_clean":"2023-10-16T20:05:01.437931+0000","last_became_active":"2023-10-16T20:04:57.421206+0000","last_became_peered":"2023-10-16T20:04:57.421206+0000","last_unstale":"2023-10-16T20:05:01.437931+0000","last_undegraded":"2023-10-16T20:05:01.437931+0000","last_fullsized":"2023-10-16T20:05:01.437931+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:11:56.834052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.439019+0000","last_change":"2023-10-16T20:04:56.411232+0000","last_active":"2023-10-16T20:05:01.439019+0000","last_peered":"2023-10-16T20:05:01.439019+0000","last_clean":"2023-10-16T20:05:01.439019+0000","last_became_active":"2023-10-16T20:04:56.410793+0000","last_became_peered":"2023-10-16T20:04:56.410793+0000","last_unstale":"2023-10-16T20:05:01.439019+0000","last_undegraded":"2023-10-16T20:05:01.439019+0000","last_fullsized":"2023-10-16T20:05:01.439019+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:19:48.110438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.685647+0000","last_change":"2023-10-16T20:05:01.764968+0000","last_active":"2023-10-16T20:05:02.685647+0000","last_peered":"2023-10-16T20:05:02.685647+0000","last_clean":"2023-10-16T20:05:02.685647+0000","last_became_active":"2023-10-16T20:05:01.763830+0000","last_became_peered":"2023-10-16T20:05:01.763830+0000","last_unstale":"2023-10-16T20:05:02.685647+0000","last_undegraded":"2023-10-16T20:05:02.685647+0000","last_fullsized":"2023-10-16T20:05:02.685647+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:39:58.794103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.688809+0000","last_change":"2023-10-16T20:05:01.764846+0000","last_active":"2023-10-16T20:05:02.688809+0000","last_peered":"2023-10-16T20:05:02.688809+0000","last_clean":"2023-10-16T20:05:02.688809+0000","last_became_active":"2023-10-16T20:05:01.763756+0000","last_became_peered":"2023-10-16T20:05:01.763756+0000","last_unstale":"2023-10-16T20:05:02.688809+0000","last_undegraded":"2023-10-16T20:05:02.688809+0000","last_fullsized":"2023-10-16T20:05:02.688809+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:10:31.962156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471094+0000","last_change":"2023-10-16T20:04:57.428882+0000","last_active":"2023-10-16T20:05:01.471094+0000","last_peered":"2023-10-16T20:05:01.471094+0000","last_clean":"2023-10-16T20:05:01.471094+0000","last_became_active":"2023-10-16T20:04:57.428591+0000","last_became_peered":"2023-10-16T20:04:57.428591+0000","last_unstale":"2023-10-16T20:05:01.471094+0000","last_undegraded":"2023-10-16T20:05:01.471094+0000","last_fullsized":"2023-10-16T20:05:01.471094+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:33:36.402743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438343+0000","last_change":"2023-10-16T20:04:56.396769+0000","last_active":"2023-10-16T20:05:01.438343+0000","last_peered":"2023-10-16T20:05:01.438343+0000","last_clean":"2023-10-16T20:05:01.438343+0000","last_became_active":"2023-10-16T20:04:56.396273+0000","last_became_peered":"2023-10-16T20:04:56.396273+0000","last_unstale":"2023-10-16T20:05:01.438343+0000","last_undegraded":"2023-10-16T20:05:01.438343+0000","last_fullsized":"2023-10-16T20:05:01.438343+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:04:32.948232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.683891+0000","last_change":"2023-10-16T20:05:01.475065+0000","last_active":"2023-10-16T20:05:02.683891+0000","last_peered":"2023-10-16T20:05:02.683891+0000","last_clean":"2023-10-16T20:05:02.683891+0000","last_became_active":"2023-10-16T20:05:01.474571+0000","last_became_peered":"2023-10-16T20:05:01.474571+0000","last_unstale":"2023-10-16T20:05:02.683891+0000","last_undegraded":"2023-10-16T20:05:02.683891+0000","last_fullsized":"2023-10-16T20:05:02.683891+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:31:38.939408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442005+0000","last_change":"2023-10-16T20:04:57.423943+0000","last_active":"2023-10-16T20:05:01.442005+0000","last_peered":"2023-10-16T20:05:01.442005+0000","last_clean":"2023-10-16T20:05:01.442005+0000","last_became_active":"2023-10-16T20:04:57.423325+0000","last_became_peered":"2023-10-16T20:04:57.423325+0000","last_unstale":"2023-10-16T20:05:01.442005+0000","last_undegraded":"2023-10-16T20:05:01.442005+0000","last_fullsized":"2023-10-16T20:05:01.442005+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:56:28.993951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442942+0000","last_change":"2023-10-16T20:04:56.397549+0000","last_active":"2023-10-16T20:05:01.442942+0000","last_peered":"2023-10-16T20:05:01.442942+0000","last_clean":"2023-10-16T20:05:01.442942+0000","last_became_active":"2023-10-16T20:04:56.396648+0000","last_became_peered":"2023-10-16T20:04:56.396648+0000","last_unstale":"2023-10-16T20:05:01.442942+0000","last_undegraded":"2023-10-16T20:05:01.442942+0000","last_fullsized":"2023-10-16T20:05:01.442942+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:58:19.945263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438803+0000","last_change":"2023-10-16T20:04:56.407155+0000","last_active":"2023-10-16T20:05:01.438803+0000","last_peered":"2023-10-16T20:05:01.438803+0000","last_clean":"2023-10-16T20:05:01.438803+0000","last_became_active":"2023-10-16T20:04:56.406828+0000","last_became_peered":"2023-10-16T20:04:56.406828+0000","last_unstale":"2023-10-16T20:05:01.438803+0000","last_undegraded":"2023-10-16T20:05:01.438803+0000","last_fullsized":"2023-10-16T20:05:01.438803+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:23:05.044462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.464536+0000","last_change":"2023-10-16T20:04:57.416841+0000","last_active":"2023-10-16T20:05:01.464536+0000","last_peered":"2023-10-16T20:05:01.464536+0000","last_clean":"2023-10-16T20:05:01.464536+0000","last_became_active":"2023-10-16T20:04:57.416338+0000","last_became_peered":"2023-10-16T20:04:57.416338+0000","last_unstale":"2023-10-16T20:05:01.464536+0000","last_undegraded":"2023-10-16T20:05:01.464536+0000","last_fullsized":"2023-10-16T20:05:01.464536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:01.396735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.445821+0000","last_change":"2023-10-16T20:05:01.481039+0000","last_active":"2023-10-16T20:05:02.445821+0000","last_peered":"2023-10-16T20:05:02.445821+0000","last_clean":"2023-10-16T20:05:02.445821+0000","last_became_active":"2023-10-16T20:05:01.480626+0000","last_became_peered":"2023-10-16T20:05:01.480626+0000","last_unstale":"2023-10-16T20:05:02.445821+0000","last_undegraded":"2023-10-16T20:05:02.445821+0000","last_fullsized":"2023-10-16T20:05:02.445821+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:21:18.109168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.441938+0000","last_change":"2023-10-16T20:04:56.401545+0000","last_active":"2023-10-16T20:05:01.441938+0000","last_peered":"2023-10-16T20:05:01.441938+0000","last_clean":"2023-10-16T20:05:01.441938+0000","last_became_active":"2023-10-16T20:04:56.401276+0000","last_became_peered":"2023-10-16T20:04:56.401276+0000","last_unstale":"2023-10-16T20:05:01.441938+0000","last_undegraded":"2023-10-16T20:05:01.441938+0000","last_fullsized":"2023-10-16T20:05:01.441938+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:04:32.948232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.447134+0000","last_change":"2023-10-16T20:04:57.425600+0000","last_active":"2023-10-16T20:05:01.447134+0000","last_peered":"2023-10-16T20:05:01.447134+0000","last_clean":"2023-10-16T20:05:01.447134+0000","last_became_active":"2023-10-16T20:04:57.425213+0000","last_became_peered":"2023-10-16T20:04:57.425213+0000","last_unstale":"2023-10-16T20:05:01.447134+0000","last_undegraded":"2023-10-16T20:05:01.447134+0000","last_fullsized":"2023-10-16T20:05:01.447134+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:33:14.623281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.686142+0000","last_change":"2023-10-16T20:05:01.479226+0000","last_active":"2023-10-16T20:05:02.686142+0000","last_peered":"2023-10-16T20:05:02.686142+0000","last_clean":"2023-10-16T20:05:02.686142+0000","last_became_active":"2023-10-16T20:05:01.478808+0000","last_became_peered":"2023-10-16T20:05:01.478808+0000","last_unstale":"2023-10-16T20:05:02.686142+0000","last_undegraded":"2023-10-16T20:05:02.686142+0000","last_fullsized":"2023-10-16T20:05:02.686142+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:33:23.371729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.440406+0000","last_change":"2023-10-16T20:04:57.409772+0000","last_active":"2023-10-16T20:05:01.440406+0000","last_peered":"2023-10-16T20:05:01.440406+0000","last_clean":"2023-10-16T20:05:01.440406+0000","last_became_active":"2023-10-16T20:04:57.409449+0000","last_became_peered":"2023-10-16T20:04:57.409449+0000","last_unstale":"2023-10-16T20:05:01.440406+0000","last_undegraded":"2023-10-16T20:05:01.440406+0000","last_fullsized":"2023-10-16T20:05:01.440406+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:13:41.559934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.448190+0000","last_change":"2023-10-16T20:04:57.418433+0000","last_active":"2023-10-16T20:05:01.448190+0000","last_peered":"2023-10-16T20:05:01.448190+0000","last_clean":"2023-10-16T20:05:01.448190+0000","last_became_active":"2023-10-16T20:04:57.418140+0000","last_became_peered":"2023-10-16T20:04:57.418140+0000","last_unstale":"2023-10-16T20:05:01.448190+0000","last_undegraded":"2023-10-16T20:05:01.448190+0000","last_fullsized":"2023-10-16T20:05:01.448190+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:41:01.253485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.437811+0000","last_change":"2023-10-16T20:04:57.430273+0000","last_active":"2023-10-16T20:05:01.437811+0000","last_peered":"2023-10-16T20:05:01.437811+0000","last_clean":"2023-10-16T20:05:01.437811+0000","last_became_active":"2023-10-16T20:04:57.427616+0000","last_became_peered":"2023-10-16T20:04:57.427616+0000","last_unstale":"2023-10-16T20:05:01.437811+0000","last_undegraded":"2023-10-16T20:05:01.437811+0000","last_fullsized":"2023-10-16T20:05:01.437811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:22:12.692981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.464484+0000","last_change":"2023-10-16T20:04:57.434154+0000","last_active":"2023-10-16T20:05:01.464484+0000","last_peered":"2023-10-16T20:05:01.464484+0000","last_clean":"2023-10-16T20:05:01.464484+0000","last_became_active":"2023-10-16T20:04:57.433876+0000","last_became_peered":"2023-10-16T20:04:57.433876+0000","last_unstale":"2023-10-16T20:05:01.464484+0000","last_undegraded":"2023-10-16T20:05:01.464484+0000","last_fullsized":"2023-10-16T20:05:01.464484+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:45:41.399175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.443384+0000","last_change":"2023-10-16T20:04:57.437575+0000","last_active":"2023-10-16T20:05:01.443384+0000","last_peered":"2023-10-16T20:05:01.443384+0000","last_clean":"2023-10-16T20:05:01.443384+0000","last_became_active":"2023-10-16T20:04:57.437261+0000","last_became_peered":"2023-10-16T20:04:57.437261+0000","last_unstale":"2023-10-16T20:05:01.443384+0000","last_undegraded":"2023-10-16T20:05:01.443384+0000","last_fullsized":"2023-10-16T20:05:01.443384+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:58:33.862646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471548+0000","last_change":"2023-10-16T20:04:57.434048+0000","last_active":"2023-10-16T20:05:01.471548+0000","last_peered":"2023-10-16T20:05:01.471548+0000","last_clean":"2023-10-16T20:05:01.471548+0000","last_became_active":"2023-10-16T20:04:57.433771+0000","last_became_peered":"2023-10-16T20:04:57.433771+0000","last_unstale":"2023-10-16T20:05:01.471548+0000","last_undegraded":"2023-10-16T20:05:01.471548+0000","last_fullsized":"2023-10-16T20:05:01.471548+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:58:33.862646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.440121+0000","last_change":"2023-10-16T20:04:57.439787+0000","last_active":"2023-10-16T20:05:01.440121+0000","last_peered":"2023-10-16T20:05:01.440121+0000","last_clean":"2023-10-16T20:05:01.440121+0000","last_became_active":"2023-10-16T20:04:57.439468+0000","last_became_peered":"2023-10-16T20:04:57.439468+0000","last_unstale":"2023-10-16T20:05:01.440121+0000","last_undegraded":"2023-10-16T20:05:01.440121+0000","last_fullsized":"2023-10-16T20:05:01.440121+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:49:52.409593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442167+0000","last_change":"2023-10-16T20:04:57.421442+0000","last_active":"2023-10-16T20:05:01.442167+0000","last_peered":"2023-10-16T20:05:01.442167+0000","last_clean":"2023-10-16T20:05:01.442167+0000","last_became_active":"2023-10-16T20:04:57.421123+0000","last_became_peered":"2023-10-16T20:04:57.421123+0000","last_unstale":"2023-10-16T20:05:01.442167+0000","last_undegraded":"2023-10-16T20:05:01.442167+0000","last_fullsized":"2023-10-16T20:05:01.442167+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:17:24.205223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438259+0000","last_change":"2023-10-16T20:04:57.412843+0000","last_active":"2023-10-16T20:05:01.438259+0000","last_peered":"2023-10-16T20:05:01.438259+0000","last_clean":"2023-10-16T20:05:01.438259+0000","last_became_active":"2023-10-16T20:04:57.412559+0000","last_became_peered":"2023-10-16T20:04:57.412559+0000","last_unstale":"2023-10-16T20:05:01.438259+0000","last_undegraded":"2023-10-16T20:05:01.438259+0000","last_fullsized":"2023-10-16T20:05:01.438259+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:11:28.851632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442830+0000","last_change":"2023-10-16T20:04:57.421462+0000","last_active":"2023-10-16T20:05:01.442830+0000","last_peered":"2023-10-16T20:05:01.442830+0000","last_clean":"2023-10-16T20:05:01.442830+0000","last_became_active":"2023-10-16T20:04:57.421107+0000","last_became_peered":"2023-10-16T20:04:57.421107+0000","last_unstale":"2023-10-16T20:05:01.442830+0000","last_undegraded":"2023-10-16T20:05:01.442830+0000","last_fullsized":"2023-10-16T20:05:01.442830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:53:20.922170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.437759+0000","last_change":"2023-10-16T20:04:57.422120+0000","last_active":"2023-10-16T20:05:01.437759+0000","last_peered":"2023-10-16T20:05:01.437759+0000","last_clean":"2023-10-16T20:05:01.437759+0000","last_became_active":"2023-10-16T20:04:57.421682+0000","last_became_peered":"2023-10-16T20:04:57.421682+0000","last_unstale":"2023-10-16T20:05:01.437759+0000","last_undegraded":"2023-10-16T20:05:01.437759+0000","last_fullsized":"2023-10-16T20:05:01.437759+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:21:09.387890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.464439+0000","last_change":"2023-10-16T20:04:57.432633+0000","last_active":"2023-10-16T20:05:01.464439+0000","last_peered":"2023-10-16T20:05:01.464439+0000","last_clean":"2023-10-16T20:05:01.464439+0000","last_became_active":"2023-10-16T20:04:57.432324+0000","last_became_peered":"2023-10-16T20:04:57.432324+0000","last_unstale":"2023-10-16T20:05:01.464439+0000","last_undegraded":"2023-10-16T20:05:01.464439+0000","last_fullsized":"2023-10-16T20:05:01.464439+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:23:06.128119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438742+0000","last_change":"2023-10-16T20:04:57.410439+0000","last_active":"2023-10-16T20:05:01.438742+0000","last_peered":"2023-10-16T20:05:01.438742+0000","last_clean":"2023-10-16T20:05:01.438742+0000","last_became_active":"2023-10-16T20:04:57.410178+0000","last_became_peered":"2023-10-16T20:04:57.410178+0000","last_unstale":"2023-10-16T20:05:01.438742+0000","last_undegraded":"2023-10-16T20:05:01.438742+0000","last_fullsized":"2023-10-16T20:05:01.438742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:33:36.402743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.758054+0000","last_change":"2023-10-16T20:04:57.436579+0000","last_active":"2023-10-16T20:05:01.758054+0000","last_peered":"2023-10-16T20:05:01.758054+0000","last_clean":"2023-10-16T20:05:01.758054+0000","last_became_active":"2023-10-16T20:04:57.436033+0000","last_became_peered":"2023-10-16T20:04:57.436033+0000","last_unstale":"2023-10-16T20:05:01.758054+0000","last_undegraded":"2023-10-16T20:05:01.758054+0000","last_fullsized":"2023-10-16T20:05:01.758054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:57:27.959012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.448091+0000","last_change":"2023-10-16T20:04:57.415573+0000","last_active":"2023-10-16T20:05:01.448091+0000","last_peered":"2023-10-16T20:05:01.448091+0000","last_clean":"2023-10-16T20:05:01.448091+0000","last_became_active":"2023-10-16T20:04:57.414461+0000","last_became_peered":"2023-10-16T20:04:57.414461+0000","last_unstale":"2023-10-16T20:05:01.448091+0000","last_undegraded":"2023-10-16T20:05:01.448091+0000","last_fullsized":"2023-10-16T20:05:01.448091+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:23:06.051449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438402+0000","last_change":"2023-10-16T20:04:57.432424+0000","last_active":"2023-10-16T20:05:01.438402+0000","last_peered":"2023-10-16T20:05:01.438402+0000","last_clean":"2023-10-16T20:05:01.438402+0000","last_became_active":"2023-10-16T20:04:57.431858+0000","last_became_peered":"2023-10-16T20:04:57.431858+0000","last_unstale":"2023-10-16T20:05:01.438402+0000","last_undegraded":"2023-10-16T20:05:01.438402+0000","last_fullsized":"2023-10-16T20:05:01.438402+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:57:27.959012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438175+0000","last_change":"2023-10-16T20:04:57.417470+0000","last_active":"2023-10-16T20:05:01.438175+0000","last_peered":"2023-10-16T20:05:01.438175+0000","last_clean":"2023-10-16T20:05:01.438175+0000","last_became_active":"2023-10-16T20:04:57.417181+0000","last_became_peered":"2023-10-16T20:04:57.417181+0000","last_unstale":"2023-10-16T20:05:01.438175+0000","last_undegraded":"2023-10-16T20:05:01.438175+0000","last_fullsized":"2023-10-16T20:05:01.438175+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:27:55.972790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471102+0000","last_change":"2023-10-16T20:04:57.431307+0000","last_active":"2023-10-16T20:05:01.471102+0000","last_peered":"2023-10-16T20:05:01.471102+0000","last_clean":"2023-10-16T20:05:01.471102+0000","last_became_active":"2023-10-16T20:04:57.430943+0000","last_became_peered":"2023-10-16T20:04:57.430943+0000","last_unstale":"2023-10-16T20:05:01.471102+0000","last_undegraded":"2023-10-16T20:05:01.471102+0000","last_fullsized":"2023-10-16T20:05:01.471102+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:17:24.205223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.463204+0000","last_change":"2023-10-16T20:04:57.428233+0000","last_active":"2023-10-16T20:05:01.463204+0000","last_peered":"2023-10-16T20:05:01.463204+0000","last_clean":"2023-10-16T20:05:01.463204+0000","last_became_active":"2023-10-16T20:04:57.427952+0000","last_became_peered":"2023-10-16T20:04:57.427952+0000","last_unstale":"2023-10-16T20:05:01.463204+0000","last_undegraded":"2023-10-16T20:05:01.463204+0000","last_fullsized":"2023-10-16T20:05:01.463204+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:57:35.297293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.441740+0000","last_change":"2023-10-16T20:04:57.425988+0000","last_active":"2023-10-16T20:05:01.441740+0000","last_peered":"2023-10-16T20:05:01.441740+0000","last_clean":"2023-10-16T20:05:01.441740+0000","last_became_active":"2023-10-16T20:04:57.425383+0000","last_became_peered":"2023-10-16T20:04:57.425383+0000","last_unstale":"2023-10-16T20:05:01.441740+0000","last_undegraded":"2023-10-16T20:05:01.441740+0000","last_fullsized":"2023-10-16T20:05:01.441740+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:23:01.141545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.447072+0000","last_change":"2023-10-16T20:04:57.415017+0000","last_active":"2023-10-16T20:05:01.447072+0000","last_peered":"2023-10-16T20:05:01.447072+0000","last_clean":"2023-10-16T20:05:01.447072+0000","last_became_active":"2023-10-16T20:04:57.414629+0000","last_became_peered":"2023-10-16T20:04:57.414629+0000","last_unstale":"2023-10-16T20:05:01.447072+0000","last_undegraded":"2023-10-16T20:05:01.447072+0000","last_fullsized":"2023-10-16T20:05:01.447072+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:41:01.253485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.439630+0000","last_change":"2023-10-16T20:04:57.421682+0000","last_active":"2023-10-16T20:05:01.439630+0000","last_peered":"2023-10-16T20:05:01.439630+0000","last_clean":"2023-10-16T20:05:01.439630+0000","last_became_active":"2023-10-16T20:04:57.421340+0000","last_became_peered":"2023-10-16T20:04:57.421340+0000","last_unstale":"2023-10-16T20:05:01.439630+0000","last_undegraded":"2023-10-16T20:05:01.439630+0000","last_fullsized":"2023-10-16T20:05:01.439630+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:56:28.993951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.440035+0000","last_change":"2023-10-16T20:04:57.415019+0000","last_active":"2023-10-16T20:05:01.440035+0000","last_peered":"2023-10-16T20:05:01.440035+0000","last_clean":"2023-10-16T20:05:01.440035+0000","last_became_active":"2023-10-16T20:04:57.414700+0000","last_became_peered":"2023-10-16T20:04:57.414700+0000","last_unstale":"2023-10-16T20:05:01.440035+0000","last_undegraded":"2023-10-16T20:05:01.440035+0000","last_fullsized":"2023-10-16T20:05:01.440035+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:17:24.205223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.758287+0000","last_change":"2023-10-16T20:04:57.436170+0000","last_active":"2023-10-16T20:05:01.758287+0000","last_peered":"2023-10-16T20:05:01.758287+0000","last_clean":"2023-10-16T20:05:01.758287+0000","last_became_active":"2023-10-16T20:04:57.435430+0000","last_became_peered":"2023-10-16T20:04:57.435430+0000","last_unstale":"2023-10-16T20:05:01.758287+0000","last_undegraded":"2023-10-16T20:05:01.758287+0000","last_fullsized":"2023-10-16T20:05:01.758287+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:39:54.738856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.441360+0000","last_change":"2023-10-16T20:04:57.429885+0000","last_active":"2023-10-16T20:05:01.441360+0000","last_peered":"2023-10-16T20:05:01.441360+0000","last_clean":"2023-10-16T20:05:01.441360+0000","last_became_active":"2023-10-16T20:04:57.429595+0000","last_became_peered":"2023-10-16T20:04:57.429595+0000","last_unstale":"2023-10-16T20:05:01.441360+0000","last_undegraded":"2023-10-16T20:05:01.441360+0000","last_fullsized":"2023-10-16T20:05:01.441360+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:49:52.409593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757637+0000","last_change":"2023-10-16T20:04:57.434436+0000","last_active":"2023-10-16T20:05:01.757637+0000","last_peered":"2023-10-16T20:05:01.757637+0000","last_clean":"2023-10-16T20:05:01.757637+0000","last_became_active":"2023-10-16T20:04:57.433525+0000","last_became_peered":"2023-10-16T20:04:57.433525+0000","last_unstale":"2023-10-16T20:05:01.757637+0000","last_undegraded":"2023-10-16T20:05:01.757637+0000","last_fullsized":"2023-10-16T20:05:01.757637+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:22:12.692981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.437681+0000","last_change":"2023-10-16T20:04:57.421933+0000","last_active":"2023-10-16T20:05:01.437681+0000","last_peered":"2023-10-16T20:05:01.437681+0000","last_clean":"2023-10-16T20:05:01.437681+0000","last_became_active":"2023-10-16T20:04:57.421342+0000","last_became_peered":"2023-10-16T20:04:57.421342+0000","last_unstale":"2023-10-16T20:05:01.437681+0000","last_undegraded":"2023-10-16T20:05:01.437681+0000","last_fullsized":"2023-10-16T20:05:01.437681+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:54:09.036700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757757+0000","last_change":"2023-10-16T20:04:57.434235+0000","last_active":"2023-10-16T20:05:01.757757+0000","last_peered":"2023-10-16T20:05:01.757757+0000","last_clean":"2023-10-16T20:05:01.757757+0000","last_became_active":"2023-10-16T20:04:57.433720+0000","last_became_peered":"2023-10-16T20:04:57.433720+0000","last_unstale":"2023-10-16T20:05:01.757757+0000","last_undegraded":"2023-10-16T20:05:01.757757+0000","last_fullsized":"2023-10-16T20:05:01.757757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:11:56.834052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438139+0000","last_change":"2023-10-16T20:04:57.431571+0000","last_active":"2023-10-16T20:05:01.438139+0000","last_peered":"2023-10-16T20:05:01.438139+0000","last_clean":"2023-10-16T20:05:01.438139+0000","last_became_active":"2023-10-16T20:04:57.429514+0000","last_became_peered":"2023-10-16T20:04:57.429514+0000","last_unstale":"2023-10-16T20:05:01.438139+0000","last_undegraded":"2023-10-16T20:05:01.438139+0000","last_fullsized":"2023-10-16T20:05:01.438139+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:31:34.884161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.464508+0000","last_change":"2023-10-16T20:04:57.412857+0000","last_active":"2023-10-16T20:05:01.464508+0000","last_peered":"2023-10-16T20:05:01.464508+0000","last_clean":"2023-10-16T20:05:01.464508+0000","last_became_active":"2023-10-16T20:04:57.412587+0000","last_became_peered":"2023-10-16T20:04:57.412587+0000","last_unstale":"2023-10-16T20:05:01.464508+0000","last_undegraded":"2023-10-16T20:05:01.464508+0000","last_fullsized":"2023-10-16T20:05:01.464508+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:42:47.128122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471650+0000","last_change":"2023-10-16T20:04:57.423730+0000","last_active":"2023-10-16T20:05:01.471650+0000","last_peered":"2023-10-16T20:05:01.471650+0000","last_clean":"2023-10-16T20:05:01.471650+0000","last_became_active":"2023-10-16T20:04:57.423300+0000","last_became_peered":"2023-10-16T20:04:57.423300+0000","last_unstale":"2023-10-16T20:05:01.471650+0000","last_undegraded":"2023-10-16T20:05:01.471650+0000","last_fullsized":"2023-10-16T20:05:01.471650+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:27:55.972790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.758256+0000","last_change":"2023-10-16T20:04:57.442158+0000","last_active":"2023-10-16T20:05:01.758256+0000","last_peered":"2023-10-16T20:05:01.758256+0000","last_clean":"2023-10-16T20:05:01.758256+0000","last_became_active":"2023-10-16T20:04:57.438707+0000","last_became_peered":"2023-10-16T20:04:57.438707+0000","last_unstale":"2023-10-16T20:05:01.758256+0000","last_undegraded":"2023-10-16T20:05:01.758256+0000","last_fullsized":"2023-10-16T20:05:01.758256+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:51:08.154913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":880640,"data_stored":701023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]}]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":294912,"data_stored":113914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]}]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":110163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":294912,"data_stored":110199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]}]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":294912,"data_stored":113914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]}]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":110073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":290816,"data_stored":110233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":888832,"data_stored":701079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T20:05:16.642 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T20:05:16.642 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-16T20:05:16.643 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-16T20:05:16.643 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-16T20:05:16.643 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-16T20:05:16.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.814+0000 7f1271777640 1 Processor -- start 2023-10-16T20:05:16.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.814+0000 7f1271777640 1 -- start start 2023-10-16T20:05:16.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.814+0000 7f1271777640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f126c0cde80 0x7f126c0d3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:16.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.814+0000 7f1271777640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f126c0cf230 0x7f126c0cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:16.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.814+0000 7f1271777640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f126c0d0c40 con 0x7f126c15c9a0 2023-10-16T20:05:16.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.814+0000 7f1271777640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f126c0d37b0 con 0x7f126c0cde80 2023-10-16T20:05:16.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.814+0000 7f1271777640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f126c0ce7e0 con 0x7f126c0cf230 2023-10-16T20:05:16.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.814+0000 7f126b7fe640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f126c0cde80 0x7f126c0d3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:16.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.814+0000 7f126b7fe640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f126c0cde80 0x7f126c0d3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40196/0 (socket says 172.21.15.88:40196) 2023-10-16T20:05:16.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.814+0000 7f126b7fe640 1 -- 172.21.15.88:0/4248181627 learned_addr learned my addr 172.21.15.88:0/4248181627 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:16.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.814+0000 7f1270f76640 1 --2- 172.21.15.88:0/4248181627 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f126c0cf230 0x7f126c0cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:16.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.814+0000 7f126affd640 1 -- 172.21.15.88:0/4248181627 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1232136174 0 0) 0x7f126c0d0c40 con 0x7f126c15c9a0 2023-10-16T20:05:16.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.814+0000 7f126b7fe640 1 -- 172.21.15.88:0/4248181627 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f126c0cf230 msgr2=0x7f126c0cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:16.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.814+0000 7f126b7fe640 1 --2- 172.21.15.88:0/4248181627 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f126c0cf230 0x7f126c0cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:16.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.814+0000 7f126b7fe640 1 -- 172.21.15.88:0/4248181627 >> v1:172.21.15.88:6789/0 conn(0x7f126c15c9a0 legacy=0x7f126c0d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:16.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.814+0000 7f126b7fe640 1 -- 172.21.15.88:0/4248181627 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f126c0ce2d0 con 0x7f126c0cde80 2023-10-16T20:05:16.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.818+0000 7f126b7fe640 1 --2- 172.21.15.88:0/4248181627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f126c0cde80 0x7f126c0d3270 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1260009870 tx=0x7f126001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc1a12ba373317b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:16.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.818+0000 7f126affd640 1 -- 172.21.15.88:0/4248181627 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12600ab650 con 0x7f126c0cde80 2023-10-16T20:05:16.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.818+0000 7f126affd640 1 -- 172.21.15.88:0/4248181627 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12600ab7b0 con 0x7f126c0cde80 2023-10-16T20:05:16.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.818+0000 7f126affd640 1 -- 172.21.15.88:0/4248181627 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12600abb40 con 0x7f126c0cde80 2023-10-16T20:05:16.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.818+0000 7f1271777640 1 -- 172.21.15.88:0/4248181627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f126c0cde80 msgr2=0x7f126c0d3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:16.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.818+0000 7f1271777640 1 --2- 172.21.15.88:0/4248181627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f126c0cde80 0x7f126c0d3270 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1260009870 tx=0x7f126001c540 comp rx=0 tx=0).stop 2023-10-16T20:05:16.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.818+0000 7f1271777640 1 -- 172.21.15.88:0/4248181627 shutdown_connections 2023-10-16T20:05:16.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.818+0000 7f1271777640 1 --2- 172.21.15.88:0/4248181627 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f126c0cf230 0x7f126c0cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:16.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.818+0000 7f1271777640 1 --2- 172.21.15.88:0/4248181627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f126c0cde80 0x7f126c0d3270 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:16.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.818+0000 7f1271777640 1 -- 172.21.15.88:0/4248181627 >> 172.21.15.88:0/4248181627 conn(0x7f126c09bc30 msgr2=0x7f126c0d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:16.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.818+0000 7f1271777640 1 -- 172.21.15.88:0/4248181627 shutdown_connections 2023-10-16T20:05:16.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.818+0000 7f1271777640 1 -- 172.21.15.88:0/4248181627 wait complete. 2023-10-16T20:05:16.822 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.818+0000 7f1271777640 1 Processor -- start 2023-10-16T20:05:16.823 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.818+0000 7f1271777640 1 -- start start 2023-10-16T20:05:16.823 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.822+0000 7f1271777640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f126c0cde80 0x7f126c154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:16.823 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.822+0000 7f1271777640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f126c0cf230 0x7f126c0b3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:16.823 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.822+0000 7f1271777640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f126c15c9a0 0x7f126c0b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:16.823 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.822+0000 7f1271777640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126c15ded0 con 0x7f126c0cf230 2023-10-16T20:05:16.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.822+0000 7f1271777640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126c15e1b0 con 0x7f126c15c9a0 2023-10-16T20:05:16.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.822+0000 7f1271777640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f126c15e490 con 0x7f126c0cde80 2023-10-16T20:05:16.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.822+0000 7f126b7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f126c0cf230 0x7f126c0b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:16.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.822+0000 7f126b7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f126c0cf230 0x7f126c0b3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40210/0 (socket says 172.21.15.88:40210) 2023-10-16T20:05:16.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.822+0000 7f126b7fe640 1 -- 172.21.15.88:0/2077718118 learned_addr learned my addr 172.21.15.88:0/2077718118 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:16.825 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.822+0000 7f126bfff640 1 --2- 172.21.15.88:0/2077718118 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f126c0cde80 0x7f126c154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:16.825 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.822+0000 7f1270f76640 1 --2- 172.21.15.88:0/2077718118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f126c15c9a0 0x7f126c0b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:16.825 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.822+0000 7f126b7fe640 1 -- 172.21.15.88:0/2077718118 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f126c0cde80 msgr2=0x7f126c154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:16.825 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.822+0000 7f126b7fe640 1 --2- 172.21.15.88:0/2077718118 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f126c0cde80 0x7f126c154d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:16.825 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.822+0000 7f126b7fe640 1 -- 172.21.15.88:0/2077718118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f126c15c9a0 msgr2=0x7f126c0b35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:16.826 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.822+0000 7f126b7fe640 1 --2- 172.21.15.88:0/2077718118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f126c15c9a0 0x7f126c0b35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:16.826 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.822+0000 7f126b7fe640 1 -- 172.21.15.88:0/2077718118 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12600094d0 con 0x7f126c0cf230 2023-10-16T20:05:16.826 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.822+0000 7f126b7fe640 1 --2- 172.21.15.88:0/2077718118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f126c0cf230 0x7f126c0b3080 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f126001a170 tx=0x7f126001ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:16.827 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.822+0000 7f1268ff9640 1 -- 172.21.15.88:0/2077718118 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12600aabf0 con 0x7f126c0cf230 2023-10-16T20:05:16.828 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.822+0000 7f1268ff9640 1 -- 172.21.15.88:0/2077718118 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12600aad50 con 0x7f126c0cf230 2023-10-16T20:05:16.828 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.822+0000 7f1268ff9640 1 -- 172.21.15.88:0/2077718118 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12600be630 con 0x7f126c0cf230 2023-10-16T20:05:16.828 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.822+0000 7f1271777640 1 -- 172.21.15.88:0/2077718118 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f126c15e770 con 0x7f126c0cf230 2023-10-16T20:05:16.828 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.822+0000 7f1271777640 1 -- 172.21.15.88:0/2077718118 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f126c155240 con 0x7f126c0cf230 2023-10-16T20:05:16.828 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.822+0000 7f1271777640 1 -- 172.21.15.88:0/2077718118 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f126c0674f0 con 0x7f126c0cf230 2023-10-16T20:05:16.829 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.826+0000 7f1268ff9640 1 -- 172.21.15.88:0/2077718118 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f12600b43f0 con 0x7f126c0cf230 2023-10-16T20:05:16.829 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.826+0000 7f1268ff9640 1 --2- 172.21.15.88:0/2077718118 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f12580664a0 0x7f1258068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:16.829 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.826+0000 7f1268ff9640 1 -- 172.21.15.88:0/2077718118 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f126011edd0 con 0x7f126c0cf230 2023-10-16T20:05:16.829 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.826+0000 7f126bfff640 1 --2- 172.21.15.88:0/2077718118 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f12580664a0 0x7f1258068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:16.829 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.826+0000 7f126bfff640 1 --2- 172.21.15.88:0/2077718118 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f12580664a0 0x7f1258068960 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f125c007fd0 tx=0x7f125c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:16.830 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:16.830+0000 7f1268ff9640 1 -- 172.21.15.88:0/2077718118 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f12600ebef0 con 0x7f126c0cf230 2023-10-16T20:05:17.019 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.018+0000 7f1271777640 1 -- 172.21.15.88:0/2077718118 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f126c15ec20 con 0x7f126c0cf230 2023-10-16T20:05:17.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.022+0000 7f1268ff9640 1 -- 172.21.15.88:0/2077718118 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+164430 (secure 0 0 0) 0x7f12600ef510 con 0x7f126c0cf230 2023-10-16T20:05:17.025 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:17.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.022+0000 7f1271777640 1 -- 172.21.15.88:0/2077718118 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f12580664a0 msgr2=0x7f1258068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:17.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.022+0000 7f1271777640 1 --2- 172.21.15.88:0/2077718118 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f12580664a0 0x7f1258068960 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f125c007fd0 tx=0x7f125c009040 comp rx=0 tx=0).stop 2023-10-16T20:05:17.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.022+0000 7f1271777640 1 -- 172.21.15.88:0/2077718118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f126c0cf230 msgr2=0x7f126c0b3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:17.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.022+0000 7f1271777640 1 --2- 172.21.15.88:0/2077718118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f126c0cf230 0x7f126c0b3080 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f126001a170 tx=0x7f126001ad80 comp rx=0 tx=0).stop 2023-10-16T20:05:17.029 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.026+0000 7f1271777640 1 -- 172.21.15.88:0/2077718118 shutdown_connections 2023-10-16T20:05:17.029 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.026+0000 7f1271777640 1 --2- 172.21.15.88:0/2077718118 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f12580664a0 0x7f1258068960 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.030 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.026+0000 7f1271777640 1 --2- 172.21.15.88:0/2077718118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f126c15c9a0 0x7f126c0b35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.030 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.026+0000 7f1271777640 1 --2- 172.21.15.88:0/2077718118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f126c0cf230 0x7f126c0b3080 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.030 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.026+0000 7f1271777640 1 --2- 172.21.15.88:0/2077718118 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f126c0cde80 0x7f126c154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.026+0000 7f1271777640 1 -- 172.21.15.88:0/2077718118 >> 172.21.15.88:0/2077718118 conn(0x7f126c09bc30 msgr2=0x7f126c0d10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:17.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.026+0000 7f1271777640 1 -- 172.21.15.88:0/2077718118 shutdown_connections 2023-10-16T20:05:17.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.026+0000 7f1271777640 1 -- 172.21.15.88:0/2077718118 wait complete. 2023-10-16T20:05:17.044 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":5,"active_gid":4110,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4078483333},{"type":"v1","addr":"172.21.15.88:6833","nonce":4078483333}]},"active_addr":"172.21.15.88:6833/4078483333","active_change":"2023-10-16T20:04:36.429071+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4107,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.88:0","nonce":1829167853}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.88:0","nonce":3783802115}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.88:0","nonce":3382788944}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.88:0","nonce":1411423973}]}]} 2023-10-16T20:05:17.048 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-16T20:05:17.049 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-16T20:05:17.049 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:05:17.186 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.186+0000 7f2aec4c9640 1 Processor -- start 2023-10-16T20:05:17.186 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.186+0000 7f2aec4c9640 1 -- start start 2023-10-16T20:05:17.187 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.186+0000 7f2aec4c9640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2ae40b2a80 0x7f2ae40cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:17.187 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.186+0000 7f2aec4c9640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ae40cba30 con 0x7f2ae415aa60 2023-10-16T20:05:17.188 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.186+0000 7f2aec4c9640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ae40cbd10 con 0x7f2ae415ae40 2023-10-16T20:05:17.188 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.186+0000 7f2aec4c9640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ae40ce9c0 con 0x7f2ae40b2a80 2023-10-16T20:05:17.188 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.186+0000 7f2aea265640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f2ae415aa60 0x7f2ae40cb150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:39166/0 (socket says 172.21.15.88:39166) 2023-10-16T20:05:17.188 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.186+0000 7f2aea265640 1 -- 172.21.15.88:0/1397670517 learned_addr learned my addr 172.21.15.88:0/1397670517 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:17.190 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.186+0000 7f2aeaa66640 1 --2- 172.21.15.88:0/1397670517 >> v2:172.21.15.94:3300/0 conn(0x7f2ae40b2a80 0x7f2ae40cf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:17.190 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.186+0000 7f2ae9263640 1 -- 172.21.15.88:0/1397670517 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1509800970 0 0) 0x7f2ae40cba30 con 0x7f2ae415aa60 2023-10-16T20:05:17.191 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.186+0000 7f2ae9263640 1 -- 172.21.15.88:0/1397670517 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ad0003530 con 0x7f2ae415aa60 2023-10-16T20:05:17.191 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.186+0000 7f2ae9263640 1 -- 172.21.15.88:0/1397670517 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2101026085 0 0) 0x7f2ad0003530 con 0x7f2ae415aa60 2023-10-16T20:05:17.191 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.186+0000 7f2ae9263640 1 -- 172.21.15.88:0/1397670517 >> v2:172.21.15.94:3300/0 conn(0x7f2ae40b2a80 msgr2=0x7f2ae40cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:17.191 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.186+0000 7f2ae9263640 1 --2- 172.21.15.88:0/1397670517 >> v2:172.21.15.94:3300/0 conn(0x7f2ae40b2a80 0x7f2ae40cf4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.191 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.186+0000 7f2ae9263640 1 -- 172.21.15.88:0/1397670517 >> v1:172.21.15.94:6789/0 conn(0x7f2ae415ae40 legacy=0x7f2ae40b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:17.192 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.186+0000 7f2ae9263640 1 -- 172.21.15.88:0/1397670517 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae40ceca0 con 0x7f2ae415aa60 2023-10-16T20:05:17.192 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.186+0000 7f2ae9263640 1 -- 172.21.15.88:0/1397670517 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f2ae0001680 con 0x7f2ae415aa60 2023-10-16T20:05:17.192 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.186+0000 7f2ae9263640 1 -- 172.21.15.88:0/1397670517 >> v1:172.21.15.88:6789/0 conn(0x7f2ae415aa60 legacy=0x7f2ae40cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:17.192 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.186+0000 7f2ae9263640 1 --2- 172.21.15.88:0/1397670517 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2ad0003980 0x7f2ad0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:17.192 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.186+0000 7f2ae9263640 1 --2- 172.21.15.88:0/1397670517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ad00047d0 0x7f2ad0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:17.193 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.186+0000 7f2ae9263640 1 --2- 172.21.15.88:0/1397670517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ad0005190 0x7f2ad000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:17.193 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.186+0000 7f2ae9263640 1 -- 172.21.15.88:0/1397670517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae40ce9c0 con 0x7f2ad00047d0 2023-10-16T20:05:17.193 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.186+0000 7f2ae9263640 1 -- 172.21.15.88:0/1397670517 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad000c030 con 0x7f2ad0005190 2023-10-16T20:05:17.193 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.186+0000 7f2ae9263640 1 -- 172.21.15.88:0/1397670517 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad000c1f0 con 0x7f2ad0003980 2023-10-16T20:05:17.193 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.186+0000 7f2ae9a64640 1 --2- 172.21.15.88:0/1397670517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ad00047d0 0x7f2ad0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:17.193 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.186+0000 7f2aeaa66640 1 --2- 172.21.15.88:0/1397670517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ad0005190 0x7f2ad000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:17.194 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.186+0000 7f2aea265640 1 --2- 172.21.15.88:0/1397670517 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2ad0003980 0x7f2ad0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:17.194 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.190+0000 7f2aeaa66640 1 -- 172.21.15.88:0/1397670517 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2ad0003980 msgr2=0x7f2ad0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:17.194 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.190+0000 7f2aeaa66640 1 --2- 172.21.15.88:0/1397670517 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2ad0003980 0x7f2ad0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.194 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.190+0000 7f2aeaa66640 1 -- 172.21.15.88:0/1397670517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ad00047d0 msgr2=0x7f2ad0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:17.194 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.190+0000 7f2aeaa66640 1 --2- 172.21.15.88:0/1397670517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ad00047d0 0x7f2ad0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.195 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.190+0000 7f2aeaa66640 1 -- 172.21.15.88:0/1397670517 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad0003530 con 0x7f2ad0005190 2023-10-16T20:05:17.195 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.190+0000 7f2ae9a64640 1 --2- 172.21.15.88:0/1397670517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ad00047d0 0x7f2ad0004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:05:17.198 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.190+0000 7f2aeaa66640 1 -- 172.21.15.88:0/1397670517 reap_dead start 2023-10-16T20:05:17.198 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.190+0000 7f2aeaa66640 1 --2- 172.21.15.88:0/1397670517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ad0005190 0x7f2ad000b990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2ae40b37f0 tx=0x7f2adc01cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe99c423c7d4f5b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:17.198 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.190+0000 7f2ae9263640 1 -- 172.21.15.88:0/1397670517 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc003ba0 con 0x7f2ad0005190 2023-10-16T20:05:17.198 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.190+0000 7f2ae9263640 1 -- 172.21.15.88:0/1397670517 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2adc00f040 con 0x7f2ad0005190 2023-10-16T20:05:17.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.190+0000 7f2ae9263640 1 -- 172.21.15.88:0/1397670517 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc0a5950 con 0x7f2ad0005190 2023-10-16T20:05:17.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.194+0000 7f2aec4c9640 1 -- 172.21.15.88:0/1397670517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ad0005190 msgr2=0x7f2ad000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:17.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.194+0000 7f2aec4c9640 1 --2- 172.21.15.88:0/1397670517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ad0005190 0x7f2ad000b990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2ae40b37f0 tx=0x7f2adc01cc40 comp rx=0 tx=0).stop 2023-10-16T20:05:17.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.194+0000 7f2aec4c9640 1 -- 172.21.15.88:0/1397670517 shutdown_connections 2023-10-16T20:05:17.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.194+0000 7f2aec4c9640 1 --2- 172.21.15.88:0/1397670517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ad0005190 0x7f2ad000b990 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.194+0000 7f2aec4c9640 1 -- 172.21.15.88:0/1397670517 >> 172.21.15.88:0/1397670517 conn(0x7f2ae40cd370 msgr2=0x7f2ae40cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:17.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.194+0000 7f2aec4c9640 1 -- 172.21.15.88:0/1397670517 shutdown_connections 2023-10-16T20:05:17.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.194+0000 7f2aec4c9640 1 -- 172.21.15.88:0/1397670517 wait complete. 2023-10-16T20:05:17.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.194+0000 7f2aec4c9640 1 Processor -- start 2023-10-16T20:05:17.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.194+0000 7f2aec4c9640 1 -- start start 2023-10-16T20:05:17.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.194+0000 7f2aec4c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ad0005190 0x7f2ae4151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:17.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.194+0000 7f2aec4c9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2ae4151710 0x7f2ae40ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:17.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.194+0000 7f2aec4c9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ae40c88b0 0x7f2ae40b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:17.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.194+0000 7f2aec4c9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae4151b90 con 0x7f2ae40c88b0 2023-10-16T20:05:17.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.194+0000 7f2aec4c9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae40ca810 con 0x7f2ad0005190 2023-10-16T20:05:17.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.194+0000 7f2aec4c9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae40c7af0 con 0x7f2ae4151710 2023-10-16T20:05:17.202 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.194+0000 7f2aeaa66640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ae40c88b0 0x7f2ae40b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:17.202 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.194+0000 7f2aeaa66640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ae40c88b0 0x7f2ae40b1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40228/0 (socket says 172.21.15.88:40228) 2023-10-16T20:05:17.202 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.194+0000 7f2aeaa66640 1 -- 172.21.15.88:0/1593023039 learned_addr learned my addr 172.21.15.88:0/1593023039 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:17.202 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.194+0000 7f2ae9a64640 1 --2- 172.21.15.88:0/1593023039 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2ae4151710 0x7f2ae40ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:17.202 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.194+0000 7f2aea265640 1 --2- 172.21.15.88:0/1593023039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ad0005190 0x7f2ae4151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:17.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.194+0000 7f2aeaa66640 1 -- 172.21.15.88:0/1593023039 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2ae4151710 msgr2=0x7f2ae40ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:17.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.194+0000 7f2aeaa66640 1 --2- 172.21.15.88:0/1593023039 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2ae4151710 0x7f2ae40ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.194+0000 7f2aeaa66640 1 -- 172.21.15.88:0/1593023039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ad0005190 msgr2=0x7f2ae4151d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:17.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.194+0000 7f2aeaa66640 1 --2- 172.21.15.88:0/1593023039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ad0005190 0x7f2ae4151d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.194+0000 7f2aeaa66640 1 -- 172.21.15.88:0/1593023039 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae40c7dd0 con 0x7f2ae40c88b0 2023-10-16T20:05:17.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.198+0000 7f2aeaa66640 1 --2- 172.21.15.88:0/1593023039 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ae40c88b0 0x7f2ae40b1f10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2adc01d4e0 tx=0x7f2adc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:17.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.198+0000 7f2adaffd640 1 -- 172.21.15.88:0/1593023039 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc0a5d60 con 0x7f2ae40c88b0 2023-10-16T20:05:17.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.198+0000 7f2adaffd640 1 -- 172.21.15.88:0/1593023039 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2adc003f80 con 0x7f2ae40c88b0 2023-10-16T20:05:17.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.198+0000 7f2aec4c9640 1 -- 172.21.15.88:0/1593023039 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ae40c80b0 con 0x7f2ae40c88b0 2023-10-16T20:05:17.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.198+0000 7f2adaffd640 1 -- 172.21.15.88:0/1593023039 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc004330 con 0x7f2ae40c88b0 2023-10-16T20:05:17.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.198+0000 7f2aec4c9640 1 -- 172.21.15.88:0/1593023039 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ae415b490 con 0x7f2ae40c88b0 2023-10-16T20:05:17.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.198+0000 7f2aec4c9640 1 -- 172.21.15.88:0/1593023039 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ae40c85d0 con 0x7f2ae40c88b0 2023-10-16T20:05:17.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.198+0000 7f2adaffd640 1 -- 172.21.15.88:0/1593023039 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2adc004490 con 0x7f2ae40c88b0 2023-10-16T20:05:17.206 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.198+0000 7f2adaffd640 1 --2- 172.21.15.88:0/1593023039 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2ac80664d0 0x7f2ac8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:17.206 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.198+0000 7f2adaffd640 1 -- 172.21.15.88:0/1593023039 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2adc11ec90 con 0x7f2ae40c88b0 2023-10-16T20:05:17.206 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.198+0000 7f2aea265640 1 --2- 172.21.15.88:0/1593023039 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2ac80664d0 0x7f2ac8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:17.206 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.202+0000 7f2aea265640 1 --2- 172.21.15.88:0/1593023039 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2ac80664d0 0x7f2ac8068990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0000c10 tx=0x7f2ae0001c10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:17.206 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.202+0000 7f2adaffd640 1 -- 172.21.15.88:0/1593023039 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2ae40c85d0 con 0x7f2ae40c88b0 2023-10-16T20:05:17.356 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.354+0000 7f2aec4c9640 1 -- 172.21.15.88:0/1593023039 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ae40c85d0 con 0x7f2ae40c88b0 2023-10-16T20:05:17.357 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.358+0000 7f2adaffd640 1 -- 172.21.15.88:0/1593023039 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18274 (secure 0 0 0) 0x7f2adc0ebf60 con 0x7f2ae40c88b0 2023-10-16T20:05:17.358 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:17.358 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:05:01.835630+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T20:04:54.442487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T20:04:55.766641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T20:04:59.475830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:05:17.359 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.358+0000 7f2aec4c9640 1 -- 172.21.15.88:0/1593023039 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2ac80664d0 msgr2=0x7f2ac8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:17.359 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.358+0000 7f2aec4c9640 1 --2- 172.21.15.88:0/1593023039 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2ac80664d0 0x7f2ac8068990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0000c10 tx=0x7f2ae0001c10 comp rx=0 tx=0).stop 2023-10-16T20:05:17.359 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.358+0000 7f2aec4c9640 1 -- 172.21.15.88:0/1593023039 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ae40c88b0 msgr2=0x7f2ae40b1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:17.360 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.358+0000 7f2aec4c9640 1 --2- 172.21.15.88:0/1593023039 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ae40c88b0 0x7f2ae40b1f10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2adc01d4e0 tx=0x7f2adc009750 comp rx=0 tx=0).stop 2023-10-16T20:05:17.360 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.358+0000 7f2aec4c9640 1 -- 172.21.15.88:0/1593023039 shutdown_connections 2023-10-16T20:05:17.360 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.358+0000 7f2aec4c9640 1 --2- 172.21.15.88:0/1593023039 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2ac80664d0 0x7f2ac8068990 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.360 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.358+0000 7f2aec4c9640 1 --2- 172.21.15.88:0/1593023039 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2ae4151710 0x7f2ae40ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.361 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.358+0000 7f2aec4c9640 1 --2- 172.21.15.88:0/1593023039 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ae40c88b0 0x7f2ae40b1f10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.361 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.358+0000 7f2aec4c9640 1 --2- 172.21.15.88:0/1593023039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ad0005190 0x7f2ae4151d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.361 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.358+0000 7f2aec4c9640 1 -- 172.21.15.88:0/1593023039 >> 172.21.15.88:0/1593023039 conn(0x7f2ae40cd370 msgr2=0x7f2ae40aefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:17.361 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.358+0000 7f2aec4c9640 1 -- 172.21.15.88:0/1593023039 shutdown_connections 2023-10-16T20:05:17.361 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.358+0000 7f2aec4c9640 1 -- 172.21.15.88:0/1593023039 wait complete. 2023-10-16T20:05:17.379 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-16T20:05:17.379 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:05:17.521 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.518+0000 7fb7547b8640 1 Processor -- start 2023-10-16T20:05:17.521 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.518+0000 7fb7547b8640 1 -- start start 2023-10-16T20:05:17.523 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.522+0000 7fb7547b8640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb74c153ea0 0x7fb74c0d1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:17.523 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.522+0000 7fb7547b8640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb74c0cd970 0x7fb74c0cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:17.523 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.522+0000 7fb7547b8640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb74c0b5510 con 0x7fb74c0d2660 2023-10-16T20:05:17.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.522+0000 7fb7547b8640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb74c15d9a0 con 0x7fb74c0cd970 2023-10-16T20:05:17.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.522+0000 7fb7547b8640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb74c0b4990 con 0x7fb74c153ea0 2023-10-16T20:05:17.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.522+0000 7fb751d53640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fb74c0d2660 0x7fb74c15d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:39174/0 (socket says 172.21.15.88:39174) 2023-10-16T20:05:17.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.522+0000 7fb751d53640 1 -- 172.21.15.88:0/3596182713 learned_addr learned my addr 172.21.15.88:0/3596182713 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:17.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.522+0000 7fb752d55640 1 --2- 172.21.15.88:0/3596182713 >> v2:172.21.15.88:3300/0 conn(0x7fb74c0cd970 0x7fb74c0cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:17.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.522+0000 7fb752554640 1 --2- 172.21.15.88:0/3596182713 >> v2:172.21.15.94:3300/0 conn(0x7fb74c153ea0 0x7fb74c0d1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:17.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.522+0000 7fb751552640 1 -- 172.21.15.88:0/3596182713 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2053197519 0 0) 0x7fb74c0b5510 con 0x7fb74c0d2660 2023-10-16T20:05:17.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.522+0000 7fb751552640 1 -- 172.21.15.88:0/3596182713 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb738003530 con 0x7fb74c0d2660 2023-10-16T20:05:17.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.522+0000 7fb752d55640 1 -- 172.21.15.88:0/3596182713 >> v2:172.21.15.94:3300/0 conn(0x7fb74c153ea0 msgr2=0x7fb74c0d1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:17.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.522+0000 7fb752d55640 1 --2- 172.21.15.88:0/3596182713 >> v2:172.21.15.94:3300/0 conn(0x7fb74c153ea0 0x7fb74c0d1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.522+0000 7fb752d55640 1 -- 172.21.15.88:0/3596182713 >> v1:172.21.15.88:6789/0 conn(0x7fb74c0d2660 legacy=0x7fb74c15d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:17.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.522+0000 7fb752d55640 1 -- 172.21.15.88:0/3596182713 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb74c0b4c70 con 0x7fb74c0cd970 2023-10-16T20:05:17.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.526+0000 7fb752d55640 1 --2- 172.21.15.88:0/3596182713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb74c0cd970 0x7fb74c0cdd80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb74c06c610 tx=0x7fb74401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57249cdd6cdc1083 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:17.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.526+0000 7fb751552640 1 -- 172.21.15.88:0/3596182713 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7440a59a0 con 0x7fb74c0cd970 2023-10-16T20:05:17.527 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.526+0000 7fb751552640 1 -- 172.21.15.88:0/3596182713 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7440a5b00 con 0x7fb74c0cd970 2023-10-16T20:05:17.527 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.526+0000 7fb751552640 1 -- 172.21.15.88:0/3596182713 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7440a5e90 con 0x7fb74c0cd970 2023-10-16T20:05:17.528 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.526+0000 7fb7547b8640 1 -- 172.21.15.88:0/3596182713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb74c0cd970 msgr2=0x7fb74c0cdd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:17.528 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.526+0000 7fb7547b8640 1 --2- 172.21.15.88:0/3596182713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb74c0cd970 0x7fb74c0cdd80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb74c06c610 tx=0x7fb74401c8e0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.528 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.526+0000 7fb7547b8640 1 -- 172.21.15.88:0/3596182713 shutdown_connections 2023-10-16T20:05:17.528 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.526+0000 7fb7547b8640 1 --2- 172.21.15.88:0/3596182713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb74c0cd970 0x7fb74c0cdd80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.526+0000 7fb7547b8640 1 --2- 172.21.15.88:0/3596182713 >> v2:172.21.15.94:3300/0 conn(0x7fb74c153ea0 0x7fb74c0d1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.526+0000 7fb7547b8640 1 -- 172.21.15.88:0/3596182713 >> 172.21.15.88:0/3596182713 conn(0x7fb74c0715c0 msgr2=0x7fb74c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:17.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.526+0000 7fb7547b8640 1 -- 172.21.15.88:0/3596182713 shutdown_connections 2023-10-16T20:05:17.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.526+0000 7fb7547b8640 1 -- 172.21.15.88:0/3596182713 wait complete. 2023-10-16T20:05:17.531 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.526+0000 7fb7547b8640 1 Processor -- start 2023-10-16T20:05:17.531 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.526+0000 7fb7547b8640 1 -- start start 2023-10-16T20:05:17.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.530+0000 7fb7547b8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb74c0cd970 0x7fb74c0c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:17.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.530+0000 7fb7547b8640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb74c0d2660 0x7fb74c0b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:17.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.530+0000 7fb7547b8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb74c153ea0 0x7fb74c0c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:17.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.530+0000 7fb7547b8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74c144ba0 con 0x7fb74c0cd970 2023-10-16T20:05:17.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.530+0000 7fb7547b8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74c0b7360 con 0x7fb74c153ea0 2023-10-16T20:05:17.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.530+0000 7fb7547b8640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb74c145730 con 0x7fb74c0d2660 2023-10-16T20:05:17.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.530+0000 7fb752554640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb74c0cd970 0x7fb74c0c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:17.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.530+0000 7fb752554640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb74c0cd970 0x7fb74c0c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40248/0 (socket says 172.21.15.88:40248) 2023-10-16T20:05:17.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.530+0000 7fb752554640 1 -- 172.21.15.88:0/37286900 learned_addr learned my addr 172.21.15.88:0/37286900 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:17.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.530+0000 7fb752d55640 1 --2- 172.21.15.88:0/37286900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb74c153ea0 0x7fb74c0c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:17.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.530+0000 7fb751d53640 1 --2- 172.21.15.88:0/37286900 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb74c0d2660 0x7fb74c0b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:17.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.530+0000 7fb752554640 1 -- 172.21.15.88:0/37286900 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb74c0d2660 msgr2=0x7fb74c0b7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:17.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.530+0000 7fb752554640 1 --2- 172.21.15.88:0/37286900 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb74c0d2660 0x7fb74c0b7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.530+0000 7fb752554640 1 -- 172.21.15.88:0/37286900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb74c153ea0 msgr2=0x7fb74c0c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:17.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.530+0000 7fb752554640 1 --2- 172.21.15.88:0/37286900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb74c153ea0 0x7fb74c0c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.530+0000 7fb752554640 1 -- 172.21.15.88:0/37286900 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb744009a70 con 0x7fb74c0cd970 2023-10-16T20:05:17.535 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.530+0000 7fb752d55640 1 --2- 172.21.15.88:0/37286900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb74c153ea0 0x7fb74c0c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:17.535 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.530+0000 7fb752554640 1 --2- 172.21.15.88:0/37286900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb74c0cd970 0x7fb74c0c8db0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb74800e870 tx=0x7fb74800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:17.535 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.530+0000 7fb742ffd640 1 -- 172.21.15.88:0/37286900 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb748009510 con 0x7fb74c0cd970 2023-10-16T20:05:17.535 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.530+0000 7fb742ffd640 1 -- 172.21.15.88:0/37286900 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb74800eec0 con 0x7fb74c0cd970 2023-10-16T20:05:17.535 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.530+0000 7fb742ffd640 1 -- 172.21.15.88:0/37286900 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74800cb40 con 0x7fb74c0cd970 2023-10-16T20:05:17.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.530+0000 7fb7547b8640 1 -- 172.21.15.88:0/37286900 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb74c145a10 con 0x7fb74c0cd970 2023-10-16T20:05:17.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.530+0000 7fb7547b8640 1 -- 172.21.15.88:0/37286900 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb74c0c9e90 con 0x7fb74c0cd970 2023-10-16T20:05:17.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.530+0000 7fb7547b8640 1 -- 172.21.15.88:0/37286900 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb74c145c00 con 0x7fb74c0cd970 2023-10-16T20:05:17.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.530+0000 7fb742ffd640 1 -- 172.21.15.88:0/37286900 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb74800ccc0 con 0x7fb74c0cd970 2023-10-16T20:05:17.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.534+0000 7fb742ffd640 1 --2- 172.21.15.88:0/37286900 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb7300664a0 0x7fb730068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:17.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.534+0000 7fb742ffd640 1 -- 172.21.15.88:0/37286900 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb748084260 con 0x7fb74c0cd970 2023-10-16T20:05:17.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.534+0000 7fb751d53640 1 --2- 172.21.15.88:0/37286900 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb7300664a0 0x7fb730068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:17.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.534+0000 7fb751d53640 1 --2- 172.21.15.88:0/37286900 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb7300664a0 0x7fb730068960 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb73c0011f0 tx=0x7fb73c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:17.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.538+0000 7fb742ffd640 1 -- 172.21.15.88:0/37286900 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb7480513e0 con 0x7fb74c0cd970 2023-10-16T20:05:17.687 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.686+0000 7fb7547b8640 1 -- 172.21.15.88:0/37286900 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb74c04e9c0 con 0x7fb74c0cd970 2023-10-16T20:05:17.688 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.686+0000 7fb742ffd640 1 -- 172.21.15.88:0/37286900 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18274 (secure 0 0 0) 0x7fb748054a00 con 0x7fb74c0cd970 2023-10-16T20:05:17.689 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:17.689 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:05:01.835630+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T20:04:54.442487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T20:04:55.766641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T20:04:59.475830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:05:17.691 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.690+0000 7fb7547b8640 1 -- 172.21.15.88:0/37286900 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb7300664a0 msgr2=0x7fb730068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:17.691 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.690+0000 7fb7547b8640 1 --2- 172.21.15.88:0/37286900 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb7300664a0 0x7fb730068960 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb73c0011f0 tx=0x7fb73c009040 comp rx=0 tx=0).stop 2023-10-16T20:05:17.691 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.690+0000 7fb7547b8640 1 -- 172.21.15.88:0/37286900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb74c0cd970 msgr2=0x7fb74c0c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:17.691 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.690+0000 7fb7547b8640 1 --2- 172.21.15.88:0/37286900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb74c0cd970 0x7fb74c0c8db0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb74800e870 tx=0x7fb74800ec70 comp rx=0 tx=0).stop 2023-10-16T20:05:17.691 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.690+0000 7fb7547b8640 1 -- 172.21.15.88:0/37286900 shutdown_connections 2023-10-16T20:05:17.692 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.690+0000 7fb7547b8640 1 --2- 172.21.15.88:0/37286900 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb7300664a0 0x7fb730068960 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.692 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.690+0000 7fb7547b8640 1 --2- 172.21.15.88:0/37286900 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb74c0d2660 0x7fb74c0b7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.692 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.690+0000 7fb7547b8640 1 --2- 172.21.15.88:0/37286900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb74c153ea0 0x7fb74c0c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.692 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.690+0000 7fb7547b8640 1 --2- 172.21.15.88:0/37286900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb74c0cd970 0x7fb74c0c8db0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.692 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.690+0000 7fb7547b8640 1 -- 172.21.15.88:0/37286900 >> 172.21.15.88:0/37286900 conn(0x7fb74c0715c0 msgr2=0x7fb74c15eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:17.693 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.690+0000 7fb7547b8640 1 -- 172.21.15.88:0/37286900 shutdown_connections 2023-10-16T20:05:17.693 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.690+0000 7fb7547b8640 1 -- 172.21.15.88:0/37286900 wait complete. 2023-10-16T20:05:17.710 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-16T20:05:17.711 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-16T20:05:17.711 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-16T20:05:17.712 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-16T20:05:17.713 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-16T20:05:17.713 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-16T20:05:17.714 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-16T20:05:17.714 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-16T20:05:17.953 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.954+0000 7f2274679640 1 Processor -- start 2023-10-16T20:05:17.954 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.954+0000 7f2274679640 1 -- start start 2023-10-16T20:05:17.955 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.954+0000 7f2274679640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f2264006050 0x7f2264058600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:17.956 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.954+0000 7f2274679640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2264053f20 0x7f226403c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:17.956 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.954+0000 7f2274679640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2264037c60 0x7f226403c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:17.956 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.954+0000 7f2274679640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2264054700 con 0x7f2264006050 2023-10-16T20:05:17.957 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.954+0000 7f2274679640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2264054420 con 0x7f2264053f20 2023-10-16T20:05:17.957 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.954+0000 7f2274679640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2264057b20 con 0x7f2264037c60 2023-10-16T20:05:17.957 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.954+0000 7f2272c16640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2264037c60 0x7f226403c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:17.958 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.954+0000 7f2271c14640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2264053f20 0x7f226403c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:17.958 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.954+0000 7f2272c16640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2264037c60 0x7f226403c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:32900/0 (socket says 172.21.15.88:32900) 2023-10-16T20:05:17.958 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.954+0000 7f2272c16640 1 -- 172.21.15.88:0/4242527250 learned_addr learned my addr 172.21.15.88:0/4242527250 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:17.958 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.954+0000 7f2272c16640 1 -- 172.21.15.88:0/4242527250 >> v2:172.21.15.94:3300/0 conn(0x7f2264053f20 msgr2=0x7f226403c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:17.959 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.954+0000 7f2272c16640 1 --2- 172.21.15.88:0/4242527250 >> v2:172.21.15.94:3300/0 conn(0x7f2264053f20 0x7f226403c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.959 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.954+0000 7f2272c16640 1 -- 172.21.15.88:0/4242527250 >> v2:172.21.15.88:3300/0 conn(0x7f2264006050 msgr2=0x7f2264058600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T20:05:17.959 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.954+0000 7f2272c16640 1 --2- 172.21.15.88:0/4242527250 >> v2:172.21.15.88:3300/0 conn(0x7f2264006050 0x7f2264058600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.959 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.954+0000 7f2272c16640 1 -- 172.21.15.88:0/4242527250 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2264057e00 con 0x7f2264037c60 2023-10-16T20:05:17.960 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.958+0000 7faccca79640 1 Processor -- start 2023-10-16T20:05:17.960 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.958+0000 7faccca79640 1 -- start start 2023-10-16T20:05:17.965 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.958+0000 7faccca79640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7facc00061b0 0x7facc003a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:17.965 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.958+0000 7faccca79640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7facc00525f0 0x7facc00e33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:17.965 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.958+0000 7faccca79640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc00e38f0 con 0x7facc00506e0 2023-10-16T20:05:17.966 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.958+0000 7faccca79640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facc00e6e40 con 0x7facc00525f0 2023-10-16T20:05:17.966 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.958+0000 7faccca79640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc004fdd0 con 0x7facc00061b0 2023-10-16T20:05:17.966 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.958+0000 7facc659c640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7facc00061b0 0x7facc003a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:17.966 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.958+0000 7facc659c640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7facc00061b0 0x7facc003a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:32910/0 (socket says 172.21.15.88:32910) 2023-10-16T20:05:17.966 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.958+0000 7facc6d9d640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7facc00506e0 0x7facc00396a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:48668/0 (socket says 172.21.15.88:48668) 2023-10-16T20:05:17.967 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.958+0000 7facc659c640 1 -- 172.21.15.88:0/2907948112 learned_addr learned my addr 172.21.15.88:0/2907948112 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:17.967 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.958+0000 7facc5d9b640 1 --2- 172.21.15.88:0/2907948112 >> v2:172.21.15.88:3300/0 conn(0x7facc00525f0 0x7facc00e33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:17.967 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.962+0000 7facc559a640 1 -- 172.21.15.88:0/2907948112 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1154782876 0 0) 0x7facc00e38f0 con 0x7facc00506e0 2023-10-16T20:05:17.967 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.962+0000 7facc559a640 1 -- 172.21.15.88:0/2907948112 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facb0003530 con 0x7facc00506e0 2023-10-16T20:05:17.967 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.962+0000 7facc659c640 1 -- 172.21.15.88:0/2907948112 >> v2:172.21.15.88:3300/0 conn(0x7facc00525f0 msgr2=0x7facc00e33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:17.968 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.962+0000 7facc659c640 1 --2- 172.21.15.88:0/2907948112 >> v2:172.21.15.88:3300/0 conn(0x7facc00525f0 0x7facc00e33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.968 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.962+0000 7facc659c640 1 -- 172.21.15.88:0/2907948112 >> v1:172.21.15.94:6789/0 conn(0x7facc00506e0 legacy=0x7facc00396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:17.968 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.962+0000 7facc659c640 1 -- 172.21.15.88:0/2907948112 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc00500b0 con 0x7facc00061b0 2023-10-16T20:05:17.968 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.966+0000 7facc659c640 1 --2- 172.21.15.88:0/2907948112 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7facc00061b0 0x7facc003a620 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7facb8009870 tx=0x7facb802d540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1606f055eafdd31b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:17.969 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.966+0000 7f2272c16640 1 --2- 172.21.15.88:0/4242527250 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2264037c60 0x7f226403c5a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2264010100 tx=0x7f225c01ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a608f6e895064be server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:17.969 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.966+0000 7facc559a640 1 -- 172.21.15.88:0/2907948112 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb80ba620 con 0x7facc00061b0 2023-10-16T20:05:17.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.966+0000 7facc559a640 1 -- 172.21.15.88:0/2907948112 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb80ba780 con 0x7facc00061b0 2023-10-16T20:05:17.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.966+0000 7facc559a640 1 -- 172.21.15.88:0/2907948112 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb80bab10 con 0x7facc00061b0 2023-10-16T20:05:17.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.966+0000 7f2271413640 1 -- 172.21.15.88:0/4242527250 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c0a89b0 con 0x7f2264037c60 2023-10-16T20:05:17.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.966+0000 7f2271413640 1 -- 172.21.15.88:0/4242527250 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f225c0a8b10 con 0x7f2264037c60 2023-10-16T20:05:17.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.966+0000 7f2271413640 1 -- 172.21.15.88:0/4242527250 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c0a8ea0 con 0x7f2264037c60 2023-10-16T20:05:17.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.966+0000 7faccca79640 1 -- 172.21.15.88:0/2907948112 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7facc00061b0 msgr2=0x7facc003a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:17.972 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.966+0000 7faccca79640 1 --2- 172.21.15.88:0/2907948112 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7facc00061b0 0x7facc003a620 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7facb8009870 tx=0x7facb802d540 comp rx=0 tx=0).stop 2023-10-16T20:05:17.972 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.966+0000 7faccca79640 1 -- 172.21.15.88:0/2907948112 shutdown_connections 2023-10-16T20:05:17.972 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.966+0000 7faccca79640 1 --2- 172.21.15.88:0/2907948112 >> v2:172.21.15.88:3300/0 conn(0x7facc00525f0 0x7facc00e33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.972 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.966+0000 7faccca79640 1 --2- 172.21.15.88:0/2907948112 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7facc00061b0 0x7facc003a620 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.966+0000 7f2274679640 1 -- 172.21.15.88:0/4242527250 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2264037c60 msgr2=0x7f226403c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:17.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.966+0000 7f2274679640 1 --2- 172.21.15.88:0/4242527250 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2264037c60 0x7f226403c5a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2264010100 tx=0x7f225c01ecf0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.966+0000 7faccca79640 1 -- 172.21.15.88:0/2907948112 >> 172.21.15.88:0/2907948112 conn(0x7facc0009fa0 msgr2=0x7facc000a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:17.974 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.966+0000 7faccca79640 1 -- 172.21.15.88:0/2907948112 shutdown_connections 2023-10-16T20:05:17.974 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.966+0000 7faccca79640 1 -- 172.21.15.88:0/2907948112 wait complete. 2023-10-16T20:05:17.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.966+0000 7f2274679640 1 -- 172.21.15.88:0/4242527250 shutdown_connections 2023-10-16T20:05:17.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.966+0000 7f2274679640 1 --2- 172.21.15.88:0/4242527250 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2264037c60 0x7f226403c5a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.966+0000 7f2274679640 1 --2- 172.21.15.88:0/4242527250 >> v2:172.21.15.94:3300/0 conn(0x7f2264053f20 0x7f226403c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.966+0000 7f2274679640 1 --2- 172.21.15.88:0/4242527250 >> v2:172.21.15.88:3300/0 conn(0x7f2264006050 0x7f2264058600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.966+0000 7f2274679640 1 -- 172.21.15.88:0/4242527250 >> 172.21.15.88:0/4242527250 conn(0x7f2264058ec0 msgr2=0x7f22640592d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:17.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.966+0000 7f2274679640 1 -- 172.21.15.88:0/4242527250 shutdown_connections 2023-10-16T20:05:17.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.966+0000 7f2274679640 1 -- 172.21.15.88:0/4242527250 wait complete. 2023-10-16T20:05:17.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.970+0000 7faccca79640 1 Processor -- start 2023-10-16T20:05:17.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.970+0000 7faccca79640 1 -- start start 2023-10-16T20:05:17.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.970+0000 7f2274679640 1 Processor -- start 2023-10-16T20:05:17.978 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.970+0000 7f2274679640 1 -- start start 2023-10-16T20:05:17.978 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.970+0000 7faccca79640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7facc00061b0 0x7facc00e45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:17.978 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.970+0000 7faccca79640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facc00506e0 0x7facc00d3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:17.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.970+0000 7faccca79640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facc00525f0 0x7facc00d38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:17.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.970+0000 7f2636206640 1 Processor -- start 2023-10-16T20:05:17.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.970+0000 7f2636206640 1 -- start start 2023-10-16T20:05:17.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f2274679640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2264006050 0x7f226404c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:17.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7facc6d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facc00525f0 0x7facc00d38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:17.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7facc6d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facc00525f0 0x7facc00d38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40264/0 (socket says 172.21.15.88:40264) 2023-10-16T20:05:17.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f2274679640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2264037c60 0x7f22640e13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:17.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f2272415640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2264006050 0x7f226404c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:17.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f2272415640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2264006050 0x7f226404c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40276/0 (socket says 172.21.15.88:40276) 2023-10-16T20:05:17.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f2272415640 1 -- 172.21.15.88:0/3511215215 learned_addr learned my addr 172.21.15.88:0/3511215215 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:17.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f2274679640 1 --2- 172.21.15.88:0/3511215215 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2264053f20 0x7f22640de6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:17.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f2274679640 1 -- 172.21.15.88:0/3511215215 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226417d0d0 con 0x7f2264006050 2023-10-16T20:05:17.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f2274679640 1 -- 172.21.15.88:0/3511215215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226404d900 con 0x7f2264037c60 2023-10-16T20:05:17.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f2274679640 1 -- 172.21.15.88:0/3511215215 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22640e1920 con 0x7f2264053f20 2023-10-16T20:05:17.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f2271c14640 1 --2- 172.21.15.88:0/3511215215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2264037c60 0x7f22640e13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:17.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7ff06bfd9640 1 Processor -- start 2023-10-16T20:05:17.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f0e72772640 1 Processor -- start 2023-10-16T20:05:17.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f2636206640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2628054480 0x7f262804f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:17.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f2636206640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2628037d60 0x7f2628038210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:17.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f2636206640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26280549f0 con 0x7f26280e78e0 2023-10-16T20:05:17.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f2636206640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26280077b0 con 0x7f2628054480 2023-10-16T20:05:17.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f2636206640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26280069f0 con 0x7f2628037d60 2023-10-16T20:05:17.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f2634a03640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2628054480 0x7f262804f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:17.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f2634a03640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2628054480 0x7f262804f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:56710/0 (socket says 172.21.15.88:56710) 2023-10-16T20:05:17.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f2634a03640 1 -- 172.21.15.88:0/3137349555 learned_addr learned my addr 172.21.15.88:0/3137349555 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:17.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7ff06bfd9640 1 -- start start 2023-10-16T20:05:17.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f2635a05640 1 --2- 172.21.15.88:0/3137349555 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2628037d60 0x7f2628038210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:17.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f0e72772640 1 -- start start 2023-10-16T20:05:17.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f2634a03640 1 -- 172.21.15.88:0/3137349555 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2628037d60 msgr2=0x7f2628038210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:17.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f2634a03640 1 --2- 172.21.15.88:0/3137349555 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2628037d60 0x7f2628038210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f2634a03640 1 -- 172.21.15.88:0/3137349555 >> v1:172.21.15.94:6789/0 conn(0x7f26280e78e0 legacy=0x7f262804f210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T20:05:17.988 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f2634a03640 1 -- 172.21.15.88:0/3137349555 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2628006cd0 con 0x7f2628054480 2023-10-16T20:05:17.988 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7facc6d9d640 1 -- 172.21.15.88:0/1732139448 learned_addr learned my addr 172.21.15.88:0/1732139448 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:17.988 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7faccca79640 1 -- 172.21.15.88:0/1732139448 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc00e4af0 con 0x7facc00525f0 2023-10-16T20:05:17.989 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7faccca79640 1 -- 172.21.15.88:0/1732139448 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc00e1920 con 0x7facc00506e0 2023-10-16T20:05:17.989 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7faccca79640 1 -- 172.21.15.88:0/1732139448 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc00e0d10 con 0x7facc00061b0 2023-10-16T20:05:17.989 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7facc659c640 1 --2- 172.21.15.88:0/1732139448 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7facc00061b0 0x7facc00e45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:17.989 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7facc5d9b640 1 --2- 172.21.15.88:0/1732139448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facc00506e0 0x7facc00d3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:17.989 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7facc659c640 1 -- 172.21.15.88:0/1732139448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facc00506e0 msgr2=0x7facc00d3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:17.990 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7facc659c640 1 --2- 172.21.15.88:0/1732139448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facc00506e0 0x7facc00d3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.990 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7facc659c640 1 -- 172.21.15.88:0/1732139448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facc00525f0 msgr2=0x7facc00d38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:17.990 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7facc659c640 1 --2- 172.21.15.88:0/1732139448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facc00525f0 0x7facc00d38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.990 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7facc659c640 1 -- 172.21.15.88:0/1732139448 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb80094d0 con 0x7facc00061b0 2023-10-16T20:05:17.990 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7facc659c640 1 --2- 172.21.15.88:0/1732139448 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7facc00061b0 0x7facc00e45b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7facb80bb000 tx=0x7facb802bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:17.991 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f2272c16640 1 --2- 172.21.15.88:0/3511215215 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2264053f20 0x7f22640de6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:17.991 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f2272415640 1 -- 172.21.15.88:0/3511215215 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2264053f20 msgr2=0x7f22640de6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:17.991 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f2272415640 1 --2- 172.21.15.88:0/3511215215 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2264053f20 0x7f22640de6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.991 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f2272415640 1 -- 172.21.15.88:0/3511215215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2264037c60 msgr2=0x7f22640e13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:17.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f2272415640 1 --2- 172.21.15.88:0/3511215215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2264037c60 0x7f22640e13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:17.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f2272415640 1 -- 172.21.15.88:0/3511215215 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f225c0094d0 con 0x7f2264006050 2023-10-16T20:05:17.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f2272c16640 1 --2- 172.21.15.88:0/3511215215 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2264053f20 0x7f22640de6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:05:17.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f2271c14640 1 --2- 172.21.15.88:0/3511215215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2264037c60 0x7f22640e13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:05:17.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f2272415640 1 --2- 172.21.15.88:0/3511215215 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2264006050 0x7f226404c9f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f226c000b70 tx=0x7f226c04f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:17.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7f2262ffd640 1 -- 172.21.15.88:0/3511215215 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c072040 con 0x7f2264006050 2023-10-16T20:05:17.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7f2262ffd640 1 -- 172.21.15.88:0/3511215215 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226c072790 con 0x7f2264006050 2023-10-16T20:05:17.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7f2262ffd640 1 -- 172.21.15.88:0/3511215215 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c072b20 con 0x7f2264006050 2023-10-16T20:05:17.994 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7facb6ffd640 1 -- 172.21.15.88:0/1732139448 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb80badc0 con 0x7facc00061b0 2023-10-16T20:05:17.994 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7facb6ffd640 1 -- 172.21.15.88:0/1732139448 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb80cd450 con 0x7facc00061b0 2023-10-16T20:05:17.994 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7facb6ffd640 1 -- 172.21.15.88:0/1732139448 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb80cd800 con 0x7facc00061b0 2023-10-16T20:05:17.994 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7faccca79640 1 -- 172.21.15.88:0/1732139448 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc00e0ff0 con 0x7facc00061b0 2023-10-16T20:05:17.995 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7ff06bfd9640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff064153ea0 0x7ff0640d1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:17.995 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7ff06bfd9640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff0640d2660 0x7ff06415d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:17.995 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7ff06bfd9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff06415d7d0 0x7ff0640cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:17.995 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7ff06bfd9640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0640b5510 con 0x7ff064153ea0 2023-10-16T20:05:17.995 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7ff06bfd9640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0640b4990 con 0x7ff0640d2660 2023-10-16T20:05:17.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7ff06bfd9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0640b4c70 con 0x7ff06415d7d0 2023-10-16T20:05:17.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7f2274679640 1 -- 172.21.15.88:0/3511215215 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22640dec20 con 0x7f2264006050 2023-10-16T20:05:17.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7f2274679640 1 -- 172.21.15.88:0/3511215215 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f226404d340 con 0x7f2264006050 2023-10-16T20:05:17.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7ff06a576640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff06415d7d0 0x7ff0640cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:17.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7ff06a576640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff06415d7d0 0x7ff0640cd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:32954/0 (socket says 172.21.15.88:32954) 2023-10-16T20:05:17.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7ff06a576640 1 -- 172.21.15.88:0/15176497 learned_addr learned my addr 172.21.15.88:0/15176497 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:17.998 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f0e72772640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0e6c158ba0 0x7f0e6c06da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:17.998 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f0e72772640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0e6c06df80 0x7f0e6c0701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:17.998 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f0e72772640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e6c06e3e0 con 0x7f0e6c0706e0 2023-10-16T20:05:17.998 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f0e72772640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e6c0ae100 con 0x7f0e6c158ba0 2023-10-16T20:05:17.998 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.974+0000 7f0e72772640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c0b1d80 con 0x7f0e6c06df80 2023-10-16T20:05:17.999 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7f0e6b7fe640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0e6c06df80 0x7f0e6c0701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:17.999 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7f0e6b7fe640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0e6c06df80 0x7f0e6c0701a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:32950/0 (socket says 172.21.15.88:32950) 2023-10-16T20:05:17.999 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7f0e6b7fe640 1 -- 172.21.15.88:0/182457991 learned_addr learned my addr 172.21.15.88:0/182457991 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:17.999 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7f0e6b7fe640 1 -- 172.21.15.88:0/182457991 >> v2:172.21.15.94:3300/0 conn(0x7f0e6c158ba0 msgr2=0x7f0e6c06da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:17.999 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7f0e6bfff640 1 --2- 172.21.15.88:0/182457991 >> v2:172.21.15.94:3300/0 conn(0x7f0e6c158ba0 0x7f0e6c06da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.000 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7f0e6b7fe640 1 --2- 172.21.15.88:0/182457991 >> v2:172.21.15.94:3300/0 conn(0x7f0e6c158ba0 0x7f0e6c06da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.000 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7f0e6b7fe640 1 -- 172.21.15.88:0/182457991 >> v1:172.21.15.94:6789/0 conn(0x7f0e6c0706e0 legacy=0x7f0e6c15c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.000 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7faccca79640 1 -- 172.21.15.88:0/1732139448 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc00e15b0 con 0x7facc00061b0 2023-10-16T20:05:18.001 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7ff069d75640 1 --2- 172.21.15.88:0/15176497 >> v2:172.21.15.88:3300/0 conn(0x7ff064153ea0 0x7ff0640d1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.001 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7ff069574640 1 --2- 172.21.15.88:0/15176497 >> v2:172.21.15.94:3300/0 conn(0x7ff0640d2660 0x7ff06415d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.001 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7f0e6b7fe640 1 -- 172.21.15.88:0/182457991 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6c0711f0 con 0x7f0e6c06df80 2023-10-16T20:05:18.002 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7fc04aa46640 1 Processor -- start 2023-10-16T20:05:18.002 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7faccca79640 1 -- 172.21.15.88:0/1732139448 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7facc004fa80 con 0x7facc00061b0 2023-10-16T20:05:18.002 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7f2274679640 1 -- 172.21.15.88:0/3511215215 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f226403cae0 con 0x7f2264006050 2023-10-16T20:05:18.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7fbe61079640 1 Processor -- start 2023-10-16T20:05:18.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7fc04aa46640 1 -- start start 2023-10-16T20:05:18.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7fbe61079640 1 -- start start 2023-10-16T20:05:18.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7ff06a576640 1 -- 172.21.15.88:0/15176497 >> v2:172.21.15.94:3300/0 conn(0x7ff0640d2660 msgr2=0x7ff06415d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:18.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7ff06a576640 1 --2- 172.21.15.88:0/15176497 >> v2:172.21.15.94:3300/0 conn(0x7ff0640d2660 0x7ff06415d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7ff06a576640 1 -- 172.21.15.88:0/15176497 >> v2:172.21.15.88:3300/0 conn(0x7ff064153ea0 msgr2=0x7ff0640d1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:18.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7ff06a576640 1 --2- 172.21.15.88:0/15176497 >> v2:172.21.15.88:3300/0 conn(0x7ff064153ea0 0x7ff0640d1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7ff06a576640 1 -- 172.21.15.88:0/15176497 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0640b4f50 con 0x7ff06415d7d0 2023-10-16T20:05:18.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.978+0000 7f2262ffd640 1 -- 172.21.15.88:0/3511215215 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f226c08f020 con 0x7f2264006050 2023-10-16T20:05:18.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7f2634a03640 1 --2- 172.21.15.88:0/3137349555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2628054480 0x7f262804f920 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f262c0098a0 tx=0x7f262c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46c47db80122ce60 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7facb6ffd640 1 -- 172.21.15.88:0/1732139448 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7facb80d63f0 con 0x7facc00061b0 2023-10-16T20:05:18.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7f2262ffd640 1 --2- 172.21.15.88:0/3511215215 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f22500664f0 0x7f22500689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7f2627fff640 1 -- 172.21.15.88:0/3137349555 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c0ac640 con 0x7f2628054480 2023-10-16T20:05:18.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7f2627fff640 1 -- 172.21.15.88:0/3137349555 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262c0ac7a0 con 0x7f2628054480 2023-10-16T20:05:18.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7f2627fff640 1 -- 172.21.15.88:0/3137349555 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c0acb30 con 0x7f2628054480 2023-10-16T20:05:18.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7f2262ffd640 1 -- 172.21.15.88:0/3511215215 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f226c0fd0f0 con 0x7f2264006050 2023-10-16T20:05:18.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7f2271c14640 1 --2- 172.21.15.88:0/3511215215 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f22500664f0 0x7f22500689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7fc04aa46640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc0440afdf0 0x7fc0440aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7fc04aa46640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc044162580 con 0x7fc0440cc830 2023-10-16T20:05:18.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7fc04aa46640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0440ae830 con 0x7fc044069e90 2023-10-16T20:05:18.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7fc04aa46640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0440adf00 con 0x7fc0440afdf0 2023-10-16T20:05:18.010 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7fbe61079640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fbe5c15cb70 0x7fbe5c161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.010 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7fbe61079640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fbe5c0cdad0 0x7fbe5c0cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.010 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7fbe61079640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe5c0cff40 0x7fbe5c0b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.010 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7fbe61079640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe5c1615a0 con 0x7fbe5c15cb70 2023-10-16T20:05:18.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7fbe61079640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe5c0d03b0 con 0x7fbe5c0cdad0 2023-10-16T20:05:18.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7fbe61079640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5c0b2190 con 0x7fbe5c0cff40 2023-10-16T20:05:18.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7f2262ffd640 1 --2- 172.21.15.88:0/3511215215 >> [v2:172.21.15.88:6800/685586021,v1:172.21.15.88:6801/685586021] conn(0x7f225006fb30 0x7f2250071f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.012 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7fbe5b59e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe5c0cff40 0x7fbe5c0b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.012 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7fbe5b59e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe5c0cff40 0x7fbe5c0b7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:32960/0 (socket says 172.21.15.88:32960) 2023-10-16T20:05:18.012 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7fbe5b59e640 1 -- 172.21.15.88:0/3288064234 learned_addr learned my addr 172.21.15.88:0/3288064234 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:18.012 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7fbe5ad9d640 1 --2- 172.21.15.88:0/3288064234 >> v2:172.21.15.88:3300/0 conn(0x7fbe5c15cb70 0x7fbe5c161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.013 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7fc043fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc0440afdf0 0x7fc0440aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.013 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7fc043fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc0440afdf0 0x7fc0440aecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40306/0 (socket says 172.21.15.88:40306) 2023-10-16T20:05:18.013 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7fc0437fe640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fc044069e90 0x7fc0440cc120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:48702/0 (socket says 172.21.15.88:48702) 2023-10-16T20:05:18.013 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7f2262ffd640 1 -- 172.21.15.88:0/3511215215 --> [v2:172.21.15.88:6800/685586021,v1:172.21.15.88:6801/685586021] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2250072640 con 0x7f225006fb30 2023-10-16T20:05:18.014 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7f2262ffd640 1 -- 172.21.15.88:0/3511215215 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f226c0ff800 con 0x7f2264006050 2023-10-16T20:05:18.014 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7fbe5b59e640 1 -- 172.21.15.88:0/3288064234 >> v2:172.21.15.94:3300/0 conn(0x7fbe5c0cdad0 msgr2=0x7fbe5c0cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:18.014 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7fbe5a59c640 1 --2- 172.21.15.88:0/3288064234 >> v2:172.21.15.94:3300/0 conn(0x7fbe5c0cdad0 0x7fbe5c0cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7facb6ffd640 1 --2- 172.21.15.88:0/1732139448 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7faca80664f0 0x7faca80689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7facb6ffd640 1 -- 172.21.15.88:0/1732139448 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7facb812f750 con 0x7facc00061b0 2023-10-16T20:05:18.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7fbe5b59e640 1 --2- 172.21.15.88:0/3288064234 >> v2:172.21.15.94:3300/0 conn(0x7fbe5c0cdad0 0x7fbe5c0cfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7facc5d9b640 1 --2- 172.21.15.88:0/1732139448 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7faca80664f0 0x7faca80689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7facb6ffd640 1 --2- 172.21.15.88:0/1732139448 >> [v2:172.21.15.88:6808/2212808191,v1:172.21.15.88:6809/2212808191] conn(0x7faca806fab0 0x7faca8071ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7fbe5b59e640 1 -- 172.21.15.88:0/3288064234 >> v2:172.21.15.88:3300/0 conn(0x7fbe5c15cb70 msgr2=0x7fbe5c161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:18.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7fbe5b59e640 1 --2- 172.21.15.88:0/3288064234 >> v2:172.21.15.88:3300/0 conn(0x7fbe5c15cb70 0x7fbe5c161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7fc0437fe640 1 -- 172.21.15.88:0/4249779990 learned_addr learned my addr 172.21.15.88:0/4249779990 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:18.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7fbe5b59e640 1 -- 172.21.15.88:0/3288064234 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe5c15e230 con 0x7fbe5c0cff40 2023-10-16T20:05:18.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7fbe5ad9d640 1 --2- 172.21.15.88:0/3288064234 >> v2:172.21.15.88:3300/0 conn(0x7fbe5c15cb70 0x7fbe5c161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:18.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7fbe5a59c640 1 --2- 172.21.15.88:0/3288064234 >> v2:172.21.15.94:3300/0 conn(0x7fbe5c0cdad0 0x7fbe5c0cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:18.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7facb6ffd640 1 -- 172.21.15.88:0/1732139448 --> [v2:172.21.15.88:6808/2212808191,v1:172.21.15.88:6809/2212808191] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faca80725c0 con 0x7faca806fab0 2023-10-16T20:05:18.019 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7fc042ffd640 1 -- 172.21.15.88:0/4249779990 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1698162144 0 0) 0x7fc0440ae830 con 0x7fc044069e90 2023-10-16T20:05:18.019 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7facc6d9d640 1 --2- 172.21.15.88:0/1732139448 >> [v2:172.21.15.88:6808/2212808191,v1:172.21.15.88:6809/2212808191] conn(0x7faca806fab0 0x7faca8071ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7f2272c16640 1 --2- 172.21.15.88:0/3511215215 >> [v2:172.21.15.88:6800/685586021,v1:172.21.15.88:6801/685586021] conn(0x7f225006fb30 0x7f2250071f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7f2271c14640 1 --2- 172.21.15.88:0/3511215215 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f22500664f0 0x7f22500689b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f22640e0750 tx=0x7f2268009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7f2272c16640 1 --2- 172.21.15.88:0/3511215215 >> [v2:172.21.15.88:6800/685586021,v1:172.21.15.88:6801/685586021] conn(0x7f225006fb30 0x7f2250071f70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7fc042ffd640 1 -- 172.21.15.88:0/4249779990 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc02c003530 con 0x7fc044069e90 2023-10-16T20:05:18.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7fc042ffd640 1 -- 172.21.15.88:0/4249779990 <== mon.1 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4121583418 0 0) 0x7fc02c003530 con 0x7fc044069e90 2023-10-16T20:05:18.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7fc042ffd640 1 -- 172.21.15.88:0/4249779990 >> v2:172.21.15.88:3300/0 conn(0x7fc0440afdf0 msgr2=0x7fc0440aecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:18.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7fc042ffd640 1 --2- 172.21.15.88:0/4249779990 >> v2:172.21.15.88:3300/0 conn(0x7fc0440afdf0 0x7fc0440aecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.982+0000 7fc042ffd640 1 -- 172.21.15.88:0/4249779990 >> v1:172.21.15.88:6789/0 conn(0x7fc0440cc830 legacy=0x7fc044161810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:18.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7fc042ffd640 1 -- 172.21.15.88:0/4249779990 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0440ae1e0 con 0x7fc044069e90 2023-10-16T20:05:18.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7fc042ffd640 1 -- 172.21.15.88:0/4249779990 <== mon.1 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7fc034001680 con 0x7fc044069e90 2023-10-16T20:05:18.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7fc042ffd640 1 -- 172.21.15.88:0/4249779990 >> v1:172.21.15.94:6789/0 conn(0x7fc044069e90 legacy=0x7fc0440cc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7fc042ffd640 1 --2- 172.21.15.88:0/4249779990 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc02c003980 0x7fc02c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7fc042ffd640 1 --2- 172.21.15.88:0/4249779990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc02c0047d0 0x7fc02c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7fc042ffd640 1 --2- 172.21.15.88:0/4249779990 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc02c005390 0x7fc02c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7fc042ffd640 1 -- 172.21.15.88:0/4249779990 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440adf00 con 0x7fc02c003980 2023-10-16T20:05:18.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7fc042ffd640 1 -- 172.21.15.88:0/4249779990 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c00c2b0 con 0x7fc02c0047d0 2023-10-16T20:05:18.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7fc042ffd640 1 -- 172.21.15.88:0/4249779990 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c00c470 con 0x7fc02c005390 2023-10-16T20:05:18.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7fc043fff640 1 --2- 172.21.15.88:0/4249779990 >> v2:172.21.15.88:3300/0 conn(0x7fc0440afdf0 0x7fc0440aecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:18.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7facc5d9b640 1 --2- 172.21.15.88:0/1732139448 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7faca80664f0 0x7faca80689b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7facc00d9e10 tx=0x7facbc009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7facb6ffd640 1 -- 172.21.15.88:0/1732139448 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7facb80fca00 con 0x7facc00061b0 2023-10-16T20:05:18.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7ff06a576640 1 --2- 172.21.15.88:0/15176497 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff06415d7d0 0x7ff0640cd970 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff064072520 tx=0x7ff05401cf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf21c203a60d4477 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7facc6d9d640 1 --2- 172.21.15.88:0/1732139448 >> [v2:172.21.15.88:6808/2212808191,v1:172.21.15.88:6809/2212808191] conn(0x7faca806fab0 0x7faca8071ef0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7f2636206640 1 -- 172.21.15.88:0/3137349555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2628054480 msgr2=0x7f262804f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7f2636206640 1 --2- 172.21.15.88:0/3137349555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2628054480 0x7f262804f920 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f262c0098a0 tx=0x7f262c01c470 comp rx=0 tx=0).stop 2023-10-16T20:05:18.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7fc0437fe640 1 --2- 172.21.15.88:0/4249779990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc02c0047d0 0x7fc02c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7ff068d73640 1 -- 172.21.15.88:0/15176497 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0540aa610 con 0x7ff06415d7d0 2023-10-16T20:05:18.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7ff068d73640 1 -- 172.21.15.88:0/15176497 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0540a6040 con 0x7ff06415d7d0 2023-10-16T20:05:18.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7ff068d73640 1 -- 172.21.15.88:0/15176497 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0540aa990 con 0x7ff06415d7d0 2023-10-16T20:05:18.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7f0e6b7fe640 1 --2- 172.21.15.88:0/182457991 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0e6c06df80 0x7f0e6c0701a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0e60009870 tx=0x7f0e6001d450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84ae464408576f4c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7f2636206640 1 -- 172.21.15.88:0/3137349555 shutdown_connections 2023-10-16T20:05:18.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7f2636206640 1 --2- 172.21.15.88:0/3137349555 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2628037d60 0x7f2628038210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7f2636206640 1 --2- 172.21.15.88:0/3137349555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2628054480 0x7f262804f920 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7f2636206640 1 -- 172.21.15.88:0/3137349555 >> 172.21.15.88:0/3137349555 conn(0x7f2628054bc0 msgr2=0x7f2628054fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:18.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7f2636206640 1 -- 172.21.15.88:0/3137349555 shutdown_connections 2023-10-16T20:05:18.029 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7fc043fff640 1 --2- 172.21.15.88:0/4249779990 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc02c003980 0x7fc02c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.029 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7fc048fe3640 1 --2- 172.21.15.88:0/4249779990 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc02c005390 0x7fc02c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.029 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7fc0437fe640 1 -- 172.21.15.88:0/4249779990 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc02c005390 msgr2=0x7fc02c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:18.029 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7fc0437fe640 1 --2- 172.21.15.88:0/4249779990 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc02c005390 0x7fc02c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.029 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7fc0437fe640 1 -- 172.21.15.88:0/4249779990 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc02c003980 msgr2=0x7fc02c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:18.030 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7fc0437fe640 1 --2- 172.21.15.88:0/4249779990 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc02c003980 0x7fc02c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.030 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7fc0437fe640 1 -- 172.21.15.88:0/4249779990 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0440ae1e0 con 0x7fc02c0047d0 2023-10-16T20:05:18.030 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7f0e6affd640 1 -- 172.21.15.88:0/182457991 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e600ad620 con 0x7f0e6c06df80 2023-10-16T20:05:18.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7fc048fe3640 1 --2- 172.21.15.88:0/4249779990 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc02c005390 0x7fc02c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:18.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7fc048fe3640 1 -- 172.21.15.88:0/4249779990 reap_dead start 2023-10-16T20:05:18.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7f2262ffd640 1 -- 172.21.15.88:0/3511215215 <== osd.0 v2:172.21.15.88:6800/685586021 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f2250072640 con 0x7f225006fb30 2023-10-16T20:05:18.032 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7f0e6affd640 1 -- 172.21.15.88:0/182457991 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e600ad780 con 0x7f0e6c06df80 2023-10-16T20:05:18.032 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7f0e6affd640 1 -- 172.21.15.88:0/182457991 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e600adb10 con 0x7f0e6c06df80 2023-10-16T20:05:18.032 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7f2636206640 1 -- 172.21.15.88:0/3137349555 wait complete. 2023-10-16T20:05:18.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7facb6ffd640 1 -- 172.21.15.88:0/1732139448 <== osd.1 v2:172.21.15.88:6808/2212808191 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7faca80725c0 con 0x7faca806fab0 2023-10-16T20:05:18.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7ff06bfd9640 1 -- 172.21.15.88:0/15176497 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff06415d7d0 msgr2=0x7ff0640cd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7ff06bfd9640 1 --2- 172.21.15.88:0/15176497 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff06415d7d0 0x7ff0640cd970 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff064072520 tx=0x7ff05401cf30 comp rx=0 tx=0).stop 2023-10-16T20:05:18.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7ff06bfd9640 1 -- 172.21.15.88:0/15176497 shutdown_connections 2023-10-16T20:05:18.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7ff06bfd9640 1 --2- 172.21.15.88:0/15176497 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff06415d7d0 0x7ff0640cd970 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7ff06bfd9640 1 --2- 172.21.15.88:0/15176497 >> v2:172.21.15.94:3300/0 conn(0x7ff0640d2660 0x7ff06415d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7ff06bfd9640 1 --2- 172.21.15.88:0/15176497 >> v2:172.21.15.88:3300/0 conn(0x7ff064153ea0 0x7ff0640d1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7ff06bfd9640 1 -- 172.21.15.88:0/15176497 >> 172.21.15.88:0/15176497 conn(0x7ff0640715c0 msgr2=0x7ff0640719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:18.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7ff06bfd9640 1 -- 172.21.15.88:0/15176497 shutdown_connections 2023-10-16T20:05:18.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7ff06bfd9640 1 -- 172.21.15.88:0/15176497 wait complete. 2023-10-16T20:05:18.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7f0e72772640 1 -- 172.21.15.88:0/182457991 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0e6c06df80 msgr2=0x7f0e6c0701a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7f0e72772640 1 --2- 172.21.15.88:0/182457991 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0e6c06df80 0x7f0e6c0701a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0e60009870 tx=0x7f0e6001d450 comp rx=0 tx=0).stop 2023-10-16T20:05:18.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7f0e72772640 1 -- 172.21.15.88:0/182457991 shutdown_connections 2023-10-16T20:05:18.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7f0e72772640 1 --2- 172.21.15.88:0/182457991 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0e6c06df80 0x7f0e6c0701a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7f0e72772640 1 --2- 172.21.15.88:0/182457991 >> v2:172.21.15.94:3300/0 conn(0x7f0e6c158ba0 0x7f0e6c06da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.986+0000 7f0e72772640 1 -- 172.21.15.88:0/182457991 >> 172.21.15.88:0/182457991 conn(0x7f0e6c0cb060 msgr2=0x7f0e6c0cb470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:18.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7f0e72772640 1 -- 172.21.15.88:0/182457991 shutdown_connections 2023-10-16T20:05:18.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7f0e72772640 1 -- 172.21.15.88:0/182457991 wait complete. 2023-10-16T20:05:18.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7fbe5b59e640 1 --2- 172.21.15.88:0/3288064234 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe5c0cff40 0x7fbe5c0b7c00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c0725a0 tx=0x7fbe5001c500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb42565a13611987 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7f2636206640 1 Processor -- start 2023-10-16T20:05:18.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7fbe59d9b640 1 -- 172.21.15.88:0/3288064234 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe500ac410 con 0x7fbe5c0cff40 2023-10-16T20:05:18.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7fbe59d9b640 1 -- 172.21.15.88:0/3288064234 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe500ac570 con 0x7fbe5c0cff40 2023-10-16T20:05:18.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7fbe59d9b640 1 -- 172.21.15.88:0/3288064234 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe500ac900 con 0x7fbe5c0cff40 2023-10-16T20:05:18.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7ff06bfd9640 1 Processor -- start 2023-10-16T20:05:18.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7f2636206640 1 -- start start 2023-10-16T20:05:18.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7ff06bfd9640 1 -- start start 2023-10-16T20:05:18.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7f73baa95640 1 Processor -- start 2023-10-16T20:05:18.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7f0e72772640 1 Processor -- start 2023-10-16T20:05:18.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7fbe61079640 1 -- 172.21.15.88:0/3288064234 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe5c0cff40 msgr2=0x7fbe5c0b7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7fbe61079640 1 --2- 172.21.15.88:0/3288064234 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe5c0cff40 0x7fbe5c0b7c00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c0725a0 tx=0x7fbe5001c500 comp rx=0 tx=0).stop 2023-10-16T20:05:18.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7f0e72772640 1 -- start start 2023-10-16T20:05:18.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7fbe61079640 1 -- 172.21.15.88:0/3288064234 shutdown_connections 2023-10-16T20:05:18.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7fbe61079640 1 --2- 172.21.15.88:0/3288064234 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe5c0cff40 0x7fbe5c0b7c00 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7fbe61079640 1 --2- 172.21.15.88:0/3288064234 >> v2:172.21.15.94:3300/0 conn(0x7fbe5c0cdad0 0x7fbe5c0cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7fbe61079640 1 --2- 172.21.15.88:0/3288064234 >> v2:172.21.15.88:3300/0 conn(0x7fbe5c15cb70 0x7fbe5c161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7fbe61079640 1 -- 172.21.15.88:0/3288064234 >> 172.21.15.88:0/3288064234 conn(0x7fbe5c0719c0 msgr2=0x7fbe5c071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:18.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7fc0437fe640 1 --2- 172.21.15.88:0/4249779990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc02c0047d0 0x7fc02c004c50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc03400c340 tx=0x7fc03401cd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b687a633ad456f74 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7fbe61079640 1 -- 172.21.15.88:0/3288064234 shutdown_connections 2023-10-16T20:05:18.044 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7f73baa95640 1 -- start start 2023-10-16T20:05:18.044 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7fc042ffd640 1 -- 172.21.15.88:0/4249779990 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc034005ae0 con 0x7fc02c0047d0 2023-10-16T20:05:18.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7fc042ffd640 1 -- 172.21.15.88:0/4249779990 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc034005c40 con 0x7fc02c0047d0 2023-10-16T20:05:18.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7fc042ffd640 1 -- 172.21.15.88:0/4249779990 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0340a58c0 con 0x7fc02c0047d0 2023-10-16T20:05:18.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7ff06bfd9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0640d2660 0x7ff0640ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7ff06bfd9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff064153ea0 0x7ff0641474b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7ff06bfd9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0640c8560 0x7ff064145af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7ff06bfd9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0640c9150 con 0x7ff0640d2660 2023-10-16T20:05:18.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7ff06bfd9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff064143cd0 con 0x7ff0640c8560 2023-10-16T20:05:18.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7ff06bfd9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff064148b40 con 0x7ff064153ea0 2023-10-16T20:05:18.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7ff069574640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff064153ea0 0x7ff0641474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7ff069574640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff064153ea0 0x7ff0641474b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:32988/0 (socket says 172.21.15.88:32988) 2023-10-16T20:05:18.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7ff069574640 1 -- 172.21.15.88:0/2373031832 learned_addr learned my addr 172.21.15.88:0/2373031832 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:18.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f0e72772640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0e6c06df80 0x7f0e6c149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.048 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f2636206640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2628037d60 0x7f26280ea5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.048 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7fc04aa46640 1 -- 172.21.15.88:0/4249779990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc02c0047d0 msgr2=0x7fc02c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.048 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7fc04aa46640 1 --2- 172.21.15.88:0/4249779990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc02c0047d0 0x7fc02c004c50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc03400c340 tx=0x7fc03401cd00 comp rx=0 tx=0).stop 2023-10-16T20:05:18.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f0e72772640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e6c0706e0 0x7f0e6c1e2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f0e72772640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e6c158ba0 0x7f0e6c0acab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f0e72772640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c14a100 con 0x7f0e6c0706e0 2023-10-16T20:05:18.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f0e72772640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c14c810 con 0x7f0e6c158ba0 2023-10-16T20:05:18.050 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f0e72772640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c14caf0 con 0x7f0e6c06df80 2023-10-16T20:05:18.050 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.990+0000 7fbe61079640 1 -- 172.21.15.88:0/3288064234 wait complete. 2023-10-16T20:05:18.051 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f0e6b7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e6c0706e0 0x7f0e6c1e2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.051 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f0e6b7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e6c0706e0 0x7f0e6c1e2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40328/0 (socket says 172.21.15.88:40328) 2023-10-16T20:05:18.051 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f0e6b7fe640 1 -- 172.21.15.88:0/1585829960 learned_addr learned my addr 172.21.15.88:0/1585829960 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:18.051 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f2636206640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2628054480 0x7f26280da980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.052 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f2636206640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26280e78e0 0x7f26280d1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.052 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7ff069574640 1 -- 172.21.15.88:0/2373031832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0640c8560 msgr2=0x7ff064145af0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-16T20:05:18.052 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7ff069574640 1 --2- 172.21.15.88:0/2373031832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0640c8560 0x7ff064145af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.053 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7ff069574640 1 -- 172.21.15.88:0/2373031832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0640d2660 msgr2=0x7ff0640ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.053 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7ff069574640 1 --2- 172.21.15.88:0/2373031832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0640d2660 0x7ff0640ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.053 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7ff069574640 1 -- 172.21.15.88:0/2373031832 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0540094d0 con 0x7ff064153ea0 2023-10-16T20:05:18.053 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f2634a03640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2628054480 0x7f26280da980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.053 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f2636206640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26280e9a10 con 0x7f26280e78e0 2023-10-16T20:05:18.054 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f2636206640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26280e8e40 con 0x7f2628054480 2023-10-16T20:05:18.054 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f2636206640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26280e90e0 con 0x7f2628037d60 2023-10-16T20:05:18.054 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7fc04aa46640 1 -- 172.21.15.88:0/4249779990 shutdown_connections 2023-10-16T20:05:18.054 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7fc04aa46640 1 --2- 172.21.15.88:0/4249779990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc02c0047d0 0x7fc02c004c50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.055 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7fc04aa46640 1 -- 172.21.15.88:0/4249779990 >> 172.21.15.88:0/4249779990 conn(0x7fc0440ce5e0 msgr2=0x7fc0440ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:18.055 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f73baa95640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f73b4160180 0x7f73b40cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.055 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f73baa95640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f73b40d12d0 0x7f73b40cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.055 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f73baa95640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73b40b3df0 con 0x7f73b4160560 2023-10-16T20:05:18.056 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f73baa95640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73b40b4280 con 0x7f73b40d12d0 2023-10-16T20:05:18.056 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f73baa95640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b40d0220 con 0x7f73b4160180 2023-10-16T20:05:18.056 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7ff069574640 1 --2- 172.21.15.88:0/2373031832 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff064153ea0 0x7ff0641474b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff05c007e60 tx=0x7ff05c00e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.056 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7ff05a7fc640 1 -- 172.21.15.88:0/2373031832 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c014620 con 0x7ff064153ea0 2023-10-16T20:05:18.057 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7ff05a7fc640 1 -- 172.21.15.88:0/2373031832 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff05c00ee90 con 0x7ff064153ea0 2023-10-16T20:05:18.057 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7ff05a7fc640 1 -- 172.21.15.88:0/2373031832 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c014620 con 0x7ff064153ea0 2023-10-16T20:05:18.057 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7ff06bfd9640 1 -- 172.21.15.88:0/2373031832 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff064148e20 con 0x7ff064153ea0 2023-10-16T20:05:18.057 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f0e6b7fe640 1 -- 172.21.15.88:0/1585829960 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0e6c06df80 msgr2=0x7f0e6c149bc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-16T20:05:18.058 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f0e6b7fe640 1 --2- 172.21.15.88:0/1585829960 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0e6c06df80 0x7f0e6c149bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.058 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f0e6b7fe640 1 -- 172.21.15.88:0/1585829960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e6c158ba0 msgr2=0x7f0e6c0acab0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-16T20:05:18.058 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f0e6b7fe640 1 --2- 172.21.15.88:0/1585829960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e6c158ba0 0x7f0e6c0acab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.058 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f0e6b7fe640 1 -- 172.21.15.88:0/1585829960 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e600094d0 con 0x7f0e6c0706e0 2023-10-16T20:05:18.059 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f2635a05640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26280e78e0 0x7f26280d1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.059 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f2635a05640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26280e78e0 0x7f26280d1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40332/0 (socket says 172.21.15.88:40332) 2023-10-16T20:05:18.059 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f2635a05640 1 -- 172.21.15.88:0/2169257607 learned_addr learned my addr 172.21.15.88:0/2169257607 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:18.059 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f2635204640 1 --2- 172.21.15.88:0/2169257607 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2628037d60 0x7f26280ea5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.060 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7fc04aa46640 1 -- 172.21.15.88:0/4249779990 shutdown_connections 2023-10-16T20:05:18.060 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7fc04aa46640 1 -- 172.21.15.88:0/4249779990 wait complete. 2023-10-16T20:05:18.060 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7fbe61079640 1 Processor -- start 2023-10-16T20:05:18.061 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7fbe61079640 1 -- start start 2023-10-16T20:05:18.061 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f0e6b7fe640 1 --2- 172.21.15.88:0/1585829960 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e6c0706e0 0x7f0e6c1e2cf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0e6001b080 tx=0x7f0e6001a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.061 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f2634a03640 1 -- 172.21.15.88:0/2169257607 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2628037d60 msgr2=0x7f26280ea5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.062 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f2634a03640 1 --2- 172.21.15.88:0/2169257607 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2628037d60 0x7f26280ea5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.062 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f2634a03640 1 -- 172.21.15.88:0/2169257607 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26280e78e0 msgr2=0x7f26280d1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.062 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f2634a03640 1 --2- 172.21.15.88:0/2169257607 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26280e78e0 0x7f26280d1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.062 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f2634a03640 1 -- 172.21.15.88:0/2169257607 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f262c0094d0 con 0x7f2628054480 2023-10-16T20:05:18.063 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7ff06bfd9640 1 -- 172.21.15.88:0/2373031832 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff064149100 con 0x7ff064153ea0 2023-10-16T20:05:18.063 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7ff06bfd9640 1 -- 172.21.15.88:0/2373031832 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff0640b4620 con 0x7ff064153ea0 2023-10-16T20:05:18.063 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f0e68ff9640 1 -- 172.21.15.88:0/1585829960 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e600b7850 con 0x7f0e6c0706e0 2023-10-16T20:05:18.064 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7f0e68ff9640 1 -- 172.21.15.88:0/1585829960 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e600b79b0 con 0x7f0e6c0706e0 2023-10-16T20:05:18.064 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7f0e68ff9640 1 -- 172.21.15.88:0/1585829960 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e600b7d20 con 0x7f0e6c0706e0 2023-10-16T20:05:18.064 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7f0e72772640 1 -- 172.21.15.88:0/1585829960 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e6c14cdd0 con 0x7f0e6c0706e0 2023-10-16T20:05:18.064 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7f0e72772640 1 -- 172.21.15.88:0/1585829960 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e6c0b3520 con 0x7f0e6c0706e0 2023-10-16T20:05:18.065 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7f2634a03640 1 --2- 172.21.15.88:0/2169257607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2628054480 0x7f26280da980 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f262c0ab000 tx=0x7f262c01acb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.065 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7f2625ffb640 1 -- 172.21.15.88:0/2169257607 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c0bd070 con 0x7f2628054480 2023-10-16T20:05:18.065 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7fc04aa46640 1 Processor -- start 2023-10-16T20:05:18.066 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7fbe61079640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe5c0cdad0 0x7fbe5c15bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.066 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7fbe61079640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe5c0cff40 0x7fbe5c0b3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.066 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7fbe61079640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe5c15cb70 0x7fbe5c15eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f73ba294640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f73b40d12d0 0x7f73b40cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7f73ba294640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f73b40d12d0 0x7f73b40cd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:56790/0 (socket says 172.21.15.88:56790) 2023-10-16T20:05:18.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7f73ba294640 1 -- 172.21.15.88:0/2154350590 learned_addr learned my addr 172.21.15.88:0/2154350590 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:18.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7f2636206640 1 -- 172.21.15.88:0/2169257607 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26280e93c0 con 0x7f2628054480 2023-10-16T20:05:18.068 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f73b9a93640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f73b4160180 0x7f73b40cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.068 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7f73ba294640 1 -- 172.21.15.88:0/2154350590 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f73b4160180 msgr2=0x7f73b40cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:18.069 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.994+0000 7f73b9292640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f73b4160560 0x7f73b40d0bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:39200/0 (socket says 172.21.15.88:39200) 2023-10-16T20:05:18.069 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7f73ba294640 1 --2- 172.21.15.88:0/2154350590 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f73b4160180 0x7f73b40cfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.069 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7f73ba294640 1 -- 172.21.15.88:0/2154350590 >> v1:172.21.15.88:6789/0 conn(0x7f73b4160560 legacy=0x7f73b40d0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:18.069 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7f73ba294640 1 -- 172.21.15.88:0/2154350590 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b40b3af0 con 0x7f73b40d12d0 2023-10-16T20:05:18.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7fbe5ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe5c0cdad0 0x7fbe5c15bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7f73b9a93640 1 --2- 172.21.15.88:0/2154350590 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f73b4160180 0x7f73b40cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:18.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7fbe5ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe5c0cdad0 0x7fbe5c15bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:56794/0 (socket says 172.21.15.88:56794) 2023-10-16T20:05:18.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7fbe61079640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5c0c8dc0 con 0x7fbe5c0cff40 2023-10-16T20:05:18.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7fbe61079640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5c0c8930 con 0x7fbe5c0cdad0 2023-10-16T20:05:18.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7fbe61079640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5c15c630 con 0x7fbe5c15cb70 2023-10-16T20:05:18.072 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7fbe5ad9d640 1 -- 172.21.15.88:0/3570662209 learned_addr learned my addr 172.21.15.88:0/3570662209 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:18.072 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7fbe5b59e640 1 --2- 172.21.15.88:0/3570662209 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe5c15cb70 0x7fbe5c15eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.072 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7f0e72772640 1 -- 172.21.15.88:0/1585829960 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0e4c000fd0 con 0x7f0e6c0706e0 2023-10-16T20:05:18.072 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7fbe5b59e640 1 -- 172.21.15.88:0/3570662209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe5c0cdad0 msgr2=0x7fbe5c15bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.073 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7fbe5b59e640 1 --2- 172.21.15.88:0/3570662209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe5c0cdad0 0x7fbe5c15bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.073 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7fbe5b59e640 1 -- 172.21.15.88:0/3570662209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe5c0cff40 msgr2=0x7fbe5c0b3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.073 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7fbe5b59e640 1 --2- 172.21.15.88:0/3570662209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe5c0cff40 0x7fbe5c0b3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.073 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7fbe5b59e640 1 -- 172.21.15.88:0/3570662209 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe500094d0 con 0x7fbe5c15cb70 2023-10-16T20:05:18.074 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7ff05a7fc640 1 -- 172.21.15.88:0/2373031832 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff05c014780 con 0x7ff064153ea0 2023-10-16T20:05:18.074 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7fbe5b59e640 1 --2- 172.21.15.88:0/3570662209 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe5c15cb70 0x7fbe5c15eef0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c0d0d80 tx=0x7fbe500192a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.075 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7fbe437fe640 1 -- 172.21.15.88:0/3570662209 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe500ac410 con 0x7fbe5c15cb70 2023-10-16T20:05:18.075 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7fbe437fe640 1 -- 172.21.15.88:0/3570662209 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe500ace20 con 0x7fbe5c15cb70 2023-10-16T20:05:18.075 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7fbe437fe640 1 -- 172.21.15.88:0/3570662209 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe500b4810 con 0x7fbe5c15cb70 2023-10-16T20:05:18.075 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7fbe61079640 1 -- 172.21.15.88:0/3570662209 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe5c0c8000 con 0x7fbe5c15cb70 2023-10-16T20:05:18.075 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7fbe61079640 1 -- 172.21.15.88:0/3570662209 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe5c0c85c0 con 0x7fbe5c15cb70 2023-10-16T20:05:18.076 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7fc04aa46640 1 -- start start 2023-10-16T20:05:18.076 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7fc04aa46640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc02c0047d0 0x7fc04415e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.076 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7fc04aa46640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc0441fcd90 0x7fc0441fd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.076 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7fc04aa46640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc044162710 0x7fc044162b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.077 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7ff05a7fc640 1 --2- 172.21.15.88:0/2373031832 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff048066500 0x7ff0480689c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.077 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.002+0000 7ff05a7fc640 1 -- 172.21.15.88:0/2373031832 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff05c083dd0 con 0x7ff064153ea0 2023-10-16T20:05:18.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.002+0000 7fc0437fe640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc0441fcd90 0x7fc0441fd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.002+0000 7fc0437fe640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc0441fcd90 0x7fc0441fd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:33024/0 (socket says 172.21.15.88:33024) 2023-10-16T20:05:18.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.002+0000 7fc0437fe640 1 -- 172.21.15.88:0/3343873656 learned_addr learned my addr 172.21.15.88:0/3343873656 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:18.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.002+0000 7fc04aa46640 1 -- 172.21.15.88:0/3343873656 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0441fd750 con 0x7fc044162710 2023-10-16T20:05:18.079 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.002+0000 7fc04aa46640 1 -- 172.21.15.88:0/3343873656 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0441633b0 con 0x7fc02c0047d0 2023-10-16T20:05:18.079 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.002+0000 7ff05a7fc640 1 --2- 172.21.15.88:0/2373031832 >> [v2:172.21.15.88:6816/3866598902,v1:172.21.15.88:6817/3866598902] conn(0x7ff04806fac0 0x7ff048071f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.079 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.002+0000 7ff069d75640 1 --2- 172.21.15.88:0/2373031832 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff048066500 0x7ff0480689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.079 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.002+0000 7ff05a7fc640 1 -- 172.21.15.88:0/2373031832 --> [v2:172.21.15.88:6816/3866598902,v1:172.21.15.88:6817/3866598902] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff0480725d0 con 0x7ff04806fac0 2023-10-16T20:05:18.080 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.002+0000 7ff06a576640 1 --2- 172.21.15.88:0/2373031832 >> [v2:172.21.15.88:6816/3866598902,v1:172.21.15.88:6817/3866598902] conn(0x7ff04806fac0 0x7ff048071f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.080 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.002+0000 7fc04aa46640 1 -- 172.21.15.88:0/3343873656 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc044163690 con 0x7fc0441fcd90 2023-10-16T20:05:18.080 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.002+0000 7ff05a7fc640 1 -- 172.21.15.88:0/2373031832 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff05c084090 con 0x7ff064153ea0 2023-10-16T20:05:18.081 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7f2636206640 1 -- 172.21.15.88:0/2169257607 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26280daec0 con 0x7f2628054480 2023-10-16T20:05:18.081 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7f2625ffb640 1 -- 172.21.15.88:0/2169257607 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262c0ac620 con 0x7f2628054480 2023-10-16T20:05:18.081 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:17.998+0000 7f2625ffb640 1 -- 172.21.15.88:0/2169257607 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c0b7850 con 0x7f2628054480 2023-10-16T20:05:18.081 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.002+0000 7f2636206640 1 -- 172.21.15.88:0/2169257607 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2610000fd0 con 0x7f2628054480 2023-10-16T20:05:18.082 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.002+0000 7ff069d75640 1 --2- 172.21.15.88:0/2373031832 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff048066500 0x7ff0480689c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff06415f850 tx=0x7ff060008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.082 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.002+0000 7fc0437fe640 1 -- 172.21.15.88:0/3343873656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc02c0047d0 msgr2=0x7fc04415e930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T20:05:18.082 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.002+0000 7fc0437fe640 1 --2- 172.21.15.88:0/3343873656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc02c0047d0 0x7fc04415e930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.002+0000 7fc0437fe640 1 -- 172.21.15.88:0/3343873656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc044162710 msgr2=0x7fc044162b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.002+0000 7fc0437fe640 1 --2- 172.21.15.88:0/3343873656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc044162710 0x7fc044162b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.002+0000 7f73ba294640 1 --2- 172.21.15.88:0/2154350590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73b40d12d0 0x7f73b40cd4a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f73ac009e50 tx=0x7f73ac01c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b23136cab157688 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.002+0000 7fc0437fe640 1 -- 172.21.15.88:0/3343873656 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc044163970 con 0x7fc0441fcd90 2023-10-16T20:05:18.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.002+0000 7f0e68ff9640 1 -- 172.21.15.88:0/1585829960 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0e600ab070 con 0x7f0e6c0706e0 2023-10-16T20:05:18.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.002+0000 7f2625ffb640 1 -- 172.21.15.88:0/2169257607 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f262c0b3040 con 0x7f2628054480 2023-10-16T20:05:18.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.002+0000 7fc0437fe640 1 --2- 172.21.15.88:0/3343873656 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc0441fcd90 0x7fc0441fd210 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc034001940 tx=0x7fc02c005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.002+0000 7fc040ff9640 1 -- 172.21.15.88:0/3343873656 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0340b4070 con 0x7fc0441fcd90 2023-10-16T20:05:18.086 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.002+0000 7fbe437fe640 1 -- 172.21.15.88:0/3570662209 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbe500b3070 con 0x7fbe5c15cb70 2023-10-16T20:05:18.086 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.002+0000 7ff06a576640 1 --2- 172.21.15.88:0/2373031832 >> [v2:172.21.15.88:6816/3866598902,v1:172.21.15.88:6817/3866598902] conn(0x7ff04806fac0 0x7ff048071f00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.087 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.002+0000 7fc04aa46640 1 -- 172.21.15.88:0/3343873656 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc044163c50 con 0x7fc0441fcd90 2023-10-16T20:05:18.087 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.002+0000 7fc04aa46640 1 -- 172.21.15.88:0/3343873656 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0441630d0 con 0x7fc0441fcd90 2023-10-16T20:05:18.087 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.002+0000 7fc040ff9640 1 -- 172.21.15.88:0/3343873656 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc034001500 con 0x7fc0441fcd90 2023-10-16T20:05:18.087 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7f2625ffb640 1 --2- 172.21.15.88:0/2169257607 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f26140664a0 0x7f2614068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.088 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7f0e68ff9640 1 --2- 172.21.15.88:0/1585829960 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0e580664f0 0x7f0e580689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.088 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7f2625ffb640 1 -- 172.21.15.88:0/2169257607 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f262c11d8d0 con 0x7f2628054480 2023-10-16T20:05:18.089 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7fc040ff9640 1 -- 172.21.15.88:0/3343873656 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0340b4070 con 0x7fc0441fcd90 2023-10-16T20:05:18.089 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7f73b8a91640 1 -- 172.21.15.88:0/2154350590 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ac0a57b0 con 0x7f73b40d12d0 2023-10-16T20:05:18.089 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7f2625ffb640 1 --2- 172.21.15.88:0/2169257607 >> [v2:172.21.15.94:6808/722063332,v1:172.21.15.94:6809/722063332] conn(0x7f261406fa60 0x7f2614071ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.090 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7fbe61079640 1 -- 172.21.15.88:0/3570662209 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbe3c000fd0 con 0x7fbe5c15cb70 2023-10-16T20:05:18.090 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7f2635204640 1 --2- 172.21.15.88:0/2169257607 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f26140664a0 0x7f2614068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.090 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7f2625ffb640 1 -- 172.21.15.88:0/2169257607 --> [v2:172.21.15.94:6808/722063332,v1:172.21.15.94:6809/722063332] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2614072570 con 0x7f261406fa60 2023-10-16T20:05:18.091 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7f2625ffb640 1 -- 172.21.15.88:0/2169257607 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f262c0eaae0 con 0x7f2628054480 2023-10-16T20:05:18.091 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7fbe437fe640 1 --2- 172.21.15.88:0/3570662209 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbe48066520 0x7fbe480689e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.091 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7f73b8a91640 1 -- 172.21.15.88:0/2154350590 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73ac0a5910 con 0x7f73b40d12d0 2023-10-16T20:05:18.092 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7f73b8a91640 1 -- 172.21.15.88:0/2154350590 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ac0a5ca0 con 0x7f73b40d12d0 2023-10-16T20:05:18.092 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7f2635a05640 1 --2- 172.21.15.88:0/2169257607 >> [v2:172.21.15.94:6808/722063332,v1:172.21.15.94:6809/722063332] conn(0x7f261406fa60 0x7f2614071ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.092 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7f0e6bfff640 1 --2- 172.21.15.88:0/1585829960 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0e580664f0 0x7f0e580689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.093 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7fbe437fe640 1 -- 172.21.15.88:0/3570662209 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbe5011dcf0 con 0x7fbe5c15cb70 2023-10-16T20:05:18.093 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7f73baa95640 1 -- 172.21.15.88:0/2154350590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73b40d12d0 msgr2=0x7f73b40cd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.093 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7f73baa95640 1 --2- 172.21.15.88:0/2154350590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73b40d12d0 0x7f73b40cd4a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f73ac009e50 tx=0x7f73ac01c910 comp rx=0 tx=0).stop 2023-10-16T20:05:18.094 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7f2635a05640 1 --2- 172.21.15.88:0/2169257607 >> [v2:172.21.15.94:6808/722063332,v1:172.21.15.94:6809/722063332] conn(0x7f261406fa60 0x7f2614071ea0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.094 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7f73baa95640 1 -- 172.21.15.88:0/2154350590 shutdown_connections 2023-10-16T20:05:18.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7f73baa95640 1 --2- 172.21.15.88:0/2154350590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73b40d12d0 0x7f73b40cd4a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7fbe437fe640 1 --2- 172.21.15.88:0/3570662209 >> [v2:172.21.15.94:6816/175389267,v1:172.21.15.94:6817/175389267] conn(0x7fbe4806fae0 0x7fbe48071f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7fbe437fe640 1 -- 172.21.15.88:0/3570662209 --> [v2:172.21.15.94:6816/175389267,v1:172.21.15.94:6817/175389267] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbe480725f0 con 0x7fbe4806fae0 2023-10-16T20:05:18.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7fbe437fe640 1 -- 172.21.15.88:0/3570662209 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe500eae60 con 0x7fbe5c15cb70 2023-10-16T20:05:18.096 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7fbe5ad9d640 1 --2- 172.21.15.88:0/3570662209 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbe48066520 0x7fbe480689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.096 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7f0e68ff9640 1 -- 172.21.15.88:0/1585829960 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0e600f02e0 con 0x7f0e6c0706e0 2023-10-16T20:05:18.097 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7f73baa95640 1 --2- 172.21.15.88:0/2154350590 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f73b4160180 0x7f73b40cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.097 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7f73baa95640 1 -- 172.21.15.88:0/2154350590 >> 172.21.15.88:0/2154350590 conn(0x7f73b4089fb0 msgr2=0x7f73b40d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:18.097 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7f73baa95640 1 -- 172.21.15.88:0/2154350590 shutdown_connections 2023-10-16T20:05:18.097 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7f73baa95640 1 -- 172.21.15.88:0/2154350590 wait complete. 2023-10-16T20:05:18.098 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7f0e68ff9640 1 --2- 172.21.15.88:0/1585829960 >> [v2:172.21.15.88:6824/1777716082,v1:172.21.15.88:6825/1777716082] conn(0x7f0e5806fb30 0x7f0e58071f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.098 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7f0e68ff9640 1 -- 172.21.15.88:0/1585829960 --> [v2:172.21.15.88:6824/1777716082,v1:172.21.15.88:6825/1777716082] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0e58072640 con 0x7f0e5806fb30 2023-10-16T20:05:18.098 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7f0e68ff9640 1 -- 172.21.15.88:0/1585829960 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e600ebf20 con 0x7f0e6c0706e0 2023-10-16T20:05:18.098 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7f0e70d0f640 1 --2- 172.21.15.88:0/1585829960 >> [v2:172.21.15.88:6824/1777716082,v1:172.21.15.88:6825/1777716082] conn(0x7f0e5806fb30 0x7f0e58071f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.099 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7fbe5a59c640 1 --2- 172.21.15.88:0/3570662209 >> [v2:172.21.15.94:6816/175389267,v1:172.21.15.94:6817/175389267] conn(0x7fbe4806fae0 0x7fbe48071f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.099 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7f2625ffb640 1 -- 172.21.15.88:0/2169257607 <== osd.4 v2:172.21.15.94:6808/722063332 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f2614072570 con 0x7f261406fa60 2023-10-16T20:05:18.099 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7f0e6bfff640 1 --2- 172.21.15.88:0/1585829960 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0e580664f0 0x7f0e580689b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c155010 tx=0x7f0e5c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.100 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7f2635204640 1 --2- 172.21.15.88:0/2169257607 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f26140664a0 0x7f2614068960 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f26200098e0 tx=0x7f2620009060 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.100 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.010+0000 7f0e70d0f640 1 --2- 172.21.15.88:0/1585829960 >> [v2:172.21.15.88:6824/1777716082,v1:172.21.15.88:6825/1777716082] conn(0x7f0e5806fb30 0x7f0e58071f70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.010+0000 7fbe5ad9d640 1 --2- 172.21.15.88:0/3570662209 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbe48066520 0x7fbe480689e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbe54003b40 tx=0x7fbe540098b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.010+0000 7fbe5a59c640 1 --2- 172.21.15.88:0/3570662209 >> [v2:172.21.15.94:6816/175389267,v1:172.21.15.94:6817/175389267] conn(0x7fbe4806fae0 0x7fbe48071f20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.006+0000 7fc040ff9640 1 -- 172.21.15.88:0/3343873656 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc0340a58c0 con 0x7fc0441fcd90 2023-10-16T20:05:18.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.010+0000 7fc04aa46640 1 -- 172.21.15.88:0/3343873656 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc024000fd0 con 0x7fc0441fcd90 2023-10-16T20:05:18.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.010+0000 7fc040ff9640 1 --2- 172.21.15.88:0/3343873656 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc030066590 0x7fc030068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.010+0000 7f0e68ff9640 1 -- 172.21.15.88:0/1585829960 <== osd.3 v2:172.21.15.88:6824/1777716082 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f0e58072640 con 0x7f0e5806fb30 2023-10-16T20:05:18.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.010+0000 7f73baa95640 1 Processor -- start 2023-10-16T20:05:18.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.010+0000 7ff05a7fc640 1 -- 172.21.15.88:0/2373031832 <== osd.2 v2:172.21.15.88:6816/3866598902 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7ff0480725d0 con 0x7ff04806fac0 2023-10-16T20:05:18.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.010+0000 7f73baa95640 1 -- start start 2023-10-16T20:05:18.104 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.010+0000 7fbe437fe640 1 -- 172.21.15.88:0/3570662209 <== osd.6 v2:172.21.15.94:6816/175389267 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fbe480725f0 con 0x7fbe4806fae0 2023-10-16T20:05:18.105 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.010+0000 7fc043fff640 1 --2- 172.21.15.88:0/3343873656 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc030066590 0x7fc030068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.105 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.010+0000 7fc040ff9640 1 -- 172.21.15.88:0/3343873656 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc03411d150 con 0x7fc0441fcd90 2023-10-16T20:05:18.105 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.010+0000 7fc043fff640 1 --2- 172.21.15.88:0/3343873656 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc030066590 0x7fc030068a50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc03c00d4a0 tx=0x7fc03c007730 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.105 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.010+0000 7fc040ff9640 1 --2- 172.21.15.88:0/3343873656 >> [v2:172.21.15.94:6800/3097673529,v1:172.21.15.94:6801/3097673529] conn(0x7fc03006fb50 0x7fc030071f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.105 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.010+0000 7fc040ff9640 1 -- 172.21.15.88:0/3343873656 --> [v2:172.21.15.94:6800/3097673529,v1:172.21.15.94:6801/3097673529] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc030072660 con 0x7fc03006fb50 2023-10-16T20:05:18.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.010+0000 7fc040ff9640 1 -- 172.21.15.88:0/3343873656 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0340b4070 con 0x7fc0441fcd90 2023-10-16T20:05:18.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.010+0000 7fc048fe3640 1 --2- 172.21.15.88:0/3343873656 >> [v2:172.21.15.94:6800/3097673529,v1:172.21.15.94:6801/3097673529] conn(0x7fc03006fb50 0x7fc030071f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.010+0000 7f73baa95640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73b40d12d0 0x7f73b414f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.010+0000 7f73baa95640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73b4160180 0x7f73b414fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.014+0000 7f73b9a93640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73b40d12d0 0x7f73b414f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.014+0000 7f73b9a93640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73b40d12d0 0x7f73b414f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:56808/0 (socket says 172.21.15.88:56808) 2023-10-16T20:05:18.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.014+0000 7fc048fe3640 1 --2- 172.21.15.88:0/3343873656 >> [v2:172.21.15.94:6800/3097673529,v1:172.21.15.94:6801/3097673529] conn(0x7fc03006fb50 0x7fc030071f90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.108 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.010+0000 7f73baa95640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f73b4160560 0x7f73b40b7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.108 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.014+0000 7f73b9a93640 1 -- 172.21.15.88:0/2982366610 learned_addr learned my addr 172.21.15.88:0/2982366610 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:18.108 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.014+0000 7f73baa95640 1 -- 172.21.15.88:0/2982366610 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b4157e90 con 0x7f73b4160180 2023-10-16T20:05:18.109 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.014+0000 7f73baa95640 1 -- 172.21.15.88:0/2982366610 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b40c80b0 con 0x7f73b40d12d0 2023-10-16T20:05:18.109 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.014+0000 7f73baa95640 1 -- 172.21.15.88:0/2982366610 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b4152220 con 0x7f73b4160560 2023-10-16T20:05:18.109 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.014+0000 7f73b9292640 1 --2- 172.21.15.88:0/2982366610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73b4160180 0x7f73b414fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.109 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.014+0000 7f73ba294640 1 --2- 172.21.15.88:0/2982366610 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f73b4160560 0x7f73b40b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.110 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.014+0000 7fc040ff9640 1 -- 172.21.15.88:0/3343873656 <== osd.5 v2:172.21.15.94:6800/3097673529 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fc030072660 con 0x7fc03006fb50 2023-10-16T20:05:18.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.014+0000 7f73b9a93640 1 -- 172.21.15.88:0/2982366610 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f73b4160560 msgr2=0x7f73b40b7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.014+0000 7f73b9a93640 1 --2- 172.21.15.88:0/2982366610 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f73b4160560 0x7f73b40b7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.014+0000 7f73b9a93640 1 -- 172.21.15.88:0/2982366610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73b4160180 msgr2=0x7f73b414fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.014+0000 7f73b9a93640 1 --2- 172.21.15.88:0/2982366610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73b4160180 0x7f73b414fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.014+0000 7f73b9a93640 1 -- 172.21.15.88:0/2982366610 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ac009b30 con 0x7f73b40d12d0 2023-10-16T20:05:18.112 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.014+0000 7f73ba294640 1 --2- 172.21.15.88:0/2982366610 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f73b4160560 0x7f73b40b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:05:18.112 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.014+0000 7f73b9292640 1 --2- 172.21.15.88:0/2982366610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73b4160180 0x7f73b414fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:18.112 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.014+0000 7faccca79640 1 -- 172.21.15.88:0/1732139448 --> [v2:172.21.15.88:6808/2212808191,v1:172.21.15.88:6809/2212808191] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7facc003a1c0 con 0x7faca806fab0 2023-10-16T20:05:18.113 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.014+0000 7f73b9a93640 1 --2- 172.21.15.88:0/2982366610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73b40d12d0 0x7f73b414f910 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f73b000c820 tx=0x7f73b000cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.113 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.018+0000 7f73aa7fc640 1 -- 172.21.15.88:0/2982366610 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b0007a30 con 0x7f73b40d12d0 2023-10-16T20:05:18.113 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.018+0000 7f73aa7fc640 1 -- 172.21.15.88:0/2982366610 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b000ced0 con 0x7f73b40d12d0 2023-10-16T20:05:18.113 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.018+0000 7facb6ffd640 1 -- 172.21.15.88:0/1732139448 <== osd.1 v2:172.21.15.88:6808/2212808191 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7facc8069e90 con 0x7faca806fab0 2023-10-16T20:05:18.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.018+0000 7f73baa95640 1 -- 172.21.15.88:0/2982366610 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73b40c7780 con 0x7f73b40d12d0 2023-10-16T20:05:18.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.018+0000 7f73aa7fc640 1 -- 172.21.15.88:0/2982366610 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b0007dd0 con 0x7f73b40d12d0 2023-10-16T20:05:18.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.018+0000 7f73baa95640 1 -- 172.21.15.88:0/2982366610 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b40c7d40 con 0x7f73b40d12d0 2023-10-16T20:05:18.115 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.018+0000 7f73baa95640 1 -- 172.21.15.88:0/2982366610 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f739c000fd0 con 0x7f73b40d12d0 2023-10-16T20:05:18.115 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.018+0000 7f73aa7fc640 1 -- 172.21.15.88:0/2982366610 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f73b00113f0 con 0x7f73b40d12d0 2023-10-16T20:05:18.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.022+0000 7faccca79640 1 -- 172.21.15.88:0/1732139448 >> [v2:172.21.15.88:6808/2212808191,v1:172.21.15.88:6809/2212808191] conn(0x7faca806fab0 msgr2=0x7faca8071ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.022+0000 7faccca79640 1 --2- 172.21.15.88:0/1732139448 >> [v2:172.21.15.88:6808/2212808191,v1:172.21.15.88:6809/2212808191] conn(0x7faca806fab0 0x7faca8071ef0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.022+0000 7faccca79640 1 -- 172.21.15.88:0/1732139448 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7faca80664f0 msgr2=0x7faca80689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.022+0000 7faccca79640 1 --2- 172.21.15.88:0/1732139448 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7faca80664f0 0x7faca80689b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7facc00d9e10 tx=0x7facbc009190 comp rx=0 tx=0).stop 2023-10-16T20:05:18.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.022+0000 7faccca79640 1 -- 172.21.15.88:0/1732139448 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7facc00061b0 msgr2=0x7facc00e45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.117 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.022+0000 7faccca79640 1 --2- 172.21.15.88:0/1732139448 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7facc00061b0 0x7facc00e45b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7facb80bb000 tx=0x7facb802bd80 comp rx=0 tx=0).stop 2023-10-16T20:05:18.117 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.022+0000 7facc6d9d640 1 -- 172.21.15.88:0/1732139448 reap_dead start 2023-10-16T20:05:18.117 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.022+0000 7faccca79640 1 -- 172.21.15.88:0/1732139448 shutdown_connections 2023-10-16T20:05:18.117 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.022+0000 7f73aa7fc640 1 --2- 172.21.15.88:0/2982366610 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f73980664f0 0x7f73980689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.022+0000 7faccca79640 1 -- 172.21.15.88:0/1732139448 >> 172.21.15.88:0/1732139448 conn(0x7facc0009fa0 msgr2=0x7facc004ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:18.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.022+0000 7faccca79640 1 -- 172.21.15.88:0/1732139448 shutdown_connections 2023-10-16T20:05:18.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.022+0000 7faccca79640 1 -- 172.21.15.88:0/1732139448 wait complete. 2023-10-16T20:05:18.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.022+0000 7f73aa7fc640 1 -- 172.21.15.88:0/2982366610 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f73b00841b0 con 0x7f73b40d12d0 2023-10-16T20:05:18.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.022+0000 7f73aa7fc640 1 --2- 172.21.15.88:0/2982366610 >> [v2:172.21.15.94:6824/1057220797,v1:172.21.15.94:6825/1057220797] conn(0x7f739806fb30 0x7f7398071f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.022+0000 7f73aa7fc640 1 -- 172.21.15.88:0/2982366610 --> [v2:172.21.15.94:6824/1057220797,v1:172.21.15.94:6825/1057220797] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7398072640 con 0x7f739806fb30 2023-10-16T20:05:18.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.022+0000 7f73aa7fc640 1 -- 172.21.15.88:0/2982366610 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73b0086790 con 0x7f73b40d12d0 2023-10-16T20:05:18.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.022+0000 7f73b9292640 1 --2- 172.21.15.88:0/2982366610 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f73980664f0 0x7f73980689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.022+0000 7f73ba294640 1 --2- 172.21.15.88:0/2982366610 >> [v2:172.21.15.94:6824/1057220797,v1:172.21.15.94:6825/1057220797] conn(0x7f739806fb30 0x7f7398071f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.026+0000 7f73b9292640 1 --2- 172.21.15.88:0/2982366610 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f73980664f0 0x7f73980689b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f73b4167c90 tx=0x7f73a4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.026+0000 7f73ba294640 1 --2- 172.21.15.88:0/2982366610 >> [v2:172.21.15.94:6824/1057220797,v1:172.21.15.94:6825/1057220797] conn(0x7f739806fb30 0x7f7398071f70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.026+0000 7f2274679640 1 -- 172.21.15.88:0/3511215215 --> [v2:172.21.15.88:6800/685586021,v1:172.21.15.88:6801/685586021] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f22640580e0 con 0x7f225006fb30 2023-10-16T20:05:18.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.026+0000 7f73aa7fc640 1 -- 172.21.15.88:0/2982366610 <== osd.7 v2:172.21.15.94:6824/1057220797 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f7398072640 con 0x7f739806fb30 2023-10-16T20:05:18.122 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.030+0000 7f2262ffd640 1 -- 172.21.15.88:0/3511215215 <== osd.0 v2:172.21.15.88:6800/685586021 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f225c0b0870 con 0x7f225006fb30 2023-10-16T20:05:18.123 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.030+0000 7f2274679640 1 -- 172.21.15.88:0/3511215215 >> [v2:172.21.15.88:6800/685586021,v1:172.21.15.88:6801/685586021] conn(0x7f225006fb30 msgr2=0x7f2250071f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.123 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.030+0000 7f2274679640 1 --2- 172.21.15.88:0/3511215215 >> [v2:172.21.15.88:6800/685586021,v1:172.21.15.88:6801/685586021] conn(0x7f225006fb30 0x7f2250071f70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.123 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.030+0000 7f2274679640 1 -- 172.21.15.88:0/3511215215 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f22500664f0 msgr2=0x7f22500689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.123 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.030+0000 7f2274679640 1 --2- 172.21.15.88:0/3511215215 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f22500664f0 0x7f22500689b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f22640e0750 tx=0x7f2268009190 comp rx=0 tx=0).stop 2023-10-16T20:05:18.123 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.030+0000 7f2274679640 1 -- 172.21.15.88:0/3511215215 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2264006050 msgr2=0x7f226404c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.124 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.030+0000 7f2274679640 1 --2- 172.21.15.88:0/3511215215 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2264006050 0x7f226404c9f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f226c000b70 tx=0x7f226c04f1c0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.124 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.030+0000 7f2272c16640 1 -- 172.21.15.88:0/3511215215 reap_dead start 2023-10-16T20:05:18.124 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.030+0000 7f2274679640 1 -- 172.21.15.88:0/3511215215 shutdown_connections 2023-10-16T20:05:18.124 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.030+0000 7f2274679640 1 -- 172.21.15.88:0/3511215215 >> 172.21.15.88:0/3511215215 conn(0x7f2264058ec0 msgr2=0x7f2264056d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:18.124 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.030+0000 7f2274679640 1 -- 172.21.15.88:0/3511215215 shutdown_connections 2023-10-16T20:05:18.124 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.030+0000 7f2274679640 1 -- 172.21.15.88:0/3511215215 wait complete. 2023-10-16T20:05:18.125 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.030+0000 7fbe61079640 1 -- 172.21.15.88:0/3570662209 --> [v2:172.21.15.94:6816/175389267,v1:172.21.15.94:6817/175389267] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbe3c002cc0 con 0x7fbe4806fae0 2023-10-16T20:05:18.125 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.034+0000 7fbe437fe640 1 -- 172.21.15.88:0/3570662209 <== osd.6 v2:172.21.15.94:6816/175389267 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbe3c002cc0 con 0x7fbe4806fae0 2023-10-16T20:05:18.126 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.034+0000 7fbe61079640 1 -- 172.21.15.88:0/3570662209 >> [v2:172.21.15.94:6816/175389267,v1:172.21.15.94:6817/175389267] conn(0x7fbe4806fae0 msgr2=0x7fbe48071f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.126 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.034+0000 7fbe61079640 1 --2- 172.21.15.88:0/3570662209 >> [v2:172.21.15.94:6816/175389267,v1:172.21.15.94:6817/175389267] conn(0x7fbe4806fae0 0x7fbe48071f20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.126 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.034+0000 7fbe61079640 1 -- 172.21.15.88:0/3570662209 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbe48066520 msgr2=0x7fbe480689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.127 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.034+0000 7fbe61079640 1 --2- 172.21.15.88:0/3570662209 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbe48066520 0x7fbe480689e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbe54003b40 tx=0x7fbe540098b0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.127 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.034+0000 7fbe61079640 1 -- 172.21.15.88:0/3570662209 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe5c15cb70 msgr2=0x7fbe5c15eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.127 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.034+0000 7fbe61079640 1 --2- 172.21.15.88:0/3570662209 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe5c15cb70 0x7fbe5c15eef0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c0d0d80 tx=0x7fbe500192a0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.127 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.038+0000 7fbe5b59e640 1 -- 172.21.15.88:0/3570662209 reap_dead start 2023-10-16T20:05:18.128 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.038+0000 7fc04aa46640 1 -- 172.21.15.88:0/3343873656 --> [v2:172.21.15.94:6800/3097673529,v1:172.21.15.94:6801/3097673529] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc024002cc0 con 0x7fc03006fb50 2023-10-16T20:05:18.128 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.038+0000 7fbe61079640 1 -- 172.21.15.88:0/3570662209 shutdown_connections 2023-10-16T20:05:18.128 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.038+0000 7fbe61079640 1 -- 172.21.15.88:0/3570662209 >> 172.21.15.88:0/3570662209 conn(0x7fbe5c0719c0 msgr2=0x7fbe5c072790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:18.129 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.038+0000 7f2636206640 1 -- 172.21.15.88:0/2169257607 --> [v2:172.21.15.94:6808/722063332,v1:172.21.15.94:6809/722063332] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2610002cc0 con 0x7f261406fa60 2023-10-16T20:05:18.129 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.038+0000 7fc040ff9640 1 -- 172.21.15.88:0/3343873656 <== osd.5 v2:172.21.15.94:6800/3097673529 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc024002cc0 con 0x7fc03006fb50 2023-10-16T20:05:18.130 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.038+0000 7fbe61079640 1 -- 172.21.15.88:0/3570662209 shutdown_connections 2023-10-16T20:05:18.130 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.038+0000 7fbe61079640 1 -- 172.21.15.88:0/3570662209 wait complete. 2023-10-16T20:05:18.130 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7f2625ffb640 1 -- 172.21.15.88:0/2169257607 <== osd.4 v2:172.21.15.94:6808/722063332 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2610002cc0 con 0x7f261406fa60 2023-10-16T20:05:18.131 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7f0e72772640 1 -- 172.21.15.88:0/1585829960 --> [v2:172.21.15.88:6824/1777716082,v1:172.21.15.88:6825/1777716082] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0e4c002cc0 con 0x7f0e5806fb30 2023-10-16T20:05:18.131 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7f261b7fe640 1 -- 172.21.15.88:0/2169257607 >> [v2:172.21.15.94:6808/722063332,v1:172.21.15.94:6809/722063332] conn(0x7f261406fa60 msgr2=0x7f2614071ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.131 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7f261b7fe640 1 --2- 172.21.15.88:0/2169257607 >> [v2:172.21.15.94:6808/722063332,v1:172.21.15.94:6809/722063332] conn(0x7f261406fa60 0x7f2614071ea0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.132 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7fc04aa46640 1 -- 172.21.15.88:0/3343873656 >> [v2:172.21.15.94:6800/3097673529,v1:172.21.15.94:6801/3097673529] conn(0x7fc03006fb50 msgr2=0x7fc030071f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.132 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7fc04aa46640 1 --2- 172.21.15.88:0/3343873656 >> [v2:172.21.15.94:6800/3097673529,v1:172.21.15.94:6801/3097673529] conn(0x7fc03006fb50 0x7fc030071f90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.132 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7f261b7fe640 1 -- 172.21.15.88:0/2169257607 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f26140664a0 msgr2=0x7f2614068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7f261b7fe640 1 --2- 172.21.15.88:0/2169257607 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f26140664a0 0x7f2614068960 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f26200098e0 tx=0x7f2620009060 comp rx=0 tx=0).stop 2023-10-16T20:05:18.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7fc04aa46640 1 -- 172.21.15.88:0/3343873656 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc030066590 msgr2=0x7fc030068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7fc04aa46640 1 --2- 172.21.15.88:0/3343873656 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc030066590 0x7fc030068a50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc03c00d4a0 tx=0x7fc03c007730 comp rx=0 tx=0).stop 2023-10-16T20:05:18.134 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7f261b7fe640 1 -- 172.21.15.88:0/2169257607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2628054480 msgr2=0x7f26280da980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.134 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7f261b7fe640 1 --2- 172.21.15.88:0/2169257607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2628054480 0x7f26280da980 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f262c0ab000 tx=0x7f262c01acb0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.134 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7fc04aa46640 1 -- 172.21.15.88:0/3343873656 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc0441fcd90 msgr2=0x7fc0441fd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7f2635a05640 1 -- 172.21.15.88:0/2169257607 reap_dead start 2023-10-16T20:05:18.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7fc04aa46640 1 --2- 172.21.15.88:0/3343873656 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc0441fcd90 0x7fc0441fd210 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc034001940 tx=0x7fc02c005800 comp rx=0 tx=0).stop 2023-10-16T20:05:18.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7ff06bfd9640 1 -- 172.21.15.88:0/2373031832 --> [v2:172.21.15.88:6816/3866598902,v1:172.21.15.88:6817/3866598902] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff0641441b0 con 0x7ff04806fac0 2023-10-16T20:05:18.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7f261b7fe640 1 -- 172.21.15.88:0/2169257607 shutdown_connections 2023-10-16T20:05:18.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7f261b7fe640 1 -- 172.21.15.88:0/2169257607 >> 172.21.15.88:0/2169257607 conn(0x7f2628054bc0 msgr2=0x7f2628054fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:18.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7f261b7fe640 1 -- 172.21.15.88:0/2169257607 shutdown_connections 2023-10-16T20:05:18.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7fc048fe3640 1 -- 172.21.15.88:0/3343873656 reap_dead start 2023-10-16T20:05:18.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7fc04aa46640 1 -- 172.21.15.88:0/3343873656 shutdown_connections 2023-10-16T20:05:18.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7fc04aa46640 1 -- 172.21.15.88:0/3343873656 >> 172.21.15.88:0/3343873656 conn(0x7fc0440ce5e0 msgr2=0x7fc0440ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:18.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7f261b7fe640 1 -- 172.21.15.88:0/2169257607 wait complete. 2023-10-16T20:05:18.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7f0e68ff9640 1 -- 172.21.15.88:0/1585829960 <== osd.3 v2:172.21.15.88:6824/1777716082 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0e4c002cc0 con 0x7f0e5806fb30 2023-10-16T20:05:18.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7fc04aa46640 1 -- 172.21.15.88:0/3343873656 shutdown_connections 2023-10-16T20:05:18.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7fc04aa46640 1 -- 172.21.15.88:0/3343873656 wait complete. 2023-10-16T20:05:18.139 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7ff05a7fc640 1 -- 172.21.15.88:0/2373031832 <== osd.2 v2:172.21.15.88:6816/3866598902 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff0641441b0 con 0x7ff04806fac0 2023-10-16T20:05:18.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7f0e72772640 1 -- 172.21.15.88:0/1585829960 >> [v2:172.21.15.88:6824/1777716082,v1:172.21.15.88:6825/1777716082] conn(0x7f0e5806fb30 msgr2=0x7f0e58071f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7f0e72772640 1 --2- 172.21.15.88:0/1585829960 >> [v2:172.21.15.88:6824/1777716082,v1:172.21.15.88:6825/1777716082] conn(0x7f0e5806fb30 0x7f0e58071f70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7f0e72772640 1 -- 172.21.15.88:0/1585829960 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0e580664f0 msgr2=0x7f0e580689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7f0e72772640 1 --2- 172.21.15.88:0/1585829960 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0e580664f0 0x7f0e580689b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c155010 tx=0x7f0e5c008040 comp rx=0 tx=0).stop 2023-10-16T20:05:18.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7f0e72772640 1 -- 172.21.15.88:0/1585829960 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e6c0706e0 msgr2=0x7f0e6c1e2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7f0e72772640 1 --2- 172.21.15.88:0/1585829960 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e6c0706e0 0x7f0e6c1e2cf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0e6001b080 tx=0x7f0e6001a1f0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.042+0000 7f0e70d0f640 1 -- 172.21.15.88:0/1585829960 reap_dead start 2023-10-16T20:05:18.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.046+0000 7ff06bfd9640 1 -- 172.21.15.88:0/2373031832 >> [v2:172.21.15.88:6816/3866598902,v1:172.21.15.88:6817/3866598902] conn(0x7ff04806fac0 msgr2=0x7ff048071f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.046+0000 7ff06bfd9640 1 --2- 172.21.15.88:0/2373031832 >> [v2:172.21.15.88:6816/3866598902,v1:172.21.15.88:6817/3866598902] conn(0x7ff04806fac0 0x7ff048071f00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.046+0000 7f0e72772640 1 -- 172.21.15.88:0/1585829960 shutdown_connections 2023-10-16T20:05:18.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.046+0000 7f0e72772640 1 -- 172.21.15.88:0/1585829960 >> 172.21.15.88:0/1585829960 conn(0x7f0e6c0cb060 msgr2=0x7f0e6c1574b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:18.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.046+0000 7ff06bfd9640 1 -- 172.21.15.88:0/2373031832 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff048066500 msgr2=0x7ff0480689c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.046+0000 7ff06bfd9640 1 --2- 172.21.15.88:0/2373031832 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff048066500 0x7ff0480689c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff06415f850 tx=0x7ff060008040 comp rx=0 tx=0).stop 2023-10-16T20:05:18.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.046+0000 7ff06bfd9640 1 -- 172.21.15.88:0/2373031832 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff064153ea0 msgr2=0x7ff0641474b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.046+0000 7ff06bfd9640 1 --2- 172.21.15.88:0/2373031832 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff064153ea0 0x7ff0641474b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff05c007e60 tx=0x7ff05c00e4b0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.046+0000 7ff06a576640 1 -- 172.21.15.88:0/2373031832 reap_dead start 2023-10-16T20:05:18.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.046+0000 7ff06bfd9640 1 -- 172.21.15.88:0/2373031832 shutdown_connections 2023-10-16T20:05:18.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.046+0000 7ff06bfd9640 1 -- 172.21.15.88:0/2373031832 >> 172.21.15.88:0/2373031832 conn(0x7ff0640715c0 msgr2=0x7ff0640719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:18.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.046+0000 7f0e72772640 1 -- 172.21.15.88:0/1585829960 shutdown_connections 2023-10-16T20:05:18.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.046+0000 7f0e72772640 1 -- 172.21.15.88:0/1585829960 wait complete. 2023-10-16T20:05:18.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.050+0000 7ff06bfd9640 1 -- 172.21.15.88:0/2373031832 shutdown_connections 2023-10-16T20:05:18.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.050+0000 7ff06bfd9640 1 -- 172.21.15.88:0/2373031832 wait complete. 2023-10-16T20:05:18.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.054+0000 7f73baa95640 1 -- 172.21.15.88:0/2982366610 --> [v2:172.21.15.94:6824/1057220797,v1:172.21.15.94:6825/1057220797] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f739c002cc0 con 0x7f739806fb30 2023-10-16T20:05:18.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.062+0000 7f73aa7fc640 1 -- 172.21.15.88:0/2982366610 <== osd.7 v2:172.21.15.94:6824/1057220797 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f739c002cc0 con 0x7f739806fb30 2023-10-16T20:05:18.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.062+0000 7f73baa95640 1 -- 172.21.15.88:0/2982366610 >> [v2:172.21.15.94:6824/1057220797,v1:172.21.15.94:6825/1057220797] conn(0x7f739806fb30 msgr2=0x7f7398071f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.062+0000 7f73baa95640 1 --2- 172.21.15.88:0/2982366610 >> [v2:172.21.15.94:6824/1057220797,v1:172.21.15.94:6825/1057220797] conn(0x7f739806fb30 0x7f7398071f70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.147 INFO:teuthology.orchestra.run.smithi088.stdout:64424509449 2023-10-16T20:05:18.147 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-16T20:05:18.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.062+0000 7f73baa95640 1 -- 172.21.15.88:0/2982366610 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f73980664f0 msgr2=0x7f73980689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.066+0000 7f73baa95640 1 --2- 172.21.15.88:0/2982366610 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f73980664f0 0x7f73980689b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f73b4167c90 tx=0x7f73a4008040 comp rx=0 tx=0).stop 2023-10-16T20:05:18.149 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.066+0000 7f73baa95640 1 -- 172.21.15.88:0/2982366610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73b40d12d0 msgr2=0x7f73b414f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.149 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.066+0000 7f73baa95640 1 --2- 172.21.15.88:0/2982366610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73b40d12d0 0x7f73b414f910 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f73b000c820 tx=0x7f73b000cce0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.149 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.066+0000 7f73ba294640 1 -- 172.21.15.88:0/2982366610 reap_dead start 2023-10-16T20:05:18.149 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.066+0000 7f73baa95640 1 -- 172.21.15.88:0/2982366610 shutdown_connections 2023-10-16T20:05:18.150 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.066+0000 7f73baa95640 1 -- 172.21.15.88:0/2982366610 >> 172.21.15.88:0/2982366610 conn(0x7f73b4089fb0 msgr2=0x7f73b40d0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:18.150 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.066+0000 7f73baa95640 1 -- 172.21.15.88:0/2982366610 shutdown_connections 2023-10-16T20:05:18.150 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.066+0000 7f73baa95640 1 -- 172.21.15.88:0/2982366610 wait complete. 2023-10-16T20:05:18.151 INFO:teuthology.orchestra.run.smithi088.stdout:55834574858 2023-10-16T20:05:18.151 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-16T20:05:18.152 INFO:teuthology.orchestra.run.smithi088.stdout:64424509448 2023-10-16T20:05:18.152 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-16T20:05:18.153 INFO:teuthology.orchestra.run.smithi088.stdout:64424509450 2023-10-16T20:05:18.154 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-16T20:05:18.154 INFO:teuthology.orchestra.run.smithi088.stdout:64424509448 2023-10-16T20:05:18.155 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-16T20:05:18.156 INFO:teuthology.orchestra.run.smithi088.stdout:64424509449 2023-10-16T20:05:18.156 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-16T20:05:18.157 INFO:teuthology.orchestra.run.smithi088.stdout:64424509448 2023-10-16T20:05:18.157 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-16T20:05:18.158 INFO:teuthology.orchestra.run.smithi088.stdout:64424509449 2023-10-16T20:05:18.159 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-16T20:05:18.428 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.426+0000 7f2efe409640 1 Processor -- start 2023-10-16T20:05:18.429 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.426+0000 7f2efe409640 1 -- start start 2023-10-16T20:05:18.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.430+0000 7f2efe409640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f2ef00e13d0 0x7f2ef0054970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.431 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.430+0000 7f2efe409640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ef0052960 con 0x7f2ef00e5070 2023-10-16T20:05:18.431 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.430+0000 7f2efe409640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ef0054eb0 con 0x7f2ef0051a30 2023-10-16T20:05:18.431 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.430+0000 7f2efe409640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ef0055110 con 0x7f2ef00e13d0 2023-10-16T20:05:18.432 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.430+0000 7f2efdc08640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f2ef0051a30 0x7f2ef003a2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:48710/0 (socket says 172.21.15.88:48710) 2023-10-16T20:05:18.432 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.430+0000 7f2efdc08640 1 -- 172.21.15.88:0/2978288384 learned_addr learned my addr 172.21.15.88:0/2978288384 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:18.432 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.430+0000 7f2eeffff640 1 -- 172.21.15.88:0/2978288384 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 414775388 0 0) 0x7f2ef0054eb0 con 0x7f2ef0051a30 2023-10-16T20:05:18.433 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.430+0000 7f2eeffff640 1 -- 172.21.15.88:0/2978288384 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ee4003530 con 0x7f2ef0051a30 2023-10-16T20:05:18.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.434+0000 7f2eeffff640 1 -- 172.21.15.88:0/2978288384 <== mon.1 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 279915467 0 0) 0x7f2ee4003530 con 0x7f2ef0051a30 2023-10-16T20:05:18.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.434+0000 7f2eeffff640 1 -- 172.21.15.88:0/2978288384 >> v2:172.21.15.88:3300/0 conn(0x7f2ef00e13d0 msgr2=0x7f2ef0054970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:18.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.434+0000 7f2eeffff640 1 --2- 172.21.15.88:0/2978288384 >> v2:172.21.15.88:3300/0 conn(0x7f2ef00e13d0 0x7f2ef0054970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.435 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.434+0000 7f2eeffff640 1 -- 172.21.15.88:0/2978288384 >> v1:172.21.15.88:6789/0 conn(0x7f2ef00e5070 legacy=0x7f2ef00561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:18.435 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.434+0000 7f2eeffff640 1 -- 172.21.15.88:0/2978288384 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef00553f0 con 0x7f2ef0051a30 2023-10-16T20:05:18.435 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.434+0000 7f2eeffff640 1 -- 172.21.15.88:0/2978288384 <== mon.1 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f2ef8069c10 con 0x7f2ef0051a30 2023-10-16T20:05:18.435 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.434+0000 7f2eeffff640 1 -- 172.21.15.88:0/2978288384 >> v1:172.21.15.94:6789/0 conn(0x7f2ef0051a30 legacy=0x7f2ef003a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.435 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.434+0000 7f2eeffff640 1 --2- 172.21.15.88:0/2978288384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ee4003980 0x7f2ee4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.436 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.434+0000 7f2eeffff640 1 --2- 172.21.15.88:0/2978288384 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2ee40047d0 0x7f2ee4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.436 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.434+0000 7f2eeffff640 1 --2- 172.21.15.88:0/2978288384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ee4005190 0x7f2ee400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.436 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.434+0000 7f2eeffff640 1 -- 172.21.15.88:0/2978288384 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef0055110 con 0x7f2ee4003980 2023-10-16T20:05:18.436 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.434+0000 7f2eeffff640 1 -- 172.21.15.88:0/2978288384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee400c150 con 0x7f2ee4005190 2023-10-16T20:05:18.436 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.434+0000 7f2eeffff640 1 -- 172.21.15.88:0/2978288384 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee400c310 con 0x7f2ee40047d0 2023-10-16T20:05:18.437 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.434+0000 7f2efdc08640 1 --2- 172.21.15.88:0/2978288384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ee4005190 0x7f2ee400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.437 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.434+0000 7f2efd407640 1 --2- 172.21.15.88:0/2978288384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ee4003980 0x7f2ee4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.434+0000 7f2efcc06640 1 --2- 172.21.15.88:0/2978288384 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2ee40047d0 0x7f2ee4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.434+0000 7f2efdc08640 1 -- 172.21.15.88:0/2978288384 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2ee40047d0 msgr2=0x7f2ee4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:18.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.434+0000 7f2efdc08640 1 --2- 172.21.15.88:0/2978288384 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2ee40047d0 0x7f2ee4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.434+0000 7f2efdc08640 1 -- 172.21.15.88:0/2978288384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ee4003980 msgr2=0x7f2ee4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:18.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.434+0000 7f2efdc08640 1 --2- 172.21.15.88:0/2978288384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ee4003980 0x7f2ee4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.434+0000 7f2efdc08640 1 -- 172.21.15.88:0/2978288384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef00553f0 con 0x7f2ee4005190 2023-10-16T20:05:18.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.434+0000 7f2efcc06640 1 --2- 172.21.15.88:0/2978288384 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2ee40047d0 0x7f2ee4004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:18.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.434+0000 7f2efd407640 1 --2- 172.21.15.88:0/2978288384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ee4003980 0x7f2ee4003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:18.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.438+0000 7f2efdc08640 1 -- 172.21.15.88:0/2978288384 reap_dead start 2023-10-16T20:05:18.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.438+0000 7f2efdc08640 1 --2- 172.21.15.88:0/2978288384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ee4005190 0x7f2ee400b990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8063b60 tx=0x7f2ef8096d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4f46df6c2efa325 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.438+0000 7f2eeffff640 1 -- 172.21.15.88:0/2978288384 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef811f850 con 0x7f2ee4005190 2023-10-16T20:05:18.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.438+0000 7f2eeffff640 1 -- 172.21.15.88:0/2978288384 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef8069e90 con 0x7f2ee4005190 2023-10-16T20:05:18.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.438+0000 7f2eeffff640 1 -- 172.21.15.88:0/2978288384 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef811fc50 con 0x7f2ee4005190 2023-10-16T20:05:18.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.438+0000 7f2efe409640 1 -- 172.21.15.88:0/2978288384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ee4005190 msgr2=0x7f2ee400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.438+0000 7f2efe409640 1 --2- 172.21.15.88:0/2978288384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ee4005190 0x7f2ee400b990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8063b60 tx=0x7f2ef8096d50 comp rx=0 tx=0).stop 2023-10-16T20:05:18.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.438+0000 7f2efe409640 1 -- 172.21.15.88:0/2978288384 shutdown_connections 2023-10-16T20:05:18.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.438+0000 7f2efe409640 1 --2- 172.21.15.88:0/2978288384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ee4005190 0x7f2ee400b990 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.438+0000 7f2efe409640 1 -- 172.21.15.88:0/2978288384 >> 172.21.15.88:0/2978288384 conn(0x7f2ef0056e30 msgr2=0x7f2ef0057240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:18.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.442+0000 7f2efe409640 1 -- 172.21.15.88:0/2978288384 shutdown_connections 2023-10-16T20:05:18.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.442+0000 7f2efe409640 1 -- 172.21.15.88:0/2978288384 wait complete. 2023-10-16T20:05:18.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.442+0000 7f566049e640 1 Processor -- start 2023-10-16T20:05:18.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.442+0000 7f566049e640 1 -- start start 2023-10-16T20:05:18.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.442+0000 7f2efe409640 1 Processor -- start 2023-10-16T20:05:18.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.442+0000 7f2efe409640 1 -- start start 2023-10-16T20:05:18.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.442+0000 7f566049e640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5658153ea0 0x7f56580d1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.442+0000 7f566049e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f56580d2660 0x7f565815d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.442+0000 7f566049e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56580b5510 con 0x7f565815d7d0 2023-10-16T20:05:18.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.442+0000 7f566049e640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56580b4990 con 0x7f5658153ea0 2023-10-16T20:05:18.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.442+0000 7f566049e640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56580b4c70 con 0x7f56580d2660 2023-10-16T20:05:18.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.446+0000 7f565e23a640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5658153ea0 0x7f56580d1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.446+0000 7f565da39640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f56580d2660 0x7f565815d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.446+0000 7f565e23a640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5658153ea0 0x7f56580d1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40396/0 (socket says 172.21.15.88:40396) 2023-10-16T20:05:18.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.446+0000 7f565e23a640 1 -- 172.21.15.88:0/348970284 learned_addr learned my addr 172.21.15.88:0/348970284 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:18.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.446+0000 7f565da39640 1 -- 172.21.15.88:0/348970284 >> v2:172.21.15.88:3300/0 conn(0x7f5658153ea0 msgr2=0x7f56580d1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:18.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.446+0000 7f565da39640 1 --2- 172.21.15.88:0/348970284 >> v2:172.21.15.88:3300/0 conn(0x7f5658153ea0 0x7f56580d1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.446+0000 7f565da39640 1 -- 172.21.15.88:0/348970284 >> v1:172.21.15.94:6789/0 conn(0x7f565815d7d0 legacy=0x7f56580cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.446+0000 7f565d238640 1 -- 172.21.15.88:0/348970284 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4292919855 0 0) 0x7f56580b5510 con 0x7f565815d7d0 2023-10-16T20:05:18.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.446+0000 7f565da39640 1 -- 172.21.15.88:0/348970284 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56580b4f50 con 0x7f56580d2660 2023-10-16T20:05:18.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.446+0000 7f2efe409640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2ee4005190 0x7f2ef004a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.446+0000 7f2efe409640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ef00e2370 0x7f2ef004adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.446+0000 7f2efe409640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ef00e1940 0x7f2ef00ce630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.446+0000 7f2efe409640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef004a130 con 0x7f2ef00e2370 2023-10-16T20:05:18.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.446+0000 7f2efe409640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef004a660 con 0x7f2ef00e1940 2023-10-16T20:05:18.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.446+0000 7f2efe409640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef004a380 con 0x7f2ee4005190 2023-10-16T20:05:18.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.446+0000 7f29ef3f8640 1 Processor -- start 2023-10-16T20:05:18.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.446+0000 7f2efdc08640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ef00e1940 0x7f2ef00ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.446+0000 7f2efdc08640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ef00e1940 0x7f2ef00ce630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:56828/0 (socket says 172.21.15.88:56828) 2023-10-16T20:05:18.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.446+0000 7f2efdc08640 1 -- 172.21.15.88:0/1487792970 learned_addr learned my addr 172.21.15.88:0/1487792970 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:18.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.446+0000 7f2efcc06640 1 --2- 172.21.15.88:0/1487792970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ef00e2370 0x7f2ef004adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.446+0000 7f2efd407640 1 --2- 172.21.15.88:0/1487792970 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2ee4005190 0x7f2ef004a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.446+0000 7f29ef3f8640 1 -- start start 2023-10-16T20:05:18.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.446+0000 7f2efd407640 1 -- 172.21.15.88:0/1487792970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ef00e1940 msgr2=0x7f2ef00ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.446+0000 7f2efd407640 1 --2- 172.21.15.88:0/1487792970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ef00e1940 0x7f2ef00ce630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.446+0000 7f2efd407640 1 -- 172.21.15.88:0/1487792970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ef00e2370 msgr2=0x7f2ef004adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.446+0000 7f2efd407640 1 --2- 172.21.15.88:0/1487792970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ef00e2370 0x7f2ef004adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.446+0000 7f2efd407640 1 -- 172.21.15.88:0/1487792970 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef004eac0 con 0x7f2ee4005190 2023-10-16T20:05:18.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.446+0000 7f2efdc08640 1 --2- 172.21.15.88:0/1487792970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ef00e1940 0x7f2ef00ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:05:18.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.450+0000 7f2efd407640 1 --2- 172.21.15.88:0/1487792970 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2ee4005190 0x7f2ef004a8b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2ef400d470 tx=0x7f2ef400d930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.450+0000 7f2eedffb640 1 -- 172.21.15.88:0/1487792970 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4014620 con 0x7f2ee4005190 2023-10-16T20:05:18.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.450+0000 7f2efe409640 1 -- 172.21.15.88:0/1487792970 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef004eda0 con 0x7f2ee4005190 2023-10-16T20:05:18.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.450+0000 7f2efe409640 1 -- 172.21.15.88:0/1487792970 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef00ced40 con 0x7f2ee4005190 2023-10-16T20:05:18.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.450+0000 7f29ef3f8640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f29e815cb70 0x7f29e8161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.450+0000 7f29ef3f8640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29e80cdad0 0x7f29e80cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.450+0000 7f2eedffb640 1 -- 172.21.15.88:0/1487792970 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef4014780 con 0x7f2ee4005190 2023-10-16T20:05:18.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.450+0000 7f2eedffb640 1 -- 172.21.15.88:0/1487792970 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4014620 con 0x7f2ee4005190 2023-10-16T20:05:18.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.450+0000 7f8b9bfd0640 1 Processor -- start 2023-10-16T20:05:18.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.450+0000 7f29ef3f8640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29e80b2190 con 0x7f29e80cff40 2023-10-16T20:05:18.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.450+0000 7f29ef3f8640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29e80d03d0 con 0x7f29e815cb70 2023-10-16T20:05:18.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.450+0000 7f29ef3f8640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e81615a0 con 0x7f29e80cdad0 2023-10-16T20:05:18.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.450+0000 7f29ec993640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29e80cdad0 0x7f29e80cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.450+0000 7f29ec993640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29e80cdad0 0x7f29e80cfa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:33064/0 (socket says 172.21.15.88:33064) 2023-10-16T20:05:18.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.450+0000 7f29ec993640 1 -- 172.21.15.88:0/3140778128 learned_addr learned my addr 172.21.15.88:0/3140778128 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:18.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.450+0000 7f8b9bfd0640 1 -- start start 2023-10-16T20:05:18.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.450+0000 7f29ed194640 1 --2- 172.21.15.88:0/3140778128 >> v2:172.21.15.88:3300/0 conn(0x7f29e815cb70 0x7f29e8161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.450+0000 7f565da39640 1 --2- 172.21.15.88:0/348970284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56580d2660 0x7f565815d290 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5658072520 tx=0x7f564801c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd67bdbe1618c82a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.450+0000 7f2efe409640 1 -- 172.21.15.88:0/1487792970 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ed8005260 con 0x7f2ee4005190 2023-10-16T20:05:18.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.450+0000 7f29ec993640 1 -- 172.21.15.88:0/3140778128 >> v2:172.21.15.88:3300/0 conn(0x7f29e815cb70 msgr2=0x7f29e8161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:18.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7f2eedffb640 1 -- 172.21.15.88:0/1487792970 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2ef40148f0 con 0x7f2ee4005190 2023-10-16T20:05:18.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7f8b9bfd0640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8b94143500 0x7f8b940b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7f8b9bfd0640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8b940b0500 0x7f8b940b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7f8b9bfd0640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b940b0e90 con 0x7f8b9415eff0 2023-10-16T20:05:18.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7f2eedffb640 1 --2- 172.21.15.88:0/1487792970 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2edc066500 0x7f2edc0689c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7f8b9bfd0640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b9415e5a0 con 0x7f8b940b0500 2023-10-16T20:05:18.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7f8b9bfd0640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b940b4280 con 0x7f8b94143500 2023-10-16T20:05:18.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7f2eedffb640 1 -- 172.21.15.88:0/1487792970 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2ef4083e80 con 0x7f2ee4005190 2023-10-16T20:05:18.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7f8b9a56d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8b940b0500 0x7f8b940b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7f565d238640 1 -- 172.21.15.88:0/348970284 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56480a8050 con 0x7f56580d2660 2023-10-16T20:05:18.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7f8b9a56d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8b940b0500 0x7f8b940b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:56832/0 (socket says 172.21.15.88:56832) 2023-10-16T20:05:18.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7f8b9a56d640 1 -- 172.21.15.88:0/2503109740 learned_addr learned my addr 172.21.15.88:0/2503109740 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:18.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7f565d238640 1 -- 172.21.15.88:0/348970284 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56480ac5f0 con 0x7f56580d2660 2023-10-16T20:05:18.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7f565d238640 1 -- 172.21.15.88:0/348970284 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56480ac980 con 0x7f56580d2660 2023-10-16T20:05:18.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7fbe8129e640 1 Processor -- start 2023-10-16T20:05:18.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7f8b99d6c640 1 --2- 172.21.15.88:0/2503109740 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8b94143500 0x7f8b940b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7fbe8129e640 1 -- start start 2023-10-16T20:05:18.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7f2efcc06640 1 --2- 172.21.15.88:0/1487792970 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2edc066500 0x7f2edc0689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7f056a46b640 1 Processor -- start 2023-10-16T20:05:18.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.450+0000 7f29ec993640 1 --2- 172.21.15.88:0/3140778128 >> v2:172.21.15.88:3300/0 conn(0x7f29e815cb70 0x7f29e8161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.450+0000 7f29ec993640 1 -- 172.21.15.88:0/3140778128 >> v1:172.21.15.94:6789/0 conn(0x7f29e80cff40 legacy=0x7f29e80b7c00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T20:05:18.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.450+0000 7f29ec993640 1 -- 172.21.15.88:0/3140778128 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e815df60 con 0x7f29e80cdad0 2023-10-16T20:05:18.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7f29ed194640 1 --2- 172.21.15.88:0/3140778128 >> v2:172.21.15.88:3300/0 conn(0x7f29e815cb70 0x7f29e8161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:18.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7f566049e640 1 -- 172.21.15.88:0/348970284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56580d2660 msgr2=0x7f565815d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7f566049e640 1 --2- 172.21.15.88:0/348970284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56580d2660 0x7f565815d290 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5658072520 tx=0x7f564801c430 comp rx=0 tx=0).stop 2023-10-16T20:05:18.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7f566049e640 1 -- 172.21.15.88:0/348970284 shutdown_connections 2023-10-16T20:05:18.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7f566049e640 1 --2- 172.21.15.88:0/348970284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56580d2660 0x7f565815d290 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5658072520 tx=0x7f564801c430 comp rx=0 tx=0).stop 2023-10-16T20:05:18.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7f566049e640 1 --2- 172.21.15.88:0/348970284 >> v2:172.21.15.88:3300/0 conn(0x7f5658153ea0 0x7f56580d1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7f566049e640 1 -- 172.21.15.88:0/348970284 >> 172.21.15.88:0/348970284 conn(0x7f56580715c0 msgr2=0x7f56580719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:18.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7f2efcc06640 1 --2- 172.21.15.88:0/1487792970 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2edc066500 0x7f2edc0689c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0057930 tx=0x7f2ee8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7fbe8129e640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fbe7c153ea0 0x7fbe7c0d1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7fbe8129e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe7c0d2660 0x7fbe7c15d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7f7d80446640 1 Processor -- start 2023-10-16T20:05:18.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7fbe8129e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe7c0b5510 con 0x7fbe7c15d7d0 2023-10-16T20:05:18.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7fbe8129e640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe7c0b4990 con 0x7fbe7c153ea0 2023-10-16T20:05:18.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7fbe8129e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c0b4c70 con 0x7fbe7c0d2660 2023-10-16T20:05:18.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7f7d80446640 1 -- start start 2023-10-16T20:05:18.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7fbe7b7fe640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fbe7c15d7d0 0x7fbe7c0cd970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:48756/0 (socket says 172.21.15.88:48756) 2023-10-16T20:05:18.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7fbe7b7fe640 1 -- 172.21.15.88:0/2818245287 learned_addr learned my addr 172.21.15.88:0/2818245287 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:18.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7fbe7a7fc640 1 --2- 172.21.15.88:0/2818245287 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe7c0d2660 0x7fbe7c15d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7fbe7affd640 1 --2- 172.21.15.88:0/2818245287 >> v2:172.21.15.88:3300/0 conn(0x7fbe7c153ea0 0x7fbe7c0d1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.475 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7f8b9a56d640 1 -- 172.21.15.88:0/2503109740 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8b94143500 msgr2=0x7f8b940b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:18.475 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7f8b9a56d640 1 --2- 172.21.15.88:0/2503109740 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8b94143500 0x7f8b940b3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.475 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7f8b9a56d640 1 -- 172.21.15.88:0/2503109740 >> v1:172.21.15.94:6789/0 conn(0x7f8b9415eff0 legacy=0x7f8b940cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:18.475 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7f8b9a56d640 1 -- 172.21.15.88:0/2503109740 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b940b4560 con 0x7f8b940b0500 2023-10-16T20:05:18.476 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.454+0000 7f056a46b640 1 -- start start 2023-10-16T20:05:18.476 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7f056a46b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f056415cb70 0x7f0564161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.476 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7f056a46b640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f05640cdad0 0x7f05640cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.476 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7f056a46b640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05640b2190 con 0x7f05640cff40 2023-10-16T20:05:18.476 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7f056a46b640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05640d03d0 con 0x7f05640cdad0 2023-10-16T20:05:18.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7f056a46b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05641615a0 con 0x7f056415cb70 2023-10-16T20:05:18.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7f0568a08640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f05640cff40 0x7f05640b7c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:39212/0 (socket says 172.21.15.88:39212) 2023-10-16T20:05:18.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7f0568a08640 1 -- 172.21.15.88:0/4175270567 learned_addr learned my addr 172.21.15.88:0/4175270567 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:18.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7fbe79ffb640 1 -- 172.21.15.88:0/2818245287 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4107558893 0 0) 0x7fbe7c0b5510 con 0x7fbe7c15d7d0 2023-10-16T20:05:18.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7fbe79ffb640 1 -- 172.21.15.88:0/2818245287 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe64003530 con 0x7fbe7c15d7d0 2023-10-16T20:05:18.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7fbe7a7fc640 1 -- 172.21.15.88:0/2818245287 >> v2:172.21.15.88:3300/0 conn(0x7fbe7c153ea0 msgr2=0x7fbe7c0d1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:18.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7fbe7a7fc640 1 --2- 172.21.15.88:0/2818245287 >> v2:172.21.15.88:3300/0 conn(0x7fbe7c153ea0 0x7fbe7c0d1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7fbe7a7fc640 1 -- 172.21.15.88:0/2818245287 >> v1:172.21.15.94:6789/0 conn(0x7fbe7c15d7d0 legacy=0x7fbe7c0cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7fbe79ffb640 1 -- 172.21.15.88:0/2818245287 <== mon.1 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 925035520 0 0) 0x7fbe64003530 con 0x7fbe7c15d7d0 2023-10-16T20:05:18.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7fbe7a7fc640 1 -- 172.21.15.88:0/2818245287 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe7c0b4f50 con 0x7fbe7c0d2660 2023-10-16T20:05:18.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7f29ec993640 1 --2- 172.21.15.88:0/3140778128 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29e80cdad0 0x7f29e80cfa00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f29e4013db0 tx=0x7f29e4024c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c235f80bc1a0366 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7f7d80446640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d780cecd0 0x7f7d780cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7f7d80446640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f7d780ce360 0x7f7d7806ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7f7d80446640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d78164f90 con 0x7f7d780c34b0 2023-10-16T20:05:18.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7f7d80446640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d78165b80 con 0x7f7d780ce360 2023-10-16T20:05:18.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7f7d80446640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d781618a0 con 0x7f7d780cecd0 2023-10-16T20:05:18.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7f0562ffd640 1 -- 172.21.15.88:0/4175270567 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3453240452 0 0) 0x7f05640b2190 con 0x7f05640cff40 2023-10-16T20:05:18.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7f0562ffd640 1 -- 172.21.15.88:0/4175270567 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f054c003530 con 0x7f05640cff40 2023-10-16T20:05:18.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7f7d7e1e2640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d780cecd0 0x7f7d780cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.482 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7f8b9a56d640 1 --2- 172.21.15.88:0/2503109740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b940b0500 0x7f8b940b0950 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c009e50 tx=0x7f8b8c01c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e65ed0ff1c61775 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.482 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7f7d7e1e2640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d780cecd0 0x7f7d780cf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:33094/0 (socket says 172.21.15.88:33094) 2023-10-16T20:05:18.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7f7d7e1e2640 1 -- 172.21.15.88:0/3721203642 learned_addr learned my addr 172.21.15.88:0/3721203642 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:18.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7f7d7e1e2640 1 -- 172.21.15.88:0/3721203642 >> v2:172.21.15.88:3300/0 conn(0x7f7d780ce360 msgr2=0x7f7d7806ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:18.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7f7d7e1e2640 1 --2- 172.21.15.88:0/3721203642 >> v2:172.21.15.88:3300/0 conn(0x7f7d780ce360 0x7f7d7806ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7f7d7e1e2640 1 -- 172.21.15.88:0/3721203642 >> v1:172.21.15.88:6789/0 conn(0x7f7d780c34b0 legacy=0x7f7d780704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:18.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7f7d7e1e2640 1 -- 172.21.15.88:0/3721203642 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d780cf5f0 con 0x7f7d780cecd0 2023-10-16T20:05:18.484 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7f8b98d6a640 1 -- 172.21.15.88:0/2503109740 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c0a5700 con 0x7f8b940b0500 2023-10-16T20:05:18.484 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7f566049e640 1 -- 172.21.15.88:0/348970284 shutdown_connections 2023-10-16T20:05:18.485 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7f8b98d6a640 1 -- 172.21.15.88:0/2503109740 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8c0a5860 con 0x7f8b940b0500 2023-10-16T20:05:18.485 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7f8b98d6a640 1 -- 172.21.15.88:0/2503109740 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c0a5bf0 con 0x7f8b940b0500 2023-10-16T20:05:18.485 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f29dffff640 1 -- 172.21.15.88:0/3140778128 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e40adb90 con 0x7f29e80cdad0 2023-10-16T20:05:18.486 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f29dffff640 1 -- 172.21.15.88:0/3140778128 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e40adcf0 con 0x7f29e80cdad0 2023-10-16T20:05:18.486 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f29dffff640 1 -- 172.21.15.88:0/3140778128 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e40b5660 con 0x7f29e80cdad0 2023-10-16T20:05:18.486 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f0563fff640 1 --2- 172.21.15.88:0/4175270567 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f056415cb70 0x7f0564161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.487 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f8b9bfd0640 1 -- 172.21.15.88:0/2503109740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b940b0500 msgr2=0x7f8b940b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.487 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f8b9bfd0640 1 --2- 172.21.15.88:0/2503109740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b940b0500 0x7f8b940b0950 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c009e50 tx=0x7f8b8c01c880 comp rx=0 tx=0).stop 2023-10-16T20:05:18.487 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f8b9bfd0640 1 -- 172.21.15.88:0/2503109740 shutdown_connections 2023-10-16T20:05:18.488 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f8b9bfd0640 1 --2- 172.21.15.88:0/2503109740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b940b0500 0x7f8b940b0950 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.488 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f8b9bfd0640 1 --2- 172.21.15.88:0/2503109740 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8b94143500 0x7f8b940b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.488 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f8b9bfd0640 1 -- 172.21.15.88:0/2503109740 >> 172.21.15.88:0/2503109740 conn(0x7f8b940ce5e0 msgr2=0x7f8b940ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:18.488 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f8b9bfd0640 1 -- 172.21.15.88:0/2503109740 shutdown_connections 2023-10-16T20:05:18.488 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f2eedffb640 1 -- 172.21.15.88:0/1487792970 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2ef4054be0 con 0x7f2ee4005190 2023-10-16T20:05:18.489 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f8b9bfd0640 1 -- 172.21.15.88:0/2503109740 wait complete. 2023-10-16T20:05:18.490 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f05637fe640 1 --2- 172.21.15.88:0/4175270567 >> v2:172.21.15.88:3300/0 conn(0x7f05640cdad0 0x7f05640cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.490 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f0562ffd640 1 -- 172.21.15.88:0/4175270567 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1321726575 0 0) 0x7f054c003530 con 0x7f05640cff40 2023-10-16T20:05:18.490 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f0562ffd640 1 -- 172.21.15.88:0/4175270567 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f056415cb70 msgr2=0x7f0564161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:18.490 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f0562ffd640 1 --2- 172.21.15.88:0/4175270567 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f056415cb70 0x7f0564161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.491 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f0562ffd640 1 -- 172.21.15.88:0/4175270567 >> v2:172.21.15.88:3300/0 conn(0x7f05640cdad0 msgr2=0x7f05640cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:18.491 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f0562ffd640 1 --2- 172.21.15.88:0/4175270567 >> v2:172.21.15.88:3300/0 conn(0x7f05640cdad0 0x7f05640cfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.491 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f0562ffd640 1 -- 172.21.15.88:0/4175270567 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056415df60 con 0x7f05640cff40 2023-10-16T20:05:18.491 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f0562ffd640 1 -- 172.21.15.88:0/4175270567 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f055c001a00 con 0x7f05640cff40 2023-10-16T20:05:18.491 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f0562ffd640 1 -- 172.21.15.88:0/4175270567 >> v1:172.21.15.88:6789/0 conn(0x7f05640cff40 legacy=0x7f05640b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.492 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f0562ffd640 1 --2- 172.21.15.88:0/4175270567 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f054c003980 0x7f054c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.492 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f0562ffd640 1 --2- 172.21.15.88:0/4175270567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f054c004370 0x7f054c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.492 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f0562ffd640 1 --2- 172.21.15.88:0/4175270567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f054c005050 0x7f054c0141a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.492 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f0562ffd640 1 -- 172.21.15.88:0/4175270567 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05640d03d0 con 0x7f054c005050 2023-10-16T20:05:18.492 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f0562ffd640 1 -- 172.21.15.88:0/4175270567 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05641615a0 con 0x7f054c004370 2023-10-16T20:05:18.493 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f0562ffd640 1 -- 172.21.15.88:0/4175270567 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054c0059e0 con 0x7f054c003980 2023-10-16T20:05:18.493 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f05637fe640 1 --2- 172.21.15.88:0/4175270567 >> v2:172.21.15.88:3300/0 conn(0x7f05640cdad0 0x7f05640cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:18.493 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f0563fff640 1 --2- 172.21.15.88:0/4175270567 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f054c003980 0x7f054c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.493 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f05637fe640 1 --2- 172.21.15.88:0/4175270567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f054c004370 0x7f054c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.494 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.458+0000 7f566049e640 1 -- 172.21.15.88:0/348970284 wait complete. 2023-10-16T20:05:18.494 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f566049e640 1 Processor -- start 2023-10-16T20:05:18.494 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f0563fff640 1 -- 172.21.15.88:0/4175270567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f054c004370 msgr2=0x7f054c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:18.495 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f0563fff640 1 --2- 172.21.15.88:0/4175270567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f054c004370 0x7f054c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.495 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f0563fff640 1 -- 172.21.15.88:0/4175270567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f054c005050 msgr2=0x7f054c0141a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:18.495 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f0563fff640 1 --2- 172.21.15.88:0/4175270567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f054c005050 0x7f054c0141a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.495 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f0563fff640 1 -- 172.21.15.88:0/4175270567 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054c003530 con 0x7f054c003980 2023-10-16T20:05:18.496 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f29ef3f8640 1 -- 172.21.15.88:0/3140778128 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29e80cdad0 msgr2=0x7f29e80cfa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.496 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f29ef3f8640 1 --2- 172.21.15.88:0/3140778128 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29e80cdad0 0x7f29e80cfa00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f29e4013db0 tx=0x7f29e4024c30 comp rx=0 tx=0).stop 2023-10-16T20:05:18.497 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f29ef3f8640 1 -- 172.21.15.88:0/3140778128 shutdown_connections 2023-10-16T20:05:18.497 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f29ef3f8640 1 --2- 172.21.15.88:0/3140778128 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29e80cdad0 0x7f29e80cfa00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.497 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f29ef3f8640 1 --2- 172.21.15.88:0/3140778128 >> v2:172.21.15.88:3300/0 conn(0x7f29e815cb70 0x7f29e8161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.497 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f29ef3f8640 1 -- 172.21.15.88:0/3140778128 >> 172.21.15.88:0/3140778128 conn(0x7f29e80719c0 msgr2=0x7f29e8071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:18.497 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f29ef3f8640 1 -- 172.21.15.88:0/3140778128 shutdown_connections 2023-10-16T20:05:18.498 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.462+0000 7f7d7e1e2640 1 --2- 172.21.15.88:0/3721203642 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d780cecd0 0x7f7d780cf0b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7d74009870 tx=0x7f7d7401c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2106f61d1bf4a08f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.498 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f7d7d1e0640 1 -- 172.21.15.88:0/3721203642 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d740aa5f0 con 0x7f7d780cecd0 2023-10-16T20:05:18.498 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f7d7d1e0640 1 -- 172.21.15.88:0/3721203642 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d740aa750 con 0x7f7d780cecd0 2023-10-16T20:05:18.499 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f7d7d1e0640 1 -- 172.21.15.88:0/3721203642 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d740aaae0 con 0x7f7d780cecd0 2023-10-16T20:05:18.499 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f566049e640 1 -- start start 2023-10-16T20:05:18.499 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f29ef3f8640 1 -- 172.21.15.88:0/3140778128 wait complete. 2023-10-16T20:05:18.500 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7fbe7a7fc640 1 --2- 172.21.15.88:0/2818245287 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe7c0d2660 0x7fbe7c15d290 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c0097c0 tx=0x7fbe6c01c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e356d506261f6110 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.500 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f8b9bfd0640 1 Processor -- start 2023-10-16T20:05:18.501 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f566049e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5658153ea0 0x7f56580d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.501 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f566049e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5658206490 0x7f56580d2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.501 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f566049e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56580c9c70 0x7f56580ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.501 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f566049e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5658149120 con 0x7f5658206490 2023-10-16T20:05:18.501 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f566049e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56582031c0 con 0x7f56580c9c70 2023-10-16T20:05:18.502 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f566049e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56580ca630 con 0x7f5658153ea0 2023-10-16T20:05:18.502 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7fbe79ffb640 1 -- 172.21.15.88:0/2818245287 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c0a4ea0 con 0x7fbe7c0d2660 2023-10-16T20:05:18.502 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7fbe79ffb640 1 -- 172.21.15.88:0/2818245287 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe6c0a7050 con 0x7fbe7c0d2660 2023-10-16T20:05:18.502 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7fbe79ffb640 1 -- 172.21.15.88:0/2818245287 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe70001680 con 0x7fbe7c0d2660 2023-10-16T20:05:18.503 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f565e23a640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5658153ea0 0x7f56580d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.503 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f565ea3b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56580c9c70 0x7f56580ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.503 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f565e23a640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5658153ea0 0x7f56580d2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:33114/0 (socket says 172.21.15.88:33114) 2023-10-16T20:05:18.504 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f565e23a640 1 -- 172.21.15.88:0/263511120 learned_addr learned my addr 172.21.15.88:0/263511120 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:18.504 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f565da39640 1 --2- 172.21.15.88:0/263511120 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5658206490 0x7f56580d2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.504 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f565ea3b640 1 -- 172.21.15.88:0/263511120 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5658153ea0 msgr2=0x7f56580d2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.504 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f565ea3b640 1 --2- 172.21.15.88:0/263511120 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5658153ea0 0x7f56580d2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.504 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f565ea3b640 1 -- 172.21.15.88:0/263511120 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5658206490 msgr2=0x7f56580d2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.505 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f565ea3b640 1 --2- 172.21.15.88:0/263511120 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5658206490 0x7f56580d2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.505 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f565ea3b640 1 -- 172.21.15.88:0/263511120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5648009420 con 0x7f56580c9c70 2023-10-16T20:05:18.505 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f565da39640 1 --2- 172.21.15.88:0/263511120 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5658206490 0x7f56580d2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:05:18.505 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7fbe8129e640 1 -- 172.21.15.88:0/2818245287 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe7c0d2660 msgr2=0x7fbe7c15d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.506 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7fbe8129e640 1 --2- 172.21.15.88:0/2818245287 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe7c0d2660 0x7fbe7c15d290 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c0097c0 tx=0x7fbe6c01c370 comp rx=0 tx=0).stop 2023-10-16T20:05:18.506 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7fbe8129e640 1 -- 172.21.15.88:0/2818245287 shutdown_connections 2023-10-16T20:05:18.506 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7fbe8129e640 1 --2- 172.21.15.88:0/2818245287 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe7c0d2660 0x7fbe7c15d290 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.506 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7fbe8129e640 1 --2- 172.21.15.88:0/2818245287 >> v2:172.21.15.88:3300/0 conn(0x7fbe7c153ea0 0x7fbe7c0d1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.506 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7fbe8129e640 1 -- 172.21.15.88:0/2818245287 >> 172.21.15.88:0/2818245287 conn(0x7fbe7c0715c0 msgr2=0x7fbe7c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:18.507 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7fbe8129e640 1 -- 172.21.15.88:0/2818245287 shutdown_connections 2023-10-16T20:05:18.507 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f8b9bfd0640 1 -- start start 2023-10-16T20:05:18.507 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f8b9bfd0640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8b940b0500 0x7f8b940716b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.508 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f8b9bfd0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b94143500 0x7f8b94157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.508 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f8b9bfd0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b9415eff0 0x7f8b941581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.508 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f8b9bfd0640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9413a720 con 0x7f8b94143500 2023-10-16T20:05:18.508 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f8b9bfd0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9415dcc0 con 0x7f8b9415eff0 2023-10-16T20:05:18.508 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f8b9bfd0640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9415c890 con 0x7f8b940b0500 2023-10-16T20:05:18.509 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7fbe8129e640 1 -- 172.21.15.88:0/2818245287 wait complete. 2023-10-16T20:05:18.509 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f8b99d6c640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8b940b0500 0x7f8b940716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.510 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f8b99d6c640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8b940b0500 0x7f8b940716b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:33128/0 (socket says 172.21.15.88:33128) 2023-10-16T20:05:18.510 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f8b99d6c640 1 -- 172.21.15.88:0/677337582 learned_addr learned my addr 172.21.15.88:0/677337582 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:18.510 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f8b9956b640 1 --2- 172.21.15.88:0/677337582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b94143500 0x7f8b94157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.510 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f8b9a56d640 1 --2- 172.21.15.88:0/677337582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b9415eff0 0x7f8b941581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.511 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f7517b80640 1 Processor -- start 2023-10-16T20:05:18.511 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f8b99d6c640 1 -- 172.21.15.88:0/677337582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b9415eff0 msgr2=0x7f8b941581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.511 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f565ea3b640 1 --2- 172.21.15.88:0/263511120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56580c9c70 0x7f56580ca0f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f56580cd1a0 tx=0x7f565400c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.512 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f8b99d6c640 1 --2- 172.21.15.88:0/677337582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b9415eff0 0x7f8b941581b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.512 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f8b99d6c640 1 -- 172.21.15.88:0/677337582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b94143500 msgr2=0x7f8b94157c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.512 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f8b99d6c640 1 --2- 172.21.15.88:0/677337582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b94143500 0x7f8b94157c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.513 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f8b99d6c640 1 -- 172.21.15.88:0/677337582 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8c009b30 con 0x7f8b940b0500 2023-10-16T20:05:18.513 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f7d80446640 1 -- 172.21.15.88:0/3721203642 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d780cecd0 msgr2=0x7f7d780cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.513 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.466+0000 7f7d80446640 1 --2- 172.21.15.88:0/3721203642 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d780cecd0 0x7f7d780cf0b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7d74009870 tx=0x7f7d7401c4f0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.514 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f7517b80640 1 -- start start 2023-10-16T20:05:18.514 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f564effd640 1 -- 172.21.15.88:0/263511120 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56540054e0 con 0x7f56580c9c70 2023-10-16T20:05:18.514 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f564effd640 1 -- 172.21.15.88:0/263511120 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5654003600 con 0x7f56580c9c70 2023-10-16T20:05:18.515 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f564effd640 1 -- 172.21.15.88:0/263511120 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56540039d0 con 0x7f56580c9c70 2023-10-16T20:05:18.515 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f566049e640 1 -- 172.21.15.88:0/263511120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5658202ee0 con 0x7f56580c9c70 2023-10-16T20:05:18.515 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f566049e640 1 -- 172.21.15.88:0/263511120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5658205c90 con 0x7f56580c9c70 2023-10-16T20:05:18.515 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f7d80446640 1 -- 172.21.15.88:0/3721203642 shutdown_connections 2023-10-16T20:05:18.516 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f29ef3f8640 1 Processor -- start 2023-10-16T20:05:18.516 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7fbe8129e640 1 Processor -- start 2023-10-16T20:05:18.517 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f29ef3f8640 1 -- start start 2023-10-16T20:05:18.517 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7fbe8129e640 1 -- start start 2023-10-16T20:05:18.517 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f7d80446640 1 --2- 172.21.15.88:0/3721203642 >> v2:172.21.15.88:3300/0 conn(0x7f7d780ce360 0x7f7d7806ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.518 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f7517b80640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f75100d0110 0x7f7510161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.518 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f7517b80640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f75100cdad0 0x7f75100b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.518 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f7517b80640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75100b2190 con 0x7f751015cb70 2023-10-16T20:05:18.519 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f7517b80640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75101615a0 con 0x7f75100d0110 2023-10-16T20:05:18.519 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f7517b80640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f751015df60 con 0x7f75100cdad0 2023-10-16T20:05:18.519 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f0563fff640 1 --2- 172.21.15.88:0/4175270567 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f054c003980 0x7f054c003da0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f05640cf9d0 tx=0x7f055801c930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b57906b2636c0e6f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.519 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f0562ffd640 1 -- 172.21.15.88:0/4175270567 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05580a5520 con 0x7f054c003980 2023-10-16T20:05:18.520 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f0562ffd640 1 -- 172.21.15.88:0/4175270567 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05580a5680 con 0x7f054c003980 2023-10-16T20:05:18.520 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f0562ffd640 1 -- 172.21.15.88:0/4175270567 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05580a5a30 con 0x7f054c003980 2023-10-16T20:05:18.520 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f7d80446640 1 --2- 172.21.15.88:0/3721203642 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d780cecd0 0x7f7d780cf0b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.521 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f7d80446640 1 -- 172.21.15.88:0/3721203642 >> 172.21.15.88:0/3721203642 conn(0x7f7d780d0820 msgr2=0x7f7d780d0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:18.521 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f7d80446640 1 -- 172.21.15.88:0/3721203642 shutdown_connections 2023-10-16T20:05:18.521 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f751591c640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f751015cb70 0x7f75100cfa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:48770/0 (socket says 172.21.15.88:48770) 2023-10-16T20:05:18.521 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f751591c640 1 -- 172.21.15.88:0/1987442349 learned_addr learned my addr 172.21.15.88:0/1987442349 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:18.522 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f751611d640 1 --2- 172.21.15.88:0/1987442349 >> v2:172.21.15.94:3300/0 conn(0x7f75100cdad0 0x7f75100b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.522 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f7d80446640 1 -- 172.21.15.88:0/3721203642 wait complete. 2023-10-16T20:05:18.522 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f566049e640 1 -- 172.21.15.88:0/263511120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f565804e9c0 con 0x7f56580c9c70 2023-10-16T20:05:18.523 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f751511b640 1 --2- 172.21.15.88:0/1987442349 >> v2:172.21.15.88:3300/0 conn(0x7f75100d0110 0x7f7510161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.523 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f8b99d6c640 1 --2- 172.21.15.88:0/677337582 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8b940b0500 0x7f8b940716b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400e870 tx=0x7f8b8400ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f751611d640 1 -- 172.21.15.88:0/1987442349 >> v2:172.21.15.88:3300/0 conn(0x7f75100d0110 msgr2=0x7f7510161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:18.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f751611d640 1 --2- 172.21.15.88:0/1987442349 >> v2:172.21.15.88:3300/0 conn(0x7f75100d0110 0x7f7510161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f751611d640 1 -- 172.21.15.88:0/1987442349 >> v1:172.21.15.94:6789/0 conn(0x7f751015cb70 legacy=0x7f75100cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7fbe8129e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe7c0d2660 0x7fbe7c14bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7fbe8129e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe7c153ea0 0x7fbe7c0ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7fbe8129e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe7c1f9980 0x7fbe7c1f9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7fbe8129e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c0c9fa0 con 0x7fbe7c0d2660 2023-10-16T20:05:18.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7fbe8129e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c1fa4b0 con 0x7fbe7c153ea0 2023-10-16T20:05:18.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7fbe8129e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c1fa790 con 0x7fbe7c1f9980 2023-10-16T20:05:18.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f751611d640 1 -- 172.21.15.88:0/1987442349 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f751015e230 con 0x7f75100cdad0 2023-10-16T20:05:18.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f751511b640 1 --2- 172.21.15.88:0/1987442349 >> v2:172.21.15.88:3300/0 conn(0x7f75100d0110 0x7f7510161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:18.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f056a46b640 1 -- 172.21.15.88:0/4175270567 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f054c003980 msgr2=0x7f054c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.527 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f056a46b640 1 --2- 172.21.15.88:0/4175270567 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f054c003980 0x7f054c003da0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f05640cf9d0 tx=0x7f055801c930 comp rx=0 tx=0).stop 2023-10-16T20:05:18.527 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f0568a08640 1 -- 172.21.15.88:0/4175270567 reap_dead start 2023-10-16T20:05:18.527 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f8b8a7fc640 1 -- 172.21.15.88:0/677337582 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b84009510 con 0x7f8b940b0500 2023-10-16T20:05:18.528 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f8b8a7fc640 1 -- 172.21.15.88:0/677337582 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8400eec0 con 0x7f8b940b0500 2023-10-16T20:05:18.528 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7fbe7b7fe640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe7c1f9980 0x7fbe7c1f9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.528 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f8b8a7fc640 1 -- 172.21.15.88:0/677337582 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8400cb40 con 0x7f8b940b0500 2023-10-16T20:05:18.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7fbe7b7fe640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe7c1f9980 0x7fbe7c1f9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:33140/0 (socket says 172.21.15.88:33140) 2023-10-16T20:05:18.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7fbe7b7fe640 1 -- 172.21.15.88:0/1961398657 learned_addr learned my addr 172.21.15.88:0/1961398657 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:18.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7fbe7affd640 1 --2- 172.21.15.88:0/1961398657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe7c0d2660 0x7fbe7c14bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.470+0000 7f056a46b640 1 -- 172.21.15.88:0/4175270567 shutdown_connections 2023-10-16T20:05:18.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7f056a46b640 1 -- 172.21.15.88:0/4175270567 >> 172.21.15.88:0/4175270567 conn(0x7f05640719c0 msgr2=0x7f0564071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:18.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7f29ef3f8640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29e80cdad0 0x7f29e8151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.531 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7fbe7a7fc640 1 --2- 172.21.15.88:0/1961398657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe7c153ea0 0x7fbe7c0ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.531 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7f056a46b640 1 -- 172.21.15.88:0/4175270567 shutdown_connections 2023-10-16T20:05:18.531 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7fbe7affd640 1 -- 172.21.15.88:0/1961398657 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe7c1f9980 msgr2=0x7fbe7c1f9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7fbe7affd640 1 --2- 172.21.15.88:0/1961398657 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe7c1f9980 0x7fbe7c1f9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7fbe7affd640 1 -- 172.21.15.88:0/1961398657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe7c153ea0 msgr2=0x7fbe7c0ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7fbe7affd640 1 --2- 172.21.15.88:0/1961398657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe7c153ea0 0x7fbe7c0ca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7fbe7affd640 1 -- 172.21.15.88:0/1961398657 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe6c009420 con 0x7fbe7c0d2660 2023-10-16T20:05:18.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7f29ef3f8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29e80cff40 0x7f29e80b6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7f29ef3f8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29e815cb70 0x7f29e80b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7f29ed194640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29e80cdad0 0x7f29e8151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7f29ed194640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29e80cdad0 0x7f29e8151000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:33142/0 (socket says 172.21.15.88:33142) 2023-10-16T20:05:18.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7f29ec993640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29e80cff40 0x7f29e80b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7f29ec993640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29e80cff40 0x7f29e80b6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:56874/0 (socket says 172.21.15.88:56874) 2023-10-16T20:05:18.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7fbe7affd640 1 --2- 172.21.15.88:0/1961398657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe7c0d2660 0x7fbe7c14bd70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbe7400efc0 tx=0x7fbe7400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.535 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7f29ed995640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29e815cb70 0x7f29e80b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.535 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7f7d80446640 1 Processor -- start 2023-10-16T20:05:18.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7fbe637fe640 1 -- 172.21.15.88:0/1961398657 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe74014480 con 0x7fbe7c0d2660 2023-10-16T20:05:18.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7fbe8129e640 1 -- 172.21.15.88:0/1961398657 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe7c0ca1b0 con 0x7fbe7c0d2660 2023-10-16T20:05:18.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7f29ed995640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29e815cb70 0x7f29e80b33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40502/0 (socket says 172.21.15.88:40502) 2023-10-16T20:05:18.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7fbe637fe640 1 -- 172.21.15.88:0/1961398657 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7400f040 con 0x7fbe7c0d2660 2023-10-16T20:05:18.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7fbe637fe640 1 -- 172.21.15.88:0/1961398657 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe740147e0 con 0x7fbe7c0d2660 2023-10-16T20:05:18.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7fbe8129e640 1 -- 172.21.15.88:0/1961398657 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe7c0cc8b0 con 0x7fbe7c0d2660 2023-10-16T20:05:18.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7f29ed194640 1 -- 172.21.15.88:0/3805510258 learned_addr learned my addr 172.21.15.88:0/3805510258 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:18.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7f29ef3f8640 1 -- 172.21.15.88:0/3805510258 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e80b23f0 con 0x7f29e815cb70 2023-10-16T20:05:18.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7f29ef3f8640 1 -- 172.21.15.88:0/3805510258 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e80b2600 con 0x7f29e80cff40 2023-10-16T20:05:18.539 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7f056a46b640 1 -- 172.21.15.88:0/4175270567 wait complete. 2023-10-16T20:05:18.539 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7f29ef3f8640 1 -- 172.21.15.88:0/3805510258 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e80b28e0 con 0x7f29e80cdad0 2023-10-16T20:05:18.540 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7f056a46b640 1 Processor -- start 2023-10-16T20:05:18.540 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7fbe8129e640 1 -- 172.21.15.88:0/1961398657 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe7c04e9c0 con 0x7fbe7c0d2660 2023-10-16T20:05:18.540 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7f056a46b640 1 -- start start 2023-10-16T20:05:18.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7f564effd640 1 -- 172.21.15.88:0/263511120 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5654003b30 con 0x7f56580c9c70 2023-10-16T20:05:18.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7f564effd640 1 --2- 172.21.15.88:0/263511120 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f563c0664d0 0x7f563c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7f564effd640 1 -- 172.21.15.88:0/263511120 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5654083980 con 0x7f56580c9c70 2023-10-16T20:05:18.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7f29ec993640 1 -- 172.21.15.88:0/3805510258 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29e80cdad0 msgr2=0x7f29e8151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7f29ec993640 1 --2- 172.21.15.88:0/3805510258 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29e80cdad0 0x7f29e8151000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7f29ec993640 1 -- 172.21.15.88:0/3805510258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29e815cb70 msgr2=0x7f29e80b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7f29ec993640 1 --2- 172.21.15.88:0/3805510258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29e815cb70 0x7f29e80b33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7f29ec993640 1 -- 172.21.15.88:0/3805510258 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e4013a10 con 0x7f29e80cff40 2023-10-16T20:05:18.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.474+0000 7f565e23a640 1 --2- 172.21.15.88:0/263511120 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f563c0664d0 0x7f563c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f7d80446640 1 -- start start 2023-10-16T20:05:18.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f8b9bfd0640 1 -- 172.21.15.88:0/677337582 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b9415c5b0 con 0x7f8b940b0500 2023-10-16T20:05:18.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f8b9bfd0640 1 -- 172.21.15.88:0/677337582 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b9413a340 con 0x7f8b940b0500 2023-10-16T20:05:18.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f565e23a640 1 --2- 172.21.15.88:0/263511120 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f563c0664d0 0x7f563c068990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f56500070b0 tx=0x7f5650009110 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f056a46b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f054c003980 0x7f056414faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f29ec993640 1 --2- 172.21.15.88:0/3805510258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29e80cff40 0x7f29e80b6420 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f29e40b4000 tx=0x7f29e40219d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f751611d640 1 --2- 172.21.15.88:0/1987442349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75100cdad0 0x7f75100b7c00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f75100c6570 tx=0x7f750801c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=552acc0e04cb8401 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f7d80446640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d780c34b0 0x7f7d78202f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f7d80446640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d780ce360 0x7f7d780b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f7d80446640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d780cecd0 0x7f7d78162620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f7d80446640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d780b5d00 con 0x7f7d780ce360 2023-10-16T20:05:18.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f7d80446640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d78206610 con 0x7f7d780c34b0 2023-10-16T20:05:18.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f7d80446640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d78203570 con 0x7f7d780cecd0 2023-10-16T20:05:18.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f751491a640 1 -- 172.21.15.88:0/1987442349 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7508004230 con 0x7f75100cdad0 2023-10-16T20:05:18.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f056a46b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0564159a50 0x7f0564159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f751491a640 1 -- 172.21.15.88:0/1987442349 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75080a7050 con 0x7f75100cdad0 2023-10-16T20:05:18.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f8b9bfd0640 1 -- 172.21.15.88:0/677337582 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b9413a990 con 0x7f8b940b0500 2023-10-16T20:05:18.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f056a46b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05641608b0 0x7f0564160d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f0563fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f054c003980 0x7f056414faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f056a46b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056415a410 con 0x7f054c003980 2023-10-16T20:05:18.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f056a46b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05640b3180 con 0x7f05641608b0 2023-10-16T20:05:18.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f056a46b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05640b2570 con 0x7f0564159a50 2023-10-16T20:05:18.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f7d7e9e3640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d780cecd0 0x7f7d78162620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f7d7e9e3640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d780cecd0 0x7f7d78162620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:33146/0 (socket says 172.21.15.88:33146) 2023-10-16T20:05:18.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f7d7e9e3640 1 -- 172.21.15.88:0/2862830094 learned_addr learned my addr 172.21.15.88:0/2862830094 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:18.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f7d7d9e1640 1 --2- 172.21.15.88:0/2862830094 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d780ce360 0x7f7d780b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.552 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f7d7e9e3640 1 -- 172.21.15.88:0/2862830094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d780c34b0 msgr2=0x7f7d78202f80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T20:05:18.552 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f7d7e9e3640 1 --2- 172.21.15.88:0/2862830094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d780c34b0 0x7f7d78202f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.552 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f7d7e9e3640 1 -- 172.21.15.88:0/2862830094 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d780ce360 msgr2=0x7f7d780b4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.552 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f7d7e9e3640 1 --2- 172.21.15.88:0/2862830094 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d780ce360 0x7f7d780b4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.552 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f7d7e9e3640 1 -- 172.21.15.88:0/2862830094 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d740094d0 con 0x7f7d780cecd0 2023-10-16T20:05:18.553 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f0568a08640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05641608b0 0x7f0564160d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.553 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f0568a08640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05641608b0 0x7f0564160d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:56880/0 (socket says 172.21.15.88:56880) 2023-10-16T20:05:18.553 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f0568a08640 1 -- 172.21.15.88:0/3368310457 learned_addr learned my addr 172.21.15.88:0/3368310457 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:18.554 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f7d7d9e1640 1 --2- 172.21.15.88:0/2862830094 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d780ce360 0x7f7d780b4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:18.554 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f05637fe640 1 --2- 172.21.15.88:0/3368310457 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0564159a50 0x7f0564159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.554 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f7d7e9e3640 1 --2- 172.21.15.88:0/2862830094 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d780cecd0 0x7f7d78162620 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7d7000b9c0 tx=0x7f7d7000be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7fbe637fe640 1 -- 172.21.15.88:0/1961398657 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbe74014940 con 0x7fbe7c0d2660 2023-10-16T20:05:18.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f751491a640 1 -- 172.21.15.88:0/1987442349 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7508004230 con 0x7f75100cdad0 2023-10-16T20:05:18.556 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f29ddffb640 1 -- 172.21.15.88:0/3805510258 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e40b5df0 con 0x7f29e80cff40 2023-10-16T20:05:18.556 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f0568a08640 1 -- 172.21.15.88:0/3368310457 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0564159a50 msgr2=0x7f0564159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.556 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f0568a08640 1 --2- 172.21.15.88:0/3368310457 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0564159a50 0x7f0564159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.556 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f0568a08640 1 -- 172.21.15.88:0/3368310457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f054c003980 msgr2=0x7f056414faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.557 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f0568a08640 1 --2- 172.21.15.88:0/3368310457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f054c003980 0x7f056414faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.557 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f29ddffb640 1 -- 172.21.15.88:0/3805510258 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e40c0480 con 0x7f29e80cff40 2023-10-16T20:05:18.558 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f0568a08640 1 -- 172.21.15.88:0/3368310457 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05640b2850 con 0x7f05641608b0 2023-10-16T20:05:18.558 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f05637fe640 1 --2- 172.21.15.88:0/3368310457 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0564159a50 0x7f0564159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:18.558 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7f29ef3f8640 1 -- 172.21.15.88:0/3805510258 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e80b2bc0 con 0x7f29e80cff40 2023-10-16T20:05:18.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7f0568a08640 1 --2- 172.21.15.88:0/3368310457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05641608b0 0x7f0564160d30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f05640c6570 tx=0x7f055c00ccd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7f0560ff9640 1 -- 172.21.15.88:0/3368310457 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c00a610 con 0x7f05641608b0 2023-10-16T20:05:18.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7f0560ff9640 1 -- 172.21.15.88:0/3368310457 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055c00a770 con 0x7f05641608b0 2023-10-16T20:05:18.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7f0560ff9640 1 -- 172.21.15.88:0/3368310457 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c00ab20 con 0x7f05641608b0 2023-10-16T20:05:18.560 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7f056a46b640 1 -- 172.21.15.88:0/3368310457 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05640b2b30 con 0x7f05641608b0 2023-10-16T20:05:18.560 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7f056a46b640 1 -- 172.21.15.88:0/3368310457 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0564161270 con 0x7f05641608b0 2023-10-16T20:05:18.560 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7f29ddffb640 1 -- 172.21.15.88:0/3805510258 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e40b5df0 con 0x7f29e80cff40 2023-10-16T20:05:18.560 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7f29ef3f8640 1 -- 172.21.15.88:0/3805510258 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e8151540 con 0x7f29e80cff40 2023-10-16T20:05:18.561 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f7d6effd640 1 -- 172.21.15.88:0/2862830094 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7000cba0 con 0x7f7d780cecd0 2023-10-16T20:05:18.561 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7f7d6effd640 1 -- 172.21.15.88:0/2862830094 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d7000cd00 con 0x7f7d780cecd0 2023-10-16T20:05:18.561 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7f7d6effd640 1 -- 172.21.15.88:0/2862830094 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d70014660 con 0x7f7d780cecd0 2023-10-16T20:05:18.561 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7f7d80446640 1 -- 172.21.15.88:0/2862830094 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d78203850 con 0x7f7d780cecd0 2023-10-16T20:05:18.561 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7f7d80446640 1 -- 172.21.15.88:0/2862830094 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d78203e10 con 0x7f7d780cecd0 2023-10-16T20:05:18.562 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7fbe637fe640 1 --2- 172.21.15.88:0/1961398657 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbe680664d0 0x7fbe68068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.562 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7fbe637fe640 1 -- 172.21.15.88:0/1961398657 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbe74085490 con 0x7fbe7c0d2660 2023-10-16T20:05:18.562 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7fbe7a7fc640 1 --2- 172.21.15.88:0/1961398657 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbe680664d0 0x7fbe68068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.563 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.478+0000 7f8b8a7fc640 1 -- 172.21.15.88:0/677337582 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8b8400cca0 con 0x7f8b940b0500 2023-10-16T20:05:18.563 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7f8b8a7fc640 1 --2- 172.21.15.88:0/677337582 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f8b78066570 0x7f8b78068a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.563 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7f8b8a7fc640 1 -- 172.21.15.88:0/677337582 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8b84084210 con 0x7f8b940b0500 2023-10-16T20:05:18.564 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7f564effd640 1 -- 172.21.15.88:0/263511120 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5654050c30 con 0x7f56580c9c70 2023-10-16T20:05:18.564 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7f7517b80640 1 -- 172.21.15.88:0/1987442349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75100cdad0 msgr2=0x7f75100b7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.564 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7f7517b80640 1 --2- 172.21.15.88:0/1987442349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75100cdad0 0x7f75100b7c00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f75100c6570 tx=0x7f750801c460 comp rx=0 tx=0).stop 2023-10-16T20:05:18.565 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7f8b9956b640 1 --2- 172.21.15.88:0/677337582 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f8b78066570 0x7f8b78068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.565 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7f054a7fc640 1 -- 172.21.15.88:0/3368310457 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f053c005260 con 0x7f05641608b0 2023-10-16T20:05:18.566 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7f29ef3f8640 1 -- 172.21.15.88:0/3805510258 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e804e9c0 con 0x7f29e80cff40 2023-10-16T20:05:18.566 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7f7d80446640 1 -- 172.21.15.88:0/2862830094 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d78206810 con 0x7f7d780cecd0 2023-10-16T20:05:18.566 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7fbe7a7fc640 1 --2- 172.21.15.88:0/1961398657 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbe680664d0 0x7fbe68068990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c0ad000 tx=0x7fbe6c0a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.567 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7f7517b80640 1 -- 172.21.15.88:0/1987442349 shutdown_connections 2023-10-16T20:05:18.567 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7f7517b80640 1 --2- 172.21.15.88:0/1987442349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75100cdad0 0x7f75100b7c00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.567 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7f7517b80640 1 --2- 172.21.15.88:0/1987442349 >> v2:172.21.15.88:3300/0 conn(0x7f75100d0110 0x7f7510161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7f7517b80640 1 -- 172.21.15.88:0/1987442349 >> 172.21.15.88:0/1987442349 conn(0x7f75100719c0 msgr2=0x7f7510071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:18.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7f7517b80640 1 -- 172.21.15.88:0/1987442349 shutdown_connections 2023-10-16T20:05:18.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7f7517b80640 1 -- 172.21.15.88:0/1987442349 wait complete. 2023-10-16T20:05:18.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.482+0000 7f8b9956b640 1 --2- 172.21.15.88:0/677337582 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f8b78066570 0x7f8b78068a30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8b94072130 tx=0x7f8b90009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.569 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.486+0000 7f29ddffb640 1 -- 172.21.15.88:0/3805510258 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f29e40c0610 con 0x7f29e80cff40 2023-10-16T20:05:18.569 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.486+0000 7f0560ff9640 1 -- 172.21.15.88:0/3368310457 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f055c00ac80 con 0x7f05641608b0 2023-10-16T20:05:18.570 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.486+0000 7f0560ff9640 1 --2- 172.21.15.88:0/3368310457 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0550066520 0x7f05500689e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.570 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.486+0000 7f0560ff9640 1 -- 172.21.15.88:0/3368310457 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f055c083930 con 0x7f05641608b0 2023-10-16T20:05:18.570 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.486+0000 7f7517b80640 1 Processor -- start 2023-10-16T20:05:18.571 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.486+0000 7f29ddffb640 1 --2- 172.21.15.88:0/3805510258 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f29cc066500 0x7f29cc0689c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.571 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.486+0000 7f0563fff640 1 --2- 172.21.15.88:0/3368310457 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0550066520 0x7f05500689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.571 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.486+0000 7f8b8a7fc640 1 -- 172.21.15.88:0/677337582 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8b84051430 con 0x7f8b940b0500 2023-10-16T20:05:18.572 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.486+0000 7f29ddffb640 1 -- 172.21.15.88:0/3805510258 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f29e4127090 con 0x7f29e80cff40 2023-10-16T20:05:18.572 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.486+0000 7f29ed194640 1 --2- 172.21.15.88:0/3805510258 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f29cc066500 0x7f29cc0689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.572 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.486+0000 7f0563fff640 1 --2- 172.21.15.88:0/3368310457 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0550066520 0x7f05500689e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f05580af000 tx=0x7f05580046d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.573 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.486+0000 7f7517b80640 1 -- start start 2023-10-16T20:05:18.573 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.486+0000 7f7517b80640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f75100cdad0 0x7f751015b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.573 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.486+0000 7f7517b80640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75100d0110 0x7f75100b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.574 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.486+0000 7f7517b80640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f751015cb70 0x7f75100b3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.574 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.486+0000 7f7517b80640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751015b9a0 con 0x7f751015cb70 2023-10-16T20:05:18.574 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.486+0000 7f7517b80640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75100cbcc0 con 0x7f75100d0110 2023-10-16T20:05:18.574 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.486+0000 7f7517b80640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75100cbfa0 con 0x7f75100cdad0 2023-10-16T20:05:18.575 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.486+0000 7f7d6effd640 1 -- 172.21.15.88:0/2862830094 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7d7001d3b0 con 0x7f7d780cecd0 2023-10-16T20:05:18.575 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.486+0000 7f29ed194640 1 --2- 172.21.15.88:0/3805510258 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f29cc066500 0x7f29cc0689c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f29e0003b40 tx=0x7f29e00098b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.575 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.486+0000 7fbe637fe640 1 -- 172.21.15.88:0/1961398657 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbe740526b0 con 0x7fbe7c0d2660 2023-10-16T20:05:18.576 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.486+0000 7f751511b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75100d0110 0x7f75100b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.577 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.486+0000 7f751591c640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f75100cdad0 0x7f751015b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.577 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.490+0000 7f751511b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75100d0110 0x7f75100b3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:56892/0 (socket says 172.21.15.88:56892) 2023-10-16T20:05:18.577 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.490+0000 7f751511b640 1 -- 172.21.15.88:0/2514620508 learned_addr learned my addr 172.21.15.88:0/2514620508 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:18.577 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.490+0000 7f751591c640 1 -- 172.21.15.88:0/2514620508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75100d0110 msgr2=0x7f75100b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.577 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.490+0000 7f751591c640 1 --2- 172.21.15.88:0/2514620508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75100d0110 0x7f75100b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.577 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.490+0000 7f751591c640 1 -- 172.21.15.88:0/2514620508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f751015cb70 msgr2=0x7f75100b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.578 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.490+0000 7f751591c640 1 --2- 172.21.15.88:0/2514620508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f751015cb70 0x7f75100b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.578 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.490+0000 7f751591c640 1 -- 172.21.15.88:0/2514620508 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7508009420 con 0x7f75100cdad0 2023-10-16T20:05:18.578 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.490+0000 7f751591c640 1 --2- 172.21.15.88:0/2514620508 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f75100cdad0 0x7f751015b460 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f750400ca70 tx=0x7f750400ce70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.578 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.490+0000 7f750e7fc640 1 -- 172.21.15.88:0/2514620508 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750400ab10 con 0x7f75100cdad0 2023-10-16T20:05:18.578 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.490+0000 7f750e7fc640 1 -- 172.21.15.88:0/2514620508 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f750400ac70 con 0x7f75100cdad0 2023-10-16T20:05:18.579 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.490+0000 7f750e7fc640 1 -- 172.21.15.88:0/2514620508 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7504012620 con 0x7f75100cdad0 2023-10-16T20:05:18.579 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.490+0000 7f7517b80640 1 -- 172.21.15.88:0/2514620508 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75100c9b60 con 0x7f75100cdad0 2023-10-16T20:05:18.579 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.490+0000 7f7517b80640 1 -- 172.21.15.88:0/2514620508 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75100ca120 con 0x7f75100cdad0 2023-10-16T20:05:18.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.490+0000 7f7d6effd640 1 --2- 172.21.15.88:0/2862830094 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7d5c066570 0x7f7d5c068a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.490+0000 7f7d6effd640 1 -- 172.21.15.88:0/2862830094 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7d70084400 con 0x7f7d780cecd0 2023-10-16T20:05:18.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.490+0000 7f7d7e1e2640 1 --2- 172.21.15.88:0/2862830094 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7d5c066570 0x7f7d5c068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.494+0000 7f7517b80640 1 -- 172.21.15.88:0/2514620508 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75100ca340 con 0x7f75100cdad0 2023-10-16T20:05:18.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.494+0000 7f29ddffb640 1 -- 172.21.15.88:0/3805510258 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f29e40f4180 con 0x7f29e80cff40 2023-10-16T20:05:18.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.494+0000 7f7d7e1e2640 1 --2- 172.21.15.88:0/2862830094 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7d5c066570 0x7f7d5c068a30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7d740ae000 tx=0x7f7d74008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.494+0000 7f7d6effd640 1 -- 172.21.15.88:0/2862830094 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7d70051570 con 0x7f7d780cecd0 2023-10-16T20:05:18.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.494+0000 7f750e7fc640 1 -- 172.21.15.88:0/2514620508 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7504007070 con 0x7f75100cdad0 2023-10-16T20:05:18.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.494+0000 7f750e7fc640 1 --2- 172.21.15.88:0/2514620508 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f74f40664a0 0x7f74f4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:18.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.498+0000 7f750e7fc640 1 -- 172.21.15.88:0/2514620508 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7504083f90 con 0x7f75100cdad0 2023-10-16T20:05:18.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.494+0000 7f0560ff9640 1 -- 172.21.15.88:0/3368310457 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f055c050ba0 con 0x7f05641608b0 2023-10-16T20:05:18.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.498+0000 7f751511b640 1 --2- 172.21.15.88:0/2514620508 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f74f40664a0 0x7f74f4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:18.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.498+0000 7f751511b640 1 --2- 172.21.15.88:0/2514620508 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f74f40664a0 0x7f74f4068960 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f75100cab90 tx=0x7f74fc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:18.584 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.502+0000 7f750e7fc640 1 -- 172.21.15.88:0/2514620508 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7504051130 con 0x7f75100cdad0 2023-10-16T20:05:18.795 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.794+0000 7f2efe409640 1 -- 172.21.15.88:0/1487792970 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2ed80057e0 con 0x7f2ee4005190 2023-10-16T20:05:18.797 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.798+0000 7f2eedffb640 1 -- 172.21.15.88:0/1487792970 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2ef4054770 con 0x7f2ee4005190 2023-10-16T20:05:18.799 INFO:teuthology.orchestra.run.smithi088.stdout:64424509448 2023-10-16T20:05:18.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.802+0000 7f2efe409640 1 -- 172.21.15.88:0/1487792970 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2edc066500 msgr2=0x7f2edc0689c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.802+0000 7f2efe409640 1 --2- 172.21.15.88:0/1487792970 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2edc066500 0x7f2edc0689c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0057930 tx=0x7f2ee8009040 comp rx=0 tx=0).stop 2023-10-16T20:05:18.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.802+0000 7f2efe409640 1 -- 172.21.15.88:0/1487792970 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2ee4005190 msgr2=0x7f2ef004a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.802+0000 7f2efe409640 1 --2- 172.21.15.88:0/1487792970 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2ee4005190 0x7f2ef004a8b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2ef400d470 tx=0x7f2ef400d930 comp rx=0 tx=0).stop 2023-10-16T20:05:18.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.802+0000 7f2efe409640 1 -- 172.21.15.88:0/1487792970 shutdown_connections 2023-10-16T20:05:18.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.802+0000 7f2efe409640 1 --2- 172.21.15.88:0/1487792970 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2edc066500 0x7f2edc0689c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.802+0000 7f2efe409640 1 --2- 172.21.15.88:0/1487792970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ef00e1940 0x7f2ef00ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.802+0000 7f2efe409640 1 --2- 172.21.15.88:0/1487792970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ef00e2370 0x7f2ef004adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.802+0000 7f2efe409640 1 --2- 172.21.15.88:0/1487792970 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2ee4005190 0x7f2ef004a8b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.802+0000 7f2efe409640 1 -- 172.21.15.88:0/1487792970 >> 172.21.15.88:0/1487792970 conn(0x7f2ef0056e30 msgr2=0x7f2ef00e2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:18.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.802+0000 7f2efe409640 1 -- 172.21.15.88:0/1487792970 shutdown_connections 2023-10-16T20:05:18.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.802+0000 7f2efe409640 1 -- 172.21.15.88:0/1487792970 wait complete. 2023-10-16T20:05:18.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.802+0000 7f566049e640 1 -- 172.21.15.88:0/263511120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f56580d2360 con 0x7f56580c9c70 2023-10-16T20:05:18.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.806+0000 7f564effd640 1 -- 172.21.15.88:0/263511120 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5654017020 con 0x7f56580c9c70 2023-10-16T20:05:18.808 INFO:teuthology.orchestra.run.smithi088.stdout:64424509447 2023-10-16T20:05:18.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.810+0000 7f566049e640 1 -- 172.21.15.88:0/263511120 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f563c0664d0 msgr2=0x7f563c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.810+0000 7f566049e640 1 --2- 172.21.15.88:0/263511120 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f563c0664d0 0x7f563c068990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f56500070b0 tx=0x7f5650009110 comp rx=0 tx=0).stop 2023-10-16T20:05:18.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.810+0000 7f566049e640 1 -- 172.21.15.88:0/263511120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56580c9c70 msgr2=0x7f56580ca0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.810+0000 7f566049e640 1 --2- 172.21.15.88:0/263511120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56580c9c70 0x7f56580ca0f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f56580cd1a0 tx=0x7f565400c870 comp rx=0 tx=0).stop 2023-10-16T20:05:18.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.810+0000 7fbe8129e640 1 -- 172.21.15.88:0/1961398657 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbe7c0cfed0 con 0x7fbe7c0d2660 2023-10-16T20:05:18.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.810+0000 7f566049e640 1 -- 172.21.15.88:0/263511120 shutdown_connections 2023-10-16T20:05:18.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.810+0000 7f566049e640 1 --2- 172.21.15.88:0/263511120 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f563c0664d0 0x7f563c068990 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.810+0000 7f566049e640 1 --2- 172.21.15.88:0/263511120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56580c9c70 0x7f56580ca0f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.810+0000 7f566049e640 1 --2- 172.21.15.88:0/263511120 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5658206490 0x7f56580d2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.810+0000 7f566049e640 1 --2- 172.21.15.88:0/263511120 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5658153ea0 0x7f56580d2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.810+0000 7f566049e640 1 -- 172.21.15.88:0/263511120 >> 172.21.15.88:0/263511120 conn(0x7f56580715c0 msgr2=0x7f565814ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:18.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.814+0000 7f566049e640 1 -- 172.21.15.88:0/263511120 shutdown_connections 2023-10-16T20:05:18.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.814+0000 7f566049e640 1 -- 172.21.15.88:0/263511120 wait complete. 2023-10-16T20:05:18.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.814+0000 7fbe637fe640 1 -- 172.21.15.88:0/1961398657 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbe74055cd0 con 0x7fbe7c0d2660 2023-10-16T20:05:18.816 INFO:teuthology.orchestra.run.smithi088.stdout:64424509447 2023-10-16T20:05:18.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.814+0000 7f8b9bfd0640 1 -- 172.21.15.88:0/677337582 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8b94139ec0 con 0x7f8b940b0500 2023-10-16T20:05:18.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.818+0000 7fbe8129e640 1 -- 172.21.15.88:0/1961398657 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbe680664d0 msgr2=0x7fbe68068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.818+0000 7fbe8129e640 1 --2- 172.21.15.88:0/1961398657 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbe680664d0 0x7fbe68068990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c0ad000 tx=0x7fbe6c0a6000 comp rx=0 tx=0).stop 2023-10-16T20:05:18.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.818+0000 7fbe8129e640 1 -- 172.21.15.88:0/1961398657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe7c0d2660 msgr2=0x7fbe7c14bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.818+0000 7fbe8129e640 1 --2- 172.21.15.88:0/1961398657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe7c0d2660 0x7fbe7c14bd70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbe7400efc0 tx=0x7fbe7400c490 comp rx=0 tx=0).stop 2023-10-16T20:05:18.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.818+0000 7fbe8129e640 1 -- 172.21.15.88:0/1961398657 shutdown_connections 2023-10-16T20:05:18.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.818+0000 7f8b8a7fc640 1 -- 172.21.15.88:0/677337582 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8b94139ec0 con 0x7f8b940b0500 2023-10-16T20:05:18.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.818+0000 7fbe8129e640 1 --2- 172.21.15.88:0/1961398657 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbe680664d0 0x7fbe68068990 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c0ad000 tx=0x7fbe6c0a6000 comp rx=0 tx=0).stop 2023-10-16T20:05:18.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.818+0000 7fbe8129e640 1 --2- 172.21.15.88:0/1961398657 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbe7c1f9980 0x7fbe7c1f9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.818+0000 7fbe8129e640 1 --2- 172.21.15.88:0/1961398657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe7c153ea0 0x7fbe7c0ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.818+0000 7fbe8129e640 1 --2- 172.21.15.88:0/1961398657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe7c0d2660 0x7fbe7c14bd70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.818+0000 7fbe8129e640 1 -- 172.21.15.88:0/1961398657 >> 172.21.15.88:0/1961398657 conn(0x7fbe7c0715c0 msgr2=0x7fbe7c0cf3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:18.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.818+0000 7fbe8129e640 1 -- 172.21.15.88:0/1961398657 shutdown_connections 2023-10-16T20:05:18.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.818+0000 7f29ef3f8640 1 -- 172.21.15.88:0/3805510258 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f29e815df60 con 0x7f29e80cff40 2023-10-16T20:05:18.821 INFO:teuthology.orchestra.run.smithi088.stdout:55834574857 2023-10-16T20:05:18.822 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.818+0000 7f7d80446640 1 -- 172.21.15.88:0/2862830094 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7d782067e0 con 0x7f7d780cecd0 2023-10-16T20:05:18.822 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.818+0000 7f29ddffb640 1 -- 172.21.15.88:0/3805510258 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f29e40ba020 con 0x7f29e80cff40 2023-10-16T20:05:18.822 INFO:teuthology.orchestra.run.smithi088.stdout:64424509447 2023-10-16T20:05:18.823 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.822+0000 7fbe8129e640 1 -- 172.21.15.88:0/1961398657 wait complete. 2023-10-16T20:05:18.823 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.822+0000 7f29ef3f8640 1 -- 172.21.15.88:0/3805510258 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f29cc066500 msgr2=0x7f29cc0689c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.822+0000 7f29ef3f8640 1 --2- 172.21.15.88:0/3805510258 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f29cc066500 0x7f29cc0689c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f29e0003b40 tx=0x7f29e00098b0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.822+0000 7f8b9bfd0640 1 -- 172.21.15.88:0/677337582 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f8b78066570 msgr2=0x7f8b78068a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.822+0000 7f8b9bfd0640 1 --2- 172.21.15.88:0/677337582 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f8b78066570 0x7f8b78068a30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8b94072130 tx=0x7f8b90009040 comp rx=0 tx=0).stop 2023-10-16T20:05:18.825 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.822+0000 7f29ef3f8640 1 -- 172.21.15.88:0/3805510258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29e80cff40 msgr2=0x7f29e80b6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.825 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.822+0000 7f29ef3f8640 1 --2- 172.21.15.88:0/3805510258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29e80cff40 0x7f29e80b6420 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f29e40b4000 tx=0x7f29e40219d0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.825 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.822+0000 7f8b9bfd0640 1 -- 172.21.15.88:0/677337582 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8b940b0500 msgr2=0x7f8b940716b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.826 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.822+0000 7f8b9bfd0640 1 --2- 172.21.15.88:0/677337582 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8b940b0500 0x7f8b940716b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400e870 tx=0x7f8b8400ec70 comp rx=0 tx=0).stop 2023-10-16T20:05:18.826 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.822+0000 7f29ef3f8640 1 -- 172.21.15.88:0/3805510258 shutdown_connections 2023-10-16T20:05:18.826 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.822+0000 7f29ef3f8640 1 --2- 172.21.15.88:0/3805510258 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f29cc066500 0x7f29cc0689c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.826 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.822+0000 7f29ef3f8640 1 --2- 172.21.15.88:0/3805510258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29e815cb70 0x7f29e80b33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.827 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.822+0000 7f29ef3f8640 1 --2- 172.21.15.88:0/3805510258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29e80cff40 0x7f29e80b6420 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.827 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.822+0000 7f7d6effd640 1 -- 172.21.15.88:0/2862830094 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7d70019020 con 0x7f7d780cecd0 2023-10-16T20:05:18.828 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.822+0000 7f29ef3f8640 1 --2- 172.21.15.88:0/3805510258 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29e80cdad0 0x7f29e8151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.828 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.822+0000 7f29ef3f8640 1 -- 172.21.15.88:0/3805510258 >> 172.21.15.88:0/3805510258 conn(0x7f29e80719c0 msgr2=0x7f29e815eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:18.828 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.822+0000 7f29ef3f8640 1 -- 172.21.15.88:0/3805510258 shutdown_connections 2023-10-16T20:05:18.828 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.822+0000 7f29ef3f8640 1 -- 172.21.15.88:0/3805510258 wait complete. 2023-10-16T20:05:18.829 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.822+0000 7f054a7fc640 1 -- 172.21.15.88:0/3368310457 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f053c0057e0 con 0x7f05641608b0 2023-10-16T20:05:18.829 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.822+0000 7f8b9bfd0640 1 -- 172.21.15.88:0/677337582 shutdown_connections 2023-10-16T20:05:18.829 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.822+0000 7f8b9bfd0640 1 --2- 172.21.15.88:0/677337582 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f8b78066570 0x7f8b78068a30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.830 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.822+0000 7f8b9bfd0640 1 --2- 172.21.15.88:0/677337582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b9415eff0 0x7f8b941581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.830 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.822+0000 7f8b9bfd0640 1 --2- 172.21.15.88:0/677337582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b94143500 0x7f8b94157c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.830 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.822+0000 7f8b9bfd0640 1 --2- 172.21.15.88:0/677337582 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8b940b0500 0x7f8b940716b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.830 INFO:teuthology.orchestra.run.smithi088.stdout:64424509448 2023-10-16T20:05:18.831 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.822+0000 7f8b9bfd0640 1 -- 172.21.15.88:0/677337582 >> 172.21.15.88:0/677337582 conn(0x7f8b940ce5e0 msgr2=0x7f8b940cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:18.831 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.822+0000 7f8b9bfd0640 1 -- 172.21.15.88:0/677337582 shutdown_connections 2023-10-16T20:05:18.831 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.822+0000 7f8b9bfd0640 1 -- 172.21.15.88:0/677337582 wait complete. 2023-10-16T20:05:18.832 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.822+0000 7f0560ff9640 1 -- 172.21.15.88:0/3368310457 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f055c0541c0 con 0x7f05641608b0 2023-10-16T20:05:18.832 INFO:teuthology.orchestra.run.smithi088.stdout:64424509449 2023-10-16T20:05:18.833 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.826+0000 7f7517b80640 1 -- 172.21.15.88:0/2514620508 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f75100cc0e0 con 0x7f75100cdad0 2023-10-16T20:05:18.833 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.826+0000 7f750e7fc640 1 -- 172.21.15.88:0/2514620508 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f75100cc0e0 con 0x7f75100cdad0 2023-10-16T20:05:18.833 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.826+0000 7f7d80446640 1 -- 172.21.15.88:0/2862830094 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7d5c066570 msgr2=0x7f7d5c068a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.834 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.826+0000 7f7d80446640 1 --2- 172.21.15.88:0/2862830094 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7d5c066570 0x7f7d5c068a30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7d740ae000 tx=0x7f7d74008040 comp rx=0 tx=0).stop 2023-10-16T20:05:18.834 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.826+0000 7f7d80446640 1 -- 172.21.15.88:0/2862830094 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d780cecd0 msgr2=0x7f7d78162620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.834 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.826+0000 7f7d80446640 1 --2- 172.21.15.88:0/2862830094 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d780cecd0 0x7f7d78162620 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7d7000b9c0 tx=0x7f7d7000be80 comp rx=0 tx=0).stop 2023-10-16T20:05:18.834 INFO:teuthology.orchestra.run.smithi088.stdout:64424509448 2023-10-16T20:05:18.835 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.826+0000 7f054a7fc640 1 -- 172.21.15.88:0/3368310457 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0550066520 msgr2=0x7f05500689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.835 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.826+0000 7f054a7fc640 1 --2- 172.21.15.88:0/3368310457 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0550066520 0x7f05500689e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f05580af000 tx=0x7f05580046d0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.835 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.826+0000 7f054a7fc640 1 -- 172.21.15.88:0/3368310457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05641608b0 msgr2=0x7f0564160d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.836 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.826+0000 7f054a7fc640 1 --2- 172.21.15.88:0/3368310457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05641608b0 0x7f0564160d30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f05640c6570 tx=0x7f055c00ccd0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.836 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.826+0000 7f054a7fc640 1 -- 172.21.15.88:0/3368310457 shutdown_connections 2023-10-16T20:05:18.836 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.826+0000 7f054a7fc640 1 --2- 172.21.15.88:0/3368310457 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0550066520 0x7f05500689e0 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f05580af000 tx=0x7f05580046d0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.836 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.826+0000 7f054a7fc640 1 --2- 172.21.15.88:0/3368310457 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0564159a50 0x7f0564159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.836 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.826+0000 7f054a7fc640 1 --2- 172.21.15.88:0/3368310457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05641608b0 0x7f0564160d30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.837 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.826+0000 7f054a7fc640 1 --2- 172.21.15.88:0/3368310457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f054c003980 0x7f056414faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.837 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.826+0000 7f054a7fc640 1 -- 172.21.15.88:0/3368310457 >> 172.21.15.88:0/3368310457 conn(0x7f05640719c0 msgr2=0x7f056415be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:18.837 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.826+0000 7f7d80446640 1 -- 172.21.15.88:0/2862830094 shutdown_connections 2023-10-16T20:05:18.837 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.826+0000 7f7d80446640 1 --2- 172.21.15.88:0/2862830094 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7d5c066570 0x7f7d5c068a30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.837 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.826+0000 7f7d80446640 1 --2- 172.21.15.88:0/2862830094 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d780cecd0 0x7f7d78162620 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.838 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.826+0000 7f7d80446640 1 --2- 172.21.15.88:0/2862830094 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d780ce360 0x7f7d780b4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.838 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.826+0000 7f7d80446640 1 --2- 172.21.15.88:0/2862830094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d780c34b0 0x7f7d78202f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.838 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.826+0000 7f7d80446640 1 -- 172.21.15.88:0/2862830094 >> 172.21.15.88:0/2862830094 conn(0x7f7d780d0820 msgr2=0x7f7d7815d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:18.838 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.830+0000 7f054a7fc640 1 -- 172.21.15.88:0/3368310457 shutdown_connections 2023-10-16T20:05:18.839 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.826+0000 7f7d80446640 1 -- 172.21.15.88:0/2862830094 shutdown_connections 2023-10-16T20:05:18.839 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.826+0000 7f7d80446640 1 -- 172.21.15.88:0/2862830094 wait complete. 2023-10-16T20:05:18.839 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.830+0000 7f054a7fc640 1 -- 172.21.15.88:0/3368310457 wait complete. 2023-10-16T20:05:18.840 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.830+0000 7f7517b80640 1 -- 172.21.15.88:0/2514620508 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f74f40664a0 msgr2=0x7f74f4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.840 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.830+0000 7f7517b80640 1 --2- 172.21.15.88:0/2514620508 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f74f40664a0 0x7f74f4068960 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f75100cab90 tx=0x7f74fc008040 comp rx=0 tx=0).stop 2023-10-16T20:05:18.841 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.830+0000 7f7517b80640 1 -- 172.21.15.88:0/2514620508 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f75100cdad0 msgr2=0x7f751015b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:18.841 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.830+0000 7f7517b80640 1 --2- 172.21.15.88:0/2514620508 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f75100cdad0 0x7f751015b460 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f750400ca70 tx=0x7f750400ce70 comp rx=0 tx=0).stop 2023-10-16T20:05:18.841 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.830+0000 7f7517b80640 1 -- 172.21.15.88:0/2514620508 shutdown_connections 2023-10-16T20:05:18.841 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.830+0000 7f7517b80640 1 --2- 172.21.15.88:0/2514620508 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f74f40664a0 0x7f74f4068960 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.841 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.830+0000 7f7517b80640 1 --2- 172.21.15.88:0/2514620508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f751015cb70 0x7f75100b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.830+0000 7f7517b80640 1 --2- 172.21.15.88:0/2514620508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75100d0110 0x7f75100b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.830+0000 7f7517b80640 1 --2- 172.21.15.88:0/2514620508 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f75100cdad0 0x7f751015b460 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:18.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.830+0000 7f7517b80640 1 -- 172.21.15.88:0/2514620508 >> 172.21.15.88:0/2514620508 conn(0x7f75100719c0 msgr2=0x7f7510167fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:18.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.834+0000 7f7517b80640 1 -- 172.21.15.88:0/2514620508 shutdown_connections 2023-10-16T20:05:18.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:18.834+0000 7f7517b80640 1 -- 172.21.15.88:0/2514620508 wait complete. 2023-10-16T20:05:18.843 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.1 2023-10-16T20:05:18.857 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.4 2023-10-16T20:05:18.867 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.6 2023-10-16T20:05:18.867 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.0 2023-10-16T20:05:18.868 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.5 2023-10-16T20:05:18.869 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.2 2023-10-16T20:05:18.870 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.3 2023-10-16T20:05:18.873 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.7 2023-10-16T20:05:19.844 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-16T20:05:19.857 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-16T20:05:19.868 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-16T20:05:19.869 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-16T20:05:19.869 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-16T20:05:19.870 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-16T20:05:19.871 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-16T20:05:19.874 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-16T20:05:20.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.070+0000 7fb7f95b5640 1 Processor -- start 2023-10-16T20:05:20.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.070+0000 7fb7f95b5640 1 -- start start 2023-10-16T20:05:20.073 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.070+0000 7fb7f95b5640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb7f415c8e0 0x7fb7f40c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.073 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.070+0000 7fb7f95b5640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb7f415ccc0 0x7fb7f40aff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.074 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.074+0000 7fb7f95b5640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f40cbe40 con 0x7fb7f40b04d0 2023-10-16T20:05:20.074 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.074+0000 7fb7f95b5640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7f40c93d0 con 0x7fb7f415c8e0 2023-10-16T20:05:20.074 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.074+0000 7fb7f95b5640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f40b0910 con 0x7fb7f415ccc0 2023-10-16T20:05:20.077 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.078+0000 7fb7f37fe640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb7f415ccc0 0x7fb7f40aff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.077 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.078+0000 7fb7f3fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb7f415c8e0 0x7fb7f40c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.078+0000 7fb7f37fe640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb7f415ccc0 0x7fb7f40aff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:33170/0 (socket says 172.21.15.88:33170) 2023-10-16T20:05:20.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.078+0000 7fb7f37fe640 1 -- 172.21.15.88:0/2860980232 learned_addr learned my addr 172.21.15.88:0/2860980232 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:20.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.078+0000 7fb7f37fe640 1 -- 172.21.15.88:0/2860980232 >> v2:172.21.15.88:3300/0 conn(0x7fb7f415c8e0 msgr2=0x7fb7f40c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:20.081 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.078+0000 7fb7f37fe640 1 --2- 172.21.15.88:0/2860980232 >> v2:172.21.15.88:3300/0 conn(0x7fb7f415c8e0 0x7fb7f40c8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.081 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.078+0000 7fb7f37fe640 1 -- 172.21.15.88:0/2860980232 >> v1:172.21.15.88:6789/0 conn(0x7fb7f40b04d0 legacy=0x7fb7f41606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:20.081 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.078+0000 7fb7f37fe640 1 -- 172.21.15.88:0/2860980232 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f40c9910 con 0x7fb7f415ccc0 2023-10-16T20:05:20.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.086+0000 7fb7f37fe640 1 --2- 172.21.15.88:0/2860980232 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb7f415ccc0 0x7fb7f40aff90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009870 tx=0x7fb7e401c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca9f69e9a54a7a53 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:20.086 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.086+0000 7fb7f2ffd640 1 -- 172.21.15.88:0/2860980232 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e40ab600 con 0x7fb7f415ccc0 2023-10-16T20:05:20.086 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.086+0000 7fb7f2ffd640 1 -- 172.21.15.88:0/2860980232 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e40a7050 con 0x7fb7f415ccc0 2023-10-16T20:05:20.086 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.086+0000 7fb7f2ffd640 1 -- 172.21.15.88:0/2860980232 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e40ab980 con 0x7fb7f415ccc0 2023-10-16T20:05:20.087 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.086+0000 7fb7f95b5640 1 -- 172.21.15.88:0/2860980232 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb7f415ccc0 msgr2=0x7fb7f40aff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.088 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.086+0000 7fb7f95b5640 1 --2- 172.21.15.88:0/2860980232 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb7f415ccc0 0x7fb7f40aff90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009870 tx=0x7fb7e401c3b0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.088 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.086+0000 7fb7f95b5640 1 -- 172.21.15.88:0/2860980232 shutdown_connections 2023-10-16T20:05:20.088 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.086+0000 7fb7f95b5640 1 --2- 172.21.15.88:0/2860980232 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb7f415ccc0 0x7fb7f40aff90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.088 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.086+0000 7fb7f95b5640 1 --2- 172.21.15.88:0/2860980232 >> v2:172.21.15.88:3300/0 conn(0x7fb7f415c8e0 0x7fb7f40c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.089 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.086+0000 7fb7f95b5640 1 -- 172.21.15.88:0/2860980232 >> 172.21.15.88:0/2860980232 conn(0x7fb7f406c170 msgr2=0x7fb7f406c580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:20.089 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.086+0000 7fb7f95b5640 1 -- 172.21.15.88:0/2860980232 shutdown_connections 2023-10-16T20:05:20.089 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.086+0000 7fb7f95b5640 1 -- 172.21.15.88:0/2860980232 wait complete. 2023-10-16T20:05:20.090 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.090+0000 7fb7f95b5640 1 Processor -- start 2023-10-16T20:05:20.091 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.090+0000 7fb7f95b5640 1 -- start start 2023-10-16T20:05:20.093 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.094+0000 7fb7f95b5640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb7f40b04d0 0x7fb7f4155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.094 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.094+0000 7fb7f95b5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7f415c8e0 0x7fb7f4153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.094 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.094+0000 7fb7f95b5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7f415ccc0 0x7fb7f415b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.094+0000 7fb7f37fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7f415c8e0 0x7fb7f4153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.094+0000 7fb7f37fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7f415c8e0 0x7fb7f4153c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40528/0 (socket says 172.21.15.88:40528) 2023-10-16T20:05:20.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.094+0000 7fb7f95b5640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f4155e70 con 0x7fb7f415c8e0 2023-10-16T20:05:20.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.094+0000 7fb7f95b5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f41554a0 con 0x7fb7f415ccc0 2023-10-16T20:05:20.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.094+0000 7fb7f95b5640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f4154890 con 0x7fb7f40b04d0 2023-10-16T20:05:20.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.094+0000 7fb7f37fe640 1 -- 172.21.15.88:0/2038259066 learned_addr learned my addr 172.21.15.88:0/2038259066 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:20.096 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.094+0000 7fb7f8db4640 1 --2- 172.21.15.88:0/2038259066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7f415ccc0 0x7fb7f415b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.096 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.094+0000 7fb7f3fff640 1 --2- 172.21.15.88:0/2038259066 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb7f40b04d0 0x7fb7f4155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.096 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.094+0000 7fb7f8db4640 1 -- 172.21.15.88:0/2038259066 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb7f40b04d0 msgr2=0x7fb7f4155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.097 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.094+0000 7fb7f8db4640 1 --2- 172.21.15.88:0/2038259066 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb7f40b04d0 0x7fb7f4155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.097 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.094+0000 7fb7f8db4640 1 -- 172.21.15.88:0/2038259066 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7f415c8e0 msgr2=0x7fb7f4153c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.097 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.094+0000 7fb7f8db4640 1 --2- 172.21.15.88:0/2038259066 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7f415c8e0 0x7fb7f4153c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.097 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.094+0000 7fb7f8db4640 1 -- 172.21.15.88:0/2038259066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e40094d0 con 0x7fb7f415ccc0 2023-10-16T20:05:20.098 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.094+0000 7fb7f8db4640 1 --2- 172.21.15.88:0/2038259066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7f415ccc0 0x7fb7f415b030 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00c7b0 tx=0x7fb7ec00cc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:20.098 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.098+0000 7fb7f0ff9640 1 -- 172.21.15.88:0/2038259066 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec003d90 con 0x7fb7f415ccc0 2023-10-16T20:05:20.098 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.098+0000 7fb7f0ff9640 1 -- 172.21.15.88:0/2038259066 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7ec00ce60 con 0x7fb7f415ccc0 2023-10-16T20:05:20.099 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.098+0000 7fb7f0ff9640 1 -- 172.21.15.88:0/2038259066 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec0052b0 con 0x7fb7f415ccc0 2023-10-16T20:05:20.099 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.098+0000 7fb7f95b5640 1 -- 172.21.15.88:0/2038259066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7f4154b70 con 0x7fb7f415ccc0 2023-10-16T20:05:20.099 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.098+0000 7fb7f95b5640 1 -- 172.21.15.88:0/2038259066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f4155130 con 0x7fb7f415ccc0 2023-10-16T20:05:20.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.102+0000 7fb7da7fc640 1 -- 172.21.15.88:0/2038259066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7cc005260 con 0x7fb7f415ccc0 2023-10-16T20:05:20.105 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.102+0000 7fb7f0ff9640 1 -- 172.21.15.88:0/2038259066 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb7ec006b20 con 0x7fb7f415ccc0 2023-10-16T20:05:20.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.102+0000 7fb7f0ff9640 1 --2- 172.21.15.88:0/2038259066 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb7e00664a0 0x7fb7e0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.102+0000 7fb7f0ff9640 1 -- 172.21.15.88:0/2038259066 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb7ec083480 con 0x7fb7f415ccc0 2023-10-16T20:05:20.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.106+0000 7fb7f3fff640 1 --2- 172.21.15.88:0/2038259066 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb7e00664a0 0x7fb7e0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.106+0000 7fb7f3fff640 1 --2- 172.21.15.88:0/2038259066 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb7e00664a0 0x7fb7e0068960 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb7f41f8ba0 tx=0x7fb7e8009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:20.108 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.106+0000 7fb7f0ff9640 1 -- 172.21.15.88:0/2038259066 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb7ec0506b0 con 0x7fb7f415ccc0 2023-10-16T20:05:20.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.142+0000 7f0d57fb8640 1 Processor -- start 2023-10-16T20:05:20.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.142+0000 7f9d59267640 1 Processor -- start 2023-10-16T20:05:20.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.142+0000 7f9d59267640 1 -- start start 2023-10-16T20:05:20.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.142+0000 7f9d59267640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f9d540c8c40 0x7f9d540c9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.142+0000 7f9d59267640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f9d54070560 0x7f9d540c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.142+0000 7f9d59267640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d5415d4e0 con 0x7f9d54158810 2023-10-16T20:05:20.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.142+0000 7f9d59267640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d540ce190 con 0x7f9d54070560 2023-10-16T20:05:20.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.142+0000 7f9d59267640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d5415ccc0 con 0x7f9d540c8c40 2023-10-16T20:05:20.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.142+0000 7f9d53fff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f9d54158810 0x7f9d5415c5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:48778/0 (socket says 172.21.15.88:48778) 2023-10-16T20:05:20.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.142+0000 7f9d53fff640 1 -- 172.21.15.88:0/1639035677 learned_addr learned my addr 172.21.15.88:0/1639035677 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:20.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.142+0000 7f9d58a66640 1 --2- 172.21.15.88:0/1639035677 >> v2:172.21.15.88:3300/0 conn(0x7f9d54070560 0x7f9d540c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.142+0000 7f0d57fb8640 1 -- start start 2023-10-16T20:05:20.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.142+0000 7f0d57fb8640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d5015cb70 0x7f0d50161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.142+0000 7f0d57fb8640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f0d500cdad0 0x7f0d500cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.142+0000 7f0d57fb8640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d500b2190 con 0x7f0d500cff40 2023-10-16T20:05:20.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.142+0000 7f0d57fb8640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d500d03d0 con 0x7f0d500cdad0 2023-10-16T20:05:20.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.142+0000 7f0d57fb8640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d501615a0 con 0x7f0d5015cb70 2023-10-16T20:05:20.149 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.142+0000 7fd706cd6640 1 Processor -- start 2023-10-16T20:05:20.149 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.146+0000 7f0d55d54640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d5015cb70 0x7f0d50161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.149 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.146+0000 7f0d56555640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f0d500cff40 0x7f0d500b7c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:39248/0 (socket says 172.21.15.88:39248) 2023-10-16T20:05:20.150 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.146+0000 7f0d56555640 1 -- 172.21.15.88:0/4018357892 learned_addr learned my addr 172.21.15.88:0/4018357892 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:20.150 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.146+0000 7f0d55553640 1 --2- 172.21.15.88:0/4018357892 >> v2:172.21.15.88:3300/0 conn(0x7f0d500cdad0 0x7f0d500cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.150 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.142+0000 7f9d52ffd640 1 -- 172.21.15.88:0/1639035677 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3231887927 0 0) 0x7f9d5415d4e0 con 0x7f9d54158810 2023-10-16T20:05:20.150 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.142+0000 7f9d52ffd640 1 -- 172.21.15.88:0/1639035677 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d3c003530 con 0x7f9d54158810 2023-10-16T20:05:20.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.142+0000 7f9d537fe640 1 --2- 172.21.15.88:0/1639035677 >> v2:172.21.15.94:3300/0 conn(0x7f9d540c8c40 0x7f9d540c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.146+0000 7f9d58a66640 1 -- 172.21.15.88:0/1639035677 >> v2:172.21.15.94:3300/0 conn(0x7f9d540c8c40 msgr2=0x7f9d540c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:20.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.146+0000 7f9d58a66640 1 --2- 172.21.15.88:0/1639035677 >> v2:172.21.15.94:3300/0 conn(0x7f9d540c8c40 0x7f9d540c9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.146+0000 7f9d58a66640 1 -- 172.21.15.88:0/1639035677 >> v1:172.21.15.94:6789/0 conn(0x7f9d54158810 legacy=0x7f9d5415c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.146+0000 7f9d58a66640 1 -- 172.21.15.88:0/1639035677 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d5415cf20 con 0x7f9d54070560 2023-10-16T20:05:20.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.142+0000 7fd706cd6640 1 -- start start 2023-10-16T20:05:20.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.146+0000 7fd706cd6640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd700158ba0 0x7fd70006da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.146+0000 7fd706cd6640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd70006df80 0x7fd7000701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.146+0000 7fd706cd6640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd7000706e0 0x7fd70015c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.146+0000 7f0d55553640 1 -- 172.21.15.88:0/4018357892 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d5015cb70 msgr2=0x7f0d50161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:20.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.146+0000 7f0d55553640 1 --2- 172.21.15.88:0/4018357892 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d5015cb70 0x7f0d50161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.146+0000 7f0d55553640 1 -- 172.21.15.88:0/4018357892 >> v1:172.21.15.88:6789/0 conn(0x7f0d500cff40 legacy=0x7f0d500b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.146+0000 7f9d537fe640 1 --2- 172.21.15.88:0/1639035677 >> v2:172.21.15.94:3300/0 conn(0x7f9d540c8c40 0x7f9d540c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:20.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.146+0000 7fd704a72640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd700158ba0 0x7fd70006da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.155 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.146+0000 7f0d55553640 1 -- 172.21.15.88:0/4018357892 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d5015df60 con 0x7f0d500cdad0 2023-10-16T20:05:20.156 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.146+0000 7fd706cd6640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7000ae100 con 0x7fd700158ba0 2023-10-16T20:05:20.156 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.146+0000 7fd706cd6640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd70006e3e0 con 0x7fd7000706e0 2023-10-16T20:05:20.156 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.146+0000 7fd706cd6640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7000b1d80 con 0x7fd70006df80 2023-10-16T20:05:20.156 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.146+0000 7fd704a72640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd700158ba0 0x7fd70006da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40554/0 (socket says 172.21.15.88:40554) 2023-10-16T20:05:20.156 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.146+0000 7fd704a72640 1 -- 172.21.15.88:0/1060827378 learned_addr learned my addr 172.21.15.88:0/1060827378 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:20.157 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.146+0000 7fd705273640 1 --2- 172.21.15.88:0/1060827378 >> v2:172.21.15.94:3300/0 conn(0x7fd7000706e0 0x7fd70015c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.157 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.146+0000 7fd6fffff640 1 --2- 172.21.15.88:0/1060827378 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd70006df80 0x7fd7000701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.157 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.150+0000 7f2bdf8c9640 1 Processor -- start 2023-10-16T20:05:20.158 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.150+0000 7fd704a72640 1 -- 172.21.15.88:0/1060827378 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd70006df80 msgr2=0x7fd7000701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:20.158 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.150+0000 7fd704a72640 1 --2- 172.21.15.88:0/1060827378 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd70006df80 0x7fd7000701a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.158 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.150+0000 7fd704a72640 1 -- 172.21.15.88:0/1060827378 >> v2:172.21.15.94:3300/0 conn(0x7fd7000706e0 msgr2=0x7fd70015c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:20.159 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.150+0000 7fd704a72640 1 --2- 172.21.15.88:0/1060827378 >> v2:172.21.15.94:3300/0 conn(0x7fd7000706e0 0x7fd70015c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.159 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.150+0000 7fd704a72640 1 -- 172.21.15.88:0/1060827378 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7000711f0 con 0x7fd700158ba0 2023-10-16T20:05:20.159 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.150+0000 7fd705273640 1 --2- 172.21.15.88:0/1060827378 >> v2:172.21.15.94:3300/0 conn(0x7fd7000706e0 0x7fd70015c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:05:20.159 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.150+0000 7f2bdf8c9640 1 -- start start 2023-10-16T20:05:20.160 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.150+0000 7f9d58a66640 1 --2- 172.21.15.88:0/1639035677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d54070560 0x7f9d540c9a90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9d5406d190 tx=0x7f9d4c01c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf8515ed16b402dc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:20.160 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.150+0000 7fc1ed565640 1 Processor -- start 2023-10-16T20:05:20.160 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.150+0000 7f2bdf8c9640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2bd80d0110 0x7f2bd8161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.161 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.150+0000 7f2bdf8c9640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f2bd80cdad0 0x7f2bd80b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.161 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.150+0000 7f2bdf8c9640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd80b2190 con 0x7f2bd815cb70 2023-10-16T20:05:20.161 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.150+0000 7f2bdf8c9640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bd81615a0 con 0x7f2bd80cdad0 2023-10-16T20:05:20.161 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.150+0000 7f2bdf8c9640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bd815df60 con 0x7f2bd80d0110 2023-10-16T20:05:20.162 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.150+0000 7fc1ed565640 1 -- start start 2023-10-16T20:05:20.162 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.150+0000 7f9d52ffd640 1 -- 172.21.15.88:0/1639035677 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c0a5870 con 0x7f9d54070560 2023-10-16T20:05:20.163 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.150+0000 7f0d55553640 1 --2- 172.21.15.88:0/4018357892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d500cdad0 0x7f0d500cfa00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0d48009d90 tx=0x7f0d4801c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c54225b1e05102 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:20.163 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.150+0000 7f9d52ffd640 1 -- 172.21.15.88:0/1639035677 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4c0a59d0 con 0x7f9d54070560 2023-10-16T20:05:20.163 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.150+0000 7f9d52ffd640 1 -- 172.21.15.88:0/1639035677 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c0a5d60 con 0x7f9d54070560 2023-10-16T20:05:20.164 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.150+0000 7f2bdd665640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f2bd815cb70 0x7f2bd80cfa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:48790/0 (socket says 172.21.15.88:48790) 2023-10-16T20:05:20.164 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.150+0000 7f2bdd665640 1 -- 172.21.15.88:0/456769091 learned_addr learned my addr 172.21.15.88:0/456769091 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:20.164 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.150+0000 7f2bdce64640 1 --2- 172.21.15.88:0/456769091 >> v2:172.21.15.94:3300/0 conn(0x7f2bd80d0110 0x7f2bd8161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.165 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.150+0000 7f0d54d52640 1 -- 172.21.15.88:0/4018357892 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d480a5900 con 0x7f0d500cdad0 2023-10-16T20:05:20.165 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.150+0000 7f0d54d52640 1 -- 172.21.15.88:0/4018357892 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d480a5a60 con 0x7f0d500cdad0 2023-10-16T20:05:20.165 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.150+0000 7f0d54d52640 1 -- 172.21.15.88:0/4018357892 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d480a5df0 con 0x7f0d500cdad0 2023-10-16T20:05:20.166 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f2bcffff640 1 -- 172.21.15.88:0/456769091 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1537839959 0 0) 0x7f2bd80b2190 con 0x7f2bd815cb70 2023-10-16T20:05:20.166 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f2bcffff640 1 -- 172.21.15.88:0/456769091 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bc4003530 con 0x7f2bd815cb70 2023-10-16T20:05:20.166 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7fc1ed565640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc1e8158000 0x7fc1e8161ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.167 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7fc1ed565640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1e80b4dc0 0x7fc1e80cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.167 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7fc1ed565640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fc1e80cfef0 0x7fc1e80d1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.167 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7fc1ed565640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1e81614e0 con 0x7fc1e8158000 2023-10-16T20:05:20.167 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7fc1ed565640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1e80d0360 con 0x7fc1e80cfef0 2023-10-16T20:05:20.168 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7fc1ed565640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e8162430 con 0x7fc1e80b4dc0 2023-10-16T20:05:20.168 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f9d59267640 1 -- 172.21.15.88:0/1639035677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d54070560 msgr2=0x7f9d540c9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.169 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7fd704a72640 1 --2- 172.21.15.88:0/1060827378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd700158ba0 0x7fd70006da40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd7000b1b50 tx=0x7fd6f00209f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a48e78947ba2eb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:20.169 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f9d59267640 1 --2- 172.21.15.88:0/1639035677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d54070560 0x7f9d540c9a90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9d5406d190 tx=0x7f9d4c01c8c0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.169 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f2bcffff640 1 -- 172.21.15.88:0/456769091 <== mon.1 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3313657608 0 0) 0x7f2bc4003530 con 0x7f2bd815cb70 2023-10-16T20:05:20.170 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f2bcffff640 1 -- 172.21.15.88:0/456769091 >> v2:172.21.15.94:3300/0 conn(0x7f2bd80d0110 msgr2=0x7f2bd8161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:20.170 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f2bcffff640 1 --2- 172.21.15.88:0/456769091 >> v2:172.21.15.94:3300/0 conn(0x7f2bd80d0110 0x7f2bd8161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.170 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f2bcffff640 1 -- 172.21.15.88:0/456769091 >> v2:172.21.15.88:3300/0 conn(0x7f2bd80cdad0 msgr2=0x7f2bd80b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:20.170 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f2bcffff640 1 --2- 172.21.15.88:0/456769091 >> v2:172.21.15.88:3300/0 conn(0x7f2bd80cdad0 0x7f2bd80b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.170 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f2bcffff640 1 -- 172.21.15.88:0/456769091 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd815e230 con 0x7f2bd815cb70 2023-10-16T20:05:20.171 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7fc1e77fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fc1e80cfef0 0x7fc1e80d1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.171 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7fc1e77fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fc1e80cfef0 0x7fc1e80d1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:56938/0 (socket says 172.21.15.88:56938) 2023-10-16T20:05:20.171 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7fc1e77fe640 1 -- 172.21.15.88:0/1106414653 learned_addr learned my addr 172.21.15.88:0/1106414653 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:20.172 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f9d59267640 1 -- 172.21.15.88:0/1639035677 shutdown_connections 2023-10-16T20:05:20.172 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f9d59267640 1 --2- 172.21.15.88:0/1639035677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d54070560 0x7f9d540c9a90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.172 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7fc1e6ffd640 1 --2- 172.21.15.88:0/1106414653 >> v2:172.21.15.88:3300/0 conn(0x7fc1e8158000 0x7fc1e8161ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.173 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f9d59267640 1 --2- 172.21.15.88:0/1639035677 >> v2:172.21.15.94:3300/0 conn(0x7f9d540c8c40 0x7f9d540c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.173 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f9d59267640 1 -- 172.21.15.88:0/1639035677 >> 172.21.15.88:0/1639035677 conn(0x7f9d5406a4c0 msgr2=0x7f9d5406c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:20.174 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f2bcffff640 1 -- 172.21.15.88:0/456769091 <== mon.1 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f2bc8001660 con 0x7f2bd815cb70 2023-10-16T20:05:20.174 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f2bcffff640 1 -- 172.21.15.88:0/456769091 >> v1:172.21.15.94:6789/0 conn(0x7f2bd815cb70 legacy=0x7f2bd80cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.174 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f2bcffff640 1 --2- 172.21.15.88:0/456769091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc40039d0 0x7f2bc4003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.174 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f2bcffff640 1 --2- 172.21.15.88:0/456769091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bc40047d0 0x7f2bc4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.174 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f2bcffff640 1 --2- 172.21.15.88:0/456769091 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bc40051e0 0x7f2bc400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.175 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f2bcffff640 1 -- 172.21.15.88:0/456769091 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd81615a0 con 0x7f2bc40039d0 2023-10-16T20:05:20.175 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f2bcffff640 1 -- 172.21.15.88:0/456769091 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd815df60 con 0x7f2bc40047d0 2023-10-16T20:05:20.175 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f2bcffff640 1 -- 172.21.15.88:0/456769091 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc4014a50 con 0x7f2bc40051e0 2023-10-16T20:05:20.175 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7fc1e67fc640 1 --2- 172.21.15.88:0/1106414653 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1e80b4dc0 0x7fc1e80cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.176 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7fc1e77fe640 1 -- 172.21.15.88:0/1106414653 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1e80b4dc0 msgr2=0x7fc1e80cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:20.176 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7fc1e77fe640 1 --2- 172.21.15.88:0/1106414653 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1e80b4dc0 0x7fc1e80cf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.176 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7fc1e77fe640 1 -- 172.21.15.88:0/1106414653 >> v2:172.21.15.88:3300/0 conn(0x7fc1e8158000 msgr2=0x7fc1e8161ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:20.176 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7fc1e77fe640 1 --2- 172.21.15.88:0/1106414653 >> v2:172.21.15.88:3300/0 conn(0x7fc1e8158000 0x7fc1e8161ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.176 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7fc1e77fe640 1 -- 172.21.15.88:0/1106414653 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e80cd0e0 con 0x7fc1e80cfef0 2023-10-16T20:05:20.177 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7fc1e6ffd640 1 --2- 172.21.15.88:0/1106414653 >> v2:172.21.15.88:3300/0 conn(0x7fc1e8158000 0x7fc1e8161ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:20.177 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7fd6ff7fe640 1 -- 172.21.15.88:0/1060827378 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f00a9a80 con 0x7fd700158ba0 2023-10-16T20:05:20.177 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7fd6ff7fe640 1 -- 172.21.15.88:0/1060827378 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f00a9be0 con 0x7fd700158ba0 2023-10-16T20:05:20.178 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f2bdce64640 1 --2- 172.21.15.88:0/456769091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bc40047d0 0x7f2bc4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.178 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f2bdd665640 1 --2- 172.21.15.88:0/456769091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc40039d0 0x7f2bc4003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.178 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7fd6ff7fe640 1 -- 172.21.15.88:0/1060827378 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f00a9a80 con 0x7fd700158ba0 2023-10-16T20:05:20.179 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f0d57fb8640 1 -- 172.21.15.88:0/4018357892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d500cdad0 msgr2=0x7f0d500cfa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.179 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f0d57fb8640 1 --2- 172.21.15.88:0/4018357892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d500cdad0 0x7f0d500cfa00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0d48009d90 tx=0x7f0d4801c990 comp rx=0 tx=0).stop 2023-10-16T20:05:20.179 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f9d59267640 1 -- 172.21.15.88:0/1639035677 shutdown_connections 2023-10-16T20:05:20.180 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7fd706cd6640 1 -- 172.21.15.88:0/1060827378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd700158ba0 msgr2=0x7fd70006da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.180 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7fd706cd6640 1 --2- 172.21.15.88:0/1060827378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd700158ba0 0x7fd70006da40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd7000b1b50 tx=0x7fd6f00209f0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.180 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f9d59267640 1 -- 172.21.15.88:0/1639035677 wait complete. 2023-10-16T20:05:20.181 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f2bdde66640 1 --2- 172.21.15.88:0/456769091 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bc40051e0 0x7f2bc400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.181 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f2bdce64640 1 -- 172.21.15.88:0/456769091 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bc40051e0 msgr2=0x7f2bc400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:20.181 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f2bdce64640 1 --2- 172.21.15.88:0/456769091 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bc40051e0 0x7f2bc400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.182 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f2bdce64640 1 -- 172.21.15.88:0/456769091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc40039d0 msgr2=0x7f2bc4003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:20.182 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f2bdce64640 1 --2- 172.21.15.88:0/456769091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc40039d0 0x7f2bc4003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.182 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f2bdce64640 1 -- 172.21.15.88:0/456769091 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc4003530 con 0x7f2bc40047d0 2023-10-16T20:05:20.182 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7f2bdde66640 1 -- 172.21.15.88:0/456769091 reap_dead start 2023-10-16T20:05:20.183 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7fd706cd6640 1 -- 172.21.15.88:0/1060827378 shutdown_connections 2023-10-16T20:05:20.183 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7fd706cd6640 1 --2- 172.21.15.88:0/1060827378 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd70006df80 0x7fd7000701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.183 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7fd706cd6640 1 --2- 172.21.15.88:0/1060827378 >> v2:172.21.15.94:3300/0 conn(0x7fd7000706e0 0x7fd70015c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.184 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7fd706cd6640 1 --2- 172.21.15.88:0/1060827378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd700158ba0 0x7fd70006da40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.184 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.154+0000 7fd706cd6640 1 -- 172.21.15.88:0/1060827378 >> 172.21.15.88:0/1060827378 conn(0x7fd7000cb060 msgr2=0x7fd7000cb470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:20.184 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.158+0000 7fd706cd6640 1 -- 172.21.15.88:0/1060827378 shutdown_connections 2023-10-16T20:05:20.184 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.158+0000 7fd706cd6640 1 -- 172.21.15.88:0/1060827378 wait complete. 2023-10-16T20:05:20.185 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.158+0000 7f21f9c79640 1 Processor -- start 2023-10-16T20:05:20.186 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.158+0000 7f0d57fb8640 1 -- 172.21.15.88:0/4018357892 shutdown_connections 2023-10-16T20:05:20.186 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.158+0000 7f0d57fb8640 1 --2- 172.21.15.88:0/4018357892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d500cdad0 0x7f0d500cfa00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.186 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.158+0000 7f0d57fb8640 1 --2- 172.21.15.88:0/4018357892 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d5015cb70 0x7f0d50161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.187 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.158+0000 7f0d57fb8640 1 -- 172.21.15.88:0/4018357892 >> 172.21.15.88:0/4018357892 conn(0x7f0d500719c0 msgr2=0x7f0d50071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:20.187 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.158+0000 7f9d59267640 1 Processor -- start 2023-10-16T20:05:20.188 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.158+0000 7f21f9c79640 1 -- start start 2023-10-16T20:05:20.189 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.158+0000 7f9d59267640 1 -- start start 2023-10-16T20:05:20.189 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.158+0000 7fd706cd6640 1 Processor -- start 2023-10-16T20:05:20.190 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.158+0000 7f0d57fb8640 1 -- 172.21.15.88:0/4018357892 shutdown_connections 2023-10-16T20:05:20.190 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.158+0000 7f0d57fb8640 1 -- 172.21.15.88:0/4018357892 wait complete. 2023-10-16T20:05:20.191 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.158+0000 7fd706cd6640 1 -- start start 2023-10-16T20:05:20.192 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f2bdce64640 1 --2- 172.21.15.88:0/456769091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bc40047d0 0x7f2bc4004c50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0007360 tx=0x7f2bd001ce20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b77778b52b5b11f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:20.192 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7fd706cd6640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd70006df80 0x7fd7001fa020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.193 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7fd706cd6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7000706e0 0x7fd70015b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.194 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f9d59267640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d54070560 0x7f9d54153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.194 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f9d59267640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d540c8c40 0x7f9d54154830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.194 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f9d59267640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d54158810 0x7f9d54154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.195 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f9d59267640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d541f6610 con 0x7f9d540c8c40 2023-10-16T20:05:20.195 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f9d59267640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d541f6330 con 0x7f9d54158810 2023-10-16T20:05:20.195 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f2bcffff640 1 -- 172.21.15.88:0/456769091 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd00a5b00 con 0x7f2bc40047d0 2023-10-16T20:05:20.196 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f2bcffff640 1 -- 172.21.15.88:0/456769091 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd00a5c60 con 0x7f2bc40047d0 2023-10-16T20:05:20.196 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f2bcffff640 1 -- 172.21.15.88:0/456769091 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd00ad440 con 0x7f2bc40047d0 2023-10-16T20:05:20.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7fd706cd6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd700158ba0 0x7fd700157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7fd706cd6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7001fa560 con 0x7fd700158ba0 2023-10-16T20:05:20.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7fd706cd6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7001fd050 con 0x7fd7000706e0 2023-10-16T20:05:20.198 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7fd706cd6640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7001fd330 con 0x7fd70006df80 2023-10-16T20:05:20.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.158+0000 7f21f9c79640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f21f413fc50 0x7f21f4142d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.158+0000 7f21f9c79640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f21f4160790 0x7f21f41402b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f21f9c79640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21f40ace60 con 0x7f21f4069e90 2023-10-16T20:05:20.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f21f9c79640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21f4140030 con 0x7f21f413fc50 2023-10-16T20:05:20.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f21f9c79640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21f4143250 con 0x7f21f4160790 2023-10-16T20:05:20.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f21f3fff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f21f4069e90 0x7f21f415d6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:48796/0 (socket says 172.21.15.88:48796) 2023-10-16T20:05:20.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f21f3fff640 1 -- 172.21.15.88:0/1332323207 learned_addr learned my addr 172.21.15.88:0/1332323207 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:20.202 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7fd6fffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7000706e0 0x7fd70015b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.202 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7fd6fffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7000706e0 0x7fd70015b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:56964/0 (socket says 172.21.15.88:56964) 2023-10-16T20:05:20.202 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7fd6fffff640 1 -- 172.21.15.88:0/3082377264 learned_addr learned my addr 172.21.15.88:0/3082377264 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:20.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7fd704a72640 1 --2- 172.21.15.88:0/3082377264 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd70006df80 0x7fd7001fa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7fc1e77fe640 1 --2- 172.21.15.88:0/1106414653 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1e80cfef0 0x7fc1e80d1ff0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc1e80d1800 tx=0x7fc1e001c9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b45db20e155013b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:20.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f9d59267640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d54159130 con 0x7f9d54070560 2023-10-16T20:05:20.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f9d537fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d540c8c40 0x7f9d54154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f9d537fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d540c8c40 0x7f9d54154830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40590/0 (socket says 172.21.15.88:40590) 2023-10-16T20:05:20.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f9d537fe640 1 -- 172.21.15.88:0/1115622399 learned_addr learned my addr 172.21.15.88:0/1115622399 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:20.206 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f9d58a66640 1 --2- 172.21.15.88:0/1115622399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d54158810 0x7f9d54154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.206 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f9d53fff640 1 --2- 172.21.15.88:0/1115622399 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d54070560 0x7f9d54153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f21f37fe640 1 --2- 172.21.15.88:0/1332323207 >> v2:172.21.15.88:3300/0 conn(0x7f21f413fc50 0x7f21f4142d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7fd705273640 1 --2- 172.21.15.88:0/3082377264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd700158ba0 0x7fd700157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f21f2ffd640 1 --2- 172.21.15.88:0/1332323207 >> v2:172.21.15.94:3300/0 conn(0x7f21f4160790 0x7f21f41402b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7fd6fffff640 1 -- 172.21.15.88:0/3082377264 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd70006df80 msgr2=0x7fd7001fa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7fd6fffff640 1 --2- 172.21.15.88:0/3082377264 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd70006df80 0x7fd7001fa020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7fd6fffff640 1 -- 172.21.15.88:0/3082377264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd700158ba0 msgr2=0x7fd700157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7fd6fffff640 1 --2- 172.21.15.88:0/3082377264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd700158ba0 0x7fd700157090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7fd6fffff640 1 -- 172.21.15.88:0/3082377264 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f8009a70 con 0x7fd7000706e0 2023-10-16T20:05:20.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f2bdf8c9640 1 -- 172.21.15.88:0/456769091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bc40047d0 msgr2=0x7f2bc4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f2bdf8c9640 1 --2- 172.21.15.88:0/456769091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bc40047d0 0x7f2bc4004c50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0007360 tx=0x7f2bd001ce20 comp rx=0 tx=0).stop 2023-10-16T20:05:20.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7fc1e5ffb640 1 -- 172.21.15.88:0/1106414653 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e00a57d0 con 0x7fc1e80cfef0 2023-10-16T20:05:20.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7fc1e5ffb640 1 -- 172.21.15.88:0/1106414653 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e00a5930 con 0x7fc1e80cfef0 2023-10-16T20:05:20.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7fc1e5ffb640 1 -- 172.21.15.88:0/1106414653 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e00a5cc0 con 0x7fc1e80cfef0 2023-10-16T20:05:20.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f9d58a66640 1 -- 172.21.15.88:0/1115622399 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d54070560 msgr2=0x7f9d54153a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f9d58a66640 1 --2- 172.21.15.88:0/1115622399 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d54070560 0x7f9d54153a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f9d58a66640 1 -- 172.21.15.88:0/1115622399 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d540c8c40 msgr2=0x7f9d54154830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f9d58a66640 1 --2- 172.21.15.88:0/1115622399 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d540c8c40 0x7f9d54154830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f9d58a66640 1 -- 172.21.15.88:0/1115622399 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4c009a70 con 0x7f9d54158810 2023-10-16T20:05:20.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7fd6fffff640 1 --2- 172.21.15.88:0/3082377264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7000706e0 0x7fd70015b960 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd6f400e8b0 tx=0x7fd6f400ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:20.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f21f37fe640 1 -- 172.21.15.88:0/1332323207 >> v2:172.21.15.94:3300/0 conn(0x7f21f4160790 msgr2=0x7f21f41402b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:20.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7fd6fd7fa640 1 -- 172.21.15.88:0/3082377264 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f400c940 con 0x7fd7000706e0 2023-10-16T20:05:20.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f0d57fb8640 1 Processor -- start 2023-10-16T20:05:20.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f9d58a66640 1 --2- 172.21.15.88:0/1115622399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d54158810 0x7f9d54154d70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0ac000 tx=0x7f9d4c019660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:20.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f2bdf8c9640 1 -- 172.21.15.88:0/456769091 shutdown_connections 2023-10-16T20:05:20.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f2bdf8c9640 1 --2- 172.21.15.88:0/456769091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bc40047d0 0x7f2bc4004c50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.217 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f2bdf8c9640 1 -- 172.21.15.88:0/456769091 >> 172.21.15.88:0/456769091 conn(0x7f2bd80719c0 msgr2=0x7f2bd8071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:20.217 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7fd6fd7fa640 1 -- 172.21.15.88:0/3082377264 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f400caa0 con 0x7fd7000706e0 2023-10-16T20:05:20.218 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7fd6fd7fa640 1 -- 172.21.15.88:0/3082377264 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f400c940 con 0x7fd7000706e0 2023-10-16T20:05:20.218 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7fd706cd6640 1 -- 172.21.15.88:0/3082377264 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f0010810 con 0x7fd7000706e0 2023-10-16T20:05:20.218 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f21f37fe640 1 --2- 172.21.15.88:0/1332323207 >> v2:172.21.15.94:3300/0 conn(0x7f21f4160790 0x7f21f41402b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f21f37fe640 1 -- 172.21.15.88:0/1332323207 >> v1:172.21.15.94:6789/0 conn(0x7f21f4069e90 legacy=0x7f21f415d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f21f37fe640 1 -- 172.21.15.88:0/1332323207 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f415d1a0 con 0x7f21f413fc50 2023-10-16T20:05:20.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7fc1ed565640 1 -- 172.21.15.88:0/1106414653 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1e80cfef0 msgr2=0x7fc1e80d1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7fc1ed565640 1 --2- 172.21.15.88:0/1106414653 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1e80cfef0 0x7fc1e80d1ff0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc1e80d1800 tx=0x7fc1e001c9d0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f9d50ff9640 1 -- 172.21.15.88:0/1115622399 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c0ad640 con 0x7f9d54158810 2023-10-16T20:05:20.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f0d57fb8640 1 -- start start 2023-10-16T20:05:20.222 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7f9d50ff9640 1 -- 172.21.15.88:0/1115622399 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4c0ad7a0 con 0x7f9d54158810 2023-10-16T20:05:20.222 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.162+0000 7fd706cd6640 1 -- 172.21.15.88:0/3082377264 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7001fd8f0 con 0x7fd7000706e0 2023-10-16T20:05:20.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7f9d59267640 1 -- 172.21.15.88:0/1115622399 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d54159410 con 0x7f9d54158810 2023-10-16T20:05:20.224 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7fc1ed565640 1 -- 172.21.15.88:0/1106414653 shutdown_connections 2023-10-16T20:05:20.224 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7fc1ed565640 1 --2- 172.21.15.88:0/1106414653 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1e80b4dc0 0x7fc1e80cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.225 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7fc1ed565640 1 --2- 172.21.15.88:0/1106414653 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1e80cfef0 0x7fc1e80d1ff0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.225 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7fc1ed565640 1 --2- 172.21.15.88:0/1106414653 >> v2:172.21.15.88:3300/0 conn(0x7fc1e8158000 0x7fc1e8161ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.225 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7fc1ed565640 1 -- 172.21.15.88:0/1106414653 >> 172.21.15.88:0/1106414653 conn(0x7fc1e80719c0 msgr2=0x7fc1e8071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:20.226 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7f9d50ff9640 1 -- 172.21.15.88:0/1115622399 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c0ad640 con 0x7f9d54158810 2023-10-16T20:05:20.226 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7f9d59267640 1 -- 172.21.15.88:0/1115622399 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d541f67e0 con 0x7f9d54158810 2023-10-16T20:05:20.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7fd706cd6640 1 -- 172.21.15.88:0/3082377264 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd70004e9c0 con 0x7fd7000706e0 2023-10-16T20:05:20.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7f0d57fb8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d500cdad0 0x7f0d5014e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.228 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7f9d59267640 1 -- 172.21.15.88:0/1115622399 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d540674f0 con 0x7f9d54158810 2023-10-16T20:05:20.228 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7f0d57fb8640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d500cff40 0x7f0d5014ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.229 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7f0d57fb8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d5015cb70 0x7f0d50155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.229 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7f0d57fb8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d501579f0 con 0x7f0d5015cb70 2023-10-16T20:05:20.229 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7f0d57fb8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d50156de0 con 0x7f0d500cdad0 2023-10-16T20:05:20.229 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7f0d57fb8640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d501570c0 con 0x7f0d500cff40 2023-10-16T20:05:20.229 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7f0d55d54640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d500cdad0 0x7f0d5014e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.230 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7f0d55d54640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d500cdad0 0x7f0d5014e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:56988/0 (socket says 172.21.15.88:56988) 2023-10-16T20:05:20.230 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7f0d55d54640 1 -- 172.21.15.88:0/3285547011 learned_addr learned my addr 172.21.15.88:0/3285547011 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:20.230 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7f0d55553640 1 --2- 172.21.15.88:0/3285547011 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d500cff40 0x7f0d5014ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.230 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7f0d56555640 1 --2- 172.21.15.88:0/3285547011 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d5015cb70 0x7f0d50155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7f0d55553640 1 -- 172.21.15.88:0/3285547011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d500cdad0 msgr2=0x7f0d5014e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7f0d55553640 1 --2- 172.21.15.88:0/3285547011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d500cdad0 0x7f0d5014e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7f0d55553640 1 -- 172.21.15.88:0/3285547011 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d5015cb70 msgr2=0x7f0d50155800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7f0d55553640 1 --2- 172.21.15.88:0/3285547011 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d5015cb70 0x7f0d50155800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7f0d55553640 1 -- 172.21.15.88:0/3285547011 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d48009a70 con 0x7f0d500cff40 2023-10-16T20:05:20.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7f0d56555640 1 --2- 172.21.15.88:0/3285547011 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d5015cb70 0x7f0d50155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:20.232 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7fd6fd7fa640 1 -- 172.21.15.88:0/3082377264 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd6f400cc10 con 0x7fd7000706e0 2023-10-16T20:05:20.232 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7f0d55553640 1 --2- 172.21.15.88:0/3285547011 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d500cff40 0x7f0d5014ebe0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0d480ac000 tx=0x7f0d48019730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:20.233 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7f0d467fc640 1 -- 172.21.15.88:0/3285547011 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d480ad850 con 0x7f0d500cff40 2023-10-16T20:05:20.233 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7f0d467fc640 1 -- 172.21.15.88:0/3285547011 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d480ad9b0 con 0x7f0d500cff40 2023-10-16T20:05:20.233 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7f0d467fc640 1 -- 172.21.15.88:0/3285547011 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d480add40 con 0x7f0d500cff40 2023-10-16T20:05:20.233 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7f0d57fb8640 1 -- 172.21.15.88:0/3285547011 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d501573a0 con 0x7f0d500cff40 2023-10-16T20:05:20.234 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7fc1ed565640 1 -- 172.21.15.88:0/1106414653 shutdown_connections 2023-10-16T20:05:20.234 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7f0d57fb8640 1 -- 172.21.15.88:0/3285547011 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d5014aa10 con 0x7f0d500cff40 2023-10-16T20:05:20.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7f9d50ff9640 1 -- 172.21.15.88:0/1115622399 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9d4c0b4050 con 0x7f9d54158810 2023-10-16T20:05:20.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7fd6fd7fa640 1 --2- 172.21.15.88:0/3082377264 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd6ec066500 0x7fd6ec0689c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7f1f0abc5640 1 Processor -- start 2023-10-16T20:05:20.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7fd6fd7fa640 1 -- 172.21.15.88:0/3082377264 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd6f4084020 con 0x7fd7000706e0 2023-10-16T20:05:20.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.166+0000 7fc1ed565640 1 -- 172.21.15.88:0/1106414653 wait complete. 2023-10-16T20:05:20.237 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7fd704a72640 1 --2- 172.21.15.88:0/3082377264 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd6ec066500 0x7fd6ec0689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.237 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7f1f0abc5640 1 -- start start 2023-10-16T20:05:20.238 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7f2bdf8c9640 1 -- 172.21.15.88:0/456769091 shutdown_connections 2023-10-16T20:05:20.238 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7f9d50ff9640 1 --2- 172.21.15.88:0/1115622399 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9d40066500 0x7f9d400689c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.238 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7f9d50ff9640 1 -- 172.21.15.88:0/1115622399 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9d4c11ece0 con 0x7f9d54158810 2023-10-16T20:05:20.239 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7fd704a72640 1 --2- 172.21.15.88:0/3082377264 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd6ec066500 0x7fd6ec0689c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd6f00b3000 tx=0x7fd6f00ab070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:20.239 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7f2bdf8c9640 1 -- 172.21.15.88:0/456769091 wait complete. 2023-10-16T20:05:20.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7fc1ed565640 1 Processor -- start 2023-10-16T20:05:20.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7f9d53fff640 1 --2- 172.21.15.88:0/1115622399 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9d40066500 0x7f9d400689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7f1f0abc5640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1f0415aa60 0x7f1f040cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7f1f0abc5640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1f0415ae40 0x7f1f040cb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7f1f0abc5640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f040cba30 con 0x7f1f040af730 2023-10-16T20:05:20.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7f1f0abc5640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f040cbd10 con 0x7f1f0415ae40 2023-10-16T20:05:20.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7f1f0abc5640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f040ce9c0 con 0x7f1f0415aa60 2023-10-16T20:05:20.242 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7f1f08961640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1f0415aa60 0x7f1f040cf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.242 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7f1f08961640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1f0415aa60 0x7f1f040cf4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:33228/0 (socket says 172.21.15.88:33228) 2023-10-16T20:05:20.242 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7f1f08961640 1 -- 172.21.15.88:0/3959356483 learned_addr learned my addr 172.21.15.88:0/3959356483 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:20.242 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7f1efbfff640 1 --2- 172.21.15.88:0/3959356483 >> v2:172.21.15.94:3300/0 conn(0x7f1f0415ae40 0x7f1f040cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7f21f37fe640 1 --2- 172.21.15.88:0/1332323207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21f413fc50 0x7f21f4142d10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f21f415d390 tx=0x7f21e801c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62c923ef7bc51dc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:20.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7f0d57fb8640 1 -- 172.21.15.88:0/3285547011 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d5004e9c0 con 0x7f0d500cff40 2023-10-16T20:05:20.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7f0d467fc640 1 -- 172.21.15.88:0/3285547011 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0d480b4070 con 0x7f0d500cff40 2023-10-16T20:05:20.244 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7f0d467fc640 1 --2- 172.21.15.88:0/3285547011 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0d340664a0 0x7f0d34068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.244 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7f0d467fc640 1 -- 172.21.15.88:0/3285547011 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0d4811da40 con 0x7f0d500cff40 2023-10-16T20:05:20.244 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7f2bdf8c9640 1 Processor -- start 2023-10-16T20:05:20.245 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7fc1ed565640 1 -- start start 2023-10-16T20:05:20.245 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7fc1ed565640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1e80b4dc0 0x7fc1e8070b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.245 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7fc1ed565640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1e80cfef0 0x7fc1e8146ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.245 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7fc1ed565640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1e8158000 0x7fc1e806dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.246 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7fc1ed565640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e8144660 con 0x7fc1e80b4dc0 2023-10-16T20:05:20.246 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7fc1ed565640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e8071060 con 0x7fc1e80cfef0 2023-10-16T20:05:20.246 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7fc1ed565640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e8155390 con 0x7fc1e8158000 2023-10-16T20:05:20.246 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7fc1e67fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1e80cfef0 0x7fc1e8146ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.246 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7fc1e67fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1e80cfef0 0x7fc1e8146ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:57004/0 (socket says 172.21.15.88:57004) 2023-10-16T20:05:20.247 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.170+0000 7fc1e67fc640 1 -- 172.21.15.88:0/2685888329 learned_addr learned my addr 172.21.15.88:0/2685888329 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:20.247 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7fc1e6ffd640 1 --2- 172.21.15.88:0/2685888329 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1e80b4dc0 0x7fc1e8070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.247 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f21f27fc640 1 -- 172.21.15.88:0/1332323207 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e80ab600 con 0x7f21f413fc50 2023-10-16T20:05:20.247 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f21f27fc640 1 -- 172.21.15.88:0/1332323207 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e80a7050 con 0x7f21f413fc50 2023-10-16T20:05:20.248 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f21f27fc640 1 -- 172.21.15.88:0/1332323207 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e80ab980 con 0x7f21f413fc50 2023-10-16T20:05:20.248 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f1efb7fe640 1 -- 172.21.15.88:0/3959356483 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 239061948 0 0) 0x7f1f040cba30 con 0x7f1f040af730 2023-10-16T20:05:20.248 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f1f08961640 1 -- 172.21.15.88:0/3959356483 >> v2:172.21.15.94:3300/0 conn(0x7f1f0415ae40 msgr2=0x7f1f040cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:20.248 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f1f08961640 1 --2- 172.21.15.88:0/3959356483 >> v2:172.21.15.94:3300/0 conn(0x7f1f0415ae40 0x7f1f040cb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.249 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f1f08961640 1 -- 172.21.15.88:0/3959356483 >> v1:172.21.15.88:6789/0 conn(0x7f1f040af730 legacy=0x7f1f040b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.249 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f1f08961640 1 -- 172.21.15.88:0/3959356483 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f040ceca0 con 0x7f1f0415aa60 2023-10-16T20:05:20.249 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f9d53fff640 1 --2- 172.21.15.88:0/1115622399 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9d40066500 0x7f9d400689c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9d44000c10 tx=0x7f9d44007200 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:20.250 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7fd6fd7fa640 1 -- 172.21.15.88:0/3082377264 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd6f40b5090 con 0x7fd7000706e0 2023-10-16T20:05:20.250 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f1efbfff640 1 --2- 172.21.15.88:0/3959356483 >> v2:172.21.15.94:3300/0 conn(0x7f1f0415ae40 0x7f1f040cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:20.251 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7fc1e77fe640 1 --2- 172.21.15.88:0/2685888329 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1e8158000 0x7fc1e806dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.253 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7fc1e67fc640 1 -- 172.21.15.88:0/2685888329 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1e8158000 msgr2=0x7fc1e806dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.253 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7fc1e67fc640 1 --2- 172.21.15.88:0/2685888329 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1e8158000 0x7fc1e806dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.253 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7fc1e67fc640 1 -- 172.21.15.88:0/2685888329 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1e80b4dc0 msgr2=0x7fc1e8070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.253 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7fc1e67fc640 1 --2- 172.21.15.88:0/2685888329 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1e80b4dc0 0x7fc1e8070b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.254 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7fc1e67fc640 1 -- 172.21.15.88:0/2685888329 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e0009b30 con 0x7fc1e80cfef0 2023-10-16T20:05:20.254 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7fc1e6ffd640 1 --2- 172.21.15.88:0/2685888329 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1e80b4dc0 0x7fc1e8070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:05:20.254 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f0d55d54640 1 --2- 172.21.15.88:0/3285547011 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0d340664a0 0x7f0d34068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.255 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f2bdf8c9640 1 -- start start 2023-10-16T20:05:20.255 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f21f9c79640 1 -- 172.21.15.88:0/1332323207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21f413fc50 msgr2=0x7f21f4142d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.255 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f21f9c79640 1 --2- 172.21.15.88:0/1332323207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21f413fc50 0x7f21f4142d10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f21f415d390 tx=0x7f21e801c380 comp rx=0 tx=0).stop 2023-10-16T20:05:20.256 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7fc1e67fc640 1 --2- 172.21.15.88:0/2685888329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1e80cfef0 0x7fc1e8146ee0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc1d800d800 tx=0x7fc1d800dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:20.256 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f21f9c79640 1 -- 172.21.15.88:0/1332323207 shutdown_connections 2023-10-16T20:05:20.256 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f21f9c79640 1 --2- 172.21.15.88:0/1332323207 >> v2:172.21.15.94:3300/0 conn(0x7f21f4160790 0x7f21f41402b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f21f9c79640 1 --2- 172.21.15.88:0/1332323207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21f413fc50 0x7f21f4142d10 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f21f9c79640 1 -- 172.21.15.88:0/1332323207 >> 172.21.15.88:0/1332323207 conn(0x7f21f40ce5e0 msgr2=0x7f21f40ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:20.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f21f9c79640 1 -- 172.21.15.88:0/1332323207 shutdown_connections 2023-10-16T20:05:20.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f21f9c79640 1 -- 172.21.15.88:0/1332323207 wait complete. 2023-10-16T20:05:20.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f0d55d54640 1 --2- 172.21.15.88:0/3285547011 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0d340664a0 0x7f0d34068960 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0d40007a70 tx=0x7f0d40008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:20.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f2bdf8c9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc40047d0 0x7f2bd80cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f2bdf8c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bd815fbf0 0x7f2bd80c5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f2bdf8c9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bd814fde0 0x7f2bd80b6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f2bdf8c9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd80cbaa0 con 0x7f2bc40047d0 2023-10-16T20:05:20.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f2bdf8c9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd80caed0 con 0x7f2bd815fbf0 2023-10-16T20:05:20.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f2bdf8c9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd814fbe0 con 0x7f2bd814fde0 2023-10-16T20:05:20.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f2bdd665640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc40047d0 0x7f2bd80cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f2bdd665640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc40047d0 0x7f2bd80cbec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40614/0 (socket says 172.21.15.88:40614) 2023-10-16T20:05:20.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f2bdd665640 1 -- 172.21.15.88:0/3570356510 learned_addr learned my addr 172.21.15.88:0/3570356510 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:20.261 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f0d467fc640 1 -- 172.21.15.88:0/3285547011 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0d480eabc0 con 0x7f0d500cff40 2023-10-16T20:05:20.261 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f2bdde66640 1 --2- 172.21.15.88:0/3570356510 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bd814fde0 0x7f2bd80b6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.261 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7f2bdce64640 1 --2- 172.21.15.88:0/3570356510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bd815fbf0 0x7f2bd80c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7f2bdde66640 1 -- 172.21.15.88:0/3570356510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bd815fbf0 msgr2=0x7f2bd80c5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7f2bdde66640 1 --2- 172.21.15.88:0/3570356510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bd815fbf0 0x7f2bd80c5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7f2bdde66640 1 -- 172.21.15.88:0/3570356510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc40047d0 msgr2=0x7f2bd80cbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7f2bdde66640 1 --2- 172.21.15.88:0/3570356510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc40047d0 0x7f2bd80cbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7f2bdde66640 1 -- 172.21.15.88:0/3570356510 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd80b5a50 con 0x7f2bd814fde0 2023-10-16T20:05:20.263 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7f21f9c79640 1 Processor -- start 2023-10-16T20:05:20.263 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.174+0000 7f1f08961640 1 --2- 172.21.15.88:0/3959356483 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1f0415aa60 0x7f1f040cf4a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009870 tx=0x7f1ef401c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a836105e4a1812d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:20.263 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7f1efb7fe640 1 -- 172.21.15.88:0/3959356483 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef40ab640 con 0x7f1f0415aa60 2023-10-16T20:05:20.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7f1efb7fe640 1 -- 172.21.15.88:0/3959356483 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef40a7070 con 0x7f1f0415aa60 2023-10-16T20:05:20.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7f1efb7fe640 1 -- 172.21.15.88:0/3959356483 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef40ab640 con 0x7f1f0415aa60 2023-10-16T20:05:20.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7fc1df7fe640 1 -- 172.21.15.88:0/2685888329 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d80096a0 con 0x7fc1e80cfef0 2023-10-16T20:05:20.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7f2bdde66640 1 --2- 172.21.15.88:0/3570356510 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bd814fde0 0x7f2bd80b6c30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80b5680 tx=0x7f2bd400e9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:20.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7fc1df7fe640 1 -- 172.21.15.88:0/2685888329 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d8009800 con 0x7fc1e80cfef0 2023-10-16T20:05:20.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7fc1df7fe640 1 -- 172.21.15.88:0/2685888329 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d8009b70 con 0x7fc1e80cfef0 2023-10-16T20:05:20.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7fc1ed565640 1 -- 172.21.15.88:0/2685888329 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1e8155670 con 0x7fc1e80cfef0 2023-10-16T20:05:20.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7fc1ed565640 1 -- 172.21.15.88:0/2685888329 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e8155c30 con 0x7fc1e80cfef0 2023-10-16T20:05:20.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7f21f9c79640 1 -- start start 2023-10-16T20:05:20.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7f2bcdffb640 1 -- 172.21.15.88:0/3570356510 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4013680 con 0x7f2bd814fde0 2023-10-16T20:05:20.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7f2bcdffb640 1 -- 172.21.15.88:0/3570356510 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd40137e0 con 0x7f2bd814fde0 2023-10-16T20:05:20.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7f2bcdffb640 1 -- 172.21.15.88:0/3570356510 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4013b90 con 0x7f2bd814fde0 2023-10-16T20:05:20.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7f2bdf8c9640 1 -- 172.21.15.88:0/3570356510 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd80cc400 con 0x7f2bd814fde0 2023-10-16T20:05:20.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7f2bdf8c9640 1 -- 172.21.15.88:0/3570356510 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd80cb3e0 con 0x7f2bd814fde0 2023-10-16T20:05:20.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7f1f0abc5640 1 -- 172.21.15.88:0/3959356483 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1f0415aa60 msgr2=0x7f1f040cf4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7f1f0abc5640 1 --2- 172.21.15.88:0/3959356483 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1f0415aa60 0x7f1f040cf4a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009870 tx=0x7f1ef401c510 comp rx=0 tx=0).stop 2023-10-16T20:05:20.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7f1f0abc5640 1 -- 172.21.15.88:0/3959356483 shutdown_connections 2023-10-16T20:05:20.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7f1f0abc5640 1 --2- 172.21.15.88:0/3959356483 >> v2:172.21.15.94:3300/0 conn(0x7f1f0415ae40 0x7f1f040cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7f1f0abc5640 1 --2- 172.21.15.88:0/3959356483 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1f0415aa60 0x7f1f040cf4a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7f1f0abc5640 1 -- 172.21.15.88:0/3959356483 >> 172.21.15.88:0/3959356483 conn(0x7f1f040cd370 msgr2=0x7f1f040cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:20.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7f1f0abc5640 1 -- 172.21.15.88:0/3959356483 shutdown_connections 2023-10-16T20:05:20.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7f1f0abc5640 1 -- 172.21.15.88:0/3959356483 wait complete. 2023-10-16T20:05:20.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7f9d50ff9640 1 -- 172.21.15.88:0/1115622399 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9d4c0ebf30 con 0x7f9d54158810 2023-10-16T20:05:20.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7f21f9c79640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21f4069e90 0x7f21f413c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7f21f9c79640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21f413fc50 0x7f21f41401b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7f21f9c79640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f21f4160790 0x7f21f41f8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.178+0000 7f1f0abc5640 1 Processor -- start 2023-10-16T20:05:20.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.182+0000 7f1f0abc5640 1 -- start start 2023-10-16T20:05:20.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.182+0000 7f2bdf8c9640 1 -- 172.21.15.88:0/3570356510 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb8005260 con 0x7f2bd814fde0 2023-10-16T20:05:20.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.182+0000 7fc1ed565640 1 -- 172.21.15.88:0/2685888329 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1e804e9c0 con 0x7fc1e80cfef0 2023-10-16T20:05:20.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.182+0000 7f1f0abc5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f040af730 0x7f1f0406af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.182+0000 7f1f0abc5640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1f0415aa60 0x7f1f04138580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.273 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.182+0000 7f1f0abc5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f0415ae40 0x7f1f0415b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.273 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.182+0000 7f2bcdffb640 1 -- 172.21.15.88:0/3570356510 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2bd4013cf0 con 0x7f2bd814fde0 2023-10-16T20:05:20.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.182+0000 7f1f0abc5640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0406c430 con 0x7f1f040af730 2023-10-16T20:05:20.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.182+0000 7f1f0abc5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0406b7b0 con 0x7f1f0415ae40 2023-10-16T20:05:20.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.182+0000 7f1f0abc5640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0406ba90 con 0x7f1f0415aa60 2023-10-16T20:05:20.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.182+0000 7f1f08961640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f040af730 0x7f1f0406af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.182+0000 7f1f08961640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f040af730 0x7f1f0406af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40630/0 (socket says 172.21.15.88:40630) 2023-10-16T20:05:20.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.182+0000 7f1f08961640 1 -- 172.21.15.88:0/4262036713 learned_addr learned my addr 172.21.15.88:0/4262036713 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:20.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.182+0000 7f1efbfff640 1 --2- 172.21.15.88:0/4262036713 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1f0415aa60 0x7f1f04138580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.276 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.182+0000 7f1f09162640 1 --2- 172.21.15.88:0/4262036713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f0415ae40 0x7f1f0415b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.276 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.182+0000 7f1f08961640 1 -- 172.21.15.88:0/4262036713 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1f0415aa60 msgr2=0x7f1f04138580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.276 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.182+0000 7f1f08961640 1 --2- 172.21.15.88:0/4262036713 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1f0415aa60 0x7f1f04138580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.276 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.182+0000 7f1f08961640 1 -- 172.21.15.88:0/4262036713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f0415ae40 msgr2=0x7f1f0415b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.276 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.182+0000 7f1f08961640 1 --2- 172.21.15.88:0/4262036713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f0415ae40 0x7f1f0415b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.277 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.182+0000 7f1f08961640 1 -- 172.21.15.88:0/4262036713 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef40094d0 con 0x7f1f040af730 2023-10-16T20:05:20.277 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.182+0000 7f1f08961640 1 --2- 172.21.15.88:0/4262036713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f040af730 0x7f1f0406af80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1ef401a140 tx=0x7f1ef40192b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:20.277 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.182+0000 7f1ef97fa640 1 -- 172.21.15.88:0/4262036713 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4003b00 con 0x7f1f040af730 2023-10-16T20:05:20.277 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.182+0000 7f1ef97fa640 1 -- 172.21.15.88:0/4262036713 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef4003c60 con 0x7f1f040af730 2023-10-16T20:05:20.277 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.182+0000 7f1f0abc5640 1 -- 172.21.15.88:0/4262036713 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f0406bd70 con 0x7f1f040af730 2023-10-16T20:05:20.277 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.182+0000 7f1ef97fa640 1 -- 172.21.15.88:0/4262036713 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef40ab440 con 0x7f1f040af730 2023-10-16T20:05:20.278 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.182+0000 7f21f37fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21f4069e90 0x7f21f413c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.278 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.182+0000 7f21f37fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21f4069e90 0x7f21f413c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40620/0 (socket says 172.21.15.88:40620) 2023-10-16T20:05:20.278 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.186+0000 7f2bcdffb640 1 --2- 172.21.15.88:0/3570356510 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2bbc0665a0 0x7f2bbc068a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.279 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.186+0000 7f2bcdffb640 1 -- 172.21.15.88:0/3570356510 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2bd4084040 con 0x7f2bd814fde0 2023-10-16T20:05:20.279 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.186+0000 7f1f0abc5640 1 -- 172.21.15.88:0/4262036713 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f0406b4c0 con 0x7f1f040af730 2023-10-16T20:05:20.279 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.186+0000 7f1f0abc5640 1 -- 172.21.15.88:0/4262036713 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f0406c570 con 0x7f1f040af730 2023-10-16T20:05:20.280 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.186+0000 7fc1df7fe640 1 -- 172.21.15.88:0/2685888329 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc1d8009cd0 con 0x7fc1e80cfef0 2023-10-16T20:05:20.280 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.186+0000 7f21f37fe640 1 -- 172.21.15.88:0/1726400111 learned_addr learned my addr 172.21.15.88:0/1726400111 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:20.280 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.186+0000 7f21f9c79640 1 -- 172.21.15.88:0/1726400111 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f41415d0 con 0x7f21f4069e90 2023-10-16T20:05:20.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.186+0000 7f21f9c79640 1 -- 172.21.15.88:0/1726400111 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f41406f0 con 0x7f21f413fc50 2023-10-16T20:05:20.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.186+0000 7f21f9c79640 1 -- 172.21.15.88:0/1726400111 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f4140950 con 0x7f21f4160790 2023-10-16T20:05:20.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.186+0000 7f21f2ffd640 1 --2- 172.21.15.88:0/1726400111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21f413fc50 0x7f21f41401b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.186+0000 7f21f3fff640 1 --2- 172.21.15.88:0/1726400111 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f21f4160790 0x7f21f41f8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.186+0000 7f21f37fe640 1 -- 172.21.15.88:0/1726400111 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f21f4160790 msgr2=0x7f21f41f8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.186+0000 7f21f37fe640 1 --2- 172.21.15.88:0/1726400111 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f21f4160790 0x7f21f41f8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.186+0000 7f21f37fe640 1 -- 172.21.15.88:0/1726400111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21f413fc50 msgr2=0x7f21f41401b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.186+0000 7f21f37fe640 1 --2- 172.21.15.88:0/1726400111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21f413fc50 0x7f21f41401b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.186+0000 7f21f37fe640 1 -- 172.21.15.88:0/1726400111 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e8009420 con 0x7f21f4069e90 2023-10-16T20:05:20.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.190+0000 7f21f37fe640 1 --2- 172.21.15.88:0/1726400111 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21f4069e90 0x7f21f413c8d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f21e8018fb0 tx=0x7f21e8019140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:20.283 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.190+0000 7f21dbfff640 1 -- 172.21.15.88:0/1726400111 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e80b4400 con 0x7f21f4069e90 2023-10-16T20:05:20.283 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.186+0000 7f2bdd665640 1 --2- 172.21.15.88:0/3570356510 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2bbc0665a0 0x7f2bbc068a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.283 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.190+0000 7f2bdd665640 1 --2- 172.21.15.88:0/3570356510 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2bbc0665a0 0x7f2bbc068a60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8001040 tx=0x7f2bc8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:20.284 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.190+0000 7f2bcdffb640 1 -- 172.21.15.88:0/3570356510 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2bd40512f0 con 0x7f2bd814fde0 2023-10-16T20:05:20.284 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.190+0000 7fc1df7fe640 1 --2- 172.21.15.88:0/2685888329 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc1c8066570 0x7fc1c8068a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.284 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.190+0000 7fc1df7fe640 1 -- 172.21.15.88:0/2685888329 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc1d8084790 con 0x7fc1e80cfef0 2023-10-16T20:05:20.285 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.190+0000 7f21f3fff640 1 --2- 172.21.15.88:0/1726400111 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f21f4160790 0x7f21f41f8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:20.285 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.190+0000 7f21f2ffd640 1 --2- 172.21.15.88:0/1726400111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21f413fc50 0x7f21f41401b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:20.285 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.190+0000 7f21dbfff640 1 -- 172.21.15.88:0/1726400111 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e80b4560 con 0x7f21f4069e90 2023-10-16T20:05:20.285 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.190+0000 7f21f9c79640 1 -- 172.21.15.88:0/1726400111 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21f4140c30 con 0x7f21f4069e90 2023-10-16T20:05:20.285 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.190+0000 7f21dbfff640 1 -- 172.21.15.88:0/1726400111 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e80b4910 con 0x7f21f4069e90 2023-10-16T20:05:20.286 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.190+0000 7f21f9c79640 1 -- 172.21.15.88:0/1726400111 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21f41411f0 con 0x7f21f4069e90 2023-10-16T20:05:20.286 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.190+0000 7f1ef97fa640 1 -- 172.21.15.88:0/4262036713 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1ef40ab5a0 con 0x7f1f040af730 2023-10-16T20:05:20.286 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.190+0000 7f1ef97fa640 1 --2- 172.21.15.88:0/4262036713 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1ef00664d0 0x7f1ef0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.287 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.190+0000 7f1ef97fa640 1 -- 172.21.15.88:0/4262036713 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1ef411ee80 con 0x7f1f040af730 2023-10-16T20:05:20.287 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.190+0000 7f1efbfff640 1 --2- 172.21.15.88:0/4262036713 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1ef00664d0 0x7f1ef0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.287 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.194+0000 7f1efbfff640 1 --2- 172.21.15.88:0/4262036713 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1ef00664d0 0x7f1ef0068990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1f0413b320 tx=0x7f1f00008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:20.288 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.194+0000 7fc1e6ffd640 1 --2- 172.21.15.88:0/2685888329 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc1c8066570 0x7fc1c8068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.288 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.198+0000 7fc1df7fe640 1 -- 172.21.15.88:0/2685888329 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc1d8051990 con 0x7fc1e80cfef0 2023-10-16T20:05:20.288 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.198+0000 7f21f9c79640 1 -- 172.21.15.88:0/1726400111 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d4005260 con 0x7f21f4069e90 2023-10-16T20:05:20.289 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.198+0000 7fc1e6ffd640 1 --2- 172.21.15.88:0/2685888329 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc1c8066570 0x7fc1c8068a30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc1e80cf2a0 tx=0x7fc1d0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:20.289 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.202+0000 7f21dbfff640 1 -- 172.21.15.88:0/1726400111 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f21e80b2040 con 0x7f21f4069e90 2023-10-16T20:05:20.289 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.202+0000 7f1ef97fa640 1 -- 172.21.15.88:0/4262036713 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1f0406c570 con 0x7f1f040af730 2023-10-16T20:05:20.290 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.202+0000 7f21dbfff640 1 --2- 172.21.15.88:0/1726400111 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f21e00664f0 0x7f21e00689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.290 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.202+0000 7f21f2ffd640 1 --2- 172.21.15.88:0/1726400111 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f21e00664f0 0x7f21e00689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.291 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.206+0000 7f21f2ffd640 1 --2- 172.21.15.88:0/1726400111 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f21e00664f0 0x7f21e00689b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f21f4141ea0 tx=0x7f21e4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:20.291 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.206+0000 7f21dbfff640 1 -- 172.21.15.88:0/1726400111 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f21e80ef2a0 con 0x7f21f4069e90 2023-10-16T20:05:20.291 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.214+0000 7f21dbfff640 1 -- 172.21.15.88:0/1726400111 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f21e80eb5e0 con 0x7f21f4069e90 2023-10-16T20:05:20.435 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.434+0000 7fb7da7fc640 1 -- 172.21.15.88:0/2038259066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb7cc005510 con 0x7fb7f415ccc0 2023-10-16T20:05:20.436 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.434+0000 7fb7f0ff9640 1 -- 172.21.15.88:0/2038259066 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb7ec053cd0 con 0x7fb7f415ccc0 2023-10-16T20:05:20.436 INFO:teuthology.orchestra.run.smithi088.stdout:64424509449 2023-10-16T20:05:20.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.438+0000 7fb7da7fc640 1 -- 172.21.15.88:0/2038259066 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb7e00664a0 msgr2=0x7fb7e0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.438+0000 7fb7da7fc640 1 --2- 172.21.15.88:0/2038259066 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb7e00664a0 0x7fb7e0068960 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb7f41f8ba0 tx=0x7fb7e8009190 comp rx=0 tx=0).stop 2023-10-16T20:05:20.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.438+0000 7fb7da7fc640 1 -- 172.21.15.88:0/2038259066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7f415ccc0 msgr2=0x7fb7f415b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.438+0000 7fb7da7fc640 1 --2- 172.21.15.88:0/2038259066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7f415ccc0 0x7fb7f415b030 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00c7b0 tx=0x7fb7ec00cc70 comp rx=0 tx=0).stop 2023-10-16T20:05:20.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.438+0000 7fb7da7fc640 1 -- 172.21.15.88:0/2038259066 shutdown_connections 2023-10-16T20:05:20.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.438+0000 7fb7da7fc640 1 --2- 172.21.15.88:0/2038259066 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb7e00664a0 0x7fb7e0068960 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.438+0000 7fb7da7fc640 1 --2- 172.21.15.88:0/2038259066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7f415ccc0 0x7fb7f415b030 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.438+0000 7fb7da7fc640 1 --2- 172.21.15.88:0/2038259066 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7f415c8e0 0x7fb7f4153c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.438+0000 7fb7da7fc640 1 --2- 172.21.15.88:0/2038259066 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb7f40b04d0 0x7fb7f4155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.438+0000 7fb7da7fc640 1 -- 172.21.15.88:0/2038259066 >> 172.21.15.88:0/2038259066 conn(0x7fb7f406c170 msgr2=0x7fb7f406c720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:20.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.442+0000 7fb7da7fc640 1 -- 172.21.15.88:0/2038259066 shutdown_connections 2023-10-16T20:05:20.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.442+0000 7fb7da7fc640 1 -- 172.21.15.88:0/2038259066 wait complete. 2023-10-16T20:05:20.474 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.1 2023-10-16T20:05:20.474 DEBUG:teuthology.parallel:result is None 2023-10-16T20:05:20.497 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.498+0000 7fd706cd6640 1 -- 172.21.15.88:0/3082377264 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd70006e720 con 0x7fd7000706e0 2023-10-16T20:05:20.498 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.498+0000 7fd6fd7fa640 1 -- 172.21.15.88:0/3082377264 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd6f4017020 con 0x7fd7000706e0 2023-10-16T20:05:20.498 INFO:teuthology.orchestra.run.smithi088.stdout:64424509448 2023-10-16T20:05:20.501 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.502+0000 7fd706cd6640 1 -- 172.21.15.88:0/3082377264 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd6ec066500 msgr2=0x7fd6ec0689c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.501 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.502+0000 7fd706cd6640 1 --2- 172.21.15.88:0/3082377264 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd6ec066500 0x7fd6ec0689c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd6f00b3000 tx=0x7fd6f00ab070 comp rx=0 tx=0).stop 2023-10-16T20:05:20.502 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.502+0000 7fd706cd6640 1 -- 172.21.15.88:0/3082377264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7000706e0 msgr2=0x7fd70015b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.502 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.502+0000 7fd706cd6640 1 --2- 172.21.15.88:0/3082377264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7000706e0 0x7fd70015b960 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd6f400e8b0 tx=0x7fd6f400ed70 comp rx=0 tx=0).stop 2023-10-16T20:05:20.502 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.502+0000 7fd706cd6640 1 -- 172.21.15.88:0/3082377264 shutdown_connections 2023-10-16T20:05:20.503 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.502+0000 7fd706cd6640 1 --2- 172.21.15.88:0/3082377264 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd6ec066500 0x7fd6ec0689c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.503 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.502+0000 7fd706cd6640 1 --2- 172.21.15.88:0/3082377264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd700158ba0 0x7fd700157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.503 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.502+0000 7fd706cd6640 1 --2- 172.21.15.88:0/3082377264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7000706e0 0x7fd70015b960 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.503 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.502+0000 7fd706cd6640 1 --2- 172.21.15.88:0/3082377264 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd70006df80 0x7fd7001fa020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.503 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.502+0000 7fd706cd6640 1 -- 172.21.15.88:0/3082377264 >> 172.21.15.88:0/3082377264 conn(0x7fd7000cb060 msgr2=0x7fd7000cd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:20.504 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.502+0000 7fd706cd6640 1 -- 172.21.15.88:0/3082377264 shutdown_connections 2023-10-16T20:05:20.504 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.502+0000 7fd706cd6640 1 -- 172.21.15.88:0/3082377264 wait complete. 2023-10-16T20:05:20.505 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.506+0000 7f9d59267640 1 -- 172.21.15.88:0/1115622399 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f9d541598c0 con 0x7f9d54158810 2023-10-16T20:05:20.506 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.506+0000 7f9d50ff9640 1 -- 172.21.15.88:0/1115622399 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9d4c0b2090 con 0x7f9d54158810 2023-10-16T20:05:20.509 INFO:teuthology.orchestra.run.smithi088.stdout:64424509450 2023-10-16T20:05:20.509 INFO:teuthology.orchestra.run.smithi088.stdout:64424509449 2023-10-16T20:05:20.510 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.506+0000 7f1f0abc5640 1 -- 172.21.15.88:0/4262036713 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f1f0406c570 con 0x7f1f040af730 2023-10-16T20:05:20.510 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.506+0000 7f1ef97fa640 1 -- 172.21.15.88:0/4262036713 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1f0406c570 con 0x7f1f040af730 2023-10-16T20:05:20.511 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.510+0000 7f9d59267640 1 -- 172.21.15.88:0/1115622399 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9d40066500 msgr2=0x7f9d400689c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.511 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.510+0000 7f9d59267640 1 --2- 172.21.15.88:0/1115622399 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9d40066500 0x7f9d400689c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9d44000c10 tx=0x7f9d44007200 comp rx=0 tx=0).stop 2023-10-16T20:05:20.512 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.510+0000 7f9d59267640 1 -- 172.21.15.88:0/1115622399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d54158810 msgr2=0x7f9d54154d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.512 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.510+0000 7f9d59267640 1 --2- 172.21.15.88:0/1115622399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d54158810 0x7f9d54154d70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0ac000 tx=0x7f9d4c019660 comp rx=0 tx=0).stop 2023-10-16T20:05:20.512 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.510+0000 7f9d59267640 1 -- 172.21.15.88:0/1115622399 shutdown_connections 2023-10-16T20:05:20.512 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.510+0000 7f9d59267640 1 --2- 172.21.15.88:0/1115622399 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9d40066500 0x7f9d400689c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.512 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.510+0000 7f9d59267640 1 --2- 172.21.15.88:0/1115622399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d54158810 0x7f9d54154d70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.513 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.510+0000 7f9d59267640 1 --2- 172.21.15.88:0/1115622399 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d540c8c40 0x7f9d54154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.513 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.510+0000 7f9d59267640 1 --2- 172.21.15.88:0/1115622399 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d54070560 0x7f9d54153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.513 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.510+0000 7f9d59267640 1 -- 172.21.15.88:0/1115622399 >> 172.21.15.88:0/1115622399 conn(0x7f9d5406a4c0 msgr2=0x7f9d54157b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:20.513 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.510+0000 7f1f0abc5640 1 -- 172.21.15.88:0/4262036713 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1ef00664d0 msgr2=0x7f1ef0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.514 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.510+0000 7f1f0abc5640 1 --2- 172.21.15.88:0/4262036713 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1ef00664d0 0x7f1ef0068990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1f0413b320 tx=0x7f1f00008040 comp rx=0 tx=0).stop 2023-10-16T20:05:20.514 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.510+0000 7f1f0abc5640 1 -- 172.21.15.88:0/4262036713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f040af730 msgr2=0x7f1f0406af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.514 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.510+0000 7f1f0abc5640 1 --2- 172.21.15.88:0/4262036713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f040af730 0x7f1f0406af80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1ef401a140 tx=0x7f1ef40192b0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.515 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.510+0000 7f9d59267640 1 -- 172.21.15.88:0/1115622399 shutdown_connections 2023-10-16T20:05:20.515 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.510+0000 7f9d59267640 1 -- 172.21.15.88:0/1115622399 wait complete. 2023-10-16T20:05:20.516 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.510+0000 7f1f0abc5640 1 -- 172.21.15.88:0/4262036713 shutdown_connections 2023-10-16T20:05:20.516 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.510+0000 7f1f0abc5640 1 --2- 172.21.15.88:0/4262036713 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1ef00664d0 0x7f1ef0068990 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.516 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.510+0000 7f1f0abc5640 1 --2- 172.21.15.88:0/4262036713 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1f0415aa60 0x7f1f04138580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.516 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.510+0000 7f1f0abc5640 1 --2- 172.21.15.88:0/4262036713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f0415ae40 0x7f1f0415b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.517 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.510+0000 7f1f0abc5640 1 --2- 172.21.15.88:0/4262036713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f040af730 0x7f1f0406af80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.517 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.510+0000 7f0d57fb8640 1 -- 172.21.15.88:0/3285547011 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f0d5015df60 con 0x7f0d500cff40 2023-10-16T20:05:20.517 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.510+0000 7f1f0abc5640 1 -- 172.21.15.88:0/4262036713 >> 172.21.15.88:0/4262036713 conn(0x7f1f040cd370 msgr2=0x7f1f04159b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:20.518 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.510+0000 7f1f0abc5640 1 -- 172.21.15.88:0/4262036713 shutdown_connections 2023-10-16T20:05:20.518 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.510+0000 7f1f0abc5640 1 -- 172.21.15.88:0/4262036713 wait complete. 2023-10-16T20:05:20.518 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.510+0000 7f0d467fc640 1 -- 172.21.15.88:0/3285547011 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0d5015df60 con 0x7f0d500cff40 2023-10-16T20:05:20.519 INFO:teuthology.orchestra.run.smithi088.stdout:64424509448 2023-10-16T20:05:20.520 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.514+0000 7f0d57fb8640 1 -- 172.21.15.88:0/3285547011 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0d340664a0 msgr2=0x7f0d34068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.520 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.514+0000 7f0d57fb8640 1 --2- 172.21.15.88:0/3285547011 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0d340664a0 0x7f0d34068960 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0d40007a70 tx=0x7f0d40008040 comp rx=0 tx=0).stop 2023-10-16T20:05:20.520 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.514+0000 7f0d57fb8640 1 -- 172.21.15.88:0/3285547011 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d500cff40 msgr2=0x7f0d5014ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.521 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.514+0000 7f0d57fb8640 1 --2- 172.21.15.88:0/3285547011 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d500cff40 0x7f0d5014ebe0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0d480ac000 tx=0x7f0d48019730 comp rx=0 tx=0).stop 2023-10-16T20:05:20.521 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.514+0000 7f0d57fb8640 1 -- 172.21.15.88:0/3285547011 shutdown_connections 2023-10-16T20:05:20.521 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.514+0000 7f0d57fb8640 1 --2- 172.21.15.88:0/3285547011 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0d340664a0 0x7f0d34068960 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.521 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.514+0000 7f0d57fb8640 1 --2- 172.21.15.88:0/3285547011 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d500cff40 0x7f0d5014ebe0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.521 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.514+0000 7f0d57fb8640 1 --2- 172.21.15.88:0/3285547011 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d5015cb70 0x7f0d50155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.522 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.514+0000 7f0d57fb8640 1 --2- 172.21.15.88:0/3285547011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d500cdad0 0x7f0d5014e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.522 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.514+0000 7f0d57fb8640 1 -- 172.21.15.88:0/3285547011 >> 172.21.15.88:0/3285547011 conn(0x7f0d500719c0 msgr2=0x7f0d50072790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:20.522 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.514+0000 7f2bdf8c9640 1 -- 172.21.15.88:0/3570356510 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2bb8005510 con 0x7f2bd814fde0 2023-10-16T20:05:20.523 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.514+0000 7f0d57fb8640 1 -- 172.21.15.88:0/3285547011 shutdown_connections 2023-10-16T20:05:20.523 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.514+0000 7f0d57fb8640 1 -- 172.21.15.88:0/3285547011 wait complete. 2023-10-16T20:05:20.523 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.518+0000 7f2bcdffb640 1 -- 172.21.15.88:0/3570356510 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2bd4054910 con 0x7f2bd814fde0 2023-10-16T20:05:20.524 INFO:teuthology.orchestra.run.smithi088.stdout:55834574858 2023-10-16T20:05:20.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.522+0000 7f2bdf8c9640 1 -- 172.21.15.88:0/3570356510 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2bbc0665a0 msgr2=0x7f2bbc068a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.522+0000 7f2bdf8c9640 1 --2- 172.21.15.88:0/3570356510 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2bbc0665a0 0x7f2bbc068a60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8001040 tx=0x7f2bc8009040 comp rx=0 tx=0).stop 2023-10-16T20:05:20.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.522+0000 7f2bdf8c9640 1 -- 172.21.15.88:0/3570356510 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bd814fde0 msgr2=0x7f2bd80b6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.522+0000 7f2bdf8c9640 1 --2- 172.21.15.88:0/3570356510 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bd814fde0 0x7f2bd80b6c30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80b5680 tx=0x7f2bd400e9b0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.522+0000 7f2bdf8c9640 1 -- 172.21.15.88:0/3570356510 shutdown_connections 2023-10-16T20:05:20.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.522+0000 7f2bdf8c9640 1 --2- 172.21.15.88:0/3570356510 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2bbc0665a0 0x7f2bbc068a60 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8001040 tx=0x7f2bc8009040 comp rx=0 tx=0).stop 2023-10-16T20:05:20.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.522+0000 7f2bdf8c9640 1 --2- 172.21.15.88:0/3570356510 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bd814fde0 0x7f2bd80b6c30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.522+0000 7f2bdf8c9640 1 --2- 172.21.15.88:0/3570356510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bd815fbf0 0x7f2bd80c5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.522+0000 7f2bdf8c9640 1 --2- 172.21.15.88:0/3570356510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc40047d0 0x7f2bd80cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.522+0000 7f2bdf8c9640 1 -- 172.21.15.88:0/3570356510 >> 172.21.15.88:0/3570356510 conn(0x7f2bd80719c0 msgr2=0x7f2bd80cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:20.527 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.526+0000 7f2bdf8c9640 1 -- 172.21.15.88:0/3570356510 shutdown_connections 2023-10-16T20:05:20.527 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.526+0000 7f2bdf8c9640 1 -- 172.21.15.88:0/3570356510 wait complete. 2023-10-16T20:05:20.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.526+0000 7fc1ed565640 1 -- 172.21.15.88:0/2685888329 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fc1e80b2e90 con 0x7fc1e80cfef0 2023-10-16T20:05:20.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.530+0000 7fc1df7fe640 1 -- 172.21.15.88:0/2685888329 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc1d8054fb0 con 0x7fc1e80cfef0 2023-10-16T20:05:20.530 INFO:teuthology.orchestra.run.smithi088.stdout:64424509448 2023-10-16T20:05:20.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.530+0000 7fc1ed565640 1 -- 172.21.15.88:0/2685888329 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc1c8066570 msgr2=0x7fc1c8068a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.530+0000 7fc1ed565640 1 --2- 172.21.15.88:0/2685888329 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc1c8066570 0x7fc1c8068a30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc1e80cf2a0 tx=0x7fc1d0008040 comp rx=0 tx=0).stop 2023-10-16T20:05:20.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.530+0000 7fc1ed565640 1 -- 172.21.15.88:0/2685888329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1e80cfef0 msgr2=0x7fc1e8146ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.530+0000 7fc1ed565640 1 --2- 172.21.15.88:0/2685888329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1e80cfef0 0x7fc1e8146ee0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc1d800d800 tx=0x7fc1d800dcc0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.530+0000 7fc1ed565640 1 -- 172.21.15.88:0/2685888329 shutdown_connections 2023-10-16T20:05:20.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.530+0000 7fc1ed565640 1 --2- 172.21.15.88:0/2685888329 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc1c8066570 0x7fc1c8068a30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.530+0000 7fc1ed565640 1 --2- 172.21.15.88:0/2685888329 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1e8158000 0x7fc1e806dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.530+0000 7fc1ed565640 1 --2- 172.21.15.88:0/2685888329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1e80cfef0 0x7fc1e8146ee0 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc1d800d800 tx=0x7fc1d800dcc0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.530+0000 7fc1ed565640 1 --2- 172.21.15.88:0/2685888329 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1e80b4dc0 0x7fc1e8070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.535 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.530+0000 7fc1ed565640 1 -- 172.21.15.88:0/2685888329 >> 172.21.15.88:0/2685888329 conn(0x7fc1e80719c0 msgr2=0x7fc1e80b3990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:20.535 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.530+0000 7fc1ed565640 1 -- 172.21.15.88:0/2685888329 shutdown_connections 2023-10-16T20:05:20.535 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.530+0000 7fc1ed565640 1 -- 172.21.15.88:0/2685888329 wait complete. 2023-10-16T20:05:20.537 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.2 2023-10-16T20:05:20.537 DEBUG:teuthology.parallel:result is None 2023-10-16T20:05:20.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.546+0000 7f21f9c79640 1 -- 172.21.15.88:0/1726400111 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f21d4005510 con 0x7f21f4069e90 2023-10-16T20:05:20.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.546+0000 7f21dbfff640 1 -- 172.21.15.88:0/1726400111 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f21e80b0020 con 0x7f21f4069e90 2023-10-16T20:05:20.546 INFO:teuthology.orchestra.run.smithi088.stdout:64424509449 2023-10-16T20:05:20.547 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.4 2023-10-16T20:05:20.547 DEBUG:teuthology.parallel:result is None 2023-10-16T20:05:20.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.546+0000 7f21f9c79640 1 -- 172.21.15.88:0/1726400111 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f21e00664f0 msgr2=0x7f21e00689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.546+0000 7f21f9c79640 1 --2- 172.21.15.88:0/1726400111 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f21e00664f0 0x7f21e00689b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f21f4141ea0 tx=0x7f21e4008040 comp rx=0 tx=0).stop 2023-10-16T20:05:20.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.546+0000 7f21f9c79640 1 -- 172.21.15.88:0/1726400111 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21f4069e90 msgr2=0x7f21f413c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.546+0000 7f21f9c79640 1 --2- 172.21.15.88:0/1726400111 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21f4069e90 0x7f21f413c8d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f21e8018fb0 tx=0x7f21e8019140 comp rx=0 tx=0).stop 2023-10-16T20:05:20.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.546+0000 7f21f9c79640 1 -- 172.21.15.88:0/1726400111 shutdown_connections 2023-10-16T20:05:20.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.546+0000 7f21f9c79640 1 --2- 172.21.15.88:0/1726400111 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f21e00664f0 0x7f21e00689b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.546+0000 7f21f9c79640 1 --2- 172.21.15.88:0/1726400111 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f21f4160790 0x7f21f41f8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.546+0000 7f21f9c79640 1 --2- 172.21.15.88:0/1726400111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21f413fc50 0x7f21f41401b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.546+0000 7f21f9c79640 1 --2- 172.21.15.88:0/1726400111 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21f4069e90 0x7f21f413c8d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.546+0000 7f21f9c79640 1 -- 172.21.15.88:0/1726400111 >> 172.21.15.88:0/1726400111 conn(0x7f21f40ce5e0 msgr2=0x7f21f415e670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:20.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.546+0000 7f21f9c79640 1 -- 172.21.15.88:0/1726400111 shutdown_connections 2023-10-16T20:05:20.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.546+0000 7f21f9c79640 1 -- 172.21.15.88:0/1726400111 wait complete. 2023-10-16T20:05:20.552 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.7 2023-10-16T20:05:20.552 DEBUG:teuthology.parallel:result is None 2023-10-16T20:05:20.553 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.6 2023-10-16T20:05:20.553 DEBUG:teuthology.parallel:result is None 2023-10-16T20:05:20.563 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.0 2023-10-16T20:05:20.563 DEBUG:teuthology.parallel:result is None 2023-10-16T20:05:20.564 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.5 2023-10-16T20:05:20.564 DEBUG:teuthology.parallel:result is None 2023-10-16T20:05:20.571 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.3 2023-10-16T20:05:20.571 DEBUG:teuthology.parallel:result is None 2023-10-16T20:05:20.571 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-16T20:05:20.572 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T20:05:20.753 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.750+0000 7fe269d0f640 1 Processor -- start 2023-10-16T20:05:20.753 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.754+0000 7fe269d0f640 1 -- start start 2023-10-16T20:05:20.755 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.754+0000 7fe269d0f640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe2640ade20 0x7fe2640b1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.755 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.754+0000 7fe269d0f640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2640b2570 con 0x7fe2640ada40 2023-10-16T20:05:20.755 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.754+0000 7fe269d0f640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2640cca50 con 0x7fe264143eb0 2023-10-16T20:05:20.755 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.754+0000 7fe269d0f640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe264144320 con 0x7fe2640ade20 2023-10-16T20:05:20.756 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.754+0000 7fe2637fe640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fe2640ada40 0x7fe26415adb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:39270/0 (socket says 172.21.15.88:39270) 2023-10-16T20:05:20.756 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.754+0000 7fe2637fe640 1 -- 172.21.15.88:0/2599016148 learned_addr learned my addr 172.21.15.88:0/2599016148 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:20.756 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.754+0000 7fe262ffd640 1 --2- 172.21.15.88:0/2599016148 >> v2:172.21.15.94:3300/0 conn(0x7fe2640ade20 0x7fe2640b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.756 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.754+0000 7fe2627fc640 1 -- 172.21.15.88:0/2599016148 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4119845031 0 0) 0x7fe2640b2570 con 0x7fe2640ada40 2023-10-16T20:05:20.756 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.754+0000 7fe2627fc640 1 -- 172.21.15.88:0/2599016148 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe24c003530 con 0x7fe2640ada40 2023-10-16T20:05:20.758 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.754+0000 7fe2627fc640 1 -- 172.21.15.88:0/2599016148 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 442035316 0 0) 0x7fe24c003530 con 0x7fe2640ada40 2023-10-16T20:05:20.758 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.754+0000 7fe2627fc640 1 -- 172.21.15.88:0/2599016148 >> v2:172.21.15.94:3300/0 conn(0x7fe2640ade20 msgr2=0x7fe2640b1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:20.758 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.754+0000 7fe2627fc640 1 --2- 172.21.15.88:0/2599016148 >> v2:172.21.15.94:3300/0 conn(0x7fe2640ade20 0x7fe2640b1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.754+0000 7fe2627fc640 1 -- 172.21.15.88:0/2599016148 >> v1:172.21.15.94:6789/0 conn(0x7fe264143eb0 legacy=0x7fe2640b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.754+0000 7fe2627fc640 1 -- 172.21.15.88:0/2599016148 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2640b1fb0 con 0x7fe2640ada40 2023-10-16T20:05:20.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.754+0000 7fe2627fc640 1 -- 172.21.15.88:0/2599016148 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7fe254001680 con 0x7fe2640ada40 2023-10-16T20:05:20.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.754+0000 7fe2627fc640 1 -- 172.21.15.88:0/2599016148 >> v1:172.21.15.88:6789/0 conn(0x7fe2640ada40 legacy=0x7fe26415adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.754+0000 7fe2627fc640 1 --2- 172.21.15.88:0/2599016148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe24c003980 0x7fe24c003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.760 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.754+0000 7fe2627fc640 1 --2- 172.21.15.88:0/2599016148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe24c004380 0x7fe24c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.760 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.754+0000 7fe2627fc640 1 --2- 172.21.15.88:0/2599016148 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe24c0050f0 0x7fe24c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.760 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.754+0000 7fe2627fc640 1 -- 172.21.15.88:0/2599016148 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe264144320 con 0x7fe24c003980 2023-10-16T20:05:20.760 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.754+0000 7fe2627fc640 1 -- 172.21.15.88:0/2599016148 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24c00c030 con 0x7fe24c004380 2023-10-16T20:05:20.760 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.754+0000 7fe2627fc640 1 -- 172.21.15.88:0/2599016148 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe24c00c1f0 con 0x7fe24c0050f0 2023-10-16T20:05:20.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.754+0000 7fe2637fe640 1 --2- 172.21.15.88:0/2599016148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe24c003980 0x7fe24c003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.754+0000 7fe262ffd640 1 --2- 172.21.15.88:0/2599016148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe24c004380 0x7fe24c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.754+0000 7fe263fff640 1 --2- 172.21.15.88:0/2599016148 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe24c0050f0 0x7fe24c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.754+0000 7fe2637fe640 1 -- 172.21.15.88:0/2599016148 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe24c0050f0 msgr2=0x7fe24c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:20.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.754+0000 7fe2637fe640 1 --2- 172.21.15.88:0/2599016148 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe24c0050f0 0x7fe24c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.754+0000 7fe2637fe640 1 -- 172.21.15.88:0/2599016148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe24c004380 msgr2=0x7fe24c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:20.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.754+0000 7fe2637fe640 1 --2- 172.21.15.88:0/2599016148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe24c004380 0x7fe24c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.754+0000 7fe2637fe640 1 -- 172.21.15.88:0/2599016148 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24c003530 con 0x7fe24c003980 2023-10-16T20:05:20.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.754+0000 7fe263fff640 1 -- 172.21.15.88:0/2599016148 reap_dead start 2023-10-16T20:05:20.765 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.758+0000 7fe2637fe640 1 --2- 172.21.15.88:0/2599016148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe24c003980 0x7fe24c003d60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe25400c210 tx=0x7fe25401cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e1a48e1271fd16 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:20.766 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.758+0000 7fe2627fc640 1 -- 172.21.15.88:0/2599016148 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe254005ae0 con 0x7fe24c003980 2023-10-16T20:05:20.766 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.758+0000 7fe2627fc640 1 -- 172.21.15.88:0/2599016148 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe254005c60 con 0x7fe24c003980 2023-10-16T20:05:20.766 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.758+0000 7fe2627fc640 1 -- 172.21.15.88:0/2599016148 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2540a5a30 con 0x7fe24c003980 2023-10-16T20:05:20.766 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.758+0000 7fe269d0f640 1 -- 172.21.15.88:0/2599016148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe24c003980 msgr2=0x7fe24c003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.766 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.758+0000 7fe269d0f640 1 --2- 172.21.15.88:0/2599016148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe24c003980 0x7fe24c003d60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe25400c210 tx=0x7fe25401cc30 comp rx=0 tx=0).stop 2023-10-16T20:05:20.767 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.758+0000 7fe269d0f640 1 -- 172.21.15.88:0/2599016148 shutdown_connections 2023-10-16T20:05:20.767 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.758+0000 7fe269d0f640 1 --2- 172.21.15.88:0/2599016148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe24c003980 0x7fe24c003d60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.767 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.758+0000 7fe269d0f640 1 -- 172.21.15.88:0/2599016148 >> 172.21.15.88:0/2599016148 conn(0x7fe2640ce5e0 msgr2=0x7fe2640ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:20.767 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.758+0000 7fe269d0f640 1 -- 172.21.15.88:0/2599016148 shutdown_connections 2023-10-16T20:05:20.767 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.758+0000 7fe269d0f640 1 -- 172.21.15.88:0/2599016148 wait complete. 2023-10-16T20:05:20.768 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.762+0000 7fe269d0f640 1 Processor -- start 2023-10-16T20:05:20.768 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.762+0000 7fe269d0f640 1 -- start start 2023-10-16T20:05:20.768 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.762+0000 7fe269d0f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe24c003980 0x7fe264133f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.768 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.762+0000 7fe269d0f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2640a9ec0 0x7fe2640aa340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.768 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.762+0000 7fe269d0f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe26413ebc0 0x7fe26413f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.769 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.762+0000 7fe269d0f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2640a75a0 con 0x7fe24c003980 2023-10-16T20:05:20.769 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.762+0000 7fe269d0f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2640aa880 con 0x7fe2640a9ec0 2023-10-16T20:05:20.769 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.762+0000 7fe269d0f640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe26413f950 con 0x7fe26413ebc0 2023-10-16T20:05:20.769 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.762+0000 7fe2637fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe24c003980 0x7fe264133f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.769 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.762+0000 7fe2637fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe24c003980 0x7fe264133f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40650/0 (socket says 172.21.15.88:40650) 2023-10-16T20:05:20.770 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.762+0000 7fe2637fe640 1 -- 172.21.15.88:0/923862240 learned_addr learned my addr 172.21.15.88:0/923862240 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:20.770 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.762+0000 7fe262ffd640 1 --2- 172.21.15.88:0/923862240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2640a9ec0 0x7fe2640aa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.770 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.762+0000 7fe263fff640 1 --2- 172.21.15.88:0/923862240 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe26413ebc0 0x7fe26413f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.770 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.762+0000 7fe2637fe640 1 -- 172.21.15.88:0/923862240 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe26413ebc0 msgr2=0x7fe26413f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.770 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.762+0000 7fe2637fe640 1 --2- 172.21.15.88:0/923862240 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe26413ebc0 0x7fe26413f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.771 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.762+0000 7fe2637fe640 1 -- 172.21.15.88:0/923862240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2640a9ec0 msgr2=0x7fe2640aa340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.771 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.762+0000 7fe2637fe640 1 --2- 172.21.15.88:0/923862240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2640a9ec0 0x7fe2640aa340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.771 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.762+0000 7fe2637fe640 1 -- 172.21.15.88:0/923862240 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe26413fc30 con 0x7fe24c003980 2023-10-16T20:05:20.771 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.762+0000 7fe2637fe640 1 --2- 172.21.15.88:0/923862240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe24c003980 0x7fe264133f50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe25400c210 tx=0x7fe24c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:20.771 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.762+0000 7fe24bfff640 1 -- 172.21.15.88:0/923862240 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2540b4070 con 0x7fe24c003980 2023-10-16T20:05:20.772 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.762+0000 7fe24bfff640 1 -- 172.21.15.88:0/923862240 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe254000fa0 con 0x7fe24c003980 2023-10-16T20:05:20.772 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.762+0000 7fe269d0f640 1 -- 172.21.15.88:0/923862240 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe26413ff10 con 0x7fe24c003980 2023-10-16T20:05:20.772 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.762+0000 7fe24bfff640 1 -- 172.21.15.88:0/923862240 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe254001660 con 0x7fe24c003980 2023-10-16T20:05:20.772 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.762+0000 7fe269d0f640 1 -- 172.21.15.88:0/923862240 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe26413f580 con 0x7fe24c003980 2023-10-16T20:05:20.772 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.766+0000 7fe269d0f640 1 -- 172.21.15.88:0/923862240 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2641403c0 con 0x7fe24c003980 2023-10-16T20:05:20.773 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.766+0000 7fe24bfff640 1 -- 172.21.15.88:0/923862240 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe2540a57f0 con 0x7fe24c003980 2023-10-16T20:05:20.773 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.766+0000 7fe24bfff640 1 --2- 172.21.15.88:0/923862240 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe2500664a0 0x7fe250068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:20.774 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.766+0000 7fe24bfff640 1 -- 172.21.15.88:0/923862240 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe25411da90 con 0x7fe24c003980 2023-10-16T20:05:20.774 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.766+0000 7fe262ffd640 1 --2- 172.21.15.88:0/923862240 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe2500664a0 0x7fe250068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:20.774 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.766+0000 7fe262ffd640 1 --2- 172.21.15.88:0/923862240 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe2500664a0 0x7fe250068960 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe26415c8c0 tx=0x7fe258008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:20.774 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.770+0000 7fe24bfff640 1 -- 172.21.15.88:0/923862240 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe2540eac30 con 0x7fe24c003980 2023-10-16T20:05:20.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.914+0000 7fe269d0f640 1 -- 172.21.15.88:0/923862240 --> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe2640b1d30 con 0x7fe2500664a0 2023-10-16T20:05:20.925 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.922+0000 7fe24bfff640 1 -- 172.21.15.88:0/923862240 <== mgr.4110 v2:172.21.15.88:6832/4078483333 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345325 (secure 0 0 0) 0x7fe258006be0 con 0x7fe2500664a0 2023-10-16T20:05:20.928 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:20.930 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2023-10-16T20:05:20.932 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.930+0000 7fe269d0f640 1 -- 172.21.15.88:0/923862240 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe2500664a0 msgr2=0x7fe250068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.932 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.930+0000 7fe269d0f640 1 --2- 172.21.15.88:0/923862240 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe2500664a0 0x7fe250068960 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe26415c8c0 tx=0x7fe258008040 comp rx=0 tx=0).stop 2023-10-16T20:05:20.932 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.930+0000 7fe269d0f640 1 -- 172.21.15.88:0/923862240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe24c003980 msgr2=0x7fe264133f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:20.932 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.930+0000 7fe269d0f640 1 --2- 172.21.15.88:0/923862240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe24c003980 0x7fe264133f50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe25400c210 tx=0x7fe24c005800 comp rx=0 tx=0).stop 2023-10-16T20:05:20.932 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.930+0000 7fe269d0f640 1 -- 172.21.15.88:0/923862240 shutdown_connections 2023-10-16T20:05:20.933 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.930+0000 7fe269d0f640 1 --2- 172.21.15.88:0/923862240 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe2500664a0 0x7fe250068960 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.933 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.930+0000 7fe269d0f640 1 --2- 172.21.15.88:0/923862240 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe26413ebc0 0x7fe26413f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.933 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.930+0000 7fe269d0f640 1 --2- 172.21.15.88:0/923862240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2640a9ec0 0x7fe2640aa340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.933 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.930+0000 7fe269d0f640 1 --2- 172.21.15.88:0/923862240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe24c003980 0x7fe264133f50 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:20.933 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.930+0000 7fe269d0f640 1 -- 172.21.15.88:0/923862240 >> 172.21.15.88:0/923862240 conn(0x7fe2640ce5e0 msgr2=0x7fe26415b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:20.934 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.930+0000 7fe269d0f640 1 -- 172.21.15.88:0/923862240 shutdown_connections 2023-10-16T20:05:20.934 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:20.930+0000 7fe269d0f640 1 -- 172.21.15.88:0/923862240 wait complete. 2023-10-16T20:05:20.949 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2023-10-16T20:05:20.449726+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219628,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755092,"statfs":{"total":773094113280,"available":772869214208,"internally_reserved":0,"allocated":3514368,"data_stored":2070598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":221236793},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2229,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003612"},"pg_stats":[{"pgid":"2.1f","version":"22'4","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.465897+0000","last_change":"2023-10-16T20:04:56.406335+0000","last_active":"2023-10-16T20:05:01.465897+0000","last_peered":"2023-10-16T20:05:01.465897+0000","last_clean":"2023-10-16T20:05:01.465897+0000","last_became_active":"2023-10-16T20:04:56.406060+0000","last_became_peered":"2023-10-16T20:04:56.406060+0000","last_unstale":"2023-10-16T20:05:01.465897+0000","last_undegraded":"2023-10-16T20:05:01.465897+0000","last_fullsized":"2023-10-16T20:05:01.465897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:15:58.928603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442061+0000","last_change":"2023-10-16T20:04:57.416039+0000","last_active":"2023-10-16T20:05:01.442061+0000","last_peered":"2023-10-16T20:05:01.442061+0000","last_clean":"2023-10-16T20:05:01.442061+0000","last_became_active":"2023-10-16T20:04:57.415746+0000","last_became_peered":"2023-10-16T20:04:57.415746+0000","last_unstale":"2023-10-16T20:05:01.442061+0000","last_undegraded":"2023-10-16T20:05:01.442061+0000","last_fullsized":"2023-10-16T20:05:01.442061+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:27:55.972790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.685606+0000","last_change":"2023-10-16T20:05:01.770262+0000","last_active":"2023-10-16T20:05:02.685606+0000","last_peered":"2023-10-16T20:05:02.685606+0000","last_clean":"2023-10-16T20:05:02.685606+0000","last_became_active":"2023-10-16T20:05:01.769688+0000","last_became_peered":"2023-10-16T20:05:01.769688+0000","last_unstale":"2023-10-16T20:05:02.685606+0000","last_undegraded":"2023-10-16T20:05:02.685606+0000","last_fullsized":"2023-10-16T20:05:02.685606+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:51:12.210160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442225+0000","last_change":"2023-10-16T20:04:56.409621+0000","last_active":"2023-10-16T20:05:01.442225+0000","last_peered":"2023-10-16T20:05:01.442225+0000","last_clean":"2023-10-16T20:05:01.442225+0000","last_became_active":"2023-10-16T20:04:56.409150+0000","last_became_peered":"2023-10-16T20:04:56.409150+0000","last_unstale":"2023-10-16T20:05:01.442225+0000","last_undegraded":"2023-10-16T20:05:01.442225+0000","last_fullsized":"2023-10-16T20:05:01.442225+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:24:16.965233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.465724+0000","last_change":"2023-10-16T20:04:57.430587+0000","last_active":"2023-10-16T20:05:01.465724+0000","last_peered":"2023-10-16T20:05:01.465724+0000","last_clean":"2023-10-16T20:05:01.465724+0000","last_became_active":"2023-10-16T20:04:57.430224+0000","last_became_peered":"2023-10-16T20:04:57.430224+0000","last_unstale":"2023-10-16T20:05:01.465724+0000","last_undegraded":"2023-10-16T20:05:01.465724+0000","last_fullsized":"2023-10-16T20:05:01.465724+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:49:52.662102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.682460+0000","last_change":"2023-10-16T20:05:01.478221+0000","last_active":"2023-10-16T20:05:02.682460+0000","last_peered":"2023-10-16T20:05:02.682460+0000","last_clean":"2023-10-16T20:05:02.682460+0000","last_became_active":"2023-10-16T20:05:01.477692+0000","last_became_peered":"2023-10-16T20:05:01.477692+0000","last_unstale":"2023-10-16T20:05:02.682460+0000","last_undegraded":"2023-10-16T20:05:02.682460+0000","last_fullsized":"2023-10-16T20:05:02.682460+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:58:37.917893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.758057+0000","last_change":"2023-10-16T20:04:56.409779+0000","last_active":"2023-10-16T20:05:01.758057+0000","last_peered":"2023-10-16T20:05:01.758057+0000","last_clean":"2023-10-16T20:05:01.758057+0000","last_became_active":"2023-10-16T20:04:56.409515+0000","last_became_peered":"2023-10-16T20:04:56.409515+0000","last_unstale":"2023-10-16T20:05:01.758057+0000","last_undegraded":"2023-10-16T20:05:01.758057+0000","last_fullsized":"2023-10-16T20:05:01.758057+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:17:23.198236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438236+0000","last_change":"2023-10-16T20:04:57.414284+0000","last_active":"2023-10-16T20:05:01.438236+0000","last_peered":"2023-10-16T20:05:01.438236+0000","last_clean":"2023-10-16T20:05:01.438236+0000","last_became_active":"2023-10-16T20:04:57.413997+0000","last_became_peered":"2023-10-16T20:04:57.413997+0000","last_unstale":"2023-10-16T20:05:01.438236+0000","last_undegraded":"2023-10-16T20:05:01.438236+0000","last_fullsized":"2023-10-16T20:05:01.438236+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:28:00.096002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.682751+0000","last_change":"2023-10-16T20:05:01.482429+0000","last_active":"2023-10-16T20:05:02.682751+0000","last_peered":"2023-10-16T20:05:02.682751+0000","last_clean":"2023-10-16T20:05:02.682751+0000","last_became_active":"2023-10-16T20:05:01.481993+0000","last_became_peered":"2023-10-16T20:05:01.481993+0000","last_unstale":"2023-10-16T20:05:02.682751+0000","last_undegraded":"2023-10-16T20:05:02.682751+0000","last_fullsized":"2023-10-16T20:05:02.682751+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:54:13.091947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757866+0000","last_change":"2023-10-16T20:04:56.412981+0000","last_active":"2023-10-16T20:05:01.757866+0000","last_peered":"2023-10-16T20:05:01.757866+0000","last_clean":"2023-10-16T20:05:01.757866+0000","last_became_active":"2023-10-16T20:04:56.412613+0000","last_became_peered":"2023-10-16T20:04:56.412613+0000","last_unstale":"2023-10-16T20:05:01.757866+0000","last_undegraded":"2023-10-16T20:05:01.757866+0000","last_fullsized":"2023-10-16T20:05:01.757866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:14:27.634593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.440267+0000","last_change":"2023-10-16T20:04:57.411866+0000","last_active":"2023-10-16T20:05:01.440267+0000","last_peered":"2023-10-16T20:05:01.440267+0000","last_clean":"2023-10-16T20:05:01.440267+0000","last_became_active":"2023-10-16T20:04:57.411523+0000","last_became_peered":"2023-10-16T20:04:57.411523+0000","last_unstale":"2023-10-16T20:05:01.440267+0000","last_undegraded":"2023-10-16T20:05:01.440267+0000","last_fullsized":"2023-10-16T20:05:01.440267+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:01:38.242317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.683398+0000","last_change":"2023-10-16T20:05:01.460546+0000","last_active":"2023-10-16T20:05:02.683398+0000","last_peered":"2023-10-16T20:05:02.683398+0000","last_clean":"2023-10-16T20:05:02.683398+0000","last_became_active":"2023-10-16T20:05:01.460120+0000","last_became_peered":"2023-10-16T20:05:01.460120+0000","last_unstale":"2023-10-16T20:05:02.683398+0000","last_undegraded":"2023-10-16T20:05:02.683398+0000","last_fullsized":"2023-10-16T20:05:02.683398+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:31:38.939408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438705+0000","last_change":"2023-10-16T20:04:57.409882+0000","last_active":"2023-10-16T20:05:01.438705+0000","last_peered":"2023-10-16T20:05:01.438705+0000","last_clean":"2023-10-16T20:05:01.438705+0000","last_became_active":"2023-10-16T20:04:57.409534+0000","last_became_peered":"2023-10-16T20:04:57.409534+0000","last_unstale":"2023-10-16T20:05:01.438705+0000","last_undegraded":"2023-10-16T20:05:01.438705+0000","last_fullsized":"2023-10-16T20:05:01.438705+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:23:01.141545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757875+0000","last_change":"2023-10-16T20:04:56.406078+0000","last_active":"2023-10-16T20:05:01.757875+0000","last_peered":"2023-10-16T20:05:01.757875+0000","last_clean":"2023-10-16T20:05:01.757875+0000","last_became_active":"2023-10-16T20:04:56.405784+0000","last_became_peered":"2023-10-16T20:04:56.405784+0000","last_unstale":"2023-10-16T20:05:01.757875+0000","last_undegraded":"2023-10-16T20:05:01.757875+0000","last_fullsized":"2023-10-16T20:05:01.757875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:13:40.552947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.683535+0000","last_change":"2023-10-16T20:05:01.477288+0000","last_active":"2023-10-16T20:05:02.683535+0000","last_peered":"2023-10-16T20:05:02.683535+0000","last_clean":"2023-10-16T20:05:02.683535+0000","last_became_active":"2023-10-16T20:05:01.476269+0000","last_became_peered":"2023-10-16T20:05:01.476269+0000","last_unstale":"2023-10-16T20:05:02.683535+0000","last_undegraded":"2023-10-16T20:05:02.683535+0000","last_fullsized":"2023-10-16T20:05:02.683535+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:28:04.151249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.441837+0000","last_change":"2023-10-16T20:04:57.426154+0000","last_active":"2023-10-16T20:05:01.441837+0000","last_peered":"2023-10-16T20:05:01.441837+0000","last_clean":"2023-10-16T20:05:01.441837+0000","last_became_active":"2023-10-16T20:04:57.425681+0000","last_became_peered":"2023-10-16T20:04:57.425681+0000","last_unstale":"2023-10-16T20:05:01.441837+0000","last_undegraded":"2023-10-16T20:05:01.441837+0000","last_fullsized":"2023-10-16T20:05:01.441837+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:54:09.036700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471304+0000","last_change":"2023-10-16T20:04:56.406283+0000","last_active":"2023-10-16T20:05:01.471304+0000","last_peered":"2023-10-16T20:05:01.471304+0000","last_clean":"2023-10-16T20:05:01.471304+0000","last_became_active":"2023-10-16T20:04:56.405952+0000","last_became_peered":"2023-10-16T20:04:56.405952+0000","last_unstale":"2023-10-16T20:05:01.471304+0000","last_undegraded":"2023-10-16T20:05:01.471304+0000","last_fullsized":"2023-10-16T20:05:01.471304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:14:27.634593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.682647+0000","last_change":"2023-10-16T20:05:01.482495+0000","last_active":"2023-10-16T20:05:02.682647+0000","last_peered":"2023-10-16T20:05:02.682647+0000","last_clean":"2023-10-16T20:05:02.682647+0000","last_became_active":"2023-10-16T20:05:01.482072+0000","last_became_peered":"2023-10-16T20:05:01.482072+0000","last_unstale":"2023-10-16T20:05:02.682647+0000","last_undegraded":"2023-10-16T20:05:02.682647+0000","last_fullsized":"2023-10-16T20:05:02.682647+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:28:00.028037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442089+0000","last_change":"2023-10-16T20:04:57.425668+0000","last_active":"2023-10-16T20:05:01.442089+0000","last_peered":"2023-10-16T20:05:01.442089+0000","last_clean":"2023-10-16T20:05:01.442089+0000","last_became_active":"2023-10-16T20:04:57.425420+0000","last_became_peered":"2023-10-16T20:04:57.425420+0000","last_unstale":"2023-10-16T20:05:01.442089+0000","last_undegraded":"2023-10-16T20:05:01.442089+0000","last_fullsized":"2023-10-16T20:05:01.442089+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:33:36.402743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471190+0000","last_change":"2023-10-16T20:04:56.403231+0000","last_active":"2023-10-16T20:05:01.471190+0000","last_peered":"2023-10-16T20:05:01.471190+0000","last_clean":"2023-10-16T20:05:01.471190+0000","last_became_active":"2023-10-16T20:04:56.402769+0000","last_became_peered":"2023-10-16T20:04:56.402769+0000","last_unstale":"2023-10-16T20:05:01.471190+0000","last_undegraded":"2023-10-16T20:05:01.471190+0000","last_fullsized":"2023-10-16T20:05:01.471190+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:55:47.529420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.445092+0000","last_change":"2023-10-16T20:05:01.768596+0000","last_active":"2023-10-16T20:05:02.445092+0000","last_peered":"2023-10-16T20:05:02.445092+0000","last_clean":"2023-10-16T20:05:02.445092+0000","last_became_active":"2023-10-16T20:05:01.768140+0000","last_became_peered":"2023-10-16T20:05:01.768140+0000","last_unstale":"2023-10-16T20:05:02.445092+0000","last_undegraded":"2023-10-16T20:05:02.445092+0000","last_fullsized":"2023-10-16T20:05:02.445092+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:01:42.297564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.447386+0000","last_change":"2023-10-16T20:04:57.407922+0000","last_active":"2023-10-16T20:05:01.447386+0000","last_peered":"2023-10-16T20:05:01.447386+0000","last_clean":"2023-10-16T20:05:01.447386+0000","last_became_active":"2023-10-16T20:04:57.407580+0000","last_became_peered":"2023-10-16T20:04:57.407580+0000","last_unstale":"2023-10-16T20:05:01.447386+0000","last_undegraded":"2023-10-16T20:05:01.447386+0000","last_fullsized":"2023-10-16T20:05:01.447386+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:24:17.972220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757524+0000","last_change":"2023-10-16T20:04:56.408300+0000","last_active":"2023-10-16T20:05:01.757524+0000","last_peered":"2023-10-16T20:05:01.757524+0000","last_clean":"2023-10-16T20:05:01.757524+0000","last_became_active":"2023-10-16T20:04:56.407908+0000","last_became_peered":"2023-10-16T20:04:56.407908+0000","last_unstale":"2023-10-16T20:05:01.757524+0000","last_undegraded":"2023-10-16T20:05:01.757524+0000","last_fullsized":"2023-10-16T20:05:01.757524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:53:19.915183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.683126+0000","last_change":"2023-10-16T20:05:01.476964+0000","last_active":"2023-10-16T20:05:02.683126+0000","last_peered":"2023-10-16T20:05:02.683126+0000","last_clean":"2023-10-16T20:05:02.683126+0000","last_became_active":"2023-10-16T20:05:01.476584+0000","last_became_peered":"2023-10-16T20:05:01.476584+0000","last_unstale":"2023-10-16T20:05:02.683126+0000","last_undegraded":"2023-10-16T20:05:02.683126+0000","last_fullsized":"2023-10-16T20:05:02.683126+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:22:16.748229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.465558+0000","last_change":"2023-10-16T20:04:57.424772+0000","last_active":"2023-10-16T20:05:01.465558+0000","last_peered":"2023-10-16T20:05:01.465558+0000","last_clean":"2023-10-16T20:05:01.465558+0000","last_became_active":"2023-10-16T20:04:57.424340+0000","last_became_peered":"2023-10-16T20:04:57.424340+0000","last_unstale":"2023-10-16T20:05:01.465558+0000","last_undegraded":"2023-10-16T20:05:01.465558+0000","last_fullsized":"2023-10-16T20:05:01.465558+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:10:08.760019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.439134+0000","last_change":"2023-10-16T20:04:56.401336+0000","last_active":"2023-10-16T20:05:01.439134+0000","last_peered":"2023-10-16T20:05:01.439134+0000","last_clean":"2023-10-16T20:05:01.439134+0000","last_became_active":"2023-10-16T20:04:56.401019+0000","last_became_peered":"2023-10-16T20:04:56.401019+0000","last_unstale":"2023-10-16T20:05:01.439134+0000","last_undegraded":"2023-10-16T20:05:01.439134+0000","last_fullsized":"2023-10-16T20:05:01.439134+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:58:19.945263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.682734+0000","last_change":"2023-10-16T20:05:01.760505+0000","last_active":"2023-10-16T20:05:02.682734+0000","last_peered":"2023-10-16T20:05:02.682734+0000","last_clean":"2023-10-16T20:05:02.682734+0000","last_became_active":"2023-10-16T20:05:01.760043+0000","last_became_peered":"2023-10-16T20:05:01.760043+0000","last_unstale":"2023-10-16T20:05:02.682734+0000","last_undegraded":"2023-10-16T20:05:02.682734+0000","last_fullsized":"2023-10-16T20:05:02.682734+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:11:32.906879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442279+0000","last_change":"2023-10-16T20:04:57.430184+0000","last_active":"2023-10-16T20:05:01.442279+0000","last_peered":"2023-10-16T20:05:01.442279+0000","last_clean":"2023-10-16T20:05:01.442279+0000","last_became_active":"2023-10-16T20:04:57.429845+0000","last_became_peered":"2023-10-16T20:04:57.429845+0000","last_unstale":"2023-10-16T20:05:01.442279+0000","last_undegraded":"2023-10-16T20:05:01.442279+0000","last_fullsized":"2023-10-16T20:05:01.442279+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:11:28.851632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471731+0000","last_change":"2023-10-16T20:04:56.403148+0000","last_active":"2023-10-16T20:05:01.471731+0000","last_peered":"2023-10-16T20:05:01.471731+0000","last_clean":"2023-10-16T20:05:01.471731+0000","last_became_active":"2023-10-16T20:04:56.402707+0000","last_became_peered":"2023-10-16T20:04:56.402707+0000","last_unstale":"2023-10-16T20:05:01.471731+0000","last_undegraded":"2023-10-16T20:05:01.471731+0000","last_fullsized":"2023-10-16T20:05:01.471731+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:53:19.915183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.685503+0000","last_change":"2023-10-16T20:05:01.764474+0000","last_active":"2023-10-16T20:05:02.685503+0000","last_peered":"2023-10-16T20:05:02.685503+0000","last_clean":"2023-10-16T20:05:02.685503+0000","last_became_active":"2023-10-16T20:05:01.763840+0000","last_became_peered":"2023-10-16T20:05:01.763840+0000","last_unstale":"2023-10-16T20:05:02.685503+0000","last_undegraded":"2023-10-16T20:05:02.685503+0000","last_fullsized":"2023-10-16T20:05:02.685503+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:57:32.014259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471448+0000","last_change":"2023-10-16T20:04:57.423634+0000","last_active":"2023-10-16T20:05:01.471448+0000","last_peered":"2023-10-16T20:05:01.471448+0000","last_clean":"2023-10-16T20:05:01.471448+0000","last_became_active":"2023-10-16T20:04:57.423321+0000","last_became_peered":"2023-10-16T20:04:57.423321+0000","last_unstale":"2023-10-16T20:05:01.471448+0000","last_undegraded":"2023-10-16T20:05:01.471448+0000","last_fullsized":"2023-10-16T20:05:01.471448+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:23:01.141545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.758143+0000","last_change":"2023-10-16T20:04:56.408656+0000","last_active":"2023-10-16T20:05:01.758143+0000","last_peered":"2023-10-16T20:05:01.758143+0000","last_clean":"2023-10-16T20:05:01.758143+0000","last_became_active":"2023-10-16T20:04:56.408192+0000","last_became_peered":"2023-10-16T20:04:56.408192+0000","last_unstale":"2023-10-16T20:05:01.758143+0000","last_undegraded":"2023-10-16T20:05:01.758143+0000","last_fullsized":"2023-10-16T20:05:01.758143+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:54:08.029713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.688512+0000","last_change":"2023-10-16T20:05:01.767238+0000","last_active":"2023-10-16T20:05:02.688512+0000","last_peered":"2023-10-16T20:05:02.688512+0000","last_clean":"2023-10-16T20:05:02.688512+0000","last_became_active":"2023-10-16T20:05:01.765827+0000","last_became_peered":"2023-10-16T20:05:01.765827+0000","last_unstale":"2023-10-16T20:05:02.688512+0000","last_undegraded":"2023-10-16T20:05:02.688512+0000","last_fullsized":"2023-10-16T20:05:02.688512+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:14:36.535545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.440300+0000","last_change":"2023-10-16T20:04:57.430055+0000","last_active":"2023-10-16T20:05:01.440300+0000","last_peered":"2023-10-16T20:05:01.440300+0000","last_clean":"2023-10-16T20:05:01.440300+0000","last_became_active":"2023-10-16T20:04:57.428753+0000","last_became_peered":"2023-10-16T20:04:57.428753+0000","last_unstale":"2023-10-16T20:05:01.440300+0000","last_undegraded":"2023-10-16T20:05:01.440300+0000","last_fullsized":"2023-10-16T20:05:01.440300+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:14:29.143380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471458+0000","last_change":"2023-10-16T20:04:56.411374+0000","last_active":"2023-10-16T20:05:01.471458+0000","last_peered":"2023-10-16T20:05:01.471458+0000","last_clean":"2023-10-16T20:05:01.471458+0000","last_became_active":"2023-10-16T20:04:56.411098+0000","last_became_peered":"2023-10-16T20:04:56.411098+0000","last_unstale":"2023-10-16T20:05:01.471458+0000","last_undegraded":"2023-10-16T20:05:01.471458+0000","last_fullsized":"2023-10-16T20:05:01.471458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:04:32.948232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.444207+0000","last_change":"2023-10-16T20:05:01.473814+0000","last_active":"2023-10-16T20:05:02.444207+0000","last_peered":"2023-10-16T20:05:02.444207+0000","last_clean":"2023-10-16T20:05:02.444207+0000","last_became_active":"2023-10-16T20:05:01.472021+0000","last_became_peered":"2023-10-16T20:05:01.472021+0000","last_unstale":"2023-10-16T20:05:02.444207+0000","last_undegraded":"2023-10-16T20:05:02.444207+0000","last_fullsized":"2023-10-16T20:05:02.444207+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:12:00.889299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.447473+0000","last_change":"2023-10-16T20:04:57.419253+0000","last_active":"2023-10-16T20:05:01.447473+0000","last_peered":"2023-10-16T20:05:01.447473+0000","last_clean":"2023-10-16T20:05:01.447473+0000","last_became_active":"2023-10-16T20:04:57.418196+0000","last_became_peered":"2023-10-16T20:04:57.418196+0000","last_unstale":"2023-10-16T20:05:01.447473+0000","last_undegraded":"2023-10-16T20:05:01.447473+0000","last_fullsized":"2023-10-16T20:05:01.447473+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:58:20.952250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.437911+0000","last_change":"2023-10-16T20:04:56.405694+0000","last_active":"2023-10-16T20:05:01.437911+0000","last_peered":"2023-10-16T20:05:01.437911+0000","last_clean":"2023-10-16T20:05:01.437911+0000","last_became_active":"2023-10-16T20:04:56.405369+0000","last_became_peered":"2023-10-16T20:04:56.405369+0000","last_unstale":"2023-10-16T20:05:01.437911+0000","last_undegraded":"2023-10-16T20:05:01.437911+0000","last_fullsized":"2023-10-16T20:05:01.437911+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:58:32.855659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.444695+0000","last_change":"2023-10-16T20:05:01.478161+0000","last_active":"2023-10-16T20:05:02.444695+0000","last_peered":"2023-10-16T20:05:02.444695+0000","last_clean":"2023-10-16T20:05:02.444695+0000","last_became_active":"2023-10-16T20:05:01.477735+0000","last_became_peered":"2023-10-16T20:05:01.477735+0000","last_unstale":"2023-10-16T20:05:02.444695+0000","last_undegraded":"2023-10-16T20:05:02.444695+0000","last_fullsized":"2023-10-16T20:05:02.444695+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:14:33.198627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471054+0000","last_change":"2023-10-16T20:04:57.427768+0000","last_active":"2023-10-16T20:05:01.471054+0000","last_peered":"2023-10-16T20:05:01.471054+0000","last_clean":"2023-10-16T20:05:01.471054+0000","last_became_active":"2023-10-16T20:04:57.422018+0000","last_became_peered":"2023-10-16T20:04:57.422018+0000","last_unstale":"2023-10-16T20:05:01.471054+0000","last_undegraded":"2023-10-16T20:05:01.471054+0000","last_fullsized":"2023-10-16T20:05:01.471054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:49:52.409593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.448426+0000","last_change":"2023-10-16T20:04:56.402767+0000","last_active":"2023-10-16T20:05:01.448426+0000","last_peered":"2023-10-16T20:05:01.448426+0000","last_clean":"2023-10-16T20:05:01.448426+0000","last_became_active":"2023-10-16T20:04:56.402122+0000","last_became_peered":"2023-10-16T20:04:56.402122+0000","last_unstale":"2023-10-16T20:05:01.448426+0000","last_undegraded":"2023-10-16T20:05:01.448426+0000","last_fullsized":"2023-10-16T20:05:01.448426+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:54:36.775882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.445579+0000","last_change":"2023-10-16T20:05:01.478710+0000","last_active":"2023-10-16T20:05:02.445579+0000","last_peered":"2023-10-16T20:05:02.445579+0000","last_clean":"2023-10-16T20:05:02.445579+0000","last_became_active":"2023-10-16T20:05:01.478350+0000","last_became_peered":"2023-10-16T20:05:01.478350+0000","last_unstale":"2023-10-16T20:05:02.445579+0000","last_undegraded":"2023-10-16T20:05:02.445579+0000","last_fullsized":"2023-10-16T20:05:02.445579+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:57:32.014259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.459302+0000","last_change":"2023-10-16T20:04:57.420638+0000","last_active":"2023-10-16T20:05:01.459302+0000","last_peered":"2023-10-16T20:05:01.459302+0000","last_clean":"2023-10-16T20:05:01.459302+0000","last_became_active":"2023-10-16T20:04:57.420348+0000","last_became_peered":"2023-10-16T20:04:57.420348+0000","last_unstale":"2023-10-16T20:05:01.459302+0000","last_undegraded":"2023-10-16T20:05:01.459302+0000","last_fullsized":"2023-10-16T20:05:01.459302+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:14:18.231180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757558+0000","last_change":"2023-10-16T20:04:56.403488+0000","last_active":"2023-10-16T20:05:01.757558+0000","last_peered":"2023-10-16T20:05:01.757558+0000","last_clean":"2023-10-16T20:05:01.757558+0000","last_became_active":"2023-10-16T20:04:56.403156+0000","last_became_peered":"2023-10-16T20:04:56.403156+0000","last_unstale":"2023-10-16T20:05:01.757558+0000","last_undegraded":"2023-10-16T20:05:01.757558+0000","last_fullsized":"2023-10-16T20:05:01.757558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:33:35.395756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.688883+0000","last_change":"2023-10-16T20:05:01.765108+0000","last_active":"2023-10-16T20:05:02.688883+0000","last_peered":"2023-10-16T20:05:02.688883+0000","last_clean":"2023-10-16T20:05:02.688883+0000","last_became_active":"2023-10-16T20:05:01.764436+0000","last_became_peered":"2023-10-16T20:05:01.764436+0000","last_unstale":"2023-10-16T20:05:02.688883+0000","last_undegraded":"2023-10-16T20:05:02.688883+0000","last_fullsized":"2023-10-16T20:05:02.688883+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:39:29.554873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442360+0000","last_change":"2023-10-16T20:04:57.429060+0000","last_active":"2023-10-16T20:05:01.442360+0000","last_peered":"2023-10-16T20:05:01.442360+0000","last_clean":"2023-10-16T20:05:01.442360+0000","last_became_active":"2023-10-16T20:04:57.428756+0000","last_became_peered":"2023-10-16T20:04:57.428756+0000","last_unstale":"2023-10-16T20:05:01.442360+0000","last_undegraded":"2023-10-16T20:05:01.442360+0000","last_fullsized":"2023-10-16T20:05:01.442360+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:21:09.387890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.437753+0000","last_change":"2023-10-16T20:04:56.392274+0000","last_active":"2023-10-16T20:05:01.437753+0000","last_peered":"2023-10-16T20:05:01.437753+0000","last_clean":"2023-10-16T20:05:01.437753+0000","last_became_active":"2023-10-16T20:04:56.391958+0000","last_became_peered":"2023-10-16T20:04:56.391958+0000","last_unstale":"2023-10-16T20:05:01.437753+0000","last_undegraded":"2023-10-16T20:05:01.437753+0000","last_fullsized":"2023-10-16T20:05:01.437753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:58:19.945263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":169,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757620+0000","last_change":"2023-10-16T20:04:46.335822+0000","last_active":"2023-10-16T20:05:01.757620+0000","last_peered":"2023-10-16T20:05:01.757620+0000","last_clean":"2023-10-16T20:05:01.757620+0000","last_became_active":"2023-10-16T20:04:46.334617+0000","last_became_peered":"2023-10-16T20:04:46.334617+0000","last_unstale":"2023-10-16T20:05:01.757620+0000","last_undegraded":"2023-10-16T20:05:01.757620+0000","last_fullsized":"2023-10-16T20:05:01.757620+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:45.306240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:45.306240+0000","last_clean_scrub_stamp":"2023-10-16T20:04:45.306240+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:42:11.153721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.682739+0000","last_change":"2023-10-16T20:05:01.760533+0000","last_active":"2023-10-16T20:05:02.682739+0000","last_peered":"2023-10-16T20:05:02.682739+0000","last_clean":"2023-10-16T20:05:02.682739+0000","last_became_active":"2023-10-16T20:05:01.760091+0000","last_became_peered":"2023-10-16T20:05:01.760091+0000","last_unstale":"2023-10-16T20:05:02.682739+0000","last_undegraded":"2023-10-16T20:05:02.682739+0000","last_fullsized":"2023-10-16T20:05:02.682739+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:22:16.748229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757962+0000","last_change":"2023-10-16T20:04:57.435770+0000","last_active":"2023-10-16T20:05:01.757962+0000","last_peered":"2023-10-16T20:05:01.757962+0000","last_clean":"2023-10-16T20:05:01.757962+0000","last_became_active":"2023-10-16T20:04:57.434914+0000","last_became_peered":"2023-10-16T20:04:57.434914+0000","last_unstale":"2023-10-16T20:05:01.757962+0000","last_undegraded":"2023-10-16T20:05:01.757962+0000","last_fullsized":"2023-10-16T20:05:01.757962+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:42:35.216191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.449802+0000","last_change":"2023-10-16T20:04:56.414794+0000","last_active":"2023-10-16T20:05:01.449802+0000","last_peered":"2023-10-16T20:05:01.449802+0000","last_clean":"2023-10-16T20:05:01.449802+0000","last_became_active":"2023-10-16T20:04:56.414400+0000","last_became_peered":"2023-10-16T20:04:56.414400+0000","last_unstale":"2023-10-16T20:05:01.449802+0000","last_undegraded":"2023-10-16T20:05:01.449802+0000","last_fullsized":"2023-10-16T20:05:01.449802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:19:48.110438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.682395+0000","last_change":"2023-10-16T20:05:01.760979+0000","last_active":"2023-10-16T20:05:02.682395+0000","last_peered":"2023-10-16T20:05:02.682395+0000","last_clean":"2023-10-16T20:05:02.682395+0000","last_became_active":"2023-10-16T20:05:01.760141+0000","last_became_peered":"2023-10-16T20:05:01.760141+0000","last_unstale":"2023-10-16T20:05:02.682395+0000","last_undegraded":"2023-10-16T20:05:02.682395+0000","last_fullsized":"2023-10-16T20:05:02.682395+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:21:13.443137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438336+0000","last_change":"2023-10-16T20:04:57.429698+0000","last_active":"2023-10-16T20:05:01.438336+0000","last_peered":"2023-10-16T20:05:01.438336+0000","last_clean":"2023-10-16T20:05:01.438336+0000","last_became_active":"2023-10-16T20:04:57.428050+0000","last_became_peered":"2023-10-16T20:04:57.428050+0000","last_unstale":"2023-10-16T20:05:01.438336+0000","last_undegraded":"2023-10-16T20:05:01.438336+0000","last_fullsized":"2023-10-16T20:05:01.438336+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:42:35.216191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471736+0000","last_change":"2023-10-16T20:04:56.409587+0000","last_active":"2023-10-16T20:05:01.471736+0000","last_peered":"2023-10-16T20:05:01.471736+0000","last_clean":"2023-10-16T20:05:01.471736+0000","last_became_active":"2023-10-16T20:04:56.408150+0000","last_became_peered":"2023-10-16T20:04:56.408150+0000","last_unstale":"2023-10-16T20:05:01.471736+0000","last_undegraded":"2023-10-16T20:05:01.471736+0000","last_fullsized":"2023-10-16T20:05:01.471736+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:13:40.552947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.688655+0000","last_change":"2023-10-16T20:05:01.476280+0000","last_active":"2023-10-16T20:05:02.688655+0000","last_peered":"2023-10-16T20:05:02.688655+0000","last_clean":"2023-10-16T20:05:02.688655+0000","last_became_active":"2023-10-16T20:05:01.475885+0000","last_became_peered":"2023-10-16T20:05:01.475885+0000","last_unstale":"2023-10-16T20:05:02.688655+0000","last_undegraded":"2023-10-16T20:05:02.688655+0000","last_fullsized":"2023-10-16T20:05:02.688655+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:01:16.289925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.758343+0000","last_change":"2023-10-16T20:04:57.442292+0000","last_active":"2023-10-16T20:05:01.758343+0000","last_peered":"2023-10-16T20:05:01.758343+0000","last_clean":"2023-10-16T20:05:01.758343+0000","last_became_active":"2023-10-16T20:04:57.439075+0000","last_became_peered":"2023-10-16T20:04:57.439075+0000","last_unstale":"2023-10-16T20:05:01.758343+0000","last_undegraded":"2023-10-16T20:05:01.758343+0000","last_fullsized":"2023-10-16T20:05:01.758343+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:25:46.207669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.448189+0000","last_change":"2023-10-16T20:04:56.398007+0000","last_active":"2023-10-16T20:05:01.448189+0000","last_peered":"2023-10-16T20:05:01.448189+0000","last_clean":"2023-10-16T20:05:01.448189+0000","last_became_active":"2023-10-16T20:04:56.397108+0000","last_became_peered":"2023-10-16T20:04:56.397108+0000","last_unstale":"2023-10-16T20:05:01.448189+0000","last_undegraded":"2023-10-16T20:05:01.448189+0000","last_fullsized":"2023-10-16T20:05:01.448189+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:24:16.965233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.684215+0000","last_change":"2023-10-16T20:05:01.763390+0000","last_active":"2023-10-16T20:05:02.684215+0000","last_peered":"2023-10-16T20:05:02.684215+0000","last_clean":"2023-10-16T20:05:02.684215+0000","last_became_active":"2023-10-16T20:05:01.760077+0000","last_became_peered":"2023-10-16T20:05:01.760077+0000","last_unstale":"2023-10-16T20:05:02.684215+0000","last_undegraded":"2023-10-16T20:05:02.684215+0000","last_fullsized":"2023-10-16T20:05:02.684215+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:10:33.807776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.758160+0000","last_change":"2023-10-16T20:04:57.437968+0000","last_active":"2023-10-16T20:05:01.758160+0000","last_peered":"2023-10-16T20:05:01.758160+0000","last_clean":"2023-10-16T20:05:01.758160+0000","last_became_active":"2023-10-16T20:04:57.437674+0000","last_became_peered":"2023-10-16T20:04:57.437674+0000","last_unstale":"2023-10-16T20:05:01.758160+0000","last_undegraded":"2023-10-16T20:05:01.758160+0000","last_fullsized":"2023-10-16T20:05:01.758160+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:01:38.242317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.447078+0000","last_change":"2023-10-16T20:04:56.408227+0000","last_active":"2023-10-16T20:05:01.447078+0000","last_peered":"2023-10-16T20:05:01.447078+0000","last_clean":"2023-10-16T20:05:01.447078+0000","last_became_active":"2023-10-16T20:04:56.407279+0000","last_became_peered":"2023-10-16T20:04:56.407279+0000","last_unstale":"2023-10-16T20:05:01.447078+0000","last_undegraded":"2023-10-16T20:05:01.447078+0000","last_fullsized":"2023-10-16T20:05:01.447078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:43:11.864176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.682415+0000","last_change":"2023-10-16T20:05:01.477611+0000","last_active":"2023-10-16T20:05:02.682415+0000","last_peered":"2023-10-16T20:05:02.682415+0000","last_clean":"2023-10-16T20:05:02.682415+0000","last_became_active":"2023-10-16T20:05:01.477172+0000","last_became_peered":"2023-10-16T20:05:01.477172+0000","last_unstale":"2023-10-16T20:05:02.682415+0000","last_undegraded":"2023-10-16T20:05:02.682415+0000","last_fullsized":"2023-10-16T20:05:02.682415+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:23:05.196792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757700+0000","last_change":"2023-10-16T20:04:57.438300+0000","last_active":"2023-10-16T20:05:01.757700+0000","last_peered":"2023-10-16T20:05:01.757700+0000","last_clean":"2023-10-16T20:05:01.757700+0000","last_became_active":"2023-10-16T20:04:57.437951+0000","last_became_peered":"2023-10-16T20:04:57.437951+0000","last_unstale":"2023-10-16T20:05:01.757700+0000","last_undegraded":"2023-10-16T20:05:01.757700+0000","last_fullsized":"2023-10-16T20:05:01.757700+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:31:34.884161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.448376+0000","last_change":"2023-10-16T20:04:56.401341+0000","last_active":"2023-10-16T20:05:01.448376+0000","last_peered":"2023-10-16T20:05:01.448376+0000","last_clean":"2023-10-16T20:05:01.448376+0000","last_became_active":"2023-10-16T20:04:56.401058+0000","last_became_peered":"2023-10-16T20:04:56.401058+0000","last_unstale":"2023-10-16T20:05:01.448376+0000","last_undegraded":"2023-10-16T20:05:01.448376+0000","last_fullsized":"2023-10-16T20:05:01.448376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:41:34.379170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.688808+0000","last_change":"2023-10-16T20:05:01.463304+0000","last_active":"2023-10-16T20:05:02.688808+0000","last_peered":"2023-10-16T20:05:02.688808+0000","last_clean":"2023-10-16T20:05:02.688808+0000","last_became_active":"2023-10-16T20:05:01.462044+0000","last_became_peered":"2023-10-16T20:05:01.462044+0000","last_unstale":"2023-10-16T20:05:02.688808+0000","last_undegraded":"2023-10-16T20:05:02.688808+0000","last_fullsized":"2023-10-16T20:05:02.688808+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:41:12.320369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.440492+0000","last_change":"2023-10-16T20:04:57.409823+0000","last_active":"2023-10-16T20:05:01.440492+0000","last_peered":"2023-10-16T20:05:01.440492+0000","last_clean":"2023-10-16T20:05:01.440492+0000","last_became_active":"2023-10-16T20:04:57.409457+0000","last_became_peered":"2023-10-16T20:04:57.409457+0000","last_unstale":"2023-10-16T20:05:01.440492+0000","last_undegraded":"2023-10-16T20:05:01.440492+0000","last_fullsized":"2023-10-16T20:05:01.440492+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:53:20.922170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.440306+0000","last_change":"2023-10-16T20:04:56.404854+0000","last_active":"2023-10-16T20:05:01.440306+0000","last_peered":"2023-10-16T20:05:01.440306+0000","last_clean":"2023-10-16T20:05:01.440306+0000","last_became_active":"2023-10-16T20:04:56.404457+0000","last_became_peered":"2023-10-16T20:04:56.404457+0000","last_unstale":"2023-10-16T20:05:01.440306+0000","last_undegraded":"2023-10-16T20:05:01.440306+0000","last_fullsized":"2023-10-16T20:05:01.440306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:41:00.246498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.686341+0000","last_change":"2023-10-16T20:05:01.769521+0000","last_active":"2023-10-16T20:05:02.686341+0000","last_peered":"2023-10-16T20:05:02.686341+0000","last_clean":"2023-10-16T20:05:02.686341+0000","last_became_active":"2023-10-16T20:05:01.768728+0000","last_became_peered":"2023-10-16T20:05:01.768728+0000","last_unstale":"2023-10-16T20:05:02.686341+0000","last_undegraded":"2023-10-16T20:05:02.686341+0000","last_fullsized":"2023-10-16T20:05:02.686341+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:01:55.141797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757851+0000","last_change":"2023-10-16T20:04:57.438646+0000","last_active":"2023-10-16T20:05:01.757851+0000","last_peered":"2023-10-16T20:05:01.757851+0000","last_clean":"2023-10-16T20:05:01.757851+0000","last_became_active":"2023-10-16T20:04:57.438375+0000","last_became_peered":"2023-10-16T20:04:57.438375+0000","last_unstale":"2023-10-16T20:05:01.757851+0000","last_undegraded":"2023-10-16T20:05:01.757851+0000","last_fullsized":"2023-10-16T20:05:01.757851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:14:29.143380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.437845+0000","last_change":"2023-10-16T20:04:56.407281+0000","last_active":"2023-10-16T20:05:01.437845+0000","last_peered":"2023-10-16T20:05:01.437845+0000","last_clean":"2023-10-16T20:05:01.437845+0000","last_became_active":"2023-10-16T20:04:56.406824+0000","last_became_peered":"2023-10-16T20:04:56.406824+0000","last_unstale":"2023-10-16T20:05:01.437845+0000","last_undegraded":"2023-10-16T20:05:01.437845+0000","last_fullsized":"2023-10-16T20:05:01.437845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:23:00.134558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.686002+0000","last_change":"2023-10-16T20:05:01.764372+0000","last_active":"2023-10-16T20:05:02.686002+0000","last_peered":"2023-10-16T20:05:02.686002+0000","last_clean":"2023-10-16T20:05:02.686002+0000","last_became_active":"2023-10-16T20:05:01.763918+0000","last_became_peered":"2023-10-16T20:05:01.763918+0000","last_unstale":"2023-10-16T20:05:02.686002+0000","last_undegraded":"2023-10-16T20:05:02.686002+0000","last_fullsized":"2023-10-16T20:05:02.686002+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:10:33.807776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.463201+0000","last_change":"2023-10-16T20:04:57.425920+0000","last_active":"2023-10-16T20:05:01.463201+0000","last_peered":"2023-10-16T20:05:01.463201+0000","last_clean":"2023-10-16T20:05:01.463201+0000","last_became_active":"2023-10-16T20:04:57.425614+0000","last_became_peered":"2023-10-16T20:04:57.425614+0000","last_unstale":"2023-10-16T20:05:01.463201+0000","last_undegraded":"2023-10-16T20:05:01.463201+0000","last_fullsized":"2023-10-16T20:05:01.463201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:28:14.992492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.464594+0000","last_change":"2023-10-16T20:04:56.407990+0000","last_active":"2023-10-16T20:05:01.464594+0000","last_peered":"2023-10-16T20:05:01.464594+0000","last_clean":"2023-10-16T20:05:01.464594+0000","last_became_active":"2023-10-16T20:04:56.407682+0000","last_became_peered":"2023-10-16T20:04:56.407682+0000","last_unstale":"2023-10-16T20:05:01.464594+0000","last_undegraded":"2023-10-16T20:05:01.464594+0000","last_fullsized":"2023-10-16T20:05:01.464594+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:25:27.337575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.685081+0000","last_change":"2023-10-16T20:05:01.761327+0000","last_active":"2023-10-16T20:05:02.685081+0000","last_peered":"2023-10-16T20:05:02.685081+0000","last_clean":"2023-10-16T20:05:02.685081+0000","last_became_active":"2023-10-16T20:05:01.760040+0000","last_became_peered":"2023-10-16T20:05:01.760040+0000","last_unstale":"2023-10-16T20:05:02.685081+0000","last_undegraded":"2023-10-16T20:05:02.685081+0000","last_fullsized":"2023-10-16T20:05:02.685081+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:33:23.371729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757841+0000","last_change":"2023-10-16T20:04:57.437602+0000","last_active":"2023-10-16T20:05:01.757841+0000","last_peered":"2023-10-16T20:05:01.757841+0000","last_clean":"2023-10-16T20:05:01.757841+0000","last_became_active":"2023-10-16T20:04:57.436601+0000","last_became_peered":"2023-10-16T20:04:57.436601+0000","last_unstale":"2023-10-16T20:05:01.757841+0000","last_undegraded":"2023-10-16T20:05:01.757841+0000","last_fullsized":"2023-10-16T20:05:01.757841+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:28:00.096002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'9","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:08.020486+0000","last_change":"2023-10-16T20:04:56.398743+0000","last_active":"2023-10-16T20:05:08.020486+0000","last_peered":"2023-10-16T20:05:08.020486+0000","last_clean":"2023-10-16T20:05:08.020486+0000","last_became_active":"2023-10-16T20:04:56.398410+0000","last_became_peered":"2023-10-16T20:04:56.398410+0000","last_unstale":"2023-10-16T20:05:08.020486+0000","last_undegraded":"2023-10-16T20:05:08.020486+0000","last_fullsized":"2023-10-16T20:05:08.020486+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:21:13.046934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.444394+0000","last_change":"2023-10-16T20:05:01.472414+0000","last_active":"2023-10-16T20:05:02.444394+0000","last_peered":"2023-10-16T20:05:02.444394+0000","last_clean":"2023-10-16T20:05:02.444394+0000","last_became_active":"2023-10-16T20:05:01.471943+0000","last_became_peered":"2023-10-16T20:05:01.471943+0000","last_unstale":"2023-10-16T20:05:02.444394+0000","last_undegraded":"2023-10-16T20:05:02.444394+0000","last_fullsized":"2023-10-16T20:05:02.444394+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:42:39.271438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.447164+0000","last_change":"2023-10-16T20:04:57.413204+0000","last_active":"2023-10-16T20:05:01.447164+0000","last_peered":"2023-10-16T20:05:01.447164+0000","last_clean":"2023-10-16T20:05:01.447164+0000","last_became_active":"2023-10-16T20:04:57.412836+0000","last_became_peered":"2023-10-16T20:04:57.412836+0000","last_unstale":"2023-10-16T20:05:01.447164+0000","last_undegraded":"2023-10-16T20:05:01.447164+0000","last_fullsized":"2023-10-16T20:05:01.447164+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:23:06.051449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471656+0000","last_change":"2023-10-16T20:04:56.414671+0000","last_active":"2023-10-16T20:05:01.471656+0000","last_peered":"2023-10-16T20:05:01.471656+0000","last_clean":"2023-10-16T20:05:01.471656+0000","last_became_active":"2023-10-16T20:04:56.414339+0000","last_became_peered":"2023-10-16T20:04:56.414339+0000","last_unstale":"2023-10-16T20:05:01.471656+0000","last_undegraded":"2023-10-16T20:05:01.471656+0000","last_fullsized":"2023-10-16T20:05:01.471656+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:21:13.046934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.683018+0000","last_change":"2023-10-16T20:05:01.482674+0000","last_active":"2023-10-16T20:05:02.683018+0000","last_peered":"2023-10-16T20:05:02.683018+0000","last_clean":"2023-10-16T20:05:02.683018+0000","last_became_active":"2023-10-16T20:05:01.482048+0000","last_became_peered":"2023-10-16T20:05:01.482048+0000","last_unstale":"2023-10-16T20:05:02.683018+0000","last_undegraded":"2023-10-16T20:05:02.683018+0000","last_fullsized":"2023-10-16T20:05:02.683018+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:21:13.443137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.465807+0000","last_change":"2023-10-16T20:04:57.430429+0000","last_active":"2023-10-16T20:05:01.465807+0000","last_peered":"2023-10-16T20:05:01.465807+0000","last_clean":"2023-10-16T20:05:01.465807+0000","last_became_active":"2023-10-16T20:04:57.429872+0000","last_became_peered":"2023-10-16T20:04:57.429872+0000","last_unstale":"2023-10-16T20:05:01.465807+0000","last_undegraded":"2023-10-16T20:05:01.465807+0000","last_fullsized":"2023-10-16T20:05:01.465807+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:47:33.537999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438017+0000","last_change":"2023-10-16T20:04:56.398061+0000","last_active":"2023-10-16T20:05:01.438017+0000","last_peered":"2023-10-16T20:05:01.438017+0000","last_clean":"2023-10-16T20:05:01.438017+0000","last_became_active":"2023-10-16T20:04:56.397712+0000","last_became_peered":"2023-10-16T20:04:56.397712+0000","last_unstale":"2023-10-16T20:05:01.438017+0000","last_undegraded":"2023-10-16T20:05:01.438017+0000","last_fullsized":"2023-10-16T20:05:01.438017+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:49:51.402606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.685658+0000","last_change":"2023-10-16T20:05:01.476635+0000","last_active":"2023-10-16T20:05:02.685658+0000","last_peered":"2023-10-16T20:05:02.685658+0000","last_clean":"2023-10-16T20:05:02.685658+0000","last_became_active":"2023-10-16T20:05:01.476053+0000","last_became_peered":"2023-10-16T20:05:01.476053+0000","last_unstale":"2023-10-16T20:05:02.685658+0000","last_undegraded":"2023-10-16T20:05:02.685658+0000","last_fullsized":"2023-10-16T20:05:02.685658+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:25:50.262916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.437931+0000","last_change":"2023-10-16T20:04:57.421476+0000","last_active":"2023-10-16T20:05:01.437931+0000","last_peered":"2023-10-16T20:05:01.437931+0000","last_clean":"2023-10-16T20:05:01.437931+0000","last_became_active":"2023-10-16T20:04:57.421206+0000","last_became_peered":"2023-10-16T20:04:57.421206+0000","last_unstale":"2023-10-16T20:05:01.437931+0000","last_undegraded":"2023-10-16T20:05:01.437931+0000","last_fullsized":"2023-10-16T20:05:01.437931+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:11:56.834052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.439019+0000","last_change":"2023-10-16T20:04:56.411232+0000","last_active":"2023-10-16T20:05:01.439019+0000","last_peered":"2023-10-16T20:05:01.439019+0000","last_clean":"2023-10-16T20:05:01.439019+0000","last_became_active":"2023-10-16T20:04:56.410793+0000","last_became_peered":"2023-10-16T20:04:56.410793+0000","last_unstale":"2023-10-16T20:05:01.439019+0000","last_undegraded":"2023-10-16T20:05:01.439019+0000","last_fullsized":"2023-10-16T20:05:01.439019+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:19:48.110438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.685647+0000","last_change":"2023-10-16T20:05:01.764968+0000","last_active":"2023-10-16T20:05:02.685647+0000","last_peered":"2023-10-16T20:05:02.685647+0000","last_clean":"2023-10-16T20:05:02.685647+0000","last_became_active":"2023-10-16T20:05:01.763830+0000","last_became_peered":"2023-10-16T20:05:01.763830+0000","last_unstale":"2023-10-16T20:05:02.685647+0000","last_undegraded":"2023-10-16T20:05:02.685647+0000","last_fullsized":"2023-10-16T20:05:02.685647+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:39:58.794103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.688809+0000","last_change":"2023-10-16T20:05:01.764846+0000","last_active":"2023-10-16T20:05:02.688809+0000","last_peered":"2023-10-16T20:05:02.688809+0000","last_clean":"2023-10-16T20:05:02.688809+0000","last_became_active":"2023-10-16T20:05:01.763756+0000","last_became_peered":"2023-10-16T20:05:01.763756+0000","last_unstale":"2023-10-16T20:05:02.688809+0000","last_undegraded":"2023-10-16T20:05:02.688809+0000","last_fullsized":"2023-10-16T20:05:02.688809+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:10:31.962156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471094+0000","last_change":"2023-10-16T20:04:57.428882+0000","last_active":"2023-10-16T20:05:01.471094+0000","last_peered":"2023-10-16T20:05:01.471094+0000","last_clean":"2023-10-16T20:05:01.471094+0000","last_became_active":"2023-10-16T20:04:57.428591+0000","last_became_peered":"2023-10-16T20:04:57.428591+0000","last_unstale":"2023-10-16T20:05:01.471094+0000","last_undegraded":"2023-10-16T20:05:01.471094+0000","last_fullsized":"2023-10-16T20:05:01.471094+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:33:36.402743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438343+0000","last_change":"2023-10-16T20:04:56.396769+0000","last_active":"2023-10-16T20:05:01.438343+0000","last_peered":"2023-10-16T20:05:01.438343+0000","last_clean":"2023-10-16T20:05:01.438343+0000","last_became_active":"2023-10-16T20:04:56.396273+0000","last_became_peered":"2023-10-16T20:04:56.396273+0000","last_unstale":"2023-10-16T20:05:01.438343+0000","last_undegraded":"2023-10-16T20:05:01.438343+0000","last_fullsized":"2023-10-16T20:05:01.438343+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:04:32.948232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.683891+0000","last_change":"2023-10-16T20:05:01.475065+0000","last_active":"2023-10-16T20:05:02.683891+0000","last_peered":"2023-10-16T20:05:02.683891+0000","last_clean":"2023-10-16T20:05:02.683891+0000","last_became_active":"2023-10-16T20:05:01.474571+0000","last_became_peered":"2023-10-16T20:05:01.474571+0000","last_unstale":"2023-10-16T20:05:02.683891+0000","last_undegraded":"2023-10-16T20:05:02.683891+0000","last_fullsized":"2023-10-16T20:05:02.683891+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:31:38.939408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442005+0000","last_change":"2023-10-16T20:04:57.423943+0000","last_active":"2023-10-16T20:05:01.442005+0000","last_peered":"2023-10-16T20:05:01.442005+0000","last_clean":"2023-10-16T20:05:01.442005+0000","last_became_active":"2023-10-16T20:04:57.423325+0000","last_became_peered":"2023-10-16T20:04:57.423325+0000","last_unstale":"2023-10-16T20:05:01.442005+0000","last_undegraded":"2023-10-16T20:05:01.442005+0000","last_fullsized":"2023-10-16T20:05:01.442005+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:56:28.993951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442942+0000","last_change":"2023-10-16T20:04:56.397549+0000","last_active":"2023-10-16T20:05:01.442942+0000","last_peered":"2023-10-16T20:05:01.442942+0000","last_clean":"2023-10-16T20:05:01.442942+0000","last_became_active":"2023-10-16T20:04:56.396648+0000","last_became_peered":"2023-10-16T20:04:56.396648+0000","last_unstale":"2023-10-16T20:05:01.442942+0000","last_undegraded":"2023-10-16T20:05:01.442942+0000","last_fullsized":"2023-10-16T20:05:01.442942+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:58:19.945263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438803+0000","last_change":"2023-10-16T20:04:56.407155+0000","last_active":"2023-10-16T20:05:01.438803+0000","last_peered":"2023-10-16T20:05:01.438803+0000","last_clean":"2023-10-16T20:05:01.438803+0000","last_became_active":"2023-10-16T20:04:56.406828+0000","last_became_peered":"2023-10-16T20:04:56.406828+0000","last_unstale":"2023-10-16T20:05:01.438803+0000","last_undegraded":"2023-10-16T20:05:01.438803+0000","last_fullsized":"2023-10-16T20:05:01.438803+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:23:05.044462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.464536+0000","last_change":"2023-10-16T20:04:57.416841+0000","last_active":"2023-10-16T20:05:01.464536+0000","last_peered":"2023-10-16T20:05:01.464536+0000","last_clean":"2023-10-16T20:05:01.464536+0000","last_became_active":"2023-10-16T20:04:57.416338+0000","last_became_peered":"2023-10-16T20:04:57.416338+0000","last_unstale":"2023-10-16T20:05:01.464536+0000","last_undegraded":"2023-10-16T20:05:01.464536+0000","last_fullsized":"2023-10-16T20:05:01.464536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:01.396735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.445821+0000","last_change":"2023-10-16T20:05:01.481039+0000","last_active":"2023-10-16T20:05:02.445821+0000","last_peered":"2023-10-16T20:05:02.445821+0000","last_clean":"2023-10-16T20:05:02.445821+0000","last_became_active":"2023-10-16T20:05:01.480626+0000","last_became_peered":"2023-10-16T20:05:01.480626+0000","last_unstale":"2023-10-16T20:05:02.445821+0000","last_undegraded":"2023-10-16T20:05:02.445821+0000","last_fullsized":"2023-10-16T20:05:02.445821+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:21:18.109168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.441938+0000","last_change":"2023-10-16T20:04:56.401545+0000","last_active":"2023-10-16T20:05:01.441938+0000","last_peered":"2023-10-16T20:05:01.441938+0000","last_clean":"2023-10-16T20:05:01.441938+0000","last_became_active":"2023-10-16T20:04:56.401276+0000","last_became_peered":"2023-10-16T20:04:56.401276+0000","last_unstale":"2023-10-16T20:05:01.441938+0000","last_undegraded":"2023-10-16T20:05:01.441938+0000","last_fullsized":"2023-10-16T20:05:01.441938+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:04:32.948232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.447134+0000","last_change":"2023-10-16T20:04:57.425600+0000","last_active":"2023-10-16T20:05:01.447134+0000","last_peered":"2023-10-16T20:05:01.447134+0000","last_clean":"2023-10-16T20:05:01.447134+0000","last_became_active":"2023-10-16T20:04:57.425213+0000","last_became_peered":"2023-10-16T20:04:57.425213+0000","last_unstale":"2023-10-16T20:05:01.447134+0000","last_undegraded":"2023-10-16T20:05:01.447134+0000","last_fullsized":"2023-10-16T20:05:01.447134+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:33:14.623281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.686142+0000","last_change":"2023-10-16T20:05:01.479226+0000","last_active":"2023-10-16T20:05:02.686142+0000","last_peered":"2023-10-16T20:05:02.686142+0000","last_clean":"2023-10-16T20:05:02.686142+0000","last_became_active":"2023-10-16T20:05:01.478808+0000","last_became_peered":"2023-10-16T20:05:01.478808+0000","last_unstale":"2023-10-16T20:05:02.686142+0000","last_undegraded":"2023-10-16T20:05:02.686142+0000","last_fullsized":"2023-10-16T20:05:02.686142+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:33:23.371729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.440406+0000","last_change":"2023-10-16T20:04:57.409772+0000","last_active":"2023-10-16T20:05:01.440406+0000","last_peered":"2023-10-16T20:05:01.440406+0000","last_clean":"2023-10-16T20:05:01.440406+0000","last_became_active":"2023-10-16T20:04:57.409449+0000","last_became_peered":"2023-10-16T20:04:57.409449+0000","last_unstale":"2023-10-16T20:05:01.440406+0000","last_undegraded":"2023-10-16T20:05:01.440406+0000","last_fullsized":"2023-10-16T20:05:01.440406+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:13:41.559934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.448190+0000","last_change":"2023-10-16T20:04:57.418433+0000","last_active":"2023-10-16T20:05:01.448190+0000","last_peered":"2023-10-16T20:05:01.448190+0000","last_clean":"2023-10-16T20:05:01.448190+0000","last_became_active":"2023-10-16T20:04:57.418140+0000","last_became_peered":"2023-10-16T20:04:57.418140+0000","last_unstale":"2023-10-16T20:05:01.448190+0000","last_undegraded":"2023-10-16T20:05:01.448190+0000","last_fullsized":"2023-10-16T20:05:01.448190+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:41:01.253485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.437811+0000","last_change":"2023-10-16T20:04:57.430273+0000","last_active":"2023-10-16T20:05:01.437811+0000","last_peered":"2023-10-16T20:05:01.437811+0000","last_clean":"2023-10-16T20:05:01.437811+0000","last_became_active":"2023-10-16T20:04:57.427616+0000","last_became_peered":"2023-10-16T20:04:57.427616+0000","last_unstale":"2023-10-16T20:05:01.437811+0000","last_undegraded":"2023-10-16T20:05:01.437811+0000","last_fullsized":"2023-10-16T20:05:01.437811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:22:12.692981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.464484+0000","last_change":"2023-10-16T20:04:57.434154+0000","last_active":"2023-10-16T20:05:01.464484+0000","last_peered":"2023-10-16T20:05:01.464484+0000","last_clean":"2023-10-16T20:05:01.464484+0000","last_became_active":"2023-10-16T20:04:57.433876+0000","last_became_peered":"2023-10-16T20:04:57.433876+0000","last_unstale":"2023-10-16T20:05:01.464484+0000","last_undegraded":"2023-10-16T20:05:01.464484+0000","last_fullsized":"2023-10-16T20:05:01.464484+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:45:41.399175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.443384+0000","last_change":"2023-10-16T20:04:57.437575+0000","last_active":"2023-10-16T20:05:01.443384+0000","last_peered":"2023-10-16T20:05:01.443384+0000","last_clean":"2023-10-16T20:05:01.443384+0000","last_became_active":"2023-10-16T20:04:57.437261+0000","last_became_peered":"2023-10-16T20:04:57.437261+0000","last_unstale":"2023-10-16T20:05:01.443384+0000","last_undegraded":"2023-10-16T20:05:01.443384+0000","last_fullsized":"2023-10-16T20:05:01.443384+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:58:33.862646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471548+0000","last_change":"2023-10-16T20:04:57.434048+0000","last_active":"2023-10-16T20:05:01.471548+0000","last_peered":"2023-10-16T20:05:01.471548+0000","last_clean":"2023-10-16T20:05:01.471548+0000","last_became_active":"2023-10-16T20:04:57.433771+0000","last_became_peered":"2023-10-16T20:04:57.433771+0000","last_unstale":"2023-10-16T20:05:01.471548+0000","last_undegraded":"2023-10-16T20:05:01.471548+0000","last_fullsized":"2023-10-16T20:05:01.471548+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:58:33.862646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.440121+0000","last_change":"2023-10-16T20:04:57.439787+0000","last_active":"2023-10-16T20:05:01.440121+0000","last_peered":"2023-10-16T20:05:01.440121+0000","last_clean":"2023-10-16T20:05:01.440121+0000","last_became_active":"2023-10-16T20:04:57.439468+0000","last_became_peered":"2023-10-16T20:04:57.439468+0000","last_unstale":"2023-10-16T20:05:01.440121+0000","last_undegraded":"2023-10-16T20:05:01.440121+0000","last_fullsized":"2023-10-16T20:05:01.440121+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:49:52.409593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442167+0000","last_change":"2023-10-16T20:04:57.421442+0000","last_active":"2023-10-16T20:05:01.442167+0000","last_peered":"2023-10-16T20:05:01.442167+0000","last_clean":"2023-10-16T20:05:01.442167+0000","last_became_active":"2023-10-16T20:04:57.421123+0000","last_became_peered":"2023-10-16T20:04:57.421123+0000","last_unstale":"2023-10-16T20:05:01.442167+0000","last_undegraded":"2023-10-16T20:05:01.442167+0000","last_fullsized":"2023-10-16T20:05:01.442167+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:17:24.205223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438259+0000","last_change":"2023-10-16T20:04:57.412843+0000","last_active":"2023-10-16T20:05:01.438259+0000","last_peered":"2023-10-16T20:05:01.438259+0000","last_clean":"2023-10-16T20:05:01.438259+0000","last_became_active":"2023-10-16T20:04:57.412559+0000","last_became_peered":"2023-10-16T20:04:57.412559+0000","last_unstale":"2023-10-16T20:05:01.438259+0000","last_undegraded":"2023-10-16T20:05:01.438259+0000","last_fullsized":"2023-10-16T20:05:01.438259+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:11:28.851632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442830+0000","last_change":"2023-10-16T20:04:57.421462+0000","last_active":"2023-10-16T20:05:01.442830+0000","last_peered":"2023-10-16T20:05:01.442830+0000","last_clean":"2023-10-16T20:05:01.442830+0000","last_became_active":"2023-10-16T20:04:57.421107+0000","last_became_peered":"2023-10-16T20:04:57.421107+0000","last_unstale":"2023-10-16T20:05:01.442830+0000","last_undegraded":"2023-10-16T20:05:01.442830+0000","last_fullsized":"2023-10-16T20:05:01.442830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:53:20.922170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.437759+0000","last_change":"2023-10-16T20:04:57.422120+0000","last_active":"2023-10-16T20:05:01.437759+0000","last_peered":"2023-10-16T20:05:01.437759+0000","last_clean":"2023-10-16T20:05:01.437759+0000","last_became_active":"2023-10-16T20:04:57.421682+0000","last_became_peered":"2023-10-16T20:04:57.421682+0000","last_unstale":"2023-10-16T20:05:01.437759+0000","last_undegraded":"2023-10-16T20:05:01.437759+0000","last_fullsized":"2023-10-16T20:05:01.437759+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:21:09.387890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.464439+0000","last_change":"2023-10-16T20:04:57.432633+0000","last_active":"2023-10-16T20:05:01.464439+0000","last_peered":"2023-10-16T20:05:01.464439+0000","last_clean":"2023-10-16T20:05:01.464439+0000","last_became_active":"2023-10-16T20:04:57.432324+0000","last_became_peered":"2023-10-16T20:04:57.432324+0000","last_unstale":"2023-10-16T20:05:01.464439+0000","last_undegraded":"2023-10-16T20:05:01.464439+0000","last_fullsized":"2023-10-16T20:05:01.464439+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:23:06.128119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438742+0000","last_change":"2023-10-16T20:04:57.410439+0000","last_active":"2023-10-16T20:05:01.438742+0000","last_peered":"2023-10-16T20:05:01.438742+0000","last_clean":"2023-10-16T20:05:01.438742+0000","last_became_active":"2023-10-16T20:04:57.410178+0000","last_became_peered":"2023-10-16T20:04:57.410178+0000","last_unstale":"2023-10-16T20:05:01.438742+0000","last_undegraded":"2023-10-16T20:05:01.438742+0000","last_fullsized":"2023-10-16T20:05:01.438742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:33:36.402743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.758054+0000","last_change":"2023-10-16T20:04:57.436579+0000","last_active":"2023-10-16T20:05:01.758054+0000","last_peered":"2023-10-16T20:05:01.758054+0000","last_clean":"2023-10-16T20:05:01.758054+0000","last_became_active":"2023-10-16T20:04:57.436033+0000","last_became_peered":"2023-10-16T20:04:57.436033+0000","last_unstale":"2023-10-16T20:05:01.758054+0000","last_undegraded":"2023-10-16T20:05:01.758054+0000","last_fullsized":"2023-10-16T20:05:01.758054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:57:27.959012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.448091+0000","last_change":"2023-10-16T20:04:57.415573+0000","last_active":"2023-10-16T20:05:01.448091+0000","last_peered":"2023-10-16T20:05:01.448091+0000","last_clean":"2023-10-16T20:05:01.448091+0000","last_became_active":"2023-10-16T20:04:57.414461+0000","last_became_peered":"2023-10-16T20:04:57.414461+0000","last_unstale":"2023-10-16T20:05:01.448091+0000","last_undegraded":"2023-10-16T20:05:01.448091+0000","last_fullsized":"2023-10-16T20:05:01.448091+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:23:06.051449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438402+0000","last_change":"2023-10-16T20:04:57.432424+0000","last_active":"2023-10-16T20:05:01.438402+0000","last_peered":"2023-10-16T20:05:01.438402+0000","last_clean":"2023-10-16T20:05:01.438402+0000","last_became_active":"2023-10-16T20:04:57.431858+0000","last_became_peered":"2023-10-16T20:04:57.431858+0000","last_unstale":"2023-10-16T20:05:01.438402+0000","last_undegraded":"2023-10-16T20:05:01.438402+0000","last_fullsized":"2023-10-16T20:05:01.438402+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:57:27.959012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438175+0000","last_change":"2023-10-16T20:04:57.417470+0000","last_active":"2023-10-16T20:05:01.438175+0000","last_peered":"2023-10-16T20:05:01.438175+0000","last_clean":"2023-10-16T20:05:01.438175+0000","last_became_active":"2023-10-16T20:04:57.417181+0000","last_became_peered":"2023-10-16T20:04:57.417181+0000","last_unstale":"2023-10-16T20:05:01.438175+0000","last_undegraded":"2023-10-16T20:05:01.438175+0000","last_fullsized":"2023-10-16T20:05:01.438175+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:27:55.972790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471102+0000","last_change":"2023-10-16T20:04:57.431307+0000","last_active":"2023-10-16T20:05:01.471102+0000","last_peered":"2023-10-16T20:05:01.471102+0000","last_clean":"2023-10-16T20:05:01.471102+0000","last_became_active":"2023-10-16T20:04:57.430943+0000","last_became_peered":"2023-10-16T20:04:57.430943+0000","last_unstale":"2023-10-16T20:05:01.471102+0000","last_undegraded":"2023-10-16T20:05:01.471102+0000","last_fullsized":"2023-10-16T20:05:01.471102+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:17:24.205223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.463204+0000","last_change":"2023-10-16T20:04:57.428233+0000","last_active":"2023-10-16T20:05:01.463204+0000","last_peered":"2023-10-16T20:05:01.463204+0000","last_clean":"2023-10-16T20:05:01.463204+0000","last_became_active":"2023-10-16T20:04:57.427952+0000","last_became_peered":"2023-10-16T20:04:57.427952+0000","last_unstale":"2023-10-16T20:05:01.463204+0000","last_undegraded":"2023-10-16T20:05:01.463204+0000","last_fullsized":"2023-10-16T20:05:01.463204+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:57:35.297293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.441740+0000","last_change":"2023-10-16T20:04:57.425988+0000","last_active":"2023-10-16T20:05:01.441740+0000","last_peered":"2023-10-16T20:05:01.441740+0000","last_clean":"2023-10-16T20:05:01.441740+0000","last_became_active":"2023-10-16T20:04:57.425383+0000","last_became_peered":"2023-10-16T20:04:57.425383+0000","last_unstale":"2023-10-16T20:05:01.441740+0000","last_undegraded":"2023-10-16T20:05:01.441740+0000","last_fullsized":"2023-10-16T20:05:01.441740+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:23:01.141545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.447072+0000","last_change":"2023-10-16T20:04:57.415017+0000","last_active":"2023-10-16T20:05:01.447072+0000","last_peered":"2023-10-16T20:05:01.447072+0000","last_clean":"2023-10-16T20:05:01.447072+0000","last_became_active":"2023-10-16T20:04:57.414629+0000","last_became_peered":"2023-10-16T20:04:57.414629+0000","last_unstale":"2023-10-16T20:05:01.447072+0000","last_undegraded":"2023-10-16T20:05:01.447072+0000","last_fullsized":"2023-10-16T20:05:01.447072+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:41:01.253485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.439630+0000","last_change":"2023-10-16T20:04:57.421682+0000","last_active":"2023-10-16T20:05:01.439630+0000","last_peered":"2023-10-16T20:05:01.439630+0000","last_clean":"2023-10-16T20:05:01.439630+0000","last_became_active":"2023-10-16T20:04:57.421340+0000","last_became_peered":"2023-10-16T20:04:57.421340+0000","last_unstale":"2023-10-16T20:05:01.439630+0000","last_undegraded":"2023-10-16T20:05:01.439630+0000","last_fullsized":"2023-10-16T20:05:01.439630+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:56:28.993951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.440035+0000","last_change":"2023-10-16T20:04:57.415019+0000","last_active":"2023-10-16T20:05:01.440035+0000","last_peered":"2023-10-16T20:05:01.440035+0000","last_clean":"2023-10-16T20:05:01.440035+0000","last_became_active":"2023-10-16T20:04:57.414700+0000","last_became_peered":"2023-10-16T20:04:57.414700+0000","last_unstale":"2023-10-16T20:05:01.440035+0000","last_undegraded":"2023-10-16T20:05:01.440035+0000","last_fullsized":"2023-10-16T20:05:01.440035+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:17:24.205223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.758287+0000","last_change":"2023-10-16T20:04:57.436170+0000","last_active":"2023-10-16T20:05:01.758287+0000","last_peered":"2023-10-16T20:05:01.758287+0000","last_clean":"2023-10-16T20:05:01.758287+0000","last_became_active":"2023-10-16T20:04:57.435430+0000","last_became_peered":"2023-10-16T20:04:57.435430+0000","last_unstale":"2023-10-16T20:05:01.758287+0000","last_undegraded":"2023-10-16T20:05:01.758287+0000","last_fullsized":"2023-10-16T20:05:01.758287+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:39:54.738856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.441360+0000","last_change":"2023-10-16T20:04:57.429885+0000","last_active":"2023-10-16T20:05:01.441360+0000","last_peered":"2023-10-16T20:05:01.441360+0000","last_clean":"2023-10-16T20:05:01.441360+0000","last_became_active":"2023-10-16T20:04:57.429595+0000","last_became_peered":"2023-10-16T20:04:57.429595+0000","last_unstale":"2023-10-16T20:05:01.441360+0000","last_undegraded":"2023-10-16T20:05:01.441360+0000","last_fullsized":"2023-10-16T20:05:01.441360+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:49:52.409593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757637+0000","last_change":"2023-10-16T20:04:57.434436+0000","last_active":"2023-10-16T20:05:01.757637+0000","last_peered":"2023-10-16T20:05:01.757637+0000","last_clean":"2023-10-16T20:05:01.757637+0000","last_became_active":"2023-10-16T20:04:57.433525+0000","last_became_peered":"2023-10-16T20:04:57.433525+0000","last_unstale":"2023-10-16T20:05:01.757637+0000","last_undegraded":"2023-10-16T20:05:01.757637+0000","last_fullsized":"2023-10-16T20:05:01.757637+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:22:12.692981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.437681+0000","last_change":"2023-10-16T20:04:57.421933+0000","last_active":"2023-10-16T20:05:01.437681+0000","last_peered":"2023-10-16T20:05:01.437681+0000","last_clean":"2023-10-16T20:05:01.437681+0000","last_became_active":"2023-10-16T20:04:57.421342+0000","last_became_peered":"2023-10-16T20:04:57.421342+0000","last_unstale":"2023-10-16T20:05:01.437681+0000","last_undegraded":"2023-10-16T20:05:01.437681+0000","last_fullsized":"2023-10-16T20:05:01.437681+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:54:09.036700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757757+0000","last_change":"2023-10-16T20:04:57.434235+0000","last_active":"2023-10-16T20:05:01.757757+0000","last_peered":"2023-10-16T20:05:01.757757+0000","last_clean":"2023-10-16T20:05:01.757757+0000","last_became_active":"2023-10-16T20:04:57.433720+0000","last_became_peered":"2023-10-16T20:04:57.433720+0000","last_unstale":"2023-10-16T20:05:01.757757+0000","last_undegraded":"2023-10-16T20:05:01.757757+0000","last_fullsized":"2023-10-16T20:05:01.757757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:11:56.834052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438139+0000","last_change":"2023-10-16T20:04:57.431571+0000","last_active":"2023-10-16T20:05:01.438139+0000","last_peered":"2023-10-16T20:05:01.438139+0000","last_clean":"2023-10-16T20:05:01.438139+0000","last_became_active":"2023-10-16T20:04:57.429514+0000","last_became_peered":"2023-10-16T20:04:57.429514+0000","last_unstale":"2023-10-16T20:05:01.438139+0000","last_undegraded":"2023-10-16T20:05:01.438139+0000","last_fullsized":"2023-10-16T20:05:01.438139+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:31:34.884161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.464508+0000","last_change":"2023-10-16T20:04:57.412857+0000","last_active":"2023-10-16T20:05:01.464508+0000","last_peered":"2023-10-16T20:05:01.464508+0000","last_clean":"2023-10-16T20:05:01.464508+0000","last_became_active":"2023-10-16T20:04:57.412587+0000","last_became_peered":"2023-10-16T20:04:57.412587+0000","last_unstale":"2023-10-16T20:05:01.464508+0000","last_undegraded":"2023-10-16T20:05:01.464508+0000","last_fullsized":"2023-10-16T20:05:01.464508+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:42:47.128122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471650+0000","last_change":"2023-10-16T20:04:57.423730+0000","last_active":"2023-10-16T20:05:01.471650+0000","last_peered":"2023-10-16T20:05:01.471650+0000","last_clean":"2023-10-16T20:05:01.471650+0000","last_became_active":"2023-10-16T20:04:57.423300+0000","last_became_peered":"2023-10-16T20:04:57.423300+0000","last_unstale":"2023-10-16T20:05:01.471650+0000","last_undegraded":"2023-10-16T20:05:01.471650+0000","last_fullsized":"2023-10-16T20:05:01.471650+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:27:55.972790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.758256+0000","last_change":"2023-10-16T20:04:57.442158+0000","last_active":"2023-10-16T20:05:01.758256+0000","last_peered":"2023-10-16T20:05:01.758256+0000","last_clean":"2023-10-16T20:05:01.758256+0000","last_became_active":"2023-10-16T20:04:57.438707+0000","last_became_peered":"2023-10-16T20:04:57.438707+0000","last_unstale":"2023-10-16T20:05:01.758256+0000","last_undegraded":"2023-10-16T20:05:01.758256+0000","last_fullsized":"2023-10-16T20:05:01.758256+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:51:08.154913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509450,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":880640,"data_stored":701023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]}]},{"osd":6,"up_from":15,"seq":64424509451,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":294912,"data_stored":113914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":110163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.264}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4159999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]}]},{"osd":4,"up_from":15,"seq":64424509449,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":294912,"data_stored":110199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]}]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":294912,"data_stored":113914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]}]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":110073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]}]},{"osd":1,"up_from":15,"seq":64424509450,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":290816,"data_stored":110233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":888832,"data_stored":701079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.9289999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.1520000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.3440000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.5250000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.9990000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.0289999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T20:05:20.955 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T20:05:21.129 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.130+0000 7f54b1572640 1 Processor -- start 2023-10-16T20:05:21.129 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.130+0000 7f54b1572640 1 -- start start 2023-10-16T20:05:21.130 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.130+0000 7f54b1572640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f54ac15d9f0 0x7f54ac161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:21.131 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.130+0000 7f54b1572640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f54ac0d26c0 0x7f54ac0cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:21.131 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.130+0000 7f54b1572640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54ac0b87e0 con 0x7f54ac0d22e0 2023-10-16T20:05:21.131 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.130+0000 7f54b1572640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54ac15df10 con 0x7f54ac0d26c0 2023-10-16T20:05:21.131 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.130+0000 7f54b1572640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54ac15e1f0 con 0x7f54ac15d9f0 2023-10-16T20:05:21.131 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.130+0000 7f54aaffd640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f54ac0d22e0 0x7f54ac15d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:39274/0 (socket says 172.21.15.88:39274) 2023-10-16T20:05:21.132 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.130+0000 7f54aaffd640 1 -- 172.21.15.88:0/2450699354 learned_addr learned my addr 172.21.15.88:0/2450699354 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:21.132 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.130+0000 7f54ab7fe640 1 --2- 172.21.15.88:0/2450699354 >> v2:172.21.15.88:3300/0 conn(0x7f54ac0d26c0 0x7f54ac0cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:21.132 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.130+0000 7f54a3fff640 1 --2- 172.21.15.88:0/2450699354 >> v2:172.21.15.94:3300/0 conn(0x7f54ac15d9f0 0x7f54ac161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:21.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.130+0000 7f54aa7fc640 1 -- 172.21.15.88:0/2450699354 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2160473661 0 0) 0x7f54ac0b87e0 con 0x7f54ac0d22e0 2023-10-16T20:05:21.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.130+0000 7f54ab7fe640 1 -- 172.21.15.88:0/2450699354 >> v2:172.21.15.94:3300/0 conn(0x7f54ac15d9f0 msgr2=0x7f54ac161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:21.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.130+0000 7f54ab7fe640 1 --2- 172.21.15.88:0/2450699354 >> v2:172.21.15.94:3300/0 conn(0x7f54ac15d9f0 0x7f54ac161450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:21.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.130+0000 7f54ab7fe640 1 -- 172.21.15.88:0/2450699354 >> v1:172.21.15.88:6789/0 conn(0x7f54ac0d22e0 legacy=0x7f54ac15d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:21.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.130+0000 7f54ab7fe640 1 -- 172.21.15.88:0/2450699354 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ac160970 con 0x7f54ac0d26c0 2023-10-16T20:05:21.134 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.134+0000 7f54ab7fe640 1 --2- 172.21.15.88:0/2450699354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54ac0d26c0 0x7f54ac0cf2e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f54ac072520 tx=0x7f54a401c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6143500f8f99d3f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:21.134 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.134+0000 7f54aa7fc640 1 -- 172.21.15.88:0/2450699354 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a40a5910 con 0x7f54ac0d26c0 2023-10-16T20:05:21.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.134+0000 7f54aa7fc640 1 -- 172.21.15.88:0/2450699354 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a40a5a70 con 0x7f54ac0d26c0 2023-10-16T20:05:21.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.134+0000 7f54aa7fc640 1 -- 172.21.15.88:0/2450699354 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a40a5e00 con 0x7f54ac0d26c0 2023-10-16T20:05:21.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.134+0000 7f54b1572640 1 -- 172.21.15.88:0/2450699354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54ac0d26c0 msgr2=0x7f54ac0cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:21.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.134+0000 7f54b1572640 1 --2- 172.21.15.88:0/2450699354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54ac0d26c0 0x7f54ac0cf2e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f54ac072520 tx=0x7f54a401c850 comp rx=0 tx=0).stop 2023-10-16T20:05:21.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.134+0000 7f54b1572640 1 -- 172.21.15.88:0/2450699354 shutdown_connections 2023-10-16T20:05:21.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.134+0000 7f54b1572640 1 --2- 172.21.15.88:0/2450699354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54ac0d26c0 0x7f54ac0cf2e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:21.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.134+0000 7f54b1572640 1 --2- 172.21.15.88:0/2450699354 >> v2:172.21.15.94:3300/0 conn(0x7f54ac15d9f0 0x7f54ac161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:21.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.134+0000 7f54b1572640 1 -- 172.21.15.88:0/2450699354 >> 172.21.15.88:0/2450699354 conn(0x7f54ac0715c0 msgr2=0x7f54ac0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:21.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.134+0000 7f54b1572640 1 -- 172.21.15.88:0/2450699354 shutdown_connections 2023-10-16T20:05:21.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.134+0000 7f54b1572640 1 -- 172.21.15.88:0/2450699354 wait complete. 2023-10-16T20:05:21.139 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.134+0000 7f54b1572640 1 Processor -- start 2023-10-16T20:05:21.139 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.134+0000 7f54b1572640 1 -- start start 2023-10-16T20:05:21.139 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.138+0000 7f54b1572640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f54ac0d22e0 0x7f54ac154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:21.139 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.138+0000 7f54b1572640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54ac0d26c0 0x7f54ac0c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:21.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.138+0000 7f54b1572640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54ac15d9f0 0x7f54ac0c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:21.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.138+0000 7f54b1572640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac1f5890 con 0x7f54ac0d26c0 2023-10-16T20:05:21.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.138+0000 7f54b1572640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac1f5ae0 con 0x7f54ac15d9f0 2023-10-16T20:05:21.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.138+0000 7f54b1572640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac0c6850 con 0x7f54ac0d22e0 2023-10-16T20:05:21.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.138+0000 7f54a3fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54ac0d26c0 0x7f54ac0c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:21.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.138+0000 7f54a3fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54ac0d26c0 0x7f54ac0c7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40668/0 (socket says 172.21.15.88:40668) 2023-10-16T20:05:21.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.138+0000 7f54a3fff640 1 -- 172.21.15.88:0/776882960 learned_addr learned my addr 172.21.15.88:0/776882960 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:21.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.138+0000 7f54ab7fe640 1 --2- 172.21.15.88:0/776882960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54ac15d9f0 0x7f54ac0c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:21.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.138+0000 7f54aaffd640 1 --2- 172.21.15.88:0/776882960 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f54ac0d22e0 0x7f54ac154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:21.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.138+0000 7f54a3fff640 1 -- 172.21.15.88:0/776882960 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f54ac0d22e0 msgr2=0x7f54ac154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:21.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.138+0000 7f54a3fff640 1 --2- 172.21.15.88:0/776882960 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f54ac0d22e0 0x7f54ac154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:21.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.138+0000 7f54a3fff640 1 -- 172.21.15.88:0/776882960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54ac15d9f0 msgr2=0x7f54ac0c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:21.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.138+0000 7f54a3fff640 1 --2- 172.21.15.88:0/776882960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54ac15d9f0 0x7f54ac0c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:21.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.138+0000 7f54a3fff640 1 -- 172.21.15.88:0/776882960 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a4009a70 con 0x7f54ac0d26c0 2023-10-16T20:05:21.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.138+0000 7f54aaffd640 1 --2- 172.21.15.88:0/776882960 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f54ac0d22e0 0x7f54ac154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:21.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.138+0000 7f54a3fff640 1 --2- 172.21.15.88:0/776882960 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54ac0d26c0 0x7f54ac0c7320 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f549400e840 tx=0x7f549400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:21.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.138+0000 7f54a37fe640 1 -- 172.21.15.88:0/776882960 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5494009510 con 0x7f54ac0d26c0 2023-10-16T20:05:21.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.138+0000 7f54a37fe640 1 -- 172.21.15.88:0/776882960 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549400ee90 con 0x7f54ac0d26c0 2023-10-16T20:05:21.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.138+0000 7f54a37fe640 1 -- 172.21.15.88:0/776882960 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549400cb80 con 0x7f54ac0d26c0 2023-10-16T20:05:21.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.138+0000 7f54b1572640 1 -- 172.21.15.88:0/776882960 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54ac0c6b30 con 0x7f54ac0d26c0 2023-10-16T20:05:21.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.138+0000 7f54b1572640 1 -- 172.21.15.88:0/776882960 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54ac14d840 con 0x7f54ac0d26c0 2023-10-16T20:05:21.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.138+0000 7f54b1572640 1 -- 172.21.15.88:0/776882960 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54ac04e9c0 con 0x7f54ac0d26c0 2023-10-16T20:05:21.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.138+0000 7f54a37fe640 1 -- 172.21.15.88:0/776882960 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f549400cce0 con 0x7f54ac0d26c0 2023-10-16T20:05:21.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.142+0000 7f54a37fe640 1 --2- 172.21.15.88:0/776882960 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f548c0664a0 0x7f548c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:21.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.142+0000 7f54a37fe640 1 -- 172.21.15.88:0/776882960 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5494084200 con 0x7f54ac0d26c0 2023-10-16T20:05:21.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.142+0000 7f54aaffd640 1 --2- 172.21.15.88:0/776882960 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f548c0664a0 0x7f548c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:21.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.142+0000 7f54aaffd640 1 --2- 172.21.15.88:0/776882960 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f548c0664a0 0x7f548c068960 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f549c000f70 tx=0x7f549c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:21.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.146+0000 7f54a37fe640 1 -- 172.21.15.88:0/776882960 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5494051420 con 0x7f54ac0d26c0 2023-10-16T20:05:21.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.294+0000 7f54b1572640 1 -- 172.21.15.88:0/776882960 --> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f54ac0c66d0 con 0x7f548c0664a0 2023-10-16T20:05:21.304 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.302+0000 7f54a37fe640 1 -- 172.21.15.88:0/776882960 <== mgr.4110 v2:172.21.15.88:6832/4078483333 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345325 (secure 0 0 0) 0x7f549c0013e0 con 0x7f548c0664a0 2023-10-16T20:05:21.308 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:21.310 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2023-10-16T20:05:21.311 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.306+0000 7f54b1572640 1 -- 172.21.15.88:0/776882960 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f548c0664a0 msgr2=0x7f548c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:21.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.306+0000 7f54b1572640 1 --2- 172.21.15.88:0/776882960 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f548c0664a0 0x7f548c068960 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f549c000f70 tx=0x7f549c009040 comp rx=0 tx=0).stop 2023-10-16T20:05:21.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.306+0000 7f54b1572640 1 -- 172.21.15.88:0/776882960 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54ac0d26c0 msgr2=0x7f54ac0c7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:21.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.306+0000 7f54b1572640 1 --2- 172.21.15.88:0/776882960 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54ac0d26c0 0x7f54ac0c7320 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f549400e840 tx=0x7f549400ec40 comp rx=0 tx=0).stop 2023-10-16T20:05:21.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.306+0000 7f54b1572640 1 -- 172.21.15.88:0/776882960 shutdown_connections 2023-10-16T20:05:21.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.306+0000 7f54b1572640 1 --2- 172.21.15.88:0/776882960 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f548c0664a0 0x7f548c068960 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:21.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.306+0000 7f54b1572640 1 --2- 172.21.15.88:0/776882960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54ac15d9f0 0x7f54ac0c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:21.313 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.306+0000 7f54b1572640 1 --2- 172.21.15.88:0/776882960 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54ac0d26c0 0x7f54ac0c7320 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:21.313 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.306+0000 7f54b1572640 1 --2- 172.21.15.88:0/776882960 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f54ac0d22e0 0x7f54ac154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:21.313 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.306+0000 7f54b1572640 1 -- 172.21.15.88:0/776882960 >> 172.21.15.88:0/776882960 conn(0x7f54ac0715c0 msgr2=0x7f54ac0b7680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:21.313 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.306+0000 7f54b1572640 1 -- 172.21.15.88:0/776882960 shutdown_connections 2023-10-16T20:05:21.313 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.310+0000 7f54b1572640 1 -- 172.21.15.88:0/776882960 wait complete. 2023-10-16T20:05:21.328 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2023-10-16T20:05:20.449726+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219628,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755092,"statfs":{"total":773094113280,"available":772869214208,"internally_reserved":0,"allocated":3514368,"data_stored":2070598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":221236793},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2229,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003612"},"pg_stats":[{"pgid":"2.1f","version":"22'4","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.465897+0000","last_change":"2023-10-16T20:04:56.406335+0000","last_active":"2023-10-16T20:05:01.465897+0000","last_peered":"2023-10-16T20:05:01.465897+0000","last_clean":"2023-10-16T20:05:01.465897+0000","last_became_active":"2023-10-16T20:04:56.406060+0000","last_became_peered":"2023-10-16T20:04:56.406060+0000","last_unstale":"2023-10-16T20:05:01.465897+0000","last_undegraded":"2023-10-16T20:05:01.465897+0000","last_fullsized":"2023-10-16T20:05:01.465897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:15:58.928603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442061+0000","last_change":"2023-10-16T20:04:57.416039+0000","last_active":"2023-10-16T20:05:01.442061+0000","last_peered":"2023-10-16T20:05:01.442061+0000","last_clean":"2023-10-16T20:05:01.442061+0000","last_became_active":"2023-10-16T20:04:57.415746+0000","last_became_peered":"2023-10-16T20:04:57.415746+0000","last_unstale":"2023-10-16T20:05:01.442061+0000","last_undegraded":"2023-10-16T20:05:01.442061+0000","last_fullsized":"2023-10-16T20:05:01.442061+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:27:55.972790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.685606+0000","last_change":"2023-10-16T20:05:01.770262+0000","last_active":"2023-10-16T20:05:02.685606+0000","last_peered":"2023-10-16T20:05:02.685606+0000","last_clean":"2023-10-16T20:05:02.685606+0000","last_became_active":"2023-10-16T20:05:01.769688+0000","last_became_peered":"2023-10-16T20:05:01.769688+0000","last_unstale":"2023-10-16T20:05:02.685606+0000","last_undegraded":"2023-10-16T20:05:02.685606+0000","last_fullsized":"2023-10-16T20:05:02.685606+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:51:12.210160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442225+0000","last_change":"2023-10-16T20:04:56.409621+0000","last_active":"2023-10-16T20:05:01.442225+0000","last_peered":"2023-10-16T20:05:01.442225+0000","last_clean":"2023-10-16T20:05:01.442225+0000","last_became_active":"2023-10-16T20:04:56.409150+0000","last_became_peered":"2023-10-16T20:04:56.409150+0000","last_unstale":"2023-10-16T20:05:01.442225+0000","last_undegraded":"2023-10-16T20:05:01.442225+0000","last_fullsized":"2023-10-16T20:05:01.442225+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:24:16.965233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.465724+0000","last_change":"2023-10-16T20:04:57.430587+0000","last_active":"2023-10-16T20:05:01.465724+0000","last_peered":"2023-10-16T20:05:01.465724+0000","last_clean":"2023-10-16T20:05:01.465724+0000","last_became_active":"2023-10-16T20:04:57.430224+0000","last_became_peered":"2023-10-16T20:04:57.430224+0000","last_unstale":"2023-10-16T20:05:01.465724+0000","last_undegraded":"2023-10-16T20:05:01.465724+0000","last_fullsized":"2023-10-16T20:05:01.465724+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:49:52.662102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.682460+0000","last_change":"2023-10-16T20:05:01.478221+0000","last_active":"2023-10-16T20:05:02.682460+0000","last_peered":"2023-10-16T20:05:02.682460+0000","last_clean":"2023-10-16T20:05:02.682460+0000","last_became_active":"2023-10-16T20:05:01.477692+0000","last_became_peered":"2023-10-16T20:05:01.477692+0000","last_unstale":"2023-10-16T20:05:02.682460+0000","last_undegraded":"2023-10-16T20:05:02.682460+0000","last_fullsized":"2023-10-16T20:05:02.682460+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:58:37.917893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.758057+0000","last_change":"2023-10-16T20:04:56.409779+0000","last_active":"2023-10-16T20:05:01.758057+0000","last_peered":"2023-10-16T20:05:01.758057+0000","last_clean":"2023-10-16T20:05:01.758057+0000","last_became_active":"2023-10-16T20:04:56.409515+0000","last_became_peered":"2023-10-16T20:04:56.409515+0000","last_unstale":"2023-10-16T20:05:01.758057+0000","last_undegraded":"2023-10-16T20:05:01.758057+0000","last_fullsized":"2023-10-16T20:05:01.758057+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:17:23.198236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438236+0000","last_change":"2023-10-16T20:04:57.414284+0000","last_active":"2023-10-16T20:05:01.438236+0000","last_peered":"2023-10-16T20:05:01.438236+0000","last_clean":"2023-10-16T20:05:01.438236+0000","last_became_active":"2023-10-16T20:04:57.413997+0000","last_became_peered":"2023-10-16T20:04:57.413997+0000","last_unstale":"2023-10-16T20:05:01.438236+0000","last_undegraded":"2023-10-16T20:05:01.438236+0000","last_fullsized":"2023-10-16T20:05:01.438236+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:28:00.096002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.682751+0000","last_change":"2023-10-16T20:05:01.482429+0000","last_active":"2023-10-16T20:05:02.682751+0000","last_peered":"2023-10-16T20:05:02.682751+0000","last_clean":"2023-10-16T20:05:02.682751+0000","last_became_active":"2023-10-16T20:05:01.481993+0000","last_became_peered":"2023-10-16T20:05:01.481993+0000","last_unstale":"2023-10-16T20:05:02.682751+0000","last_undegraded":"2023-10-16T20:05:02.682751+0000","last_fullsized":"2023-10-16T20:05:02.682751+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:54:13.091947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757866+0000","last_change":"2023-10-16T20:04:56.412981+0000","last_active":"2023-10-16T20:05:01.757866+0000","last_peered":"2023-10-16T20:05:01.757866+0000","last_clean":"2023-10-16T20:05:01.757866+0000","last_became_active":"2023-10-16T20:04:56.412613+0000","last_became_peered":"2023-10-16T20:04:56.412613+0000","last_unstale":"2023-10-16T20:05:01.757866+0000","last_undegraded":"2023-10-16T20:05:01.757866+0000","last_fullsized":"2023-10-16T20:05:01.757866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:14:27.634593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.440267+0000","last_change":"2023-10-16T20:04:57.411866+0000","last_active":"2023-10-16T20:05:01.440267+0000","last_peered":"2023-10-16T20:05:01.440267+0000","last_clean":"2023-10-16T20:05:01.440267+0000","last_became_active":"2023-10-16T20:04:57.411523+0000","last_became_peered":"2023-10-16T20:04:57.411523+0000","last_unstale":"2023-10-16T20:05:01.440267+0000","last_undegraded":"2023-10-16T20:05:01.440267+0000","last_fullsized":"2023-10-16T20:05:01.440267+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:01:38.242317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.683398+0000","last_change":"2023-10-16T20:05:01.460546+0000","last_active":"2023-10-16T20:05:02.683398+0000","last_peered":"2023-10-16T20:05:02.683398+0000","last_clean":"2023-10-16T20:05:02.683398+0000","last_became_active":"2023-10-16T20:05:01.460120+0000","last_became_peered":"2023-10-16T20:05:01.460120+0000","last_unstale":"2023-10-16T20:05:02.683398+0000","last_undegraded":"2023-10-16T20:05:02.683398+0000","last_fullsized":"2023-10-16T20:05:02.683398+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:31:38.939408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438705+0000","last_change":"2023-10-16T20:04:57.409882+0000","last_active":"2023-10-16T20:05:01.438705+0000","last_peered":"2023-10-16T20:05:01.438705+0000","last_clean":"2023-10-16T20:05:01.438705+0000","last_became_active":"2023-10-16T20:04:57.409534+0000","last_became_peered":"2023-10-16T20:04:57.409534+0000","last_unstale":"2023-10-16T20:05:01.438705+0000","last_undegraded":"2023-10-16T20:05:01.438705+0000","last_fullsized":"2023-10-16T20:05:01.438705+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:23:01.141545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757875+0000","last_change":"2023-10-16T20:04:56.406078+0000","last_active":"2023-10-16T20:05:01.757875+0000","last_peered":"2023-10-16T20:05:01.757875+0000","last_clean":"2023-10-16T20:05:01.757875+0000","last_became_active":"2023-10-16T20:04:56.405784+0000","last_became_peered":"2023-10-16T20:04:56.405784+0000","last_unstale":"2023-10-16T20:05:01.757875+0000","last_undegraded":"2023-10-16T20:05:01.757875+0000","last_fullsized":"2023-10-16T20:05:01.757875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:13:40.552947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.683535+0000","last_change":"2023-10-16T20:05:01.477288+0000","last_active":"2023-10-16T20:05:02.683535+0000","last_peered":"2023-10-16T20:05:02.683535+0000","last_clean":"2023-10-16T20:05:02.683535+0000","last_became_active":"2023-10-16T20:05:01.476269+0000","last_became_peered":"2023-10-16T20:05:01.476269+0000","last_unstale":"2023-10-16T20:05:02.683535+0000","last_undegraded":"2023-10-16T20:05:02.683535+0000","last_fullsized":"2023-10-16T20:05:02.683535+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:28:04.151249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.441837+0000","last_change":"2023-10-16T20:04:57.426154+0000","last_active":"2023-10-16T20:05:01.441837+0000","last_peered":"2023-10-16T20:05:01.441837+0000","last_clean":"2023-10-16T20:05:01.441837+0000","last_became_active":"2023-10-16T20:04:57.425681+0000","last_became_peered":"2023-10-16T20:04:57.425681+0000","last_unstale":"2023-10-16T20:05:01.441837+0000","last_undegraded":"2023-10-16T20:05:01.441837+0000","last_fullsized":"2023-10-16T20:05:01.441837+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:54:09.036700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471304+0000","last_change":"2023-10-16T20:04:56.406283+0000","last_active":"2023-10-16T20:05:01.471304+0000","last_peered":"2023-10-16T20:05:01.471304+0000","last_clean":"2023-10-16T20:05:01.471304+0000","last_became_active":"2023-10-16T20:04:56.405952+0000","last_became_peered":"2023-10-16T20:04:56.405952+0000","last_unstale":"2023-10-16T20:05:01.471304+0000","last_undegraded":"2023-10-16T20:05:01.471304+0000","last_fullsized":"2023-10-16T20:05:01.471304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:14:27.634593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.682647+0000","last_change":"2023-10-16T20:05:01.482495+0000","last_active":"2023-10-16T20:05:02.682647+0000","last_peered":"2023-10-16T20:05:02.682647+0000","last_clean":"2023-10-16T20:05:02.682647+0000","last_became_active":"2023-10-16T20:05:01.482072+0000","last_became_peered":"2023-10-16T20:05:01.482072+0000","last_unstale":"2023-10-16T20:05:02.682647+0000","last_undegraded":"2023-10-16T20:05:02.682647+0000","last_fullsized":"2023-10-16T20:05:02.682647+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:28:00.028037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442089+0000","last_change":"2023-10-16T20:04:57.425668+0000","last_active":"2023-10-16T20:05:01.442089+0000","last_peered":"2023-10-16T20:05:01.442089+0000","last_clean":"2023-10-16T20:05:01.442089+0000","last_became_active":"2023-10-16T20:04:57.425420+0000","last_became_peered":"2023-10-16T20:04:57.425420+0000","last_unstale":"2023-10-16T20:05:01.442089+0000","last_undegraded":"2023-10-16T20:05:01.442089+0000","last_fullsized":"2023-10-16T20:05:01.442089+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:33:36.402743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471190+0000","last_change":"2023-10-16T20:04:56.403231+0000","last_active":"2023-10-16T20:05:01.471190+0000","last_peered":"2023-10-16T20:05:01.471190+0000","last_clean":"2023-10-16T20:05:01.471190+0000","last_became_active":"2023-10-16T20:04:56.402769+0000","last_became_peered":"2023-10-16T20:04:56.402769+0000","last_unstale":"2023-10-16T20:05:01.471190+0000","last_undegraded":"2023-10-16T20:05:01.471190+0000","last_fullsized":"2023-10-16T20:05:01.471190+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:55:47.529420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.445092+0000","last_change":"2023-10-16T20:05:01.768596+0000","last_active":"2023-10-16T20:05:02.445092+0000","last_peered":"2023-10-16T20:05:02.445092+0000","last_clean":"2023-10-16T20:05:02.445092+0000","last_became_active":"2023-10-16T20:05:01.768140+0000","last_became_peered":"2023-10-16T20:05:01.768140+0000","last_unstale":"2023-10-16T20:05:02.445092+0000","last_undegraded":"2023-10-16T20:05:02.445092+0000","last_fullsized":"2023-10-16T20:05:02.445092+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:01:42.297564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.447386+0000","last_change":"2023-10-16T20:04:57.407922+0000","last_active":"2023-10-16T20:05:01.447386+0000","last_peered":"2023-10-16T20:05:01.447386+0000","last_clean":"2023-10-16T20:05:01.447386+0000","last_became_active":"2023-10-16T20:04:57.407580+0000","last_became_peered":"2023-10-16T20:04:57.407580+0000","last_unstale":"2023-10-16T20:05:01.447386+0000","last_undegraded":"2023-10-16T20:05:01.447386+0000","last_fullsized":"2023-10-16T20:05:01.447386+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:24:17.972220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757524+0000","last_change":"2023-10-16T20:04:56.408300+0000","last_active":"2023-10-16T20:05:01.757524+0000","last_peered":"2023-10-16T20:05:01.757524+0000","last_clean":"2023-10-16T20:05:01.757524+0000","last_became_active":"2023-10-16T20:04:56.407908+0000","last_became_peered":"2023-10-16T20:04:56.407908+0000","last_unstale":"2023-10-16T20:05:01.757524+0000","last_undegraded":"2023-10-16T20:05:01.757524+0000","last_fullsized":"2023-10-16T20:05:01.757524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:53:19.915183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.683126+0000","last_change":"2023-10-16T20:05:01.476964+0000","last_active":"2023-10-16T20:05:02.683126+0000","last_peered":"2023-10-16T20:05:02.683126+0000","last_clean":"2023-10-16T20:05:02.683126+0000","last_became_active":"2023-10-16T20:05:01.476584+0000","last_became_peered":"2023-10-16T20:05:01.476584+0000","last_unstale":"2023-10-16T20:05:02.683126+0000","last_undegraded":"2023-10-16T20:05:02.683126+0000","last_fullsized":"2023-10-16T20:05:02.683126+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:22:16.748229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.465558+0000","last_change":"2023-10-16T20:04:57.424772+0000","last_active":"2023-10-16T20:05:01.465558+0000","last_peered":"2023-10-16T20:05:01.465558+0000","last_clean":"2023-10-16T20:05:01.465558+0000","last_became_active":"2023-10-16T20:04:57.424340+0000","last_became_peered":"2023-10-16T20:04:57.424340+0000","last_unstale":"2023-10-16T20:05:01.465558+0000","last_undegraded":"2023-10-16T20:05:01.465558+0000","last_fullsized":"2023-10-16T20:05:01.465558+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:10:08.760019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.439134+0000","last_change":"2023-10-16T20:04:56.401336+0000","last_active":"2023-10-16T20:05:01.439134+0000","last_peered":"2023-10-16T20:05:01.439134+0000","last_clean":"2023-10-16T20:05:01.439134+0000","last_became_active":"2023-10-16T20:04:56.401019+0000","last_became_peered":"2023-10-16T20:04:56.401019+0000","last_unstale":"2023-10-16T20:05:01.439134+0000","last_undegraded":"2023-10-16T20:05:01.439134+0000","last_fullsized":"2023-10-16T20:05:01.439134+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:58:19.945263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.682734+0000","last_change":"2023-10-16T20:05:01.760505+0000","last_active":"2023-10-16T20:05:02.682734+0000","last_peered":"2023-10-16T20:05:02.682734+0000","last_clean":"2023-10-16T20:05:02.682734+0000","last_became_active":"2023-10-16T20:05:01.760043+0000","last_became_peered":"2023-10-16T20:05:01.760043+0000","last_unstale":"2023-10-16T20:05:02.682734+0000","last_undegraded":"2023-10-16T20:05:02.682734+0000","last_fullsized":"2023-10-16T20:05:02.682734+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:11:32.906879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442279+0000","last_change":"2023-10-16T20:04:57.430184+0000","last_active":"2023-10-16T20:05:01.442279+0000","last_peered":"2023-10-16T20:05:01.442279+0000","last_clean":"2023-10-16T20:05:01.442279+0000","last_became_active":"2023-10-16T20:04:57.429845+0000","last_became_peered":"2023-10-16T20:04:57.429845+0000","last_unstale":"2023-10-16T20:05:01.442279+0000","last_undegraded":"2023-10-16T20:05:01.442279+0000","last_fullsized":"2023-10-16T20:05:01.442279+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:11:28.851632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471731+0000","last_change":"2023-10-16T20:04:56.403148+0000","last_active":"2023-10-16T20:05:01.471731+0000","last_peered":"2023-10-16T20:05:01.471731+0000","last_clean":"2023-10-16T20:05:01.471731+0000","last_became_active":"2023-10-16T20:04:56.402707+0000","last_became_peered":"2023-10-16T20:04:56.402707+0000","last_unstale":"2023-10-16T20:05:01.471731+0000","last_undegraded":"2023-10-16T20:05:01.471731+0000","last_fullsized":"2023-10-16T20:05:01.471731+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:53:19.915183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.685503+0000","last_change":"2023-10-16T20:05:01.764474+0000","last_active":"2023-10-16T20:05:02.685503+0000","last_peered":"2023-10-16T20:05:02.685503+0000","last_clean":"2023-10-16T20:05:02.685503+0000","last_became_active":"2023-10-16T20:05:01.763840+0000","last_became_peered":"2023-10-16T20:05:01.763840+0000","last_unstale":"2023-10-16T20:05:02.685503+0000","last_undegraded":"2023-10-16T20:05:02.685503+0000","last_fullsized":"2023-10-16T20:05:02.685503+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:57:32.014259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471448+0000","last_change":"2023-10-16T20:04:57.423634+0000","last_active":"2023-10-16T20:05:01.471448+0000","last_peered":"2023-10-16T20:05:01.471448+0000","last_clean":"2023-10-16T20:05:01.471448+0000","last_became_active":"2023-10-16T20:04:57.423321+0000","last_became_peered":"2023-10-16T20:04:57.423321+0000","last_unstale":"2023-10-16T20:05:01.471448+0000","last_undegraded":"2023-10-16T20:05:01.471448+0000","last_fullsized":"2023-10-16T20:05:01.471448+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:23:01.141545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.758143+0000","last_change":"2023-10-16T20:04:56.408656+0000","last_active":"2023-10-16T20:05:01.758143+0000","last_peered":"2023-10-16T20:05:01.758143+0000","last_clean":"2023-10-16T20:05:01.758143+0000","last_became_active":"2023-10-16T20:04:56.408192+0000","last_became_peered":"2023-10-16T20:04:56.408192+0000","last_unstale":"2023-10-16T20:05:01.758143+0000","last_undegraded":"2023-10-16T20:05:01.758143+0000","last_fullsized":"2023-10-16T20:05:01.758143+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:54:08.029713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.688512+0000","last_change":"2023-10-16T20:05:01.767238+0000","last_active":"2023-10-16T20:05:02.688512+0000","last_peered":"2023-10-16T20:05:02.688512+0000","last_clean":"2023-10-16T20:05:02.688512+0000","last_became_active":"2023-10-16T20:05:01.765827+0000","last_became_peered":"2023-10-16T20:05:01.765827+0000","last_unstale":"2023-10-16T20:05:02.688512+0000","last_undegraded":"2023-10-16T20:05:02.688512+0000","last_fullsized":"2023-10-16T20:05:02.688512+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:14:36.535545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.440300+0000","last_change":"2023-10-16T20:04:57.430055+0000","last_active":"2023-10-16T20:05:01.440300+0000","last_peered":"2023-10-16T20:05:01.440300+0000","last_clean":"2023-10-16T20:05:01.440300+0000","last_became_active":"2023-10-16T20:04:57.428753+0000","last_became_peered":"2023-10-16T20:04:57.428753+0000","last_unstale":"2023-10-16T20:05:01.440300+0000","last_undegraded":"2023-10-16T20:05:01.440300+0000","last_fullsized":"2023-10-16T20:05:01.440300+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:14:29.143380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471458+0000","last_change":"2023-10-16T20:04:56.411374+0000","last_active":"2023-10-16T20:05:01.471458+0000","last_peered":"2023-10-16T20:05:01.471458+0000","last_clean":"2023-10-16T20:05:01.471458+0000","last_became_active":"2023-10-16T20:04:56.411098+0000","last_became_peered":"2023-10-16T20:04:56.411098+0000","last_unstale":"2023-10-16T20:05:01.471458+0000","last_undegraded":"2023-10-16T20:05:01.471458+0000","last_fullsized":"2023-10-16T20:05:01.471458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:04:32.948232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.444207+0000","last_change":"2023-10-16T20:05:01.473814+0000","last_active":"2023-10-16T20:05:02.444207+0000","last_peered":"2023-10-16T20:05:02.444207+0000","last_clean":"2023-10-16T20:05:02.444207+0000","last_became_active":"2023-10-16T20:05:01.472021+0000","last_became_peered":"2023-10-16T20:05:01.472021+0000","last_unstale":"2023-10-16T20:05:02.444207+0000","last_undegraded":"2023-10-16T20:05:02.444207+0000","last_fullsized":"2023-10-16T20:05:02.444207+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:12:00.889299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.447473+0000","last_change":"2023-10-16T20:04:57.419253+0000","last_active":"2023-10-16T20:05:01.447473+0000","last_peered":"2023-10-16T20:05:01.447473+0000","last_clean":"2023-10-16T20:05:01.447473+0000","last_became_active":"2023-10-16T20:04:57.418196+0000","last_became_peered":"2023-10-16T20:04:57.418196+0000","last_unstale":"2023-10-16T20:05:01.447473+0000","last_undegraded":"2023-10-16T20:05:01.447473+0000","last_fullsized":"2023-10-16T20:05:01.447473+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:58:20.952250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.437911+0000","last_change":"2023-10-16T20:04:56.405694+0000","last_active":"2023-10-16T20:05:01.437911+0000","last_peered":"2023-10-16T20:05:01.437911+0000","last_clean":"2023-10-16T20:05:01.437911+0000","last_became_active":"2023-10-16T20:04:56.405369+0000","last_became_peered":"2023-10-16T20:04:56.405369+0000","last_unstale":"2023-10-16T20:05:01.437911+0000","last_undegraded":"2023-10-16T20:05:01.437911+0000","last_fullsized":"2023-10-16T20:05:01.437911+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:58:32.855659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.444695+0000","last_change":"2023-10-16T20:05:01.478161+0000","last_active":"2023-10-16T20:05:02.444695+0000","last_peered":"2023-10-16T20:05:02.444695+0000","last_clean":"2023-10-16T20:05:02.444695+0000","last_became_active":"2023-10-16T20:05:01.477735+0000","last_became_peered":"2023-10-16T20:05:01.477735+0000","last_unstale":"2023-10-16T20:05:02.444695+0000","last_undegraded":"2023-10-16T20:05:02.444695+0000","last_fullsized":"2023-10-16T20:05:02.444695+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:14:33.198627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471054+0000","last_change":"2023-10-16T20:04:57.427768+0000","last_active":"2023-10-16T20:05:01.471054+0000","last_peered":"2023-10-16T20:05:01.471054+0000","last_clean":"2023-10-16T20:05:01.471054+0000","last_became_active":"2023-10-16T20:04:57.422018+0000","last_became_peered":"2023-10-16T20:04:57.422018+0000","last_unstale":"2023-10-16T20:05:01.471054+0000","last_undegraded":"2023-10-16T20:05:01.471054+0000","last_fullsized":"2023-10-16T20:05:01.471054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:49:52.409593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.448426+0000","last_change":"2023-10-16T20:04:56.402767+0000","last_active":"2023-10-16T20:05:01.448426+0000","last_peered":"2023-10-16T20:05:01.448426+0000","last_clean":"2023-10-16T20:05:01.448426+0000","last_became_active":"2023-10-16T20:04:56.402122+0000","last_became_peered":"2023-10-16T20:04:56.402122+0000","last_unstale":"2023-10-16T20:05:01.448426+0000","last_undegraded":"2023-10-16T20:05:01.448426+0000","last_fullsized":"2023-10-16T20:05:01.448426+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:54:36.775882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.445579+0000","last_change":"2023-10-16T20:05:01.478710+0000","last_active":"2023-10-16T20:05:02.445579+0000","last_peered":"2023-10-16T20:05:02.445579+0000","last_clean":"2023-10-16T20:05:02.445579+0000","last_became_active":"2023-10-16T20:05:01.478350+0000","last_became_peered":"2023-10-16T20:05:01.478350+0000","last_unstale":"2023-10-16T20:05:02.445579+0000","last_undegraded":"2023-10-16T20:05:02.445579+0000","last_fullsized":"2023-10-16T20:05:02.445579+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:57:32.014259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.459302+0000","last_change":"2023-10-16T20:04:57.420638+0000","last_active":"2023-10-16T20:05:01.459302+0000","last_peered":"2023-10-16T20:05:01.459302+0000","last_clean":"2023-10-16T20:05:01.459302+0000","last_became_active":"2023-10-16T20:04:57.420348+0000","last_became_peered":"2023-10-16T20:04:57.420348+0000","last_unstale":"2023-10-16T20:05:01.459302+0000","last_undegraded":"2023-10-16T20:05:01.459302+0000","last_fullsized":"2023-10-16T20:05:01.459302+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:14:18.231180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757558+0000","last_change":"2023-10-16T20:04:56.403488+0000","last_active":"2023-10-16T20:05:01.757558+0000","last_peered":"2023-10-16T20:05:01.757558+0000","last_clean":"2023-10-16T20:05:01.757558+0000","last_became_active":"2023-10-16T20:04:56.403156+0000","last_became_peered":"2023-10-16T20:04:56.403156+0000","last_unstale":"2023-10-16T20:05:01.757558+0000","last_undegraded":"2023-10-16T20:05:01.757558+0000","last_fullsized":"2023-10-16T20:05:01.757558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:33:35.395756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.688883+0000","last_change":"2023-10-16T20:05:01.765108+0000","last_active":"2023-10-16T20:05:02.688883+0000","last_peered":"2023-10-16T20:05:02.688883+0000","last_clean":"2023-10-16T20:05:02.688883+0000","last_became_active":"2023-10-16T20:05:01.764436+0000","last_became_peered":"2023-10-16T20:05:01.764436+0000","last_unstale":"2023-10-16T20:05:02.688883+0000","last_undegraded":"2023-10-16T20:05:02.688883+0000","last_fullsized":"2023-10-16T20:05:02.688883+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:39:29.554873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442360+0000","last_change":"2023-10-16T20:04:57.429060+0000","last_active":"2023-10-16T20:05:01.442360+0000","last_peered":"2023-10-16T20:05:01.442360+0000","last_clean":"2023-10-16T20:05:01.442360+0000","last_became_active":"2023-10-16T20:04:57.428756+0000","last_became_peered":"2023-10-16T20:04:57.428756+0000","last_unstale":"2023-10-16T20:05:01.442360+0000","last_undegraded":"2023-10-16T20:05:01.442360+0000","last_fullsized":"2023-10-16T20:05:01.442360+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:21:09.387890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.437753+0000","last_change":"2023-10-16T20:04:56.392274+0000","last_active":"2023-10-16T20:05:01.437753+0000","last_peered":"2023-10-16T20:05:01.437753+0000","last_clean":"2023-10-16T20:05:01.437753+0000","last_became_active":"2023-10-16T20:04:56.391958+0000","last_became_peered":"2023-10-16T20:04:56.391958+0000","last_unstale":"2023-10-16T20:05:01.437753+0000","last_undegraded":"2023-10-16T20:05:01.437753+0000","last_fullsized":"2023-10-16T20:05:01.437753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:58:19.945263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":169,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757620+0000","last_change":"2023-10-16T20:04:46.335822+0000","last_active":"2023-10-16T20:05:01.757620+0000","last_peered":"2023-10-16T20:05:01.757620+0000","last_clean":"2023-10-16T20:05:01.757620+0000","last_became_active":"2023-10-16T20:04:46.334617+0000","last_became_peered":"2023-10-16T20:04:46.334617+0000","last_unstale":"2023-10-16T20:05:01.757620+0000","last_undegraded":"2023-10-16T20:05:01.757620+0000","last_fullsized":"2023-10-16T20:05:01.757620+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:45.306240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:45.306240+0000","last_clean_scrub_stamp":"2023-10-16T20:04:45.306240+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:42:11.153721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.682739+0000","last_change":"2023-10-16T20:05:01.760533+0000","last_active":"2023-10-16T20:05:02.682739+0000","last_peered":"2023-10-16T20:05:02.682739+0000","last_clean":"2023-10-16T20:05:02.682739+0000","last_became_active":"2023-10-16T20:05:01.760091+0000","last_became_peered":"2023-10-16T20:05:01.760091+0000","last_unstale":"2023-10-16T20:05:02.682739+0000","last_undegraded":"2023-10-16T20:05:02.682739+0000","last_fullsized":"2023-10-16T20:05:02.682739+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:22:16.748229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757962+0000","last_change":"2023-10-16T20:04:57.435770+0000","last_active":"2023-10-16T20:05:01.757962+0000","last_peered":"2023-10-16T20:05:01.757962+0000","last_clean":"2023-10-16T20:05:01.757962+0000","last_became_active":"2023-10-16T20:04:57.434914+0000","last_became_peered":"2023-10-16T20:04:57.434914+0000","last_unstale":"2023-10-16T20:05:01.757962+0000","last_undegraded":"2023-10-16T20:05:01.757962+0000","last_fullsized":"2023-10-16T20:05:01.757962+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:42:35.216191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.449802+0000","last_change":"2023-10-16T20:04:56.414794+0000","last_active":"2023-10-16T20:05:01.449802+0000","last_peered":"2023-10-16T20:05:01.449802+0000","last_clean":"2023-10-16T20:05:01.449802+0000","last_became_active":"2023-10-16T20:04:56.414400+0000","last_became_peered":"2023-10-16T20:04:56.414400+0000","last_unstale":"2023-10-16T20:05:01.449802+0000","last_undegraded":"2023-10-16T20:05:01.449802+0000","last_fullsized":"2023-10-16T20:05:01.449802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:19:48.110438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.682395+0000","last_change":"2023-10-16T20:05:01.760979+0000","last_active":"2023-10-16T20:05:02.682395+0000","last_peered":"2023-10-16T20:05:02.682395+0000","last_clean":"2023-10-16T20:05:02.682395+0000","last_became_active":"2023-10-16T20:05:01.760141+0000","last_became_peered":"2023-10-16T20:05:01.760141+0000","last_unstale":"2023-10-16T20:05:02.682395+0000","last_undegraded":"2023-10-16T20:05:02.682395+0000","last_fullsized":"2023-10-16T20:05:02.682395+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:21:13.443137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438336+0000","last_change":"2023-10-16T20:04:57.429698+0000","last_active":"2023-10-16T20:05:01.438336+0000","last_peered":"2023-10-16T20:05:01.438336+0000","last_clean":"2023-10-16T20:05:01.438336+0000","last_became_active":"2023-10-16T20:04:57.428050+0000","last_became_peered":"2023-10-16T20:04:57.428050+0000","last_unstale":"2023-10-16T20:05:01.438336+0000","last_undegraded":"2023-10-16T20:05:01.438336+0000","last_fullsized":"2023-10-16T20:05:01.438336+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:42:35.216191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471736+0000","last_change":"2023-10-16T20:04:56.409587+0000","last_active":"2023-10-16T20:05:01.471736+0000","last_peered":"2023-10-16T20:05:01.471736+0000","last_clean":"2023-10-16T20:05:01.471736+0000","last_became_active":"2023-10-16T20:04:56.408150+0000","last_became_peered":"2023-10-16T20:04:56.408150+0000","last_unstale":"2023-10-16T20:05:01.471736+0000","last_undegraded":"2023-10-16T20:05:01.471736+0000","last_fullsized":"2023-10-16T20:05:01.471736+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:13:40.552947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.688655+0000","last_change":"2023-10-16T20:05:01.476280+0000","last_active":"2023-10-16T20:05:02.688655+0000","last_peered":"2023-10-16T20:05:02.688655+0000","last_clean":"2023-10-16T20:05:02.688655+0000","last_became_active":"2023-10-16T20:05:01.475885+0000","last_became_peered":"2023-10-16T20:05:01.475885+0000","last_unstale":"2023-10-16T20:05:02.688655+0000","last_undegraded":"2023-10-16T20:05:02.688655+0000","last_fullsized":"2023-10-16T20:05:02.688655+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:01:16.289925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.758343+0000","last_change":"2023-10-16T20:04:57.442292+0000","last_active":"2023-10-16T20:05:01.758343+0000","last_peered":"2023-10-16T20:05:01.758343+0000","last_clean":"2023-10-16T20:05:01.758343+0000","last_became_active":"2023-10-16T20:04:57.439075+0000","last_became_peered":"2023-10-16T20:04:57.439075+0000","last_unstale":"2023-10-16T20:05:01.758343+0000","last_undegraded":"2023-10-16T20:05:01.758343+0000","last_fullsized":"2023-10-16T20:05:01.758343+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:25:46.207669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.448189+0000","last_change":"2023-10-16T20:04:56.398007+0000","last_active":"2023-10-16T20:05:01.448189+0000","last_peered":"2023-10-16T20:05:01.448189+0000","last_clean":"2023-10-16T20:05:01.448189+0000","last_became_active":"2023-10-16T20:04:56.397108+0000","last_became_peered":"2023-10-16T20:04:56.397108+0000","last_unstale":"2023-10-16T20:05:01.448189+0000","last_undegraded":"2023-10-16T20:05:01.448189+0000","last_fullsized":"2023-10-16T20:05:01.448189+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:24:16.965233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.684215+0000","last_change":"2023-10-16T20:05:01.763390+0000","last_active":"2023-10-16T20:05:02.684215+0000","last_peered":"2023-10-16T20:05:02.684215+0000","last_clean":"2023-10-16T20:05:02.684215+0000","last_became_active":"2023-10-16T20:05:01.760077+0000","last_became_peered":"2023-10-16T20:05:01.760077+0000","last_unstale":"2023-10-16T20:05:02.684215+0000","last_undegraded":"2023-10-16T20:05:02.684215+0000","last_fullsized":"2023-10-16T20:05:02.684215+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:10:33.807776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.758160+0000","last_change":"2023-10-16T20:04:57.437968+0000","last_active":"2023-10-16T20:05:01.758160+0000","last_peered":"2023-10-16T20:05:01.758160+0000","last_clean":"2023-10-16T20:05:01.758160+0000","last_became_active":"2023-10-16T20:04:57.437674+0000","last_became_peered":"2023-10-16T20:04:57.437674+0000","last_unstale":"2023-10-16T20:05:01.758160+0000","last_undegraded":"2023-10-16T20:05:01.758160+0000","last_fullsized":"2023-10-16T20:05:01.758160+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:01:38.242317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.447078+0000","last_change":"2023-10-16T20:04:56.408227+0000","last_active":"2023-10-16T20:05:01.447078+0000","last_peered":"2023-10-16T20:05:01.447078+0000","last_clean":"2023-10-16T20:05:01.447078+0000","last_became_active":"2023-10-16T20:04:56.407279+0000","last_became_peered":"2023-10-16T20:04:56.407279+0000","last_unstale":"2023-10-16T20:05:01.447078+0000","last_undegraded":"2023-10-16T20:05:01.447078+0000","last_fullsized":"2023-10-16T20:05:01.447078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:43:11.864176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.682415+0000","last_change":"2023-10-16T20:05:01.477611+0000","last_active":"2023-10-16T20:05:02.682415+0000","last_peered":"2023-10-16T20:05:02.682415+0000","last_clean":"2023-10-16T20:05:02.682415+0000","last_became_active":"2023-10-16T20:05:01.477172+0000","last_became_peered":"2023-10-16T20:05:01.477172+0000","last_unstale":"2023-10-16T20:05:02.682415+0000","last_undegraded":"2023-10-16T20:05:02.682415+0000","last_fullsized":"2023-10-16T20:05:02.682415+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:23:05.196792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757700+0000","last_change":"2023-10-16T20:04:57.438300+0000","last_active":"2023-10-16T20:05:01.757700+0000","last_peered":"2023-10-16T20:05:01.757700+0000","last_clean":"2023-10-16T20:05:01.757700+0000","last_became_active":"2023-10-16T20:04:57.437951+0000","last_became_peered":"2023-10-16T20:04:57.437951+0000","last_unstale":"2023-10-16T20:05:01.757700+0000","last_undegraded":"2023-10-16T20:05:01.757700+0000","last_fullsized":"2023-10-16T20:05:01.757700+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:31:34.884161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.448376+0000","last_change":"2023-10-16T20:04:56.401341+0000","last_active":"2023-10-16T20:05:01.448376+0000","last_peered":"2023-10-16T20:05:01.448376+0000","last_clean":"2023-10-16T20:05:01.448376+0000","last_became_active":"2023-10-16T20:04:56.401058+0000","last_became_peered":"2023-10-16T20:04:56.401058+0000","last_unstale":"2023-10-16T20:05:01.448376+0000","last_undegraded":"2023-10-16T20:05:01.448376+0000","last_fullsized":"2023-10-16T20:05:01.448376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:41:34.379170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.688808+0000","last_change":"2023-10-16T20:05:01.463304+0000","last_active":"2023-10-16T20:05:02.688808+0000","last_peered":"2023-10-16T20:05:02.688808+0000","last_clean":"2023-10-16T20:05:02.688808+0000","last_became_active":"2023-10-16T20:05:01.462044+0000","last_became_peered":"2023-10-16T20:05:01.462044+0000","last_unstale":"2023-10-16T20:05:02.688808+0000","last_undegraded":"2023-10-16T20:05:02.688808+0000","last_fullsized":"2023-10-16T20:05:02.688808+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:41:12.320369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.440492+0000","last_change":"2023-10-16T20:04:57.409823+0000","last_active":"2023-10-16T20:05:01.440492+0000","last_peered":"2023-10-16T20:05:01.440492+0000","last_clean":"2023-10-16T20:05:01.440492+0000","last_became_active":"2023-10-16T20:04:57.409457+0000","last_became_peered":"2023-10-16T20:04:57.409457+0000","last_unstale":"2023-10-16T20:05:01.440492+0000","last_undegraded":"2023-10-16T20:05:01.440492+0000","last_fullsized":"2023-10-16T20:05:01.440492+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:53:20.922170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.440306+0000","last_change":"2023-10-16T20:04:56.404854+0000","last_active":"2023-10-16T20:05:01.440306+0000","last_peered":"2023-10-16T20:05:01.440306+0000","last_clean":"2023-10-16T20:05:01.440306+0000","last_became_active":"2023-10-16T20:04:56.404457+0000","last_became_peered":"2023-10-16T20:04:56.404457+0000","last_unstale":"2023-10-16T20:05:01.440306+0000","last_undegraded":"2023-10-16T20:05:01.440306+0000","last_fullsized":"2023-10-16T20:05:01.440306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:41:00.246498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.686341+0000","last_change":"2023-10-16T20:05:01.769521+0000","last_active":"2023-10-16T20:05:02.686341+0000","last_peered":"2023-10-16T20:05:02.686341+0000","last_clean":"2023-10-16T20:05:02.686341+0000","last_became_active":"2023-10-16T20:05:01.768728+0000","last_became_peered":"2023-10-16T20:05:01.768728+0000","last_unstale":"2023-10-16T20:05:02.686341+0000","last_undegraded":"2023-10-16T20:05:02.686341+0000","last_fullsized":"2023-10-16T20:05:02.686341+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:01:55.141797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757851+0000","last_change":"2023-10-16T20:04:57.438646+0000","last_active":"2023-10-16T20:05:01.757851+0000","last_peered":"2023-10-16T20:05:01.757851+0000","last_clean":"2023-10-16T20:05:01.757851+0000","last_became_active":"2023-10-16T20:04:57.438375+0000","last_became_peered":"2023-10-16T20:04:57.438375+0000","last_unstale":"2023-10-16T20:05:01.757851+0000","last_undegraded":"2023-10-16T20:05:01.757851+0000","last_fullsized":"2023-10-16T20:05:01.757851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:14:29.143380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.437845+0000","last_change":"2023-10-16T20:04:56.407281+0000","last_active":"2023-10-16T20:05:01.437845+0000","last_peered":"2023-10-16T20:05:01.437845+0000","last_clean":"2023-10-16T20:05:01.437845+0000","last_became_active":"2023-10-16T20:04:56.406824+0000","last_became_peered":"2023-10-16T20:04:56.406824+0000","last_unstale":"2023-10-16T20:05:01.437845+0000","last_undegraded":"2023-10-16T20:05:01.437845+0000","last_fullsized":"2023-10-16T20:05:01.437845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:23:00.134558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.686002+0000","last_change":"2023-10-16T20:05:01.764372+0000","last_active":"2023-10-16T20:05:02.686002+0000","last_peered":"2023-10-16T20:05:02.686002+0000","last_clean":"2023-10-16T20:05:02.686002+0000","last_became_active":"2023-10-16T20:05:01.763918+0000","last_became_peered":"2023-10-16T20:05:01.763918+0000","last_unstale":"2023-10-16T20:05:02.686002+0000","last_undegraded":"2023-10-16T20:05:02.686002+0000","last_fullsized":"2023-10-16T20:05:02.686002+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:10:33.807776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.463201+0000","last_change":"2023-10-16T20:04:57.425920+0000","last_active":"2023-10-16T20:05:01.463201+0000","last_peered":"2023-10-16T20:05:01.463201+0000","last_clean":"2023-10-16T20:05:01.463201+0000","last_became_active":"2023-10-16T20:04:57.425614+0000","last_became_peered":"2023-10-16T20:04:57.425614+0000","last_unstale":"2023-10-16T20:05:01.463201+0000","last_undegraded":"2023-10-16T20:05:01.463201+0000","last_fullsized":"2023-10-16T20:05:01.463201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:28:14.992492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.464594+0000","last_change":"2023-10-16T20:04:56.407990+0000","last_active":"2023-10-16T20:05:01.464594+0000","last_peered":"2023-10-16T20:05:01.464594+0000","last_clean":"2023-10-16T20:05:01.464594+0000","last_became_active":"2023-10-16T20:04:56.407682+0000","last_became_peered":"2023-10-16T20:04:56.407682+0000","last_unstale":"2023-10-16T20:05:01.464594+0000","last_undegraded":"2023-10-16T20:05:01.464594+0000","last_fullsized":"2023-10-16T20:05:01.464594+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:25:27.337575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.685081+0000","last_change":"2023-10-16T20:05:01.761327+0000","last_active":"2023-10-16T20:05:02.685081+0000","last_peered":"2023-10-16T20:05:02.685081+0000","last_clean":"2023-10-16T20:05:02.685081+0000","last_became_active":"2023-10-16T20:05:01.760040+0000","last_became_peered":"2023-10-16T20:05:01.760040+0000","last_unstale":"2023-10-16T20:05:02.685081+0000","last_undegraded":"2023-10-16T20:05:02.685081+0000","last_fullsized":"2023-10-16T20:05:02.685081+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:33:23.371729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757841+0000","last_change":"2023-10-16T20:04:57.437602+0000","last_active":"2023-10-16T20:05:01.757841+0000","last_peered":"2023-10-16T20:05:01.757841+0000","last_clean":"2023-10-16T20:05:01.757841+0000","last_became_active":"2023-10-16T20:04:57.436601+0000","last_became_peered":"2023-10-16T20:04:57.436601+0000","last_unstale":"2023-10-16T20:05:01.757841+0000","last_undegraded":"2023-10-16T20:05:01.757841+0000","last_fullsized":"2023-10-16T20:05:01.757841+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:28:00.096002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'9","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:08.020486+0000","last_change":"2023-10-16T20:04:56.398743+0000","last_active":"2023-10-16T20:05:08.020486+0000","last_peered":"2023-10-16T20:05:08.020486+0000","last_clean":"2023-10-16T20:05:08.020486+0000","last_became_active":"2023-10-16T20:04:56.398410+0000","last_became_peered":"2023-10-16T20:04:56.398410+0000","last_unstale":"2023-10-16T20:05:08.020486+0000","last_undegraded":"2023-10-16T20:05:08.020486+0000","last_fullsized":"2023-10-16T20:05:08.020486+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:21:13.046934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.444394+0000","last_change":"2023-10-16T20:05:01.472414+0000","last_active":"2023-10-16T20:05:02.444394+0000","last_peered":"2023-10-16T20:05:02.444394+0000","last_clean":"2023-10-16T20:05:02.444394+0000","last_became_active":"2023-10-16T20:05:01.471943+0000","last_became_peered":"2023-10-16T20:05:01.471943+0000","last_unstale":"2023-10-16T20:05:02.444394+0000","last_undegraded":"2023-10-16T20:05:02.444394+0000","last_fullsized":"2023-10-16T20:05:02.444394+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:42:39.271438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.447164+0000","last_change":"2023-10-16T20:04:57.413204+0000","last_active":"2023-10-16T20:05:01.447164+0000","last_peered":"2023-10-16T20:05:01.447164+0000","last_clean":"2023-10-16T20:05:01.447164+0000","last_became_active":"2023-10-16T20:04:57.412836+0000","last_became_peered":"2023-10-16T20:04:57.412836+0000","last_unstale":"2023-10-16T20:05:01.447164+0000","last_undegraded":"2023-10-16T20:05:01.447164+0000","last_fullsized":"2023-10-16T20:05:01.447164+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:23:06.051449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471656+0000","last_change":"2023-10-16T20:04:56.414671+0000","last_active":"2023-10-16T20:05:01.471656+0000","last_peered":"2023-10-16T20:05:01.471656+0000","last_clean":"2023-10-16T20:05:01.471656+0000","last_became_active":"2023-10-16T20:04:56.414339+0000","last_became_peered":"2023-10-16T20:04:56.414339+0000","last_unstale":"2023-10-16T20:05:01.471656+0000","last_undegraded":"2023-10-16T20:05:01.471656+0000","last_fullsized":"2023-10-16T20:05:01.471656+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:21:13.046934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.683018+0000","last_change":"2023-10-16T20:05:01.482674+0000","last_active":"2023-10-16T20:05:02.683018+0000","last_peered":"2023-10-16T20:05:02.683018+0000","last_clean":"2023-10-16T20:05:02.683018+0000","last_became_active":"2023-10-16T20:05:01.482048+0000","last_became_peered":"2023-10-16T20:05:01.482048+0000","last_unstale":"2023-10-16T20:05:02.683018+0000","last_undegraded":"2023-10-16T20:05:02.683018+0000","last_fullsized":"2023-10-16T20:05:02.683018+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:21:13.443137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.465807+0000","last_change":"2023-10-16T20:04:57.430429+0000","last_active":"2023-10-16T20:05:01.465807+0000","last_peered":"2023-10-16T20:05:01.465807+0000","last_clean":"2023-10-16T20:05:01.465807+0000","last_became_active":"2023-10-16T20:04:57.429872+0000","last_became_peered":"2023-10-16T20:04:57.429872+0000","last_unstale":"2023-10-16T20:05:01.465807+0000","last_undegraded":"2023-10-16T20:05:01.465807+0000","last_fullsized":"2023-10-16T20:05:01.465807+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:47:33.537999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438017+0000","last_change":"2023-10-16T20:04:56.398061+0000","last_active":"2023-10-16T20:05:01.438017+0000","last_peered":"2023-10-16T20:05:01.438017+0000","last_clean":"2023-10-16T20:05:01.438017+0000","last_became_active":"2023-10-16T20:04:56.397712+0000","last_became_peered":"2023-10-16T20:04:56.397712+0000","last_unstale":"2023-10-16T20:05:01.438017+0000","last_undegraded":"2023-10-16T20:05:01.438017+0000","last_fullsized":"2023-10-16T20:05:01.438017+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:49:51.402606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.685658+0000","last_change":"2023-10-16T20:05:01.476635+0000","last_active":"2023-10-16T20:05:02.685658+0000","last_peered":"2023-10-16T20:05:02.685658+0000","last_clean":"2023-10-16T20:05:02.685658+0000","last_became_active":"2023-10-16T20:05:01.476053+0000","last_became_peered":"2023-10-16T20:05:01.476053+0000","last_unstale":"2023-10-16T20:05:02.685658+0000","last_undegraded":"2023-10-16T20:05:02.685658+0000","last_fullsized":"2023-10-16T20:05:02.685658+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:25:50.262916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.437931+0000","last_change":"2023-10-16T20:04:57.421476+0000","last_active":"2023-10-16T20:05:01.437931+0000","last_peered":"2023-10-16T20:05:01.437931+0000","last_clean":"2023-10-16T20:05:01.437931+0000","last_became_active":"2023-10-16T20:04:57.421206+0000","last_became_peered":"2023-10-16T20:04:57.421206+0000","last_unstale":"2023-10-16T20:05:01.437931+0000","last_undegraded":"2023-10-16T20:05:01.437931+0000","last_fullsized":"2023-10-16T20:05:01.437931+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:11:56.834052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.439019+0000","last_change":"2023-10-16T20:04:56.411232+0000","last_active":"2023-10-16T20:05:01.439019+0000","last_peered":"2023-10-16T20:05:01.439019+0000","last_clean":"2023-10-16T20:05:01.439019+0000","last_became_active":"2023-10-16T20:04:56.410793+0000","last_became_peered":"2023-10-16T20:04:56.410793+0000","last_unstale":"2023-10-16T20:05:01.439019+0000","last_undegraded":"2023-10-16T20:05:01.439019+0000","last_fullsized":"2023-10-16T20:05:01.439019+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:19:48.110438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.685647+0000","last_change":"2023-10-16T20:05:01.764968+0000","last_active":"2023-10-16T20:05:02.685647+0000","last_peered":"2023-10-16T20:05:02.685647+0000","last_clean":"2023-10-16T20:05:02.685647+0000","last_became_active":"2023-10-16T20:05:01.763830+0000","last_became_peered":"2023-10-16T20:05:01.763830+0000","last_unstale":"2023-10-16T20:05:02.685647+0000","last_undegraded":"2023-10-16T20:05:02.685647+0000","last_fullsized":"2023-10-16T20:05:02.685647+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:39:58.794103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.688809+0000","last_change":"2023-10-16T20:05:01.764846+0000","last_active":"2023-10-16T20:05:02.688809+0000","last_peered":"2023-10-16T20:05:02.688809+0000","last_clean":"2023-10-16T20:05:02.688809+0000","last_became_active":"2023-10-16T20:05:01.763756+0000","last_became_peered":"2023-10-16T20:05:01.763756+0000","last_unstale":"2023-10-16T20:05:02.688809+0000","last_undegraded":"2023-10-16T20:05:02.688809+0000","last_fullsized":"2023-10-16T20:05:02.688809+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:10:31.962156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471094+0000","last_change":"2023-10-16T20:04:57.428882+0000","last_active":"2023-10-16T20:05:01.471094+0000","last_peered":"2023-10-16T20:05:01.471094+0000","last_clean":"2023-10-16T20:05:01.471094+0000","last_became_active":"2023-10-16T20:04:57.428591+0000","last_became_peered":"2023-10-16T20:04:57.428591+0000","last_unstale":"2023-10-16T20:05:01.471094+0000","last_undegraded":"2023-10-16T20:05:01.471094+0000","last_fullsized":"2023-10-16T20:05:01.471094+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:33:36.402743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438343+0000","last_change":"2023-10-16T20:04:56.396769+0000","last_active":"2023-10-16T20:05:01.438343+0000","last_peered":"2023-10-16T20:05:01.438343+0000","last_clean":"2023-10-16T20:05:01.438343+0000","last_became_active":"2023-10-16T20:04:56.396273+0000","last_became_peered":"2023-10-16T20:04:56.396273+0000","last_unstale":"2023-10-16T20:05:01.438343+0000","last_undegraded":"2023-10-16T20:05:01.438343+0000","last_fullsized":"2023-10-16T20:05:01.438343+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:04:32.948232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.683891+0000","last_change":"2023-10-16T20:05:01.475065+0000","last_active":"2023-10-16T20:05:02.683891+0000","last_peered":"2023-10-16T20:05:02.683891+0000","last_clean":"2023-10-16T20:05:02.683891+0000","last_became_active":"2023-10-16T20:05:01.474571+0000","last_became_peered":"2023-10-16T20:05:01.474571+0000","last_unstale":"2023-10-16T20:05:02.683891+0000","last_undegraded":"2023-10-16T20:05:02.683891+0000","last_fullsized":"2023-10-16T20:05:02.683891+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:31:38.939408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442005+0000","last_change":"2023-10-16T20:04:57.423943+0000","last_active":"2023-10-16T20:05:01.442005+0000","last_peered":"2023-10-16T20:05:01.442005+0000","last_clean":"2023-10-16T20:05:01.442005+0000","last_became_active":"2023-10-16T20:04:57.423325+0000","last_became_peered":"2023-10-16T20:04:57.423325+0000","last_unstale":"2023-10-16T20:05:01.442005+0000","last_undegraded":"2023-10-16T20:05:01.442005+0000","last_fullsized":"2023-10-16T20:05:01.442005+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:56:28.993951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442942+0000","last_change":"2023-10-16T20:04:56.397549+0000","last_active":"2023-10-16T20:05:01.442942+0000","last_peered":"2023-10-16T20:05:01.442942+0000","last_clean":"2023-10-16T20:05:01.442942+0000","last_became_active":"2023-10-16T20:04:56.396648+0000","last_became_peered":"2023-10-16T20:04:56.396648+0000","last_unstale":"2023-10-16T20:05:01.442942+0000","last_undegraded":"2023-10-16T20:05:01.442942+0000","last_fullsized":"2023-10-16T20:05:01.442942+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:58:19.945263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438803+0000","last_change":"2023-10-16T20:04:56.407155+0000","last_active":"2023-10-16T20:05:01.438803+0000","last_peered":"2023-10-16T20:05:01.438803+0000","last_clean":"2023-10-16T20:05:01.438803+0000","last_became_active":"2023-10-16T20:04:56.406828+0000","last_became_peered":"2023-10-16T20:04:56.406828+0000","last_unstale":"2023-10-16T20:05:01.438803+0000","last_undegraded":"2023-10-16T20:05:01.438803+0000","last_fullsized":"2023-10-16T20:05:01.438803+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:23:05.044462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.464536+0000","last_change":"2023-10-16T20:04:57.416841+0000","last_active":"2023-10-16T20:05:01.464536+0000","last_peered":"2023-10-16T20:05:01.464536+0000","last_clean":"2023-10-16T20:05:01.464536+0000","last_became_active":"2023-10-16T20:04:57.416338+0000","last_became_peered":"2023-10-16T20:04:57.416338+0000","last_unstale":"2023-10-16T20:05:01.464536+0000","last_undegraded":"2023-10-16T20:05:01.464536+0000","last_fullsized":"2023-10-16T20:05:01.464536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:01.396735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.445821+0000","last_change":"2023-10-16T20:05:01.481039+0000","last_active":"2023-10-16T20:05:02.445821+0000","last_peered":"2023-10-16T20:05:02.445821+0000","last_clean":"2023-10-16T20:05:02.445821+0000","last_became_active":"2023-10-16T20:05:01.480626+0000","last_became_peered":"2023-10-16T20:05:01.480626+0000","last_unstale":"2023-10-16T20:05:02.445821+0000","last_undegraded":"2023-10-16T20:05:02.445821+0000","last_fullsized":"2023-10-16T20:05:02.445821+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:21:18.109168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.441938+0000","last_change":"2023-10-16T20:04:56.401545+0000","last_active":"2023-10-16T20:05:01.441938+0000","last_peered":"2023-10-16T20:05:01.441938+0000","last_clean":"2023-10-16T20:05:01.441938+0000","last_became_active":"2023-10-16T20:04:56.401276+0000","last_became_peered":"2023-10-16T20:04:56.401276+0000","last_unstale":"2023-10-16T20:05:01.441938+0000","last_undegraded":"2023-10-16T20:05:01.441938+0000","last_fullsized":"2023-10-16T20:05:01.441938+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:55.367617+0000","last_clean_scrub_stamp":"2023-10-16T20:04:55.367617+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:04:32.948232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.447134+0000","last_change":"2023-10-16T20:04:57.425600+0000","last_active":"2023-10-16T20:05:01.447134+0000","last_peered":"2023-10-16T20:05:01.447134+0000","last_clean":"2023-10-16T20:05:01.447134+0000","last_became_active":"2023-10-16T20:04:57.425213+0000","last_became_peered":"2023-10-16T20:04:57.425213+0000","last_unstale":"2023-10-16T20:05:01.447134+0000","last_undegraded":"2023-10-16T20:05:01.447134+0000","last_fullsized":"2023-10-16T20:05:01.447134+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:33:14.623281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-16T20:05:02.686142+0000","last_change":"2023-10-16T20:05:01.479226+0000","last_active":"2023-10-16T20:05:02.686142+0000","last_peered":"2023-10-16T20:05:02.686142+0000","last_clean":"2023-10-16T20:05:02.686142+0000","last_became_active":"2023-10-16T20:05:01.478808+0000","last_became_peered":"2023-10-16T20:05:01.478808+0000","last_unstale":"2023-10-16T20:05:02.686142+0000","last_undegraded":"2023-10-16T20:05:02.686142+0000","last_fullsized":"2023-10-16T20:05:02.686142+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:05:00.429852+0000","last_clean_scrub_stamp":"2023-10-16T20:05:00.429852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:33:23.371729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.440406+0000","last_change":"2023-10-16T20:04:57.409772+0000","last_active":"2023-10-16T20:05:01.440406+0000","last_peered":"2023-10-16T20:05:01.440406+0000","last_clean":"2023-10-16T20:05:01.440406+0000","last_became_active":"2023-10-16T20:04:57.409449+0000","last_became_peered":"2023-10-16T20:04:57.409449+0000","last_unstale":"2023-10-16T20:05:01.440406+0000","last_undegraded":"2023-10-16T20:05:01.440406+0000","last_fullsized":"2023-10-16T20:05:01.440406+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:13:41.559934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.448190+0000","last_change":"2023-10-16T20:04:57.418433+0000","last_active":"2023-10-16T20:05:01.448190+0000","last_peered":"2023-10-16T20:05:01.448190+0000","last_clean":"2023-10-16T20:05:01.448190+0000","last_became_active":"2023-10-16T20:04:57.418140+0000","last_became_peered":"2023-10-16T20:04:57.418140+0000","last_unstale":"2023-10-16T20:05:01.448190+0000","last_undegraded":"2023-10-16T20:05:01.448190+0000","last_fullsized":"2023-10-16T20:05:01.448190+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:41:01.253485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.437811+0000","last_change":"2023-10-16T20:04:57.430273+0000","last_active":"2023-10-16T20:05:01.437811+0000","last_peered":"2023-10-16T20:05:01.437811+0000","last_clean":"2023-10-16T20:05:01.437811+0000","last_became_active":"2023-10-16T20:04:57.427616+0000","last_became_peered":"2023-10-16T20:04:57.427616+0000","last_unstale":"2023-10-16T20:05:01.437811+0000","last_undegraded":"2023-10-16T20:05:01.437811+0000","last_fullsized":"2023-10-16T20:05:01.437811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:22:12.692981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.464484+0000","last_change":"2023-10-16T20:04:57.434154+0000","last_active":"2023-10-16T20:05:01.464484+0000","last_peered":"2023-10-16T20:05:01.464484+0000","last_clean":"2023-10-16T20:05:01.464484+0000","last_became_active":"2023-10-16T20:04:57.433876+0000","last_became_peered":"2023-10-16T20:04:57.433876+0000","last_unstale":"2023-10-16T20:05:01.464484+0000","last_undegraded":"2023-10-16T20:05:01.464484+0000","last_fullsized":"2023-10-16T20:05:01.464484+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:45:41.399175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.443384+0000","last_change":"2023-10-16T20:04:57.437575+0000","last_active":"2023-10-16T20:05:01.443384+0000","last_peered":"2023-10-16T20:05:01.443384+0000","last_clean":"2023-10-16T20:05:01.443384+0000","last_became_active":"2023-10-16T20:04:57.437261+0000","last_became_peered":"2023-10-16T20:04:57.437261+0000","last_unstale":"2023-10-16T20:05:01.443384+0000","last_undegraded":"2023-10-16T20:05:01.443384+0000","last_fullsized":"2023-10-16T20:05:01.443384+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:58:33.862646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471548+0000","last_change":"2023-10-16T20:04:57.434048+0000","last_active":"2023-10-16T20:05:01.471548+0000","last_peered":"2023-10-16T20:05:01.471548+0000","last_clean":"2023-10-16T20:05:01.471548+0000","last_became_active":"2023-10-16T20:04:57.433771+0000","last_became_peered":"2023-10-16T20:04:57.433771+0000","last_unstale":"2023-10-16T20:05:01.471548+0000","last_undegraded":"2023-10-16T20:05:01.471548+0000","last_fullsized":"2023-10-16T20:05:01.471548+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:58:33.862646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.440121+0000","last_change":"2023-10-16T20:04:57.439787+0000","last_active":"2023-10-16T20:05:01.440121+0000","last_peered":"2023-10-16T20:05:01.440121+0000","last_clean":"2023-10-16T20:05:01.440121+0000","last_became_active":"2023-10-16T20:04:57.439468+0000","last_became_peered":"2023-10-16T20:04:57.439468+0000","last_unstale":"2023-10-16T20:05:01.440121+0000","last_undegraded":"2023-10-16T20:05:01.440121+0000","last_fullsized":"2023-10-16T20:05:01.440121+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:49:52.409593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442167+0000","last_change":"2023-10-16T20:04:57.421442+0000","last_active":"2023-10-16T20:05:01.442167+0000","last_peered":"2023-10-16T20:05:01.442167+0000","last_clean":"2023-10-16T20:05:01.442167+0000","last_became_active":"2023-10-16T20:04:57.421123+0000","last_became_peered":"2023-10-16T20:04:57.421123+0000","last_unstale":"2023-10-16T20:05:01.442167+0000","last_undegraded":"2023-10-16T20:05:01.442167+0000","last_fullsized":"2023-10-16T20:05:01.442167+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:17:24.205223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438259+0000","last_change":"2023-10-16T20:04:57.412843+0000","last_active":"2023-10-16T20:05:01.438259+0000","last_peered":"2023-10-16T20:05:01.438259+0000","last_clean":"2023-10-16T20:05:01.438259+0000","last_became_active":"2023-10-16T20:04:57.412559+0000","last_became_peered":"2023-10-16T20:04:57.412559+0000","last_unstale":"2023-10-16T20:05:01.438259+0000","last_undegraded":"2023-10-16T20:05:01.438259+0000","last_fullsized":"2023-10-16T20:05:01.438259+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:11:28.851632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.442830+0000","last_change":"2023-10-16T20:04:57.421462+0000","last_active":"2023-10-16T20:05:01.442830+0000","last_peered":"2023-10-16T20:05:01.442830+0000","last_clean":"2023-10-16T20:05:01.442830+0000","last_became_active":"2023-10-16T20:04:57.421107+0000","last_became_peered":"2023-10-16T20:04:57.421107+0000","last_unstale":"2023-10-16T20:05:01.442830+0000","last_undegraded":"2023-10-16T20:05:01.442830+0000","last_fullsized":"2023-10-16T20:05:01.442830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:53:20.922170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.437759+0000","last_change":"2023-10-16T20:04:57.422120+0000","last_active":"2023-10-16T20:05:01.437759+0000","last_peered":"2023-10-16T20:05:01.437759+0000","last_clean":"2023-10-16T20:05:01.437759+0000","last_became_active":"2023-10-16T20:04:57.421682+0000","last_became_peered":"2023-10-16T20:04:57.421682+0000","last_unstale":"2023-10-16T20:05:01.437759+0000","last_undegraded":"2023-10-16T20:05:01.437759+0000","last_fullsized":"2023-10-16T20:05:01.437759+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:21:09.387890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.464439+0000","last_change":"2023-10-16T20:04:57.432633+0000","last_active":"2023-10-16T20:05:01.464439+0000","last_peered":"2023-10-16T20:05:01.464439+0000","last_clean":"2023-10-16T20:05:01.464439+0000","last_became_active":"2023-10-16T20:04:57.432324+0000","last_became_peered":"2023-10-16T20:04:57.432324+0000","last_unstale":"2023-10-16T20:05:01.464439+0000","last_undegraded":"2023-10-16T20:05:01.464439+0000","last_fullsized":"2023-10-16T20:05:01.464439+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:23:06.128119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438742+0000","last_change":"2023-10-16T20:04:57.410439+0000","last_active":"2023-10-16T20:05:01.438742+0000","last_peered":"2023-10-16T20:05:01.438742+0000","last_clean":"2023-10-16T20:05:01.438742+0000","last_became_active":"2023-10-16T20:04:57.410178+0000","last_became_peered":"2023-10-16T20:04:57.410178+0000","last_unstale":"2023-10-16T20:05:01.438742+0000","last_undegraded":"2023-10-16T20:05:01.438742+0000","last_fullsized":"2023-10-16T20:05:01.438742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:33:36.402743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.758054+0000","last_change":"2023-10-16T20:04:57.436579+0000","last_active":"2023-10-16T20:05:01.758054+0000","last_peered":"2023-10-16T20:05:01.758054+0000","last_clean":"2023-10-16T20:05:01.758054+0000","last_became_active":"2023-10-16T20:04:57.436033+0000","last_became_peered":"2023-10-16T20:04:57.436033+0000","last_unstale":"2023-10-16T20:05:01.758054+0000","last_undegraded":"2023-10-16T20:05:01.758054+0000","last_fullsized":"2023-10-16T20:05:01.758054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:57:27.959012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.448091+0000","last_change":"2023-10-16T20:04:57.415573+0000","last_active":"2023-10-16T20:05:01.448091+0000","last_peered":"2023-10-16T20:05:01.448091+0000","last_clean":"2023-10-16T20:05:01.448091+0000","last_became_active":"2023-10-16T20:04:57.414461+0000","last_became_peered":"2023-10-16T20:04:57.414461+0000","last_unstale":"2023-10-16T20:05:01.448091+0000","last_undegraded":"2023-10-16T20:05:01.448091+0000","last_fullsized":"2023-10-16T20:05:01.448091+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:23:06.051449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438402+0000","last_change":"2023-10-16T20:04:57.432424+0000","last_active":"2023-10-16T20:05:01.438402+0000","last_peered":"2023-10-16T20:05:01.438402+0000","last_clean":"2023-10-16T20:05:01.438402+0000","last_became_active":"2023-10-16T20:04:57.431858+0000","last_became_peered":"2023-10-16T20:04:57.431858+0000","last_unstale":"2023-10-16T20:05:01.438402+0000","last_undegraded":"2023-10-16T20:05:01.438402+0000","last_fullsized":"2023-10-16T20:05:01.438402+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:57:27.959012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438175+0000","last_change":"2023-10-16T20:04:57.417470+0000","last_active":"2023-10-16T20:05:01.438175+0000","last_peered":"2023-10-16T20:05:01.438175+0000","last_clean":"2023-10-16T20:05:01.438175+0000","last_became_active":"2023-10-16T20:04:57.417181+0000","last_became_peered":"2023-10-16T20:04:57.417181+0000","last_unstale":"2023-10-16T20:05:01.438175+0000","last_undegraded":"2023-10-16T20:05:01.438175+0000","last_fullsized":"2023-10-16T20:05:01.438175+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:27:55.972790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471102+0000","last_change":"2023-10-16T20:04:57.431307+0000","last_active":"2023-10-16T20:05:01.471102+0000","last_peered":"2023-10-16T20:05:01.471102+0000","last_clean":"2023-10-16T20:05:01.471102+0000","last_became_active":"2023-10-16T20:04:57.430943+0000","last_became_peered":"2023-10-16T20:04:57.430943+0000","last_unstale":"2023-10-16T20:05:01.471102+0000","last_undegraded":"2023-10-16T20:05:01.471102+0000","last_fullsized":"2023-10-16T20:05:01.471102+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:17:24.205223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.463204+0000","last_change":"2023-10-16T20:04:57.428233+0000","last_active":"2023-10-16T20:05:01.463204+0000","last_peered":"2023-10-16T20:05:01.463204+0000","last_clean":"2023-10-16T20:05:01.463204+0000","last_became_active":"2023-10-16T20:04:57.427952+0000","last_became_peered":"2023-10-16T20:04:57.427952+0000","last_unstale":"2023-10-16T20:05:01.463204+0000","last_undegraded":"2023-10-16T20:05:01.463204+0000","last_fullsized":"2023-10-16T20:05:01.463204+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:57:35.297293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.441740+0000","last_change":"2023-10-16T20:04:57.425988+0000","last_active":"2023-10-16T20:05:01.441740+0000","last_peered":"2023-10-16T20:05:01.441740+0000","last_clean":"2023-10-16T20:05:01.441740+0000","last_became_active":"2023-10-16T20:04:57.425383+0000","last_became_peered":"2023-10-16T20:04:57.425383+0000","last_unstale":"2023-10-16T20:05:01.441740+0000","last_undegraded":"2023-10-16T20:05:01.441740+0000","last_fullsized":"2023-10-16T20:05:01.441740+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:23:01.141545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.447072+0000","last_change":"2023-10-16T20:04:57.415017+0000","last_active":"2023-10-16T20:05:01.447072+0000","last_peered":"2023-10-16T20:05:01.447072+0000","last_clean":"2023-10-16T20:05:01.447072+0000","last_became_active":"2023-10-16T20:04:57.414629+0000","last_became_peered":"2023-10-16T20:04:57.414629+0000","last_unstale":"2023-10-16T20:05:01.447072+0000","last_undegraded":"2023-10-16T20:05:01.447072+0000","last_fullsized":"2023-10-16T20:05:01.447072+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:41:01.253485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.439630+0000","last_change":"2023-10-16T20:04:57.421682+0000","last_active":"2023-10-16T20:05:01.439630+0000","last_peered":"2023-10-16T20:05:01.439630+0000","last_clean":"2023-10-16T20:05:01.439630+0000","last_became_active":"2023-10-16T20:04:57.421340+0000","last_became_peered":"2023-10-16T20:04:57.421340+0000","last_unstale":"2023-10-16T20:05:01.439630+0000","last_undegraded":"2023-10-16T20:05:01.439630+0000","last_fullsized":"2023-10-16T20:05:01.439630+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:56:28.993951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.440035+0000","last_change":"2023-10-16T20:04:57.415019+0000","last_active":"2023-10-16T20:05:01.440035+0000","last_peered":"2023-10-16T20:05:01.440035+0000","last_clean":"2023-10-16T20:05:01.440035+0000","last_became_active":"2023-10-16T20:04:57.414700+0000","last_became_peered":"2023-10-16T20:04:57.414700+0000","last_unstale":"2023-10-16T20:05:01.440035+0000","last_undegraded":"2023-10-16T20:05:01.440035+0000","last_fullsized":"2023-10-16T20:05:01.440035+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:17:24.205223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.758287+0000","last_change":"2023-10-16T20:04:57.436170+0000","last_active":"2023-10-16T20:05:01.758287+0000","last_peered":"2023-10-16T20:05:01.758287+0000","last_clean":"2023-10-16T20:05:01.758287+0000","last_became_active":"2023-10-16T20:04:57.435430+0000","last_became_peered":"2023-10-16T20:04:57.435430+0000","last_unstale":"2023-10-16T20:05:01.758287+0000","last_undegraded":"2023-10-16T20:05:01.758287+0000","last_fullsized":"2023-10-16T20:05:01.758287+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:39:54.738856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.441360+0000","last_change":"2023-10-16T20:04:57.429885+0000","last_active":"2023-10-16T20:05:01.441360+0000","last_peered":"2023-10-16T20:05:01.441360+0000","last_clean":"2023-10-16T20:05:01.441360+0000","last_became_active":"2023-10-16T20:04:57.429595+0000","last_became_peered":"2023-10-16T20:04:57.429595+0000","last_unstale":"2023-10-16T20:05:01.441360+0000","last_undegraded":"2023-10-16T20:05:01.441360+0000","last_fullsized":"2023-10-16T20:05:01.441360+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:49:52.409593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757637+0000","last_change":"2023-10-16T20:04:57.434436+0000","last_active":"2023-10-16T20:05:01.757637+0000","last_peered":"2023-10-16T20:05:01.757637+0000","last_clean":"2023-10-16T20:05:01.757637+0000","last_became_active":"2023-10-16T20:04:57.433525+0000","last_became_peered":"2023-10-16T20:04:57.433525+0000","last_unstale":"2023-10-16T20:05:01.757637+0000","last_undegraded":"2023-10-16T20:05:01.757637+0000","last_fullsized":"2023-10-16T20:05:01.757637+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:22:12.692981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.437681+0000","last_change":"2023-10-16T20:04:57.421933+0000","last_active":"2023-10-16T20:05:01.437681+0000","last_peered":"2023-10-16T20:05:01.437681+0000","last_clean":"2023-10-16T20:05:01.437681+0000","last_became_active":"2023-10-16T20:04:57.421342+0000","last_became_peered":"2023-10-16T20:04:57.421342+0000","last_unstale":"2023-10-16T20:05:01.437681+0000","last_undegraded":"2023-10-16T20:05:01.437681+0000","last_fullsized":"2023-10-16T20:05:01.437681+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:54:09.036700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.757757+0000","last_change":"2023-10-16T20:04:57.434235+0000","last_active":"2023-10-16T20:05:01.757757+0000","last_peered":"2023-10-16T20:05:01.757757+0000","last_clean":"2023-10-16T20:05:01.757757+0000","last_became_active":"2023-10-16T20:04:57.433720+0000","last_became_peered":"2023-10-16T20:04:57.433720+0000","last_unstale":"2023-10-16T20:05:01.757757+0000","last_undegraded":"2023-10-16T20:05:01.757757+0000","last_fullsized":"2023-10-16T20:05:01.757757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:11:56.834052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.438139+0000","last_change":"2023-10-16T20:04:57.431571+0000","last_active":"2023-10-16T20:05:01.438139+0000","last_peered":"2023-10-16T20:05:01.438139+0000","last_clean":"2023-10-16T20:05:01.438139+0000","last_became_active":"2023-10-16T20:04:57.429514+0000","last_became_peered":"2023-10-16T20:04:57.429514+0000","last_unstale":"2023-10-16T20:05:01.438139+0000","last_undegraded":"2023-10-16T20:05:01.438139+0000","last_fullsized":"2023-10-16T20:05:01.438139+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:31:34.884161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.464508+0000","last_change":"2023-10-16T20:04:57.412857+0000","last_active":"2023-10-16T20:05:01.464508+0000","last_peered":"2023-10-16T20:05:01.464508+0000","last_clean":"2023-10-16T20:05:01.464508+0000","last_became_active":"2023-10-16T20:04:57.412587+0000","last_became_peered":"2023-10-16T20:04:57.412587+0000","last_unstale":"2023-10-16T20:05:01.464508+0000","last_undegraded":"2023-10-16T20:05:01.464508+0000","last_fullsized":"2023-10-16T20:05:01.464508+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:42:47.128122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.471650+0000","last_change":"2023-10-16T20:04:57.423730+0000","last_active":"2023-10-16T20:05:01.471650+0000","last_peered":"2023-10-16T20:05:01.471650+0000","last_clean":"2023-10-16T20:05:01.471650+0000","last_became_active":"2023-10-16T20:04:57.423300+0000","last_became_peered":"2023-10-16T20:04:57.423300+0000","last_unstale":"2023-10-16T20:05:01.471650+0000","last_undegraded":"2023-10-16T20:05:01.471650+0000","last_fullsized":"2023-10-16T20:05:01.471650+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:27:55.972790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-16T20:05:01.758256+0000","last_change":"2023-10-16T20:04:57.442158+0000","last_active":"2023-10-16T20:05:01.758256+0000","last_peered":"2023-10-16T20:05:01.758256+0000","last_clean":"2023-10-16T20:05:01.758256+0000","last_became_active":"2023-10-16T20:04:57.438707+0000","last_became_peered":"2023-10-16T20:04:57.438707+0000","last_unstale":"2023-10-16T20:05:01.758256+0000","last_undegraded":"2023-10-16T20:05:01.758256+0000","last_fullsized":"2023-10-16T20:05:01.758256+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:56.374604+0000","last_clean_scrub_stamp":"2023-10-16T20:04:56.374604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:51:08.154913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509450,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":880640,"data_stored":701023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]}]},{"osd":6,"up_from":15,"seq":64424509451,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":294912,"data_stored":113914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":110163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.264}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4159999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]}]},{"osd":4,"up_from":15,"seq":64424509449,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":294912,"data_stored":110199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]}]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":294912,"data_stored":113914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]}]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":110073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]}]},{"osd":1,"up_from":15,"seq":64424509450,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":290816,"data_stored":110233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":888832,"data_stored":701079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.9289999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.1520000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.3440000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.5250000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.9990000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.0289999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T20:05:21.335 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-16T20:05:21.335 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-16T20:05:21.335 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-16T20:05:21.336 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-16T20:05:21.513 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.510+0000 7f9b8a180640 1 Processor -- start 2023-10-16T20:05:21.513 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.510+0000 7f9b8a180640 1 -- start start 2023-10-16T20:05:21.515 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.514+0000 7f9b8a180640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f9b8415cb70 0x7f9b84161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:21.515 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.514+0000 7f9b8a180640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b840cdad0 0x7f9b840cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:21.515 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.514+0000 7f9b8a180640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b840b2190 con 0x7f9b840cff40 2023-10-16T20:05:21.515 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.514+0000 7f9b8a180640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b840d03d0 con 0x7f9b8415cb70 2023-10-16T20:05:21.515 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.514+0000 7f9b8a180640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b841615a0 con 0x7f9b840cdad0 2023-10-16T20:05:21.516 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.514+0000 7f9b837fe640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f9b8415cb70 0x7f9b84161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:21.516 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.514+0000 7f9b837fe640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f9b8415cb70 0x7f9b84161060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40670/0 (socket says 172.21.15.88:40670) 2023-10-16T20:05:21.516 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.514+0000 7f9b83fff640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f9b840cff40 0x7f9b840b7c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:39282/0 (socket says 172.21.15.88:39282) 2023-10-16T20:05:21.516 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.514+0000 7f9b837fe640 1 -- 172.21.15.88:0/260824247 learned_addr learned my addr 172.21.15.88:0/260824247 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:21.516 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.514+0000 7f9b82ffd640 1 --2- 172.21.15.88:0/260824247 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b840cdad0 0x7f9b840cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:21.517 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.514+0000 7f9b837fe640 1 -- 172.21.15.88:0/260824247 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b840cdad0 msgr2=0x7f9b840cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:21.517 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.514+0000 7f9b837fe640 1 --2- 172.21.15.88:0/260824247 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b840cdad0 0x7f9b840cfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:21.517 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.514+0000 7f9b827fc640 1 -- 172.21.15.88:0/260824247 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4096623563 0 0) 0x7f9b840b2190 con 0x7f9b840cff40 2023-10-16T20:05:21.517 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.514+0000 7f9b837fe640 1 -- 172.21.15.88:0/260824247 >> v1:172.21.15.88:6789/0 conn(0x7f9b840cff40 legacy=0x7f9b840b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:21.517 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.514+0000 7f9b837fe640 1 -- 172.21.15.88:0/260824247 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8415df60 con 0x7f9b8415cb70 2023-10-16T20:05:21.519 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.518+0000 7f9b837fe640 1 --2- 172.21.15.88:0/260824247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b8415cb70 0x7f9b84161060 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9b74009870 tx=0x7f9b7401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcade511067fbf3c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:21.519 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.518+0000 7f9b827fc640 1 -- 172.21.15.88:0/260824247 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b740a7070 con 0x7f9b8415cb70 2023-10-16T20:05:21.519 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.518+0000 7f9b827fc640 1 -- 172.21.15.88:0/260824247 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b740ab430 con 0x7f9b8415cb70 2023-10-16T20:05:21.519 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.518+0000 7f9b827fc640 1 -- 172.21.15.88:0/260824247 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b740ab7c0 con 0x7f9b8415cb70 2023-10-16T20:05:21.520 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.518+0000 7f9b8a180640 1 -- 172.21.15.88:0/260824247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b8415cb70 msgr2=0x7f9b84161060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:21.520 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.518+0000 7f9b8a180640 1 --2- 172.21.15.88:0/260824247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b8415cb70 0x7f9b84161060 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9b74009870 tx=0x7f9b7401c540 comp rx=0 tx=0).stop 2023-10-16T20:05:21.520 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.518+0000 7f9b8a180640 1 -- 172.21.15.88:0/260824247 shutdown_connections 2023-10-16T20:05:21.520 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.518+0000 7f9b8a180640 1 --2- 172.21.15.88:0/260824247 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b840cdad0 0x7f9b840cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:21.520 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.518+0000 7f9b8a180640 1 --2- 172.21.15.88:0/260824247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b8415cb70 0x7f9b84161060 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:21.521 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.518+0000 7f9b8a180640 1 -- 172.21.15.88:0/260824247 >> 172.21.15.88:0/260824247 conn(0x7f9b840719c0 msgr2=0x7f9b84071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:21.521 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.518+0000 7f9b8a180640 1 -- 172.21.15.88:0/260824247 shutdown_connections 2023-10-16T20:05:21.521 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.518+0000 7f9b8a180640 1 -- 172.21.15.88:0/260824247 wait complete. 2023-10-16T20:05:21.523 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.518+0000 7f9b8a180640 1 Processor -- start 2023-10-16T20:05:21.523 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.518+0000 7f9b8a180640 1 -- start start 2023-10-16T20:05:21.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.522+0000 7f9b8a180640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b840cdad0 0x7f9b84154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:21.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.522+0000 7f9b8a180640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b840cff40 0x7f9b8415b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:21.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.522+0000 7f9b8a180640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b8415cb70 0x7f9b8415b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:21.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.522+0000 7f9b8a180640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8415fa00 con 0x7f9b840cdad0 2023-10-16T20:05:21.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.522+0000 7f9b8a180640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b84157790 con 0x7f9b840cff40 2023-10-16T20:05:21.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.522+0000 7f9b8a180640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b84157a70 con 0x7f9b8415cb70 2023-10-16T20:05:21.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.522+0000 7f9b837fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b840cdad0 0x7f9b84154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:21.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.522+0000 7f9b837fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b840cdad0 0x7f9b84154c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40686/0 (socket says 172.21.15.88:40686) 2023-10-16T20:05:21.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.522+0000 7f9b837fe640 1 -- 172.21.15.88:0/1314769853 learned_addr learned my addr 172.21.15.88:0/1314769853 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:21.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.522+0000 7f9b83fff640 1 --2- 172.21.15.88:0/1314769853 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b8415cb70 0x7f9b8415b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:21.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.522+0000 7f9b82ffd640 1 --2- 172.21.15.88:0/1314769853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b840cff40 0x7f9b8415b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:21.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.522+0000 7f9b837fe640 1 -- 172.21.15.88:0/1314769853 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b8415cb70 msgr2=0x7f9b8415b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:21.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.522+0000 7f9b837fe640 1 --2- 172.21.15.88:0/1314769853 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b8415cb70 0x7f9b8415b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:21.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.522+0000 7f9b837fe640 1 -- 172.21.15.88:0/1314769853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b840cff40 msgr2=0x7f9b8415b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:21.527 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.522+0000 7f9b837fe640 1 --2- 172.21.15.88:0/1314769853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b840cff40 0x7f9b8415b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:21.527 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.522+0000 7f9b837fe640 1 -- 172.21.15.88:0/1314769853 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b740094d0 con 0x7f9b840cdad0 2023-10-16T20:05:21.527 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.522+0000 7f9b82ffd640 1 --2- 172.21.15.88:0/1314769853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b840cff40 0x7f9b8415b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:21.527 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.522+0000 7f9b837fe640 1 --2- 172.21.15.88:0/1314769853 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b840cdad0 0x7f9b84154c00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9b740ad040 tx=0x7f9b740192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:21.527 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.522+0000 7f9b6bfff640 1 -- 172.21.15.88:0/1314769853 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b740b4620 con 0x7f9b840cdad0 2023-10-16T20:05:21.528 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.522+0000 7f9b6bfff640 1 -- 172.21.15.88:0/1314769853 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b740b4780 con 0x7f9b840cdad0 2023-10-16T20:05:21.528 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.522+0000 7f9b8a180640 1 -- 172.21.15.88:0/1314769853 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b84157d50 con 0x7f9b840cdad0 2023-10-16T20:05:21.528 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.522+0000 7f9b6bfff640 1 -- 172.21.15.88:0/1314769853 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b740b4b30 con 0x7f9b840cdad0 2023-10-16T20:05:21.528 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.522+0000 7f9b8a180640 1 -- 172.21.15.88:0/1314769853 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b84158310 con 0x7f9b840cdad0 2023-10-16T20:05:21.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.522+0000 7f9b8a180640 1 -- 172.21.15.88:0/1314769853 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b8404e9c0 con 0x7f9b840cdad0 2023-10-16T20:05:21.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.522+0000 7f9b6bfff640 1 -- 172.21.15.88:0/1314769853 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9b740b4c90 con 0x7f9b840cdad0 2023-10-16T20:05:21.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.526+0000 7f9b6bfff640 1 --2- 172.21.15.88:0/1314769853 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9b700664d0 0x7f9b70068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:21.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.526+0000 7f9b6bfff640 1 -- 172.21.15.88:0/1314769853 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9b7411e310 con 0x7f9b840cdad0 2023-10-16T20:05:21.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.526+0000 7f9b82ffd640 1 --2- 172.21.15.88:0/1314769853 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9b700664d0 0x7f9b70068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:21.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.526+0000 7f9b82ffd640 1 --2- 172.21.15.88:0/1314769853 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9b700664d0 0x7f9b70068990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9b840b7bf0 tx=0x7f9b78008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:21.531 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.530+0000 7f9b6bfff640 1 -- 172.21.15.88:0/1314769853 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9b740eb400 con 0x7f9b840cdad0 2023-10-16T20:05:21.710 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.710+0000 7f9b8a180640 1 -- 172.21.15.88:0/1314769853 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9b8415df60 con 0x7f9b840cdad0 2023-10-16T20:05:21.711 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.710+0000 7f9b6bfff640 1 -- 172.21.15.88:0/1314769853 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f9b8415df60 con 0x7f9b840cdad0 2023-10-16T20:05:21.711 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:21.711 INFO:teuthology.orchestra.run.smithi088.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-16T20:05:21.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.710+0000 7f9b8a180640 1 -- 172.21.15.88:0/1314769853 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9b700664d0 msgr2=0x7f9b70068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:21.713 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.710+0000 7f9b8a180640 1 --2- 172.21.15.88:0/1314769853 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9b700664d0 0x7f9b70068990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9b840b7bf0 tx=0x7f9b78008040 comp rx=0 tx=0).stop 2023-10-16T20:05:21.713 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.710+0000 7f9b8a180640 1 -- 172.21.15.88:0/1314769853 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b840cdad0 msgr2=0x7f9b84154c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:21.713 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.710+0000 7f9b8a180640 1 --2- 172.21.15.88:0/1314769853 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b840cdad0 0x7f9b84154c00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9b740ad040 tx=0x7f9b740192e0 comp rx=0 tx=0).stop 2023-10-16T20:05:21.713 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.710+0000 7f9b8a180640 1 -- 172.21.15.88:0/1314769853 shutdown_connections 2023-10-16T20:05:21.713 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.710+0000 7f9b8a180640 1 --2- 172.21.15.88:0/1314769853 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9b700664d0 0x7f9b70068990 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:21.714 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.710+0000 7f9b8a180640 1 --2- 172.21.15.88:0/1314769853 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b8415cb70 0x7f9b8415b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:21.714 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.710+0000 7f9b8a180640 1 --2- 172.21.15.88:0/1314769853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b840cff40 0x7f9b8415b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:21.714 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.710+0000 7f9b8a180640 1 --2- 172.21.15.88:0/1314769853 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b840cdad0 0x7f9b84154c00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:21.715 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.710+0000 7f9b8a180640 1 -- 172.21.15.88:0/1314769853 >> 172.21.15.88:0/1314769853 conn(0x7f9b840719c0 msgr2=0x7f9b84167b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:21.715 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.710+0000 7f9b8a180640 1 -- 172.21.15.88:0/1314769853 shutdown_connections 2023-10-16T20:05:21.716 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.710+0000 7f9b8a180640 1 -- 172.21.15.88:0/1314769853 wait complete. 2023-10-16T20:05:21.731 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-16T20:05:21.732 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:05:21.869 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.866+0000 7f1e367aa640 1 Processor -- start 2023-10-16T20:05:21.869 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.866+0000 7f1e367aa640 1 -- start start 2023-10-16T20:05:21.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.870+0000 7f1e367aa640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1e30160cc0 0x7f1e300ca800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:21.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.870+0000 7f1e367aa640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e301610a0 0x7f1e300ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:21.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.870+0000 7f1e367aa640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1e300ae710 0x7f1e300cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:21.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.870+0000 7f1e367aa640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e300cc570 con 0x7f1e30160cc0 2023-10-16T20:05:21.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.870+0000 7f1e367aa640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e300cc850 con 0x7f1e300ae710 2023-10-16T20:05:21.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.870+0000 7f1e367aa640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e300ccb30 con 0x7f1e301610a0 2023-10-16T20:05:21.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.870+0000 7f1e2ffff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1e30160cc0 0x7f1e300ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:21.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.870+0000 7f1e2ffff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1e30160cc0 0x7f1e300ca800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40694/0 (socket says 172.21.15.88:40694) 2023-10-16T20:05:21.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.870+0000 7f1e2ffff640 1 -- 172.21.15.88:0/2540340473 learned_addr learned my addr 172.21.15.88:0/2540340473 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:21.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.870+0000 7f1e2f7fe640 1 --2- 172.21.15.88:0/2540340473 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e301610a0 0x7f1e300ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:21.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.870+0000 7f1e34d47640 1 --2- 172.21.15.88:0/2540340473 >> v2:172.21.15.94:3300/0 conn(0x7f1e300ae710 0x7f1e300cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:21.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.870+0000 7f1e2ffff640 1 -- 172.21.15.88:0/2540340473 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e301610a0 msgr2=0x7f1e300ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:21.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.870+0000 7f1e2ffff640 1 --2- 172.21.15.88:0/2540340473 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e301610a0 0x7f1e300ae1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:21.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.870+0000 7f1e2ffff640 1 -- 172.21.15.88:0/2540340473 >> v2:172.21.15.94:3300/0 conn(0x7f1e300ae710 msgr2=0x7f1e300cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:21.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.870+0000 7f1e2ffff640 1 --2- 172.21.15.88:0/2540340473 >> v2:172.21.15.94:3300/0 conn(0x7f1e300ae710 0x7f1e300cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:21.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.870+0000 7f1e2ffff640 1 -- 172.21.15.88:0/2540340473 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e300cce10 con 0x7f1e30160cc0 2023-10-16T20:05:21.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.870+0000 7f1e34d47640 1 --2- 172.21.15.88:0/2540340473 >> v2:172.21.15.94:3300/0 conn(0x7f1e300ae710 0x7f1e300cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:21.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.874+0000 7f1e2ffff640 1 --2- 172.21.15.88:0/2540340473 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e30160cc0 0x7f1e300ca800 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1e3006afc0 tx=0x7f1e2001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ce1f727bae6152 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:21.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.874+0000 7f1e2effd640 1 -- 172.21.15.88:0/2540340473 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e200ac600 con 0x7f1e30160cc0 2023-10-16T20:05:21.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.874+0000 7f1e2effd640 1 -- 172.21.15.88:0/2540340473 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e200a8050 con 0x7f1e30160cc0 2023-10-16T20:05:21.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.874+0000 7f1e2effd640 1 -- 172.21.15.88:0/2540340473 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e200ac980 con 0x7f1e30160cc0 2023-10-16T20:05:21.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.874+0000 7f1e367aa640 1 -- 172.21.15.88:0/2540340473 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e30160cc0 msgr2=0x7f1e300ca800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:21.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.874+0000 7f1e367aa640 1 --2- 172.21.15.88:0/2540340473 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e30160cc0 0x7f1e300ca800 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1e3006afc0 tx=0x7f1e2001c4d0 comp rx=0 tx=0).stop 2023-10-16T20:05:21.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.874+0000 7f1e367aa640 1 -- 172.21.15.88:0/2540340473 shutdown_connections 2023-10-16T20:05:21.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.874+0000 7f1e367aa640 1 --2- 172.21.15.88:0/2540340473 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e301610a0 0x7f1e300ae1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:21.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.874+0000 7f1e367aa640 1 --2- 172.21.15.88:0/2540340473 >> v2:172.21.15.94:3300/0 conn(0x7f1e300ae710 0x7f1e300cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:21.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.874+0000 7f1e367aa640 1 --2- 172.21.15.88:0/2540340473 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e30160cc0 0x7f1e300ca800 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:21.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.874+0000 7f1e367aa640 1 -- 172.21.15.88:0/2540340473 >> 172.21.15.88:0/2540340473 conn(0x7f1e300ce5e0 msgr2=0x7f1e300ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:21.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.874+0000 7f1e367aa640 1 -- 172.21.15.88:0/2540340473 shutdown_connections 2023-10-16T20:05:21.878 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.874+0000 7f1e367aa640 1 -- 172.21.15.88:0/2540340473 wait complete. 2023-10-16T20:05:21.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.874+0000 7f1e367aa640 1 Processor -- start 2023-10-16T20:05:21.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.874+0000 7f1e367aa640 1 -- start start 2023-10-16T20:05:21.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.878+0000 7f1e367aa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e300ae710 0x7f1e3015aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:21.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.878+0000 7f1e367aa640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e30160cc0 0x7f1e30159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:21.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.878+0000 7f1e367aa640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e301610a0 0x7f1e30151780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:21.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.878+0000 7f1e367aa640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e30153730 con 0x7f1e301610a0 2023-10-16T20:05:21.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.878+0000 7f1e367aa640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e300b4250 con 0x7f1e300ae710 2023-10-16T20:05:21.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.878+0000 7f1e367aa640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e300b18b0 con 0x7f1e30160cc0 2023-10-16T20:05:21.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.878+0000 7f1e34d47640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e301610a0 0x7f1e30151780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:21.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.878+0000 7f1e34d47640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e301610a0 0x7f1e30151780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40702/0 (socket says 172.21.15.88:40702) 2023-10-16T20:05:21.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.878+0000 7f1e34d47640 1 -- 172.21.15.88:0/2689010905 learned_addr learned my addr 172.21.15.88:0/2689010905 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:21.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.878+0000 7f1e2f7fe640 1 --2- 172.21.15.88:0/2689010905 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e30160cc0 0x7f1e30159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:21.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.878+0000 7f1e2ffff640 1 --2- 172.21.15.88:0/2689010905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e300ae710 0x7f1e3015aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:21.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.878+0000 7f1e34d47640 1 -- 172.21.15.88:0/2689010905 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e30160cc0 msgr2=0x7f1e30159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:21.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.878+0000 7f1e34d47640 1 --2- 172.21.15.88:0/2689010905 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e30160cc0 0x7f1e30159f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:21.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.878+0000 7f1e34d47640 1 -- 172.21.15.88:0/2689010905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e300ae710 msgr2=0x7f1e3015aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:21.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.878+0000 7f1e34d47640 1 --2- 172.21.15.88:0/2689010905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e300ae710 0x7f1e3015aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:21.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.878+0000 7f1e34d47640 1 -- 172.21.15.88:0/2689010905 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e200094d0 con 0x7f1e301610a0 2023-10-16T20:05:21.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.878+0000 7f1e34d47640 1 --2- 172.21.15.88:0/2689010905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e301610a0 0x7f1e30151780 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1e28007ef0 tx=0x7f1e2800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:21.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.878+0000 7f1e2cff9640 1 -- 172.21.15.88:0/2689010905 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e28003ba0 con 0x7f1e301610a0 2023-10-16T20:05:21.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.878+0000 7f1e2cff9640 1 -- 172.21.15.88:0/2689010905 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e2800f040 con 0x7f1e301610a0 2023-10-16T20:05:21.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.878+0000 7f1e2cff9640 1 -- 172.21.15.88:0/2689010905 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2800de80 con 0x7f1e301610a0 2023-10-16T20:05:21.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.878+0000 7f1e367aa640 1 -- 172.21.15.88:0/2689010905 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e30155ff0 con 0x7f1e301610a0 2023-10-16T20:05:21.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.878+0000 7f1e367aa640 1 -- 172.21.15.88:0/2689010905 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e300b4780 con 0x7f1e301610a0 2023-10-16T20:05:21.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.878+0000 7f1e367aa640 1 -- 172.21.15.88:0/2689010905 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e3004e9c0 con 0x7f1e301610a0 2023-10-16T20:05:21.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.878+0000 7f1e2cff9640 1 -- 172.21.15.88:0/2689010905 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1e28004030 con 0x7f1e301610a0 2023-10-16T20:05:21.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.882+0000 7f1e2cff9640 1 --2- 172.21.15.88:0/2689010905 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1e1c0664a0 0x7f1e1c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:21.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.882+0000 7f1e2cff9640 1 -- 172.21.15.88:0/2689010905 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1e28084380 con 0x7f1e301610a0 2023-10-16T20:05:21.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.882+0000 7f1e2ffff640 1 --2- 172.21.15.88:0/2689010905 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1e1c0664a0 0x7f1e1c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:21.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.882+0000 7f1e2ffff640 1 --2- 172.21.15.88:0/2689010905 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1e1c0664a0 0x7f1e1c068960 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1e200ae000 tx=0x7f1e200a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:21.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:21.886+0000 7f1e2cff9640 1 -- 172.21.15.88:0/2689010905 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1e280515b0 con 0x7f1e301610a0 2023-10-16T20:05:22.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.038+0000 7f1e367aa640 1 -- 172.21.15.88:0/2689010905 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e30161710 con 0x7f1e301610a0 2023-10-16T20:05:22.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.038+0000 7f1e2cff9640 1 -- 172.21.15.88:0/2689010905 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18274 (secure 0 0 0) 0x7f1e28054bd0 con 0x7f1e301610a0 2023-10-16T20:05:22.039 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:22.040 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:05:01.835630+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T20:04:54.442487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T20:04:55.766641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T20:04:59.475830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:05:22.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.038+0000 7f1e367aa640 1 -- 172.21.15.88:0/2689010905 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1e1c0664a0 msgr2=0x7f1e1c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:22.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.038+0000 7f1e367aa640 1 --2- 172.21.15.88:0/2689010905 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1e1c0664a0 0x7f1e1c068960 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1e200ae000 tx=0x7f1e200a7000 comp rx=0 tx=0).stop 2023-10-16T20:05:22.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.038+0000 7f1e367aa640 1 -- 172.21.15.88:0/2689010905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e301610a0 msgr2=0x7f1e30151780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:22.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.038+0000 7f1e367aa640 1 --2- 172.21.15.88:0/2689010905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e301610a0 0x7f1e30151780 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1e28007ef0 tx=0x7f1e2800d4a0 comp rx=0 tx=0).stop 2023-10-16T20:05:22.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.038+0000 7f1e367aa640 1 -- 172.21.15.88:0/2689010905 shutdown_connections 2023-10-16T20:05:22.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.038+0000 7f1e367aa640 1 --2- 172.21.15.88:0/2689010905 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1e1c0664a0 0x7f1e1c068960 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:22.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.038+0000 7f1e367aa640 1 --2- 172.21.15.88:0/2689010905 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e30160cc0 0x7f1e30159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:22.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.038+0000 7f1e367aa640 1 --2- 172.21.15.88:0/2689010905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e301610a0 0x7f1e30151780 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:22.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.038+0000 7f1e367aa640 1 --2- 172.21.15.88:0/2689010905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e300ae710 0x7f1e3015aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:22.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.038+0000 7f1e367aa640 1 -- 172.21.15.88:0/2689010905 >> 172.21.15.88:0/2689010905 conn(0x7f1e300ce5e0 msgr2=0x7f1e30156430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:22.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.042+0000 7f1e367aa640 1 -- 172.21.15.88:0/2689010905 shutdown_connections 2023-10-16T20:05:22.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.042+0000 7f1e367aa640 1 -- 172.21.15.88:0/2689010905 wait complete. 2023-10-16T20:05:22.060 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T20:04:54.442487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T20:04:55.766641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T20:04:59.475830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T20:05:22.061 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:05:22.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.198+0000 7f163c6c9640 1 Processor -- start 2023-10-16T20:05:22.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.198+0000 7f163c6c9640 1 -- start start 2023-10-16T20:05:22.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.202+0000 7f163c6c9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f16340b0da0 0x7f1634159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:22.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.202+0000 7f163c6c9640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f163415ad30 con 0x7f163415a1f0 2023-10-16T20:05:22.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.202+0000 7f163c6c9640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16340b4640 con 0x7f16340b4260 2023-10-16T20:05:22.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.202+0000 7f163c6c9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f163415a770 con 0x7f16340b0da0 2023-10-16T20:05:22.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.202+0000 7f163ac66640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f163415a1f0 0x7f163406d3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:60756/0 (socket says 172.21.15.88:60756) 2023-10-16T20:05:22.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.202+0000 7f163ac66640 1 -- 172.21.15.88:0/3005210400 learned_addr learned my addr 172.21.15.88:0/3005210400 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:22.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.202+0000 7f1639c64640 1 --2- 172.21.15.88:0/3005210400 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f16340b0da0 0x7f1634159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:22.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.202+0000 7f1639463640 1 -- 172.21.15.88:0/3005210400 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4067010046 0 0) 0x7f163415ad30 con 0x7f163415a1f0 2023-10-16T20:05:22.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.202+0000 7f1639463640 1 -- 172.21.15.88:0/3005210400 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1620003530 con 0x7f163415a1f0 2023-10-16T20:05:22.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.202+0000 7f1639463640 1 -- 172.21.15.88:0/3005210400 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3499808106 0 0) 0x7f16340b4640 con 0x7f16340b4260 2023-10-16T20:05:22.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.202+0000 7f1639463640 1 -- 172.21.15.88:0/3005210400 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f163415ad30 con 0x7f16340b4260 2023-10-16T20:05:22.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.202+0000 7f1639463640 1 -- 172.21.15.88:0/3005210400 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4032955817 0 0) 0x7f1620003530 con 0x7f163415a1f0 2023-10-16T20:05:22.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.202+0000 7f1639c64640 1 -- 172.21.15.88:0/3005210400 >> v1:172.21.15.94:6789/0 conn(0x7f16340b4260 legacy=0x7f163415dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:22.206 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.202+0000 7f1639c64640 1 -- 172.21.15.88:0/3005210400 >> v1:172.21.15.88:6789/0 conn(0x7f163415a1f0 legacy=0x7f163406d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:22.206 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.202+0000 7f1639c64640 1 -- 172.21.15.88:0/3005210400 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163415aa50 con 0x7f16340b0da0 2023-10-16T20:05:22.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.206+0000 7f1639c64640 1 --2- 172.21.15.88:0/3005210400 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f16340b0da0 0x7f1634159cb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f16240097f0 tx=0x7f162401c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b90922492939c09 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:22.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.206+0000 7f1639463640 1 -- 172.21.15.88:0/3005210400 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162c001a00 con 0x7f16340b0da0 2023-10-16T20:05:22.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.206+0000 7f1639463640 1 -- 172.21.15.88:0/3005210400 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16240a7050 con 0x7f16340b0da0 2023-10-16T20:05:22.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.206+0000 7f1639463640 1 -- 172.21.15.88:0/3005210400 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16240ab620 con 0x7f16340b0da0 2023-10-16T20:05:22.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.206+0000 7f163c6c9640 1 -- 172.21.15.88:0/3005210400 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f16340b0da0 msgr2=0x7f1634159cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:22.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.206+0000 7f163c6c9640 1 --2- 172.21.15.88:0/3005210400 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f16340b0da0 0x7f1634159cb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f16240097f0 tx=0x7f162401c350 comp rx=0 tx=0).stop 2023-10-16T20:05:22.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.206+0000 7f163c6c9640 1 -- 172.21.15.88:0/3005210400 shutdown_connections 2023-10-16T20:05:22.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.206+0000 7f163c6c9640 1 --2- 172.21.15.88:0/3005210400 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f16340b0da0 0x7f1634159cb0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:22.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.206+0000 7f163c6c9640 1 -- 172.21.15.88:0/3005210400 >> 172.21.15.88:0/3005210400 conn(0x7f16340cb080 msgr2=0x7f16340cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:22.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.206+0000 7f163c6c9640 1 -- 172.21.15.88:0/3005210400 shutdown_connections 2023-10-16T20:05:22.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.206+0000 7f163c6c9640 1 -- 172.21.15.88:0/3005210400 wait complete. 2023-10-16T20:05:22.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.206+0000 7f163c6c9640 1 Processor -- start 2023-10-16T20:05:22.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.206+0000 7f163c6c9640 1 -- start start 2023-10-16T20:05:22.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.210+0000 7f163c6c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16340b0da0 0x7f1634151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:22.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.210+0000 7f163c6c9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f16340b4260 0x7f16341499a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:22.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.210+0000 7f163c6c9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f163415a1f0 0x7f1634148440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:22.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.210+0000 7f163c6c9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16341574b0 con 0x7f163415a1f0 2023-10-16T20:05:22.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.210+0000 7f163c6c9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16341526d0 con 0x7f16340b0da0 2023-10-16T20:05:22.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.210+0000 7f163c6c9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1634149ee0 con 0x7f16340b4260 2023-10-16T20:05:22.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.210+0000 7f163ac66640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f163415a1f0 0x7f1634148440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:22.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.210+0000 7f163ac66640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f163415a1f0 0x7f1634148440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38010/0 (socket says 172.21.15.88:38010) 2023-10-16T20:05:22.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.210+0000 7f163ac66640 1 -- 172.21.15.88:0/111708484 learned_addr learned my addr 172.21.15.88:0/111708484 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:22.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.210+0000 7f163ac66640 1 -- 172.21.15.88:0/111708484 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f16340b4260 msgr2=0x7f16341499a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:22.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.210+0000 7f163ac66640 1 --2- 172.21.15.88:0/111708484 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f16340b4260 0x7f16341499a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:22.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.210+0000 7f163ac66640 1 -- 172.21.15.88:0/111708484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16340b0da0 msgr2=0x7f1634151a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:22.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.210+0000 7f163ac66640 1 --2- 172.21.15.88:0/111708484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16340b0da0 0x7f1634151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:22.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.210+0000 7f163ac66640 1 -- 172.21.15.88:0/111708484 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1624009420 con 0x7f163415a1f0 2023-10-16T20:05:22.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.210+0000 7f163ac66640 1 --2- 172.21.15.88:0/111708484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f163415a1f0 0x7f1634148440 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f162c001f90 tx=0x7f162c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:22.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.210+0000 7f162affd640 1 -- 172.21.15.88:0/111708484 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162c012400 con 0x7f163415a1f0 2023-10-16T20:05:22.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.210+0000 7f162affd640 1 -- 172.21.15.88:0/111708484 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f162c012560 con 0x7f163415a1f0 2023-10-16T20:05:22.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.210+0000 7f163c6c9640 1 -- 172.21.15.88:0/111708484 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1634152110 con 0x7f163415a1f0 2023-10-16T20:05:22.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.210+0000 7f162affd640 1 -- 172.21.15.88:0/111708484 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162c012910 con 0x7f163415a1f0 2023-10-16T20:05:22.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.210+0000 7f163c6c9640 1 -- 172.21.15.88:0/111708484 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16341570d0 con 0x7f163415a1f0 2023-10-16T20:05:22.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.210+0000 7f163c6c9640 1 -- 172.21.15.88:0/111708484 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16341575f0 con 0x7f163415a1f0 2023-10-16T20:05:22.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.210+0000 7f162affd640 1 -- 172.21.15.88:0/111708484 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f162c012a70 con 0x7f163415a1f0 2023-10-16T20:05:22.217 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.214+0000 7f162affd640 1 --2- 172.21.15.88:0/111708484 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f16180664a0 0x7f1618068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:22.217 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.214+0000 7f162affd640 1 -- 172.21.15.88:0/111708484 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f162c084a60 con 0x7f163415a1f0 2023-10-16T20:05:22.217 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.214+0000 7f163a465640 1 --2- 172.21.15.88:0/111708484 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f16180664a0 0x7f1618068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:22.217 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.214+0000 7f163a465640 1 --2- 172.21.15.88:0/111708484 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f16180664a0 0x7f1618068960 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1630007fd0 tx=0x7f1630007960 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:22.218 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.218+0000 7f162affd640 1 -- 172.21.15.88:0/111708484 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f16341575f0 con 0x7f163415a1f0 2023-10-16T20:05:22.368 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.366+0000 7f163c6c9640 1 -- 172.21.15.88:0/111708484 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f16341575f0 con 0x7f163415a1f0 2023-10-16T20:05:22.368 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.366+0000 7f162affd640 1 -- 172.21.15.88:0/111708484 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f16341575f0 con 0x7f163415a1f0 2023-10-16T20:05:22.369 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:05:22.370 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.370+0000 7f163c6c9640 1 -- 172.21.15.88:0/111708484 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f16180664a0 msgr2=0x7f1618068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:22.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.370+0000 7f163c6c9640 1 --2- 172.21.15.88:0/111708484 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f16180664a0 0x7f1618068960 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1630007fd0 tx=0x7f1630007960 comp rx=0 tx=0).stop 2023-10-16T20:05:22.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.370+0000 7f163c6c9640 1 -- 172.21.15.88:0/111708484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f163415a1f0 msgr2=0x7f1634148440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:22.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.370+0000 7f163c6c9640 1 --2- 172.21.15.88:0/111708484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f163415a1f0 0x7f1634148440 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f162c001f90 tx=0x7f162c00a430 comp rx=0 tx=0).stop 2023-10-16T20:05:22.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.370+0000 7f163c6c9640 1 -- 172.21.15.88:0/111708484 shutdown_connections 2023-10-16T20:05:22.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.370+0000 7f163c6c9640 1 --2- 172.21.15.88:0/111708484 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f16180664a0 0x7f1618068960 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:22.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.370+0000 7f163c6c9640 1 --2- 172.21.15.88:0/111708484 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f16340b4260 0x7f16341499a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:22.372 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.370+0000 7f163c6c9640 1 --2- 172.21.15.88:0/111708484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f163415a1f0 0x7f1634148440 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:22.372 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.370+0000 7f163c6c9640 1 --2- 172.21.15.88:0/111708484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16340b0da0 0x7f1634151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:22.372 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.370+0000 7f163c6c9640 1 -- 172.21.15.88:0/111708484 >> 172.21.15.88:0/111708484 conn(0x7f16340cb080 msgr2=0x7f163406c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:22.372 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.370+0000 7f163c6c9640 1 -- 172.21.15.88:0/111708484 shutdown_connections 2023-10-16T20:05:22.373 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.370+0000 7f163c6c9640 1 -- 172.21.15.88:0/111708484 wait complete. 2023-10-16T20:05:22.390 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T20:05:22.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.530+0000 7fadb566b640 1 Processor -- start 2023-10-16T20:05:22.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.530+0000 7fadb566b640 1 -- start start 2023-10-16T20:05:22.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.530+0000 7fadb566b640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fadb015cb70 0x7fadb00b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:22.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.530+0000 7fadb566b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fadb00b4530 0x7fadb00b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:22.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.530+0000 7fadb566b640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadb00b4ed0 con 0x7fadb00d0bd0 2023-10-16T20:05:22.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.530+0000 7fadb566b640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadb00b7d20 con 0x7fadb015cb70 2023-10-16T20:05:22.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.530+0000 7fadb566b640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadb00b7110 con 0x7fadb00b4530 2023-10-16T20:05:22.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.534+0000 7fadaeffd640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fadb015cb70 0x7fadb00b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:22.535 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.534+0000 7fadaeffd640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fadb015cb70 0x7fadb00b83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38020/0 (socket says 172.21.15.88:38020) 2023-10-16T20:05:22.535 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.534+0000 7fadaeffd640 1 -- 172.21.15.88:0/1703420288 learned_addr learned my addr 172.21.15.88:0/1703420288 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:22.535 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.534+0000 7fada67fc640 1 --2- 172.21.15.88:0/1703420288 >> v2:172.21.15.94:3300/0 conn(0x7fadb00b4530 0x7fadb00b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:22.535 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.534+0000 7fadaeffd640 1 -- 172.21.15.88:0/1703420288 >> v2:172.21.15.94:3300/0 conn(0x7fadb00b4530 msgr2=0x7fadb00b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:22.535 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.534+0000 7fadaeffd640 1 --2- 172.21.15.88:0/1703420288 >> v2:172.21.15.94:3300/0 conn(0x7fadb00b4530 0x7fadb00b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:22.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.534+0000 7fadaeffd640 1 -- 172.21.15.88:0/1703420288 >> v1:172.21.15.94:6789/0 conn(0x7fadb00d0bd0 legacy=0x7fadb00d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:22.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.534+0000 7fadaeffd640 1 -- 172.21.15.88:0/1703420288 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb00b73f0 con 0x7fadb015cb70 2023-10-16T20:05:22.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.534+0000 7fadaeffd640 1 --2- 172.21.15.88:0/1703420288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fadb015cb70 0x7fadb00b83d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fadb00d0500 tx=0x7fada001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d40be12f5fc0055 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:22.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.534+0000 7fadae7fc640 1 -- 172.21.15.88:0/1703420288 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada00a89f0 con 0x7fadb015cb70 2023-10-16T20:05:22.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.534+0000 7fadae7fc640 1 -- 172.21.15.88:0/1703420288 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada00a8b50 con 0x7fadb015cb70 2023-10-16T20:05:22.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.534+0000 7fadae7fc640 1 -- 172.21.15.88:0/1703420288 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada00b0440 con 0x7fadb015cb70 2023-10-16T20:05:22.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.538+0000 7fadb566b640 1 -- 172.21.15.88:0/1703420288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fadb015cb70 msgr2=0x7fadb00b83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:22.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.538+0000 7fadb566b640 1 --2- 172.21.15.88:0/1703420288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fadb015cb70 0x7fadb00b83d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fadb00d0500 tx=0x7fada001ecf0 comp rx=0 tx=0).stop 2023-10-16T20:05:22.539 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.538+0000 7fadb566b640 1 -- 172.21.15.88:0/1703420288 shutdown_connections 2023-10-16T20:05:22.539 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.538+0000 7fadb566b640 1 --2- 172.21.15.88:0/1703420288 >> v2:172.21.15.94:3300/0 conn(0x7fadb00b4530 0x7fadb00b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:22.539 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.538+0000 7fadb566b640 1 --2- 172.21.15.88:0/1703420288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fadb015cb70 0x7fadb00b83d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:22.539 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.538+0000 7fadb566b640 1 -- 172.21.15.88:0/1703420288 >> 172.21.15.88:0/1703420288 conn(0x7fadb006b6e0 msgr2=0x7fadb006baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:22.540 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.538+0000 7fadb566b640 1 -- 172.21.15.88:0/1703420288 shutdown_connections 2023-10-16T20:05:22.540 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.538+0000 7fadb566b640 1 -- 172.21.15.88:0/1703420288 wait complete. 2023-10-16T20:05:22.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.538+0000 7fadb566b640 1 Processor -- start 2023-10-16T20:05:22.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.538+0000 7fadb566b640 1 -- start start 2023-10-16T20:05:22.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.538+0000 7fadb566b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fadb00b4530 0x7fadb01562e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:22.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.538+0000 7fadb566b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fadb00d0bd0 0x7fadb01f2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:22.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.538+0000 7fadb566b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadb015cb70 0x7fadb0158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:22.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.538+0000 7fadb566b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb015dc80 con 0x7fadb00d0bd0 2023-10-16T20:05:22.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.538+0000 7fadb566b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb015df60 con 0x7fadb015cb70 2023-10-16T20:05:22.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.538+0000 7fadb566b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb015f0a0 con 0x7fadb00b4530 2023-10-16T20:05:22.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.538+0000 7fada67fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fadb00d0bd0 0x7fadb01f2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:22.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.538+0000 7fada67fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fadb00d0bd0 0x7fadb01f2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38022/0 (socket says 172.21.15.88:38022) 2023-10-16T20:05:22.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.538+0000 7fada67fc640 1 -- 172.21.15.88:0/2606436250 learned_addr learned my addr 172.21.15.88:0/2606436250 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:22.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.542+0000 7fadaeffd640 1 --2- 172.21.15.88:0/2606436250 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fadb00b4530 0x7fadb01562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:22.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.542+0000 7fada67fc640 1 -- 172.21.15.88:0/2606436250 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fadb00b4530 msgr2=0x7fadb01562e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:22.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.542+0000 7fadaf7fe640 1 --2- 172.21.15.88:0/2606436250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadb015cb70 0x7fadb0158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:22.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.542+0000 7fada67fc640 1 --2- 172.21.15.88:0/2606436250 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fadb00b4530 0x7fadb01562e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:22.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.542+0000 7fada67fc640 1 -- 172.21.15.88:0/2606436250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadb015cb70 msgr2=0x7fadb0158d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:22.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.542+0000 7fada67fc640 1 --2- 172.21.15.88:0/2606436250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadb015cb70 0x7fadb0158d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:22.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.542+0000 7fada67fc640 1 -- 172.21.15.88:0/2606436250 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada00094d0 con 0x7fadb00d0bd0 2023-10-16T20:05:22.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.542+0000 7fadaf7fe640 1 --2- 172.21.15.88:0/2606436250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadb015cb70 0x7fadb0158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:22.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.542+0000 7fadaeffd640 1 --2- 172.21.15.88:0/2606436250 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fadb00b4530 0x7fadb01562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:22.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.542+0000 7fada67fc640 1 --2- 172.21.15.88:0/2606436250 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fadb00d0bd0 0x7fadb01f2140 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fad9800d800 tx=0x7fad9800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:22.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.542+0000 7fada7fff640 1 -- 172.21.15.88:0/2606436250 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9800b6c0 con 0x7fadb00d0bd0 2023-10-16T20:05:22.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.542+0000 7fada7fff640 1 -- 172.21.15.88:0/2606436250 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad9800b820 con 0x7fadb00d0bd0 2023-10-16T20:05:22.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.542+0000 7fada7fff640 1 -- 172.21.15.88:0/2606436250 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9800bb90 con 0x7fadb00d0bd0 2023-10-16T20:05:22.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.542+0000 7fadb566b640 1 -- 172.21.15.88:0/2606436250 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadb015f380 con 0x7fadb00d0bd0 2023-10-16T20:05:22.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.542+0000 7fadb566b640 1 -- 172.21.15.88:0/2606436250 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadb0156820 con 0x7fadb00d0bd0 2023-10-16T20:05:22.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.542+0000 7fadb566b640 1 -- 172.21.15.88:0/2606436250 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadb0072e60 con 0x7fadb00d0bd0 2023-10-16T20:05:22.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.542+0000 7fada7fff640 1 -- 172.21.15.88:0/2606436250 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fad9800bcf0 con 0x7fadb00d0bd0 2023-10-16T20:05:22.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.542+0000 7fada7fff640 1 --2- 172.21.15.88:0/2606436250 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fad900664d0 0x7fad90068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:22.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.542+0000 7fada7fff640 1 -- 172.21.15.88:0/2606436250 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fad980846a0 con 0x7fadb00d0bd0 2023-10-16T20:05:22.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.542+0000 7fadaeffd640 1 --2- 172.21.15.88:0/2606436250 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fad900664d0 0x7fad90068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:22.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.546+0000 7fadaeffd640 1 --2- 172.21.15.88:0/2606436250 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fad900664d0 0x7fad90068990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fada00b2000 tx=0x7fada00aa070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:22.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.546+0000 7fada7fff640 1 -- 172.21.15.88:0/2606436250 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fad98051820 con 0x7fadb00d0bd0 2023-10-16T20:05:22.698 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.698+0000 7fadb566b640 1 -- 172.21.15.88:0/2606436250 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fadb004e9c0 con 0x7fadb00d0bd0 2023-10-16T20:05:22.699 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.698+0000 7fada7fff640 1 -- 172.21.15.88:0/2606436250 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fad98054e40 con 0x7fadb00d0bd0 2023-10-16T20:05:22.700 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:05:22.701 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.698+0000 7fadb566b640 1 -- 172.21.15.88:0/2606436250 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fad900664d0 msgr2=0x7fad90068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:22.701 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.698+0000 7fadb566b640 1 --2- 172.21.15.88:0/2606436250 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fad900664d0 0x7fad90068990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fada00b2000 tx=0x7fada00aa070 comp rx=0 tx=0).stop 2023-10-16T20:05:22.702 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.698+0000 7fadb566b640 1 -- 172.21.15.88:0/2606436250 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fadb00d0bd0 msgr2=0x7fadb01f2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:22.702 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.698+0000 7fadb566b640 1 --2- 172.21.15.88:0/2606436250 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fadb00d0bd0 0x7fadb01f2140 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fad9800d800 tx=0x7fad9800dcc0 comp rx=0 tx=0).stop 2023-10-16T20:05:22.702 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.698+0000 7fadb566b640 1 -- 172.21.15.88:0/2606436250 shutdown_connections 2023-10-16T20:05:22.702 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.698+0000 7fadb566b640 1 --2- 172.21.15.88:0/2606436250 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fad900664d0 0x7fad90068990 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:22.703 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.698+0000 7fadb566b640 1 --2- 172.21.15.88:0/2606436250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadb015cb70 0x7fadb0158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:22.703 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.698+0000 7fadb566b640 1 --2- 172.21.15.88:0/2606436250 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fadb00d0bd0 0x7fadb01f2140 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:22.703 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.698+0000 7fadb566b640 1 --2- 172.21.15.88:0/2606436250 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fadb00b4530 0x7fadb01562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:22.703 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.698+0000 7fadb566b640 1 -- 172.21.15.88:0/2606436250 >> 172.21.15.88:0/2606436250 conn(0x7fadb006b6e0 msgr2=0x7fadb0157690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:22.704 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.702+0000 7fadb566b640 1 -- 172.21.15.88:0/2606436250 shutdown_connections 2023-10-16T20:05:22.704 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.702+0000 7fadb566b640 1 -- 172.21.15.88:0/2606436250 wait complete. 2023-10-16T20:05:22.720 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T20:05:22.860 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.858+0000 7f8dd88c8640 1 Processor -- start 2023-10-16T20:05:22.860 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.858+0000 7f8dd88c8640 1 -- start start 2023-10-16T20:05:22.861 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.862+0000 7f8dd88c8640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8dd40cde80 0x7f8dd40d3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:22.862 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.862+0000 7f8dd88c8640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f8dd40cf230 0x7f8dd40cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:22.862 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.862+0000 7f8dd88c8640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dd40d0c40 con 0x7f8dd415c9a0 2023-10-16T20:05:22.862 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.862+0000 7f8dd88c8640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dd40d37b0 con 0x7f8dd40cf230 2023-10-16T20:05:22.863 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.862+0000 7f8dd88c8640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd40ce7e0 con 0x7f8dd40cde80 2023-10-16T20:05:22.863 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.862+0000 7f8dd3fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f8dd40cf230 0x7f8dd40cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:22.863 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.862+0000 7f8dd3fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f8dd40cf230 0x7f8dd40cf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38024/0 (socket says 172.21.15.88:38024) 2023-10-16T20:05:22.863 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.862+0000 7f8dd3fff640 1 -- 172.21.15.88:0/2755895542 learned_addr learned my addr 172.21.15.88:0/2755895542 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:22.863 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.862+0000 7f8dd2ffd640 1 --2- 172.21.15.88:0/2755895542 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8dd40cde80 0x7f8dd40d3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:22.864 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.862+0000 7f8dd3fff640 1 -- 172.21.15.88:0/2755895542 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8dd40cde80 msgr2=0x7f8dd40d3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:22.864 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.862+0000 7f8dd3fff640 1 --2- 172.21.15.88:0/2755895542 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8dd40cde80 0x7f8dd40d3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:22.864 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.862+0000 7f8dd3fff640 1 -- 172.21.15.88:0/2755895542 >> v1:172.21.15.94:6789/0 conn(0x7f8dd415c9a0 legacy=0x7f8dd40d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:22.865 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.862+0000 7f8dd3fff640 1 -- 172.21.15.88:0/2755895542 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd40ce2a0 con 0x7f8dd40cf230 2023-10-16T20:05:22.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.862+0000 7f8dd3fff640 1 --2- 172.21.15.88:0/2755895542 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8dd40cf230 0x7f8dd40cf6a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc009e50 tx=0x7f8dcc01ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b09a988229148f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:22.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.866+0000 7f8dd27fc640 1 -- 172.21.15.88:0/2755895542 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc0a56e0 con 0x7f8dd40cf230 2023-10-16T20:05:22.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.866+0000 7f8dd27fc640 1 -- 172.21.15.88:0/2755895542 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dcc0a5840 con 0x7f8dd40cf230 2023-10-16T20:05:22.867 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.866+0000 7f8dd27fc640 1 -- 172.21.15.88:0/2755895542 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc0a5bd0 con 0x7f8dd40cf230 2023-10-16T20:05:22.867 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.866+0000 7f8dd88c8640 1 -- 172.21.15.88:0/2755895542 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8dd40cf230 msgr2=0x7f8dd40cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:22.867 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.866+0000 7f8dd88c8640 1 --2- 172.21.15.88:0/2755895542 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8dd40cf230 0x7f8dd40cf6a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc009e50 tx=0x7f8dcc01ca10 comp rx=0 tx=0).stop 2023-10-16T20:05:22.867 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.866+0000 7f8dd88c8640 1 -- 172.21.15.88:0/2755895542 shutdown_connections 2023-10-16T20:05:22.868 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.866+0000 7f8dd88c8640 1 --2- 172.21.15.88:0/2755895542 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8dd40cf230 0x7f8dd40cf6a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:22.868 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.866+0000 7f8dd88c8640 1 --2- 172.21.15.88:0/2755895542 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8dd40cde80 0x7f8dd40d3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:22.868 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.866+0000 7f8dd88c8640 1 -- 172.21.15.88:0/2755895542 >> 172.21.15.88:0/2755895542 conn(0x7f8dd409bc30 msgr2=0x7f8dd40d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:22.868 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.866+0000 7f8dd88c8640 1 -- 172.21.15.88:0/2755895542 shutdown_connections 2023-10-16T20:05:22.868 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.866+0000 7f8dd88c8640 1 -- 172.21.15.88:0/2755895542 wait complete. 2023-10-16T20:05:22.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.866+0000 7f8dd88c8640 1 Processor -- start 2023-10-16T20:05:22.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.866+0000 7f8dd88c8640 1 -- start start 2023-10-16T20:05:22.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.866+0000 7f8dd88c8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8dd40cde80 0x7f8dd40b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:22.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.866+0000 7f8dd88c8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8dd40cf230 0x7f8dd40c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:22.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.866+0000 7f8dd88c8640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8dd415c9a0 0x7f8dd4152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:22.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.866+0000 7f8dd88c8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd40cb280 con 0x7f8dd40cde80 2023-10-16T20:05:22.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.866+0000 7f8dd88c8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd41546c0 con 0x7f8dd40cf230 2023-10-16T20:05:22.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.866+0000 7f8dd88c8640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd4154100 con 0x7f8dd415c9a0 2023-10-16T20:05:22.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.866+0000 7f8dd37fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8dd40cde80 0x7f8dd40b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:22.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.870+0000 7f8dd37fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8dd40cde80 0x7f8dd40b54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38032/0 (socket says 172.21.15.88:38032) 2023-10-16T20:05:22.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.870+0000 7f8dd37fe640 1 -- 172.21.15.88:0/3563810266 learned_addr learned my addr 172.21.15.88:0/3563810266 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:22.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.870+0000 7f8dd3fff640 1 --2- 172.21.15.88:0/3563810266 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8dd415c9a0 0x7f8dd4152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:22.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.870+0000 7f8dd2ffd640 1 --2- 172.21.15.88:0/3563810266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8dd40cf230 0x7f8dd40c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:22.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.870+0000 7f8dd37fe640 1 -- 172.21.15.88:0/3563810266 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8dd415c9a0 msgr2=0x7f8dd4152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:22.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.870+0000 7f8dd37fe640 1 --2- 172.21.15.88:0/3563810266 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8dd415c9a0 0x7f8dd4152a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:22.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.870+0000 7f8dd37fe640 1 -- 172.21.15.88:0/3563810266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8dd40cf230 msgr2=0x7f8dd40c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:22.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.870+0000 7f8dd37fe640 1 --2- 172.21.15.88:0/3563810266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8dd40cf230 0x7f8dd40c9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:22.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.870+0000 7f8dd37fe640 1 -- 172.21.15.88:0/3563810266 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dcc009b30 con 0x7f8dd40cde80 2023-10-16T20:05:22.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.870+0000 7f8dd37fe640 1 --2- 172.21.15.88:0/3563810266 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8dd40cde80 0x7f8dd40b54e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400c400 tx=0x7f8dc400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:22.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.870+0000 7f8dbbfff640 1 -- 172.21.15.88:0/3563810266 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc40052d0 con 0x7f8dd40cde80 2023-10-16T20:05:22.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.870+0000 7f8dbbfff640 1 -- 172.21.15.88:0/3563810266 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc4005430 con 0x7f8dd40cde80 2023-10-16T20:05:22.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.870+0000 7f8dd88c8640 1 -- 172.21.15.88:0/3563810266 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dd415cf00 con 0x7f8dd40cde80 2023-10-16T20:05:22.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.870+0000 7f8dbbfff640 1 -- 172.21.15.88:0/3563810266 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4003890 con 0x7f8dd40cde80 2023-10-16T20:05:22.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.870+0000 7f8dd88c8640 1 -- 172.21.15.88:0/3563810266 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dd40cb490 con 0x7f8dd40cde80 2023-10-16T20:05:22.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.870+0000 7f8dd88c8640 1 -- 172.21.15.88:0/3563810266 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dd40674f0 con 0x7f8dd40cde80 2023-10-16T20:05:22.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.870+0000 7f8dbbfff640 1 -- 172.21.15.88:0/3563810266 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8dc40039f0 con 0x7f8dd40cde80 2023-10-16T20:05:22.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.870+0000 7f8dbbfff640 1 --2- 172.21.15.88:0/3563810266 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f8dc00664a0 0x7f8dc0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:22.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.870+0000 7f8dbbfff640 1 -- 172.21.15.88:0/3563810266 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8dc4083690 con 0x7f8dd40cde80 2023-10-16T20:05:22.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.874+0000 7f8dd2ffd640 1 --2- 172.21.15.88:0/3563810266 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f8dc00664a0 0x7f8dc0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:22.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.874+0000 7f8dd2ffd640 1 --2- 172.21.15.88:0/3563810266 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f8dc00664a0 0x7f8dc0068960 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40c5de0 tx=0x7f8dc8009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:22.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:22.874+0000 7f8dbbfff640 1 -- 172.21.15.88:0/3563810266 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8dc40508d0 con 0x7f8dd40cde80 2023-10-16T20:05:23.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.022+0000 7f8dd88c8640 1 -- 172.21.15.88:0/3563810266 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8dd40cff40 con 0x7f8dd40cde80 2023-10-16T20:05:23.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.026+0000 7f8dbbfff640 1 -- 172.21.15.88:0/3563810266 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8dc4053ef0 con 0x7f8dd40cde80 2023-10-16T20:05:23.026 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2023-10-16T20:05:23.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.026+0000 7f8dd88c8640 1 -- 172.21.15.88:0/3563810266 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f8dc00664a0 msgr2=0x7f8dc0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:23.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.026+0000 7f8dd88c8640 1 --2- 172.21.15.88:0/3563810266 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f8dc00664a0 0x7f8dc0068960 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40c5de0 tx=0x7f8dc8009190 comp rx=0 tx=0).stop 2023-10-16T20:05:23.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.026+0000 7f8dd88c8640 1 -- 172.21.15.88:0/3563810266 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8dd40cde80 msgr2=0x7f8dd40b54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:23.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.026+0000 7f8dd88c8640 1 --2- 172.21.15.88:0/3563810266 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8dd40cde80 0x7f8dd40b54e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400c400 tx=0x7f8dc400c8c0 comp rx=0 tx=0).stop 2023-10-16T20:05:23.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.026+0000 7f8dd88c8640 1 -- 172.21.15.88:0/3563810266 shutdown_connections 2023-10-16T20:05:23.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.026+0000 7f8dd88c8640 1 --2- 172.21.15.88:0/3563810266 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f8dc00664a0 0x7f8dc0068960 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:23.029 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.026+0000 7f8dd88c8640 1 --2- 172.21.15.88:0/3563810266 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8dd415c9a0 0x7f8dd4152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:23.029 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.026+0000 7f8dd88c8640 1 --2- 172.21.15.88:0/3563810266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8dd40cf230 0x7f8dd40c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:23.029 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.026+0000 7f8dd88c8640 1 --2- 172.21.15.88:0/3563810266 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8dd40cde80 0x7f8dd40b54e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:23.029 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.026+0000 7f8dd88c8640 1 -- 172.21.15.88:0/3563810266 >> 172.21.15.88:0/3563810266 conn(0x7f8dd409bc30 msgr2=0x7f8dd40b4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:23.029 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.026+0000 7f8dd88c8640 1 -- 172.21.15.88:0/3563810266 shutdown_connections 2023-10-16T20:05:23.030 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.026+0000 7f8dd88c8640 1 -- 172.21.15.88:0/3563810266 wait complete. 2023-10-16T20:05:23.046 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T20:05:23.186 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.186+0000 7efd7ae46640 1 Processor -- start 2023-10-16T20:05:23.187 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.186+0000 7efd7ae46640 1 -- start start 2023-10-16T20:05:23.188 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.186+0000 7efd7ae46640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7efd7415aae0 0x7efd740d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:23.188 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.186+0000 7efd7ae46640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7efd740b6270 0x7efd740cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:23.188 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.186+0000 7efd7ae46640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd740b6700 con 0x7efd7415aec0 2023-10-16T20:05:23.189 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.186+0000 7efd7ae46640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd7415ea10 con 0x7efd740b6270 2023-10-16T20:05:23.189 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.186+0000 7efd7ae46640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd7415ecf0 con 0x7efd7415aae0 2023-10-16T20:05:23.189 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.186+0000 7efd73fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7efd740b6270 0x7efd740cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:23.189 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.186+0000 7efd73fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7efd740b6270 0x7efd740cb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38036/0 (socket says 172.21.15.88:38036) 2023-10-16T20:05:23.189 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.186+0000 7efd73fff640 1 -- 172.21.15.88:0/2743345727 learned_addr learned my addr 172.21.15.88:0/2743345727 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:23.190 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.186+0000 7efd793e3640 1 --1- 172.21.15.88:0/2743345727 >> v1:172.21.15.88:6789/0 conn(0x7efd7415aec0 0x7efd740ced50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:60766/0 (socket says 172.21.15.88:60766) 2023-10-16T20:05:23.190 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.186+0000 7efd78be2640 1 --2- 172.21.15.88:0/2743345727 >> v2:172.21.15.94:3300/0 conn(0x7efd7415aae0 0x7efd740d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:23.190 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.186+0000 7efd73fff640 1 -- 172.21.15.88:0/2743345727 >> v2:172.21.15.94:3300/0 conn(0x7efd7415aae0 msgr2=0x7efd740d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:23.190 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.186+0000 7efd73fff640 1 --2- 172.21.15.88:0/2743345727 >> v2:172.21.15.94:3300/0 conn(0x7efd7415aae0 0x7efd740d0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:23.191 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.186+0000 7efd73fff640 1 -- 172.21.15.88:0/2743345727 >> v1:172.21.15.88:6789/0 conn(0x7efd7415aec0 legacy=0x7efd740ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:23.191 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.186+0000 7efd73fff640 1 -- 172.21.15.88:0/2743345727 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd7415efd0 con 0x7efd740b6270 2023-10-16T20:05:23.191 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.186+0000 7efd737fe640 1 -- 172.21.15.88:0/2743345727 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4193841486 0 0) 0x7efd740b6700 con 0x7efd7415aec0 2023-10-16T20:05:23.192 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.190+0000 7efd73fff640 1 --2- 172.21.15.88:0/2743345727 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd740b6270 0x7efd740cb6b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efd740b3240 tx=0x7efd6001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79efa9a2e3173770 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:23.192 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.190+0000 7efd737fe640 1 -- 172.21.15.88:0/2743345727 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd600a89f0 con 0x7efd740b6270 2023-10-16T20:05:23.193 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.190+0000 7efd737fe640 1 -- 172.21.15.88:0/2743345727 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd600a8b50 con 0x7efd740b6270 2023-10-16T20:05:23.193 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.190+0000 7efd737fe640 1 -- 172.21.15.88:0/2743345727 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd600b0440 con 0x7efd740b6270 2023-10-16T20:05:23.193 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.190+0000 7efd7ae46640 1 -- 172.21.15.88:0/2743345727 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd740b6270 msgr2=0x7efd740cb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:23.193 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.190+0000 7efd7ae46640 1 --2- 172.21.15.88:0/2743345727 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd740b6270 0x7efd740cb6b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efd740b3240 tx=0x7efd6001ecf0 comp rx=0 tx=0).stop 2023-10-16T20:05:23.193 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.190+0000 7efd7ae46640 1 -- 172.21.15.88:0/2743345727 shutdown_connections 2023-10-16T20:05:23.194 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.190+0000 7efd7ae46640 1 --2- 172.21.15.88:0/2743345727 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd740b6270 0x7efd740cb6b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:23.194 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.190+0000 7efd7ae46640 1 --2- 172.21.15.88:0/2743345727 >> v2:172.21.15.94:3300/0 conn(0x7efd7415aae0 0x7efd740d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:23.194 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.190+0000 7efd7ae46640 1 -- 172.21.15.88:0/2743345727 >> 172.21.15.88:0/2743345727 conn(0x7efd740cd3f0 msgr2=0x7efd740cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:23.194 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.190+0000 7efd7ae46640 1 -- 172.21.15.88:0/2743345727 shutdown_connections 2023-10-16T20:05:23.195 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.190+0000 7efd7ae46640 1 -- 172.21.15.88:0/2743345727 wait complete. 2023-10-16T20:05:23.196 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.194+0000 7efd7ae46640 1 Processor -- start 2023-10-16T20:05:23.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.194+0000 7efd7ae46640 1 -- start start 2023-10-16T20:05:23.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.194+0000 7efd7ae46640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd7415aae0 0x7efd7414ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:23.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.194+0000 7efd7ae46640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efd7415aec0 0x7efd740ca390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:23.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.194+0000 7efd7ae46640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd740c79b0 0x7efd740c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:23.198 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.194+0000 7efd7ae46640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd740c8bf0 con 0x7efd7415aae0 2023-10-16T20:05:23.198 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.194+0000 7efd7ae46640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd740b59b0 con 0x7efd740c79b0 2023-10-16T20:05:23.198 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.194+0000 7efd7ae46640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd740c7f70 con 0x7efd7415aec0 2023-10-16T20:05:23.198 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.194+0000 7efd78be2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd7415aae0 0x7efd7414ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:23.198 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.194+0000 7efd78be2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd7415aae0 0x7efd7414ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38048/0 (socket says 172.21.15.88:38048) 2023-10-16T20:05:23.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.194+0000 7efd78be2640 1 -- 172.21.15.88:0/925548318 learned_addr learned my addr 172.21.15.88:0/925548318 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:23.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.194+0000 7efd73fff640 1 --2- 172.21.15.88:0/925548318 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efd7415aec0 0x7efd740ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:23.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.194+0000 7efd793e3640 1 --2- 172.21.15.88:0/925548318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd740c79b0 0x7efd740c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:23.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.194+0000 7efd78be2640 1 -- 172.21.15.88:0/925548318 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efd7415aec0 msgr2=0x7efd740ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:23.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.194+0000 7efd78be2640 1 --2- 172.21.15.88:0/925548318 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efd7415aec0 0x7efd740ca390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:23.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.194+0000 7efd78be2640 1 -- 172.21.15.88:0/925548318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd740c79b0 msgr2=0x7efd740c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:23.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.194+0000 7efd78be2640 1 --2- 172.21.15.88:0/925548318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd740c79b0 0x7efd740c9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:23.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.194+0000 7efd78be2640 1 -- 172.21.15.88:0/925548318 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd600094d0 con 0x7efd7415aae0 2023-10-16T20:05:23.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.194+0000 7efd78be2640 1 --2- 172.21.15.88:0/925548318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd7415aae0 0x7efd7414ebb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efd6800e930 tx=0x7efd6800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:23.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.194+0000 7efd717fa640 1 -- 172.21.15.88:0/925548318 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd68009690 con 0x7efd7415aae0 2023-10-16T20:05:23.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.194+0000 7efd717fa640 1 -- 172.21.15.88:0/925548318 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd6800ca00 con 0x7efd7415aae0 2023-10-16T20:05:23.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.194+0000 7efd717fa640 1 -- 172.21.15.88:0/925548318 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6800cd70 con 0x7efd7415aae0 2023-10-16T20:05:23.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.194+0000 7efd7ae46640 1 -- 172.21.15.88:0/925548318 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd740c8250 con 0x7efd7415aae0 2023-10-16T20:05:23.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.194+0000 7efd7ae46640 1 -- 172.21.15.88:0/925548318 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd740c8810 con 0x7efd7415aae0 2023-10-16T20:05:23.202 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.198+0000 7efd7ae46640 1 -- 172.21.15.88:0/925548318 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd7404e9c0 con 0x7efd7415aae0 2023-10-16T20:05:23.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.198+0000 7efd717fa640 1 -- 172.21.15.88:0/925548318 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7efd68013470 con 0x7efd7415aae0 2023-10-16T20:05:23.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.198+0000 7efd717fa640 1 --2- 172.21.15.88:0/925548318 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7efd580664d0 0x7efd58068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:23.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.198+0000 7efd717fa640 1 -- 172.21.15.88:0/925548318 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7efd68084370 con 0x7efd7415aae0 2023-10-16T20:05:23.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.198+0000 7efd73fff640 1 --2- 172.21.15.88:0/925548318 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7efd580664d0 0x7efd58068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:23.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.198+0000 7efd73fff640 1 --2- 172.21.15.88:0/925548318 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7efd580664d0 0x7efd58068990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efd600b2000 tx=0x7efd60009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:23.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.202+0000 7efd717fa640 1 -- 172.21.15.88:0/925548318 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7efd68051660 con 0x7efd7415aae0 2023-10-16T20:05:23.353 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.350+0000 7efd7ae46640 1 -- 172.21.15.88:0/925548318 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7efd740ce620 con 0x7efd7415aae0 2023-10-16T20:05:23.353 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.354+0000 7efd717fa640 1 -- 172.21.15.88:0/925548318 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7efd68054c80 con 0x7efd7415aae0 2023-10-16T20:05:23.354 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:05:23.355 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.354+0000 7efd7ae46640 1 -- 172.21.15.88:0/925548318 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7efd580664d0 msgr2=0x7efd58068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:23.355 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.354+0000 7efd7ae46640 1 --2- 172.21.15.88:0/925548318 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7efd580664d0 0x7efd58068990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efd600b2000 tx=0x7efd60009040 comp rx=0 tx=0).stop 2023-10-16T20:05:23.356 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.354+0000 7efd7ae46640 1 -- 172.21.15.88:0/925548318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd7415aae0 msgr2=0x7efd7414ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:23.356 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.354+0000 7efd7ae46640 1 --2- 172.21.15.88:0/925548318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd7415aae0 0x7efd7414ebb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efd6800e930 tx=0x7efd6800edf0 comp rx=0 tx=0).stop 2023-10-16T20:05:23.356 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.354+0000 7efd7ae46640 1 -- 172.21.15.88:0/925548318 shutdown_connections 2023-10-16T20:05:23.356 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.354+0000 7efd7ae46640 1 --2- 172.21.15.88:0/925548318 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7efd580664d0 0x7efd58068990 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:23.356 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.354+0000 7efd7ae46640 1 --2- 172.21.15.88:0/925548318 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efd7415aec0 0x7efd740ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:23.357 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.354+0000 7efd7ae46640 1 --2- 172.21.15.88:0/925548318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd740c79b0 0x7efd740c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:23.357 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.354+0000 7efd7ae46640 1 --2- 172.21.15.88:0/925548318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd7415aae0 0x7efd7414ebb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:23.357 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.354+0000 7efd7ae46640 1 -- 172.21.15.88:0/925548318 >> 172.21.15.88:0/925548318 conn(0x7efd740cd3f0 msgr2=0x7efd74158710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:23.357 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.354+0000 7efd7ae46640 1 -- 172.21.15.88:0/925548318 shutdown_connections 2023-10-16T20:05:23.357 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.354+0000 7efd7ae46640 1 -- 172.21.15.88:0/925548318 wait complete. 2023-10-16T20:05:23.374 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T20:05:23.512 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.510+0000 7f8659180640 1 Processor -- start 2023-10-16T20:05:23.512 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.510+0000 7f8659180640 1 -- start start 2023-10-16T20:05:23.514 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.510+0000 7f8659180640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f86540b2a80 0x7f86540ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:23.514 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.510+0000 7f8659180640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f86540cea00 0x7f865406bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:23.514 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.510+0000 7f8659180640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f865406b490 con 0x7f865415cb80 2023-10-16T20:05:23.514 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.514+0000 7f8659180640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86540cf180 con 0x7f86540cea00 2023-10-16T20:05:23.515 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.514+0000 7f8659180640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f865406b700 con 0x7f86540b2a80 2023-10-16T20:05:23.515 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.514+0000 7f8652d9d640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f865415cb80 0x7f86540b2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:60772/0 (socket says 172.21.15.88:60772) 2023-10-16T20:05:23.515 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.514+0000 7f8652d9d640 1 -- 172.21.15.88:0/3737544726 learned_addr learned my addr 172.21.15.88:0/3737544726 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:23.515 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.514+0000 7f865359e640 1 --2- 172.21.15.88:0/3737544726 >> v2:172.21.15.88:3300/0 conn(0x7f86540cea00 0x7f865406bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:23.515 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.514+0000 7f864a59c640 1 --2- 172.21.15.88:0/3737544726 >> v2:172.21.15.94:3300/0 conn(0x7f86540b2a80 0x7f86540ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:23.516 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.514+0000 7f865259c640 1 -- 172.21.15.88:0/3737544726 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1791157348 0 0) 0x7f865406b490 con 0x7f865415cb80 2023-10-16T20:05:23.516 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.514+0000 7f865259c640 1 -- 172.21.15.88:0/3737544726 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8640003530 con 0x7f865415cb80 2023-10-16T20:05:23.516 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.514+0000 7f865359e640 1 -- 172.21.15.88:0/3737544726 >> v2:172.21.15.94:3300/0 conn(0x7f86540b2a80 msgr2=0x7f86540ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:23.516 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.514+0000 7f865359e640 1 --2- 172.21.15.88:0/3737544726 >> v2:172.21.15.94:3300/0 conn(0x7f86540b2a80 0x7f86540ce4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:23.516 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.514+0000 7f865359e640 1 -- 172.21.15.88:0/3737544726 >> v1:172.21.15.88:6789/0 conn(0x7f865415cb80 legacy=0x7f86540b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:23.516 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.514+0000 7f865359e640 1 -- 172.21.15.88:0/3737544726 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f865406b9e0 con 0x7f86540cea00 2023-10-16T20:05:23.518 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.514+0000 7f865359e640 1 --2- 172.21.15.88:0/3737544726 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86540cea00 0x7f865406bf00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f86540cd220 tx=0x7f864c01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96b5ef38aa2d845 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:23.518 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.514+0000 7f865259c640 1 -- 172.21.15.88:0/3737544726 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c0a5900 con 0x7f86540cea00 2023-10-16T20:05:23.518 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.514+0000 7f865259c640 1 -- 172.21.15.88:0/3737544726 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864c0a5a60 con 0x7f86540cea00 2023-10-16T20:05:23.518 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.514+0000 7f865259c640 1 -- 172.21.15.88:0/3737544726 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c0a5df0 con 0x7f86540cea00 2023-10-16T20:05:23.519 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.518+0000 7f8659180640 1 -- 172.21.15.88:0/3737544726 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86540cea00 msgr2=0x7f865406bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:23.519 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.518+0000 7f8659180640 1 --2- 172.21.15.88:0/3737544726 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86540cea00 0x7f865406bf00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f86540cd220 tx=0x7f864c01c910 comp rx=0 tx=0).stop 2023-10-16T20:05:23.519 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.518+0000 7f8659180640 1 -- 172.21.15.88:0/3737544726 shutdown_connections 2023-10-16T20:05:23.519 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.518+0000 7f8659180640 1 --2- 172.21.15.88:0/3737544726 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86540cea00 0x7f865406bf00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:23.519 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.518+0000 7f8659180640 1 --2- 172.21.15.88:0/3737544726 >> v2:172.21.15.94:3300/0 conn(0x7f86540b2a80 0x7f86540ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:23.520 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.518+0000 7f8659180640 1 -- 172.21.15.88:0/3737544726 >> 172.21.15.88:0/3737544726 conn(0x7f86540cd370 msgr2=0x7f86540cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:23.520 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.518+0000 7f8659180640 1 -- 172.21.15.88:0/3737544726 shutdown_connections 2023-10-16T20:05:23.520 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.518+0000 7f8659180640 1 -- 172.21.15.88:0/3737544726 wait complete. 2023-10-16T20:05:23.522 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.518+0000 7f8659180640 1 Processor -- start 2023-10-16T20:05:23.522 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.518+0000 7f8659180640 1 -- start start 2023-10-16T20:05:23.522 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.518+0000 7f8659180640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f86540b2a80 0x7f8654156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:23.522 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.518+0000 7f8659180640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86540cea00 0x7f86540c43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:23.523 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.518+0000 7f8659180640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f865415cb80 0x7f86540c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:23.523 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.518+0000 7f8659180640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86540c8330 con 0x7f865415cb80 2023-10-16T20:05:23.523 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.518+0000 7f8659180640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86540c8610 con 0x7f86540cea00 2023-10-16T20:05:23.523 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.518+0000 7f8659180640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86540c88f0 con 0x7f86540b2a80 2023-10-16T20:05:23.523 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.518+0000 7f865359e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f865415cb80 0x7f86540c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:23.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.518+0000 7f865359e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f865415cb80 0x7f86540c79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38060/0 (socket says 172.21.15.88:38060) 2023-10-16T20:05:23.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.518+0000 7f865359e640 1 -- 172.21.15.88:0/3957360918 learned_addr learned my addr 172.21.15.88:0/3957360918 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:23.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.522+0000 7f865359e640 1 -- 172.21.15.88:0/3957360918 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f86540b2a80 msgr2=0x7f8654156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:23.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.522+0000 7f865359e640 1 --2- 172.21.15.88:0/3957360918 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f86540b2a80 0x7f8654156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:23.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.522+0000 7f864a59c640 1 --2- 172.21.15.88:0/3957360918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86540cea00 0x7f86540c43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:23.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.522+0000 7f865359e640 1 -- 172.21.15.88:0/3957360918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86540cea00 msgr2=0x7f86540c43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:23.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.522+0000 7f865359e640 1 --2- 172.21.15.88:0/3957360918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86540cea00 0x7f86540c43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:23.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.522+0000 7f865359e640 1 -- 172.21.15.88:0/3957360918 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f864c009a70 con 0x7f865415cb80 2023-10-16T20:05:23.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.522+0000 7f864a59c640 1 --2- 172.21.15.88:0/3957360918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86540cea00 0x7f86540c43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:23.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.522+0000 7f865359e640 1 --2- 172.21.15.88:0/3957360918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f865415cb80 0x7f86540c79c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f864c019520 tx=0x7f864c0196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:23.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.522+0000 7f864bfff640 1 -- 172.21.15.88:0/3957360918 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c0b9860 con 0x7f865415cb80 2023-10-16T20:05:23.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.522+0000 7f864bfff640 1 -- 172.21.15.88:0/3957360918 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864c0a58f0 con 0x7f865415cb80 2023-10-16T20:05:23.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.522+0000 7f864bfff640 1 -- 172.21.15.88:0/3957360918 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c0b9c00 con 0x7f865415cb80 2023-10-16T20:05:23.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.522+0000 7f8659180640 1 -- 172.21.15.88:0/3957360918 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86540c9560 con 0x7f865415cb80 2023-10-16T20:05:23.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.522+0000 7f8659180640 1 -- 172.21.15.88:0/3957360918 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86540c4930 con 0x7f865415cb80 2023-10-16T20:05:23.527 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.522+0000 7f8659180640 1 -- 172.21.15.88:0/3957360918 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f865404e9c0 con 0x7f865415cb80 2023-10-16T20:05:23.527 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.522+0000 7f864bfff640 1 -- 172.21.15.88:0/3957360918 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f864c0ad440 con 0x7f865415cb80 2023-10-16T20:05:23.527 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.522+0000 7f864bfff640 1 --2- 172.21.15.88:0/3957360918 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f86340664d0 0x7f8634068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:23.528 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.522+0000 7f864bfff640 1 -- 172.21.15.88:0/3957360918 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f864c11ef90 con 0x7f865415cb80 2023-10-16T20:05:23.528 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.522+0000 7f8652d9d640 1 --2- 172.21.15.88:0/3957360918 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f86340664d0 0x7f8634068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:23.528 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.526+0000 7f8652d9d640 1 --2- 172.21.15.88:0/3957360918 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f86340664d0 0x7f8634068990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f863c007fd0 tx=0x7f863c007940 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:23.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.526+0000 7f864bfff640 1 -- 172.21.15.88:0/3957360918 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f864c0ec1a0 con 0x7f865415cb80 2023-10-16T20:05:23.707 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.706+0000 7f8659180640 1 -- 172.21.15.88:0/3957360918 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86540c9a10 con 0x7f865415cb80 2023-10-16T20:05:23.708 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.706+0000 7f864bfff640 1 -- 172.21.15.88:0/3957360918 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4906 (secure 0 0 0) 0x7f864c0ef7c0 con 0x7f865415cb80 2023-10-16T20:05:23.709 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:23.709 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3803627113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3803627113},{"type":"v1","addr":"172.21.15.94:6834","nonce":3803627113}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3758093239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":3758093239},{"type":"v1","addr":"172.21.15.88:6837","nonce":3758093239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/3836772385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3836772385},{"type":"v1","addr":"172.21.15.88:6836","nonce":3836772385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T20:04:56.759719+0000","modified":"2023-10-16T20:05:14.569618+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6835/555575750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":555575750},{"type":"v1","addr":"172.21.15.94:6835","nonce":555575750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T20:05:23.709 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 9 2023-10-16T20:05:23.710 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.710+0000 7f8659180640 1 -- 172.21.15.88:0/3957360918 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f86340664d0 msgr2=0x7f8634068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:23.711 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.710+0000 7f8659180640 1 --2- 172.21.15.88:0/3957360918 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f86340664d0 0x7f8634068990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f863c007fd0 tx=0x7f863c007940 comp rx=0 tx=0).stop 2023-10-16T20:05:23.711 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.710+0000 7f8659180640 1 -- 172.21.15.88:0/3957360918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f865415cb80 msgr2=0x7f86540c79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:23.711 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.710+0000 7f8659180640 1 --2- 172.21.15.88:0/3957360918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f865415cb80 0x7f86540c79c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f864c019520 tx=0x7f864c0196b0 comp rx=0 tx=0).stop 2023-10-16T20:05:23.711 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.710+0000 7f8659180640 1 -- 172.21.15.88:0/3957360918 shutdown_connections 2023-10-16T20:05:23.711 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.710+0000 7f8659180640 1 --2- 172.21.15.88:0/3957360918 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f86340664d0 0x7f8634068990 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:23.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.710+0000 7f8659180640 1 --2- 172.21.15.88:0/3957360918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f865415cb80 0x7f86540c79c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:23.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.710+0000 7f8659180640 1 --2- 172.21.15.88:0/3957360918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86540cea00 0x7f86540c43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:23.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.710+0000 7f8659180640 1 --2- 172.21.15.88:0/3957360918 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f86540b2a80 0x7f8654156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:23.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.710+0000 7f8659180640 1 -- 172.21.15.88:0/3957360918 >> 172.21.15.88:0/3957360918 conn(0x7f86540cd370 msgr2=0x7f865415be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:23.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.710+0000 7f8659180640 1 -- 172.21.15.88:0/3957360918 shutdown_connections 2023-10-16T20:05:23.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.710+0000 7f8659180640 1 -- 172.21.15.88:0/3957360918 wait complete. 2023-10-16T20:05:23.730 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:05:23.868 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.866+0000 7fd3c172a640 1 Processor -- start 2023-10-16T20:05:23.868 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.866+0000 7fd3c172a640 1 -- start start 2023-10-16T20:05:23.869 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.866+0000 7fd3c172a640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd3bc15d9f0 0x7fd3bc161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:23.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.866+0000 7fd3c172a640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3bc0d26c0 0x7fd3bc0cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:23.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.866+0000 7fd3c172a640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3bc0b87e0 con 0x7fd3bc0d22e0 2023-10-16T20:05:23.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.866+0000 7fd3c172a640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3bc15df10 con 0x7fd3bc15d9f0 2023-10-16T20:05:23.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.866+0000 7fd3c172a640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc15e1f0 con 0x7fd3bc0d26c0 2023-10-16T20:05:23.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.870+0000 7fd3b3fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd3bc15d9f0 0x7fd3bc161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:23.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.870+0000 7fd3b3fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd3bc15d9f0 0x7fd3bc161450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38074/0 (socket says 172.21.15.88:38074) 2023-10-16T20:05:23.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.870+0000 7fd3b3fff640 1 -- 172.21.15.88:0/3486728189 learned_addr learned my addr 172.21.15.88:0/3486728189 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:23.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.870+0000 7fd3baffd640 1 --1- 172.21.15.88:0/3486728189 >> v1:172.21.15.88:6789/0 conn(0x7fd3bc0d22e0 0x7fd3bc15d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:60788/0 (socket says 172.21.15.88:60788) 2023-10-16T20:05:23.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.870+0000 7fd3bb7fe640 1 --2- 172.21.15.88:0/3486728189 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3bc0d26c0 0x7fd3bc0cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:23.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.870+0000 7fd3b3fff640 1 -- 172.21.15.88:0/3486728189 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3bc0d26c0 msgr2=0x7fd3bc0cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:23.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.870+0000 7fd3b3fff640 1 --2- 172.21.15.88:0/3486728189 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3bc0d26c0 0x7fd3bc0cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:23.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.870+0000 7fd3b3fff640 1 -- 172.21.15.88:0/3486728189 >> v1:172.21.15.88:6789/0 conn(0x7fd3bc0d22e0 legacy=0x7fd3bc15d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:23.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.870+0000 7fd3b3fff640 1 -- 172.21.15.88:0/3486728189 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3bc160970 con 0x7fd3bc15d9f0 2023-10-16T20:05:23.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.870+0000 7fd3b3fff640 1 --2- 172.21.15.88:0/3486728189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd3bc15d9f0 0x7fd3bc161450 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4009870 tx=0x7fd3a401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a24ad5af38ad3b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:23.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.870+0000 7fd3ba7fc640 1 -- 172.21.15.88:0/3486728189 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a40ab680 con 0x7fd3bc15d9f0 2023-10-16T20:05:23.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.870+0000 7fd3ba7fc640 1 -- 172.21.15.88:0/3486728189 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a40ab7e0 con 0x7fd3bc15d9f0 2023-10-16T20:05:23.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.870+0000 7fd3ba7fc640 1 -- 172.21.15.88:0/3486728189 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a40abb70 con 0x7fd3bc15d9f0 2023-10-16T20:05:23.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.874+0000 7fd3c172a640 1 -- 172.21.15.88:0/3486728189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd3bc15d9f0 msgr2=0x7fd3bc161450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:23.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.874+0000 7fd3c172a640 1 --2- 172.21.15.88:0/3486728189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd3bc15d9f0 0x7fd3bc161450 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4009870 tx=0x7fd3a401c480 comp rx=0 tx=0).stop 2023-10-16T20:05:23.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.874+0000 7fd3c172a640 1 -- 172.21.15.88:0/3486728189 shutdown_connections 2023-10-16T20:05:23.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.874+0000 7fd3c172a640 1 --2- 172.21.15.88:0/3486728189 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3bc0d26c0 0x7fd3bc0cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:23.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.874+0000 7fd3c172a640 1 --2- 172.21.15.88:0/3486728189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd3bc15d9f0 0x7fd3bc161450 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:23.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.874+0000 7fd3c172a640 1 -- 172.21.15.88:0/3486728189 >> 172.21.15.88:0/3486728189 conn(0x7fd3bc0715c0 msgr2=0x7fd3bc0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:23.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.874+0000 7fd3c172a640 1 -- 172.21.15.88:0/3486728189 shutdown_connections 2023-10-16T20:05:23.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.874+0000 7fd3c172a640 1 -- 172.21.15.88:0/3486728189 wait complete. 2023-10-16T20:05:23.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.874+0000 7fd3c172a640 1 Processor -- start 2023-10-16T20:05:23.878 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.874+0000 7fd3c172a640 1 -- start start 2023-10-16T20:05:23.878 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.874+0000 7fd3c172a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3bc0d22e0 0x7fd3bc155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:23.878 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.874+0000 7fd3c172a640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3bc0d26c0 0x7fd3bc14d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:23.878 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.874+0000 7fd3c172a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd3bc15d9f0 0x7fd3bc0c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:23.879 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.874+0000 7fd3c172a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc0b8640 con 0x7fd3bc15d9f0 2023-10-16T20:05:23.879 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.874+0000 7fd3c172a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc155b80 con 0x7fd3bc0d22e0 2023-10-16T20:05:23.879 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.874+0000 7fd3c172a640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc14d8c0 con 0x7fd3bc0d26c0 2023-10-16T20:05:23.879 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.874+0000 7fd3bb7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd3bc15d9f0 0x7fd3bc0c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:23.879 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.874+0000 7fd3bb7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd3bc15d9f0 0x7fd3bc0c7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38086/0 (socket says 172.21.15.88:38086) 2023-10-16T20:05:23.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.874+0000 7fd3bb7fe640 1 -- 172.21.15.88:0/2886628560 learned_addr learned my addr 172.21.15.88:0/2886628560 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:23.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.874+0000 7fd3baffd640 1 --2- 172.21.15.88:0/2886628560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3bc0d22e0 0x7fd3bc155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:23.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.874+0000 7fd3b3fff640 1 --2- 172.21.15.88:0/2886628560 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3bc0d26c0 0x7fd3bc14d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:23.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.874+0000 7fd3bb7fe640 1 -- 172.21.15.88:0/2886628560 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3bc0d26c0 msgr2=0x7fd3bc14d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:23.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.874+0000 7fd3bb7fe640 1 --2- 172.21.15.88:0/2886628560 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3bc0d26c0 0x7fd3bc14d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:23.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.874+0000 7fd3bb7fe640 1 -- 172.21.15.88:0/2886628560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3bc0d22e0 msgr2=0x7fd3bc155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:23.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.874+0000 7fd3bb7fe640 1 --2- 172.21.15.88:0/2886628560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3bc0d22e0 0x7fd3bc155640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:23.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.874+0000 7fd3bb7fe640 1 -- 172.21.15.88:0/2886628560 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a40094d0 con 0x7fd3bc15d9f0 2023-10-16T20:05:23.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.878+0000 7fd3bb7fe640 1 --2- 172.21.15.88:0/2886628560 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd3bc15d9f0 0x7fd3bc0c7a30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4007e40 tx=0x7fd3b400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:23.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.878+0000 7fd3b37fe640 1 -- 172.21.15.88:0/2886628560 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4014620 con 0x7fd3bc15d9f0 2023-10-16T20:05:23.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.878+0000 7fd3b37fe640 1 -- 172.21.15.88:0/2886628560 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b400ee90 con 0x7fd3bc15d9f0 2023-10-16T20:05:23.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.878+0000 7fd3c172a640 1 -- 172.21.15.88:0/2886628560 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3bc155db0 con 0x7fd3bc15d9f0 2023-10-16T20:05:23.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.878+0000 7fd3b37fe640 1 -- 172.21.15.88:0/2886628560 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b40149e0 con 0x7fd3bc15d9f0 2023-10-16T20:05:23.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.878+0000 7fd3c172a640 1 -- 172.21.15.88:0/2886628560 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3bc156370 con 0x7fd3bc15d9f0 2023-10-16T20:05:23.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.878+0000 7fd3c172a640 1 -- 172.21.15.88:0/2886628560 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3bc04e9c0 con 0x7fd3bc15d9f0 2023-10-16T20:05:23.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.878+0000 7fd3b37fe640 1 -- 172.21.15.88:0/2886628560 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd3b4014b40 con 0x7fd3bc15d9f0 2023-10-16T20:05:23.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.878+0000 7fd3b37fe640 1 --2- 172.21.15.88:0/2886628560 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd39c0664a0 0x7fd39c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:23.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.878+0000 7fd3b37fe640 1 -- 172.21.15.88:0/2886628560 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd3b40849c0 con 0x7fd3bc15d9f0 2023-10-16T20:05:23.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.878+0000 7fd3baffd640 1 --2- 172.21.15.88:0/2886628560 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd39c0664a0 0x7fd39c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:23.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.882+0000 7fd3baffd640 1 --2- 172.21.15.88:0/2886628560 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd39c0664a0 0x7fd39c068960 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac000f70 tx=0x7fd3ac007e50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:23.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:23.882+0000 7fd3b37fe640 1 -- 172.21.15.88:0/2886628560 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd3b4051bf0 con 0x7fd3bc15d9f0 2023-10-16T20:05:24.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.034+0000 7fd3c172a640 1 -- 172.21.15.88:0/2886628560 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3bc0d0840 con 0x7fd3bc15d9f0 2023-10-16T20:05:24.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.034+0000 7fd3b37fe640 1 -- 172.21.15.88:0/2886628560 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18274 (secure 0 0 0) 0x7fd3b4055210 con 0x7fd3bc15d9f0 2023-10-16T20:05:24.035 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:24.036 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:05:01.835630+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T20:04:54.442487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T20:04:55.766641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T20:04:59.475830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:05:24.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.034+0000 7fd3c172a640 1 -- 172.21.15.88:0/2886628560 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd39c0664a0 msgr2=0x7fd39c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:24.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.034+0000 7fd3c172a640 1 --2- 172.21.15.88:0/2886628560 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd39c0664a0 0x7fd39c068960 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac000f70 tx=0x7fd3ac007e50 comp rx=0 tx=0).stop 2023-10-16T20:05:24.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.034+0000 7fd3c172a640 1 -- 172.21.15.88:0/2886628560 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd3bc15d9f0 msgr2=0x7fd3bc0c7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:24.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.034+0000 7fd3c172a640 1 --2- 172.21.15.88:0/2886628560 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd3bc15d9f0 0x7fd3bc0c7a30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4007e40 tx=0x7fd3b400e4b0 comp rx=0 tx=0).stop 2023-10-16T20:05:24.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.034+0000 7fd3c172a640 1 -- 172.21.15.88:0/2886628560 shutdown_connections 2023-10-16T20:05:24.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.034+0000 7fd3c172a640 1 --2- 172.21.15.88:0/2886628560 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd39c0664a0 0x7fd39c068960 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:24.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.034+0000 7fd3c172a640 1 --2- 172.21.15.88:0/2886628560 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3bc0d26c0 0x7fd3bc14d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:24.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.034+0000 7fd3c172a640 1 --2- 172.21.15.88:0/2886628560 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd3bc15d9f0 0x7fd3bc0c7a30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:24.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.034+0000 7fd3c172a640 1 --2- 172.21.15.88:0/2886628560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3bc0d22e0 0x7fd3bc155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:24.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.034+0000 7fd3c172a640 1 -- 172.21.15.88:0/2886628560 >> 172.21.15.88:0/2886628560 conn(0x7fd3bc0715c0 msgr2=0x7fd3bc15d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:24.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.038+0000 7fd3c172a640 1 -- 172.21.15.88:0/2886628560 shutdown_connections 2023-10-16T20:05:24.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.038+0000 7fd3c172a640 1 -- 172.21.15.88:0/2886628560 wait complete. 2023-10-16T20:05:24.056 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T20:04:54.442487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T20:04:55.766641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T20:04:59.475830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T20:05:24.057 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:05:24.196 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.194+0000 7fba56a54640 1 Processor -- start 2023-10-16T20:05:24.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.194+0000 7fba56a54640 1 -- start start 2023-10-16T20:05:24.198 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.198+0000 7fba56a54640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fba5015a820 0x7fba500cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:24.198 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.198+0000 7fba56a54640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fba5015ac00 0x7fba500b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:24.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.198+0000 7fba56a54640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba500ce4b0 con 0x7fba500b2930 2023-10-16T20:05:24.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.198+0000 7fba56a54640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba500b2da0 con 0x7fba5015a820 2023-10-16T20:05:24.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.198+0000 7fba56a54640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba5015e270 con 0x7fba5015ac00 2023-10-16T20:05:24.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.198+0000 7fba54ff1640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fba500b2930 0x7fba5006c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:60798/0 (socket says 172.21.15.88:60798) 2023-10-16T20:05:24.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.198+0000 7fba54ff1640 1 -- 172.21.15.88:0/3174217467 learned_addr learned my addr 172.21.15.88:0/3174217467 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:24.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.198+0000 7fba4f7fe640 1 --2- 172.21.15.88:0/3174217467 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fba5015ac00 0x7fba500b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:24.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.198+0000 7fba4ffff640 1 --2- 172.21.15.88:0/3174217467 >> v2:172.21.15.94:3300/0 conn(0x7fba5015a820 0x7fba500cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:24.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.198+0000 7fba4effd640 1 -- 172.21.15.88:0/3174217467 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 733134112 0 0) 0x7fba500ce4b0 con 0x7fba500b2930 2023-10-16T20:05:24.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.198+0000 7fba4effd640 1 -- 172.21.15.88:0/3174217467 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba38003530 con 0x7fba500b2930 2023-10-16T20:05:24.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.198+0000 7fba4ffff640 1 -- 172.21.15.88:0/3174217467 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fba5015ac00 msgr2=0x7fba500b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:24.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.198+0000 7fba4ffff640 1 --2- 172.21.15.88:0/3174217467 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fba5015ac00 0x7fba500b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:24.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.198+0000 7fba4ffff640 1 -- 172.21.15.88:0/3174217467 >> v1:172.21.15.88:6789/0 conn(0x7fba500b2930 legacy=0x7fba5006c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:24.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.198+0000 7fba4ffff640 1 -- 172.21.15.88:0/3174217467 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba500ce700 con 0x7fba5015a820 2023-10-16T20:05:24.202 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.202+0000 7fba4ffff640 1 --2- 172.21.15.88:0/3174217467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba5015a820 0x7fba500cf1e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fba400097c0 tx=0x7fba4001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c12385a701e0bf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:24.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.202+0000 7fba4effd640 1 -- 172.21.15.88:0/3174217467 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba400ad660 con 0x7fba5015a820 2023-10-16T20:05:24.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.202+0000 7fba4effd640 1 -- 172.21.15.88:0/3174217467 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba400ad7c0 con 0x7fba5015a820 2023-10-16T20:05:24.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.202+0000 7fba4effd640 1 -- 172.21.15.88:0/3174217467 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba400adb50 con 0x7fba5015a820 2023-10-16T20:05:24.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.202+0000 7fba56a54640 1 -- 172.21.15.88:0/3174217467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba5015a820 msgr2=0x7fba500cf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:24.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.202+0000 7fba56a54640 1 --2- 172.21.15.88:0/3174217467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba5015a820 0x7fba500cf1e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fba400097c0 tx=0x7fba4001c450 comp rx=0 tx=0).stop 2023-10-16T20:05:24.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.202+0000 7fba56a54640 1 -- 172.21.15.88:0/3174217467 shutdown_connections 2023-10-16T20:05:24.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.202+0000 7fba56a54640 1 --2- 172.21.15.88:0/3174217467 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fba5015ac00 0x7fba500b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:24.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.202+0000 7fba56a54640 1 --2- 172.21.15.88:0/3174217467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba5015a820 0x7fba500cf1e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:24.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.202+0000 7fba56a54640 1 -- 172.21.15.88:0/3174217467 >> 172.21.15.88:0/3174217467 conn(0x7fba500cd3f0 msgr2=0x7fba500cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:24.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.202+0000 7fba56a54640 1 -- 172.21.15.88:0/3174217467 shutdown_connections 2023-10-16T20:05:24.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.202+0000 7fba56a54640 1 -- 172.21.15.88:0/3174217467 wait complete. 2023-10-16T20:05:24.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.202+0000 7fba56a54640 1 Processor -- start 2023-10-16T20:05:24.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.202+0000 7fba56a54640 1 -- start start 2023-10-16T20:05:24.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.206+0000 7fba56a54640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fba500b2930 0x7fba500b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:24.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.206+0000 7fba56a54640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba5015a820 0x7fba500b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:24.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.206+0000 7fba56a54640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba5015ac00 0x7fba501f8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:24.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.206+0000 7fba56a54640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5015ea10 con 0x7fba5015ac00 2023-10-16T20:05:24.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.206+0000 7fba56a54640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba500b16c0 con 0x7fba5015a820 2023-10-16T20:05:24.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.206+0000 7fba56a54640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba500b19a0 con 0x7fba500b2930 2023-10-16T20:05:24.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.206+0000 7fba54ff1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba5015ac00 0x7fba501f8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:24.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.206+0000 7fba54ff1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba5015ac00 0x7fba501f8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38094/0 (socket says 172.21.15.88:38094) 2023-10-16T20:05:24.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.206+0000 7fba54ff1640 1 -- 172.21.15.88:0/3442032504 learned_addr learned my addr 172.21.15.88:0/3442032504 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:24.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.206+0000 7fba4ffff640 1 --2- 172.21.15.88:0/3442032504 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fba500b2930 0x7fba500b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:24.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.206+0000 7fba4f7fe640 1 --2- 172.21.15.88:0/3442032504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba5015a820 0x7fba500b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:24.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.206+0000 7fba54ff1640 1 -- 172.21.15.88:0/3442032504 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fba500b2930 msgr2=0x7fba500b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:24.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.206+0000 7fba54ff1640 1 --2- 172.21.15.88:0/3442032504 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fba500b2930 0x7fba500b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:24.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.206+0000 7fba54ff1640 1 -- 172.21.15.88:0/3442032504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba5015a820 msgr2=0x7fba500b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:24.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.206+0000 7fba54ff1640 1 --2- 172.21.15.88:0/3442032504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba5015a820 0x7fba500b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:24.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.206+0000 7fba54ff1640 1 -- 172.21.15.88:0/3442032504 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba40009420 con 0x7fba5015ac00 2023-10-16T20:05:24.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.206+0000 7fba54ff1640 1 --2- 172.21.15.88:0/3442032504 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba5015ac00 0x7fba501f8fe0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fba4800cb40 tx=0x7fba4800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:24.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.206+0000 7fba4cff9640 1 -- 172.21.15.88:0/3442032504 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4800aca0 con 0x7fba5015ac00 2023-10-16T20:05:24.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.206+0000 7fba4cff9640 1 -- 172.21.15.88:0/3442032504 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4800ae00 con 0x7fba5015ac00 2023-10-16T20:05:24.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.206+0000 7fba56a54640 1 -- 172.21.15.88:0/3442032504 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba500b1c80 con 0x7fba5015ac00 2023-10-16T20:05:24.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.206+0000 7fba4cff9640 1 -- 172.21.15.88:0/3442032504 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba48012620 con 0x7fba5015ac00 2023-10-16T20:05:24.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.206+0000 7fba56a54640 1 -- 172.21.15.88:0/3442032504 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba500b4510 con 0x7fba5015ac00 2023-10-16T20:05:24.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.206+0000 7fba56a54640 1 -- 172.21.15.88:0/3442032504 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba500b4760 con 0x7fba5015ac00 2023-10-16T20:05:24.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.206+0000 7fba4cff9640 1 -- 172.21.15.88:0/3442032504 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fba48007070 con 0x7fba5015ac00 2023-10-16T20:05:24.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.210+0000 7fba4cff9640 1 --2- 172.21.15.88:0/3442032504 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fba3c0664a0 0x7fba3c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:24.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.210+0000 7fba4cff9640 1 -- 172.21.15.88:0/3442032504 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fba480841e0 con 0x7fba5015ac00 2023-10-16T20:05:24.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.210+0000 7fba4ffff640 1 --2- 172.21.15.88:0/3442032504 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fba3c0664a0 0x7fba3c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:24.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.210+0000 7fba4ffff640 1 --2- 172.21.15.88:0/3442032504 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fba3c0664a0 0x7fba3c068960 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fba400af000 tx=0x7fba400a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:24.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.214+0000 7fba4cff9640 1 -- 172.21.15.88:0/3442032504 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fba500b4760 con 0x7fba5015ac00 2023-10-16T20:05:24.363 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.362+0000 7fba56a54640 1 -- 172.21.15.88:0/3442032504 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fba500b4760 con 0x7fba5015ac00 2023-10-16T20:05:24.364 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.362+0000 7fba4cff9640 1 -- 172.21.15.88:0/3442032504 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fba500b4760 con 0x7fba5015ac00 2023-10-16T20:05:24.364 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:05:24.366 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.366+0000 7fba56a54640 1 -- 172.21.15.88:0/3442032504 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fba3c0664a0 msgr2=0x7fba3c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:24.366 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.366+0000 7fba56a54640 1 --2- 172.21.15.88:0/3442032504 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fba3c0664a0 0x7fba3c068960 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fba400af000 tx=0x7fba400a7000 comp rx=0 tx=0).stop 2023-10-16T20:05:24.366 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.366+0000 7fba56a54640 1 -- 172.21.15.88:0/3442032504 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba5015ac00 msgr2=0x7fba501f8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:24.367 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.366+0000 7fba56a54640 1 --2- 172.21.15.88:0/3442032504 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba5015ac00 0x7fba501f8fe0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fba4800cb40 tx=0x7fba4800a060 comp rx=0 tx=0).stop 2023-10-16T20:05:24.367 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.366+0000 7fba56a54640 1 -- 172.21.15.88:0/3442032504 shutdown_connections 2023-10-16T20:05:24.367 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.366+0000 7fba56a54640 1 --2- 172.21.15.88:0/3442032504 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fba3c0664a0 0x7fba3c068960 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:24.367 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.366+0000 7fba56a54640 1 --2- 172.21.15.88:0/3442032504 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba5015ac00 0x7fba501f8fe0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:24.367 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.366+0000 7fba56a54640 1 --2- 172.21.15.88:0/3442032504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba5015a820 0x7fba500b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:24.368 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.366+0000 7fba56a54640 1 --2- 172.21.15.88:0/3442032504 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fba500b2930 0x7fba500b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:24.368 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.366+0000 7fba56a54640 1 -- 172.21.15.88:0/3442032504 >> 172.21.15.88:0/3442032504 conn(0x7fba500cd3f0 msgr2=0x7fba5015c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:24.368 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.366+0000 7fba56a54640 1 -- 172.21.15.88:0/3442032504 shutdown_connections 2023-10-16T20:05:24.368 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.366+0000 7fba56a54640 1 -- 172.21.15.88:0/3442032504 wait complete. 2023-10-16T20:05:24.386 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T20:05:24.527 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.526+0000 7f9cdd380640 1 Processor -- start 2023-10-16T20:05:24.527 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.526+0000 7f9cdd380640 1 -- start start 2023-10-16T20:05:24.528 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.526+0000 7f9cdd380640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f9cd80d1be0 0x7f9cd80b4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:24.528 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.526+0000 7f9cdd380640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cd80b7d20 con 0x7f9cd815cb70 2023-10-16T20:05:24.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.526+0000 7f9cdd380640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cd80b4ea0 con 0x7f9cd80b4a70 2023-10-16T20:05:24.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.526+0000 7f9cdd380640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cd80b7110 con 0x7f9cd80d1be0 2023-10-16T20:05:24.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.526+0000 7f9cce7fc640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f9cd80d1be0 0x7f9cd80b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:24.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.526+0000 7f9cd6ffd640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f9cd815cb70 0x7f9cd80b83d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:60812/0 (socket says 172.21.15.88:60812) 2023-10-16T20:05:24.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.526+0000 7f9cd6ffd640 1 -- 172.21.15.88:0/1650023834 learned_addr learned my addr 172.21.15.88:0/1650023834 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:24.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.526+0000 7f9cce7fc640 1 -- 172.21.15.88:0/1650023834 >> v1:172.21.15.94:6789/0 conn(0x7f9cd80b4a70 legacy=0x7f9cd80b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:24.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.526+0000 7f9cce7fc640 1 -- 172.21.15.88:0/1650023834 >> v1:172.21.15.88:6789/0 conn(0x7f9cd815cb70 legacy=0x7f9cd80b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:24.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.526+0000 7f9cce7fc640 1 -- 172.21.15.88:0/1650023834 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd80b73f0 con 0x7f9cd80d1be0 2023-10-16T20:05:24.531 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.530+0000 7f9cce7fc640 1 --2- 172.21.15.88:0/1650023834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9cd80d1be0 0x7f9cd80b4530 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00097c0 tx=0x7f9cc001ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625cc2e1668b252d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:24.531 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.530+0000 7f9cd67fc640 1 -- 172.21.15.88:0/1650023834 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc00a8930 con 0x7f9cd80d1be0 2023-10-16T20:05:24.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.530+0000 7f9cd67fc640 1 -- 172.21.15.88:0/1650023834 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc00a8a90 con 0x7f9cd80d1be0 2023-10-16T20:05:24.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.530+0000 7f9cd67fc640 1 -- 172.21.15.88:0/1650023834 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc00a8e20 con 0x7f9cd80d1be0 2023-10-16T20:05:24.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.530+0000 7f9cdd380640 1 -- 172.21.15.88:0/1650023834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9cd80d1be0 msgr2=0x7f9cd80b4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:24.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.530+0000 7f9cdd380640 1 --2- 172.21.15.88:0/1650023834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9cd80d1be0 0x7f9cd80b4530 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00097c0 tx=0x7f9cc001ec30 comp rx=0 tx=0).stop 2023-10-16T20:05:24.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.530+0000 7f9cdd380640 1 -- 172.21.15.88:0/1650023834 shutdown_connections 2023-10-16T20:05:24.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.530+0000 7f9cdd380640 1 --2- 172.21.15.88:0/1650023834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9cd80d1be0 0x7f9cd80b4530 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:24.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.530+0000 7f9cdd380640 1 -- 172.21.15.88:0/1650023834 >> 172.21.15.88:0/1650023834 conn(0x7f9cd806b6e0 msgr2=0x7f9cd806baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:24.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.530+0000 7f9cdd380640 1 -- 172.21.15.88:0/1650023834 shutdown_connections 2023-10-16T20:05:24.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.530+0000 7f9cdd380640 1 -- 172.21.15.88:0/1650023834 wait complete. 2023-10-16T20:05:24.535 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.534+0000 7f9cdd380640 1 Processor -- start 2023-10-16T20:05:24.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.534+0000 7f9cdd380640 1 -- start start 2023-10-16T20:05:24.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.534+0000 7f9cdd380640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cd80b4a70 0x7f9cd80c6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:24.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.534+0000 7f9cdd380640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9cd80d1be0 0x7f9cd815e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:24.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.534+0000 7f9cdd380640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9cd815cb70 0x7f9cd80c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:24.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.534+0000 7f9cdd380640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd814d360 con 0x7f9cd80d1be0 2023-10-16T20:05:24.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.534+0000 7f9cdd380640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd80c5f80 con 0x7f9cd80b4a70 2023-10-16T20:05:24.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.534+0000 7f9cdd380640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd815ed60 con 0x7f9cd815cb70 2023-10-16T20:05:24.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.534+0000 7f9cce7fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9cd80d1be0 0x7f9cd815e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:24.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.534+0000 7f9cce7fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9cd80d1be0 0x7f9cd815e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38120/0 (socket says 172.21.15.88:38120) 2023-10-16T20:05:24.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.534+0000 7f9cce7fc640 1 -- 172.21.15.88:0/874875139 learned_addr learned my addr 172.21.15.88:0/874875139 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:24.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.534+0000 7f9cce7fc640 1 -- 172.21.15.88:0/874875139 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9cd815cb70 msgr2=0x7f9cd80c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:24.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.534+0000 7f9cce7fc640 1 --2- 172.21.15.88:0/874875139 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9cd815cb70 0x7f9cd80c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:24.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.534+0000 7f9cce7fc640 1 -- 172.21.15.88:0/874875139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cd80b4a70 msgr2=0x7f9cd80c6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:24.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.534+0000 7f9cce7fc640 1 --2- 172.21.15.88:0/874875139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cd80b4a70 0x7f9cd80c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:24.539 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.534+0000 7f9cce7fc640 1 -- 172.21.15.88:0/874875139 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc0009420 con 0x7f9cd80d1be0 2023-10-16T20:05:24.540 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.534+0000 7f9cce7fc640 1 --2- 172.21.15.88:0/874875139 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9cd80d1be0 0x7f9cd815e660 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0009710 tx=0x7f9cc001b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:24.540 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.534+0000 7f9ccffff640 1 -- 172.21.15.88:0/874875139 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc00b0e40 con 0x7f9cd80d1be0 2023-10-16T20:05:24.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.534+0000 7f9ccffff640 1 -- 172.21.15.88:0/874875139 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc00b9410 con 0x7f9cd80d1be0 2023-10-16T20:05:24.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.534+0000 7f9ccffff640 1 -- 172.21.15.88:0/874875139 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc00b97c0 con 0x7f9cd80d1be0 2023-10-16T20:05:24.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.534+0000 7f9cdd380640 1 -- 172.21.15.88:0/874875139 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd815f040 con 0x7f9cd80d1be0 2023-10-16T20:05:24.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.534+0000 7f9cdd380640 1 -- 172.21.15.88:0/874875139 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd815f600 con 0x7f9cd80d1be0 2023-10-16T20:05:24.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.538+0000 7f9cdd380640 1 -- 172.21.15.88:0/874875139 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cd80667b0 con 0x7f9cd80d1be0 2023-10-16T20:05:24.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.538+0000 7f9ccffff640 1 -- 172.21.15.88:0/874875139 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9cc00b7040 con 0x7f9cd80d1be0 2023-10-16T20:05:24.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.538+0000 7f9ccffff640 1 --2- 172.21.15.88:0/874875139 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9cb80664a0 0x7f9cb8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:24.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.538+0000 7f9ccffff640 1 -- 172.21.15.88:0/874875139 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9cc0121fe0 con 0x7f9cd80d1be0 2023-10-16T20:05:24.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.538+0000 7f9cd6ffd640 1 --2- 172.21.15.88:0/874875139 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9cb80664a0 0x7f9cb8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:24.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.538+0000 7f9cd6ffd640 1 --2- 172.21.15.88:0/874875139 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9cb80664a0 0x7f9cb8068960 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8001560 tx=0x7f9cc8007a50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:24.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.542+0000 7f9ccffff640 1 -- 172.21.15.88:0/874875139 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9cc00ef040 con 0x7f9cd80d1be0 2023-10-16T20:05:24.692 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.690+0000 7f9cdd380640 1 -- 172.21.15.88:0/874875139 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9cd80667b0 con 0x7f9cd80d1be0 2023-10-16T20:05:24.693 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.690+0000 7f9ccffff640 1 -- 172.21.15.88:0/874875139 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9cc00b5020 con 0x7f9cd80d1be0 2023-10-16T20:05:24.693 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:05:24.694 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.694+0000 7f9cdd380640 1 -- 172.21.15.88:0/874875139 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9cb80664a0 msgr2=0x7f9cb8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:24.695 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.694+0000 7f9cdd380640 1 --2- 172.21.15.88:0/874875139 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9cb80664a0 0x7f9cb8068960 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8001560 tx=0x7f9cc8007a50 comp rx=0 tx=0).stop 2023-10-16T20:05:24.695 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.694+0000 7f9cdd380640 1 -- 172.21.15.88:0/874875139 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9cd80d1be0 msgr2=0x7f9cd815e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:24.695 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.694+0000 7f9cdd380640 1 --2- 172.21.15.88:0/874875139 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9cd80d1be0 0x7f9cd815e660 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0009710 tx=0x7f9cc001b9d0 comp rx=0 tx=0).stop 2023-10-16T20:05:24.695 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.694+0000 7f9cdd380640 1 -- 172.21.15.88:0/874875139 shutdown_connections 2023-10-16T20:05:24.696 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.694+0000 7f9cdd380640 1 --2- 172.21.15.88:0/874875139 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9cb80664a0 0x7f9cb8068960 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:24.696 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.694+0000 7f9cdd380640 1 --2- 172.21.15.88:0/874875139 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9cd815cb70 0x7f9cd80c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:24.696 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.694+0000 7f9cdd380640 1 --2- 172.21.15.88:0/874875139 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9cd80d1be0 0x7f9cd815e660 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:24.696 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.694+0000 7f9cdd380640 1 --2- 172.21.15.88:0/874875139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cd80b4a70 0x7f9cd80c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:24.696 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.694+0000 7f9cdd380640 1 -- 172.21.15.88:0/874875139 >> 172.21.15.88:0/874875139 conn(0x7f9cd806b6e0 msgr2=0x7f9cd80d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:24.697 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.694+0000 7f9cdd380640 1 -- 172.21.15.88:0/874875139 shutdown_connections 2023-10-16T20:05:24.697 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.694+0000 7f9cdd380640 1 -- 172.21.15.88:0/874875139 wait complete. 2023-10-16T20:05:24.713 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T20:05:24.854 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.854+0000 7f7d5a70f640 1 Processor -- start 2023-10-16T20:05:24.854 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.854+0000 7f7d5a70f640 1 -- start start 2023-10-16T20:05:24.856 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.854+0000 7f7d5a70f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d5415a820 0x7f7d540cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:24.856 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.854+0000 7f7d5a70f640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f7d540b2b00 0x7f7d5406c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:24.856 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.854+0000 7f7d5a70f640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d540ce4b0 con 0x7f7d5415ac00 2023-10-16T20:05:24.856 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.854+0000 7f7d5a70f640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d5415e270 con 0x7f7d540b2b00 2023-10-16T20:05:24.857 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.854+0000 7f7d5a70f640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d540ce700 con 0x7f7d5415a820 2023-10-16T20:05:24.857 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.854+0000 7f7d58cac640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f7d540b2b00 0x7f7d5406c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:24.857 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.854+0000 7f7d58cac640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f7d540b2b00 0x7f7d5406c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38122/0 (socket says 172.21.15.88:38122) 2023-10-16T20:05:24.857 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.854+0000 7f7d58cac640 1 -- 172.21.15.88:0/4085979283 learned_addr learned my addr 172.21.15.88:0/4085979283 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:24.857 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.854+0000 7f7d53fff640 1 --2- 172.21.15.88:0/4085979283 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d5415a820 0x7f7d540cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:24.858 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.854+0000 7f7d58cac640 1 -- 172.21.15.88:0/4085979283 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d5415a820 msgr2=0x7f7d540cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:24.858 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.854+0000 7f7d58cac640 1 --2- 172.21.15.88:0/4085979283 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d5415a820 0x7f7d540cf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:24.858 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.854+0000 7f7d58cac640 1 -- 172.21.15.88:0/4085979283 >> v1:172.21.15.94:6789/0 conn(0x7f7d5415ac00 legacy=0x7f7d540b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:24.858 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.854+0000 7f7d58cac640 1 -- 172.21.15.88:0/4085979283 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d540ce9e0 con 0x7f7d540b2b00 2023-10-16T20:05:24.859 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.858+0000 7f7d58cac640 1 --2- 172.21.15.88:0/4085979283 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d540b2b00 0x7f7d5406c340 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c009e50 tx=0x7f7d4c01c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e85dfeeb481fe6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:24.859 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.858+0000 7f7d52ffd640 1 -- 172.21.15.88:0/4085979283 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5406cea0 con 0x7f7d540b2b00 2023-10-16T20:05:24.859 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.858+0000 7f7d52ffd640 1 -- 172.21.15.88:0/4085979283 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d4c0a58d0 con 0x7f7d540b2b00 2023-10-16T20:05:24.860 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.858+0000 7f7d52ffd640 1 -- 172.21.15.88:0/4085979283 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c0a5c60 con 0x7f7d540b2b00 2023-10-16T20:05:24.861 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.858+0000 7f7d5a70f640 1 -- 172.21.15.88:0/4085979283 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d540b2b00 msgr2=0x7f7d5406c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:24.861 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.858+0000 7f7d5a70f640 1 --2- 172.21.15.88:0/4085979283 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d540b2b00 0x7f7d5406c340 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c009e50 tx=0x7f7d4c01c9e0 comp rx=0 tx=0).stop 2023-10-16T20:05:24.861 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.858+0000 7f7d5a70f640 1 -- 172.21.15.88:0/4085979283 shutdown_connections 2023-10-16T20:05:24.861 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.858+0000 7f7d5a70f640 1 --2- 172.21.15.88:0/4085979283 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d540b2b00 0x7f7d5406c340 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:24.861 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.858+0000 7f7d5a70f640 1 --2- 172.21.15.88:0/4085979283 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d5415a820 0x7f7d540cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:24.862 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.858+0000 7f7d5a70f640 1 -- 172.21.15.88:0/4085979283 >> 172.21.15.88:0/4085979283 conn(0x7f7d540cd3f0 msgr2=0x7f7d540cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:24.862 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.858+0000 7f7d5a70f640 1 -- 172.21.15.88:0/4085979283 shutdown_connections 2023-10-16T20:05:24.862 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.858+0000 7f7d5a70f640 1 -- 172.21.15.88:0/4085979283 wait complete. 2023-10-16T20:05:24.864 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.862+0000 7f7d5a70f640 1 Processor -- start 2023-10-16T20:05:24.864 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.862+0000 7f7d5a70f640 1 -- start start 2023-10-16T20:05:24.865 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.862+0000 7f7d5a70f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d540b2b00 0x7f7d540b1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:24.865 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.862+0000 7f7d5a70f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d5415a820 0x7f7d5415c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:24.865 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.862+0000 7f7d5a70f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d5415ac00 0x7f7d540c4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:24.865 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.862+0000 7f7d5a70f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d540cc5d0 con 0x7f7d540b2b00 2023-10-16T20:05:24.865 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.862+0000 7f7d5a70f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d540b3ea0 con 0x7f7d5415ac00 2023-10-16T20:05:24.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.862+0000 7f7d5a70f640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d540b1450 con 0x7f7d5415a820 2023-10-16T20:05:24.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.862+0000 7f7d53fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d540b2b00 0x7f7d540b1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:24.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.862+0000 7f7d53fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d540b2b00 0x7f7d540b1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38130/0 (socket says 172.21.15.88:38130) 2023-10-16T20:05:24.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.862+0000 7f7d53fff640 1 -- 172.21.15.88:0/2795566957 learned_addr learned my addr 172.21.15.88:0/2795566957 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:24.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.862+0000 7f7d537fe640 1 --2- 172.21.15.88:0/2795566957 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d5415a820 0x7f7d5415c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:24.867 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.862+0000 7f7d58cac640 1 --2- 172.21.15.88:0/2795566957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d5415ac00 0x7f7d540c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:24.867 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.862+0000 7f7d53fff640 1 -- 172.21.15.88:0/2795566957 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d5415a820 msgr2=0x7f7d5415c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:24.867 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.862+0000 7f7d53fff640 1 --2- 172.21.15.88:0/2795566957 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d5415a820 0x7f7d5415c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:24.867 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.862+0000 7f7d53fff640 1 -- 172.21.15.88:0/2795566957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d5415ac00 msgr2=0x7f7d540c4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:24.867 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.862+0000 7f7d53fff640 1 --2- 172.21.15.88:0/2795566957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d5415ac00 0x7f7d540c4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:24.868 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.862+0000 7f7d53fff640 1 -- 172.21.15.88:0/2795566957 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4c009b30 con 0x7f7d540b2b00 2023-10-16T20:05:24.868 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.862+0000 7f7d537fe640 1 --2- 172.21.15.88:0/2795566957 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d5415a820 0x7f7d5415c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:24.868 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.862+0000 7f7d53fff640 1 --2- 172.21.15.88:0/2795566957 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d540b2b00 0x7f7d540b1c80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7d4400e960 tx=0x7f7d4400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:24.868 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.862+0000 7f7d50ff9640 1 -- 172.21.15.88:0/2795566957 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d44009690 con 0x7f7d540b2b00 2023-10-16T20:05:24.868 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.862+0000 7f7d50ff9640 1 -- 172.21.15.88:0/2795566957 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d4400ca00 con 0x7f7d540b2b00 2023-10-16T20:05:24.869 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.862+0000 7f7d50ff9640 1 -- 172.21.15.88:0/2795566957 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4400cd70 con 0x7f7d540b2b00 2023-10-16T20:05:24.869 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.862+0000 7f7d5a70f640 1 -- 172.21.15.88:0/2795566957 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d540b08d0 con 0x7f7d540b2b00 2023-10-16T20:05:24.869 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.862+0000 7f7d5a70f640 1 -- 172.21.15.88:0/2795566957 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d540b0e90 con 0x7f7d540b2b00 2023-10-16T20:05:24.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.862+0000 7f7d5a70f640 1 -- 172.21.15.88:0/2795566957 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d5404e9c0 con 0x7f7d540b2b00 2023-10-16T20:05:24.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.866+0000 7f7d50ff9640 1 -- 172.21.15.88:0/2795566957 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7d44013470 con 0x7f7d540b2b00 2023-10-16T20:05:24.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.866+0000 7f7d50ff9640 1 --2- 172.21.15.88:0/2795566957 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7d400664a0 0x7f7d40068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:24.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.866+0000 7f7d50ff9640 1 -- 172.21.15.88:0/2795566957 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7d44084390 con 0x7f7d540b2b00 2023-10-16T20:05:24.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.866+0000 7f7d537fe640 1 --2- 172.21.15.88:0/2795566957 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7d400664a0 0x7f7d40068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:24.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.866+0000 7f7d537fe640 1 --2- 172.21.15.88:0/2795566957 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7d400664a0 0x7f7d40068960 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7d540b19e0 tx=0x7f7d48009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:24.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:24.870+0000 7f7d50ff9640 1 -- 172.21.15.88:0/2795566957 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7d44051500 con 0x7f7d540b2b00 2023-10-16T20:05:25.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.018+0000 7f7d5a70f640 1 -- 172.21.15.88:0/2795566957 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7d5406b110 con 0x7f7d540b2b00 2023-10-16T20:05:25.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.018+0000 7f7d50ff9640 1 -- 172.21.15.88:0/2795566957 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7d44054b20 con 0x7f7d540b2b00 2023-10-16T20:05:25.021 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2023-10-16T20:05:25.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.022+0000 7f7d5a70f640 1 -- 172.21.15.88:0/2795566957 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7d400664a0 msgr2=0x7f7d40068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:25.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.022+0000 7f7d5a70f640 1 --2- 172.21.15.88:0/2795566957 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7d400664a0 0x7f7d40068960 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7d540b19e0 tx=0x7f7d48009040 comp rx=0 tx=0).stop 2023-10-16T20:05:25.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.022+0000 7f7d5a70f640 1 -- 172.21.15.88:0/2795566957 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d540b2b00 msgr2=0x7f7d540b1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:25.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.022+0000 7f7d5a70f640 1 --2- 172.21.15.88:0/2795566957 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d540b2b00 0x7f7d540b1c80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7d4400e960 tx=0x7f7d4400ee20 comp rx=0 tx=0).stop 2023-10-16T20:05:25.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.022+0000 7f7d5a70f640 1 -- 172.21.15.88:0/2795566957 shutdown_connections 2023-10-16T20:05:25.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.022+0000 7f7d5a70f640 1 --2- 172.21.15.88:0/2795566957 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7d400664a0 0x7f7d40068960 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:25.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.022+0000 7f7d5a70f640 1 --2- 172.21.15.88:0/2795566957 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d5415a820 0x7f7d5415c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:25.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.022+0000 7f7d5a70f640 1 --2- 172.21.15.88:0/2795566957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d5415ac00 0x7f7d540c4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:25.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.022+0000 7f7d5a70f640 1 --2- 172.21.15.88:0/2795566957 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d540b2b00 0x7f7d540b1c80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:25.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.022+0000 7f7d5a70f640 1 -- 172.21.15.88:0/2795566957 >> 172.21.15.88:0/2795566957 conn(0x7f7d540cd3f0 msgr2=0x7f7d5415b300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:25.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.022+0000 7f7d5a70f640 1 -- 172.21.15.88:0/2795566957 shutdown_connections 2023-10-16T20:05:25.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.022+0000 7f7d5a70f640 1 -- 172.21.15.88:0/2795566957 wait complete. 2023-10-16T20:05:25.041 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T20:05:25.180 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.178+0000 7f2bcac17640 1 Processor -- start 2023-10-16T20:05:25.180 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.178+0000 7f2bcac17640 1 -- start start 2023-10-16T20:05:25.182 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.182+0000 7f2bcac17640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f2bc40cf940 0x7f2bc40d3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:25.182 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.182+0000 7f2bcac17640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc40d0c40 con 0x7f2bc415c9a0 2023-10-16T20:05:25.182 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.182+0000 7f2bcac17640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc40ce2d0 con 0x7f2bc40cde80 2023-10-16T20:05:25.182 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.182+0000 7f2bcac17640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bc40d37b0 con 0x7f2bc40cf940 2023-10-16T20:05:25.183 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.182+0000 7f2bc9c15640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f2bc415c9a0 0x7f2bc40d10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:60820/0 (socket says 172.21.15.88:60820) 2023-10-16T20:05:25.183 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.182+0000 7f2bc9c15640 1 -- 172.21.15.88:0/483459596 learned_addr learned my addr 172.21.15.88:0/483459596 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:25.183 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.182+0000 7f2bca416640 1 --2- 172.21.15.88:0/483459596 >> v2:172.21.15.88:3300/0 conn(0x7f2bc40cf940 0x7f2bc40d3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:25.184 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.182+0000 7f2bc8c13640 1 -- 172.21.15.88:0/483459596 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3294063386 0 0) 0x7f2bc40d0c40 con 0x7f2bc415c9a0 2023-10-16T20:05:25.184 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.182+0000 7f2bc8c13640 1 -- 172.21.15.88:0/483459596 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bb0003530 con 0x7f2bc415c9a0 2023-10-16T20:05:25.184 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.182+0000 7f2bca416640 1 -- 172.21.15.88:0/483459596 >> v1:172.21.15.94:6789/0 conn(0x7f2bc40cde80 legacy=0x7f2bc40cf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:25.184 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.182+0000 7f2bca416640 1 -- 172.21.15.88:0/483459596 >> v1:172.21.15.88:6789/0 conn(0x7f2bc415c9a0 legacy=0x7f2bc40d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:25.185 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.182+0000 7f2bca416640 1 -- 172.21.15.88:0/483459596 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc40d0630 con 0x7f2bc40cf940 2023-10-16T20:05:25.185 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.186+0000 7f2bca416640 1 --2- 172.21.15.88:0/483459596 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc40cf940 0x7f2bc40d3270 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc009d90 tx=0x7f2bbc01c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5213c915d99fbd26 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:25.186 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.186+0000 7f2bc8c13640 1 -- 172.21.15.88:0/483459596 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc0a5870 con 0x7f2bc40cf940 2023-10-16T20:05:25.186 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.186+0000 7f2bc8c13640 1 -- 172.21.15.88:0/483459596 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bbc0a59d0 con 0x7f2bc40cf940 2023-10-16T20:05:25.186 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.186+0000 7f2bc8c13640 1 -- 172.21.15.88:0/483459596 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc0a5d60 con 0x7f2bc40cf940 2023-10-16T20:05:25.187 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.186+0000 7f2bcac17640 1 -- 172.21.15.88:0/483459596 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc40cf940 msgr2=0x7f2bc40d3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:25.187 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.186+0000 7f2bcac17640 1 --2- 172.21.15.88:0/483459596 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc40cf940 0x7f2bc40d3270 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc009d90 tx=0x7f2bbc01c8c0 comp rx=0 tx=0).stop 2023-10-16T20:05:25.187 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.186+0000 7f2bcac17640 1 -- 172.21.15.88:0/483459596 shutdown_connections 2023-10-16T20:05:25.188 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.186+0000 7f2bcac17640 1 --2- 172.21.15.88:0/483459596 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc40cf940 0x7f2bc40d3270 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:25.188 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.186+0000 7f2bcac17640 1 -- 172.21.15.88:0/483459596 >> 172.21.15.88:0/483459596 conn(0x7f2bc409bc30 msgr2=0x7f2bc40d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:25.188 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.186+0000 7f2bcac17640 1 -- 172.21.15.88:0/483459596 shutdown_connections 2023-10-16T20:05:25.188 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.186+0000 7f2bcac17640 1 -- 172.21.15.88:0/483459596 wait complete. 2023-10-16T20:05:25.190 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.186+0000 7f2bcac17640 1 Processor -- start 2023-10-16T20:05:25.191 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.186+0000 7f2bcac17640 1 -- start start 2023-10-16T20:05:25.191 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.190+0000 7f2bcac17640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bc40cde80 0x7f2bc4155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:25.191 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.190+0000 7f2bcac17640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc40cf940 0x7f2bc40cc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:25.191 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.190+0000 7f2bcac17640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bc415c9a0 0x7f2bc4153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:25.191 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.190+0000 7f2bcac17640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc40c6140 con 0x7f2bc40cf940 2023-10-16T20:05:25.192 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.190+0000 7f2bcac17640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc40c6420 con 0x7f2bc415c9a0 2023-10-16T20:05:25.192 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.190+0000 7f2bcac17640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc40c9a30 con 0x7f2bc40cde80 2023-10-16T20:05:25.192 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.190+0000 7f2bc9414640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc40cf940 0x7f2bc40cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:25.192 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.190+0000 7f2bc9414640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc40cf940 0x7f2bc40cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38158/0 (socket says 172.21.15.88:38158) 2023-10-16T20:05:25.192 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.190+0000 7f2bc9414640 1 -- 172.21.15.88:0/1216373997 learned_addr learned my addr 172.21.15.88:0/1216373997 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:25.193 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.190+0000 7f2bc9c15640 1 --2- 172.21.15.88:0/1216373997 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bc40cde80 0x7f2bc4155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:25.193 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.190+0000 7f2bca416640 1 --2- 172.21.15.88:0/1216373997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bc415c9a0 0x7f2bc4153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:25.193 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.190+0000 7f2bc9414640 1 -- 172.21.15.88:0/1216373997 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bc40cde80 msgr2=0x7f2bc4155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:25.193 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.190+0000 7f2bc9414640 1 --2- 172.21.15.88:0/1216373997 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bc40cde80 0x7f2bc4155a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:25.194 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.190+0000 7f2bc9414640 1 -- 172.21.15.88:0/1216373997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bc415c9a0 msgr2=0x7f2bc4153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:25.194 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.190+0000 7f2bc9414640 1 --2- 172.21.15.88:0/1216373997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bc415c9a0 0x7f2bc4153630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:25.194 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.190+0000 7f2bc9414640 1 -- 172.21.15.88:0/1216373997 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bbc009a70 con 0x7f2bc40cf940 2023-10-16T20:05:25.194 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.190+0000 7f2bc9414640 1 --2- 172.21.15.88:0/1216373997 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc40cf940 0x7f2bc40cc160 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2bb400c530 tx=0x7f2bb400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:25.194 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.190+0000 7f2bba7fc640 1 -- 172.21.15.88:0/1216373997 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb4005220 con 0x7f2bc40cf940 2023-10-16T20:05:25.195 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.190+0000 7f2bba7fc640 1 -- 172.21.15.88:0/1216373997 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb4005380 con 0x7f2bc40cf940 2023-10-16T20:05:25.195 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.190+0000 7f2bcac17640 1 -- 172.21.15.88:0/1216373997 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc415cf00 con 0x7f2bc40cf940 2023-10-16T20:05:25.195 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.190+0000 7f2bba7fc640 1 -- 172.21.15.88:0/1216373997 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb4007270 con 0x7f2bc40cf940 2023-10-16T20:05:25.195 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.190+0000 7f2bcac17640 1 -- 172.21.15.88:0/1216373997 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc40c9fb0 con 0x7f2bc40cf940 2023-10-16T20:05:25.196 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.190+0000 7f2bcac17640 1 -- 172.21.15.88:0/1216373997 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc40674f0 con 0x7f2bc40cf940 2023-10-16T20:05:25.196 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.190+0000 7f2bba7fc640 1 -- 172.21.15.88:0/1216373997 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2bb40073d0 con 0x7f2bc40cf940 2023-10-16T20:05:25.196 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.194+0000 7f2bba7fc640 1 --2- 172.21.15.88:0/1216373997 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2ba80664a0 0x7f2ba8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:25.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.194+0000 7f2bba7fc640 1 -- 172.21.15.88:0/1216373997 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2bb4083950 con 0x7f2bc40cf940 2023-10-16T20:05:25.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.194+0000 7f2bc9c15640 1 --2- 172.21.15.88:0/1216373997 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2ba80664a0 0x7f2ba8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:25.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.194+0000 7f2bc9c15640 1 --2- 172.21.15.88:0/1216373997 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2ba80664a0 0x7f2ba8068960 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2bc00010f0 tx=0x7f2bc0007bf0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:25.198 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.198+0000 7f2bba7fc640 1 -- 172.21.15.88:0/1216373997 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2bb4050b80 con 0x7f2bc40cf940 2023-10-16T20:05:25.351 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.350+0000 7f2bcac17640 1 -- 172.21.15.88:0/1216373997 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2bc40c6560 con 0x7f2bc40cf940 2023-10-16T20:05:25.351 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.350+0000 7f2bba7fc640 1 -- 172.21.15.88:0/1216373997 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2bb40541a0 con 0x7f2bc40cf940 2023-10-16T20:05:25.352 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:05:25.353 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.354+0000 7f2bcac17640 1 -- 172.21.15.88:0/1216373997 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2ba80664a0 msgr2=0x7f2ba8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:25.354 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.354+0000 7f2bcac17640 1 --2- 172.21.15.88:0/1216373997 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2ba80664a0 0x7f2ba8068960 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2bc00010f0 tx=0x7f2bc0007bf0 comp rx=0 tx=0).stop 2023-10-16T20:05:25.354 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.354+0000 7f2bcac17640 1 -- 172.21.15.88:0/1216373997 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc40cf940 msgr2=0x7f2bc40cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:25.354 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.354+0000 7f2bcac17640 1 --2- 172.21.15.88:0/1216373997 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc40cf940 0x7f2bc40cc160 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2bb400c530 tx=0x7f2bb400c9f0 comp rx=0 tx=0).stop 2023-10-16T20:05:25.354 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.354+0000 7f2bcac17640 1 -- 172.21.15.88:0/1216373997 shutdown_connections 2023-10-16T20:05:25.355 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.354+0000 7f2bcac17640 1 --2- 172.21.15.88:0/1216373997 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2ba80664a0 0x7f2ba8068960 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:25.355 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.354+0000 7f2bcac17640 1 --2- 172.21.15.88:0/1216373997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bc415c9a0 0x7f2bc4153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:25.355 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.354+0000 7f2bcac17640 1 --2- 172.21.15.88:0/1216373997 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc40cf940 0x7f2bc40cc160 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:25.355 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.354+0000 7f2bcac17640 1 --2- 172.21.15.88:0/1216373997 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2bc40cde80 0x7f2bc4155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:25.355 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.354+0000 7f2bcac17640 1 -- 172.21.15.88:0/1216373997 >> 172.21.15.88:0/1216373997 conn(0x7f2bc409bc30 msgr2=0x7f2bc40d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:25.356 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.354+0000 7f2bcac17640 1 -- 172.21.15.88:0/1216373997 shutdown_connections 2023-10-16T20:05:25.356 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.354+0000 7f2bcac17640 1 -- 172.21.15.88:0/1216373997 wait complete. 2023-10-16T20:05:25.372 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T20:05:25.512 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.510+0000 7f14e1f65640 1 Processor -- start 2023-10-16T20:05:25.513 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.510+0000 7f14e1f65640 1 -- start start 2023-10-16T20:05:25.513 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.514+0000 7f14e1f65640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14dc153ea0 0x7f14dc0d1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:25.514 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.514+0000 7f14e1f65640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14dc0b5510 con 0x7f14dc0cd970 2023-10-16T20:05:25.514 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.514+0000 7f14e1f65640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14dc15d9a0 con 0x7f14dc0d2660 2023-10-16T20:05:25.514 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.514+0000 7f14e1f65640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc0b4990 con 0x7f14dc153ea0 2023-10-16T20:05:25.514 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.514+0000 7f14dbfff640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f14dc0cd970 0x7f14dc0cdde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:60822/0 (socket says 172.21.15.88:60822) 2023-10-16T20:05:25.515 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.514+0000 7f14dbfff640 1 -- 172.21.15.88:0/1537923264 learned_addr learned my addr 172.21.15.88:0/1537923264 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:25.516 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.514+0000 7f14db7fe640 1 --2- 172.21.15.88:0/1537923264 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14dc153ea0 0x7f14dc0d1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:25.517 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.514+0000 7f14daffd640 1 -- 172.21.15.88:0/1537923264 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3212339155 0 0) 0x7f14dc0b5510 con 0x7f14dc0cd970 2023-10-16T20:05:25.517 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.514+0000 7f14daffd640 1 -- 172.21.15.88:0/1537923264 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14c8003530 con 0x7f14dc0cd970 2023-10-16T20:05:25.517 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.514+0000 7f14daffd640 1 -- 172.21.15.88:0/1537923264 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3064077091 0 0) 0x7f14dc15d9a0 con 0x7f14dc0d2660 2023-10-16T20:05:25.517 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.514+0000 7f14daffd640 1 -- 172.21.15.88:0/1537923264 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14dc0b5510 con 0x7f14dc0d2660 2023-10-16T20:05:25.518 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.514+0000 7f14daffd640 1 -- 172.21.15.88:0/1537923264 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4267462493 0 0) 0x7f14c8003530 con 0x7f14dc0cd970 2023-10-16T20:05:25.518 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.514+0000 7f14daffd640 1 -- 172.21.15.88:0/1537923264 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14dc153ea0 msgr2=0x7f14dc0d1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:25.518 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.514+0000 7f14daffd640 1 --2- 172.21.15.88:0/1537923264 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14dc153ea0 0x7f14dc0d1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:25.518 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.514+0000 7f14daffd640 1 -- 172.21.15.88:0/1537923264 >> v1:172.21.15.94:6789/0 conn(0x7f14dc0d2660 legacy=0x7f14dc15d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:25.518 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.514+0000 7f14daffd640 1 -- 172.21.15.88:0/1537923264 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14dc0b4c70 con 0x7f14dc0cd970 2023-10-16T20:05:25.518 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.514+0000 7f14daffd640 1 -- 172.21.15.88:0/1537923264 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f14d4001a00 con 0x7f14dc0cd970 2023-10-16T20:05:25.519 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.514+0000 7f14daffd640 1 -- 172.21.15.88:0/1537923264 >> v1:172.21.15.88:6789/0 conn(0x7f14dc0cd970 legacy=0x7f14dc0cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:25.519 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.514+0000 7f14daffd640 1 --2- 172.21.15.88:0/1537923264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14c8003b40 0x7f14c8003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:25.519 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.514+0000 7f14daffd640 1 --2- 172.21.15.88:0/1537923264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14c80047d0 0x7f14c8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:25.519 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.514+0000 7f14daffd640 1 --2- 172.21.15.88:0/1537923264 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14dc153ea0 0x7f14c8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:25.519 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.514+0000 7f14daffd640 1 -- 172.21.15.88:0/1537923264 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc0b4990 con 0x7f14c8003b40 2023-10-16T20:05:25.520 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.514+0000 7f14daffd640 1 -- 172.21.15.88:0/1537923264 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c800ba40 con 0x7f14c80047d0 2023-10-16T20:05:25.520 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.514+0000 7f14daffd640 1 -- 172.21.15.88:0/1537923264 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14c800bc00 con 0x7f14dc153ea0 2023-10-16T20:05:25.520 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.514+0000 7f14db7fe640 1 --2- 172.21.15.88:0/1537923264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14c8003b40 0x7f14c8003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:25.520 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.514+0000 7f14d3fff640 1 --2- 172.21.15.88:0/1537923264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14c80047d0 0x7f14c8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:25.520 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.514+0000 7f14dbfff640 1 --2- 172.21.15.88:0/1537923264 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14dc153ea0 0x7f14c8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:25.521 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.514+0000 7f14db7fe640 1 -- 172.21.15.88:0/1537923264 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14dc153ea0 msgr2=0x7f14c8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:25.521 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.514+0000 7f14db7fe640 1 --2- 172.21.15.88:0/1537923264 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14dc153ea0 0x7f14c8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:25.521 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.514+0000 7f14db7fe640 1 -- 172.21.15.88:0/1537923264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14c80047d0 msgr2=0x7f14c8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:25.521 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.514+0000 7f14db7fe640 1 --2- 172.21.15.88:0/1537923264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14c80047d0 0x7f14c8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:25.521 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.514+0000 7f14db7fe640 1 -- 172.21.15.88:0/1537923264 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c8003530 con 0x7f14c8003b40 2023-10-16T20:05:25.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.518+0000 7f14db7fe640 1 --2- 172.21.15.88:0/1537923264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14c8003b40 0x7f14c8003f40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f14cc009ae0 tx=0x7f14cc01ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316dd8f77fc8140b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:25.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.518+0000 7f14daffd640 1 -- 172.21.15.88:0/1537923264 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc0a6510 con 0x7f14c8003b40 2023-10-16T20:05:25.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.518+0000 7f14daffd640 1 -- 172.21.15.88:0/1537923264 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14cc0a6670 con 0x7f14c8003b40 2023-10-16T20:05:25.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.518+0000 7f14daffd640 1 -- 172.21.15.88:0/1537923264 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc0a6a20 con 0x7f14c8003b40 2023-10-16T20:05:25.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.518+0000 7f14e1f65640 1 -- 172.21.15.88:0/1537923264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14c8003b40 msgr2=0x7f14c8003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:25.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.518+0000 7f14e1f65640 1 --2- 172.21.15.88:0/1537923264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14c8003b40 0x7f14c8003f40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f14cc009ae0 tx=0x7f14cc01ca20 comp rx=0 tx=0).stop 2023-10-16T20:05:25.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.518+0000 7f14dbfff640 1 -- 172.21.15.88:0/1537923264 reap_dead start 2023-10-16T20:05:25.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.518+0000 7f14e1f65640 1 -- 172.21.15.88:0/1537923264 shutdown_connections 2023-10-16T20:05:25.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.518+0000 7f14e1f65640 1 -- 172.21.15.88:0/1537923264 >> 172.21.15.88:0/1537923264 conn(0x7f14dc0715c0 msgr2=0x7f14dc0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:25.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.518+0000 7f14e1f65640 1 -- 172.21.15.88:0/1537923264 shutdown_connections 2023-10-16T20:05:25.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.518+0000 7f14e1f65640 1 -- 172.21.15.88:0/1537923264 wait complete. 2023-10-16T20:05:25.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.518+0000 7f14e1f65640 1 Processor -- start 2023-10-16T20:05:25.527 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.518+0000 7f14e1f65640 1 -- start start 2023-10-16T20:05:25.527 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.522+0000 7f14e1f65640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14c8003b40 0x7f14dc154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:25.527 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.522+0000 7f14e1f65640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14dc155570 0x7f14dc0cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:25.527 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.522+0000 7f14e1f65640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14dc0b7310 0x7f14dc0b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:25.527 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.522+0000 7f14e1f65640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc0b7cd0 con 0x7f14dc155570 2023-10-16T20:05:25.528 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.522+0000 7f14e1f65640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc0cbf60 con 0x7f14c8003b40 2023-10-16T20:05:25.528 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.522+0000 7f14e1f65640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc0cb350 con 0x7f14dc0b7310 2023-10-16T20:05:25.528 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.522+0000 7f14d3fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14dc155570 0x7f14dc0cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:25.528 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.522+0000 7f14d3fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14dc155570 0x7f14dc0cc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38164/0 (socket says 172.21.15.88:38164) 2023-10-16T20:05:25.528 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.522+0000 7f14d3fff640 1 -- 172.21.15.88:0/1825154787 learned_addr learned my addr 172.21.15.88:0/1825154787 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:25.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.522+0000 7f14db7fe640 1 --2- 172.21.15.88:0/1825154787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14c8003b40 0x7f14dc154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:25.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.522+0000 7f14dbfff640 1 --2- 172.21.15.88:0/1825154787 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14dc0b7310 0x7f14dc0b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:25.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.522+0000 7f14d3fff640 1 -- 172.21.15.88:0/1825154787 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14dc0b7310 msgr2=0x7f14dc0b7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:25.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.522+0000 7f14d3fff640 1 --2- 172.21.15.88:0/1825154787 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14dc0b7310 0x7f14dc0b7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:25.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.522+0000 7f14d3fff640 1 -- 172.21.15.88:0/1825154787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14c8003b40 msgr2=0x7f14dc154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:25.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.522+0000 7f14d3fff640 1 --2- 172.21.15.88:0/1825154787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14c8003b40 0x7f14dc154280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:25.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.522+0000 7f14d3fff640 1 -- 172.21.15.88:0/1825154787 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14dc0cb630 con 0x7f14dc155570 2023-10-16T20:05:25.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.522+0000 7f14d3fff640 1 --2- 172.21.15.88:0/1825154787 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14dc155570 0x7f14dc0cc3f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f14c400e7b0 tx=0x7f14c400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:25.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.522+0000 7f14d8ff9640 1 -- 172.21.15.88:0/1825154787 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c40050d0 con 0x7f14dc155570 2023-10-16T20:05:25.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.522+0000 7f14d8ff9640 1 -- 172.21.15.88:0/1825154787 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c4005250 con 0x7f14dc155570 2023-10-16T20:05:25.531 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.522+0000 7f14e1f65640 1 -- 172.21.15.88:0/1825154787 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14dc0cb910 con 0x7f14dc155570 2023-10-16T20:05:25.531 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.522+0000 7f14d8ff9640 1 -- 172.21.15.88:0/1825154787 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c40067e0 con 0x7f14dc155570 2023-10-16T20:05:25.531 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.522+0000 7f14e1f65640 1 -- 172.21.15.88:0/1825154787 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14dc1559f0 con 0x7f14dc155570 2023-10-16T20:05:25.531 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.522+0000 7f14e1f65640 1 -- 172.21.15.88:0/1825154787 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14dc04e9c0 con 0x7f14dc155570 2023-10-16T20:05:25.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.526+0000 7f14d8ff9640 1 -- 172.21.15.88:0/1825154787 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f14c4006940 con 0x7f14dc155570 2023-10-16T20:05:25.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.526+0000 7f14d8ff9640 1 --2- 172.21.15.88:0/1825154787 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f14bc0664f0 0x7f14bc0689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:25.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.526+0000 7f14d8ff9640 1 -- 172.21.15.88:0/1825154787 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f14c4083c40 con 0x7f14dc155570 2023-10-16T20:05:25.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.526+0000 7f14db7fe640 1 --2- 172.21.15.88:0/1825154787 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f14bc0664f0 0x7f14bc0689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:25.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.526+0000 7f14db7fe640 1 --2- 172.21.15.88:0/1825154787 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f14bc0664f0 0x7f14bc0689b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f14cc0af000 tx=0x7f14cc0046d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:25.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.530+0000 7f14d8ff9640 1 -- 172.21.15.88:0/1825154787 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f14c4050e60 con 0x7f14dc155570 2023-10-16T20:05:25.709 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.710+0000 7f14e1f65640 1 -- 172.21.15.88:0/1825154787 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14dc0d2360 con 0x7f14dc155570 2023-10-16T20:05:25.710 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.710+0000 7f14d8ff9640 1 -- 172.21.15.88:0/1825154787 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4906 (secure 0 0 0) 0x7f14c4054480 con 0x7f14dc155570 2023-10-16T20:05:25.710 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:25.711 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3803627113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3803627113},{"type":"v1","addr":"172.21.15.94:6834","nonce":3803627113}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3758093239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":3758093239},{"type":"v1","addr":"172.21.15.88:6837","nonce":3758093239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/3836772385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3836772385},{"type":"v1","addr":"172.21.15.88:6836","nonce":3836772385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T20:04:56.759719+0000","modified":"2023-10-16T20:05:14.569618+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6835/555575750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":555575750},{"type":"v1","addr":"172.21.15.94:6835","nonce":555575750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T20:05:25.711 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 9 2023-10-16T20:05:25.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.710+0000 7f14e1f65640 1 -- 172.21.15.88:0/1825154787 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f14bc0664f0 msgr2=0x7f14bc0689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:25.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.710+0000 7f14e1f65640 1 --2- 172.21.15.88:0/1825154787 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f14bc0664f0 0x7f14bc0689b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f14cc0af000 tx=0x7f14cc0046d0 comp rx=0 tx=0).stop 2023-10-16T20:05:25.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.710+0000 7f14e1f65640 1 -- 172.21.15.88:0/1825154787 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14dc155570 msgr2=0x7f14dc0cc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:25.713 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.710+0000 7f14e1f65640 1 --2- 172.21.15.88:0/1825154787 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14dc155570 0x7f14dc0cc3f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f14c400e7b0 tx=0x7f14c400ec70 comp rx=0 tx=0).stop 2023-10-16T20:05:25.713 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.710+0000 7f14e1f65640 1 -- 172.21.15.88:0/1825154787 shutdown_connections 2023-10-16T20:05:25.713 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.710+0000 7f14e1f65640 1 --2- 172.21.15.88:0/1825154787 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f14bc0664f0 0x7f14bc0689b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:25.713 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.710+0000 7f14e1f65640 1 --2- 172.21.15.88:0/1825154787 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14dc0b7310 0x7f14dc0b7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:25.713 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.710+0000 7f14e1f65640 1 --2- 172.21.15.88:0/1825154787 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14dc155570 0x7f14dc0cc3f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:25.713 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.710+0000 7f14e1f65640 1 --2- 172.21.15.88:0/1825154787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14c8003b40 0x7f14dc154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:25.714 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.710+0000 7f14e1f65640 1 -- 172.21.15.88:0/1825154787 >> 172.21.15.88:0/1825154787 conn(0x7f14dc0715c0 msgr2=0x7f14dc0d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:25.714 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.710+0000 7f14e1f65640 1 -- 172.21.15.88:0/1825154787 shutdown_connections 2023-10-16T20:05:25.714 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.710+0000 7f14e1f65640 1 -- 172.21.15.88:0/1825154787 wait complete. 2023-10-16T20:05:25.731 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:05:25.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.870+0000 7f3aff1c4640 1 Processor -- start 2023-10-16T20:05:25.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.870+0000 7f3aff1c4640 1 -- start start 2023-10-16T20:05:25.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.870+0000 7f3aff1c4640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f3af80d22e0 0x7f3af8161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:25.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.870+0000 7f3aff1c4640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3af80d26c0 0x7f3af815d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:25.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.870+0000 7f3aff1c4640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3af80b87e0 con 0x7f3af815d820 2023-10-16T20:05:25.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.870+0000 7f3aff1c4640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3af815dc50 con 0x7f3af80d22e0 2023-10-16T20:05:25.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.870+0000 7f3aff1c4640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3af815df10 con 0x7f3af80d26c0 2023-10-16T20:05:25.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.870+0000 7f3afcf60640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f3af80d22e0 0x7f3af8161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:25.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.870+0000 7f3afcf60640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f3af80d22e0 0x7f3af8161450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38176/0 (socket says 172.21.15.88:38176) 2023-10-16T20:05:25.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.870+0000 7f3afcf60640 1 -- 172.21.15.88:0/3377451500 learned_addr learned my addr 172.21.15.88:0/3377451500 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:25.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.870+0000 7f3af7fff640 1 --2- 172.21.15.88:0/3377451500 >> v2:172.21.15.94:3300/0 conn(0x7f3af80d26c0 0x7f3af815d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:25.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.870+0000 7f3afcf60640 1 -- 172.21.15.88:0/3377451500 >> v2:172.21.15.94:3300/0 conn(0x7f3af80d26c0 msgr2=0x7f3af815d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:25.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.870+0000 7f3afcf60640 1 --2- 172.21.15.88:0/3377451500 >> v2:172.21.15.94:3300/0 conn(0x7f3af80d26c0 0x7f3af815d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:25.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.870+0000 7f3afcf60640 1 -- 172.21.15.88:0/3377451500 >> v1:172.21.15.94:6789/0 conn(0x7f3af815d820 legacy=0x7f3af80cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:25.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.870+0000 7f3afcf60640 1 -- 172.21.15.88:0/3377451500 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af815e1f0 con 0x7f3af80d22e0 2023-10-16T20:05:25.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.874+0000 7f3afcf60640 1 --2- 172.21.15.88:0/3377451500 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3af80d22e0 0x7f3af8161450 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3af80724f0 tx=0x7f3aec01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a1b3546183d1c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:25.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.874+0000 7f3af77fe640 1 -- 172.21.15.88:0/3377451500 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec004230 con 0x7f3af80d22e0 2023-10-16T20:05:25.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.874+0000 7f3af77fe640 1 -- 172.21.15.88:0/3377451500 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aec0a7050 con 0x7f3af80d22e0 2023-10-16T20:05:25.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.874+0000 7f3af77fe640 1 -- 172.21.15.88:0/3377451500 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec003620 con 0x7f3af80d22e0 2023-10-16T20:05:25.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.874+0000 7f3aff1c4640 1 -- 172.21.15.88:0/3377451500 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3af80d22e0 msgr2=0x7f3af8161450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:25.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.874+0000 7f3aff1c4640 1 --2- 172.21.15.88:0/3377451500 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3af80d22e0 0x7f3af8161450 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3af80724f0 tx=0x7f3aec01c440 comp rx=0 tx=0).stop 2023-10-16T20:05:25.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.874+0000 7f3aff1c4640 1 -- 172.21.15.88:0/3377451500 shutdown_connections 2023-10-16T20:05:25.878 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.874+0000 7f3aff1c4640 1 --2- 172.21.15.88:0/3377451500 >> v2:172.21.15.94:3300/0 conn(0x7f3af80d26c0 0x7f3af815d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:25.878 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.874+0000 7f3aff1c4640 1 --2- 172.21.15.88:0/3377451500 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3af80d22e0 0x7f3af8161450 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:25.878 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.874+0000 7f3aff1c4640 1 -- 172.21.15.88:0/3377451500 >> 172.21.15.88:0/3377451500 conn(0x7f3af80715c0 msgr2=0x7f3af80719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:25.878 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.874+0000 7f3aff1c4640 1 -- 172.21.15.88:0/3377451500 shutdown_connections 2023-10-16T20:05:25.879 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.874+0000 7f3aff1c4640 1 -- 172.21.15.88:0/3377451500 wait complete. 2023-10-16T20:05:25.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.878+0000 7f3aff1c4640 1 Processor -- start 2023-10-16T20:05:25.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.878+0000 7f3aff1c4640 1 -- start start 2023-10-16T20:05:25.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.878+0000 7f3aff1c4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3af80d22e0 0x7f3af81492f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:25.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.878+0000 7f3aff1c4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3af80d26c0 0x7f3af80b6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:25.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.878+0000 7f3aff1c4640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3af815d820 0x7f3af80c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:25.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.878+0000 7f3aff1c4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af8155eb0 con 0x7f3af80d26c0 2023-10-16T20:05:25.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.878+0000 7f3aff1c4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af8149830 con 0x7f3af80d22e0 2023-10-16T20:05:25.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.878+0000 7f3aff1c4640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af8155bd0 con 0x7f3af815d820 2023-10-16T20:05:25.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.878+0000 7f3af7fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3af80d26c0 0x7f3af80b6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:25.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.878+0000 7f3af7fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3af80d26c0 0x7f3af80b6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38182/0 (socket says 172.21.15.88:38182) 2023-10-16T20:05:25.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.878+0000 7f3af7fff640 1 -- 172.21.15.88:0/1216251017 learned_addr learned my addr 172.21.15.88:0/1216251017 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:25.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.878+0000 7f3afcf60640 1 --2- 172.21.15.88:0/1216251017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3af80d22e0 0x7f3af81492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:25.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.878+0000 7f3afd761640 1 --2- 172.21.15.88:0/1216251017 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3af815d820 0x7f3af80c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:25.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.878+0000 7f3af7fff640 1 -- 172.21.15.88:0/1216251017 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3af815d820 msgr2=0x7f3af80c9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:25.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.878+0000 7f3af7fff640 1 --2- 172.21.15.88:0/1216251017 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3af815d820 0x7f3af80c9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:25.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.878+0000 7f3af7fff640 1 -- 172.21.15.88:0/1216251017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3af80d22e0 msgr2=0x7f3af81492f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:25.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.878+0000 7f3af7fff640 1 --2- 172.21.15.88:0/1216251017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3af80d22e0 0x7f3af81492f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:25.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.878+0000 7f3af7fff640 1 -- 172.21.15.88:0/1216251017 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aec0094d0 con 0x7f3af80d26c0 2023-10-16T20:05:25.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.878+0000 7f3afcf60640 1 --2- 172.21.15.88:0/1216251017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3af80d22e0 0x7f3af81492f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:05:25.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.878+0000 7f3af7fff640 1 --2- 172.21.15.88:0/1216251017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3af80d26c0 0x7f3af80b6df0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400d800 tx=0x7f3ae400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:25.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.878+0000 7f3af57fa640 1 -- 172.21.15.88:0/1216251017 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae400b6c0 con 0x7f3af80d26c0 2023-10-16T20:05:25.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.878+0000 7f3af57fa640 1 -- 172.21.15.88:0/1216251017 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ae400b820 con 0x7f3af80d26c0 2023-10-16T20:05:25.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.878+0000 7f3af57fa640 1 -- 172.21.15.88:0/1216251017 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae400bb90 con 0x7f3af80d26c0 2023-10-16T20:05:25.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.878+0000 7f3aff1c4640 1 -- 172.21.15.88:0/1216251017 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3af80cc5a0 con 0x7f3af80d26c0 2023-10-16T20:05:25.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.878+0000 7f3aff1c4640 1 -- 172.21.15.88:0/1216251017 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af8155810 con 0x7f3af80d26c0 2023-10-16T20:05:25.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.882+0000 7f3aff1c4640 1 -- 172.21.15.88:0/1216251017 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3af80cc400 con 0x7f3af80d26c0 2023-10-16T20:05:25.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.882+0000 7f3af57fa640 1 -- 172.21.15.88:0/1216251017 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3ae400bcf0 con 0x7f3af80d26c0 2023-10-16T20:05:25.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.882+0000 7f3af57fa640 1 --2- 172.21.15.88:0/1216251017 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3adc0664a0 0x7f3adc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:25.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.882+0000 7f3af57fa640 1 -- 172.21.15.88:0/1216251017 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3ae4084610 con 0x7f3af80d26c0 2023-10-16T20:05:25.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.882+0000 7f3afcf60640 1 --2- 172.21.15.88:0/1216251017 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3adc0664a0 0x7f3adc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:25.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.882+0000 7f3afcf60640 1 --2- 172.21.15.88:0/1216251017 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3adc0664a0 0x7f3adc068960 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3aec0ad000 tx=0x7f3aec0a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:25.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:25.886+0000 7f3af57fa640 1 -- 172.21.15.88:0/1216251017 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3af80cc400 con 0x7f3af80d26c0 2023-10-16T20:05:26.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.034+0000 7f3aff1c4640 1 -- 172.21.15.88:0/1216251017 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3af8149010 con 0x7f3af80d26c0 2023-10-16T20:05:26.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.034+0000 7f3af57fa640 1 -- 172.21.15.88:0/1216251017 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18274 (secure 0 0 0) 0x7f3af8149010 con 0x7f3af80d26c0 2023-10-16T20:05:26.037 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:26.037 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:05:01.835630+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T20:04:54.442487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T20:04:55.766641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T20:04:59.475830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:05:26.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.038+0000 7f3aff1c4640 1 -- 172.21.15.88:0/1216251017 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3adc0664a0 msgr2=0x7f3adc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:26.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.038+0000 7f3aff1c4640 1 --2- 172.21.15.88:0/1216251017 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3adc0664a0 0x7f3adc068960 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3aec0ad000 tx=0x7f3aec0a6000 comp rx=0 tx=0).stop 2023-10-16T20:05:26.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.038+0000 7f3aff1c4640 1 -- 172.21.15.88:0/1216251017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3af80d26c0 msgr2=0x7f3af80b6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:26.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.038+0000 7f3aff1c4640 1 --2- 172.21.15.88:0/1216251017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3af80d26c0 0x7f3af80b6df0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400d800 tx=0x7f3ae400dcc0 comp rx=0 tx=0).stop 2023-10-16T20:05:26.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.038+0000 7f3aff1c4640 1 -- 172.21.15.88:0/1216251017 shutdown_connections 2023-10-16T20:05:26.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.038+0000 7f3aff1c4640 1 --2- 172.21.15.88:0/1216251017 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3adc0664a0 0x7f3adc068960 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:26.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.038+0000 7f3aff1c4640 1 --2- 172.21.15.88:0/1216251017 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3af815d820 0x7f3af80c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:26.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.038+0000 7f3aff1c4640 1 --2- 172.21.15.88:0/1216251017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3af80d26c0 0x7f3af80b6df0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:26.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.038+0000 7f3aff1c4640 1 --2- 172.21.15.88:0/1216251017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3af80d22e0 0x7f3af81492f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:26.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.038+0000 7f3aff1c4640 1 -- 172.21.15.88:0/1216251017 >> 172.21.15.88:0/1216251017 conn(0x7f3af80715c0 msgr2=0x7f3af80719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:26.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.038+0000 7f3aff1c4640 1 -- 172.21.15.88:0/1216251017 shutdown_connections 2023-10-16T20:05:26.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.038+0000 7f3aff1c4640 1 -- 172.21.15.88:0/1216251017 wait complete. 2023-10-16T20:05:26.059 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T20:05:26.196 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.194+0000 7f4833b2a640 1 Processor -- start 2023-10-16T20:05:26.196 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.194+0000 7f4833b2a640 1 -- start start 2023-10-16T20:05:26.198 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.198+0000 7f4833b2a640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f482c160cc0 0x7f482c0ca800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:26.198 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.198+0000 7f4833b2a640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f482c1610a0 0x7f482c0ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:26.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.198+0000 7f4833b2a640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f482c0cc570 con 0x7f482c0ae710 2023-10-16T20:05:26.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.198+0000 7f4833b2a640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f482c0cc850 con 0x7f482c160cc0 2023-10-16T20:05:26.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.198+0000 7f4833b2a640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f482c0ccb30 con 0x7f482c1610a0 2023-10-16T20:05:26.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.198+0000 7f48310c5640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f482c1610a0 0x7f482c0ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:26.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.198+0000 7f48310c5640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f482c1610a0 0x7f482c0ae1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:41198/0 (socket says 172.21.15.88:41198) 2023-10-16T20:05:26.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.198+0000 7f48310c5640 1 -- 172.21.15.88:0/708170648 learned_addr learned my addr 172.21.15.88:0/708170648 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:26.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.198+0000 7f48318c6640 1 --2- 172.21.15.88:0/708170648 >> v2:172.21.15.94:3300/0 conn(0x7f482c160cc0 0x7f482c0ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:26.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.198+0000 7f48308c4640 1 -- 172.21.15.88:0/708170648 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4200256663 0 0) 0x7f482c0cc570 con 0x7f482c0ae710 2023-10-16T20:05:26.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.198+0000 7f48308c4640 1 -- 172.21.15.88:0/708170648 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4818003530 con 0x7f482c0ae710 2023-10-16T20:05:26.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.198+0000 7f48310c5640 1 -- 172.21.15.88:0/708170648 >> v2:172.21.15.94:3300/0 conn(0x7f482c160cc0 msgr2=0x7f482c0ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:26.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.198+0000 7f48310c5640 1 --2- 172.21.15.88:0/708170648 >> v2:172.21.15.94:3300/0 conn(0x7f482c160cc0 0x7f482c0ca800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:26.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.198+0000 7f48310c5640 1 -- 172.21.15.88:0/708170648 >> v1:172.21.15.94:6789/0 conn(0x7f482c0ae710 legacy=0x7f482c0cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:26.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.198+0000 7f48310c5640 1 -- 172.21.15.88:0/708170648 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482c0cce10 con 0x7f482c1610a0 2023-10-16T20:05:26.202 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.202+0000 7f48310c5640 1 --2- 172.21.15.88:0/708170648 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f482c1610a0 0x7f482c0ae1d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f481c009870 tx=0x7f481c01c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3e300c4a9058c02 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:26.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.202+0000 7f48308c4640 1 -- 172.21.15.88:0/708170648 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c0a95f0 con 0x7f482c1610a0 2023-10-16T20:05:26.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.202+0000 7f48308c4640 1 -- 172.21.15.88:0/708170648 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f481c0a9750 con 0x7f482c1610a0 2023-10-16T20:05:26.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.202+0000 7f48308c4640 1 -- 172.21.15.88:0/708170648 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c0a9ae0 con 0x7f482c1610a0 2023-10-16T20:05:26.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.202+0000 7f4833b2a640 1 -- 172.21.15.88:0/708170648 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f482c1610a0 msgr2=0x7f482c0ae1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:26.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.202+0000 7f4833b2a640 1 --2- 172.21.15.88:0/708170648 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f482c1610a0 0x7f482c0ae1d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f481c009870 tx=0x7f481c01c510 comp rx=0 tx=0).stop 2023-10-16T20:05:26.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.202+0000 7f4833b2a640 1 -- 172.21.15.88:0/708170648 shutdown_connections 2023-10-16T20:05:26.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.202+0000 7f4833b2a640 1 --2- 172.21.15.88:0/708170648 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f482c1610a0 0x7f482c0ae1d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:26.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.202+0000 7f4833b2a640 1 --2- 172.21.15.88:0/708170648 >> v2:172.21.15.94:3300/0 conn(0x7f482c160cc0 0x7f482c0ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:26.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.202+0000 7f4833b2a640 1 -- 172.21.15.88:0/708170648 >> 172.21.15.88:0/708170648 conn(0x7f482c0ce5e0 msgr2=0x7f482c0ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:26.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.202+0000 7f4833b2a640 1 -- 172.21.15.88:0/708170648 shutdown_connections 2023-10-16T20:05:26.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.202+0000 7f4833b2a640 1 -- 172.21.15.88:0/708170648 wait complete. 2023-10-16T20:05:26.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.202+0000 7f4833b2a640 1 Processor -- start 2023-10-16T20:05:26.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.202+0000 7f4833b2a640 1 -- start start 2023-10-16T20:05:26.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.202+0000 7f4833b2a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f482c0ae710 0x7f482c156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:26.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.206+0000 7f4833b2a640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f482c160cc0 0x7f482c155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:26.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.206+0000 7f4833b2a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f482c1610a0 0x7f482c162160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:26.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.206+0000 7f4833b2a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c1505d0 con 0x7f482c1610a0 2023-10-16T20:05:26.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.206+0000 7f4833b2a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c14f9c0 con 0x7f482c0ae710 2023-10-16T20:05:26.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.206+0000 7f4833b2a640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f482c14fca0 con 0x7f482c160cc0 2023-10-16T20:05:26.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.206+0000 7f48320c7640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f482c1610a0 0x7f482c162160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:26.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.206+0000 7f48310c5640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f482c160cc0 0x7f482c155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:26.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.206+0000 7f48320c7640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f482c1610a0 0x7f482c162160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38196/0 (socket says 172.21.15.88:38196) 2023-10-16T20:05:26.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.206+0000 7f48320c7640 1 -- 172.21.15.88:0/3926107504 learned_addr learned my addr 172.21.15.88:0/3926107504 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:26.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.206+0000 7f48318c6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f482c0ae710 0x7f482c156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:26.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.206+0000 7f48320c7640 1 -- 172.21.15.88:0/3926107504 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f482c160cc0 msgr2=0x7f482c155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:26.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.206+0000 7f48320c7640 1 --2- 172.21.15.88:0/3926107504 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f482c160cc0 0x7f482c155d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:26.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.206+0000 7f48320c7640 1 -- 172.21.15.88:0/3926107504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f482c0ae710 msgr2=0x7f482c156bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:26.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.206+0000 7f48320c7640 1 --2- 172.21.15.88:0/3926107504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f482c0ae710 0x7f482c156bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:26.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.206+0000 7f48320c7640 1 -- 172.21.15.88:0/3926107504 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481c0094d0 con 0x7f482c1610a0 2023-10-16T20:05:26.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.206+0000 7f48320c7640 1 --2- 172.21.15.88:0/3926107504 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f482c1610a0 0x7f482c162160 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f482400c730 tx=0x7f482400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:26.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.206+0000 7f48227fc640 1 -- 172.21.15.88:0/3926107504 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4824003b80 con 0x7f482c1610a0 2023-10-16T20:05:26.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.206+0000 7f48227fc640 1 -- 172.21.15.88:0/3926107504 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4824003ce0 con 0x7f482c1610a0 2023-10-16T20:05:26.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.206+0000 7f4833b2a640 1 -- 172.21.15.88:0/3926107504 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f482c14ff80 con 0x7f482c1610a0 2023-10-16T20:05:26.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.206+0000 7f48227fc640 1 -- 172.21.15.88:0/3926107504 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482400aa90 con 0x7f482c1610a0 2023-10-16T20:05:26.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.206+0000 7f4833b2a640 1 -- 172.21.15.88:0/3926107504 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f482c150be0 con 0x7f482c1610a0 2023-10-16T20:05:26.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.206+0000 7f4833b2a640 1 -- 172.21.15.88:0/3926107504 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f482c150780 con 0x7f482c1610a0 2023-10-16T20:05:26.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.206+0000 7f48227fc640 1 -- 172.21.15.88:0/3926107504 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f482400abf0 con 0x7f482c1610a0 2023-10-16T20:05:26.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.210+0000 7f48227fc640 1 --2- 172.21.15.88:0/3926107504 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f48100664d0 0x7f4810068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:26.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.210+0000 7f48227fc640 1 -- 172.21.15.88:0/3926107504 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4824083e90 con 0x7f482c1610a0 2023-10-16T20:05:26.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.210+0000 7f48318c6640 1 --2- 172.21.15.88:0/3926107504 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f48100664d0 0x7f4810068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:26.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.210+0000 7f48318c6640 1 --2- 172.21.15.88:0/3926107504 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f48100664d0 0x7f4810068990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4828009b60 tx=0x7f4828009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:26.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.214+0000 7f48227fc640 1 -- 172.21.15.88:0/3926107504 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f482c150780 con 0x7f482c1610a0 2023-10-16T20:05:26.221 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T20:05:26.224 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T20:05:26.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.394+0000 7f4833b2a640 1 -- 172.21.15.88:0/3926107504 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f482c150780 con 0x7f482c1610a0 2023-10-16T20:05:26.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.398+0000 7f48227fc640 1 -- 172.21.15.88:0/3926107504 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4906 (secure 0 0 0) 0x7f482c150780 con 0x7f482c1610a0 2023-10-16T20:05:26.398 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:26.399 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3803627113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3803627113},{"type":"v1","addr":"172.21.15.94:6834","nonce":3803627113}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3758093239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":3758093239},{"type":"v1","addr":"172.21.15.88:6837","nonce":3758093239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/3836772385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3836772385},{"type":"v1","addr":"172.21.15.88:6836","nonce":3836772385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T20:04:56.759719+0000","modified":"2023-10-16T20:05:14.569618+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6835/555575750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":555575750},{"type":"v1","addr":"172.21.15.94:6835","nonce":555575750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T20:05:26.400 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 9 2023-10-16T20:05:26.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.398+0000 7f4833b2a640 1 -- 172.21.15.88:0/3926107504 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f48100664d0 msgr2=0x7f4810068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:26.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.398+0000 7f4833b2a640 1 --2- 172.21.15.88:0/3926107504 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f48100664d0 0x7f4810068990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4828009b60 tx=0x7f4828009190 comp rx=0 tx=0).stop 2023-10-16T20:05:26.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.398+0000 7f4833b2a640 1 -- 172.21.15.88:0/3926107504 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f482c1610a0 msgr2=0x7f482c162160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:26.401 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.398+0000 7f4833b2a640 1 --2- 172.21.15.88:0/3926107504 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f482c1610a0 0x7f482c162160 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f482400c730 tx=0x7f482400cbf0 comp rx=0 tx=0).stop 2023-10-16T20:05:26.401 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.398+0000 7f4833b2a640 1 -- 172.21.15.88:0/3926107504 shutdown_connections 2023-10-16T20:05:26.401 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.398+0000 7f4833b2a640 1 --2- 172.21.15.88:0/3926107504 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f48100664d0 0x7f4810068990 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:26.401 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.398+0000 7f4833b2a640 1 --2- 172.21.15.88:0/3926107504 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f482c160cc0 0x7f482c155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:26.401 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.398+0000 7f4833b2a640 1 --2- 172.21.15.88:0/3926107504 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f482c1610a0 0x7f482c162160 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:26.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.398+0000 7f4833b2a640 1 --2- 172.21.15.88:0/3926107504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f482c0ae710 0x7f482c156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:26.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.398+0000 7f4833b2a640 1 -- 172.21.15.88:0/3926107504 >> 172.21.15.88:0/3926107504 conn(0x7f482c0ce5e0 msgr2=0x7f482c154980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:26.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.398+0000 7f4833b2a640 1 -- 172.21.15.88:0/3926107504 shutdown_connections 2023-10-16T20:05:26.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.398+0000 7f4833b2a640 1 -- 172.21.15.88:0/3926107504 wait complete. 2023-10-16T20:05:26.821 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T20:04:56.759719+0000', 'modified': '2023-10-16T20:05:14.569618+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4272}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4272': {'gid': 4272, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.94:6835/555575750', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6833', 'nonce': 555575750}, {'type': 'v1', 'addr': '172.21.15.94:6835', 'nonce': 555575750}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T20:05:26.821 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T20:05:26.822 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-16T20:05:26.962 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.962+0000 7fef90b35640 1 Processor -- start 2023-10-16T20:05:26.962 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.962+0000 7fef90b35640 1 -- start start 2023-10-16T20:05:26.963 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.962+0000 7fef90b35640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef8c0d1470 0x7fef8c0cf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:26.964 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.962+0000 7fef90b35640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fef8c0cd090 0x7fef8c0cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:26.964 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.962+0000 7fef90b35640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef8c15d010 con 0x7fef8c15cc30 2023-10-16T20:05:26.964 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.962+0000 7fef90b35640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef8c0cda60 con 0x7fef8c0cd090 2023-10-16T20:05:26.964 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.962+0000 7fef90b35640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c06ba00 con 0x7fef8c0d1470 2023-10-16T20:05:26.964 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.962+0000 7fef83fff640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef8c0d1470 0x7fef8c0cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:26.965 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.962+0000 7fef83fff640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef8c0d1470 0x7fef8c0cf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:41210/0 (socket says 172.21.15.88:41210) 2023-10-16T20:05:26.965 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.962+0000 7fef8a59c640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fef8c15cc30 0x7fef8c06be90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:40288/0 (socket says 172.21.15.88:40288) 2023-10-16T20:05:26.965 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.962+0000 7fef8a59c640 1 -- 172.21.15.88:0/4102361389 learned_addr learned my addr 172.21.15.88:0/4102361389 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:26.965 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.962+0000 7fef8ad9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fef8c0cd090 0x7fef8c0cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:26.966 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.962+0000 7fef8ad9d640 1 -- 172.21.15.88:0/4102361389 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef8c0d1470 msgr2=0x7fef8c0cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:26.966 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.962+0000 7fef8ad9d640 1 --2- 172.21.15.88:0/4102361389 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef8c0d1470 0x7fef8c0cf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:26.966 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.962+0000 7fef8ad9d640 1 -- 172.21.15.88:0/4102361389 >> v1:172.21.15.94:6789/0 conn(0x7fef8c15cc30 legacy=0x7fef8c06be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:26.967 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.962+0000 7fef8ad9d640 1 -- 172.21.15.88:0/4102361389 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8c0cfe40 con 0x7fef8c0cd090 2023-10-16T20:05:26.967 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.962+0000 7fef83fff640 1 --2- 172.21.15.88:0/4102361389 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef8c0d1470 0x7fef8c0cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:05:26.967 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.962+0000 7fef89d9b640 1 -- 172.21.15.88:0/4102361389 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1403714013 0 0) 0x7fef8c15d010 con 0x7fef8c15cc30 2023-10-16T20:05:26.968 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.966+0000 7fef8ad9d640 1 --2- 172.21.15.88:0/4102361389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef8c0cd090 0x7fef8c0cd520 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fef84009d90 tx=0x7fef8401c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=167b6533a474f419 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:26.968 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.966+0000 7fef89d9b640 1 -- 172.21.15.88:0/4102361389 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef840a5950 con 0x7fef8c0cd090 2023-10-16T20:05:26.969 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.966+0000 7fef89d9b640 1 -- 172.21.15.88:0/4102361389 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef840a5ab0 con 0x7fef8c0cd090 2023-10-16T20:05:26.969 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.966+0000 7fef89d9b640 1 -- 172.21.15.88:0/4102361389 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef840a5e40 con 0x7fef8c0cd090 2023-10-16T20:05:26.969 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.966+0000 7fef90b35640 1 -- 172.21.15.88:0/4102361389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef8c0cd090 msgr2=0x7fef8c0cd520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:26.969 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.966+0000 7fef90b35640 1 --2- 172.21.15.88:0/4102361389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef8c0cd090 0x7fef8c0cd520 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fef84009d90 tx=0x7fef8401c890 comp rx=0 tx=0).stop 2023-10-16T20:05:26.969 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.966+0000 7fef90b35640 1 -- 172.21.15.88:0/4102361389 shutdown_connections 2023-10-16T20:05:26.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.966+0000 7fef90b35640 1 --2- 172.21.15.88:0/4102361389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef8c0cd090 0x7fef8c0cd520 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:26.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.966+0000 7fef90b35640 1 --2- 172.21.15.88:0/4102361389 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef8c0d1470 0x7fef8c0cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:26.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.966+0000 7fef90b35640 1 -- 172.21.15.88:0/4102361389 >> 172.21.15.88:0/4102361389 conn(0x7fef8c099b80 msgr2=0x7fef8c0d0810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:26.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.966+0000 7fef90b35640 1 -- 172.21.15.88:0/4102361389 shutdown_connections 2023-10-16T20:05:26.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.966+0000 7fef90b35640 1 -- 172.21.15.88:0/4102361389 wait complete. 2023-10-16T20:05:26.972 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.970+0000 7fef90b35640 1 Processor -- start 2023-10-16T20:05:26.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.970+0000 7fef90b35640 1 -- start start 2023-10-16T20:05:26.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.970+0000 7fef90b35640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef8c0cd090 0x7fef8c0b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:26.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.970+0000 7fef90b35640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef8c0d1470 0x7fef8c0b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:26.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.970+0000 7fef90b35640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef8c15cc30 0x7fef8c0c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:26.974 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.970+0000 7fef90b35640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c0add70 con 0x7fef8c15cc30 2023-10-16T20:05:26.974 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.970+0000 7fef90b35640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c0ad480 con 0x7fef8c0d1470 2023-10-16T20:05:26.974 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.970+0000 7fef90b35640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c0ad720 con 0x7fef8c0cd090 2023-10-16T20:05:26.974 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.970+0000 7fef8ad9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef8c15cc30 0x7fef8c0c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:26.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.970+0000 7fef8ad9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef8c15cc30 0x7fef8c0c99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38200/0 (socket says 172.21.15.88:38200) 2023-10-16T20:05:26.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.970+0000 7fef8ad9d640 1 -- 172.21.15.88:0/1483207785 learned_addr learned my addr 172.21.15.88:0/1483207785 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:26.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.970+0000 7fef83fff640 1 --2- 172.21.15.88:0/1483207785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef8c0d1470 0x7fef8c0b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:26.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.970+0000 7fef8ad9d640 1 -- 172.21.15.88:0/1483207785 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef8c0cd090 msgr2=0x7fef8c0b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:26.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.970+0000 7fef8ad9d640 1 --2- 172.21.15.88:0/1483207785 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef8c0cd090 0x7fef8c0b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:26.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.970+0000 7fef8ad9d640 1 -- 172.21.15.88:0/1483207785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef8c0d1470 msgr2=0x7fef8c0b5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:26.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.970+0000 7fef8ad9d640 1 --2- 172.21.15.88:0/1483207785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef8c0d1470 0x7fef8c0b5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:26.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.970+0000 7fef8ad9d640 1 -- 172.21.15.88:0/1483207785 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef74009420 con 0x7fef8c15cc30 2023-10-16T20:05:26.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.970+0000 7fef83fff640 1 --2- 172.21.15.88:0/1483207785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef8c0d1470 0x7fef8c0b5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:26.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.970+0000 7fef8ad9d640 1 --2- 172.21.15.88:0/1483207785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef8c15cc30 0x7fef8c0c99a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fef840b0040 tx=0x7fef84019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:26.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.970+0000 7fef82ffd640 1 -- 172.21.15.88:0/1483207785 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef840add90 con 0x7fef8c15cc30 2023-10-16T20:05:26.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.970+0000 7fef82ffd640 1 -- 172.21.15.88:0/1483207785 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef84009450 con 0x7fef8c15cc30 2023-10-16T20:05:26.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.970+0000 7fef90b35640 1 -- 172.21.15.88:0/1483207785 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef84009a70 con 0x7fef8c15cc30 2023-10-16T20:05:26.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.970+0000 7fef82ffd640 1 -- 172.21.15.88:0/1483207785 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef840b9690 con 0x7fef8c15cc30 2023-10-16T20:05:26.978 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.970+0000 7fef90b35640 1 -- 172.21.15.88:0/1483207785 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef8c0ca740 con 0x7fef8c15cc30 2023-10-16T20:05:26.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.974+0000 7fef82ffd640 1 -- 172.21.15.88:0/1483207785 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fef840b97f0 con 0x7fef8c15cc30 2023-10-16T20:05:26.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.974+0000 7fef82ffd640 1 --2- 172.21.15.88:0/1483207785 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fef6c0664d0 0x7fef6c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:26.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.974+0000 7fef82ffd640 1 -- 172.21.15.88:0/1483207785 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fef8411fa20 con 0x7fef8c15cc30 2023-10-16T20:05:26.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.974+0000 7fef8a59c640 1 --2- 172.21.15.88:0/1483207785 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fef6c0664d0 0x7fef6c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:26.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.974+0000 7fef93eb5000 1 Processor -- start 2023-10-16T20:05:26.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.974+0000 7fef8a59c640 1 --2- 172.21.15.88:0/1483207785 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fef6c0664d0 0x7fef6c068990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0015f0 tx=0x7fef7c0079c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:26.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.974+0000 7fef93eb5000 1 -- start start 2023-10-16T20:05:26.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.974+0000 7fef93eb5000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972edcc8a0 0x55972edccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:26.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.974+0000 7fef93eb5000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55972edcd1c0 0x55972ed82710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:26.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.974+0000 7fef93eb5000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55972ed82c50 0x55972ed83030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:26.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.974+0000 7fef93eb5000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x55972eb77cc0 con 0x55972ed82c50 2023-10-16T20:05:26.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.974+0000 7fef93eb5000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55972ebda920 con 0x55972edcc8a0 2023-10-16T20:05:26.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.974+0000 7fef93eb5000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x55972ebd7e50 con 0x55972edcd1c0 2023-10-16T20:05:26.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.978+0000 7fef8ad9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55972ed82c50 0x55972ed83030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:26.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.978+0000 7fef8ad9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55972ed82c50 0x55972ed83030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38208/0 (socket says 172.21.15.88:38208) 2023-10-16T20:05:26.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.978+0000 7fef8ad9d640 1 -- 172.21.15.88:0/757643869 learned_addr learned my addr 172.21.15.88:0/757643869 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:26.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.978+0000 7fef8a59c640 1 --2- 172.21.15.88:0/757643869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972edcc8a0 0x55972edccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:26.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.978+0000 7fef83fff640 1 --2- 172.21.15.88:0/757643869 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55972edcd1c0 0x55972ed82710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:26.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.978+0000 7fef8ad9d640 1 -- 172.21.15.88:0/757643869 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55972edcd1c0 msgr2=0x55972ed82710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:26.988 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.978+0000 7fef8ad9d640 1 --2- 172.21.15.88:0/757643869 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55972edcd1c0 0x55972ed82710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:26.988 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.978+0000 7fef8ad9d640 1 -- 172.21.15.88:0/757643869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972edcc8a0 msgr2=0x55972edccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:26.988 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.978+0000 7fef8ad9d640 1 --2- 172.21.15.88:0/757643869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972edcc8a0 0x55972edccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:26.988 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.978+0000 7fef8ad9d640 1 -- 172.21.15.88:0/757643869 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55972e8cc890 con 0x55972ed82c50 2023-10-16T20:05:26.988 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.978+0000 7fef8ad9d640 1 --2- 172.21.15.88:0/757643869 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55972ed82c50 0x55972ed83030 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fef840f0440 tx=0x7fef840f0160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c722ba00553caab server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:26.989 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.978+0000 7fef6bfff640 1 -- 172.21.15.88:0/757643869 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef840f0740 con 0x55972ed82c50 2023-10-16T20:05:26.989 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.978+0000 7fef6bfff640 1 -- 172.21.15.88:0/757643869 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef840f4890 con 0x55972ed82c50 2023-10-16T20:05:26.989 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.978+0000 7fef6bfff640 1 -- 172.21.15.88:0/757643869 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef840f8ad0 con 0x55972ed82c50 2023-10-16T20:05:26.989 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.978+0000 7fef93eb5000 1 -- 172.21.15.88:0/757643869 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55972ed82c50 msgr2=0x55972ed83030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:26.989 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.978+0000 7fef93eb5000 1 --2- 172.21.15.88:0/757643869 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55972ed82c50 0x55972ed83030 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fef840f0440 tx=0x7fef840f0160 comp rx=0 tx=0).stop 2023-10-16T20:05:26.990 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.978+0000 7fef93eb5000 1 -- 172.21.15.88:0/757643869 shutdown_connections 2023-10-16T20:05:26.990 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.978+0000 7fef93eb5000 1 --2- 172.21.15.88:0/757643869 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55972edcd1c0 0x55972ed82710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:26.990 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.978+0000 7fef93eb5000 1 --2- 172.21.15.88:0/757643869 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55972ed82c50 0x55972ed83030 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:26.990 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.978+0000 7fef93eb5000 1 --2- 172.21.15.88:0/757643869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972edcc8a0 0x55972edccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:26.991 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.978+0000 7fef93eb5000 1 -- 172.21.15.88:0/757643869 >> 172.21.15.88:0/757643869 conn(0x55972ed79fc0 msgr2=0x55972ed80ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:26.991 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.978+0000 7fef93eb5000 1 -- 172.21.15.88:0/757643869 shutdown_connections 2023-10-16T20:05:26.991 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.978+0000 7fef93eb5000 1 -- 172.21.15.88:0/757643869 wait complete. 2023-10-16T20:05:26.991 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.978+0000 7fef93eb5000 1 -- start start 2023-10-16T20:05:26.991 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.978+0000 7fef93eb5000 1 Processor -- start 2023-10-16T20:05:26.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef93eb5000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972ed82c50 0x55972ed8d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:26.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef93eb5000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55972edcc8a0 0x55972ed8dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:26.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef93eb5000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55972edcd1c0 0x55972ed8e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:26.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef93eb5000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x55972ebd3740 con 0x55972edcc8a0 2023-10-16T20:05:26.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef93eb5000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55972eb92d10 con 0x55972ed82c50 2023-10-16T20:05:26.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef93eb5000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x55972eb929b0 con 0x55972edcd1c0 2023-10-16T20:05:26.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef93eb5000 10 client.0 _ll_register_callbacks cb 0x55972ed6e390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T20:05:26.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef83fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55972edcc8a0 0x55972ed8dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:26.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef83fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55972edcc8a0 0x55972ed8dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38220/0 (socket says 172.21.15.88:38220) 2023-10-16T20:05:26.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef83fff640 1 -- 172.21.15.88:0/2552892761 learned_addr learned my addr 172.21.15.88:0/2552892761 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:26.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef8a59c640 1 --2- 172.21.15.88:0/2552892761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972ed82c50 0x55972ed8d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:26.994 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef8ad9d640 1 --2- 172.21.15.88:0/2552892761 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55972edcd1c0 0x55972ed8e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:26.994 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef83fff640 1 -- 172.21.15.88:0/2552892761 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55972edcd1c0 msgr2=0x55972ed8e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:26.994 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef83fff640 1 --2- 172.21.15.88:0/2552892761 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55972edcd1c0 0x55972ed8e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:26.994 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef83fff640 1 -- 172.21.15.88:0/2552892761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972ed82c50 msgr2=0x55972ed8d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:26.995 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef83fff640 1 --2- 172.21.15.88:0/2552892761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972ed82c50 0x55972ed8d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:26.995 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef83fff640 1 -- 172.21.15.88:0/2552892761 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef8c0ca740 con 0x55972edcc8a0 2023-10-16T20:05:26.995 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef83fff640 1 --2- 172.21.15.88:0/2552892761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55972edcc8a0 0x55972ed8dda0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fef74009f10 tx=0x7fef740136b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b055a9b8ce9f7c8b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:26.995 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef697fa640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2023-10-16T20:05:26.995 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef697fa640 1 -- 172.21.15.88:0/2552892761 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef74013ee0 con 0x55972edcc8a0 2023-10-16T20:05:26.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef93eb5000 1 -- 172.21.15.88:0/2552892761 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55972eb87370 con 0x55972edcc8a0 2023-10-16T20:05:26.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef697fa640 1 -- 172.21.15.88:0/2552892761 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fef74004220 con 0x55972edcc8a0 2023-10-16T20:05:26.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef697fa640 1 -- 172.21.15.88:0/2552892761 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef740043e0 con 0x55972edcc8a0 2023-10-16T20:05:26.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef93eb5000 10 client.4827 fetch_fsmap learned FSMap version 9 2023-10-16T20:05:26.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef93eb5000 1 -- 172.21.15.88:0/2552892761 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55972eb95460 con 0x55972edcc8a0 2023-10-16T20:05:26.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef697fa640 1 -- 172.21.15.88:0/2552892761 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55972eb87370 con 0x55972edcc8a0 2023-10-16T20:05:26.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef697fa640 1 -- 172.21.15.88:0/2552892761 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fef74016670 con 0x55972edcc8a0 2023-10-16T20:05:26.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef93eb5000 10 client.4827 fetch_fsmap finished waiting for FSMap version 9 2023-10-16T20:05:26.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef93eb5000 10 client.4827 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-16T20:05:26.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef93eb5000 20 client.4827 populate_metadata read hostname 'smithi088' 2023-10-16T20:05:26.998 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef93eb5000 1 --2- 172.21.15.88:0/2552892761 >> [v2:172.21.15.94:6833/555575750,v1:172.21.15.94:6835/555575750] conn(0x55972eddb930 0x55972edddd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:26.998 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef93eb5000 4 client.4827 mds_command: new command op to 4272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T20:05:26.998 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef93eb5000 1 -- 172.21.15.88:0/2552892761 --> [v2:172.21.15.94:6833/555575750,v1:172.21.15.94:6835/555575750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55972eb6d2d0 con 0x55972eddb930 2023-10-16T20:05:26.998 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef8a59c640 1 --2- 172.21.15.88:0/2552892761 >> [v2:172.21.15.94:6833/555575750,v1:172.21.15.94:6835/555575750] conn(0x55972eddb930 0x55972edddd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:26.999 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef697fa640 10 client.4827 ms_handle_connect on v2:172.21.15.94:6833/555575750 2023-10-16T20:05:26.999 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef8a59c640 1 --2- 172.21.15.88:0/2552892761 >> [v2:172.21.15.94:6833/555575750,v1:172.21.15.94:6835/555575750] conn(0x55972eddb930 0x55972edddd20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3963aca398284fed server_cookie=c94b998e5f71ada9 in_seq=0 out_seq=0 2023-10-16T20:05:27.002 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef697fa640 1 -- 172.21.15.88:0/2552892761 <== mds.0 v2:172.21.15.94:6833/555575750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55972eb6d2d0 con 0x55972eddb930 2023-10-16T20:05:27.002 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef697fa640 10 client.4827 handle_command_reply: tid=0 2023-10-16T20:05:27.002 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef93eb5000 1 client.4827 shutdown 2023-10-16T20:05:27.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef93eb5000 1 -- 172.21.15.88:0/2552892761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55972edcc8a0 msgr2=0x55972ed8dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:27.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.982+0000 7fef93eb5000 1 --2- 172.21.15.88:0/2552892761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55972edcc8a0 0x55972ed8dda0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fef74009f10 tx=0x7fef740136b0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.986+0000 7fef93eb5000 1 -- 172.21.15.88:0/2552892761 shutdown_connections 2023-10-16T20:05:27.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.986+0000 7fef93eb5000 1 --2- 172.21.15.88:0/2552892761 >> [v2:172.21.15.94:6833/555575750,v1:172.21.15.94:6835/555575750] conn(0x55972eddb930 0x55972edddd20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.986+0000 7fef93eb5000 1 --2- 172.21.15.88:0/2552892761 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55972edcd1c0 0x55972ed8e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.986+0000 7fef93eb5000 1 --2- 172.21.15.88:0/2552892761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55972edcc8a0 0x55972ed8dda0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.986+0000 7fef697fa640 0 client.4827 ms_handle_reset on v2:172.21.15.94:6833/555575750 2023-10-16T20:05:27.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.986+0000 7fef93eb5000 1 --2- 172.21.15.88:0/2552892761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972ed82c50 0x55972ed8d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.986+0000 7fef93eb5000 1 -- 172.21.15.88:0/2552892761 >> 172.21.15.88:0/2552892761 conn(0x55972ed79fc0 msgr2=0x55972ed816f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:27.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.986+0000 7fef93eb5000 1 -- 172.21.15.88:0/2552892761 shutdown_connections 2023-10-16T20:05:27.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.986+0000 7fef93eb5000 1 -- 172.21.15.88:0/2552892761 wait complete. 2023-10-16T20:05:27.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.986+0000 7fef93eb5000 20 client.4827 trim_cache size 0 max 16384 2023-10-16T20:05:27.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.998+0000 7fef93eb5000 1 Processor -- start 2023-10-16T20:05:27.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.998+0000 7fef93eb5000 1 -- start start 2023-10-16T20:05:27.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.998+0000 7fef93eb5000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55972ed82c50 0x55972ee26f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:27.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.998+0000 7fef93eb5000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55972edcc8a0 0x55972ee27460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:27.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:26.998+0000 7fef93eb5000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972edcd1c0 0x55972ee279a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:27.010 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef93eb5000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x55972e88d940 con 0x55972ed82c50 2023-10-16T20:05:27.010 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef93eb5000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55972ebd7740 con 0x55972edcd1c0 2023-10-16T20:05:27.010 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef93eb5000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x55972ed85a60 con 0x55972edcc8a0 2023-10-16T20:05:27.010 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef8a59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55972ed82c50 0x55972ee26f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:27.010 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef8a59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55972ed82c50 0x55972ee26f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38222/0 (socket says 172.21.15.88:38222) 2023-10-16T20:05:27.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef8a59c640 1 -- 172.21.15.88:0/4065432253 learned_addr learned my addr 172.21.15.88:0/4065432253 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:27.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef8ad9d640 1 --2- 172.21.15.88:0/4065432253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972edcd1c0 0x55972ee279a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:27.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef83fff640 1 --2- 172.21.15.88:0/4065432253 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55972edcc8a0 0x55972ee27460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:27.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef8a59c640 1 -- 172.21.15.88:0/4065432253 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55972edcc8a0 msgr2=0x55972ee27460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:27.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef8a59c640 1 --2- 172.21.15.88:0/4065432253 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55972edcc8a0 0x55972ee27460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef8a59c640 1 -- 172.21.15.88:0/4065432253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972edcd1c0 msgr2=0x55972ee279a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:27.012 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef8a59c640 1 --2- 172.21.15.88:0/4065432253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972edcd1c0 0x55972ee279a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.012 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef8a59c640 1 -- 172.21.15.88:0/4065432253 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55972eb7b760 con 0x55972ed82c50 2023-10-16T20:05:27.012 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef8ad9d640 1 --2- 172.21.15.88:0/4065432253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972edcd1c0 0x55972ee279a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:27.012 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef83fff640 1 --2- 172.21.15.88:0/4065432253 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55972edcc8a0 0x55972ee27460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:27.012 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef8a59c640 1 --2- 172.21.15.88:0/4065432253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55972ed82c50 0x55972ee26f20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fef7c007040 tx=0x7fef7c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb133c7d0b86d3ed server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:27.013 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef80ff9640 1 -- 172.21.15.88:0/4065432253 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c01d040 con 0x55972ed82c50 2023-10-16T20:05:27.013 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef80ff9640 1 -- 172.21.15.88:0/4065432253 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef7c0259f0 con 0x55972ed82c50 2023-10-16T20:05:27.013 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef80ff9640 1 -- 172.21.15.88:0/4065432253 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c025d80 con 0x55972ed82c50 2023-10-16T20:05:27.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef93eb5000 1 -- 172.21.15.88:0/4065432253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55972ed82c50 msgr2=0x55972ee26f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:27.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef93eb5000 1 --2- 172.21.15.88:0/4065432253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55972ed82c50 0x55972ee26f20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fef7c007040 tx=0x7fef7c010000 comp rx=0 tx=0).stop 2023-10-16T20:05:27.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef93eb5000 1 -- 172.21.15.88:0/4065432253 shutdown_connections 2023-10-16T20:05:27.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef93eb5000 1 --2- 172.21.15.88:0/4065432253 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55972edcc8a0 0x55972ee27460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef93eb5000 1 --2- 172.21.15.88:0/4065432253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972edcd1c0 0x55972ee279a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef93eb5000 1 --2- 172.21.15.88:0/4065432253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55972ed82c50 0x55972ee26f20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef93eb5000 1 -- 172.21.15.88:0/4065432253 >> 172.21.15.88:0/4065432253 conn(0x55972ed79fc0 msgr2=0x55972eddec30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:27.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef93eb5000 1 -- 172.21.15.88:0/4065432253 shutdown_connections 2023-10-16T20:05:27.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef93eb5000 1 -- 172.21.15.88:0/4065432253 wait complete. 2023-10-16T20:05:27.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef93eb5000 1 -- start start 2023-10-16T20:05:27.019 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef93eb5000 1 Processor -- start 2023-10-16T20:05:27.019 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef93eb5000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55972ed82c50 0x55972edcb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:27.019 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef93eb5000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972edcc8a0 0x55972ede70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:27.019 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef93eb5000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55972edcd1c0 0x55972ede7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:27.019 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef93eb5000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x55972e8cd1b0 con 0x55972edcd1c0 2023-10-16T20:05:27.019 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef93eb5000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55972ebd88b0 con 0x55972edcc8a0 2023-10-16T20:05:27.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef93eb5000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x55972eba9a10 con 0x55972ed82c50 2023-10-16T20:05:27.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef93eb5000 10 client.0 _ll_register_callbacks cb 0x7fef840a5960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T20:05:27.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef8ad9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55972edcd1c0 0x55972ede7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:27.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef8ad9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55972edcd1c0 0x55972ede7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38234/0 (socket says 172.21.15.88:38234) 2023-10-16T20:05:27.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.002+0000 7fef8ad9d640 1 -- 172.21.15.88:0/2840865266 learned_addr learned my addr 172.21.15.88:0/2840865266 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:27.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef83fff640 1 --2- 172.21.15.88:0/2840865266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972edcc8a0 0x55972ede70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:27.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef8a59c640 1 --2- 172.21.15.88:0/2840865266 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55972ed82c50 0x55972edcb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:27.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef8ad9d640 1 -- 172.21.15.88:0/2840865266 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55972ed82c50 msgr2=0x55972edcb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:27.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef8ad9d640 1 --2- 172.21.15.88:0/2840865266 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55972ed82c50 0x55972edcb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef8ad9d640 1 -- 172.21.15.88:0/2840865266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972edcc8a0 msgr2=0x55972ede70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:27.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef8ad9d640 1 --2- 172.21.15.88:0/2840865266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972edcc8a0 0x55972ede70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef8ad9d640 1 -- 172.21.15.88:0/2840865266 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7c005850 con 0x55972edcd1c0 2023-10-16T20:05:27.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef8a59c640 1 --2- 172.21.15.88:0/2840865266 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55972ed82c50 0x55972edcb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:27.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef83fff640 1 --2- 172.21.15.88:0/2840865266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972edcc8a0 0x55972ede70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:27.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef8ad9d640 1 --2- 172.21.15.88:0/2840865266 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55972edcd1c0 0x55972ede7620 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fef840f0440 tx=0x7fef840f01d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b1bd25ef2c7c75 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:27.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef697fa640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2023-10-16T20:05:27.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef697fa640 1 -- 172.21.15.88:0/2840865266 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef840f0ba0 con 0x55972edcd1c0 2023-10-16T20:05:27.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef697fa640 1 -- 172.21.15.88:0/2840865266 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fef840f0740 con 0x55972edcd1c0 2023-10-16T20:05:27.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef93eb5000 1 -- 172.21.15.88:0/2840865266 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55972ed86520 con 0x55972edcd1c0 2023-10-16T20:05:27.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef697fa640 1 -- 172.21.15.88:0/2840865266 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef840f8ad0 con 0x55972edcd1c0 2023-10-16T20:05:27.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef93eb5000 10 client.4833 fetch_fsmap learned FSMap version 9 2023-10-16T20:05:27.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef93eb5000 1 -- 172.21.15.88:0/2840865266 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55972e8cccc0 con 0x55972edcd1c0 2023-10-16T20:05:27.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef697fa640 1 -- 172.21.15.88:0/2840865266 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55972ed86520 con 0x55972edcd1c0 2023-10-16T20:05:27.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef697fa640 1 -- 172.21.15.88:0/2840865266 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fef84005d20 con 0x55972edcd1c0 2023-10-16T20:05:27.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef93eb5000 10 client.4833 fetch_fsmap finished waiting for FSMap version 9 2023-10-16T20:05:27.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef93eb5000 10 client.4833 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-16T20:05:27.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef93eb5000 20 client.4833 populate_metadata read hostname 'smithi088' 2023-10-16T20:05:27.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef93eb5000 1 --2- 172.21.15.88:0/2840865266 >> [v2:172.21.15.94:6833/555575750,v1:172.21.15.94:6835/555575750] conn(0x55972ee2b350 0x55972ee2b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:27.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef93eb5000 4 client.4833 mds_command: new command op to 4272 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T20:05:27.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef93eb5000 1 -- 172.21.15.88:0/2840865266 --> [v2:172.21.15.94:6833/555575750,v1:172.21.15.94:6835/555575750] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55972eb6b6f0 con 0x55972ee2b350 2023-10-16T20:05:27.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef8a59c640 1 --2- 172.21.15.88:0/2840865266 >> [v2:172.21.15.94:6833/555575750,v1:172.21.15.94:6835/555575750] conn(0x55972ee2b350 0x55972ee2b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:27.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef697fa640 10 client.4833 ms_handle_connect on v2:172.21.15.94:6833/555575750 2023-10-16T20:05:27.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef8a59c640 1 --2- 172.21.15.88:0/2840865266 >> [v2:172.21.15.94:6833/555575750,v1:172.21.15.94:6835/555575750] conn(0x55972ee2b350 0x55972ee2b730 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a55774bf8c21549 server_cookie=5cce551ce933d72d in_seq=0 out_seq=0 2023-10-16T20:05:27.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef697fa640 1 -- 172.21.15.88:0/2840865266 <== mds.0 v2:172.21.15.94:6833/555575750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55972eb6b6f0 con 0x55972ee2b350 2023-10-16T20:05:27.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef697fa640 10 client.4833 handle_command_reply: tid=0 2023-10-16T20:05:27.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef93eb5000 1 client.4833 shutdown 2023-10-16T20:05:27.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef93eb5000 1 -- 172.21.15.88:0/2840865266 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55972edcd1c0 msgr2=0x55972ede7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:27.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef93eb5000 1 --2- 172.21.15.88:0/2840865266 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55972edcd1c0 0x55972ede7620 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fef840f0440 tx=0x7fef840f01d0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef93eb5000 1 -- 172.21.15.88:0/2840865266 shutdown_connections 2023-10-16T20:05:27.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef93eb5000 1 --2- 172.21.15.88:0/2840865266 >> [v2:172.21.15.94:6833/555575750,v1:172.21.15.94:6835/555575750] conn(0x55972ee2b350 0x55972ee2b730 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef93eb5000 1 --2- 172.21.15.88:0/2840865266 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55972edcd1c0 0x55972ede7620 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef93eb5000 1 --2- 172.21.15.88:0/2840865266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972edcc8a0 0x55972ede70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef93eb5000 1 --2- 172.21.15.88:0/2840865266 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55972ed82c50 0x55972edcb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef697fa640 0 client.4833 ms_handle_reset on v2:172.21.15.94:6833/555575750 2023-10-16T20:05:27.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef93eb5000 1 -- 172.21.15.88:0/2840865266 >> 172.21.15.88:0/2840865266 conn(0x55972ed79fc0 msgr2=0x55972eddf330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:27.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef93eb5000 1 -- 172.21.15.88:0/2840865266 shutdown_connections 2023-10-16T20:05:27.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef93eb5000 1 -- 172.21.15.88:0/2840865266 wait complete. 2023-10-16T20:05:27.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef93eb5000 20 client.4833 trim_cache size 0 max 16384 2023-10-16T20:05:27.028 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2023-10-16T20:05:27.029 INFO:teuthology.orchestra.run.smithi088.stdout: "cluster_fsid": "d11187c0-3dd8-4c92-8d13-6ec536230748", 2023-10-16T20:05:27.029 INFO:teuthology.orchestra.run.smithi088.stdout: "whoami": 0, 2023-10-16T20:05:27.029 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 4272, 2023-10-16T20:05:27.029 INFO:teuthology.orchestra.run.smithi088.stdout: "want_state": "up:active", 2023-10-16T20:05:27.029 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "up:active", 2023-10-16T20:05:27.030 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "cephfs", 2023-10-16T20:05:27.030 INFO:teuthology.orchestra.run.smithi088.stdout: "rank_uptime": 29.599657529000002, 2023-10-16T20:05:27.030 INFO:teuthology.orchestra.run.smithi088.stdout: "mdsmap_epoch": 9, 2023-10-16T20:05:27.030 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch": 27, 2023-10-16T20:05:27.030 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch_barrier": 27, 2023-10-16T20:05:27.031 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 37.218887475000002 2023-10-16T20:05:27.031 INFO:teuthology.orchestra.run.smithi088.stdout:} 2023-10-16T20:05:27.032 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef90b35640 1 -- 172.21.15.88:0/1483207785 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fef6c0664d0 msgr2=0x7fef6c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:27.032 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef90b35640 1 --2- 172.21.15.88:0/1483207785 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fef6c0664d0 0x7fef6c068990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0015f0 tx=0x7fef7c0079c0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.032 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef90b35640 1 -- 172.21.15.88:0/1483207785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef8c15cc30 msgr2=0x7fef8c0c99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:27.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef90b35640 1 --2- 172.21.15.88:0/1483207785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef8c15cc30 0x7fef8c0c99a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fef840b0040 tx=0x7fef84019630 comp rx=0 tx=0).stop 2023-10-16T20:05:27.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef90b35640 1 -- 172.21.15.88:0/1483207785 shutdown_connections 2023-10-16T20:05:27.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef90b35640 1 --2- 172.21.15.88:0/1483207785 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fef6c0664d0 0x7fef6c068990 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef90b35640 1 --2- 172.21.15.88:0/1483207785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef8c15cc30 0x7fef8c0c99a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef90b35640 1 --2- 172.21.15.88:0/1483207785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef8c0d1470 0x7fef8c0b5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef90b35640 1 --2- 172.21.15.88:0/1483207785 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef8c0cd090 0x7fef8c0b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef90b35640 1 -- 172.21.15.88:0/1483207785 >> 172.21.15.88:0/1483207785 conn(0x7fef8c099b80 msgr2=0x7fef8c15bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:27.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef90b35640 1 -- 172.21.15.88:0/1483207785 shutdown_connections 2023-10-16T20:05:27.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.006+0000 7fef90b35640 1 -- 172.21.15.88:0/1483207785 wait complete. 2023-10-16T20:05:27.035 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-10-16T20:05:27.046 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-10-16T20:05:27.046 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-16T20:05:27.046 INFO:tasks.ceph_fuse:client.0 config is {} 2023-10-16T20:05:27.047 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-16T20:05:27.047 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T20:05:27.047 INFO:tasks.ceph_fuse:client.2 config is {} 2023-10-16T20:05:27.047 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-16T20:05:27.048 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-16T20:05:27.048 INFO:tasks.ceph_fuse:client.1 config is {} 2023-10-16T20:05:27.048 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-16T20:05:27.048 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-16T20:05:27.049 INFO:tasks.ceph_fuse:client.3 config is {} 2023-10-16T20:05:27.049 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-16T20:05:27.049 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-16T20:05:27.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:27.050 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2023-10-16T20:05:27.060 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-10-16T20:05:27.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:27.061 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-16T20:05:27.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:27.125 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link delete ceph-brx 2023-10-16T20:05:27.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:27.316 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2023-10-16T20:05:27.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:27.327 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2023-10-16T20:05:27.393 INFO:teuthology.orchestra.run.smithi094.stderr:Cannot find device "ceph-brx" 2023-10-16T20:05:27.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T20:05:27.396 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-10-16T20:05:27.396 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T20:05:27.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.534+0000 7f24bca09640 1 Processor -- start 2023-10-16T20:05:27.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.534+0000 7f24bca09640 1 -- start start 2023-10-16T20:05:27.535 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.534+0000 7f24bca09640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f24b815c9a0 0x7f24b80cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:27.535 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.534+0000 7f24bca09640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f24b80cf230 0x7f24b80cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:27.535 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.534+0000 7f24bca09640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24b80d0c40 con 0x7f24b80d3270 2023-10-16T20:05:27.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.534+0000 7f24bca09640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24b80ce7e0 con 0x7f24b815c9a0 2023-10-16T20:05:27.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.534+0000 7f24bca09640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24b80d0630 con 0x7f24b80cf230 2023-10-16T20:05:27.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.534+0000 7f24b77fe640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f24b815c9a0 0x7f24b80cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:27.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.534+0000 7f24b77fe640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f24b815c9a0 0x7f24b80cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38244/0 (socket says 172.21.15.88:38244) 2023-10-16T20:05:27.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.534+0000 7f24b77fe640 1 -- 172.21.15.88:0/1220215535 learned_addr learned my addr 172.21.15.88:0/1220215535 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:27.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.534+0000 7f24b77fe640 1 -- 172.21.15.88:0/1220215535 >> v2:172.21.15.94:3300/0 conn(0x7f24b80cf230 msgr2=0x7f24b80cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:27.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.534+0000 7f24b77fe640 1 --2- 172.21.15.88:0/1220215535 >> v2:172.21.15.94:3300/0 conn(0x7f24b80cf230 0x7f24b80cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.534+0000 7f24b77fe640 1 -- 172.21.15.88:0/1220215535 >> v1:172.21.15.88:6789/0 conn(0x7f24b80d3270 legacy=0x7f24b80d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:27.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.534+0000 7f24b77fe640 1 -- 172.21.15.88:0/1220215535 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b80ce520 con 0x7f24b815c9a0 2023-10-16T20:05:27.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.534+0000 7f24b67fc640 1 -- 172.21.15.88:0/1220215535 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3217462184 0 0) 0x7f24b80d0c40 con 0x7f24b80d3270 2023-10-16T20:05:27.539 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.538+0000 7f24b77fe640 1 --2- 172.21.15.88:0/1220215535 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24b815c9a0 0x7f24b80cde80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f24b80c63a0 tx=0x7f24a801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa364dbb1d7d3d7e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:27.539 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.538+0000 7f24b67fc640 1 -- 172.21.15.88:0/1220215535 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a80aa650 con 0x7f24b815c9a0 2023-10-16T20:05:27.539 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.538+0000 7f24b67fc640 1 -- 172.21.15.88:0/1220215535 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a80aa7b0 con 0x7f24b815c9a0 2023-10-16T20:05:27.539 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.538+0000 7f24b67fc640 1 -- 172.21.15.88:0/1220215535 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a80aab40 con 0x7f24b815c9a0 2023-10-16T20:05:27.540 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.538+0000 7f24bca09640 1 -- 172.21.15.88:0/1220215535 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24b815c9a0 msgr2=0x7f24b80cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:27.540 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.538+0000 7f24bca09640 1 --2- 172.21.15.88:0/1220215535 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24b815c9a0 0x7f24b80cde80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f24b80c63a0 tx=0x7f24a801c4d0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.538+0000 7f24bca09640 1 -- 172.21.15.88:0/1220215535 shutdown_connections 2023-10-16T20:05:27.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.538+0000 7f24bca09640 1 --2- 172.21.15.88:0/1220215535 >> v2:172.21.15.94:3300/0 conn(0x7f24b80cf230 0x7f24b80cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.538+0000 7f24bca09640 1 --2- 172.21.15.88:0/1220215535 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24b815c9a0 0x7f24b80cde80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.538+0000 7f24bca09640 1 -- 172.21.15.88:0/1220215535 >> 172.21.15.88:0/1220215535 conn(0x7f24b809bc30 msgr2=0x7f24b80d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:27.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.538+0000 7f24bca09640 1 -- 172.21.15.88:0/1220215535 shutdown_connections 2023-10-16T20:05:27.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.538+0000 7f24bca09640 1 -- 172.21.15.88:0/1220215535 wait complete. 2023-10-16T20:05:27.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.542+0000 7f24bca09640 1 Processor -- start 2023-10-16T20:05:27.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.542+0000 7f24bca09640 1 -- start start 2023-10-16T20:05:27.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.542+0000 7f24bca09640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24b80cf230 0x7f24b80c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:27.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.542+0000 7f24bca09640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24b80d3270 0x7f24b8152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:27.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.542+0000 7f24bca09640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f24b815c9a0 0x7f24b8157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:27.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.542+0000 7f24bca09640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b8157100 con 0x7f24b80d3270 2023-10-16T20:05:27.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.542+0000 7f24bca09640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b80c8b00 con 0x7f24b80cf230 2023-10-16T20:05:27.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.542+0000 7f24bca09640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b80c8da0 con 0x7f24b815c9a0 2023-10-16T20:05:27.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.542+0000 7f24b6ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24b80d3270 0x7f24b8152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:27.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.542+0000 7f24b6ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24b80d3270 0x7f24b8152160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38260/0 (socket says 172.21.15.88:38260) 2023-10-16T20:05:27.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.542+0000 7f24b6ffd640 1 -- 172.21.15.88:0/4080200509 learned_addr learned my addr 172.21.15.88:0/4080200509 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:27.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.542+0000 7f24b7fff640 1 --2- 172.21.15.88:0/4080200509 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f24b815c9a0 0x7f24b8157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:27.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.542+0000 7f24b77fe640 1 --2- 172.21.15.88:0/4080200509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24b80cf230 0x7f24b80c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:27.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.542+0000 7f24b6ffd640 1 -- 172.21.15.88:0/4080200509 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f24b815c9a0 msgr2=0x7f24b8157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:27.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.542+0000 7f24b6ffd640 1 --2- 172.21.15.88:0/4080200509 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f24b815c9a0 0x7f24b8157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.542+0000 7f24b6ffd640 1 -- 172.21.15.88:0/4080200509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24b80cf230 msgr2=0x7f24b80c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:27.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.542+0000 7f24b6ffd640 1 --2- 172.21.15.88:0/4080200509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24b80cf230 0x7f24b80c6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.542+0000 7f24b6ffd640 1 -- 172.21.15.88:0/4080200509 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a8009470 con 0x7f24b80d3270 2023-10-16T20:05:27.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.542+0000 7f24b77fe640 1 --2- 172.21.15.88:0/4080200509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24b80cf230 0x7f24b80c6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:27.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.542+0000 7f24b7fff640 1 --2- 172.21.15.88:0/4080200509 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f24b815c9a0 0x7f24b8157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:27.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.542+0000 7f24b6ffd640 1 --2- 172.21.15.88:0/4080200509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24b80d3270 0x7f24b8152160 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f24ac001ec0 tx=0x7f24ac00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:27.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.542+0000 7f249ffff640 1 -- 172.21.15.88:0/4080200509 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ac0054e0 con 0x7f24b80d3270 2023-10-16T20:05:27.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.542+0000 7f249ffff640 1 -- 172.21.15.88:0/4080200509 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24ac003600 con 0x7f24b80d3270 2023-10-16T20:05:27.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.542+0000 7f24bca09640 1 -- 172.21.15.88:0/4080200509 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b80c9080 con 0x7f24b80d3270 2023-10-16T20:05:27.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.542+0000 7f249ffff640 1 -- 172.21.15.88:0/4080200509 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ac0039b0 con 0x7f24b80d3270 2023-10-16T20:05:27.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.542+0000 7f24bca09640 1 -- 172.21.15.88:0/4080200509 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b8153950 con 0x7f24b80d3270 2023-10-16T20:05:27.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.542+0000 7f24bca09640 1 -- 172.21.15.88:0/4080200509 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24b8157240 con 0x7f24b80d3270 2023-10-16T20:05:27.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.546+0000 7f249ffff640 1 -- 172.21.15.88:0/4080200509 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f24ac003b10 con 0x7f24b80d3270 2023-10-16T20:05:27.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.546+0000 7f249ffff640 1 --2- 172.21.15.88:0/4080200509 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f24a40664a0 0x7f24a4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:27.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.546+0000 7f249ffff640 1 -- 172.21.15.88:0/4080200509 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f24ac0837f0 con 0x7f24b80d3270 2023-10-16T20:05:27.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.546+0000 7f24b77fe640 1 --2- 172.21.15.88:0/4080200509 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f24a40664a0 0x7f24a4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:27.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.546+0000 7f24b77fe640 1 --2- 172.21.15.88:0/4080200509 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f24a40664a0 0x7f24a4068960 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f24a80ae000 tx=0x7f24a80a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:27.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.550+0000 7f249ffff640 1 -- 172.21.15.88:0/4080200509 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f24b8157240 con 0x7f24b80d3270 2023-10-16T20:05:27.732 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.730+0000 7f24bca09640 1 -- 172.21.15.88:0/4080200509 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f24b8157240 con 0x7f24b80d3270 2023-10-16T20:05:27.732 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.730+0000 7f249ffff640 1 -- 172.21.15.88:0/4080200509 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f24b8157240 con 0x7f24b80d3270 2023-10-16T20:05:27.733 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-16T20:05:27.734 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.734+0000 7f24bca09640 1 -- 172.21.15.88:0/4080200509 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f24a40664a0 msgr2=0x7f24a4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:27.734 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.734+0000 7f24bca09640 1 --2- 172.21.15.88:0/4080200509 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f24a40664a0 0x7f24a4068960 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f24a80ae000 tx=0x7f24a80a7000 comp rx=0 tx=0).stop 2023-10-16T20:05:27.735 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.734+0000 7f24bca09640 1 -- 172.21.15.88:0/4080200509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24b80d3270 msgr2=0x7f24b8152160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:27.735 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.734+0000 7f24bca09640 1 --2- 172.21.15.88:0/4080200509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24b80d3270 0x7f24b8152160 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f24ac001ec0 tx=0x7f24ac00c8e0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.735 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.734+0000 7f24bca09640 1 -- 172.21.15.88:0/4080200509 shutdown_connections 2023-10-16T20:05:27.735 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.734+0000 7f24bca09640 1 --2- 172.21.15.88:0/4080200509 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f24a40664a0 0x7f24a4068960 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.735 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.734+0000 7f24bca09640 1 --2- 172.21.15.88:0/4080200509 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f24b815c9a0 0x7f24b8157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.736 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.734+0000 7f24bca09640 1 --2- 172.21.15.88:0/4080200509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24b80d3270 0x7f24b8152160 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.736 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.734+0000 7f24bca09640 1 --2- 172.21.15.88:0/4080200509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24b80cf230 0x7f24b80c6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:27.736 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.734+0000 7f24bca09640 1 -- 172.21.15.88:0/4080200509 >> 172.21.15.88:0/4080200509 conn(0x7f24b809bc30 msgr2=0x7f24b806c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:27.736 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.734+0000 7f24bca09640 1 -- 172.21.15.88:0/4080200509 shutdown_connections 2023-10-16T20:05:27.736 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:27.734+0000 7f24bca09640 1 -- 172.21.15.88:0/4080200509 wait complete. 2023-10-16T20:05:27.753 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T20:05:27.753 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T20:05:27.753 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi088.front.sepia.ceph.com 2023-10-16T20:05:27.754 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T20:05:27.754 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T20:05:27.754 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T20:05:27.754 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T20:05:27.755 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T20:05:27.755 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T20:05:27.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:27.756 DEBUG:teuthology.orchestra.run.smithi088:> ip addr 2023-10-16T20:05:27.765 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T20:05:27.765 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T20:05:27.765 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T20:05:27.766 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:27.766 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2023-10-16T20:05:27.766 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:27.766 INFO:teuthology.orchestra.run.smithi088.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:05:27.767 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f0 brd ff:ff:ff:ff:ff:ff 2023-10-16T20:05:27.767 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f0 2023-10-16T20:05:27.767 INFO:teuthology.orchestra.run.smithi088.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:05:27.767 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f1 brd ff:ff:ff:ff:ff:ff 2023-10-16T20:05:27.767 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f1 2023-10-16T20:05:27.768 INFO:teuthology.orchestra.run.smithi088.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:05:27.768 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2c brd ff:ff:ff:ff:ff:ff 2023-10-16T20:05:27.768 INFO:teuthology.orchestra.run.smithi088.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T20:05:27.768 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2d brd ff:ff:ff:ff:ff:ff 2023-10-16T20:05:27.768 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.21.15.88/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-16T20:05:27.769 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft 42404sec preferred_lft 42404sec 2023-10-16T20:05:27.769 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link 2023-10-16T20:05:27.769 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:27.769 INFO:teuthology.orchestra.run.smithi088.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-16T20:05:27.769 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 02:42:d9:3b:fb:39 brd ff:ff:ff:ff:ff:ff 2023-10-16T20:05:27.769 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-16T20:05:27.770 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:27.771 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-16T20:05:27.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:27.771 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec bash -c ' 2023-10-16T20:05:27.772 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2023-10-16T20:05:27.772 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link add name ceph-brx type bridge 2023-10-16T20:05:27.772 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip addr flush dev ceph-brx 2023-10-16T20:05:27.773 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set ceph-brx up 2023-10-16T20:05:27.773 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-16T20:05:27.773 DEBUG:teuthology.orchestra.run.smithi088:> ') 2023-10-16T20:05:27.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:27.884 DEBUG:teuthology.orchestra.run.smithi088:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T20:05:27.903 INFO:teuthology.orchestra.run.smithi088.stdout:1 2023-10-16T20:05:27.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:27.906 DEBUG:teuthology.orchestra.run.smithi088:> route 2023-10-16T20:05:27.959 INFO:teuthology.orchestra.run.smithi088.stdout:Kernel IP routing table 2023-10-16T20:05:27.960 INFO:teuthology.orchestra.run.smithi088.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T20:05:27.960 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-16T20:05:27.960 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T20:05:27.960 INFO:teuthology.orchestra.run.smithi088.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-16T20:05:27.961 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T20:05:27.961 INFO:teuthology.orchestra.run.smithi088.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:05:27.961 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:05:27.961 INFO:teuthology.orchestra.run.smithi088.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:05:27.961 INFO:teuthology.orchestra.run.smithi088.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T20:05:27.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:27.962 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec bash -c ' 2023-10-16T20:05:27.962 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2023-10-16T20:05:27.963 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T20:05:27.963 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T20:05:27.963 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T20:05:27.963 DEBUG:teuthology.orchestra.run.smithi088:> ') 2023-10-16T20:05:28.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:28.051 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2023-10-16T20:05:28.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:28.061 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list-id 2023-10-16T20:05:28.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:28.113 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec bash -c ' 2023-10-16T20:05:28.113 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2023-10-16T20:05:28.114 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:05:28.114 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-16T20:05:28.114 DEBUG:teuthology.orchestra.run.smithi088:> ') 2023-10-16T20:05:28.194 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-16T20:05:28.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:28.195 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec bash -c ' 2023-10-16T20:05:28.195 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2023-10-16T20:05:28.196 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-16T20:05:28.196 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-16T20:05:28.196 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-16T20:05:28.196 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-16T20:05:28.197 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-16T20:05:28.197 DEBUG:teuthology.orchestra.run.smithi088:> ') 2023-10-16T20:05:28.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:28.333 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec bash -c ' 2023-10-16T20:05:28.333 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2023-10-16T20:05:28.333 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set brx.0 up 2023-10-16T20:05:28.333 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set dev brx.0 master ceph-brx 2023-10-16T20:05:28.334 DEBUG:teuthology.orchestra.run.smithi088:> ') 2023-10-16T20:05:28.374 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-16T20:05:28.374 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T20:05:28.375 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T20:05:28.425 INFO:teuthology.orchestra.run.smithi088.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T20:05:28.425 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T20:05:28.425 DEBUG:teuthology.orchestra.run.smithi088:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T20:05:28.480 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2023-10-16T20:05:28.541 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T20:05:28.598 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2023-10-16T20:05:28.626 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2023-10-16T20:05:28.626 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2023-10-16T20:05:28.626 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2023-10-16T20:05:28.627 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2023-10-16T20:05:28.627 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2023-10-16T20:05:28.627 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2023-10-16T20:05:28.627 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2023-10-16T20:05:28.627 INFO:teuthology.orchestra.run.smithi088.stdout:/run/lock 2023-10-16T20:05:28.627 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2023-10-16T20:05:28.628 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2023-10-16T20:05:28.628 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2023-10-16T20:05:28.628 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T20:05:28.628 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2023-10-16T20:05:28.628 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2023-10-16T20:05:28.629 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2023-10-16T20:05:28.629 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2023-10-16T20:05:28.629 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/fs/nfsd 2023-10-16T20:05:28.629 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2023-10-16T20:05:28.629 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2023-10-16T20:05:28.629 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T20:05:28.630 INFO:teuthology.orchestra.run.smithi088.stdout:/run/rpc_pipefs 2023-10-16T20:05:28.630 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2023-10-16T20:05:28.630 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2023-10-16T20:05:28.630 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T20:05:28.630 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T20:05:28.631 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T20:05:28.631 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T20:05:28.631 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2023-10-16T20:05:28.631 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:05:28.631 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:05:28.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:28.632 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2023-10-16T20:05:28.680 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-16T20:05:28.681 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-16T20:05:28.727 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2023-10-16T20:05:28.749 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T20:05:28.803 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2023-10-16T20:05:28.803 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2023-10-16T20:05:28.803 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2023-10-16T20:05:28.803 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2023-10-16T20:05:28.804 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2023-10-16T20:05:28.804 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2023-10-16T20:05:28.804 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2023-10-16T20:05:28.804 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2023-10-16T20:05:28.804 INFO:teuthology.orchestra.run.smithi088.stdout:/run/lock 2023-10-16T20:05:28.805 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2023-10-16T20:05:28.805 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2023-10-16T20:05:28.805 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2023-10-16T20:05:28.805 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T20:05:28.805 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2023-10-16T20:05:28.806 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2023-10-16T20:05:28.806 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2023-10-16T20:05:28.806 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2023-10-16T20:05:28.806 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/fs/nfsd 2023-10-16T20:05:28.806 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2023-10-16T20:05:28.807 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2023-10-16T20:05:28.807 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T20:05:28.807 INFO:teuthology.orchestra.run.smithi088.stdout:/run/rpc_pipefs 2023-10-16T20:05:28.807 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2023-10-16T20:05:28.807 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2023-10-16T20:05:28.808 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T20:05:28.808 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T20:05:28.808 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T20:05:28.808 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T20:05:28.809 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2023-10-16T20:05:28.809 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:05:28.809 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:05:28.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:28.810 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2023-10-16T20:05:28.822 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:2023-10-16T20:05:28.822+0000 7f0ad18bf580 -1 init, newargv = 0x55beaa1dadd0 newargc=15 2023-10-16T20:05:28.822 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[45902]: starting ceph client 2023-10-16T20:05:28.830 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[45902]: starting fuse 2023-10-16T20:05:28.831 INFO:teuthology.orchestra.run.smithi088.stdout:43 2023-10-16T20:05:28.832 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-10-16T20:05:28.832 DEBUG:teuthology.orchestra.run.smithi088:> sudo stdin-killer -- python3 -c ' 2023-10-16T20:05:28.832 DEBUG:teuthology.orchestra.run.smithi088:> import glob 2023-10-16T20:05:28.832 DEBUG:teuthology.orchestra.run.smithi088:> import re 2023-10-16T20:05:28.833 DEBUG:teuthology.orchestra.run.smithi088:> import os 2023-10-16T20:05:28.833 DEBUG:teuthology.orchestra.run.smithi088:> import subprocess 2023-10-16T20:05:28.833 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:05:28.834 DEBUG:teuthology.orchestra.run.smithi088:> def _find_admin_socket(client_name): 2023-10-16T20:05:28.834 DEBUG:teuthology.orchestra.run.smithi088:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-16T20:05:28.834 DEBUG:teuthology.orchestra.run.smithi088:> files = glob.glob(asok_path) 2023-10-16T20:05:28.835 DEBUG:teuthology.orchestra.run.smithi088:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-16T20:05:28.835 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:05:28.835 DEBUG:teuthology.orchestra.run.smithi088:> # Given a non-glob path, it better be there 2023-10-16T20:05:28.836 DEBUG:teuthology.orchestra.run.smithi088:> if "*" not in asok_path: 2023-10-16T20:05:28.836 DEBUG:teuthology.orchestra.run.smithi088:> assert(len(files) == 1) 2023-10-16T20:05:28.836 DEBUG:teuthology.orchestra.run.smithi088:> return files[0] 2023-10-16T20:05:28.836 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:05:28.837 DEBUG:teuthology.orchestra.run.smithi088:> for f in files: 2023-10-16T20:05:28.837 DEBUG:teuthology.orchestra.run.smithi088:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-16T20:05:28.837 DEBUG:teuthology.orchestra.run.smithi088:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-16T20:05:28.838 DEBUG:teuthology.orchestra.run.smithi088:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-16T20:05:28.838 DEBUG:teuthology.orchestra.run.smithi088:> contents = proc_f.read() 2023-10-16T20:05:28.838 DEBUG:teuthology.orchestra.run.smithi088:> if mountpoint in contents: 2023-10-16T20:05:28.839 DEBUG:teuthology.orchestra.run.smithi088:> return f 2023-10-16T20:05:28.839 DEBUG:teuthology.orchestra.run.smithi088:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-16T20:05:28.839 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:05:28.840 DEBUG:teuthology.orchestra.run.smithi088:> print(_find_admin_socket("client.0")) 2023-10-16T20:05:28.840 DEBUG:teuthology.orchestra.run.smithi088:> ' 2023-10-16T20:05:28.953 INFO:teuthology.orchestra.run.smithi088.stdout:/var/run/ceph/ceph-client.0.45902.asok 2023-10-16T20:05:28.957 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-16T20:05:28.964 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.45902.asok 2023-10-16T20:05:28.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:28.965 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.45902.asok status 2023-10-16T20:05:29.133 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2023-10-16T20:05:29.133 INFO:teuthology.orchestra.run.smithi088.stdout: "metadata": { 2023-10-16T20:05:29.133 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_sha1": "4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0", 2023-10-16T20:05:29.134 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_version": "ceph version 18.0.0-6730-g4ba8a190 (4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0) reef (dev)", 2023-10-16T20:05:29.134 INFO:teuthology.orchestra.run.smithi088.stdout: "entity_id": "0", 2023-10-16T20:05:29.134 INFO:teuthology.orchestra.run.smithi088.stdout: "hostname": "smithi088", 2023-10-16T20:05:29.134 INFO:teuthology.orchestra.run.smithi088.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-16T20:05:29.135 INFO:teuthology.orchestra.run.smithi088.stdout: "pid": "45902", 2023-10-16T20:05:29.135 INFO:teuthology.orchestra.run.smithi088.stdout: "root": "/" 2023-10-16T20:05:29.135 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:05:29.135 INFO:teuthology.orchestra.run.smithi088.stdout: "dentry_count": 0, 2023-10-16T20:05:29.136 INFO:teuthology.orchestra.run.smithi088.stdout: "dentry_pinned_count": 0, 2023-10-16T20:05:29.136 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 4851, 2023-10-16T20:05:29.136 INFO:teuthology.orchestra.run.smithi088.stdout: "inst": { 2023-10-16T20:05:29.136 INFO:teuthology.orchestra.run.smithi088.stdout: "name": { 2023-10-16T20:05:29.137 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "client", 2023-10-16T20:05:29.137 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 4851 2023-10-16T20:05:29.137 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:05:29.137 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": { 2023-10-16T20:05:29.138 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2023-10-16T20:05:29.138 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "192.168.0.1:0", 2023-10-16T20:05:29.138 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 2077805299 2023-10-16T20:05:29.138 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:05:29.139 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:05:29.139 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": { 2023-10-16T20:05:29.139 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2023-10-16T20:05:29.139 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "192.168.0.1:0", 2023-10-16T20:05:29.139 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 2077805299 2023-10-16T20:05:29.140 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:05:29.140 INFO:teuthology.orchestra.run.smithi088.stdout: "inst_str": "client.4851 192.168.0.1:0/2077805299", 2023-10-16T20:05:29.140 INFO:teuthology.orchestra.run.smithi088.stdout: "addr_str": "192.168.0.1:0/2077805299", 2023-10-16T20:05:29.140 INFO:teuthology.orchestra.run.smithi088.stdout: "inode_count": 1, 2023-10-16T20:05:29.140 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_epoch": 9, 2023-10-16T20:05:29.141 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_epoch": 27, 2023-10-16T20:05:29.141 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_epoch_barrier": 0, 2023-10-16T20:05:29.141 INFO:teuthology.orchestra.run.smithi088.stdout: "blocklisted": false, 2023-10-16T20:05:29.141 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "cephfs" 2023-10-16T20:05:29.142 INFO:teuthology.orchestra.run.smithi088.stdout:} 2023-10-16T20:05:29.148 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T20:05:29.285 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.282+0000 7fde28446640 1 Processor -- start 2023-10-16T20:05:29.285 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.282+0000 7fde28446640 1 -- start start 2023-10-16T20:05:29.287 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.286+0000 7fde28446640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fde2015cb80 0x7fde200b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:29.287 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.286+0000 7fde28446640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fde200cebd0 0x7fde2006bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:29.287 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.286+0000 7fde28446640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde2006b490 con 0x7fde200b28b0 2023-10-16T20:05:29.287 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.286+0000 7fde28446640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde200b2d20 con 0x7fde2015cb80 2023-10-16T20:05:29.287 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.286+0000 7fde28446640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde200cf180 con 0x7fde200cebd0 2023-10-16T20:05:29.288 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.286+0000 7fde261e2640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fde2015cb80 0x7fde200b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:29.288 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.286+0000 7fde261e2640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fde2015cb80 0x7fde200b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38270/0 (socket says 172.21.15.88:38270) 2023-10-16T20:05:29.288 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.286+0000 7fde261e2640 1 -- 172.21.15.88:0/1834546737 learned_addr learned my addr 172.21.15.88:0/1834546737 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:29.289 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.286+0000 7fde269e3640 1 --2- 172.21.15.88:0/1834546737 >> v2:172.21.15.94:3300/0 conn(0x7fde200cebd0 0x7fde2006bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:29.289 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.286+0000 7fde261e2640 1 -- 172.21.15.88:0/1834546737 >> v2:172.21.15.94:3300/0 conn(0x7fde200cebd0 msgr2=0x7fde2006bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:29.289 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.286+0000 7fde261e2640 1 --2- 172.21.15.88:0/1834546737 >> v2:172.21.15.94:3300/0 conn(0x7fde200cebd0 0x7fde2006bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:29.289 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.286+0000 7fde261e2640 1 -- 172.21.15.88:0/1834546737 >> v1:172.21.15.94:6789/0 conn(0x7fde200b28b0 legacy=0x7fde200ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:29.289 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.286+0000 7fde261e2640 1 -- 172.21.15.88:0/1834546737 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde2006b700 con 0x7fde2015cb80 2023-10-16T20:05:29.290 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.290+0000 7fde261e2640 1 --2- 172.21.15.88:0/1834546737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde2015cb80 0x7fde200b2370 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fde200c43c0 tx=0x7fde1c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=777d29e616a0914b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:29.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.290+0000 7fde251e0640 1 -- 172.21.15.88:0/1834546737 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c0ab600 con 0x7fde2015cb80 2023-10-16T20:05:29.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.290+0000 7fde251e0640 1 -- 172.21.15.88:0/1834546737 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde1c0a7050 con 0x7fde2015cb80 2023-10-16T20:05:29.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.290+0000 7fde251e0640 1 -- 172.21.15.88:0/1834546737 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c0ab980 con 0x7fde2015cb80 2023-10-16T20:05:29.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.290+0000 7fde28446640 1 -- 172.21.15.88:0/1834546737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde2015cb80 msgr2=0x7fde200b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:29.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.290+0000 7fde28446640 1 --2- 172.21.15.88:0/1834546737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde2015cb80 0x7fde200b2370 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fde200c43c0 tx=0x7fde1c01c500 comp rx=0 tx=0).stop 2023-10-16T20:05:29.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.290+0000 7fde28446640 1 -- 172.21.15.88:0/1834546737 shutdown_connections 2023-10-16T20:05:29.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.290+0000 7fde28446640 1 --2- 172.21.15.88:0/1834546737 >> v2:172.21.15.94:3300/0 conn(0x7fde200cebd0 0x7fde2006bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:29.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.294+0000 7fde28446640 1 --2- 172.21.15.88:0/1834546737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde2015cb80 0x7fde200b2370 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:29.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.294+0000 7fde28446640 1 -- 172.21.15.88:0/1834546737 >> 172.21.15.88:0/1834546737 conn(0x7fde200cd370 msgr2=0x7fde200cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:29.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.294+0000 7fde28446640 1 -- 172.21.15.88:0/1834546737 shutdown_connections 2023-10-16T20:05:29.295 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.294+0000 7fde28446640 1 -- 172.21.15.88:0/1834546737 wait complete. 2023-10-16T20:05:29.295 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.294+0000 7fde28446640 1 Processor -- start 2023-10-16T20:05:29.295 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.294+0000 7fde28446640 1 -- start start 2023-10-16T20:05:29.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.294+0000 7fde28446640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fde200b28b0 0x7fde200c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:29.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.294+0000 7fde28446640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde200cebd0 0x7fde20144b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:29.298 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.294+0000 7fde28446640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde2015cb80 0x7fde20141870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:29.298 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.294+0000 7fde28446640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde200c45a0 con 0x7fde2015cb80 2023-10-16T20:05:29.298 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.294+0000 7fde28446640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde201450a0 con 0x7fde200cebd0 2023-10-16T20:05:29.298 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.294+0000 7fde28446640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde20141db0 con 0x7fde200b28b0 2023-10-16T20:05:29.298 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.294+0000 7fde269e3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde2015cb80 0x7fde20141870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:29.299 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.294+0000 7fde269e3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde2015cb80 0x7fde20141870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38286/0 (socket says 172.21.15.88:38286) 2023-10-16T20:05:29.299 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.294+0000 7fde269e3640 1 -- 172.21.15.88:0/3300954171 learned_addr learned my addr 172.21.15.88:0/3300954171 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:29.299 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.294+0000 7fde261e2640 1 --2- 172.21.15.88:0/3300954171 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fde200b28b0 0x7fde200c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:29.299 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.294+0000 7fde259e1640 1 --2- 172.21.15.88:0/3300954171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde200cebd0 0x7fde20144b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:29.299 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.294+0000 7fde269e3640 1 -- 172.21.15.88:0/3300954171 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fde200b28b0 msgr2=0x7fde200c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:29.300 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.294+0000 7fde269e3640 1 --2- 172.21.15.88:0/3300954171 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fde200b28b0 0x7fde200c6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:29.300 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.294+0000 7fde269e3640 1 -- 172.21.15.88:0/3300954171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde200cebd0 msgr2=0x7fde20144b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:29.300 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.294+0000 7fde269e3640 1 --2- 172.21.15.88:0/3300954171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde200cebd0 0x7fde20144b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:29.300 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.294+0000 7fde269e3640 1 -- 172.21.15.88:0/3300954171 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1c0094d0 con 0x7fde2015cb80 2023-10-16T20:05:29.300 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.294+0000 7fde269e3640 1 --2- 172.21.15.88:0/3300954171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde2015cb80 0x7fde20141870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fde18007e40 tx=0x7fde1800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:29.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.298+0000 7fde16ffd640 1 -- 172.21.15.88:0/3300954171 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde18014620 con 0x7fde2015cb80 2023-10-16T20:05:29.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.298+0000 7fde16ffd640 1 -- 172.21.15.88:0/3300954171 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde1800ee90 con 0x7fde2015cb80 2023-10-16T20:05:29.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.298+0000 7fde16ffd640 1 -- 172.21.15.88:0/3300954171 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde180149e0 con 0x7fde2015cb80 2023-10-16T20:05:29.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.298+0000 7fde28446640 1 -- 172.21.15.88:0/3300954171 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde200c6810 con 0x7fde2015cb80 2023-10-16T20:05:29.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.298+0000 7fde28446640 1 -- 172.21.15.88:0/3300954171 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde200c6dd0 con 0x7fde2015cb80 2023-10-16T20:05:29.302 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.298+0000 7fde28446640 1 -- 172.21.15.88:0/3300954171 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde20142270 con 0x7fde2015cb80 2023-10-16T20:05:29.302 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.298+0000 7fde16ffd640 1 -- 172.21.15.88:0/3300954171 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fde18014b40 con 0x7fde2015cb80 2023-10-16T20:05:29.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.298+0000 7fde16ffd640 1 --2- 172.21.15.88:0/3300954171 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fde040664a0 0x7fde04068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:29.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.298+0000 7fde16ffd640 1 -- 172.21.15.88:0/3300954171 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fde18084ab0 con 0x7fde2015cb80 2023-10-16T20:05:29.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.298+0000 7fde261e2640 1 --2- 172.21.15.88:0/3300954171 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fde040664a0 0x7fde04068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:29.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.298+0000 7fde261e2640 1 --2- 172.21.15.88:0/3300954171 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fde040664a0 0x7fde04068960 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fde1c0ad000 tx=0x7fde1c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:29.304 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.302+0000 7fde16ffd640 1 -- 172.21.15.88:0/3300954171 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fde20142270 con 0x7fde2015cb80 2023-10-16T20:05:29.491 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.490+0000 7fde28446640 1 -- 172.21.15.88:0/3300954171 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fde20142270 con 0x7fde2015cb80 2023-10-16T20:05:29.492 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.490+0000 7fde16ffd640 1 -- 172.21.15.88:0/3300954171 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fde20142270 con 0x7fde2015cb80 2023-10-16T20:05:29.492 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-16T20:05:29.494 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.494+0000 7fde28446640 1 -- 172.21.15.88:0/3300954171 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fde040664a0 msgr2=0x7fde04068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:29.494 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.494+0000 7fde28446640 1 --2- 172.21.15.88:0/3300954171 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fde040664a0 0x7fde04068960 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fde1c0ad000 tx=0x7fde1c008040 comp rx=0 tx=0).stop 2023-10-16T20:05:29.494 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.494+0000 7fde28446640 1 -- 172.21.15.88:0/3300954171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde2015cb80 msgr2=0x7fde20141870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:29.494 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.494+0000 7fde28446640 1 --2- 172.21.15.88:0/3300954171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde2015cb80 0x7fde20141870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fde18007e40 tx=0x7fde1800e4b0 comp rx=0 tx=0).stop 2023-10-16T20:05:29.495 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.494+0000 7fde28446640 1 -- 172.21.15.88:0/3300954171 shutdown_connections 2023-10-16T20:05:29.495 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.494+0000 7fde28446640 1 --2- 172.21.15.88:0/3300954171 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fde040664a0 0x7fde04068960 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:29.495 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.494+0000 7fde28446640 1 --2- 172.21.15.88:0/3300954171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde2015cb80 0x7fde20141870 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:29.495 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.494+0000 7fde28446640 1 --2- 172.21.15.88:0/3300954171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde200cebd0 0x7fde20144b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:29.495 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.494+0000 7fde28446640 1 --2- 172.21.15.88:0/3300954171 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fde200b28b0 0x7fde200c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:29.495 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.494+0000 7fde28446640 1 -- 172.21.15.88:0/3300954171 >> 172.21.15.88:0/3300954171 conn(0x7fde200cd370 msgr2=0x7fde20158850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:29.496 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.494+0000 7fde28446640 1 -- 172.21.15.88:0/3300954171 shutdown_connections 2023-10-16T20:05:29.496 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:29.494+0000 7fde28446640 1 -- 172.21.15.88:0/3300954171 wait complete. 2023-10-16T20:05:29.512 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T20:05:29.512 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T20:05:29.513 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi088.front.sepia.ceph.com 2023-10-16T20:05:29.513 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-16T20:05:29.513 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-16T20:05:29.513 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T20:05:29.513 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T20:05:29.514 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T20:05:29.514 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-16T20:05:29.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:29.515 DEBUG:teuthology.orchestra.run.smithi088:> ip addr 2023-10-16T20:05:29.524 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T20:05:29.525 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T20:05:29.525 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T20:05:29.525 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:29.525 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2023-10-16T20:05:29.525 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:29.526 INFO:teuthology.orchestra.run.smithi088.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:05:29.526 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f0 brd ff:ff:ff:ff:ff:ff 2023-10-16T20:05:29.526 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f0 2023-10-16T20:05:29.526 INFO:teuthology.orchestra.run.smithi088.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:05:29.526 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f1 brd ff:ff:ff:ff:ff:ff 2023-10-16T20:05:29.527 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f1 2023-10-16T20:05:29.527 INFO:teuthology.orchestra.run.smithi088.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:05:29.527 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2c brd ff:ff:ff:ff:ff:ff 2023-10-16T20:05:29.527 INFO:teuthology.orchestra.run.smithi088.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T20:05:29.527 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2d brd ff:ff:ff:ff:ff:ff 2023-10-16T20:05:29.527 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.21.15.88/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-16T20:05:29.528 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft 42402sec preferred_lft 42402sec 2023-10-16T20:05:29.528 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link 2023-10-16T20:05:29.528 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:29.528 INFO:teuthology.orchestra.run.smithi088.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-16T20:05:29.528 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 02:42:d9:3b:fb:39 brd ff:ff:ff:ff:ff:ff 2023-10-16T20:05:29.529 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-16T20:05:29.529 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:29.529 INFO:teuthology.orchestra.run.smithi088.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T20:05:29.529 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-16T20:05:29.529 INFO:teuthology.orchestra.run.smithi088.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T20:05:29.529 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:29.530 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link tentative 2023-10-16T20:05:29.530 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:29.530 INFO:teuthology.orchestra.run.smithi088.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T20:05:29.530 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:05:29.530 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-16T20:05:29.530 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:29.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:29.531 DEBUG:teuthology.orchestra.run.smithi088:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T20:05:29.587 INFO:teuthology.orchestra.run.smithi088.stdout:1 2023-10-16T20:05:29.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:29.589 DEBUG:teuthology.orchestra.run.smithi088:> route 2023-10-16T20:05:29.643 INFO:teuthology.orchestra.run.smithi088.stdout:Kernel IP routing table 2023-10-16T20:05:29.643 INFO:teuthology.orchestra.run.smithi088.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T20:05:29.644 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-16T20:05:29.644 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T20:05:29.644 INFO:teuthology.orchestra.run.smithi088.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-16T20:05:29.644 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T20:05:29.644 INFO:teuthology.orchestra.run.smithi088.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:05:29.644 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:05:29.645 INFO:teuthology.orchestra.run.smithi088.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:05:29.645 INFO:teuthology.orchestra.run.smithi088.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T20:05:29.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:29.645 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec bash -c ' 2023-10-16T20:05:29.646 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2023-10-16T20:05:29.646 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T20:05:29.646 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T20:05:29.647 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T20:05:29.647 DEBUG:teuthology.orchestra.run.smithi088:> ') 2023-10-16T20:05:29.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:29.735 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2023-10-16T20:05:29.745 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T20:05:29.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:29.746 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list-id 2023-10-16T20:05:29.796 INFO:teuthology.orchestra.run.smithi088.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-16T20:05:29.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:29.798 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec bash -c ' 2023-10-16T20:05:29.799 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2023-10-16T20:05:29.799 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:05:29.799 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-16T20:05:29.800 DEBUG:teuthology.orchestra.run.smithi088:> ') 2023-10-16T20:05:29.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:29.880 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-16T20:05:29.942 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T20:05:29.942 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T20:05:29.942 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T20:05:29.942 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:29.943 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2023-10-16T20:05:29.943 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:29.943 INFO:teuthology.orchestra.run.smithi088.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T20:05:29.943 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether da:d3:1e:e5:00:53 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-16T20:05:29.944 INFO:teuthology.orchestra.run.smithi088.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-16T20:05:29.944 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:29.944 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::d8d3:1eff:fee5:53/64 scope link tentative 2023-10-16T20:05:29.944 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:29.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:29.946 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-16T20:05:30.010 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T20:05:30.010 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T20:05:30.010 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T20:05:30.011 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:30.011 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2023-10-16T20:05:30.011 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:30.011 INFO:teuthology.orchestra.run.smithi088.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T20:05:30.012 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether da:d3:1e:e5:00:53 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-16T20:05:30.012 INFO:teuthology.orchestra.run.smithi088.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-16T20:05:30.012 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:30.012 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::d8d3:1eff:fee5:53/64 scope link tentative 2023-10-16T20:05:30.013 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:30.014 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-16T20:05:30.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:30.015 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec bash -c ' 2023-10-16T20:05:30.015 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2023-10-16T20:05:30.015 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-16T20:05:30.016 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-16T20:05:30.016 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-16T20:05:30.016 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-16T20:05:30.016 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-16T20:05:30.017 DEBUG:teuthology.orchestra.run.smithi088:> ') 2023-10-16T20:05:30.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:30.153 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec bash -c ' 2023-10-16T20:05:30.154 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2023-10-16T20:05:30.154 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set brx.1 up 2023-10-16T20:05:30.154 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set dev brx.1 master ceph-brx 2023-10-16T20:05:30.155 DEBUG:teuthology.orchestra.run.smithi088:> ') 2023-10-16T20:05:30.194 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-10-16T20:05:30.195 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T20:05:30.195 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-16T20:05:30.245 INFO:teuthology.orchestra.run.smithi088.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-16T20:05:30.245 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T20:05:30.246 DEBUG:teuthology.orchestra.run.smithi088:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-16T20:05:30.297 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2023-10-16T20:05:30.362 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T20:05:30.415 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2023-10-16T20:05:30.415 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2023-10-16T20:05:30.415 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2023-10-16T20:05:30.415 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2023-10-16T20:05:30.416 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2023-10-16T20:05:30.416 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2023-10-16T20:05:30.416 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2023-10-16T20:05:30.416 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2023-10-16T20:05:30.416 INFO:teuthology.orchestra.run.smithi088.stdout:/run/lock 2023-10-16T20:05:30.417 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2023-10-16T20:05:30.417 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2023-10-16T20:05:30.417 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2023-10-16T20:05:30.417 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T20:05:30.418 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2023-10-16T20:05:30.418 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2023-10-16T20:05:30.418 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2023-10-16T20:05:30.418 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2023-10-16T20:05:30.419 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/fs/nfsd 2023-10-16T20:05:30.419 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2023-10-16T20:05:30.420 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2023-10-16T20:05:30.420 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T20:05:30.420 INFO:teuthology.orchestra.run.smithi088.stdout:/run/rpc_pipefs 2023-10-16T20:05:30.420 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2023-10-16T20:05:30.420 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2023-10-16T20:05:30.421 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T20:05:30.421 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T20:05:30.421 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T20:05:30.421 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T20:05:30.421 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2023-10-16T20:05:30.422 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:05:30.422 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:05:30.422 INFO:teuthology.orchestra.run.smithi088.stdout:/home/ubuntu/cephtest/mnt.0 2023-10-16T20:05:30.422 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:05:30.422 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:05:30.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:30.424 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2023-10-16T20:05:30.477 INFO:teuthology.orchestra.run.smithi088.stdout:43 2023-10-16T20:05:30.478 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2023-10-16T20:05:30.478 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-10-16T20:05:30.528 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2023-10-16T20:05:30.550 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T20:05:30.607 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2023-10-16T20:05:30.607 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2023-10-16T20:05:30.607 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2023-10-16T20:05:30.607 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2023-10-16T20:05:30.608 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2023-10-16T20:05:30.608 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2023-10-16T20:05:30.608 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2023-10-16T20:05:30.608 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2023-10-16T20:05:30.608 INFO:teuthology.orchestra.run.smithi088.stdout:/run/lock 2023-10-16T20:05:30.609 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2023-10-16T20:05:30.609 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2023-10-16T20:05:30.609 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2023-10-16T20:05:30.609 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T20:05:30.610 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2023-10-16T20:05:30.610 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2023-10-16T20:05:30.610 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2023-10-16T20:05:30.610 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2023-10-16T20:05:30.610 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/fs/nfsd 2023-10-16T20:05:30.611 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2023-10-16T20:05:30.611 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2023-10-16T20:05:30.611 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T20:05:30.611 INFO:teuthology.orchestra.run.smithi088.stdout:/run/rpc_pipefs 2023-10-16T20:05:30.611 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2023-10-16T20:05:30.612 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2023-10-16T20:05:30.612 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T20:05:30.612 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T20:05:30.612 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T20:05:30.612 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T20:05:30.613 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2023-10-16T20:05:30.613 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:05:30.613 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:05:30.613 INFO:teuthology.orchestra.run.smithi088.stdout:/home/ubuntu/cephtest/mnt.0 2023-10-16T20:05:30.613 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:05:30.614 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:05:30.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:30.615 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2023-10-16T20:05:30.622 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi088.stderr:2023-10-16T20:05:30.622+0000 7f03e3b15580 -1 init, newargv = 0x5606ca7a4c60 newargc=15 2023-10-16T20:05:30.623 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi088.stderr:ceph-fuse[46037]: starting ceph client 2023-10-16T20:05:30.629 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi088.stderr:ceph-fuse[46037]: starting fuse 2023-10-16T20:05:30.631 INFO:teuthology.orchestra.run.smithi088.stdout:43 2023-10-16T20:05:30.631 INFO:teuthology.orchestra.run.smithi088.stdout:44 2023-10-16T20:05:30.631 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2023-10-16T20:05:30.632 DEBUG:teuthology.orchestra.run.smithi088:> sudo stdin-killer -- python3 -c ' 2023-10-16T20:05:30.632 DEBUG:teuthology.orchestra.run.smithi088:> import glob 2023-10-16T20:05:30.632 DEBUG:teuthology.orchestra.run.smithi088:> import re 2023-10-16T20:05:30.633 DEBUG:teuthology.orchestra.run.smithi088:> import os 2023-10-16T20:05:30.633 DEBUG:teuthology.orchestra.run.smithi088:> import subprocess 2023-10-16T20:05:30.633 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:05:30.633 DEBUG:teuthology.orchestra.run.smithi088:> def _find_admin_socket(client_name): 2023-10-16T20:05:30.634 DEBUG:teuthology.orchestra.run.smithi088:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-10-16T20:05:30.634 DEBUG:teuthology.orchestra.run.smithi088:> files = glob.glob(asok_path) 2023-10-16T20:05:30.634 DEBUG:teuthology.orchestra.run.smithi088:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-10-16T20:05:30.635 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:05:30.635 DEBUG:teuthology.orchestra.run.smithi088:> # Given a non-glob path, it better be there 2023-10-16T20:05:30.635 DEBUG:teuthology.orchestra.run.smithi088:> if "*" not in asok_path: 2023-10-16T20:05:30.635 DEBUG:teuthology.orchestra.run.smithi088:> assert(len(files) == 1) 2023-10-16T20:05:30.636 DEBUG:teuthology.orchestra.run.smithi088:> return files[0] 2023-10-16T20:05:30.636 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:05:30.636 DEBUG:teuthology.orchestra.run.smithi088:> for f in files: 2023-10-16T20:05:30.637 DEBUG:teuthology.orchestra.run.smithi088:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-16T20:05:30.637 DEBUG:teuthology.orchestra.run.smithi088:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-16T20:05:30.637 DEBUG:teuthology.orchestra.run.smithi088:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-16T20:05:30.637 DEBUG:teuthology.orchestra.run.smithi088:> contents = proc_f.read() 2023-10-16T20:05:30.638 DEBUG:teuthology.orchestra.run.smithi088:> if mountpoint in contents: 2023-10-16T20:05:30.638 DEBUG:teuthology.orchestra.run.smithi088:> return f 2023-10-16T20:05:30.638 DEBUG:teuthology.orchestra.run.smithi088:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-16T20:05:30.639 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:05:30.639 DEBUG:teuthology.orchestra.run.smithi088:> print(_find_admin_socket("client.2")) 2023-10-16T20:05:30.639 DEBUG:teuthology.orchestra.run.smithi088:> ' 2023-10-16T20:05:30.756 INFO:teuthology.orchestra.run.smithi088.stdout:/var/run/ceph/ceph-client.2.46037.asok 2023-10-16T20:05:30.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-16T20:05:30.767 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.46037.asok 2023-10-16T20:05:30.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:30.768 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.46037.asok status 2023-10-16T20:05:30.940 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2023-10-16T20:05:30.941 INFO:teuthology.orchestra.run.smithi088.stdout: "metadata": { 2023-10-16T20:05:30.941 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_sha1": "4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0", 2023-10-16T20:05:30.941 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_version": "ceph version 18.0.0-6730-g4ba8a190 (4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0) reef (dev)", 2023-10-16T20:05:30.941 INFO:teuthology.orchestra.run.smithi088.stdout: "entity_id": "2", 2023-10-16T20:05:30.942 INFO:teuthology.orchestra.run.smithi088.stdout: "hostname": "smithi088", 2023-10-16T20:05:30.942 INFO:teuthology.orchestra.run.smithi088.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-10-16T20:05:30.942 INFO:teuthology.orchestra.run.smithi088.stdout: "pid": "46037", 2023-10-16T20:05:30.942 INFO:teuthology.orchestra.run.smithi088.stdout: "root": "/" 2023-10-16T20:05:30.942 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:05:30.943 INFO:teuthology.orchestra.run.smithi088.stdout: "dentry_count": 0, 2023-10-16T20:05:30.943 INFO:teuthology.orchestra.run.smithi088.stdout: "dentry_pinned_count": 0, 2023-10-16T20:05:30.943 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 4866, 2023-10-16T20:05:30.943 INFO:teuthology.orchestra.run.smithi088.stdout: "inst": { 2023-10-16T20:05:30.943 INFO:teuthology.orchestra.run.smithi088.stdout: "name": { 2023-10-16T20:05:30.944 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "client", 2023-10-16T20:05:30.944 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 4866 2023-10-16T20:05:30.944 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:05:30.944 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": { 2023-10-16T20:05:30.944 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2023-10-16T20:05:30.944 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "192.168.0.2:0", 2023-10-16T20:05:30.945 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 1749317561 2023-10-16T20:05:30.945 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:05:30.945 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:05:30.946 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": { 2023-10-16T20:05:30.946 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2023-10-16T20:05:30.946 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "192.168.0.2:0", 2023-10-16T20:05:30.946 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 1749317561 2023-10-16T20:05:30.946 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:05:30.947 INFO:teuthology.orchestra.run.smithi088.stdout: "inst_str": "client.4866 192.168.0.2:0/1749317561", 2023-10-16T20:05:30.947 INFO:teuthology.orchestra.run.smithi088.stdout: "addr_str": "192.168.0.2:0/1749317561", 2023-10-16T20:05:30.947 INFO:teuthology.orchestra.run.smithi088.stdout: "inode_count": 1, 2023-10-16T20:05:30.947 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_epoch": 9, 2023-10-16T20:05:30.947 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_epoch": 27, 2023-10-16T20:05:30.948 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_epoch_barrier": 0, 2023-10-16T20:05:30.948 INFO:teuthology.orchestra.run.smithi088.stdout: "blocklisted": false, 2023-10-16T20:05:30.948 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "cephfs" 2023-10-16T20:05:30.948 INFO:teuthology.orchestra.run.smithi088.stdout:} 2023-10-16T20:05:30.955 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T20:05:31.093 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.090+0000 7f0a13009640 1 Processor -- start 2023-10-16T20:05:31.093 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.090+0000 7f0a13009640 1 -- start start 2023-10-16T20:05:31.096 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a13009640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a0c15cb80 0x7f0a0c0b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:31.096 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a13009640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0a0c0b28b0 0x7f0a0c0ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:31.097 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a13009640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a0c0b2cf0 con 0x7f0a0c0cea00 2023-10-16T20:05:31.097 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a13009640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a0c06b490 con 0x7f0a0c0b28b0 2023-10-16T20:05:31.097 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a13009640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c0cf180 con 0x7f0a0c15cb80 2023-10-16T20:05:31.097 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a115a6640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f0a0c0cea00 0x7f0a0c06ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:60842/0 (socket says 172.21.15.88:60842) 2023-10-16T20:05:31.097 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a115a6640 1 -- 172.21.15.88:0/2333752563 learned_addr learned my addr 172.21.15.88:0/2333752563 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:31.098 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a0bfff640 1 --2- 172.21.15.88:0/2333752563 >> v2:172.21.15.94:3300/0 conn(0x7f0a0c0b28b0 0x7f0a0c0ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:31.098 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a10da5640 1 --2- 172.21.15.88:0/2333752563 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a0c15cb80 0x7f0a0c0b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:31.098 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a0b7fe640 1 -- 172.21.15.88:0/2333752563 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1131430684 0 0) 0x7f0a0c0b2cf0 con 0x7f0a0c0cea00 2023-10-16T20:05:31.098 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a0b7fe640 1 -- 172.21.15.88:0/2333752563 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09fc003530 con 0x7f0a0c0cea00 2023-10-16T20:05:31.098 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a0b7fe640 1 -- 172.21.15.88:0/2333752563 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2818737788 0 0) 0x7f09fc003530 con 0x7f0a0c0cea00 2023-10-16T20:05:31.099 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a0b7fe640 1 -- 172.21.15.88:0/2333752563 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a0c15cb80 msgr2=0x7f0a0c0b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:31.099 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a0b7fe640 1 --2- 172.21.15.88:0/2333752563 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a0c15cb80 0x7f0a0c0b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:31.099 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a0b7fe640 1 -- 172.21.15.88:0/2333752563 >> v2:172.21.15.94:3300/0 conn(0x7f0a0c0b28b0 msgr2=0x7f0a0c0ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:31.099 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a0b7fe640 1 --2- 172.21.15.88:0/2333752563 >> v2:172.21.15.94:3300/0 conn(0x7f0a0c0b28b0 0x7f0a0c0ce4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:31.099 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a0b7fe640 1 -- 172.21.15.88:0/2333752563 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0c06b700 con 0x7f0a0c0cea00 2023-10-16T20:05:31.100 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a0b7fe640 1 -- 172.21.15.88:0/2333752563 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f0a04001a00 con 0x7f0a0c0cea00 2023-10-16T20:05:31.100 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a0bfff640 1 --2- 172.21.15.88:0/2333752563 >> v2:172.21.15.94:3300/0 conn(0x7f0a0c0b28b0 0x7f0a0c0ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:05:31.100 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a0b7fe640 1 -- 172.21.15.88:0/2333752563 >> v1:172.21.15.88:6789/0 conn(0x7f0a0c0cea00 legacy=0x7f0a0c06ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:31.100 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a0b7fe640 1 --2- 172.21.15.88:0/2333752563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09fc003980 0x7f09fc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:31.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a0b7fe640 1 --2- 172.21.15.88:0/2333752563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09fc0047d0 0x7f09fc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:31.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a0b7fe640 1 --2- 172.21.15.88:0/2333752563 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a0c15cb80 0x7f09fc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:31.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a0b7fe640 1 -- 172.21.15.88:0/2333752563 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c06b490 con 0x7f09fc003980 2023-10-16T20:05:31.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a0b7fe640 1 -- 172.21.15.88:0/2333752563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c0cf180 con 0x7f09fc0047d0 2023-10-16T20:05:31.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a0b7fe640 1 -- 172.21.15.88:0/2333752563 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc00b990 con 0x7f0a0c15cb80 2023-10-16T20:05:31.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a10da5640 1 --2- 172.21.15.88:0/2333752563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09fc003980 0x7f09fc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:31.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a0bfff640 1 --2- 172.21.15.88:0/2333752563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09fc0047d0 0x7f09fc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:31.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a115a6640 1 --2- 172.21.15.88:0/2333752563 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a0c15cb80 0x7f09fc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:31.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a10da5640 1 -- 172.21.15.88:0/2333752563 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a0c15cb80 msgr2=0x7f09fc005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:31.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a10da5640 1 --2- 172.21.15.88:0/2333752563 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a0c15cb80 0x7f09fc005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:31.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a10da5640 1 -- 172.21.15.88:0/2333752563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09fc0047d0 msgr2=0x7f09fc004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:31.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a10da5640 1 --2- 172.21.15.88:0/2333752563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09fc0047d0 0x7f09fc004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:31.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a10da5640 1 -- 172.21.15.88:0/2333752563 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09fc003530 con 0x7f09fc003980 2023-10-16T20:05:31.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.094+0000 7f0a115a6640 1 --2- 172.21.15.88:0/2333752563 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a0c15cb80 0x7f09fc005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:31.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.098+0000 7f0a10da5640 1 --2- 172.21.15.88:0/2333752563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09fc003980 0x7f09fc003da0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0a00009ac0 tx=0x7f0a0001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d1d8448853e53b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:31.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.098+0000 7f0a0b7fe640 1 -- 172.21.15.88:0/2333752563 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a000a6410 con 0x7f09fc003980 2023-10-16T20:05:31.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.098+0000 7f0a0b7fe640 1 -- 172.21.15.88:0/2333752563 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a000a6570 con 0x7f09fc003980 2023-10-16T20:05:31.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.098+0000 7f0a0b7fe640 1 -- 172.21.15.88:0/2333752563 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a000a6900 con 0x7f09fc003980 2023-10-16T20:05:31.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.098+0000 7f0a13009640 1 -- 172.21.15.88:0/2333752563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09fc003980 msgr2=0x7f09fc003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:31.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.098+0000 7f0a13009640 1 --2- 172.21.15.88:0/2333752563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09fc003980 0x7f09fc003da0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0a00009ac0 tx=0x7f0a0001ca00 comp rx=0 tx=0).stop 2023-10-16T20:05:31.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.098+0000 7f0a115a6640 1 -- 172.21.15.88:0/2333752563 reap_dead start 2023-10-16T20:05:31.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.098+0000 7f0a13009640 1 -- 172.21.15.88:0/2333752563 shutdown_connections 2023-10-16T20:05:31.108 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.098+0000 7f0a13009640 1 -- 172.21.15.88:0/2333752563 >> 172.21.15.88:0/2333752563 conn(0x7f0a0c0cd370 msgr2=0x7f0a0c0cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:31.108 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.098+0000 7f0a13009640 1 -- 172.21.15.88:0/2333752563 shutdown_connections 2023-10-16T20:05:31.108 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.098+0000 7f0a13009640 1 -- 172.21.15.88:0/2333752563 wait complete. 2023-10-16T20:05:31.108 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.102+0000 7f0a13009640 1 Processor -- start 2023-10-16T20:05:31.108 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.102+0000 7f0a13009640 1 -- start start 2023-10-16T20:05:31.109 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.102+0000 7f0a13009640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f09fc003980 0x7f0a0c15acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:31.109 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.102+0000 7f0a13009640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a0c1f5410 0x7f0a0c0b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:31.109 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.102+0000 7f0a13009640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a0c0b4450 0x7f0a0c0c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:31.109 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.102+0000 7f0a13009640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c15b230 con 0x7f0a0c0b4450 2023-10-16T20:05:31.109 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.102+0000 7f0a13009640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c0c9930 con 0x7f0a0c1f5410 2023-10-16T20:05:31.109 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.102+0000 7f0a13009640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c0b4960 con 0x7f09fc003980 2023-10-16T20:05:31.110 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.102+0000 7f0a115a6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a0c0b4450 0x7f0a0c0c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:31.110 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.102+0000 7f0a115a6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a0c0b4450 0x7f0a0c0c58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38300/0 (socket says 172.21.15.88:38300) 2023-10-16T20:05:31.110 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.102+0000 7f0a115a6640 1 -- 172.21.15.88:0/4246522290 learned_addr learned my addr 172.21.15.88:0/4246522290 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:31.110 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.102+0000 7f0a10da5640 1 --2- 172.21.15.88:0/4246522290 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f09fc003980 0x7f0a0c15acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:31.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.102+0000 7f0a0bfff640 1 --2- 172.21.15.88:0/4246522290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a0c1f5410 0x7f0a0c0b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:31.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.102+0000 7f0a115a6640 1 -- 172.21.15.88:0/4246522290 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f09fc003980 msgr2=0x7f0a0c15acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:31.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.102+0000 7f0a115a6640 1 --2- 172.21.15.88:0/4246522290 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f09fc003980 0x7f0a0c15acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:31.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.102+0000 7f0a115a6640 1 -- 172.21.15.88:0/4246522290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a0c1f5410 msgr2=0x7f0a0c0b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:31.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.102+0000 7f0a115a6640 1 --2- 172.21.15.88:0/4246522290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a0c1f5410 0x7f0a0c0b3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:31.112 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.102+0000 7f0a115a6640 1 -- 172.21.15.88:0/4246522290 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0c0c9000 con 0x7f0a0c0b4450 2023-10-16T20:05:31.112 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.102+0000 7f0a115a6640 1 --2- 172.21.15.88:0/4246522290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a0c0b4450 0x7f0a0c0c58e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0c43c0 tx=0x7f0a0400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:31.112 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.102+0000 7f0a097fa640 1 -- 172.21.15.88:0/4246522290 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a04017070 con 0x7f0a0c0b4450 2023-10-16T20:05:31.112 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.102+0000 7f0a097fa640 1 -- 172.21.15.88:0/4246522290 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0400eb30 con 0x7f0a0c0b4450 2023-10-16T20:05:31.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.102+0000 7f0a13009640 1 -- 172.21.15.88:0/4246522290 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a0c0c92e0 con 0x7f0a0c0b4450 2023-10-16T20:05:31.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.102+0000 7f0a097fa640 1 -- 172.21.15.88:0/4246522290 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0400ecc0 con 0x7f0a0c0b4450 2023-10-16T20:05:31.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.102+0000 7f0a13009640 1 -- 172.21.15.88:0/4246522290 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a0c0b6060 con 0x7f0a0c0b4450 2023-10-16T20:05:31.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.102+0000 7f0a13009640 1 -- 172.21.15.88:0/4246522290 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a0c04e9c0 con 0x7f0a0c0b4450 2023-10-16T20:05:31.115 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.106+0000 7f0a097fa640 1 -- 172.21.15.88:0/4246522290 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0a04013420 con 0x7f0a0c0b4450 2023-10-16T20:05:31.115 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.106+0000 7f0a097fa640 1 --2- 172.21.15.88:0/4246522290 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f09f00664d0 0x7f09f0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:31.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.106+0000 7f0a097fa640 1 -- 172.21.15.88:0/4246522290 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0a04083b00 con 0x7f0a0c0b4450 2023-10-16T20:05:31.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.106+0000 7f0a10da5640 1 --2- 172.21.15.88:0/4246522290 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f09f00664d0 0x7f09f0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:31.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.106+0000 7f0a10da5640 1 --2- 172.21.15.88:0/4246522290 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f09f00664d0 0x7f09f0068990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0a000ad000 tx=0x7f0a000b2040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:31.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.110+0000 7f0a097fa640 1 -- 172.21.15.88:0/4246522290 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0a04050dd0 con 0x7f0a0c0b4450 2023-10-16T20:05:31.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.294+0000 7f0a13009640 1 -- 172.21.15.88:0/4246522290 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0a0c0c9a70 con 0x7f0a0c0b4450 2023-10-16T20:05:31.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.294+0000 7f0a097fa640 1 -- 172.21.15.88:0/4246522290 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f0a040543f0 con 0x7f0a0c0b4450 2023-10-16T20:05:31.295 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-16T20:05:31.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.294+0000 7f0a13009640 1 -- 172.21.15.88:0/4246522290 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f09f00664d0 msgr2=0x7f09f0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:31.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.294+0000 7f0a13009640 1 --2- 172.21.15.88:0/4246522290 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f09f00664d0 0x7f09f0068990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0a000ad000 tx=0x7f0a000b2040 comp rx=0 tx=0).stop 2023-10-16T20:05:31.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.294+0000 7f0a13009640 1 -- 172.21.15.88:0/4246522290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a0c0b4450 msgr2=0x7f0a0c0c58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:31.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.294+0000 7f0a13009640 1 --2- 172.21.15.88:0/4246522290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a0c0b4450 0x7f0a0c0c58e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0c43c0 tx=0x7f0a0400e530 comp rx=0 tx=0).stop 2023-10-16T20:05:31.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.294+0000 7f0a13009640 1 -- 172.21.15.88:0/4246522290 shutdown_connections 2023-10-16T20:05:31.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.294+0000 7f0a13009640 1 --2- 172.21.15.88:0/4246522290 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f09f00664d0 0x7f09f0068990 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:31.298 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.294+0000 7f0a13009640 1 --2- 172.21.15.88:0/4246522290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a0c0b4450 0x7f0a0c0c58e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:31.298 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.294+0000 7f0a13009640 1 --2- 172.21.15.88:0/4246522290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a0c1f5410 0x7f0a0c0b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:31.298 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.294+0000 7f0a13009640 1 --2- 172.21.15.88:0/4246522290 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f09fc003980 0x7f0a0c15acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:31.298 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.294+0000 7f0a13009640 1 -- 172.21.15.88:0/4246522290 >> 172.21.15.88:0/4246522290 conn(0x7f0a0c0cd370 msgr2=0x7f0a0c0cc030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:31.298 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.294+0000 7f0a13009640 1 -- 172.21.15.88:0/4246522290 shutdown_connections 2023-10-16T20:05:31.299 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:31.294+0000 7f0a13009640 1 -- 172.21.15.88:0/4246522290 wait complete. 2023-10-16T20:05:31.315 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T20:05:31.315 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T20:05:31.316 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2023-10-16T20:05:31.316 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-16T20:05:31.316 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-16T20:05:31.316 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T20:05:31.316 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T20:05:31.317 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T20:05:31.317 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-16T20:05:31.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:31.317 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2023-10-16T20:05:31.328 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T20:05:31.328 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T20:05:31.328 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T20:05:31.328 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:31.328 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2023-10-16T20:05:31.329 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:31.329 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:05:31.329 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2023-10-16T20:05:31.329 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2023-10-16T20:05:31.330 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:05:31.330 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2023-10-16T20:05:31.330 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2023-10-16T20:05:31.330 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:05:31.330 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2023-10-16T20:05:31.331 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T20:05:31.331 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2023-10-16T20:05:31.331 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-16T20:05:31.331 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42377sec preferred_lft 42377sec 2023-10-16T20:05:31.331 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2023-10-16T20:05:31.331 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:31.332 INFO:teuthology.orchestra.run.smithi094.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-16T20:05:31.332 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 02:42:0f:b3:8e:fc brd ff:ff:ff:ff:ff:ff 2023-10-16T20:05:31.332 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-16T20:05:31.332 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:31.333 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-16T20:05:31.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:31.334 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec bash -c ' 2023-10-16T20:05:31.334 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2023-10-16T20:05:31.334 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add name ceph-brx type bridge 2023-10-16T20:05:31.334 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr flush dev ceph-brx 2023-10-16T20:05:31.335 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set ceph-brx up 2023-10-16T20:05:31.335 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-16T20:05:31.335 DEBUG:teuthology.orchestra.run.smithi094:> ') 2023-10-16T20:05:31.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:31.453 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T20:05:31.473 INFO:teuthology.orchestra.run.smithi094.stdout:1 2023-10-16T20:05:31.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:31.475 DEBUG:teuthology.orchestra.run.smithi094:> route 2023-10-16T20:05:31.553 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2023-10-16T20:05:31.553 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T20:05:31.553 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-16T20:05:31.553 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T20:05:31.553 INFO:teuthology.orchestra.run.smithi094.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-16T20:05:31.554 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T20:05:31.554 INFO:teuthology.orchestra.run.smithi094.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:05:31.554 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:05:31.554 INFO:teuthology.orchestra.run.smithi094.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:05:31.554 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T20:05:31.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:31.555 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec bash -c ' 2023-10-16T20:05:31.555 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2023-10-16T20:05:31.556 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T20:05:31.556 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T20:05:31.556 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T20:05:31.557 DEBUG:teuthology.orchestra.run.smithi094:> ') 2023-10-16T20:05:31.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:31.651 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2023-10-16T20:05:31.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:31.661 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list-id 2023-10-16T20:05:31.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:31.712 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec bash -c ' 2023-10-16T20:05:31.713 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2023-10-16T20:05:31.713 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T20:05:31.713 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-16T20:05:31.713 DEBUG:teuthology.orchestra.run.smithi094:> ') 2023-10-16T20:05:31.796 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-16T20:05:31.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:31.797 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec bash -c ' 2023-10-16T20:05:31.797 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2023-10-16T20:05:31.797 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-16T20:05:31.797 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-16T20:05:31.798 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-16T20:05:31.798 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-16T20:05:31.798 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-16T20:05:31.799 DEBUG:teuthology.orchestra.run.smithi094:> ') 2023-10-16T20:05:31.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:31.966 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec bash -c ' 2023-10-16T20:05:31.967 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2023-10-16T20:05:31.967 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.0 up 2023-10-16T20:05:31.967 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set dev brx.0 master ceph-brx 2023-10-16T20:05:31.968 DEBUG:teuthology.orchestra.run.smithi094:> ') 2023-10-16T20:05:32.008 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-16T20:05:32.009 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T20:05:32.009 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-16T20:05:32.064 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-16T20:05:32.064 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T20:05:32.064 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-16T20:05:32.115 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2023-10-16T20:05:32.181 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T20:05:32.233 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2023-10-16T20:05:32.233 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2023-10-16T20:05:32.234 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2023-10-16T20:05:32.234 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2023-10-16T20:05:32.234 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2023-10-16T20:05:32.234 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2023-10-16T20:05:32.234 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2023-10-16T20:05:32.235 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2023-10-16T20:05:32.235 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2023-10-16T20:05:32.235 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2023-10-16T20:05:32.235 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2023-10-16T20:05:32.235 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2023-10-16T20:05:32.235 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T20:05:32.236 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2023-10-16T20:05:32.236 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2023-10-16T20:05:32.236 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2023-10-16T20:05:32.236 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2023-10-16T20:05:32.236 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2023-10-16T20:05:32.237 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2023-10-16T20:05:32.237 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2023-10-16T20:05:32.237 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T20:05:32.237 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2023-10-16T20:05:32.237 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2023-10-16T20:05:32.237 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2023-10-16T20:05:32.238 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-4 2023-10-16T20:05:32.238 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-5 2023-10-16T20:05:32.238 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-6 2023-10-16T20:05:32.238 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-7 2023-10-16T20:05:32.239 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2023-10-16T20:05:32.239 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T20:05:32.239 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T20:05:32.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:32.240 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2023-10-16T20:05:32.296 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-16T20:05:32.296 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-16T20:05:32.342 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2023-10-16T20:05:32.364 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T20:05:32.422 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2023-10-16T20:05:32.422 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2023-10-16T20:05:32.422 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2023-10-16T20:05:32.422 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2023-10-16T20:05:32.423 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2023-10-16T20:05:32.423 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2023-10-16T20:05:32.423 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2023-10-16T20:05:32.423 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2023-10-16T20:05:32.424 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2023-10-16T20:05:32.424 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2023-10-16T20:05:32.424 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2023-10-16T20:05:32.424 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2023-10-16T20:05:32.424 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T20:05:32.424 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2023-10-16T20:05:32.425 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2023-10-16T20:05:32.425 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2023-10-16T20:05:32.425 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2023-10-16T20:05:32.425 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2023-10-16T20:05:32.425 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2023-10-16T20:05:32.426 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2023-10-16T20:05:32.426 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T20:05:32.426 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2023-10-16T20:05:32.426 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2023-10-16T20:05:32.426 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2023-10-16T20:05:32.427 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-4 2023-10-16T20:05:32.427 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-5 2023-10-16T20:05:32.427 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-6 2023-10-16T20:05:32.427 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-7 2023-10-16T20:05:32.427 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2023-10-16T20:05:32.428 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T20:05:32.428 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T20:05:32.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:32.429 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2023-10-16T20:05:32.447 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi094.stderr:2023-10-16T20:05:32.442+0000 7fecc5644580 -1 init, newargv = 0x556b09622e10 newargc=15 2023-10-16T20:05:32.448 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi094.stderr:ceph-fuse[42669]: starting ceph client 2023-10-16T20:05:32.455 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi094.stderr:ceph-fuse[42669]: starting fuse 2023-10-16T20:05:32.457 INFO:teuthology.orchestra.run.smithi094.stdout:43 2023-10-16T20:05:32.457 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-10-16T20:05:32.458 DEBUG:teuthology.orchestra.run.smithi094:> sudo stdin-killer -- python3 -c ' 2023-10-16T20:05:32.458 DEBUG:teuthology.orchestra.run.smithi094:> import glob 2023-10-16T20:05:32.458 DEBUG:teuthology.orchestra.run.smithi094:> import re 2023-10-16T20:05:32.459 DEBUG:teuthology.orchestra.run.smithi094:> import os 2023-10-16T20:05:32.459 DEBUG:teuthology.orchestra.run.smithi094:> import subprocess 2023-10-16T20:05:32.459 DEBUG:teuthology.orchestra.run.smithi094:> 2023-10-16T20:05:32.460 DEBUG:teuthology.orchestra.run.smithi094:> def _find_admin_socket(client_name): 2023-10-16T20:05:32.460 DEBUG:teuthology.orchestra.run.smithi094:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-16T20:05:32.460 DEBUG:teuthology.orchestra.run.smithi094:> files = glob.glob(asok_path) 2023-10-16T20:05:32.460 DEBUG:teuthology.orchestra.run.smithi094:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-16T20:05:32.461 DEBUG:teuthology.orchestra.run.smithi094:> 2023-10-16T20:05:32.461 DEBUG:teuthology.orchestra.run.smithi094:> # Given a non-glob path, it better be there 2023-10-16T20:05:32.461 DEBUG:teuthology.orchestra.run.smithi094:> if "*" not in asok_path: 2023-10-16T20:05:32.462 DEBUG:teuthology.orchestra.run.smithi094:> assert(len(files) == 1) 2023-10-16T20:05:32.462 DEBUG:teuthology.orchestra.run.smithi094:> return files[0] 2023-10-16T20:05:32.462 DEBUG:teuthology.orchestra.run.smithi094:> 2023-10-16T20:05:32.463 DEBUG:teuthology.orchestra.run.smithi094:> for f in files: 2023-10-16T20:05:32.463 DEBUG:teuthology.orchestra.run.smithi094:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-16T20:05:32.463 DEBUG:teuthology.orchestra.run.smithi094:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-16T20:05:32.463 DEBUG:teuthology.orchestra.run.smithi094:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-16T20:05:32.464 DEBUG:teuthology.orchestra.run.smithi094:> contents = proc_f.read() 2023-10-16T20:05:32.464 DEBUG:teuthology.orchestra.run.smithi094:> if mountpoint in contents: 2023-10-16T20:05:32.464 DEBUG:teuthology.orchestra.run.smithi094:> return f 2023-10-16T20:05:32.465 DEBUG:teuthology.orchestra.run.smithi094:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-16T20:05:32.465 DEBUG:teuthology.orchestra.run.smithi094:> 2023-10-16T20:05:32.465 DEBUG:teuthology.orchestra.run.smithi094:> print(_find_admin_socket("client.1")) 2023-10-16T20:05:32.465 DEBUG:teuthology.orchestra.run.smithi094:> ' 2023-10-16T20:05:32.585 INFO:teuthology.orchestra.run.smithi094.stdout:/var/run/ceph/ceph-client.1.42669.asok 2023-10-16T20:05:32.588 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:05:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-16T20:05:32.596 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.42669.asok 2023-10-16T20:05:32.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:32.597 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.42669.asok status 2023-10-16T20:05:32.770 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2023-10-16T20:05:32.771 INFO:teuthology.orchestra.run.smithi094.stdout: "metadata": { 2023-10-16T20:05:32.771 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_sha1": "4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0", 2023-10-16T20:05:32.771 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 18.0.0-6730-g4ba8a190 (4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0) reef (dev)", 2023-10-16T20:05:32.771 INFO:teuthology.orchestra.run.smithi094.stdout: "entity_id": "1", 2023-10-16T20:05:32.772 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": "smithi094", 2023-10-16T20:05:32.772 INFO:teuthology.orchestra.run.smithi094.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-16T20:05:32.772 INFO:teuthology.orchestra.run.smithi094.stdout: "pid": "42669", 2023-10-16T20:05:32.772 INFO:teuthology.orchestra.run.smithi094.stdout: "root": "/" 2023-10-16T20:05:32.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2023-10-16T20:05:32.773 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_count": 0, 2023-10-16T20:05:32.773 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_pinned_count": 0, 2023-10-16T20:05:32.773 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 4780, 2023-10-16T20:05:32.773 INFO:teuthology.orchestra.run.smithi094.stdout: "inst": { 2023-10-16T20:05:32.773 INFO:teuthology.orchestra.run.smithi094.stdout: "name": { 2023-10-16T20:05:32.774 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "client", 2023-10-16T20:05:32.774 INFO:teuthology.orchestra.run.smithi094.stdout: "num": 4780 2023-10-16T20:05:32.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2023-10-16T20:05:32.774 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2023-10-16T20:05:32.774 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2023-10-16T20:05:32.775 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2023-10-16T20:05:32.775 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1062791059 2023-10-16T20:05:32.775 INFO:teuthology.orchestra.run.smithi094.stdout: } 2023-10-16T20:05:32.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2023-10-16T20:05:32.775 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2023-10-16T20:05:32.775 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2023-10-16T20:05:32.776 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2023-10-16T20:05:32.776 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1062791059 2023-10-16T20:05:32.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2023-10-16T20:05:32.776 INFO:teuthology.orchestra.run.smithi094.stdout: "inst_str": "client.4780 192.168.0.1:0/1062791059", 2023-10-16T20:05:32.776 INFO:teuthology.orchestra.run.smithi094.stdout: "addr_str": "192.168.0.1:0/1062791059", 2023-10-16T20:05:32.777 INFO:teuthology.orchestra.run.smithi094.stdout: "inode_count": 1, 2023-10-16T20:05:32.777 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_epoch": 9, 2023-10-16T20:05:32.777 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch": 27, 2023-10-16T20:05:32.777 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch_barrier": 0, 2023-10-16T20:05:32.777 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklisted": false, 2023-10-16T20:05:32.777 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs" 2023-10-16T20:05:32.778 INFO:teuthology.orchestra.run.smithi094.stdout:} 2023-10-16T20:05:32.786 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T20:05:32.932 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.930+0000 7fadf029e640 1 Processor -- start 2023-10-16T20:05:32.933 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.930+0000 7fadf029e640 1 -- start start 2023-10-16T20:05:32.934 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.934+0000 7fadf029e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fade80715c0 0x7fade806ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:32.934 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.934+0000 7fadf029e640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fade806b4e0 0x7fade80c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:32.934 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.934+0000 7fadf029e640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fade80ae720 con 0x7fade806b100 2023-10-16T20:05:32.934 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.934+0000 7fadf029e640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade80c9a10 con 0x7fade806b4e0 2023-10-16T20:05:32.935 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.934+0000 7fadf029e640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade806d650 con 0x7fade80715c0 2023-10-16T20:05:32.935 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.934+0000 7fadee03a640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fade806b100 0x7fade806c370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:54690/0 (socket says 172.21.15.88:54690) 2023-10-16T20:05:32.936 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.934+0000 7fadee03a640 1 -- 172.21.15.88:0/3001992493 learned_addr learned my addr 172.21.15.88:0/3001992493 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:32.936 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.934+0000 7fadee83b640 1 --2- 172.21.15.88:0/3001992493 >> v2:172.21.15.88:3300/0 conn(0x7fade806b4e0 0x7fade80c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:32.936 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.934+0000 7faded839640 1 --2- 172.21.15.88:0/3001992493 >> v2:172.21.15.94:3300/0 conn(0x7fade80715c0 0x7fade806ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:32.936 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.934+0000 7fadee83b640 1 -- 172.21.15.88:0/3001992493 >> v2:172.21.15.94:3300/0 conn(0x7fade80715c0 msgr2=0x7fade806ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:32.936 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.934+0000 7fadee83b640 1 --2- 172.21.15.88:0/3001992493 >> v2:172.21.15.94:3300/0 conn(0x7fade80715c0 0x7fade806ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:32.937 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.934+0000 7fadee83b640 1 -- 172.21.15.88:0/3001992493 >> v1:172.21.15.88:6789/0 conn(0x7fade806b100 legacy=0x7fade806c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:32.937 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.934+0000 7fadee83b640 1 -- 172.21.15.88:0/3001992493 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade80c9420 con 0x7fade806b4e0 2023-10-16T20:05:32.937 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.934+0000 7faded038640 1 -- 172.21.15.88:0/3001992493 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 6980649 0 0) 0x7fade4001120 con 0x7fade806b100 2023-10-16T20:05:32.938 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.938+0000 7fadee83b640 1 --2- 172.21.15.88:0/3001992493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fade806b4e0 0x7fade80c9ea0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fade80cce10 tx=0x7fade001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43ccd79b6bc9fee server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:32.938 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.938+0000 7faded038640 1 -- 172.21.15.88:0/3001992493 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade00a59d0 con 0x7fade806b4e0 2023-10-16T20:05:32.938 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.938+0000 7faded038640 1 -- 172.21.15.88:0/3001992493 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade00a5b30 con 0x7fade806b4e0 2023-10-16T20:05:32.938 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.938+0000 7faded038640 1 -- 172.21.15.88:0/3001992493 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade00ad440 con 0x7fade806b4e0 2023-10-16T20:05:32.943 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.938+0000 7fadf029e640 1 -- 172.21.15.88:0/3001992493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fade806b4e0 msgr2=0x7fade80c9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:32.943 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.938+0000 7fadf029e640 1 --2- 172.21.15.88:0/3001992493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fade806b4e0 0x7fade80c9ea0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fade80cce10 tx=0x7fade001c910 comp rx=0 tx=0).stop 2023-10-16T20:05:32.943 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.938+0000 7fadf029e640 1 -- 172.21.15.88:0/3001992493 shutdown_connections 2023-10-16T20:05:32.944 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.938+0000 7fadf029e640 1 --2- 172.21.15.88:0/3001992493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fade806b4e0 0x7fade80c9ea0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:32.944 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.938+0000 7fadf029e640 1 --2- 172.21.15.88:0/3001992493 >> v2:172.21.15.94:3300/0 conn(0x7fade80715c0 0x7fade806ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:32.944 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.938+0000 7fadf029e640 1 -- 172.21.15.88:0/3001992493 >> 172.21.15.88:0/3001992493 conn(0x7fade8070aa0 msgr2=0x7fade8070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:32.944 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.938+0000 7fadf029e640 1 -- 172.21.15.88:0/3001992493 shutdown_connections 2023-10-16T20:05:32.944 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.938+0000 7fadf029e640 1 -- 172.21.15.88:0/3001992493 wait complete. 2023-10-16T20:05:32.944 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.938+0000 7fadf029e640 1 Processor -- start 2023-10-16T20:05:32.945 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.938+0000 7fadf029e640 1 -- start start 2023-10-16T20:05:32.945 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.942+0000 7fadf029e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fade806b100 0x7fade8155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:32.945 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.942+0000 7fadf029e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fade806b4e0 0x7fade80c4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:32.945 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.942+0000 7fadf029e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fade80715c0 0x7fade81525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:32.945 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.942+0000 7fadf029e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade80c4270 con 0x7fade806b4e0 2023-10-16T20:05:32.946 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.942+0000 7fadf029e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade8159c90 con 0x7fade80715c0 2023-10-16T20:05:32.946 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.942+0000 7fadf029e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade80c4510 con 0x7fade806b100 2023-10-16T20:05:32.946 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.942+0000 7faded839640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fade806b4e0 0x7fade80c4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:32.946 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.942+0000 7faded839640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fade806b4e0 0x7fade80c4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46992/0 (socket says 172.21.15.88:46992) 2023-10-16T20:05:32.946 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.942+0000 7faded839640 1 -- 172.21.15.88:0/1351581191 learned_addr learned my addr 172.21.15.88:0/1351581191 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:32.947 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.942+0000 7fadee03a640 1 --2- 172.21.15.88:0/1351581191 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fade806b100 0x7fade8155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:32.947 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.942+0000 7fadee83b640 1 --2- 172.21.15.88:0/1351581191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fade80715c0 0x7fade81525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:32.947 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.942+0000 7faded839640 1 -- 172.21.15.88:0/1351581191 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fade806b100 msgr2=0x7fade8155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:32.947 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.942+0000 7faded839640 1 --2- 172.21.15.88:0/1351581191 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fade806b100 0x7fade8155300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:32.947 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.942+0000 7faded839640 1 -- 172.21.15.88:0/1351581191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fade80715c0 msgr2=0x7fade81525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:32.947 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.942+0000 7faded839640 1 --2- 172.21.15.88:0/1351581191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fade80715c0 0x7fade81525d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:32.948 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.942+0000 7faded839640 1 -- 172.21.15.88:0/1351581191 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade0009a70 con 0x7fade806b4e0 2023-10-16T20:05:32.948 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.942+0000 7faded839640 1 --2- 172.21.15.88:0/1351581191 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fade806b4e0 0x7fade80c4d10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fadd800e870 tx=0x7fadd800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:32.948 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.942+0000 7faddeffd640 1 -- 172.21.15.88:0/1351581191 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd8009510 con 0x7fade806b4e0 2023-10-16T20:05:32.948 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.942+0000 7faddeffd640 1 -- 172.21.15.88:0/1351581191 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd800eec0 con 0x7fade806b4e0 2023-10-16T20:05:32.948 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.942+0000 7faddeffd640 1 -- 172.21.15.88:0/1351581191 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd800cb40 con 0x7fade806b4e0 2023-10-16T20:05:32.949 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.942+0000 7fadf029e640 1 -- 172.21.15.88:0/1351581191 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade80c47f0 con 0x7fade806b4e0 2023-10-16T20:05:32.949 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.942+0000 7fadf029e640 1 -- 172.21.15.88:0/1351581191 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade8152b10 con 0x7fade806b4e0 2023-10-16T20:05:32.950 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.942+0000 7fadf029e640 1 -- 172.21.15.88:0/1351581191 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fade804e9c0 con 0x7fade806b4e0 2023-10-16T20:05:32.950 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.942+0000 7faddeffd640 1 -- 172.21.15.88:0/1351581191 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fadd800cca0 con 0x7fade806b4e0 2023-10-16T20:05:32.950 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.946+0000 7faddeffd640 1 --2- 172.21.15.88:0/1351581191 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fadcc0664a0 0x7fadcc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:32.951 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.946+0000 7faddeffd640 1 -- 172.21.15.88:0/1351581191 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fadd8084290 con 0x7fade806b4e0 2023-10-16T20:05:32.951 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.946+0000 7fadee03a640 1 --2- 172.21.15.88:0/1351581191 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fadcc0664a0 0x7fadcc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:32.951 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.946+0000 7fadee03a640 1 --2- 172.21.15.88:0/1351581191 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fadcc0664a0 0x7fadcc068960 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fade4000f70 tx=0x7fade4007ba0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:32.951 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:32.950+0000 7faddeffd640 1 -- 172.21.15.88:0/1351581191 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fadd8051430 con 0x7fade806b4e0 2023-10-16T20:05:33.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:33.134+0000 7fadf029e640 1 -- 172.21.15.88:0/1351581191 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fade8065e70 con 0x7fade806b4e0 2023-10-16T20:05:33.134 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:33.134+0000 7faddeffd640 1 -- 172.21.15.88:0/1351581191 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fadd8054a50 con 0x7fade806b4e0 2023-10-16T20:05:33.134 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-16T20:05:33.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:33.134+0000 7fadf029e640 1 -- 172.21.15.88:0/1351581191 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fadcc0664a0 msgr2=0x7fadcc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:33.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:33.134+0000 7fadf029e640 1 --2- 172.21.15.88:0/1351581191 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fadcc0664a0 0x7fadcc068960 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fade4000f70 tx=0x7fade4007ba0 comp rx=0 tx=0).stop 2023-10-16T20:05:33.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:33.134+0000 7fadf029e640 1 -- 172.21.15.88:0/1351581191 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fade806b4e0 msgr2=0x7fade80c4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:33.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:33.134+0000 7fadf029e640 1 --2- 172.21.15.88:0/1351581191 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fade806b4e0 0x7fade80c4d10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fadd800e870 tx=0x7fadd800ec70 comp rx=0 tx=0).stop 2023-10-16T20:05:33.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:33.134+0000 7fadf029e640 1 -- 172.21.15.88:0/1351581191 shutdown_connections 2023-10-16T20:05:33.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:33.134+0000 7fadf029e640 1 --2- 172.21.15.88:0/1351581191 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fadcc0664a0 0x7fadcc068960 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:33.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:33.134+0000 7fadf029e640 1 --2- 172.21.15.88:0/1351581191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fade80715c0 0x7fade81525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:33.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:33.134+0000 7fadf029e640 1 --2- 172.21.15.88:0/1351581191 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fade806b4e0 0x7fade80c4d10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:33.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:33.134+0000 7fadf029e640 1 --2- 172.21.15.88:0/1351581191 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fade806b100 0x7fade8155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:33.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:33.134+0000 7fadf029e640 1 -- 172.21.15.88:0/1351581191 >> 172.21.15.88:0/1351581191 conn(0x7fade8070aa0 msgr2=0x7fade80cd230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:33.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:33.134+0000 7fadf029e640 1 -- 172.21.15.88:0/1351581191 shutdown_connections 2023-10-16T20:05:33.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:33.134+0000 7fadf029e640 1 -- 172.21.15.88:0/1351581191 wait complete. 2023-10-16T20:05:33.154 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T20:05:33.155 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T20:05:33.155 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2023-10-16T20:05:33.155 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-16T20:05:33.155 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-16T20:05:33.156 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T20:05:33.156 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T20:05:33.156 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T20:05:33.156 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-16T20:05:33.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:33.157 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2023-10-16T20:05:33.166 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T20:05:33.167 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T20:05:33.167 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T20:05:33.167 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:33.167 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2023-10-16T20:05:33.167 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:33.168 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:05:33.168 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2023-10-16T20:05:33.168 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2023-10-16T20:05:33.168 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:05:33.168 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2023-10-16T20:05:33.169 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2023-10-16T20:05:33.169 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:05:33.169 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2023-10-16T20:05:33.169 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T20:05:33.169 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2023-10-16T20:05:33.170 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-16T20:05:33.170 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42375sec preferred_lft 42375sec 2023-10-16T20:05:33.170 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2023-10-16T20:05:33.170 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:33.170 INFO:teuthology.orchestra.run.smithi094.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-16T20:05:33.171 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 02:42:0f:b3:8e:fc brd ff:ff:ff:ff:ff:ff 2023-10-16T20:05:33.171 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-16T20:05:33.171 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:33.171 INFO:teuthology.orchestra.run.smithi094.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T20:05:33.171 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-16T20:05:33.171 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T20:05:33.172 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:33.172 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link tentative 2023-10-16T20:05:33.172 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:33.172 INFO:teuthology.orchestra.run.smithi094.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T20:05:33.172 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T20:05:33.173 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link tentative 2023-10-16T20:05:33.173 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:33.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:33.174 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T20:05:33.238 INFO:teuthology.orchestra.run.smithi094.stdout:1 2023-10-16T20:05:33.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:33.241 DEBUG:teuthology.orchestra.run.smithi094:> route 2023-10-16T20:05:33.294 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2023-10-16T20:05:33.294 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T20:05:33.295 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-16T20:05:33.295 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T20:05:33.295 INFO:teuthology.orchestra.run.smithi094.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-16T20:05:33.295 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T20:05:33.295 INFO:teuthology.orchestra.run.smithi094.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:05:33.296 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:05:33.296 INFO:teuthology.orchestra.run.smithi094.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:05:33.296 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T20:05:33.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:33.297 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec bash -c ' 2023-10-16T20:05:33.297 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2023-10-16T20:05:33.297 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T20:05:33.298 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T20:05:33.298 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T20:05:33.298 DEBUG:teuthology.orchestra.run.smithi094:> ') 2023-10-16T20:05:33.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:33.385 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2023-10-16T20:05:33.394 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-16T20:05:33.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:33.395 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list-id 2023-10-16T20:05:33.449 INFO:teuthology.orchestra.run.smithi094.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-16T20:05:33.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:33.449 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec bash -c ' 2023-10-16T20:05:33.450 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2023-10-16T20:05:33.450 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-16T20:05:33.450 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-16T20:05:33.450 DEBUG:teuthology.orchestra.run.smithi094:> ') 2023-10-16T20:05:33.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:33.534 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-16T20:05:33.600 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T20:05:33.600 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T20:05:33.600 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T20:05:33.600 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:33.601 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2023-10-16T20:05:33.601 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:33.601 INFO:teuthology.orchestra.run.smithi094.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T20:05:33.601 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether b6:c0:80:48:f5:d9 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-16T20:05:33.601 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-16T20:05:33.602 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:33.602 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::b4c0:80ff:fe48:f5d9/64 scope link 2023-10-16T20:05:33.602 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:33.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:33.603 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-16T20:05:33.668 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T20:05:33.669 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T20:05:33.669 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T20:05:33.669 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:33.669 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2023-10-16T20:05:33.669 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:33.670 INFO:teuthology.orchestra.run.smithi094.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T20:05:33.670 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether b6:c0:80:48:f5:d9 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-16T20:05:33.670 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-16T20:05:33.670 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:33.670 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::b4c0:80ff:fe48:f5d9/64 scope link 2023-10-16T20:05:33.670 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:05:33.671 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-16T20:05:33.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:33.672 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec bash -c ' 2023-10-16T20:05:33.672 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2023-10-16T20:05:33.673 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-16T20:05:33.673 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-16T20:05:33.673 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-16T20:05:33.673 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-16T20:05:33.674 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-16T20:05:33.674 DEBUG:teuthology.orchestra.run.smithi094:> ') 2023-10-16T20:05:33.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:33.815 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec bash -c ' 2023-10-16T20:05:33.815 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2023-10-16T20:05:33.816 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.1 up 2023-10-16T20:05:33.816 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set dev brx.1 master ceph-brx 2023-10-16T20:05:33.816 DEBUG:teuthology.orchestra.run.smithi094:> ') 2023-10-16T20:05:33.855 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-10-16T20:05:33.856 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T20:05:33.856 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-16T20:05:33.912 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-16T20:05:33.912 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T20:05:33.913 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-16T20:05:33.968 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2023-10-16T20:05:34.032 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T20:05:34.085 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2023-10-16T20:05:34.086 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2023-10-16T20:05:34.086 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2023-10-16T20:05:34.086 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2023-10-16T20:05:34.086 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2023-10-16T20:05:34.086 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2023-10-16T20:05:34.087 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2023-10-16T20:05:34.087 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2023-10-16T20:05:34.087 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2023-10-16T20:05:34.087 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2023-10-16T20:05:34.087 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2023-10-16T20:05:34.088 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2023-10-16T20:05:34.088 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T20:05:34.088 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2023-10-16T20:05:34.088 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2023-10-16T20:05:34.088 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2023-10-16T20:05:34.089 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2023-10-16T20:05:34.089 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2023-10-16T20:05:34.089 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2023-10-16T20:05:34.089 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2023-10-16T20:05:34.089 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T20:05:34.089 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2023-10-16T20:05:34.090 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2023-10-16T20:05:34.090 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2023-10-16T20:05:34.090 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-4 2023-10-16T20:05:34.090 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-5 2023-10-16T20:05:34.090 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-6 2023-10-16T20:05:34.091 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-7 2023-10-16T20:05:34.091 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2023-10-16T20:05:34.091 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T20:05:34.091 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T20:05:34.091 INFO:teuthology.orchestra.run.smithi094.stdout:/home/ubuntu/cephtest/mnt.1 2023-10-16T20:05:34.091 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-16T20:05:34.092 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-16T20:05:34.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:34.093 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2023-10-16T20:05:34.144 INFO:teuthology.orchestra.run.smithi094.stdout:43 2023-10-16T20:05:34.144 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2023-10-16T20:05:34.144 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-10-16T20:05:34.191 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2023-10-16T20:05:34.212 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T20:05:34.265 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2023-10-16T20:05:34.265 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2023-10-16T20:05:34.266 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2023-10-16T20:05:34.266 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2023-10-16T20:05:34.266 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2023-10-16T20:05:34.266 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2023-10-16T20:05:34.267 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2023-10-16T20:05:34.267 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2023-10-16T20:05:34.267 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2023-10-16T20:05:34.267 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2023-10-16T20:05:34.267 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2023-10-16T20:05:34.268 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2023-10-16T20:05:34.268 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T20:05:34.268 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2023-10-16T20:05:34.268 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2023-10-16T20:05:34.268 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2023-10-16T20:05:34.269 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2023-10-16T20:05:34.269 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2023-10-16T20:05:34.269 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2023-10-16T20:05:34.269 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2023-10-16T20:05:34.269 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T20:05:34.270 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2023-10-16T20:05:34.270 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2023-10-16T20:05:34.270 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2023-10-16T20:05:34.270 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-4 2023-10-16T20:05:34.270 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-5 2023-10-16T20:05:34.271 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-6 2023-10-16T20:05:34.271 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-7 2023-10-16T20:05:34.271 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2023-10-16T20:05:34.271 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T20:05:34.271 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T20:05:34.272 INFO:teuthology.orchestra.run.smithi094.stdout:/home/ubuntu/cephtest/mnt.1 2023-10-16T20:05:34.272 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-16T20:05:34.272 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-16T20:05:34.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:34.273 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2023-10-16T20:05:34.286 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi094.stderr:2023-10-16T20:05:34.282+0000 7f440df70580 -1 init, newargv = 0x556ef4224dc0 newargc=15 2023-10-16T20:05:34.286 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi094.stderr:ceph-fuse[42777]: starting ceph client 2023-10-16T20:05:34.293 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi094.stderr:ceph-fuse[42777]: starting fuse 2023-10-16T20:05:34.295 INFO:teuthology.orchestra.run.smithi094.stdout:43 2023-10-16T20:05:34.295 INFO:teuthology.orchestra.run.smithi094.stdout:44 2023-10-16T20:05:34.295 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2023-10-16T20:05:34.296 DEBUG:teuthology.orchestra.run.smithi094:> sudo stdin-killer -- python3 -c ' 2023-10-16T20:05:34.296 DEBUG:teuthology.orchestra.run.smithi094:> import glob 2023-10-16T20:05:34.296 DEBUG:teuthology.orchestra.run.smithi094:> import re 2023-10-16T20:05:34.297 DEBUG:teuthology.orchestra.run.smithi094:> import os 2023-10-16T20:05:34.297 DEBUG:teuthology.orchestra.run.smithi094:> import subprocess 2023-10-16T20:05:34.297 DEBUG:teuthology.orchestra.run.smithi094:> 2023-10-16T20:05:34.297 DEBUG:teuthology.orchestra.run.smithi094:> def _find_admin_socket(client_name): 2023-10-16T20:05:34.298 DEBUG:teuthology.orchestra.run.smithi094:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-10-16T20:05:34.298 DEBUG:teuthology.orchestra.run.smithi094:> files = glob.glob(asok_path) 2023-10-16T20:05:34.298 DEBUG:teuthology.orchestra.run.smithi094:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-10-16T20:05:34.299 DEBUG:teuthology.orchestra.run.smithi094:> 2023-10-16T20:05:34.299 DEBUG:teuthology.orchestra.run.smithi094:> # Given a non-glob path, it better be there 2023-10-16T20:05:34.299 DEBUG:teuthology.orchestra.run.smithi094:> if "*" not in asok_path: 2023-10-16T20:05:34.300 DEBUG:teuthology.orchestra.run.smithi094:> assert(len(files) == 1) 2023-10-16T20:05:34.300 DEBUG:teuthology.orchestra.run.smithi094:> return files[0] 2023-10-16T20:05:34.300 DEBUG:teuthology.orchestra.run.smithi094:> 2023-10-16T20:05:34.300 DEBUG:teuthology.orchestra.run.smithi094:> for f in files: 2023-10-16T20:05:34.301 DEBUG:teuthology.orchestra.run.smithi094:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-16T20:05:34.301 DEBUG:teuthology.orchestra.run.smithi094:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-16T20:05:34.301 DEBUG:teuthology.orchestra.run.smithi094:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-16T20:05:34.301 DEBUG:teuthology.orchestra.run.smithi094:> contents = proc_f.read() 2023-10-16T20:05:34.302 DEBUG:teuthology.orchestra.run.smithi094:> if mountpoint in contents: 2023-10-16T20:05:34.302 DEBUG:teuthology.orchestra.run.smithi094:> return f 2023-10-16T20:05:34.302 DEBUG:teuthology.orchestra.run.smithi094:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-16T20:05:34.303 DEBUG:teuthology.orchestra.run.smithi094:> 2023-10-16T20:05:34.303 DEBUG:teuthology.orchestra.run.smithi094:> print(_find_admin_socket("client.3")) 2023-10-16T20:05:34.303 DEBUG:teuthology.orchestra.run.smithi094:> ' 2023-10-16T20:05:34.420 INFO:teuthology.orchestra.run.smithi094.stdout:/var/run/ceph/ceph-client.3.42777.asok 2023-10-16T20:05:34.423 INFO:teuthology.orchestra.run.smithi094.stderr:2023-10-16T20:05:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-16T20:05:34.431 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.42777.asok 2023-10-16T20:05:34.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:34.432 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.42777.asok status 2023-10-16T20:05:34.607 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2023-10-16T20:05:34.607 INFO:teuthology.orchestra.run.smithi094.stdout: "metadata": { 2023-10-16T20:05:34.607 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_sha1": "4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0", 2023-10-16T20:05:34.608 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 18.0.0-6730-g4ba8a190 (4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0) reef (dev)", 2023-10-16T20:05:34.608 INFO:teuthology.orchestra.run.smithi094.stdout: "entity_id": "3", 2023-10-16T20:05:34.608 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": "smithi094", 2023-10-16T20:05:34.608 INFO:teuthology.orchestra.run.smithi094.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-10-16T20:05:34.608 INFO:teuthology.orchestra.run.smithi094.stdout: "pid": "42777", 2023-10-16T20:05:34.609 INFO:teuthology.orchestra.run.smithi094.stdout: "root": "/" 2023-10-16T20:05:34.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2023-10-16T20:05:34.609 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_count": 0, 2023-10-16T20:05:34.609 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_pinned_count": 0, 2023-10-16T20:05:34.609 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 4643, 2023-10-16T20:05:34.610 INFO:teuthology.orchestra.run.smithi094.stdout: "inst": { 2023-10-16T20:05:34.610 INFO:teuthology.orchestra.run.smithi094.stdout: "name": { 2023-10-16T20:05:34.610 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "client", 2023-10-16T20:05:34.610 INFO:teuthology.orchestra.run.smithi094.stdout: "num": 4643 2023-10-16T20:05:34.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2023-10-16T20:05:34.611 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2023-10-16T20:05:34.611 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2023-10-16T20:05:34.611 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.2:0", 2023-10-16T20:05:34.611 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 42942018 2023-10-16T20:05:34.611 INFO:teuthology.orchestra.run.smithi094.stdout: } 2023-10-16T20:05:34.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2023-10-16T20:05:34.612 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2023-10-16T20:05:34.612 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2023-10-16T20:05:34.612 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.2:0", 2023-10-16T20:05:34.612 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 42942018 2023-10-16T20:05:34.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2023-10-16T20:05:34.613 INFO:teuthology.orchestra.run.smithi094.stdout: "inst_str": "client.4643 192.168.0.2:0/42942018", 2023-10-16T20:05:34.613 INFO:teuthology.orchestra.run.smithi094.stdout: "addr_str": "192.168.0.2:0/42942018", 2023-10-16T20:05:34.613 INFO:teuthology.orchestra.run.smithi094.stdout: "inode_count": 1, 2023-10-16T20:05:34.614 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_epoch": 9, 2023-10-16T20:05:34.614 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch": 27, 2023-10-16T20:05:34.614 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch_barrier": 0, 2023-10-16T20:05:34.614 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklisted": false, 2023-10-16T20:05:34.615 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs" 2023-10-16T20:05:34.615 INFO:teuthology.orchestra.run.smithi094.stdout:} 2023-10-16T20:05:34.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:34.622 DEBUG:teuthology.orchestra.run.smithi088:> stat --file-system '--printf=%T 2023-10-16T20:05:34.622 DEBUG:teuthology.orchestra.run.smithi088:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-16T20:05:34.632 INFO:teuthology.orchestra.run.smithi088.stdout:fuseblk 2023-10-16T20:05:34.632 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-16T20:05:34.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:34.633 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T20:05:34.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:34.694 DEBUG:teuthology.orchestra.run.smithi088:> stat --file-system '--printf=%T 2023-10-16T20:05:34.694 DEBUG:teuthology.orchestra.run.smithi088:> ' -- /home/ubuntu/cephtest/mnt.2 2023-10-16T20:05:34.745 INFO:teuthology.orchestra.run.smithi088.stdout:fuseblk 2023-10-16T20:05:34.746 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-10-16T20:05:34.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:34.746 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-16T20:05:34.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:34.813 DEBUG:teuthology.orchestra.run.smithi094:> stat --file-system '--printf=%T 2023-10-16T20:05:34.813 DEBUG:teuthology.orchestra.run.smithi094:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-16T20:05:34.823 INFO:teuthology.orchestra.run.smithi094.stdout:fuseblk 2023-10-16T20:05:34.824 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-16T20:05:34.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:34.824 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-16T20:05:34.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:34.887 DEBUG:teuthology.orchestra.run.smithi094:> stat --file-system '--printf=%T 2023-10-16T20:05:34.888 DEBUG:teuthology.orchestra.run.smithi094:> ' -- /home/ubuntu/cephtest/mnt.3 2023-10-16T20:05:34.936 INFO:teuthology.orchestra.run.smithi094.stdout:fuseblk 2023-10-16T20:05:34.936 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-10-16T20:05:34.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:34.937 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-16T20:05:34.999 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-16T20:05:35.015 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:05:35.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.150+0000 7f3f73a46640 1 Processor -- start 2023-10-16T20:05:35.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.150+0000 7f3f73a46640 1 -- start start 2023-10-16T20:05:35.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.154+0000 7f3f73a46640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3f6c0d22e0 0x7f3f6c161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:35.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.154+0000 7f3f73a46640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f6c0d26c0 0x7f3f6c15d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:35.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.154+0000 7f3f73a46640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f6c0b87e0 con 0x7f3f6c15d820 2023-10-16T20:05:35.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.154+0000 7f3f73a46640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f6c15dc70 con 0x7f3f6c0d22e0 2023-10-16T20:05:35.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.154+0000 7f3f73a46640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c15df10 con 0x7f3f6c0d26c0 2023-10-16T20:05:35.155 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.154+0000 7f3f70fe1640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f6c0d26c0 0x7f3f6c15d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:35.155 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.154+0000 7f3f717e2640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3f6c0d22e0 0x7f3f6c161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:35.155 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.154+0000 7f3f70fe1640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f6c0d26c0 0x7f3f6c15d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:59282/0 (socket says 172.21.15.88:59282) 2023-10-16T20:05:35.155 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.154+0000 7f3f70fe1640 1 -- 172.21.15.88:0/2652851695 learned_addr learned my addr 172.21.15.88:0/2652851695 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:35.156 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.154+0000 7f3f70fe1640 1 -- 172.21.15.88:0/2652851695 >> v2:172.21.15.94:3300/0 conn(0x7f3f6c0d22e0 msgr2=0x7f3f6c161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:35.156 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.154+0000 7f3f70fe1640 1 --2- 172.21.15.88:0/2652851695 >> v2:172.21.15.94:3300/0 conn(0x7f3f6c0d22e0 0x7f3f6c161450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:35.156 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.154+0000 7f3f70fe1640 1 -- 172.21.15.88:0/2652851695 >> v1:172.21.15.94:6789/0 conn(0x7f3f6c15d820 legacy=0x7f3f6c0cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:35.157 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.154+0000 7f3f70fe1640 1 -- 172.21.15.88:0/2652851695 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f6c15e1f0 con 0x7f3f6c0d26c0 2023-10-16T20:05:35.157 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.154+0000 7f3f717e2640 1 --2- 172.21.15.88:0/2652851695 >> v2:172.21.15.94:3300/0 conn(0x7f3f6c0d22e0 0x7f3f6c161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:05:35.158 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.154+0000 7f3f70fe1640 1 --2- 172.21.15.88:0/2652851695 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f6c0d26c0 0x7f3f6c15d2e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c009870 tx=0x7f3f5c01c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c7bab7d848129a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:35.158 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.158+0000 7f3f63fff640 1 -- 172.21.15.88:0/2652851695 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c0ad640 con 0x7f3f6c0d26c0 2023-10-16T20:05:35.159 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.158+0000 7f3f63fff640 1 -- 172.21.15.88:0/2652851695 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5c0ad7a0 con 0x7f3f6c0d26c0 2023-10-16T20:05:35.159 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.158+0000 7f3f63fff640 1 -- 172.21.15.88:0/2652851695 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c0adb30 con 0x7f3f6c0d26c0 2023-10-16T20:05:35.159 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.158+0000 7f3f73a46640 1 -- 172.21.15.88:0/2652851695 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f6c0d26c0 msgr2=0x7f3f6c15d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:35.159 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.158+0000 7f3f73a46640 1 --2- 172.21.15.88:0/2652851695 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f6c0d26c0 0x7f3f6c15d2e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c009870 tx=0x7f3f5c01c480 comp rx=0 tx=0).stop 2023-10-16T20:05:35.159 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.158+0000 7f3f73a46640 1 -- 172.21.15.88:0/2652851695 shutdown_connections 2023-10-16T20:05:35.160 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.158+0000 7f3f73a46640 1 --2- 172.21.15.88:0/2652851695 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f6c0d26c0 0x7f3f6c15d2e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:35.160 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.158+0000 7f3f73a46640 1 --2- 172.21.15.88:0/2652851695 >> v2:172.21.15.94:3300/0 conn(0x7f3f6c0d22e0 0x7f3f6c161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:35.160 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.158+0000 7f3f73a46640 1 -- 172.21.15.88:0/2652851695 >> 172.21.15.88:0/2652851695 conn(0x7f3f6c0715c0 msgr2=0x7f3f6c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:35.160 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.158+0000 7f3f73a46640 1 -- 172.21.15.88:0/2652851695 shutdown_connections 2023-10-16T20:05:35.161 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.158+0000 7f3f73a46640 1 -- 172.21.15.88:0/2652851695 wait complete. 2023-10-16T20:05:35.163 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.158+0000 7f3f73a46640 1 Processor -- start 2023-10-16T20:05:35.163 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.158+0000 7f3f73a46640 1 -- start start 2023-10-16T20:05:35.163 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.158+0000 7f3f73a46640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f6c0d22e0 0x7f3f6c15f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:35.163 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.158+0000 7f3f73a46640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f6c0d26c0 0x7f3f6c14e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:35.164 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.158+0000 7f3f73a46640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f6c15d820 0x7f3f6c14e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:35.164 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.158+0000 7f3f73a46640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c14f9b0 con 0x7f3f6c0d22e0 2023-10-16T20:05:35.164 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.158+0000 7f3f73a46640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c0c7350 con 0x7f3f6c0d26c0 2023-10-16T20:05:35.164 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.158+0000 7f3f73a46640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c150380 con 0x7f3f6c15d820 2023-10-16T20:05:35.164 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.162+0000 7f3f717e2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f6c0d22e0 0x7f3f6c15f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:35.165 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.162+0000 7f3f70fe1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f6c0d26c0 0x7f3f6c14e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:35.165 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.162+0000 7f3f70fe1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f6c0d26c0 0x7f3f6c14e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:56910/0 (socket says 172.21.15.88:56910) 2023-10-16T20:05:35.165 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.162+0000 7f3f71fe3640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f6c15d820 0x7f3f6c14e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:35.165 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.162+0000 7f3f70fe1640 1 -- 172.21.15.88:0/1007812043 learned_addr learned my addr 172.21.15.88:0/1007812043 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:35.165 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.162+0000 7f3f70fe1640 1 -- 172.21.15.88:0/1007812043 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f6c15d820 msgr2=0x7f3f6c14e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:35.166 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.162+0000 7f3f70fe1640 1 --2- 172.21.15.88:0/1007812043 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f6c15d820 0x7f3f6c14e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:35.166 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.162+0000 7f3f70fe1640 1 -- 172.21.15.88:0/1007812043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f6c0d22e0 msgr2=0x7f3f6c15f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:35.166 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.162+0000 7f3f70fe1640 1 --2- 172.21.15.88:0/1007812043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f6c0d22e0 0x7f3f6c15f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:35.166 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.162+0000 7f3f70fe1640 1 -- 172.21.15.88:0/1007812043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f68009420 con 0x7f3f6c0d26c0 2023-10-16T20:05:35.166 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.162+0000 7f3f717e2640 1 --2- 172.21.15.88:0/1007812043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f6c0d22e0 0x7f3f6c15f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:05:35.167 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.162+0000 7f3f70fe1640 1 --2- 172.21.15.88:0/1007812043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f6c0d26c0 0x7f3f6c14e230 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c0b0040 tx=0x7f3f5c019220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:35.167 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.162+0000 7f3f61ffb640 1 -- 172.21.15.88:0/1007812043 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c0bc070 con 0x7f3f6c0d26c0 2023-10-16T20:05:35.167 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.162+0000 7f3f61ffb640 1 -- 172.21.15.88:0/1007812043 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5c0a6040 con 0x7f3f6c0d26c0 2023-10-16T20:05:35.167 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.162+0000 7f3f73a46640 1 -- 172.21.15.88:0/1007812043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f5c0094d0 con 0x7f3f6c0d26c0 2023-10-16T20:05:35.167 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.162+0000 7f3f61ffb640 1 -- 172.21.15.88:0/1007812043 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c0b86e0 con 0x7f3f6c0d26c0 2023-10-16T20:05:35.168 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.162+0000 7f3f73a46640 1 -- 172.21.15.88:0/1007812043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f6c14f080 con 0x7f3f6c0d26c0 2023-10-16T20:05:35.169 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.162+0000 7f3f73a46640 1 -- 172.21.15.88:0/1007812043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f6c0c74c0 con 0x7f3f6c0d26c0 2023-10-16T20:05:35.169 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.162+0000 7f3f61ffb640 1 -- 172.21.15.88:0/1007812043 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3f5c0b8a50 con 0x7f3f6c0d26c0 2023-10-16T20:05:35.169 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.166+0000 7f3f61ffb640 1 --2- 172.21.15.88:0/1007812043 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3f500664a0 0x7f3f50068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:35.169 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.166+0000 7f3f61ffb640 1 -- 172.21.15.88:0/1007812043 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3f5c11e880 con 0x7f3f6c0d26c0 2023-10-16T20:05:35.169 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.166+0000 7f3f717e2640 1 --2- 172.21.15.88:0/1007812043 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3f500664a0 0x7f3f50068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:35.170 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.166+0000 7f3f717e2640 1 --2- 172.21.15.88:0/1007812043 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3f500664a0 0x7f3f50068960 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3f68009fd0 tx=0x7f3f68009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:35.170 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.166+0000 7f3f61ffb640 1 -- 172.21.15.88:0/1007812043 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3f5c0ebd50 con 0x7f3f6c0d26c0 2023-10-16T20:05:35.319 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.318+0000 7f3f73a46640 1 -- 172.21.15.88:0/1007812043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f6c0c7490 con 0x7f3f6c0d26c0 2023-10-16T20:05:35.321 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.318+0000 7f3f61ffb640 1 -- 172.21.15.88:0/1007812043 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18274 (secure 0 0 0) 0x7f3f5c0ef370 con 0x7f3f6c0d26c0 2023-10-16T20:05:35.321 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:35.322 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:05:01.835630+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T20:04:54.442487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T20:04:55.766641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T20:04:59.475830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:05:35.323 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.322+0000 7f3f73a46640 1 -- 172.21.15.88:0/1007812043 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3f500664a0 msgr2=0x7f3f50068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:35.323 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.322+0000 7f3f73a46640 1 --2- 172.21.15.88:0/1007812043 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3f500664a0 0x7f3f50068960 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3f68009fd0 tx=0x7f3f68009190 comp rx=0 tx=0).stop 2023-10-16T20:05:35.323 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.322+0000 7f3f73a46640 1 -- 172.21.15.88:0/1007812043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f6c0d26c0 msgr2=0x7f3f6c14e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:35.323 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.322+0000 7f3f73a46640 1 --2- 172.21.15.88:0/1007812043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f6c0d26c0 0x7f3f6c14e230 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c0b0040 tx=0x7f3f5c019220 comp rx=0 tx=0).stop 2023-10-16T20:05:35.324 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.322+0000 7f3f73a46640 1 -- 172.21.15.88:0/1007812043 shutdown_connections 2023-10-16T20:05:35.324 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.322+0000 7f3f73a46640 1 --2- 172.21.15.88:0/1007812043 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3f500664a0 0x7f3f50068960 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:35.324 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.322+0000 7f3f73a46640 1 --2- 172.21.15.88:0/1007812043 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f6c15d820 0x7f3f6c14e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:35.324 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.322+0000 7f3f73a46640 1 --2- 172.21.15.88:0/1007812043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f6c0d26c0 0x7f3f6c14e230 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:35.324 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.322+0000 7f3f73a46640 1 --2- 172.21.15.88:0/1007812043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f6c0d22e0 0x7f3f6c15f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:35.325 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.322+0000 7f3f73a46640 1 -- 172.21.15.88:0/1007812043 >> 172.21.15.88:0/1007812043 conn(0x7f3f6c0715c0 msgr2=0x7f3f6c0719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:35.325 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.322+0000 7f3f73a46640 1 -- 172.21.15.88:0/1007812043 shutdown_connections 2023-10-16T20:05:35.325 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.322+0000 7f3f73a46640 1 -- 172.21.15.88:0/1007812043 wait complete. 2023-10-16T20:05:35.342 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T20:04:54.442487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T20:04:55.766641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T20:04:59.475830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T20:05:35.343 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:05:35.484 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.482+0000 7f8689532640 1 Processor -- start 2023-10-16T20:05:35.485 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.482+0000 7f8689532640 1 -- start start 2023-10-16T20:05:35.486 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.486+0000 7f8689532640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f86840cf940 0x7f86840d3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:35.487 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.486+0000 7f8689532640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86840d0c40 con 0x7f86840cde80 2023-10-16T20:05:35.487 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.486+0000 7f8689532640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86840ce2d0 con 0x7f868415c9a0 2023-10-16T20:05:35.487 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.486+0000 7f8689532640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86840d37b0 con 0x7f86840cf940 2023-10-16T20:05:35.487 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.486+0000 7f86837fe640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f86840cde80 0x7f86840cf230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:54702/0 (socket says 172.21.15.88:54702) 2023-10-16T20:05:35.487 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.486+0000 7f86837fe640 1 -- 172.21.15.88:0/3819659282 learned_addr learned my addr 172.21.15.88:0/3819659282 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:35.488 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.486+0000 7f8688d31640 1 --2- 172.21.15.88:0/3819659282 >> v2:172.21.15.88:3300/0 conn(0x7f86840cf940 0x7f86840d3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:35.488 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.486+0000 7f8682ffd640 1 -- 172.21.15.88:0/3819659282 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1334947438 0 0) 0x7f86840d0c40 con 0x7f86840cde80 2023-10-16T20:05:35.488 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.486+0000 7f8682ffd640 1 -- 172.21.15.88:0/3819659282 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f866c003530 con 0x7f86840cde80 2023-10-16T20:05:35.488 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.486+0000 7f8688d31640 1 -- 172.21.15.88:0/3819659282 >> v1:172.21.15.94:6789/0 conn(0x7f868415c9a0 legacy=0x7f86840d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:35.488 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.486+0000 7f8682ffd640 1 -- 172.21.15.88:0/3819659282 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4007949269 0 0) 0x7f86840ce2d0 con 0x7f868415c9a0 2023-10-16T20:05:35.488 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.486+0000 7f8688d31640 1 -- 172.21.15.88:0/3819659282 >> v1:172.21.15.88:6789/0 conn(0x7f86840cde80 legacy=0x7f86840cf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:35.489 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.486+0000 7f8688d31640 1 -- 172.21.15.88:0/3819659282 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86840d0630 con 0x7f86840cf940 2023-10-16T20:05:35.489 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.490+0000 7f8688d31640 1 --2- 172.21.15.88:0/3819659282 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86840cf940 0x7f86840d3270 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f867c009d90 tx=0x7f867c01c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da43975b2e010353 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:35.490 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.490+0000 7f8682ffd640 1 -- 172.21.15.88:0/3819659282 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c0a5870 con 0x7f86840cf940 2023-10-16T20:05:35.490 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.490+0000 7f8682ffd640 1 -- 172.21.15.88:0/3819659282 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867c0a59d0 con 0x7f86840cf940 2023-10-16T20:05:35.490 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.490+0000 7f8682ffd640 1 -- 172.21.15.88:0/3819659282 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c0a5d60 con 0x7f86840cf940 2023-10-16T20:05:35.491 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.490+0000 7f8689532640 1 -- 172.21.15.88:0/3819659282 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86840cf940 msgr2=0x7f86840d3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:35.491 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.490+0000 7f8689532640 1 --2- 172.21.15.88:0/3819659282 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86840cf940 0x7f86840d3270 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f867c009d90 tx=0x7f867c01c8c0 comp rx=0 tx=0).stop 2023-10-16T20:05:35.491 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.490+0000 7f8689532640 1 -- 172.21.15.88:0/3819659282 shutdown_connections 2023-10-16T20:05:35.491 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.490+0000 7f8689532640 1 --2- 172.21.15.88:0/3819659282 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86840cf940 0x7f86840d3270 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:35.492 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.490+0000 7f8689532640 1 -- 172.21.15.88:0/3819659282 >> 172.21.15.88:0/3819659282 conn(0x7f868409bc30 msgr2=0x7f86840d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:35.492 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.490+0000 7f8689532640 1 -- 172.21.15.88:0/3819659282 shutdown_connections 2023-10-16T20:05:35.492 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.490+0000 7f8689532640 1 -- 172.21.15.88:0/3819659282 wait complete. 2023-10-16T20:05:35.492 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.490+0000 7f8689532640 1 Processor -- start 2023-10-16T20:05:35.493 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.490+0000 7f8689532640 1 -- start start 2023-10-16T20:05:35.494 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.494+0000 7f8689532640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86840cde80 0x7f868415f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:35.494 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.494+0000 7f8689532640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86840cf940 0x7f8684153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:35.495 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.494+0000 7f8689532640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f868415c9a0 0x7f86840b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:35.495 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.494+0000 7f8689532640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86841f6080 con 0x7f86840cde80 2023-10-16T20:05:35.495 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.494+0000 7f8689532640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86841f4fe0 con 0x7f86840cf940 2023-10-16T20:05:35.495 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.494+0000 7f8689532640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86841f52c0 con 0x7f868415c9a0 2023-10-16T20:05:35.495 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.494+0000 7f8683fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86840cde80 0x7f868415f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:35.496 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.494+0000 7f8683fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86840cde80 0x7f868415f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47024/0 (socket says 172.21.15.88:47024) 2023-10-16T20:05:35.496 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.494+0000 7f8683fff640 1 -- 172.21.15.88:0/395230016 learned_addr learned my addr 172.21.15.88:0/395230016 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:35.496 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.494+0000 7f8683fff640 1 -- 172.21.15.88:0/395230016 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f868415c9a0 msgr2=0x7f86840b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:35.496 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.494+0000 7f8683fff640 1 --2- 172.21.15.88:0/395230016 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f868415c9a0 0x7f86840b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:35.496 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.494+0000 7f8683fff640 1 -- 172.21.15.88:0/395230016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86840cf940 msgr2=0x7f8684153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:35.496 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.494+0000 7f8683fff640 1 --2- 172.21.15.88:0/395230016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86840cf940 0x7f8684153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:35.497 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.494+0000 7f8683fff640 1 -- 172.21.15.88:0/395230016 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c009a70 con 0x7f86840cde80 2023-10-16T20:05:35.497 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.494+0000 7f8683fff640 1 --2- 172.21.15.88:0/395230016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86840cde80 0x7f868415f9c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f867400c410 tx=0x7f867400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:35.497 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.494+0000 7f8680ff9640 1 -- 172.21.15.88:0/395230016 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86740054e0 con 0x7f86840cde80 2023-10-16T20:05:35.497 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.494+0000 7f8680ff9640 1 -- 172.21.15.88:0/395230016 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8674003650 con 0x7f86840cde80 2023-10-16T20:05:35.497 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.494+0000 7f8689532640 1 -- 172.21.15.88:0/395230016 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f868415cf00 con 0x7f86840cde80 2023-10-16T20:05:35.498 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.494+0000 7f8680ff9640 1 -- 172.21.15.88:0/395230016 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8674003a00 con 0x7f86840cde80 2023-10-16T20:05:35.498 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.494+0000 7f8689532640 1 -- 172.21.15.88:0/395230016 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86841f5880 con 0x7f86840cde80 2023-10-16T20:05:35.499 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.494+0000 7f8689532640 1 -- 172.21.15.88:0/395230016 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86840674f0 con 0x7f86840cde80 2023-10-16T20:05:35.499 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.494+0000 7f8680ff9640 1 -- 172.21.15.88:0/395230016 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8674003b60 con 0x7f86840cde80 2023-10-16T20:05:35.499 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.494+0000 7f8680ff9640 1 --2- 172.21.15.88:0/395230016 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f86700664a0 0x7f8670068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:35.499 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.498+0000 7f8680ff9640 1 -- 172.21.15.88:0/395230016 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8674083850 con 0x7f86840cde80 2023-10-16T20:05:35.500 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.498+0000 7f86837fe640 1 --2- 172.21.15.88:0/395230016 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f86700664a0 0x7f8670068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:35.500 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.498+0000 7f86837fe640 1 --2- 172.21.15.88:0/395230016 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f86700664a0 0x7f8670068960 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8678007fd0 tx=0x7f8678007900 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:35.501 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.498+0000 7f8680ff9640 1 -- 172.21.15.88:0/395230016 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8674050940 con 0x7f86840cde80 2023-10-16T20:05:35.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.650+0000 7f8689532640 1 -- 172.21.15.88:0/395230016 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f86840cff40 con 0x7f86840cde80 2023-10-16T20:05:35.651 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.650+0000 7f8680ff9640 1 -- 172.21.15.88:0/395230016 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8674017020 con 0x7f86840cde80 2023-10-16T20:05:35.651 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:05:35.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.650+0000 7f8689532640 1 -- 172.21.15.88:0/395230016 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f86700664a0 msgr2=0x7f8670068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:35.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.650+0000 7f8689532640 1 --2- 172.21.15.88:0/395230016 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f86700664a0 0x7f8670068960 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8678007fd0 tx=0x7f8678007900 comp rx=0 tx=0).stop 2023-10-16T20:05:35.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.650+0000 7f8689532640 1 -- 172.21.15.88:0/395230016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86840cde80 msgr2=0x7f868415f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:35.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.650+0000 7f8689532640 1 --2- 172.21.15.88:0/395230016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86840cde80 0x7f868415f9c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f867400c410 tx=0x7f867400c8d0 comp rx=0 tx=0).stop 2023-10-16T20:05:35.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.650+0000 7f8689532640 1 -- 172.21.15.88:0/395230016 shutdown_connections 2023-10-16T20:05:35.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.650+0000 7f8689532640 1 --2- 172.21.15.88:0/395230016 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f86700664a0 0x7f8670068960 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:35.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.650+0000 7f8689532640 1 --2- 172.21.15.88:0/395230016 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f868415c9a0 0x7f86840b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:35.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.650+0000 7f8689532640 1 --2- 172.21.15.88:0/395230016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86840cf940 0x7f8684153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:35.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.650+0000 7f8689532640 1 --2- 172.21.15.88:0/395230016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86840cde80 0x7f868415f9c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:35.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.650+0000 7f8689532640 1 -- 172.21.15.88:0/395230016 >> 172.21.15.88:0/395230016 conn(0x7f868409bc30 msgr2=0x7f86840b14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:35.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.650+0000 7f8689532640 1 -- 172.21.15.88:0/395230016 shutdown_connections 2023-10-16T20:05:35.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.650+0000 7f8689532640 1 -- 172.21.15.88:0/395230016 wait complete. 2023-10-16T20:05:35.672 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T20:05:35.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.810+0000 7fd27d446640 1 Processor -- start 2023-10-16T20:05:35.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.810+0000 7fd27d446640 1 -- start start 2023-10-16T20:05:35.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd27d446640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd27815c960 0x7fd2780cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:35.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd27d446640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd2780ce450 0x7fd2780cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:35.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd27d446640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2780d05f0 con 0x7fd2780cf7d0 2023-10-16T20:05:35.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd27d446640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2780cfc60 con 0x7fd2780ce450 2023-10-16T20:05:35.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd27d446640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2780d08d0 con 0x7fd27815c960 2023-10-16T20:05:35.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd2777fe640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fd2780cf7d0 0x7fd278160550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:54712/0 (socket says 172.21.15.88:54712) 2023-10-16T20:05:35.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd2777fe640 1 -- 172.21.15.88:0/2848542689 learned_addr learned my addr 172.21.15.88:0/2848542689 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:35.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd276ffd640 1 --2- 172.21.15.88:0/2848542689 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd27815c960 0x7fd2780cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:35.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd26e7fc640 1 --2- 172.21.15.88:0/2848542689 >> v2:172.21.15.94:3300/0 conn(0x7fd2780ce450 0x7fd2780cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:35.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd2767fc640 1 -- 172.21.15.88:0/2848542689 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2159947270 0 0) 0x7fd2780d05f0 con 0x7fd2780cf7d0 2023-10-16T20:05:35.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd2767fc640 1 -- 172.21.15.88:0/2848542689 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd264003530 con 0x7fd2780cf7d0 2023-10-16T20:05:35.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd2767fc640 1 -- 172.21.15.88:0/2848542689 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1297417431 0 0) 0x7fd264003530 con 0x7fd2780cf7d0 2023-10-16T20:05:35.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd2767fc640 1 -- 172.21.15.88:0/2848542689 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd27815c960 msgr2=0x7fd2780cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:35.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd2767fc640 1 --2- 172.21.15.88:0/2848542689 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd27815c960 0x7fd2780cdf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:35.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd2767fc640 1 -- 172.21.15.88:0/2848542689 >> v2:172.21.15.94:3300/0 conn(0x7fd2780ce450 msgr2=0x7fd2780cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:35.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd2767fc640 1 --2- 172.21.15.88:0/2848542689 >> v2:172.21.15.94:3300/0 conn(0x7fd2780ce450 0x7fd2780cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:35.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd2767fc640 1 -- 172.21.15.88:0/2848542689 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2780d0bb0 con 0x7fd2780cf7d0 2023-10-16T20:05:35.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd2767fc640 1 -- 172.21.15.88:0/2848542689 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7fd270001680 con 0x7fd2780cf7d0 2023-10-16T20:05:35.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd2767fc640 1 -- 172.21.15.88:0/2848542689 >> v1:172.21.15.88:6789/0 conn(0x7fd2780cf7d0 legacy=0x7fd278160550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:35.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd2767fc640 1 --2- 172.21.15.88:0/2848542689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd264003980 0x7fd264003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:35.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd2767fc640 1 --2- 172.21.15.88:0/2848542689 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd27815c960 0x7fd2640047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:35.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd2767fc640 1 --2- 172.21.15.88:0/2848542689 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd264004d10 0x7fd264005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:35.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd2767fc640 1 -- 172.21.15.88:0/2848542689 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2780cfc60 con 0x7fd264004d10 2023-10-16T20:05:35.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd2767fc640 1 -- 172.21.15.88:0/2848542689 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2780d08d0 con 0x7fd264003980 2023-10-16T20:05:35.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd2767fc640 1 -- 172.21.15.88:0/2848542689 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd26400b990 con 0x7fd27815c960 2023-10-16T20:05:35.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd2777fe640 1 --2- 172.21.15.88:0/2848542689 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd264004d10 0x7fd264005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:35.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd276ffd640 1 --2- 172.21.15.88:0/2848542689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd264003980 0x7fd264003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:35.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd26e7fc640 1 --2- 172.21.15.88:0/2848542689 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd27815c960 0x7fd2640047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:35.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd2777fe640 1 -- 172.21.15.88:0/2848542689 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd27815c960 msgr2=0x7fd2640047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:35.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd2777fe640 1 --2- 172.21.15.88:0/2848542689 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd27815c960 0x7fd2640047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:35.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd2777fe640 1 -- 172.21.15.88:0/2848542689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd264003980 msgr2=0x7fd264003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:35.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd2777fe640 1 --2- 172.21.15.88:0/2848542689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd264003980 0x7fd264003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:35.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.814+0000 7fd2777fe640 1 -- 172.21.15.88:0/2848542689 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd264003530 con 0x7fd264004d10 2023-10-16T20:05:35.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.818+0000 7fd2777fe640 1 --2- 172.21.15.88:0/2848542689 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd264004d10 0x7fd264005190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd27000c240 tx=0x7fd27001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1968a46ae462a3a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:35.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.818+0000 7fd2767fc640 1 -- 172.21.15.88:0/2848542689 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2700a5960 con 0x7fd264004d10 2023-10-16T20:05:35.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.818+0000 7fd2767fc640 1 -- 172.21.15.88:0/2848542689 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2700a5ac0 con 0x7fd264004d10 2023-10-16T20:05:35.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.818+0000 7fd2767fc640 1 -- 172.21.15.88:0/2848542689 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2700a5e90 con 0x7fd264004d10 2023-10-16T20:05:35.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.818+0000 7fd27d446640 1 -- 172.21.15.88:0/2848542689 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd264004d10 msgr2=0x7fd264005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:35.825 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.818+0000 7fd27d446640 1 --2- 172.21.15.88:0/2848542689 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd264004d10 0x7fd264005190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd27000c240 tx=0x7fd27001cc60 comp rx=0 tx=0).stop 2023-10-16T20:05:35.825 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.818+0000 7fd2777fe640 1 -- 172.21.15.88:0/2848542689 reap_dead start 2023-10-16T20:05:35.825 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.818+0000 7fd27d446640 1 -- 172.21.15.88:0/2848542689 shutdown_connections 2023-10-16T20:05:35.825 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.818+0000 7fd27d446640 1 -- 172.21.15.88:0/2848542689 >> 172.21.15.88:0/2848542689 conn(0x7fd27809bbd0 msgr2=0x7fd2780d2880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:35.825 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.818+0000 7fd27d446640 1 -- 172.21.15.88:0/2848542689 shutdown_connections 2023-10-16T20:05:35.826 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.818+0000 7fd27d446640 1 -- 172.21.15.88:0/2848542689 wait complete. 2023-10-16T20:05:35.826 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.818+0000 7fd27d446640 1 Processor -- start 2023-10-16T20:05:35.826 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.822+0000 7fd27d446640 1 -- start start 2023-10-16T20:05:35.826 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.822+0000 7fd27d446640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd264004d10 0x7fd2781f1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:35.826 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.822+0000 7fd27d446640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2780b3b80 0x7fd2780b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:35.827 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.822+0000 7fd27d446640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2781f58d0 0x7fd2781f5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:35.827 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.822+0000 7fd27d446640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2781f62c0 con 0x7fd2781f58d0 2023-10-16T20:05:35.827 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.822+0000 7fd27d446640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2780b4d00 con 0x7fd2780b3b80 2023-10-16T20:05:35.827 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.822+0000 7fd27d446640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2781f21f0 con 0x7fd264004d10 2023-10-16T20:05:35.827 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.822+0000 7fd2777fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2781f58d0 0x7fd2781f5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:35.827 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.822+0000 7fd2777fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2781f58d0 0x7fd2781f5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47040/0 (socket says 172.21.15.88:47040) 2023-10-16T20:05:35.828 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.822+0000 7fd276ffd640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd264004d10 0x7fd2781f1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:35.828 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.822+0000 7fd2777fe640 1 -- 172.21.15.88:0/2188101318 learned_addr learned my addr 172.21.15.88:0/2188101318 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:35.828 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.822+0000 7fd26e7fc640 1 --2- 172.21.15.88:0/2188101318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2780b3b80 0x7fd2780b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:35.828 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.822+0000 7fd2777fe640 1 -- 172.21.15.88:0/2188101318 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd264004d10 msgr2=0x7fd2781f1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:35.828 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.822+0000 7fd2777fe640 1 --2- 172.21.15.88:0/2188101318 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd264004d10 0x7fd2781f1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:35.829 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.822+0000 7fd2777fe640 1 -- 172.21.15.88:0/2188101318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2780b3b80 msgr2=0x7fd2780b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:35.829 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.822+0000 7fd2777fe640 1 --2- 172.21.15.88:0/2188101318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2780b3b80 0x7fd2780b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:35.829 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.822+0000 7fd2777fe640 1 -- 172.21.15.88:0/2188101318 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2780b40f0 con 0x7fd2781f58d0 2023-10-16T20:05:35.829 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.822+0000 7fd2777fe640 1 --2- 172.21.15.88:0/2188101318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2781f58d0 0x7fd2781f5d80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd27000c240 tx=0x7fd264005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:35.829 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.822+0000 7fd26ffff640 1 -- 172.21.15.88:0/2188101318 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2700a5960 con 0x7fd2781f58d0 2023-10-16T20:05:35.830 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.822+0000 7fd26ffff640 1 -- 172.21.15.88:0/2188101318 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2700072d0 con 0x7fd2781f58d0 2023-10-16T20:05:35.830 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.822+0000 7fd26ffff640 1 -- 172.21.15.88:0/2188101318 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd270007640 con 0x7fd2781f58d0 2023-10-16T20:05:35.830 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.822+0000 7fd27d446640 1 -- 172.21.15.88:0/2188101318 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2780b43d0 con 0x7fd2781f58d0 2023-10-16T20:05:35.830 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.822+0000 7fd27d446640 1 -- 172.21.15.88:0/2188101318 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2780b4990 con 0x7fd2781f58d0 2023-10-16T20:05:35.831 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.822+0000 7fd27d446640 1 -- 172.21.15.88:0/2188101318 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2780b4eb0 con 0x7fd2781f58d0 2023-10-16T20:05:35.831 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.826+0000 7fd26ffff640 1 -- 172.21.15.88:0/2188101318 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd2700af440 con 0x7fd2781f58d0 2023-10-16T20:05:35.832 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.826+0000 7fd26ffff640 1 --2- 172.21.15.88:0/2188101318 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd2580664d0 0x7fd258068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:35.832 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.826+0000 7fd26ffff640 1 -- 172.21.15.88:0/2188101318 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd2700b5030 con 0x7fd2781f58d0 2023-10-16T20:05:35.832 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.826+0000 7fd276ffd640 1 --2- 172.21.15.88:0/2188101318 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd2580664d0 0x7fd258068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:35.832 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.826+0000 7fd276ffd640 1 --2- 172.21.15.88:0/2188101318 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd2580664d0 0x7fd258068990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd2780b2280 tx=0x7fd26800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:35.832 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.830+0000 7fd26ffff640 1 -- 172.21.15.88:0/2188101318 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd2780b4eb0 con 0x7fd2781f58d0 2023-10-16T20:05:35.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.978+0000 7fd27d446640 1 -- 172.21.15.88:0/2188101318 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd2780b4eb0 con 0x7fd2781f58d0 2023-10-16T20:05:35.978 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.978+0000 7fd26ffff640 1 -- 172.21.15.88:0/2188101318 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd2780b4eb0 con 0x7fd2781f58d0 2023-10-16T20:05:35.978 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:05:35.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.978+0000 7fd27d446640 1 -- 172.21.15.88:0/2188101318 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd2580664d0 msgr2=0x7fd258068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:35.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.978+0000 7fd27d446640 1 --2- 172.21.15.88:0/2188101318 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd2580664d0 0x7fd258068990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd2780b2280 tx=0x7fd26800b040 comp rx=0 tx=0).stop 2023-10-16T20:05:35.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.978+0000 7fd27d446640 1 -- 172.21.15.88:0/2188101318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2781f58d0 msgr2=0x7fd2781f5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:35.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.978+0000 7fd27d446640 1 --2- 172.21.15.88:0/2188101318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2781f58d0 0x7fd2781f5d80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd27000c240 tx=0x7fd264005800 comp rx=0 tx=0).stop 2023-10-16T20:05:35.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.978+0000 7fd27d446640 1 -- 172.21.15.88:0/2188101318 shutdown_connections 2023-10-16T20:05:35.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.978+0000 7fd27d446640 1 --2- 172.21.15.88:0/2188101318 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd2580664d0 0x7fd258068990 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:35.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.978+0000 7fd27d446640 1 --2- 172.21.15.88:0/2188101318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2781f58d0 0x7fd2781f5d80 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:35.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.978+0000 7fd27d446640 1 --2- 172.21.15.88:0/2188101318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2780b3b80 0x7fd2780b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:35.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.978+0000 7fd27d446640 1 --2- 172.21.15.88:0/2188101318 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd264004d10 0x7fd2781f1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:35.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.978+0000 7fd27d446640 1 -- 172.21.15.88:0/2188101318 >> 172.21.15.88:0/2188101318 conn(0x7fd27809bbd0 msgr2=0x7fd27806c750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:35.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.978+0000 7fd27d446640 1 -- 172.21.15.88:0/2188101318 shutdown_connections 2023-10-16T20:05:35.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:35.978+0000 7fd27d446640 1 -- 172.21.15.88:0/2188101318 wait complete. 2023-10-16T20:05:35.999 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T20:05:36.139 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.138+0000 7f742199d640 1 Processor -- start 2023-10-16T20:05:36.139 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.138+0000 7f742199d640 1 -- start start 2023-10-16T20:05:36.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.138+0000 7f742199d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f741c0b0da0 0x7f741c159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:36.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.138+0000 7f742199d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f741c15a1f0 0x7f741c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:36.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.138+0000 7f742199d640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f741c15ad30 con 0x7f741c0b4260 2023-10-16T20:05:36.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.138+0000 7f742199d640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f741c0b4640 con 0x7f741c0b0da0 2023-10-16T20:05:36.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.138+0000 7f742199d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f741c15a770 con 0x7f741c15a1f0 2023-10-16T20:05:36.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.138+0000 7f741affd640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f741c0b4260 0x7f741c15dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:54726/0 (socket says 172.21.15.88:54726) 2023-10-16T20:05:36.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.138+0000 7f741affd640 1 -- 172.21.15.88:0/3163425061 learned_addr learned my addr 172.21.15.88:0/3163425061 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:36.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.138+0000 7f74127fc640 1 --2- 172.21.15.88:0/3163425061 >> v2:172.21.15.88:3300/0 conn(0x7f741c0b0da0 0x7f741c159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:36.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.138+0000 7f741b7fe640 1 --2- 172.21.15.88:0/3163425061 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f741c15a1f0 0x7f741c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:36.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.138+0000 7f741a7fc640 1 -- 172.21.15.88:0/3163425061 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1385428059 0 0) 0x7f741c15ad30 con 0x7f741c0b4260 2023-10-16T20:05:36.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.138+0000 7f741a7fc640 1 -- 172.21.15.88:0/3163425061 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7408005740 con 0x7f741c0b4260 2023-10-16T20:05:36.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.138+0000 7f74127fc640 1 -- 172.21.15.88:0/3163425061 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f741c15a1f0 msgr2=0x7f741c06d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:36.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.138+0000 7f74127fc640 1 --2- 172.21.15.88:0/3163425061 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f741c15a1f0 0x7f741c06d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:36.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.138+0000 7f74127fc640 1 -- 172.21.15.88:0/3163425061 >> v1:172.21.15.88:6789/0 conn(0x7f741c0b4260 legacy=0x7f741c15dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:36.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.138+0000 7f74127fc640 1 -- 172.21.15.88:0/3163425061 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f741c15aa50 con 0x7f741c0b0da0 2023-10-16T20:05:36.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.142+0000 7f74127fc640 1 --2- 172.21.15.88:0/3163425061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f741c0b0da0 0x7f741c159cb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7404009d90 tx=0x7f740401c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35647390d62d54fb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:36.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.142+0000 7f741a7fc640 1 -- 172.21.15.88:0/3163425061 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74040a5780 con 0x7f741c0b0da0 2023-10-16T20:05:36.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.142+0000 7f741a7fc640 1 -- 172.21.15.88:0/3163425061 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74040a58e0 con 0x7f741c0b0da0 2023-10-16T20:05:36.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.142+0000 7f741a7fc640 1 -- 172.21.15.88:0/3163425061 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74040a5c70 con 0x7f741c0b0da0 2023-10-16T20:05:36.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.142+0000 7f742199d640 1 -- 172.21.15.88:0/3163425061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f741c0b0da0 msgr2=0x7f741c159cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:36.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.142+0000 7f742199d640 1 --2- 172.21.15.88:0/3163425061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f741c0b0da0 0x7f741c159cb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7404009d90 tx=0x7f740401c8c0 comp rx=0 tx=0).stop 2023-10-16T20:05:36.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.142+0000 7f742199d640 1 -- 172.21.15.88:0/3163425061 shutdown_connections 2023-10-16T20:05:36.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.142+0000 7f742199d640 1 --2- 172.21.15.88:0/3163425061 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f741c15a1f0 0x7f741c06d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:36.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.142+0000 7f742199d640 1 --2- 172.21.15.88:0/3163425061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f741c0b0da0 0x7f741c159cb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:36.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.142+0000 7f742199d640 1 -- 172.21.15.88:0/3163425061 >> 172.21.15.88:0/3163425061 conn(0x7f741c0cb080 msgr2=0x7f741c0cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:36.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.142+0000 7f742199d640 1 -- 172.21.15.88:0/3163425061 shutdown_connections 2023-10-16T20:05:36.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.142+0000 7f742199d640 1 -- 172.21.15.88:0/3163425061 wait complete. 2023-10-16T20:05:36.150 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.146+0000 7f742199d640 1 Processor -- start 2023-10-16T20:05:36.150 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.146+0000 7f742199d640 1 -- start start 2023-10-16T20:05:36.150 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.146+0000 7f742199d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f741c0b0da0 0x7f741c155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:36.150 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.146+0000 7f742199d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f741c0b4260 0x7f741c150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:36.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.146+0000 7f742199d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f741c15a1f0 0x7f741c149d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:36.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.146+0000 7f742199d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741c156710 con 0x7f741c0b0da0 2023-10-16T20:05:36.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.146+0000 7f742199d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741c151960 con 0x7f741c15a1f0 2023-10-16T20:05:36.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.146+0000 7f742199d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f741c1510e0 con 0x7f741c0b4260 2023-10-16T20:05:36.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.146+0000 7f741affd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f741c0b0da0 0x7f741c155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:36.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.146+0000 7f741affd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f741c0b0da0 0x7f741c155d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47058/0 (socket says 172.21.15.88:47058) 2023-10-16T20:05:36.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.146+0000 7f741affd640 1 -- 172.21.15.88:0/2410910761 learned_addr learned my addr 172.21.15.88:0/2410910761 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:36.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.146+0000 7f74127fc640 1 --2- 172.21.15.88:0/2410910761 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f741c0b4260 0x7f741c150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:36.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.146+0000 7f741b7fe640 1 --2- 172.21.15.88:0/2410910761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f741c15a1f0 0x7f741c149d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:36.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.146+0000 7f741affd640 1 -- 172.21.15.88:0/2410910761 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f741c0b4260 msgr2=0x7f741c150ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:36.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.146+0000 7f741affd640 1 --2- 172.21.15.88:0/2410910761 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f741c0b4260 0x7f741c150ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:36.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.146+0000 7f741affd640 1 -- 172.21.15.88:0/2410910761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f741c15a1f0 msgr2=0x7f741c149d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:36.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.146+0000 7f741affd640 1 --2- 172.21.15.88:0/2410910761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f741c15a1f0 0x7f741c149d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:36.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.146+0000 7f741affd640 1 -- 172.21.15.88:0/2410910761 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7404009a70 con 0x7f741c0b0da0 2023-10-16T20:05:36.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.146+0000 7f741b7fe640 1 --2- 172.21.15.88:0/2410910761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f741c15a1f0 0x7f741c149d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:36.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.146+0000 7f741affd640 1 --2- 172.21.15.88:0/2410910761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f741c0b0da0 0x7f741c155d00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f740c00c500 tx=0x7f740c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:36.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.146+0000 7f7418ff9640 1 -- 172.21.15.88:0/2410910761 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c005cf0 con 0x7f741c0b0da0 2023-10-16T20:05:36.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.146+0000 7f7418ff9640 1 -- 172.21.15.88:0/2410910761 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740c005e50 con 0x7f741c0b0da0 2023-10-16T20:05:36.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.146+0000 7f742199d640 1 -- 172.21.15.88:0/2410910761 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f741c156f20 con 0x7f741c0b0da0 2023-10-16T20:05:36.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.146+0000 7f7418ff9640 1 -- 172.21.15.88:0/2410910761 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c0072f0 con 0x7f741c0b0da0 2023-10-16T20:05:36.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.146+0000 7f742199d640 1 -- 172.21.15.88:0/2410910761 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f741c156c40 con 0x7f741c0b0da0 2023-10-16T20:05:36.155 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.150+0000 7f742199d640 1 -- 172.21.15.88:0/2410910761 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f741c151710 con 0x7f741c0b0da0 2023-10-16T20:05:36.156 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.150+0000 7f7418ff9640 1 -- 172.21.15.88:0/2410910761 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f740c007450 con 0x7f741c0b0da0 2023-10-16T20:05:36.156 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.150+0000 7f7418ff9640 1 --2- 172.21.15.88:0/2410910761 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f73fc0664d0 0x7f73fc068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:36.156 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.150+0000 7f7418ff9640 1 -- 172.21.15.88:0/2410910761 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f740c083680 con 0x7f741c0b0da0 2023-10-16T20:05:36.156 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.150+0000 7f74127fc640 1 --2- 172.21.15.88:0/2410910761 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f73fc0664d0 0x7f73fc068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:36.156 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.150+0000 7f74127fc640 1 --2- 172.21.15.88:0/2410910761 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f73fc0664d0 0x7f73fc068990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f74040af000 tx=0x7f74040a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:36.157 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.154+0000 7f7418ff9640 1 -- 172.21.15.88:0/2410910761 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f740c0508b0 con 0x7f741c0b0da0 2023-10-16T20:05:36.306 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.306+0000 7f742199d640 1 -- 172.21.15.88:0/2410910761 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f741c04e9c0 con 0x7f741c0b0da0 2023-10-16T20:05:36.307 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.306+0000 7f7418ff9640 1 -- 172.21.15.88:0/2410910761 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f740c053ed0 con 0x7f741c0b0da0 2023-10-16T20:05:36.307 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2023-10-16T20:05:36.308 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.306+0000 7f742199d640 1 -- 172.21.15.88:0/2410910761 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f73fc0664d0 msgr2=0x7f73fc068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:36.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.306+0000 7f742199d640 1 --2- 172.21.15.88:0/2410910761 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f73fc0664d0 0x7f73fc068990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f74040af000 tx=0x7f74040a7070 comp rx=0 tx=0).stop 2023-10-16T20:05:36.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.306+0000 7f742199d640 1 -- 172.21.15.88:0/2410910761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f741c0b0da0 msgr2=0x7f741c155d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:36.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.306+0000 7f742199d640 1 --2- 172.21.15.88:0/2410910761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f741c0b0da0 0x7f741c155d00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f740c00c500 tx=0x7f740c00c9c0 comp rx=0 tx=0).stop 2023-10-16T20:05:36.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.306+0000 7f742199d640 1 -- 172.21.15.88:0/2410910761 shutdown_connections 2023-10-16T20:05:36.310 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.306+0000 7f742199d640 1 --2- 172.21.15.88:0/2410910761 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f73fc0664d0 0x7f73fc068990 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:36.310 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.306+0000 7f742199d640 1 --2- 172.21.15.88:0/2410910761 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f741c0b4260 0x7f741c150ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:36.310 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.306+0000 7f742199d640 1 --2- 172.21.15.88:0/2410910761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f741c15a1f0 0x7f741c149d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:36.310 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.306+0000 7f742199d640 1 --2- 172.21.15.88:0/2410910761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f741c0b0da0 0x7f741c155d00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:36.310 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.306+0000 7f742199d640 1 -- 172.21.15.88:0/2410910761 >> 172.21.15.88:0/2410910761 conn(0x7f741c0cb080 msgr2=0x7f741c0b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:36.311 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.306+0000 7f742199d640 1 -- 172.21.15.88:0/2410910761 shutdown_connections 2023-10-16T20:05:36.311 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.306+0000 7f742199d640 1 -- 172.21.15.88:0/2410910761 wait complete. 2023-10-16T20:05:36.328 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T20:05:36.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.473+0000 7f5cdd446640 1 Processor -- start 2023-10-16T20:05:36.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.473+0000 7f5cdd446640 1 -- start start 2023-10-16T20:05:36.475 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.473+0000 7f5cdd446640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5cd815aa50 0x7f5cd80ca420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:36.475 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.473+0000 7f5cdd446640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5cd80ca960 0x7f5cd80ad0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:36.476 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.473+0000 7f5cdd446640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cd80cae00 con 0x7f5cd806c2f0 2023-10-16T20:05:36.476 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.473+0000 7f5cdd446640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd80cb0e0 con 0x7f5cd80ca960 2023-10-16T20:05:36.476 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.473+0000 7f5cdd446640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd80cb3c0 con 0x7f5cd815aa50 2023-10-16T20:05:36.476 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.473+0000 7f5cd7fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5cd80ca960 0x7f5cd80ad0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:36.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.473+0000 7f5cd7fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5cd80ca960 0x7f5cd80ad0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47070/0 (socket says 172.21.15.88:47070) 2023-10-16T20:05:36.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.473+0000 7f5cd7fff640 1 -- 172.21.15.88:0/328280514 learned_addr learned my addr 172.21.15.88:0/328280514 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:36.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.473+0000 7f5cd6ffd640 1 --2- 172.21.15.88:0/328280514 >> v2:172.21.15.94:3300/0 conn(0x7f5cd815aa50 0x7f5cd80ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:36.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.473+0000 7f5cd7fff640 1 -- 172.21.15.88:0/328280514 >> v2:172.21.15.94:3300/0 conn(0x7f5cd815aa50 msgr2=0x7f5cd80ca420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:36.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.473+0000 7f5cd7fff640 1 --2- 172.21.15.88:0/328280514 >> v2:172.21.15.94:3300/0 conn(0x7f5cd815aa50 0x7f5cd80ca420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:36.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.473+0000 7f5cd7fff640 1 -- 172.21.15.88:0/328280514 >> v1:172.21.15.94:6789/0 conn(0x7f5cd806c2f0 legacy=0x7f5cd80ac990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:36.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.473+0000 7f5cd7fff640 1 -- 172.21.15.88:0/328280514 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd80cb6a0 con 0x7f5cd80ca960 2023-10-16T20:05:36.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.477+0000 7f5cd7fff640 1 --2- 172.21.15.88:0/328280514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cd80ca960 0x7f5cd80ad0a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8071530 tx=0x7f5cd001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=604948a8555ea392 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:36.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.477+0000 7f5cd5ffb640 1 -- 172.21.15.88:0/328280514 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd00a5880 con 0x7f5cd80ca960 2023-10-16T20:05:36.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.477+0000 7f5cd5ffb640 1 -- 172.21.15.88:0/328280514 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd00a59e0 con 0x7f5cd80ca960 2023-10-16T20:05:36.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.477+0000 7f5cd5ffb640 1 -- 172.21.15.88:0/328280514 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd00a5d70 con 0x7f5cd80ca960 2023-10-16T20:05:36.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.477+0000 7f5cdd446640 1 -- 172.21.15.88:0/328280514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cd80ca960 msgr2=0x7f5cd80ad0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:36.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.477+0000 7f5cdd446640 1 --2- 172.21.15.88:0/328280514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cd80ca960 0x7f5cd80ad0a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8071530 tx=0x7f5cd001c9a0 comp rx=0 tx=0).stop 2023-10-16T20:05:36.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.477+0000 7f5cdd446640 1 -- 172.21.15.88:0/328280514 shutdown_connections 2023-10-16T20:05:36.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.477+0000 7f5cdd446640 1 --2- 172.21.15.88:0/328280514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cd80ca960 0x7f5cd80ad0a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:36.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.477+0000 7f5cdd446640 1 --2- 172.21.15.88:0/328280514 >> v2:172.21.15.94:3300/0 conn(0x7f5cd815aa50 0x7f5cd80ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:36.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.477+0000 7f5cdd446640 1 -- 172.21.15.88:0/328280514 >> 172.21.15.88:0/328280514 conn(0x7f5cd80cfdd0 msgr2=0x7f5cd80d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:36.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.477+0000 7f5cdd446640 1 -- 172.21.15.88:0/328280514 shutdown_connections 2023-10-16T20:05:36.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.477+0000 7f5cdd446640 1 -- 172.21.15.88:0/328280514 wait complete. 2023-10-16T20:05:36.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.477+0000 7f5cdd446640 1 Processor -- start 2023-10-16T20:05:36.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.477+0000 7f5cdd446640 1 -- start start 2023-10-16T20:05:36.484 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.481+0000 7f5cdd446640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cd806c2f0 0x7f5cd81f5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:36.484 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.481+0000 7f5cdd446640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cd80ca960 0x7f5cd8151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:36.484 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.481+0000 7f5cdd446640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5cd815aa50 0x7f5cd8162e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:36.484 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.481+0000 7f5cdd446640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd8163f90 con 0x7f5cd80ca960 2023-10-16T20:05:36.484 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.481+0000 7f5cdd446640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd81636f0 con 0x7f5cd806c2f0 2023-10-16T20:05:36.485 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.481+0000 7f5cdd446640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd81639d0 con 0x7f5cd815aa50 2023-10-16T20:05:36.485 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.481+0000 7f5cd67fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cd80ca960 0x7f5cd8151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:36.485 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.481+0000 7f5cd67fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cd80ca960 0x7f5cd8151da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47074/0 (socket says 172.21.15.88:47074) 2023-10-16T20:05:36.485 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.481+0000 7f5cd67fc640 1 -- 172.21.15.88:0/3808120967 learned_addr learned my addr 172.21.15.88:0/3808120967 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:36.486 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.481+0000 7f5cd6ffd640 1 --2- 172.21.15.88:0/3808120967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cd806c2f0 0x7f5cd81f5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:36.486 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.481+0000 7f5cd7fff640 1 --2- 172.21.15.88:0/3808120967 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5cd815aa50 0x7f5cd8162e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:36.486 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.481+0000 7f5cd67fc640 1 -- 172.21.15.88:0/3808120967 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5cd815aa50 msgr2=0x7f5cd8162e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:36.486 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.481+0000 7f5cd67fc640 1 --2- 172.21.15.88:0/3808120967 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5cd815aa50 0x7f5cd8162e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:36.486 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.481+0000 7f5cd67fc640 1 -- 172.21.15.88:0/3808120967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cd806c2f0 msgr2=0x7f5cd81f5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:36.487 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.481+0000 7f5cd67fc640 1 --2- 172.21.15.88:0/3808120967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cd806c2f0 0x7f5cd81f5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:36.487 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.481+0000 7f5cd67fc640 1 -- 172.21.15.88:0/3808120967 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd0009b30 con 0x7f5cd80ca960 2023-10-16T20:05:36.487 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.481+0000 7f5cd67fc640 1 --2- 172.21.15.88:0/3808120967 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cd80ca960 0x7f5cd8151da0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00c500 tx=0x7f5ccc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:36.487 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.481+0000 7f5cbf7fe640 1 -- 172.21.15.88:0/3808120967 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc005220 con 0x7f5cd80ca960 2023-10-16T20:05:36.487 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.481+0000 7f5cbf7fe640 1 -- 172.21.15.88:0/3808120967 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ccc005380 con 0x7f5cd80ca960 2023-10-16T20:05:36.488 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.481+0000 7f5cbf7fe640 1 -- 172.21.15.88:0/3808120967 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc007270 con 0x7f5cd80ca960 2023-10-16T20:05:36.488 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.481+0000 7f5cdd446640 1 -- 172.21.15.88:0/3808120967 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd8163cb0 con 0x7f5cd80ca960 2023-10-16T20:05:36.488 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.481+0000 7f5cdd446640 1 -- 172.21.15.88:0/3808120967 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd81522e0 con 0x7f5cd80ca960 2023-10-16T20:05:36.489 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.481+0000 7f5cdd446640 1 -- 172.21.15.88:0/3808120967 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cd815c920 con 0x7f5cd80ca960 2023-10-16T20:05:36.489 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.481+0000 7f5cbf7fe640 1 -- 172.21.15.88:0/3808120967 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5ccc0073d0 con 0x7f5cd80ca960 2023-10-16T20:05:36.489 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.485+0000 7f5cbf7fe640 1 --2- 172.21.15.88:0/3808120967 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5cc40664a0 0x7f5cc4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:36.489 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.485+0000 7f5cbf7fe640 1 -- 172.21.15.88:0/3808120967 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5ccc0839c0 con 0x7f5cd80ca960 2023-10-16T20:05:36.490 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.485+0000 7f5cd6ffd640 1 --2- 172.21.15.88:0/3808120967 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5cc40664a0 0x7f5cc4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:36.490 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.485+0000 7f5cd6ffd640 1 --2- 172.21.15.88:0/3808120967 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5cc40664a0 0x7f5cc4068960 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8009ac0 tx=0x7f5cc8009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:36.490 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.489+0000 7f5cbf7fe640 1 -- 172.21.15.88:0/3808120967 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5cd815c920 con 0x7f5cd80ca960 2023-10-16T20:05:36.640 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.637+0000 7f5cdd446640 1 -- 172.21.15.88:0/3808120967 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5cd815c920 con 0x7f5cd80ca960 2023-10-16T20:05:36.641 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.637+0000 7f5cbf7fe640 1 -- 172.21.15.88:0/3808120967 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5cd815c920 con 0x7f5cd80ca960 2023-10-16T20:05:36.641 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:05:36.642 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.641+0000 7f5cdd446640 1 -- 172.21.15.88:0/3808120967 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5cc40664a0 msgr2=0x7f5cc4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:36.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.641+0000 7f5cdd446640 1 --2- 172.21.15.88:0/3808120967 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5cc40664a0 0x7f5cc4068960 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8009ac0 tx=0x7f5cc8009190 comp rx=0 tx=0).stop 2023-10-16T20:05:36.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.641+0000 7f5cdd446640 1 -- 172.21.15.88:0/3808120967 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cd80ca960 msgr2=0x7f5cd8151da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:36.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.641+0000 7f5cdd446640 1 --2- 172.21.15.88:0/3808120967 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cd80ca960 0x7f5cd8151da0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00c500 tx=0x7f5ccc00c9c0 comp rx=0 tx=0).stop 2023-10-16T20:05:36.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.641+0000 7f5cdd446640 1 -- 172.21.15.88:0/3808120967 shutdown_connections 2023-10-16T20:05:36.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.641+0000 7f5cdd446640 1 --2- 172.21.15.88:0/3808120967 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5cc40664a0 0x7f5cc4068960 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:36.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.641+0000 7f5cdd446640 1 --2- 172.21.15.88:0/3808120967 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5cd815aa50 0x7f5cd8162e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:36.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.641+0000 7f5cdd446640 1 --2- 172.21.15.88:0/3808120967 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cd80ca960 0x7f5cd8151da0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:36.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.641+0000 7f5cdd446640 1 --2- 172.21.15.88:0/3808120967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cd806c2f0 0x7f5cd81f5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:36.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.641+0000 7f5cdd446640 1 -- 172.21.15.88:0/3808120967 >> 172.21.15.88:0/3808120967 conn(0x7f5cd80cfdd0 msgr2=0x7f5cd80afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:36.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.641+0000 7f5cdd446640 1 -- 172.21.15.88:0/3808120967 shutdown_connections 2023-10-16T20:05:36.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.641+0000 7f5cdd446640 1 -- 172.21.15.88:0/3808120967 wait complete. 2023-10-16T20:05:36.661 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:05:36.798 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.797+0000 7fd5fc291640 1 Processor -- start 2023-10-16T20:05:36.799 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.797+0000 7fd5fc291640 1 -- start start 2023-10-16T20:05:36.800 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.797+0000 7fd5fc291640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd5f4069e90 0x7fd5f406a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:36.800 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.797+0000 7fd5fc291640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5f406a830 con 0x7fd5f41599f0 2023-10-16T20:05:36.800 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.797+0000 7fd5fc291640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5f40b1260 con 0x7fd5f40b05a0 2023-10-16T20:05:36.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.797+0000 7fd5fc291640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5f40b3140 con 0x7fd5f4069e90 2023-10-16T20:05:36.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.797+0000 7fd5fa02d640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fd5f41599f0 0x7fd5f40b2a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:54730/0 (socket says 172.21.15.88:54730) 2023-10-16T20:05:36.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.797+0000 7fd5fa02d640 1 -- 172.21.15.88:0/2145270493 learned_addr learned my addr 172.21.15.88:0/2145270493 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:36.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.797+0000 7fd5f982c640 1 --2- 172.21.15.88:0/2145270493 >> v2:172.21.15.88:3300/0 conn(0x7fd5f4069e90 0x7fd5f406a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:36.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.797+0000 7fd5f902b640 1 -- 172.21.15.88:0/2145270493 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3197680458 0 0) 0x7fd5f406a830 con 0x7fd5f41599f0 2023-10-16T20:05:36.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.797+0000 7fd5f902b640 1 -- 172.21.15.88:0/2145270493 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5e0005740 con 0x7fd5f41599f0 2023-10-16T20:05:36.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.797+0000 7fd5f982c640 1 -- 172.21.15.88:0/2145270493 >> v1:172.21.15.94:6789/0 conn(0x7fd5f40b05a0 legacy=0x7fd5f40b0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:36.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.797+0000 7fd5f982c640 1 -- 172.21.15.88:0/2145270493 >> v1:172.21.15.88:6789/0 conn(0x7fd5f41599f0 legacy=0x7fd5f40b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:36.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.797+0000 7fd5f982c640 1 -- 172.21.15.88:0/2145270493 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f40ae960 con 0x7fd5f4069e90 2023-10-16T20:05:36.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.801+0000 7fd5f982c640 1 --2- 172.21.15.88:0/2145270493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5f4069e90 0x7fd5f406a2f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40097c0 tx=0x7fd5e401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd5b2070de0b592 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:36.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.801+0000 7fd5f902b640 1 -- 172.21.15.88:0/2145270493 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e40ab610 con 0x7fd5f4069e90 2023-10-16T20:05:36.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.801+0000 7fd5f902b640 1 -- 172.21.15.88:0/2145270493 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e40ab770 con 0x7fd5f4069e90 2023-10-16T20:05:36.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.801+0000 7fd5f902b640 1 -- 172.21.15.88:0/2145270493 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e40abb00 con 0x7fd5f4069e90 2023-10-16T20:05:36.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.801+0000 7fd5fc291640 1 -- 172.21.15.88:0/2145270493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5f4069e90 msgr2=0x7fd5f406a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:36.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.801+0000 7fd5fc291640 1 --2- 172.21.15.88:0/2145270493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5f4069e90 0x7fd5f406a2f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40097c0 tx=0x7fd5e401c410 comp rx=0 tx=0).stop 2023-10-16T20:05:36.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.801+0000 7fd5fc291640 1 -- 172.21.15.88:0/2145270493 shutdown_connections 2023-10-16T20:05:36.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.801+0000 7fd5fc291640 1 --2- 172.21.15.88:0/2145270493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5f4069e90 0x7fd5f406a2f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:36.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.801+0000 7fd5fc291640 1 -- 172.21.15.88:0/2145270493 >> 172.21.15.88:0/2145270493 conn(0x7fd5f40ce5e0 msgr2=0x7fd5f40ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:36.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.801+0000 7fd5fc291640 1 -- 172.21.15.88:0/2145270493 shutdown_connections 2023-10-16T20:05:36.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.801+0000 7fd5fc291640 1 -- 172.21.15.88:0/2145270493 wait complete. 2023-10-16T20:05:36.808 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.805+0000 7fd5fc291640 1 Processor -- start 2023-10-16T20:05:36.808 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.805+0000 7fd5fc291640 1 -- start start 2023-10-16T20:05:36.809 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.805+0000 7fd5fc291640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd5f4069e90 0x7fd5f4146de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:36.809 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.805+0000 7fd5fc291640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5f40b05a0 0x7fd5f4150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:36.809 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.805+0000 7fd5fc291640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5f41599f0 0x7fd5f4151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:36.809 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.805+0000 7fd5fc291640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f4147320 con 0x7fd5f40b05a0 2023-10-16T20:05:36.809 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.805+0000 7fd5fc291640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f4151c30 con 0x7fd5f41599f0 2023-10-16T20:05:36.810 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.805+0000 7fd5fc291640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f414dcb0 con 0x7fd5f4069e90 2023-10-16T20:05:36.810 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.805+0000 7fd5f982c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5f40b05a0 0x7fd5f4150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:36.810 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.805+0000 7fd5f982c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5f40b05a0 0x7fd5f4150de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47088/0 (socket says 172.21.15.88:47088) 2023-10-16T20:05:36.810 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.805+0000 7fd5f982c640 1 -- 172.21.15.88:0/3178894844 learned_addr learned my addr 172.21.15.88:0/3178894844 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:36.810 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.805+0000 7fd5fa02d640 1 --2- 172.21.15.88:0/3178894844 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd5f4069e90 0x7fd5f4146de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:36.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.805+0000 7fd5fa82e640 1 --2- 172.21.15.88:0/3178894844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5f41599f0 0x7fd5f4151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:36.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.805+0000 7fd5f982c640 1 -- 172.21.15.88:0/3178894844 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd5f4069e90 msgr2=0x7fd5f4146de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:36.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.805+0000 7fd5f982c640 1 --2- 172.21.15.88:0/3178894844 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd5f4069e90 0x7fd5f4146de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:36.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.805+0000 7fd5f982c640 1 -- 172.21.15.88:0/3178894844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5f41599f0 msgr2=0x7fd5f4151320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:36.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.805+0000 7fd5f982c640 1 --2- 172.21.15.88:0/3178894844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5f41599f0 0x7fd5f4151320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:36.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.805+0000 7fd5f982c640 1 -- 172.21.15.88:0/3178894844 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e4009420 con 0x7fd5f40b05a0 2023-10-16T20:05:36.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.805+0000 7fd5fa82e640 1 --2- 172.21.15.88:0/3178894844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5f41599f0 0x7fd5f4151320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:36.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.805+0000 7fd5f982c640 1 --2- 172.21.15.88:0/3178894844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5f40b05a0 0x7fd5f4150de0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40ad040 tx=0x7fd5e401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:36.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.805+0000 7fd5eaffd640 1 -- 172.21.15.88:0/3178894844 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e40bf610 con 0x7fd5f40b05a0 2023-10-16T20:05:36.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.805+0000 7fd5eaffd640 1 -- 172.21.15.88:0/3178894844 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e40bf770 con 0x7fd5f40b05a0 2023-10-16T20:05:36.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.805+0000 7fd5eaffd640 1 -- 172.21.15.88:0/3178894844 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e40bfb00 con 0x7fd5f40b05a0 2023-10-16T20:05:36.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.805+0000 7fd5fc291640 1 -- 172.21.15.88:0/3178894844 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f40cc600 con 0x7fd5f40b05a0 2023-10-16T20:05:36.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.805+0000 7fd5fc291640 1 -- 172.21.15.88:0/3178894844 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f40ccbc0 con 0x7fd5f40b05a0 2023-10-16T20:05:36.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.809+0000 7fd5fc291640 1 -- 172.21.15.88:0/3178894844 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5f40ccdb0 con 0x7fd5f40b05a0 2023-10-16T20:05:36.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.809+0000 7fd5eaffd640 1 -- 172.21.15.88:0/3178894844 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd5e40b5400 con 0x7fd5f40b05a0 2023-10-16T20:05:36.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.809+0000 7fd5eaffd640 1 --2- 172.21.15.88:0/3178894844 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd5d80664a0 0x7fd5d8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:36.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.809+0000 7fd5eaffd640 1 -- 172.21.15.88:0/3178894844 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd5e411ec70 con 0x7fd5f40b05a0 2023-10-16T20:05:36.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.809+0000 7fd5fa02d640 1 --2- 172.21.15.88:0/3178894844 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd5d80664a0 0x7fd5d8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:36.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.809+0000 7fd5fa02d640 1 --2- 172.21.15.88:0/3178894844 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd5d80664a0 0x7fd5d8068960 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd5f00011f0 tx=0x7fd5f0007d40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:36.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.813+0000 7fd5eaffd640 1 -- 172.21.15.88:0/3178894844 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd5e40ebfc0 con 0x7fd5f40b05a0 2023-10-16T20:05:36.965 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.965+0000 7fd5fc291640 1 -- 172.21.15.88:0/3178894844 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5f40b1260 con 0x7fd5f40b05a0 2023-10-16T20:05:36.967 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.965+0000 7fd5eaffd640 1 -- 172.21.15.88:0/3178894844 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18274 (secure 0 0 0) 0x7fd5f40b1260 con 0x7fd5f40b05a0 2023-10-16T20:05:36.967 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:36.967 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:05:01.835630+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T20:04:54.442487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T20:04:55.766641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T20:04:59.475830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:05:36.969 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.965+0000 7fd5fc291640 1 -- 172.21.15.88:0/3178894844 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd5d80664a0 msgr2=0x7fd5d8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:36.969 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.965+0000 7fd5fc291640 1 --2- 172.21.15.88:0/3178894844 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd5d80664a0 0x7fd5d8068960 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd5f00011f0 tx=0x7fd5f0007d40 comp rx=0 tx=0).stop 2023-10-16T20:05:36.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.965+0000 7fd5fc291640 1 -- 172.21.15.88:0/3178894844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5f40b05a0 msgr2=0x7fd5f4150de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:36.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.965+0000 7fd5fc291640 1 --2- 172.21.15.88:0/3178894844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5f40b05a0 0x7fd5f4150de0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40ad040 tx=0x7fd5e401ac50 comp rx=0 tx=0).stop 2023-10-16T20:05:36.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.965+0000 7fd5fc291640 1 -- 172.21.15.88:0/3178894844 shutdown_connections 2023-10-16T20:05:36.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.965+0000 7fd5fc291640 1 --2- 172.21.15.88:0/3178894844 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd5d80664a0 0x7fd5d8068960 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:36.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.965+0000 7fd5fc291640 1 --2- 172.21.15.88:0/3178894844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5f41599f0 0x7fd5f4151320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:36.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.965+0000 7fd5fc291640 1 --2- 172.21.15.88:0/3178894844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5f40b05a0 0x7fd5f4150de0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:36.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.965+0000 7fd5fc291640 1 --2- 172.21.15.88:0/3178894844 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd5f4069e90 0x7fd5f4146de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:36.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.965+0000 7fd5fc291640 1 -- 172.21.15.88:0/3178894844 >> 172.21.15.88:0/3178894844 conn(0x7fd5f40ce5e0 msgr2=0x7fd5f4072790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:36.972 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.965+0000 7fd5fc291640 1 -- 172.21.15.88:0/3178894844 shutdown_connections 2023-10-16T20:05:36.972 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:36.965+0000 7fd5fc291640 1 -- 172.21.15.88:0/3178894844 wait complete. 2023-10-16T20:05:36.988 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T20:04:54.442487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T20:04:55.766641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T20:04:59.475830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T20:05:36.990 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:05:37.131 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.129+0000 7f680e874640 1 Processor -- start 2023-10-16T20:05:37.132 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.129+0000 7f680e874640 1 -- start start 2023-10-16T20:05:37.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.129+0000 7f680e874640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f68080cb5f0 0x7f6808070ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:37.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.129+0000 7f680e874640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68080b0030 con 0x7f680815a730 2023-10-16T20:05:37.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.129+0000 7f680e874640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68080b0310 con 0x7f68080b1360 2023-10-16T20:05:37.134 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.129+0000 7f680e874640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68080b0cb0 con 0x7f68080cb5f0 2023-10-16T20:05:37.134 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.129+0000 7f680d071640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f68080cb5f0 0x7f6808070ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:37.134 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.129+0000 7f680d872640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f680815a730 0x7f68080caee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:54736/0 (socket says 172.21.15.88:54736) 2023-10-16T20:05:37.134 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.129+0000 7f680d872640 1 -- 172.21.15.88:0/616950190 learned_addr learned my addr 172.21.15.88:0/616950190 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:37.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.133+0000 7f680d071640 1 -- 172.21.15.88:0/616950190 >> v1:172.21.15.94:6789/0 conn(0x7f68080b1360 legacy=0x7f680815e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:37.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.133+0000 7f680d071640 1 -- 172.21.15.88:0/616950190 >> v1:172.21.15.88:6789/0 conn(0x7f680815a730 legacy=0x7f68080caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:37.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.133+0000 7f680c870640 1 -- 172.21.15.88:0/616950190 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2500346166 0 0) 0x7f68080b0030 con 0x7f680815a730 2023-10-16T20:05:37.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.133+0000 7f680d071640 1 -- 172.21.15.88:0/616950190 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68080b05f0 con 0x7f68080cb5f0 2023-10-16T20:05:37.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.133+0000 7f680d071640 1 --2- 172.21.15.88:0/616950190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68080cb5f0 0x7f6808070ab0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f67f80097c0 tx=0x7f67f801c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f45f36535d70d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:37.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.133+0000 7f680c870640 1 -- 172.21.15.88:0/616950190 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f80ab600 con 0x7f68080cb5f0 2023-10-16T20:05:37.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.133+0000 7f680c870640 1 -- 172.21.15.88:0/616950190 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f80a7050 con 0x7f68080cb5f0 2023-10-16T20:05:37.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.133+0000 7f680c870640 1 -- 172.21.15.88:0/616950190 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f80ab980 con 0x7f68080cb5f0 2023-10-16T20:05:37.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.133+0000 7f680e874640 1 -- 172.21.15.88:0/616950190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68080cb5f0 msgr2=0x7f6808070ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:37.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.133+0000 7f680e874640 1 --2- 172.21.15.88:0/616950190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68080cb5f0 0x7f6808070ab0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f67f80097c0 tx=0x7f67f801c330 comp rx=0 tx=0).stop 2023-10-16T20:05:37.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.133+0000 7f680e874640 1 -- 172.21.15.88:0/616950190 shutdown_connections 2023-10-16T20:05:37.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.133+0000 7f680e874640 1 --2- 172.21.15.88:0/616950190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68080cb5f0 0x7f6808070ab0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:37.139 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.137+0000 7f680e874640 1 -- 172.21.15.88:0/616950190 >> 172.21.15.88:0/616950190 conn(0x7f680806a4c0 msgr2=0x7f680806c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:37.139 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.137+0000 7f680e874640 1 -- 172.21.15.88:0/616950190 shutdown_connections 2023-10-16T20:05:37.139 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.137+0000 7f680e874640 1 -- 172.21.15.88:0/616950190 wait complete. 2023-10-16T20:05:37.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.137+0000 7f680e874640 1 Processor -- start 2023-10-16T20:05:37.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.137+0000 7f680e874640 1 -- start start 2023-10-16T20:05:37.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.137+0000 7f680e874640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68080b1360 0x7f680815b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:37.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.137+0000 7f680e874640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68080cb5f0 0x7f68081530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:37.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.137+0000 7f680e874640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f680815a730 0x7f68080c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:37.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.137+0000 7f680e874640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68080c6910 con 0x7f68080cb5f0 2023-10-16T20:05:37.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.137+0000 7f680e874640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68080c62e0 con 0x7f68080b1360 2023-10-16T20:05:37.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.137+0000 7f680e874640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f680815be70 con 0x7f680815a730 2023-10-16T20:05:37.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.137+0000 7f680d071640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68080cb5f0 0x7f68081530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:37.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.137+0000 7f680d071640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68080cb5f0 0x7f68081530c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47100/0 (socket says 172.21.15.88:47100) 2023-10-16T20:05:37.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.137+0000 7f680d071640 1 -- 172.21.15.88:0/1608274559 learned_addr learned my addr 172.21.15.88:0/1608274559 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:37.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.137+0000 7f680d872640 1 --2- 172.21.15.88:0/1608274559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68080b1360 0x7f680815b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:37.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.137+0000 7f680e073640 1 --2- 172.21.15.88:0/1608274559 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f680815a730 0x7f68080c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:37.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.137+0000 7f680d071640 1 -- 172.21.15.88:0/1608274559 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f680815a730 msgr2=0x7f68080c6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:37.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.137+0000 7f680d071640 1 --2- 172.21.15.88:0/1608274559 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f680815a730 0x7f68080c6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:37.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.137+0000 7f680d071640 1 -- 172.21.15.88:0/1608274559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68080b1360 msgr2=0x7f680815b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:37.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.137+0000 7f680d071640 1 --2- 172.21.15.88:0/1608274559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68080b1360 0x7f680815b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:37.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.137+0000 7f680d071640 1 -- 172.21.15.88:0/1608274559 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f8009420 con 0x7f68080cb5f0 2023-10-16T20:05:37.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.137+0000 7f680d071640 1 --2- 172.21.15.88:0/1608274559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68080cb5f0 0x7f68081530c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f67f80ae040 tx=0x7f67f80190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:37.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.141+0000 7f67fe7fc640 1 -- 172.21.15.88:0/1608274559 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f80b4840 con 0x7f68080cb5f0 2023-10-16T20:05:37.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.141+0000 7f67fe7fc640 1 -- 172.21.15.88:0/1608274559 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f80b49c0 con 0x7f68080cb5f0 2023-10-16T20:05:37.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.141+0000 7f680e874640 1 -- 172.21.15.88:0/1608274559 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68080c8250 con 0x7f68080cb5f0 2023-10-16T20:05:37.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.141+0000 7f67fe7fc640 1 -- 172.21.15.88:0/1608274559 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f80b4d70 con 0x7f68080cb5f0 2023-10-16T20:05:37.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.141+0000 7f680e874640 1 -- 172.21.15.88:0/1608274559 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68080c6530 con 0x7f68080cb5f0 2023-10-16T20:05:37.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.141+0000 7f680e874640 1 -- 172.21.15.88:0/1608274559 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68080674f0 con 0x7f68080cb5f0 2023-10-16T20:05:37.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.141+0000 7f67fe7fc640 1 -- 172.21.15.88:0/1608274559 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f67f80b2040 con 0x7f68080cb5f0 2023-10-16T20:05:37.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.141+0000 7f67fe7fc640 1 --2- 172.21.15.88:0/1608274559 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f67ec0664a0 0x7f67ec068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:37.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.141+0000 7f67fe7fc640 1 -- 172.21.15.88:0/1608274559 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f67f811e440 con 0x7f68080cb5f0 2023-10-16T20:05:37.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.141+0000 7f680d872640 1 --2- 172.21.15.88:0/1608274559 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f67ec0664a0 0x7f67ec068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:37.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.145+0000 7f680d872640 1 --2- 172.21.15.88:0/1608274559 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f67ec0664a0 0x7f67ec068960 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6804000f70 tx=0x7f6804007240 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:37.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.145+0000 7f67fe7fc640 1 -- 172.21.15.88:0/1608274559 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f67f80eb670 con 0x7f68080cb5f0 2023-10-16T20:05:37.308 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.305+0000 7f680e874640 1 -- 172.21.15.88:0/1608274559 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f680806cee0 con 0x7f68080cb5f0 2023-10-16T20:05:37.308 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.305+0000 7f67fe7fc640 1 -- 172.21.15.88:0/1608274559 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f67f80b8020 con 0x7f68080cb5f0 2023-10-16T20:05:37.308 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:05:37.310 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.309+0000 7f680e874640 1 -- 172.21.15.88:0/1608274559 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f67ec0664a0 msgr2=0x7f67ec068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:37.310 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.309+0000 7f680e874640 1 --2- 172.21.15.88:0/1608274559 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f67ec0664a0 0x7f67ec068960 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6804000f70 tx=0x7f6804007240 comp rx=0 tx=0).stop 2023-10-16T20:05:37.310 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.309+0000 7f680e874640 1 -- 172.21.15.88:0/1608274559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68080cb5f0 msgr2=0x7f68081530c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:37.310 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.309+0000 7f680e874640 1 --2- 172.21.15.88:0/1608274559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68080cb5f0 0x7f68081530c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f67f80ae040 tx=0x7f67f80190d0 comp rx=0 tx=0).stop 2023-10-16T20:05:37.311 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.309+0000 7f680e874640 1 -- 172.21.15.88:0/1608274559 shutdown_connections 2023-10-16T20:05:37.311 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.309+0000 7f680e874640 1 --2- 172.21.15.88:0/1608274559 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f67ec0664a0 0x7f67ec068960 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:37.311 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.309+0000 7f680e874640 1 --2- 172.21.15.88:0/1608274559 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f680815a730 0x7f68080c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:37.311 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.309+0000 7f680e874640 1 --2- 172.21.15.88:0/1608274559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68080cb5f0 0x7f68081530c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:37.311 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.309+0000 7f680e874640 1 --2- 172.21.15.88:0/1608274559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68080b1360 0x7f680815b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:37.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.309+0000 7f680e874640 1 -- 172.21.15.88:0/1608274559 >> 172.21.15.88:0/1608274559 conn(0x7f680806a4c0 msgr2=0x7f68080b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:37.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.309+0000 7f680e874640 1 -- 172.21.15.88:0/1608274559 shutdown_connections 2023-10-16T20:05:37.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.309+0000 7f680e874640 1 -- 172.21.15.88:0/1608274559 wait complete. 2023-10-16T20:05:37.329 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T20:05:37.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.473+0000 7f144f19d640 1 Processor -- start 2023-10-16T20:05:37.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.473+0000 7f144f19d640 1 -- start start 2023-10-16T20:05:37.475 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.473+0000 7f144f19d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f144815aae0 0x7f14480d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:37.476 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.473+0000 7f144f19d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14480b6270 0x7f14480cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:37.476 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.473+0000 7f144f19d640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14480b6700 con 0x7f144815aec0 2023-10-16T20:05:37.476 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.473+0000 7f144f19d640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f144815ea10 con 0x7f144815aae0 2023-10-16T20:05:37.476 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.473+0000 7f144f19d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f144815ecf0 con 0x7f14480b6270 2023-10-16T20:05:37.476 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.473+0000 7f144cf39640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f144815aae0 0x7f14480d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:37.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.473+0000 7f144cf39640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f144815aae0 0x7f14480d0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47102/0 (socket says 172.21.15.88:47102) 2023-10-16T20:05:37.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.473+0000 7f144cf39640 1 -- 172.21.15.88:0/2118341142 learned_addr learned my addr 172.21.15.88:0/2118341142 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:37.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.473+0000 7f143ffff640 1 --2- 172.21.15.88:0/2118341142 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14480b6270 0x7f14480cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:37.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.473+0000 7f144cf39640 1 -- 172.21.15.88:0/2118341142 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14480b6270 msgr2=0x7f14480cb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:37.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.473+0000 7f144cf39640 1 --2- 172.21.15.88:0/2118341142 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14480b6270 0x7f14480cb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:37.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.473+0000 7f144cf39640 1 -- 172.21.15.88:0/2118341142 >> v1:172.21.15.94:6789/0 conn(0x7f144815aec0 legacy=0x7f14480ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:37.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.473+0000 7f144cf39640 1 -- 172.21.15.88:0/2118341142 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f144815efd0 con 0x7f144815aae0 2023-10-16T20:05:37.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.477+0000 7f144cf39640 1 --2- 172.21.15.88:0/2118341142 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f144815aae0 0x7f14480d0380 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1444009870 tx=0x7f144401ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290c895547e6def9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:37.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.477+0000 7f143f7fe640 1 -- 172.21.15.88:0/2118341142 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14440a8a30 con 0x7f144815aae0 2023-10-16T20:05:37.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.477+0000 7f143f7fe640 1 -- 172.21.15.88:0/2118341142 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14440a8b90 con 0x7f144815aae0 2023-10-16T20:05:37.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.477+0000 7f143f7fe640 1 -- 172.21.15.88:0/2118341142 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14440b0440 con 0x7f144815aae0 2023-10-16T20:05:37.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.477+0000 7f144f19d640 1 -- 172.21.15.88:0/2118341142 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f144815aae0 msgr2=0x7f14480d0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:37.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.477+0000 7f144f19d640 1 --2- 172.21.15.88:0/2118341142 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f144815aae0 0x7f14480d0380 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1444009870 tx=0x7f144401ed30 comp rx=0 tx=0).stop 2023-10-16T20:05:37.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.477+0000 7f144f19d640 1 -- 172.21.15.88:0/2118341142 shutdown_connections 2023-10-16T20:05:37.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.477+0000 7f144f19d640 1 --2- 172.21.15.88:0/2118341142 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14480b6270 0x7f14480cb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:37.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.477+0000 7f144f19d640 1 --2- 172.21.15.88:0/2118341142 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f144815aae0 0x7f14480d0380 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:37.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.477+0000 7f144f19d640 1 -- 172.21.15.88:0/2118341142 >> 172.21.15.88:0/2118341142 conn(0x7f14480cd3f0 msgr2=0x7f14480cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:37.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.477+0000 7f144f19d640 1 -- 172.21.15.88:0/2118341142 shutdown_connections 2023-10-16T20:05:37.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.477+0000 7f144f19d640 1 -- 172.21.15.88:0/2118341142 wait complete. 2023-10-16T20:05:37.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.477+0000 7f144f19d640 1 Processor -- start 2023-10-16T20:05:37.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.477+0000 7f144f19d640 1 -- start start 2023-10-16T20:05:37.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.481+0000 7f144f19d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f144815aae0 0x7f144815db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:37.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.481+0000 7f144f19d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f144815aec0 0x7f144814d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:37.484 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.481+0000 7f144f19d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f144814a4c0 0x7f14480b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:37.484 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.481+0000 7f144f19d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144814ded0 con 0x7f144814a4c0 2023-10-16T20:05:37.484 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.481+0000 7f144f19d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144814aef0 con 0x7f144815aae0 2023-10-16T20:05:37.484 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.481+0000 7f144f19d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f144814b790 con 0x7f144815aec0 2023-10-16T20:05:37.484 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.481+0000 7f144d73a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f144814a4c0 0x7f14480b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:37.485 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.481+0000 7f144d73a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f144814a4c0 0x7f14480b44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47104/0 (socket says 172.21.15.88:47104) 2023-10-16T20:05:37.485 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.481+0000 7f144d73a640 1 -- 172.21.15.88:0/3883178921 learned_addr learned my addr 172.21.15.88:0/3883178921 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:37.485 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.481+0000 7f143ffff640 1 --2- 172.21.15.88:0/3883178921 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f144815aec0 0x7f144814d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:37.485 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.481+0000 7f144cf39640 1 --2- 172.21.15.88:0/3883178921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f144815aae0 0x7f144815db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:37.485 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.481+0000 7f144d73a640 1 -- 172.21.15.88:0/3883178921 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f144815aec0 msgr2=0x7f144814d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:37.486 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.481+0000 7f144d73a640 1 --2- 172.21.15.88:0/3883178921 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f144815aec0 0x7f144814d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:37.486 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.481+0000 7f144d73a640 1 -- 172.21.15.88:0/3883178921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f144815aae0 msgr2=0x7f144815db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:37.486 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.481+0000 7f144d73a640 1 --2- 172.21.15.88:0/3883178921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f144815aae0 0x7f144815db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:37.486 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.481+0000 7f144d73a640 1 -- 172.21.15.88:0/3883178921 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14440094d0 con 0x7f144814a4c0 2023-10-16T20:05:37.486 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.481+0000 7f144d73a640 1 --2- 172.21.15.88:0/3883178921 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f144814a4c0 0x7f14480b44f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f144000c760 tx=0x7f144000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:37.488 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.481+0000 7f143d7fa640 1 -- 172.21.15.88:0/3883178921 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1440003b80 con 0x7f144814a4c0 2023-10-16T20:05:37.488 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.481+0000 7f143d7fa640 1 -- 172.21.15.88:0/3883178921 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1440003ce0 con 0x7f144814a4c0 2023-10-16T20:05:37.488 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.481+0000 7f144f19d640 1 -- 172.21.15.88:0/3883178921 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f144815e110 con 0x7f144814a4c0 2023-10-16T20:05:37.488 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.481+0000 7f143d7fa640 1 -- 172.21.15.88:0/3883178921 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144000aa30 con 0x7f144814a4c0 2023-10-16T20:05:37.489 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.481+0000 7f144f19d640 1 -- 172.21.15.88:0/3883178921 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f144814b420 con 0x7f144814a4c0 2023-10-16T20:05:37.489 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.481+0000 7f144f19d640 1 -- 172.21.15.88:0/3883178921 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f144804e9c0 con 0x7f144814a4c0 2023-10-16T20:05:37.489 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.485+0000 7f143d7fa640 1 -- 172.21.15.88:0/3883178921 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f144000ab90 con 0x7f144814a4c0 2023-10-16T20:05:37.489 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.485+0000 7f143d7fa640 1 --2- 172.21.15.88:0/3883178921 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f14340664a0 0x7f1434068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:37.489 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.485+0000 7f143d7fa640 1 -- 172.21.15.88:0/3883178921 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1440083e20 con 0x7f144814a4c0 2023-10-16T20:05:37.490 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.485+0000 7f144cf39640 1 --2- 172.21.15.88:0/3883178921 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f14340664a0 0x7f1434068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:37.490 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.485+0000 7f144cf39640 1 --2- 172.21.15.88:0/3883178921 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f14340664a0 0x7f1434068960 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f14440b2000 tx=0x7f1444009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:37.491 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.489+0000 7f143d7fa640 1 -- 172.21.15.88:0/3883178921 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1440051000 con 0x7f144814a4c0 2023-10-16T20:05:37.641 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.637+0000 7f144f19d640 1 -- 172.21.15.88:0/3883178921 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f144814b8d0 con 0x7f144814a4c0 2023-10-16T20:05:37.641 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.637+0000 7f143d7fa640 1 -- 172.21.15.88:0/3883178921 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1440054620 con 0x7f144814a4c0 2023-10-16T20:05:37.641 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:05:37.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.641+0000 7f144f19d640 1 -- 172.21.15.88:0/3883178921 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f14340664a0 msgr2=0x7f1434068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:37.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.641+0000 7f144f19d640 1 --2- 172.21.15.88:0/3883178921 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f14340664a0 0x7f1434068960 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f14440b2000 tx=0x7f1444009040 comp rx=0 tx=0).stop 2023-10-16T20:05:37.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.641+0000 7f144f19d640 1 -- 172.21.15.88:0/3883178921 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f144814a4c0 msgr2=0x7f14480b44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:37.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.641+0000 7f144f19d640 1 --2- 172.21.15.88:0/3883178921 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f144814a4c0 0x7f14480b44f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f144000c760 tx=0x7f144000cc20 comp rx=0 tx=0).stop 2023-10-16T20:05:37.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.641+0000 7f144f19d640 1 -- 172.21.15.88:0/3883178921 shutdown_connections 2023-10-16T20:05:37.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.641+0000 7f144f19d640 1 --2- 172.21.15.88:0/3883178921 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f14340664a0 0x7f1434068960 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:37.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.641+0000 7f144f19d640 1 --2- 172.21.15.88:0/3883178921 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f144815aec0 0x7f144814d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:37.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.641+0000 7f144f19d640 1 --2- 172.21.15.88:0/3883178921 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f144814a4c0 0x7f14480b44f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:37.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.641+0000 7f144f19d640 1 --2- 172.21.15.88:0/3883178921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f144815aae0 0x7f144815db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:37.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.641+0000 7f144f19d640 1 -- 172.21.15.88:0/3883178921 >> 172.21.15.88:0/3883178921 conn(0x7f14480cd3f0 msgr2=0x7f14480caf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:37.646 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.641+0000 7f144f19d640 1 -- 172.21.15.88:0/3883178921 shutdown_connections 2023-10-16T20:05:37.646 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.641+0000 7f144f19d640 1 -- 172.21.15.88:0/3883178921 wait complete. 2023-10-16T20:05:37.662 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T20:05:37.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.801+0000 7ff5b3d3e640 1 Processor -- start 2023-10-16T20:05:37.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.801+0000 7ff5b3d3e640 1 -- start start 2023-10-16T20:05:37.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.801+0000 7ff5b3d3e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff5ac164570 0x7ff5ac0af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:37.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.801+0000 7ff5b3d3e640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5ac0ae840 con 0x7ff5ac070250 2023-10-16T20:05:37.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.801+0000 7ff5b3d3e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5ac0aeb20 con 0x7ff5ac0afe20 2023-10-16T20:05:37.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.801+0000 7ff5b3d3e640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5ac0aee00 con 0x7ff5ac164570 2023-10-16T20:05:37.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.801+0000 7ff5b22db640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7ff5ac070250 0x7ff5ac0b17c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:54740/0 (socket says 172.21.15.88:54740) 2023-10-16T20:05:37.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.801+0000 7ff5b22db640 1 -- 172.21.15.88:0/3569912849 learned_addr learned my addr 172.21.15.88:0/3569912849 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:37.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.801+0000 7ff5b1ada640 1 --2- 172.21.15.88:0/3569912849 >> v2:172.21.15.94:3300/0 conn(0x7ff5ac164570 0x7ff5ac0af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:37.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.801+0000 7ff5b0ad8640 1 -- 172.21.15.88:0/3569912849 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2869567193 0 0) 0x7ff5ac0ae840 con 0x7ff5ac070250 2023-10-16T20:05:37.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.801+0000 7ff5b0ad8640 1 -- 172.21.15.88:0/3569912849 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff598003530 con 0x7ff5ac070250 2023-10-16T20:05:37.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.801+0000 7ff5b0ad8640 1 -- 172.21.15.88:0/3569912849 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1368154587 0 0) 0x7ff598003530 con 0x7ff5ac070250 2023-10-16T20:05:37.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.801+0000 7ff5b1ada640 1 -- 172.21.15.88:0/3569912849 >> v1:172.21.15.94:6789/0 conn(0x7ff5ac0afe20 legacy=0x7ff5ac0addf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:37.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.801+0000 7ff5b1ada640 1 -- 172.21.15.88:0/3569912849 >> v1:172.21.15.88:6789/0 conn(0x7ff5ac070250 legacy=0x7ff5ac0b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:37.808 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.801+0000 7ff5b1ada640 1 -- 172.21.15.88:0/3569912849 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ac0af0e0 con 0x7ff5ac164570 2023-10-16T20:05:37.809 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.805+0000 7ff5b1ada640 1 --2- 172.21.15.88:0/3569912849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5ac164570 0x7ff5ac0af8e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80097c0 tx=0x7ff5a801c2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56eb07cae4ee816f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:37.809 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.805+0000 7ff5b0ad8640 1 -- 172.21.15.88:0/3569912849 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4001a00 con 0x7ff5ac164570 2023-10-16T20:05:37.809 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.805+0000 7ff5b0ad8640 1 -- 172.21.15.88:0/3569912849 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5a80a7050 con 0x7ff5ac164570 2023-10-16T20:05:37.810 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.805+0000 7ff5b0ad8640 1 -- 172.21.15.88:0/3569912849 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a80ab600 con 0x7ff5ac164570 2023-10-16T20:05:37.810 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.805+0000 7ff5b3d3e640 1 -- 172.21.15.88:0/3569912849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5ac164570 msgr2=0x7ff5ac0af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:37.810 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.805+0000 7ff5b3d3e640 1 --2- 172.21.15.88:0/3569912849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5ac164570 0x7ff5ac0af8e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80097c0 tx=0x7ff5a801c2e0 comp rx=0 tx=0).stop 2023-10-16T20:05:37.810 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.805+0000 7ff5b3d3e640 1 -- 172.21.15.88:0/3569912849 shutdown_connections 2023-10-16T20:05:37.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.805+0000 7ff5b3d3e640 1 --2- 172.21.15.88:0/3569912849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5ac164570 0x7ff5ac0af8e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:37.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.805+0000 7ff5b3d3e640 1 -- 172.21.15.88:0/3569912849 >> 172.21.15.88:0/3569912849 conn(0x7ff5ac0cb080 msgr2=0x7ff5ac0cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:37.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.805+0000 7ff5b3d3e640 1 -- 172.21.15.88:0/3569912849 shutdown_connections 2023-10-16T20:05:37.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.805+0000 7ff5b3d3e640 1 -- 172.21.15.88:0/3569912849 wait complete. 2023-10-16T20:05:37.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.809+0000 7ff5b3d3e640 1 Processor -- start 2023-10-16T20:05:37.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.809+0000 7ff5b3d3e640 1 -- start start 2023-10-16T20:05:37.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.809+0000 7ff5b3d3e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5ac070250 0x7ff5ac1e96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:37.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.809+0000 7ff5b3d3e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5ac0afe20 0x7ff5ac06d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:37.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.809+0000 7ff5b3d3e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5ac164570 0x7ff5ac14f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:37.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.809+0000 7ff5b3d3e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac1ead20 con 0x7ff5ac070250 2023-10-16T20:05:37.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.809+0000 7ff5b3d3e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac1e9c80 con 0x7ff5ac164570 2023-10-16T20:05:37.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.809+0000 7ff5b3d3e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac1e9f60 con 0x7ff5ac0afe20 2023-10-16T20:05:37.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.809+0000 7ff5b1ada640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5ac070250 0x7ff5ac1e96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:37.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.809+0000 7ff5b1ada640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5ac070250 0x7ff5ac1e96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47106/0 (socket says 172.21.15.88:47106) 2023-10-16T20:05:37.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.809+0000 7ff5b1ada640 1 -- 172.21.15.88:0/402860959 learned_addr learned my addr 172.21.15.88:0/402860959 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:37.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.809+0000 7ff5b12d9640 1 --2- 172.21.15.88:0/402860959 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5ac0afe20 0x7ff5ac06d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:37.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.809+0000 7ff5b22db640 1 --2- 172.21.15.88:0/402860959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5ac164570 0x7ff5ac14f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:37.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.809+0000 7ff5b1ada640 1 -- 172.21.15.88:0/402860959 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5ac0afe20 msgr2=0x7ff5ac06d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:37.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.809+0000 7ff5b1ada640 1 --2- 172.21.15.88:0/402860959 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5ac0afe20 0x7ff5ac06d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:37.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.809+0000 7ff5b1ada640 1 -- 172.21.15.88:0/402860959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5ac164570 msgr2=0x7ff5ac14f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:37.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.809+0000 7ff5b1ada640 1 --2- 172.21.15.88:0/402860959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5ac164570 0x7ff5ac14f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:37.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.809+0000 7ff5b1ada640 1 -- 172.21.15.88:0/402860959 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a8009420 con 0x7ff5ac070250 2023-10-16T20:05:37.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.809+0000 7ff5b22db640 1 --2- 172.21.15.88:0/402860959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5ac164570 0x7ff5ac14f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:37.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.809+0000 7ff5b1ada640 1 --2- 172.21.15.88:0/402860959 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5ac070250 0x7ff5ac1e96a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8000c10 tx=0x7ff5a8019080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:37.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.809+0000 7ff5a27fc640 1 -- 172.21.15.88:0/402860959 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a80bb070 con 0x7ff5ac070250 2023-10-16T20:05:37.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.809+0000 7ff5a27fc640 1 -- 172.21.15.88:0/402860959 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5a80044c0 con 0x7ff5ac070250 2023-10-16T20:05:37.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.809+0000 7ff5b3d3e640 1 -- 172.21.15.88:0/402860959 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5ac1ea240 con 0x7ff5ac070250 2023-10-16T20:05:37.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.809+0000 7ff5a27fc640 1 -- 172.21.15.88:0/402860959 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a80b5610 con 0x7ff5ac070250 2023-10-16T20:05:37.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.809+0000 7ff5b3d3e640 1 -- 172.21.15.88:0/402860959 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5ac1ea800 con 0x7ff5ac070250 2023-10-16T20:05:37.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.813+0000 7ff5b3d3e640 1 -- 172.21.15.88:0/402860959 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5ac04e9c0 con 0x7ff5ac070250 2023-10-16T20:05:37.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.813+0000 7ff5a27fc640 1 -- 172.21.15.88:0/402860959 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff5a80b2050 con 0x7ff5ac070250 2023-10-16T20:05:37.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.813+0000 7ff5a27fc640 1 --2- 172.21.15.88:0/402860959 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff5900664a0 0x7ff590068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:37.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.813+0000 7ff5a27fc640 1 -- 172.21.15.88:0/402860959 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff5a811e260 con 0x7ff5ac070250 2023-10-16T20:05:37.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.813+0000 7ff5b12d9640 1 --2- 172.21.15.88:0/402860959 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff5900664a0 0x7ff590068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:37.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.813+0000 7ff5b12d9640 1 --2- 172.21.15.88:0/402860959 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff5900664a0 0x7ff590068960 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff59c001ec0 tx=0x7ff59c00b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:37.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.817+0000 7ff5a27fc640 1 -- 172.21.15.88:0/402860959 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff5a80eb310 con 0x7ff5ac070250 2023-10-16T20:05:37.968 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.965+0000 7ff5b3d3e640 1 -- 172.21.15.88:0/402860959 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff5ac163510 con 0x7ff5ac070250 2023-10-16T20:05:37.969 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.965+0000 7ff5a27fc640 1 -- 172.21.15.88:0/402860959 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff5a80ee930 con 0x7ff5ac070250 2023-10-16T20:05:37.969 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2023-10-16T20:05:37.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.969+0000 7ff5b3d3e640 1 -- 172.21.15.88:0/402860959 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff5900664a0 msgr2=0x7ff590068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:37.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.969+0000 7ff5b3d3e640 1 --2- 172.21.15.88:0/402860959 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff5900664a0 0x7ff590068960 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff59c001ec0 tx=0x7ff59c00b040 comp rx=0 tx=0).stop 2023-10-16T20:05:37.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.969+0000 7ff5b3d3e640 1 -- 172.21.15.88:0/402860959 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5ac070250 msgr2=0x7ff5ac1e96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:37.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.969+0000 7ff5b3d3e640 1 --2- 172.21.15.88:0/402860959 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5ac070250 0x7ff5ac1e96a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8000c10 tx=0x7ff5a8019080 comp rx=0 tx=0).stop 2023-10-16T20:05:37.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.969+0000 7ff5b3d3e640 1 -- 172.21.15.88:0/402860959 shutdown_connections 2023-10-16T20:05:37.972 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.969+0000 7ff5b3d3e640 1 --2- 172.21.15.88:0/402860959 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff5900664a0 0x7ff590068960 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:37.972 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.969+0000 7ff5b3d3e640 1 --2- 172.21.15.88:0/402860959 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5ac0afe20 0x7ff5ac06d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:37.972 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.969+0000 7ff5b3d3e640 1 --2- 172.21.15.88:0/402860959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5ac164570 0x7ff5ac14f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:37.972 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.969+0000 7ff5b3d3e640 1 --2- 172.21.15.88:0/402860959 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5ac070250 0x7ff5ac1e96a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:37.972 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.969+0000 7ff5b3d3e640 1 -- 172.21.15.88:0/402860959 >> 172.21.15.88:0/402860959 conn(0x7ff5ac0cb080 msgr2=0x7ff5ac15dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:37.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.969+0000 7ff5b3d3e640 1 -- 172.21.15.88:0/402860959 shutdown_connections 2023-10-16T20:05:37.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:37.969+0000 7ff5b3d3e640 1 -- 172.21.15.88:0/402860959 wait complete. 2023-10-16T20:05:37.989 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T20:05:38.130 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.129+0000 7fdbfcbd0640 1 Processor -- start 2023-10-16T20:05:38.130 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.129+0000 7fdbfcbd0640 1 -- start start 2023-10-16T20:05:38.131 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.129+0000 7fdbfcbd0640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdbf815cac0 0x7fdbf80ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:38.131 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.129+0000 7fdbfcbd0640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbf806b840 con 0x7fdbf815cea0 2023-10-16T20:05:38.131 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.129+0000 7fdbfcbd0640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbf806a3a0 con 0x7fdbf80ae130 2023-10-16T20:05:38.132 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.129+0000 7fdbfcbd0640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf80cac10 con 0x7fdbf815cac0 2023-10-16T20:05:38.132 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.129+0000 7fdbf5d9b640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fdbf815cea0 0x7fdbf81607d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:54744/0 (socket says 172.21.15.88:54744) 2023-10-16T20:05:38.132 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.129+0000 7fdbf5d9b640 1 -- 172.21.15.88:0/2864773446 learned_addr learned my addr 172.21.15.88:0/2864773446 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:38.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.129+0000 7fdbf659c640 1 --2- 172.21.15.88:0/2864773446 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdbf815cac0 0x7fdbf80ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:38.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.129+0000 7fdbf559a640 1 -- 172.21.15.88:0/2864773446 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 78139036 0 0) 0x7fdbf806b840 con 0x7fdbf815cea0 2023-10-16T20:05:38.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.129+0000 7fdbf559a640 1 -- 172.21.15.88:0/2864773446 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbe0003530 con 0x7fdbf815cea0 2023-10-16T20:05:38.134 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.129+0000 7fdbf559a640 1 -- 172.21.15.88:0/2864773446 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2570441256 0 0) 0x7fdbe0003530 con 0x7fdbf815cea0 2023-10-16T20:05:38.134 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.129+0000 7fdbf659c640 1 -- 172.21.15.88:0/2864773446 >> v1:172.21.15.94:6789/0 conn(0x7fdbf80ae130 legacy=0x7fdbf8160ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:38.134 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.129+0000 7fdbf659c640 1 -- 172.21.15.88:0/2864773446 >> v1:172.21.15.88:6789/0 conn(0x7fdbf815cea0 legacy=0x7fdbf81607d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:38.134 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.129+0000 7fdbf659c640 1 -- 172.21.15.88:0/2864773446 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf81615f0 con 0x7fdbf815cac0 2023-10-16T20:05:38.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.133+0000 7fdbf659c640 1 --2- 172.21.15.88:0/2864773446 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdbf815cac0 0x7fdbf80ccc60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdbe80097f0 tx=0x7fdbe801c320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4798537c24c1e5e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:38.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.133+0000 7fdbf559a640 1 -- 172.21.15.88:0/2864773446 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec001680 con 0x7fdbf815cac0 2023-10-16T20:05:38.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.133+0000 7fdbf559a640 1 -- 172.21.15.88:0/2864773446 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe80a7050 con 0x7fdbf815cac0 2023-10-16T20:05:38.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.133+0000 7fdbf559a640 1 -- 172.21.15.88:0/2864773446 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe80ab620 con 0x7fdbf815cac0 2023-10-16T20:05:38.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.133+0000 7fdbfcbd0640 1 -- 172.21.15.88:0/2864773446 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdbf815cac0 msgr2=0x7fdbf80ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:38.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.133+0000 7fdbfcbd0640 1 --2- 172.21.15.88:0/2864773446 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdbf815cac0 0x7fdbf80ccc60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdbe80097f0 tx=0x7fdbe801c320 comp rx=0 tx=0).stop 2023-10-16T20:05:38.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.133+0000 7fdbfcbd0640 1 -- 172.21.15.88:0/2864773446 shutdown_connections 2023-10-16T20:05:38.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.133+0000 7fdbfcbd0640 1 --2- 172.21.15.88:0/2864773446 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdbf815cac0 0x7fdbf80ccc60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:38.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.133+0000 7fdbfcbd0640 1 -- 172.21.15.88:0/2864773446 >> 172.21.15.88:0/2864773446 conn(0x7fdbf80b0970 msgr2=0x7fdbf80b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:38.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.133+0000 7fdbfcbd0640 1 -- 172.21.15.88:0/2864773446 shutdown_connections 2023-10-16T20:05:38.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.133+0000 7fdbfcbd0640 1 -- 172.21.15.88:0/2864773446 wait complete. 2023-10-16T20:05:38.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.137+0000 7fdbfcbd0640 1 Processor -- start 2023-10-16T20:05:38.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.137+0000 7fdbfcbd0640 1 -- start start 2023-10-16T20:05:38.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.137+0000 7fdbfcbd0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbf80ae130 0x7fdbf815bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:38.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.137+0000 7fdbfcbd0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbf815cac0 0x7fdbf80b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:38.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.137+0000 7fdbfcbd0640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdbf815cea0 0x7fdbf80b2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:38.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.137+0000 7fdbfcbd0640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf80b4120 con 0x7fdbf80ae130 2023-10-16T20:05:38.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.137+0000 7fdbfcbd0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf80b48f0 con 0x7fdbf815cac0 2023-10-16T20:05:38.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.137+0000 7fdbfcbd0640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf80c2e80 con 0x7fdbf815cea0 2023-10-16T20:05:38.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.137+0000 7fdbf659c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbf80ae130 0x7fdbf815bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:38.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.137+0000 7fdbf659c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbf80ae130 0x7fdbf815bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47122/0 (socket says 172.21.15.88:47122) 2023-10-16T20:05:38.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.137+0000 7fdbf659c640 1 -- 172.21.15.88:0/383151493 learned_addr learned my addr 172.21.15.88:0/383151493 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:38.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.137+0000 7fdbf6d9d640 1 --2- 172.21.15.88:0/383151493 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdbf815cea0 0x7fdbf80b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:38.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.137+0000 7fdbf5d9b640 1 --2- 172.21.15.88:0/383151493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbf815cac0 0x7fdbf80b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:38.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.137+0000 7fdbf659c640 1 -- 172.21.15.88:0/383151493 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdbf815cea0 msgr2=0x7fdbf80b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:38.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.137+0000 7fdbf659c640 1 --2- 172.21.15.88:0/383151493 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdbf815cea0 0x7fdbf80b2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:38.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.137+0000 7fdbf659c640 1 -- 172.21.15.88:0/383151493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbf815cac0 msgr2=0x7fdbf80b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:38.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.137+0000 7fdbf659c640 1 --2- 172.21.15.88:0/383151493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbf815cac0 0x7fdbf80b3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:38.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.137+0000 7fdbf659c640 1 -- 172.21.15.88:0/383151493 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe8009420 con 0x7fdbf80ae130 2023-10-16T20:05:38.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.137+0000 7fdbf6d9d640 1 --2- 172.21.15.88:0/383151493 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdbf815cea0 0x7fdbf80b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:38.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.137+0000 7fdbf659c640 1 --2- 172.21.15.88:0/383151493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbf80ae130 0x7fdbf815bef0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdbe801a9d0 tx=0x7fdbe80190c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:38.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.137+0000 7fdbe6ffd640 1 -- 172.21.15.88:0/383151493 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe80b4810 con 0x7fdbf80ae130 2023-10-16T20:05:38.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.137+0000 7fdbe6ffd640 1 -- 172.21.15.88:0/383151493 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe80b4970 con 0x7fdbf80ae130 2023-10-16T20:05:38.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.137+0000 7fdbe6ffd640 1 -- 172.21.15.88:0/383151493 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe80b4ce0 con 0x7fdbf80ae130 2023-10-16T20:05:38.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.137+0000 7fdbfcbd0640 1 -- 172.21.15.88:0/383151493 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbf80b4330 con 0x7fdbf80ae130 2023-10-16T20:05:38.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.137+0000 7fdbfcbd0640 1 -- 172.21.15.88:0/383151493 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf80c2b10 con 0x7fdbf80ae130 2023-10-16T20:05:38.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.137+0000 7fdbfcbd0640 1 -- 172.21.15.88:0/383151493 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbf804e9c0 con 0x7fdbf80ae130 2023-10-16T20:05:38.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.141+0000 7fdbe6ffd640 1 -- 172.21.15.88:0/383151493 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdbe80b2040 con 0x7fdbf80ae130 2023-10-16T20:05:38.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.141+0000 7fdbe6ffd640 1 --2- 172.21.15.88:0/383151493 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fdbd80664d0 0x7fdbd8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:38.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.141+0000 7fdbe6ffd640 1 -- 172.21.15.88:0/383151493 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fdbe811e260 con 0x7fdbf80ae130 2023-10-16T20:05:38.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.141+0000 7fdbf5d9b640 1 --2- 172.21.15.88:0/383151493 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fdbd80664d0 0x7fdbd8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:38.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.141+0000 7fdbf5d9b640 1 --2- 172.21.15.88:0/383151493 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fdbd80664d0 0x7fdbd8068990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdbf81f6080 tx=0x7fdbec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:38.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.145+0000 7fdbe6ffd640 1 -- 172.21.15.88:0/383151493 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdbe80eb310 con 0x7fdbf80ae130 2023-10-16T20:05:38.305 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.301+0000 7fdbfcbd0640 1 -- 172.21.15.88:0/383151493 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdbf8065cb0 con 0x7fdbf80ae130 2023-10-16T20:05:38.305 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.305+0000 7fdbe6ffd640 1 -- 172.21.15.88:0/383151493 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdbe80f3e60 con 0x7fdbf80ae130 2023-10-16T20:05:38.306 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:05:38.308 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.305+0000 7fdbfcbd0640 1 -- 172.21.15.88:0/383151493 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fdbd80664d0 msgr2=0x7fdbd8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:38.308 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.305+0000 7fdbfcbd0640 1 --2- 172.21.15.88:0/383151493 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fdbd80664d0 0x7fdbd8068990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdbf81f6080 tx=0x7fdbec009040 comp rx=0 tx=0).stop 2023-10-16T20:05:38.308 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.305+0000 7fdbfcbd0640 1 -- 172.21.15.88:0/383151493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbf80ae130 msgr2=0x7fdbf815bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:38.308 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.305+0000 7fdbfcbd0640 1 --2- 172.21.15.88:0/383151493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbf80ae130 0x7fdbf815bef0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdbe801a9d0 tx=0x7fdbe80190c0 comp rx=0 tx=0).stop 2023-10-16T20:05:38.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.305+0000 7fdbfcbd0640 1 -- 172.21.15.88:0/383151493 shutdown_connections 2023-10-16T20:05:38.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.305+0000 7fdbfcbd0640 1 --2- 172.21.15.88:0/383151493 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fdbd80664d0 0x7fdbd8068990 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:38.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.305+0000 7fdbfcbd0640 1 --2- 172.21.15.88:0/383151493 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdbf815cea0 0x7fdbf80b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:38.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.305+0000 7fdbfcbd0640 1 --2- 172.21.15.88:0/383151493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbf815cac0 0x7fdbf80b3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:38.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.305+0000 7fdbfcbd0640 1 --2- 172.21.15.88:0/383151493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbf80ae130 0x7fdbf815bef0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:38.310 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.305+0000 7fdbfcbd0640 1 -- 172.21.15.88:0/383151493 >> 172.21.15.88:0/383151493 conn(0x7fdbf80b0970 msgr2=0x7fdbf80afea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:38.310 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.305+0000 7fdbfcbd0640 1 -- 172.21.15.88:0/383151493 shutdown_connections 2023-10-16T20:05:38.310 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.305+0000 7fdbfcbd0640 1 -- 172.21.15.88:0/383151493 wait complete. 2023-10-16T20:05:38.327 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:05:38.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.465+0000 7f4ea9b1d640 1 Processor -- start 2023-10-16T20:05:38.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.465+0000 7f4ea9b1d640 1 -- start start 2023-10-16T20:05:38.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.465+0000 7f4ea9b1d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4ea40ada40 0x7f4ea40b1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:38.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.465+0000 7f4ea9b1d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4ea40ade20 0x7f4ea4143eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:38.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.465+0000 7f4ea9b1d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4ea415adb0 0x7f4ea40b0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:38.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.465+0000 7f4ea9b1d640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ea40b2570 con 0x7f4ea415adb0 2023-10-16T20:05:38.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.465+0000 7f4ea9b1d640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ea40cca50 con 0x7f4ea40ada40 2023-10-16T20:05:38.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.465+0000 7f4ea9b1d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea40b1fb0 con 0x7f4ea40ade20 2023-10-16T20:05:38.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.465+0000 7f4ea3fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4ea415adb0 0x7f4ea40b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:38.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.465+0000 7f4ea3fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4ea415adb0 0x7f4ea40b0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47136/0 (socket says 172.21.15.88:47136) 2023-10-16T20:05:38.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.465+0000 7f4ea3fff640 1 -- 172.21.15.88:0/2813154372 learned_addr learned my addr 172.21.15.88:0/2813154372 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:38.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.465+0000 7f4ea37fe640 1 --2- 172.21.15.88:0/2813154372 >> v2:172.21.15.94:3300/0 conn(0x7f4ea40ada40 0x7f4ea40b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:38.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.465+0000 7f4ea2ffd640 1 --2- 172.21.15.88:0/2813154372 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4ea40ade20 0x7f4ea4143eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:38.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.465+0000 7f4ea3fff640 1 -- 172.21.15.88:0/2813154372 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4ea40ade20 msgr2=0x7f4ea4143eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:38.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.465+0000 7f4ea3fff640 1 --2- 172.21.15.88:0/2813154372 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4ea40ade20 0x7f4ea4143eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:38.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.465+0000 7f4ea3fff640 1 -- 172.21.15.88:0/2813154372 >> v2:172.21.15.94:3300/0 conn(0x7f4ea40ada40 msgr2=0x7f4ea40b1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:38.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.465+0000 7f4ea3fff640 1 --2- 172.21.15.88:0/2813154372 >> v2:172.21.15.94:3300/0 conn(0x7f4ea40ada40 0x7f4ea40b1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:38.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.465+0000 7f4ea3fff640 1 -- 172.21.15.88:0/2813154372 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea40b2290 con 0x7f4ea415adb0 2023-10-16T20:05:38.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.469+0000 7f4ea3fff640 1 --2- 172.21.15.88:0/2813154372 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ea415adb0 0x7f4ea40b0500 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4ea40b10f0 tx=0x7f4e9c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973ee62e67fabba0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:38.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.469+0000 7f4ea27fc640 1 -- 172.21.15.88:0/2813154372 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c0a55c0 con 0x7f4ea415adb0 2023-10-16T20:05:38.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.469+0000 7f4ea27fc640 1 -- 172.21.15.88:0/2813154372 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9c0a5720 con 0x7f4ea415adb0 2023-10-16T20:05:38.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.469+0000 7f4ea27fc640 1 -- 172.21.15.88:0/2813154372 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c0a5ab0 con 0x7f4ea415adb0 2023-10-16T20:05:38.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.469+0000 7f4ea9b1d640 1 -- 172.21.15.88:0/2813154372 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ea415adb0 msgr2=0x7f4ea40b0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:38.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.469+0000 7f4ea9b1d640 1 --2- 172.21.15.88:0/2813154372 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ea415adb0 0x7f4ea40b0500 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4ea40b10f0 tx=0x7f4e9c01c940 comp rx=0 tx=0).stop 2023-10-16T20:05:38.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.469+0000 7f4ea9b1d640 1 -- 172.21.15.88:0/2813154372 shutdown_connections 2023-10-16T20:05:38.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.469+0000 7f4ea9b1d640 1 --2- 172.21.15.88:0/2813154372 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4ea40ade20 0x7f4ea4143eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:38.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.469+0000 7f4ea9b1d640 1 --2- 172.21.15.88:0/2813154372 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ea415adb0 0x7f4ea40b0500 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:38.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.469+0000 7f4ea9b1d640 1 --2- 172.21.15.88:0/2813154372 >> v2:172.21.15.94:3300/0 conn(0x7f4ea40ada40 0x7f4ea40b1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:38.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.469+0000 7f4ea9b1d640 1 -- 172.21.15.88:0/2813154372 >> 172.21.15.88:0/2813154372 conn(0x7f4ea40ce5e0 msgr2=0x7f4ea40ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:38.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.469+0000 7f4ea9b1d640 1 -- 172.21.15.88:0/2813154372 shutdown_connections 2023-10-16T20:05:38.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.469+0000 7f4ea9b1d640 1 -- 172.21.15.88:0/2813154372 wait complete. 2023-10-16T20:05:38.476 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.469+0000 7f4ea9b1d640 1 Processor -- start 2023-10-16T20:05:38.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.473+0000 7f4ea9b1d640 1 -- start start 2023-10-16T20:05:38.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.473+0000 7f4ea9b1d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ea40ada40 0x7f4ea4136a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:38.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.473+0000 7f4ea9b1d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ea40ade20 0x7f4ea4158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:38.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.473+0000 7f4ea9b1d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4ea415adb0 0x7f4ea4139cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:38.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.473+0000 7f4ea9b1d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea415a1b0 con 0x7f4ea40ada40 2023-10-16T20:05:38.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.473+0000 7f4ea9b1d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea4159320 con 0x7f4ea40ade20 2023-10-16T20:05:38.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.473+0000 7f4ea9b1d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea4159530 con 0x7f4ea415adb0 2023-10-16T20:05:38.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.473+0000 7f4ea37fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ea40ada40 0x7f4ea4136a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:38.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.473+0000 7f4ea37fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ea40ada40 0x7f4ea4136a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47142/0 (socket says 172.21.15.88:47142) 2023-10-16T20:05:38.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.473+0000 7f4ea37fe640 1 -- 172.21.15.88:0/179042452 learned_addr learned my addr 172.21.15.88:0/179042452 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:38.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.473+0000 7f4ea2ffd640 1 --2- 172.21.15.88:0/179042452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ea40ade20 0x7f4ea4158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:38.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.473+0000 7f4ea3fff640 1 --2- 172.21.15.88:0/179042452 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4ea415adb0 0x7f4ea4139cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:38.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.473+0000 7f4ea37fe640 1 -- 172.21.15.88:0/179042452 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4ea415adb0 msgr2=0x7f4ea4139cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:38.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.473+0000 7f4ea37fe640 1 --2- 172.21.15.88:0/179042452 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4ea415adb0 0x7f4ea4139cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:38.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.473+0000 7f4ea37fe640 1 -- 172.21.15.88:0/179042452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ea40ade20 msgr2=0x7f4ea4158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:38.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.473+0000 7f4ea37fe640 1 --2- 172.21.15.88:0/179042452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ea40ade20 0x7f4ea4158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:38.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.473+0000 7f4ea37fe640 1 -- 172.21.15.88:0/179042452 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9c009b30 con 0x7f4ea40ada40 2023-10-16T20:05:38.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.473+0000 7f4ea37fe640 1 --2- 172.21.15.88:0/179042452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ea40ada40 0x7f4ea4136a60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4e9400e870 tx=0x7f4e9400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:38.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.473+0000 7f4e8bfff640 1 -- 172.21.15.88:0/179042452 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e94009510 con 0x7f4ea40ada40 2023-10-16T20:05:38.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.473+0000 7f4e8bfff640 1 -- 172.21.15.88:0/179042452 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9400eec0 con 0x7f4ea40ada40 2023-10-16T20:05:38.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.473+0000 7f4ea9b1d640 1 -- 172.21.15.88:0/179042452 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea4159810 con 0x7f4ea40ada40 2023-10-16T20:05:38.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.473+0000 7f4e8bfff640 1 -- 172.21.15.88:0/179042452 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9400cb40 con 0x7f4ea40ada40 2023-10-16T20:05:38.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.473+0000 7f4ea9b1d640 1 -- 172.21.15.88:0/179042452 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea4159dd0 con 0x7f4ea40ada40 2023-10-16T20:05:38.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.473+0000 7f4ea9b1d640 1 -- 172.21.15.88:0/179042452 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea4138020 con 0x7f4ea40ada40 2023-10-16T20:05:38.482 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.477+0000 7f4e8bfff640 1 -- 172.21.15.88:0/179042452 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4e9400cca0 con 0x7f4ea40ada40 2023-10-16T20:05:38.482 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.477+0000 7f4e8bfff640 1 --2- 172.21.15.88:0/179042452 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4e900664a0 0x7f4e90068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:38.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.477+0000 7f4e8bfff640 1 -- 172.21.15.88:0/179042452 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4e94084230 con 0x7f4ea40ada40 2023-10-16T20:05:38.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.477+0000 7f4ea2ffd640 1 --2- 172.21.15.88:0/179042452 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4e900664a0 0x7f4e90068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:38.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.477+0000 7f4ea2ffd640 1 --2- 172.21.15.88:0/179042452 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4e900664a0 0x7f4e90068960 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4133ce0 tx=0x7f4e98009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:38.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.481+0000 7f4e8bfff640 1 -- 172.21.15.88:0/179042452 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4e94051430 con 0x7f4ea40ada40 2023-10-16T20:05:38.636 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.633+0000 7f4ea9b1d640 1 -- 172.21.15.88:0/179042452 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ea404e9c0 con 0x7f4ea40ada40 2023-10-16T20:05:38.638 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.637+0000 7f4e8bfff640 1 -- 172.21.15.88:0/179042452 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18661 (secure 0 0 0) 0x7f4e94054a50 con 0x7f4ea40ada40 2023-10-16T20:05:38.638 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:38.638 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":29,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:05:38.202959+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T20:04:54.442487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T20:04:55.766641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T20:04:59.475830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":1}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:05:38.640 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.637+0000 7f4ea9b1d640 1 -- 172.21.15.88:0/179042452 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4e900664a0 msgr2=0x7f4e90068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:38.640 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.637+0000 7f4ea9b1d640 1 --2- 172.21.15.88:0/179042452 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4e900664a0 0x7f4e90068960 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4133ce0 tx=0x7f4e98009190 comp rx=0 tx=0).stop 2023-10-16T20:05:38.640 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.637+0000 7f4ea9b1d640 1 -- 172.21.15.88:0/179042452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ea40ada40 msgr2=0x7f4ea4136a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:38.641 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.637+0000 7f4ea9b1d640 1 --2- 172.21.15.88:0/179042452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ea40ada40 0x7f4ea4136a60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4e9400e870 tx=0x7f4e9400ec70 comp rx=0 tx=0).stop 2023-10-16T20:05:38.641 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.637+0000 7f4ea9b1d640 1 -- 172.21.15.88:0/179042452 shutdown_connections 2023-10-16T20:05:38.641 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.637+0000 7f4ea9b1d640 1 --2- 172.21.15.88:0/179042452 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4e900664a0 0x7f4e90068960 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:38.641 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.637+0000 7f4ea9b1d640 1 --2- 172.21.15.88:0/179042452 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4ea415adb0 0x7f4ea4139cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:38.641 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.637+0000 7f4ea9b1d640 1 --2- 172.21.15.88:0/179042452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ea40ade20 0x7f4ea4158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:38.641 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.637+0000 7f4ea9b1d640 1 --2- 172.21.15.88:0/179042452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ea40ada40 0x7f4ea4136a60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:38.642 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.637+0000 7f4ea9b1d640 1 -- 172.21.15.88:0/179042452 >> 172.21.15.88:0/179042452 conn(0x7f4ea40ce5e0 msgr2=0x7f4ea4071670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:38.642 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.637+0000 7f4ea9b1d640 1 -- 172.21.15.88:0/179042452 shutdown_connections 2023-10-16T20:05:38.642 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.637+0000 7f4ea9b1d640 1 -- 172.21.15.88:0/179042452 wait complete. 2023-10-16T20:05:38.659 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T20:04:54.442487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T20:04:55.766641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T20:04:59.475830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T20:05:38.660 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:05:38.800 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.797+0000 7f683e69c640 1 Processor -- start 2023-10-16T20:05:38.800 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.797+0000 7f683e69c640 1 -- start start 2023-10-16T20:05:38.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.797+0000 7f683e69c640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f68380b4260 0x7f683815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:38.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.797+0000 7f683e69c640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f683806d3f0 0x7f683806d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:38.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.797+0000 7f683e69c640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f683815ad30 con 0x7f68380b0da0 2023-10-16T20:05:38.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.797+0000 7f683e69c640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f683806dda0 con 0x7f68380b4260 2023-10-16T20:05:38.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.797+0000 7f683e69c640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68380b4640 con 0x7f683806d3f0 2023-10-16T20:05:38.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.801+0000 7f6837fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f68380b4260 0x7f683815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:38.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.801+0000 7f6837fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f68380b4260 0x7f683815dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47150/0 (socket says 172.21.15.88:47150) 2023-10-16T20:05:38.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.801+0000 7f6837fff640 1 -- 172.21.15.88:0/2804103812 learned_addr learned my addr 172.21.15.88:0/2804103812 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:38.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.801+0000 7f683cc39640 1 --2- 172.21.15.88:0/2804103812 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f683806d3f0 0x7f683806d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:38.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.801+0000 7f6837fff640 1 -- 172.21.15.88:0/2804103812 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f683806d3f0 msgr2=0x7f683806d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:38.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.801+0000 7f6837fff640 1 --2- 172.21.15.88:0/2804103812 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f683806d3f0 0x7f683806d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:38.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.801+0000 7f6837fff640 1 -- 172.21.15.88:0/2804103812 >> v1:172.21.15.94:6789/0 conn(0x7f68380b0da0 legacy=0x7f6838159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:38.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.801+0000 7f6837fff640 1 -- 172.21.15.88:0/2804103812 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683815a770 con 0x7f68380b4260 2023-10-16T20:05:38.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.801+0000 7f683cc39640 1 --2- 172.21.15.88:0/2804103812 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f683806d3f0 0x7f683806d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:38.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.801+0000 7f6837fff640 1 --2- 172.21.15.88:0/2804103812 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68380b4260 0x7f683815dc90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f68280098a0 tx=0x7f682801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7685de84c3c9c90 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:38.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.801+0000 7f6836ffd640 1 -- 172.21.15.88:0/2804103812 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68280ab660 con 0x7f68380b4260 2023-10-16T20:05:38.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.801+0000 7f6836ffd640 1 -- 172.21.15.88:0/2804103812 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68280ab7c0 con 0x7f68380b4260 2023-10-16T20:05:38.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.801+0000 7f6836ffd640 1 -- 172.21.15.88:0/2804103812 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68280abb50 con 0x7f68380b4260 2023-10-16T20:05:38.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.805+0000 7f683e69c640 1 -- 172.21.15.88:0/2804103812 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68380b4260 msgr2=0x7f683815dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:38.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.805+0000 7f683e69c640 1 --2- 172.21.15.88:0/2804103812 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68380b4260 0x7f683815dc90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f68280098a0 tx=0x7f682801c460 comp rx=0 tx=0).stop 2023-10-16T20:05:38.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.805+0000 7f683e69c640 1 -- 172.21.15.88:0/2804103812 shutdown_connections 2023-10-16T20:05:38.808 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.805+0000 7f683e69c640 1 --2- 172.21.15.88:0/2804103812 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f683806d3f0 0x7f683806d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:38.808 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.805+0000 7f683e69c640 1 --2- 172.21.15.88:0/2804103812 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68380b4260 0x7f683815dc90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:38.808 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.805+0000 7f683e69c640 1 -- 172.21.15.88:0/2804103812 >> 172.21.15.88:0/2804103812 conn(0x7f68380cb080 msgr2=0x7f68380cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:38.808 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.805+0000 7f683e69c640 1 -- 172.21.15.88:0/2804103812 shutdown_connections 2023-10-16T20:05:38.808 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.805+0000 7f683e69c640 1 -- 172.21.15.88:0/2804103812 wait complete. 2023-10-16T20:05:38.809 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.805+0000 7f683e69c640 1 Processor -- start 2023-10-16T20:05:38.809 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.805+0000 7f683e69c640 1 -- start start 2023-10-16T20:05:38.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.805+0000 7f683e69c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f683806d3f0 0x7f68380cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:38.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.805+0000 7f683e69c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68380b0da0 0x7f6838151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:38.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.805+0000 7f683e69c640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f68380b4260 0x7f68381431f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:38.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.805+0000 7f683e69c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68381525f0 con 0x7f683806d3f0 2023-10-16T20:05:38.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.805+0000 7f683e69c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6838151740 con 0x7f68380b0da0 2023-10-16T20:05:38.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.805+0000 7f683e69c640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68381519e0 con 0x7f68380b4260 2023-10-16T20:05:38.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.805+0000 7f6837fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f683806d3f0 0x7f68380cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:38.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.805+0000 7f6837fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f683806d3f0 0x7f68380cd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47164/0 (socket says 172.21.15.88:47164) 2023-10-16T20:05:38.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.805+0000 7f6837fff640 1 -- 172.21.15.88:0/2769691925 learned_addr learned my addr 172.21.15.88:0/2769691925 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:38.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.805+0000 7f683cc39640 1 --2- 172.21.15.88:0/2769691925 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f68380b4260 0x7f68381431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:38.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.805+0000 7f68377fe640 1 --2- 172.21.15.88:0/2769691925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68380b0da0 0x7f6838151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:38.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.809+0000 7f6837fff640 1 -- 172.21.15.88:0/2769691925 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f68380b4260 msgr2=0x7f68381431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:38.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.809+0000 7f6837fff640 1 --2- 172.21.15.88:0/2769691925 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f68380b4260 0x7f68381431f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:38.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.809+0000 7f6837fff640 1 -- 172.21.15.88:0/2769691925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68380b0da0 msgr2=0x7f6838151200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:38.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.809+0000 7f6837fff640 1 --2- 172.21.15.88:0/2769691925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68380b0da0 0x7f6838151200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:38.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.809+0000 7f6837fff640 1 -- 172.21.15.88:0/2769691925 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68280094d0 con 0x7f683806d3f0 2023-10-16T20:05:38.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.809+0000 7f6837fff640 1 --2- 172.21.15.88:0/2769691925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f683806d3f0 0x7f68380cd1a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f68280ad040 tx=0x7f682801aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:38.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.809+0000 7f6834ff9640 1 -- 172.21.15.88:0/2769691925 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68280b4800 con 0x7f683806d3f0 2023-10-16T20:05:38.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.809+0000 7f6834ff9640 1 -- 172.21.15.88:0/2769691925 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68280b49a0 con 0x7f683806d3f0 2023-10-16T20:05:38.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.809+0000 7f6834ff9640 1 -- 172.21.15.88:0/2769691925 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68280b4d50 con 0x7f683806d3f0 2023-10-16T20:05:38.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.809+0000 7f683e69c640 1 -- 172.21.15.88:0/2769691925 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6838151cc0 con 0x7f683806d3f0 2023-10-16T20:05:38.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.809+0000 7f683e69c640 1 -- 172.21.15.88:0/2769691925 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6838152280 con 0x7f683806d3f0 2023-10-16T20:05:38.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.809+0000 7f683e69c640 1 -- 172.21.15.88:0/2769691925 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f683804e9c0 con 0x7f683806d3f0 2023-10-16T20:05:38.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.809+0000 7f6834ff9640 1 -- 172.21.15.88:0/2769691925 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f68280a9070 con 0x7f683806d3f0 2023-10-16T20:05:38.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.813+0000 7f6834ff9640 1 --2- 172.21.15.88:0/2769691925 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f68240664d0 0x7f6824068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:38.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.813+0000 7f6834ff9640 1 -- 172.21.15.88:0/2769691925 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f682811eb50 con 0x7f683806d3f0 2023-10-16T20:05:38.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.813+0000 7f68377fe640 1 --2- 172.21.15.88:0/2769691925 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f68240664d0 0x7f6824068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:38.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.813+0000 7f68377fe640 1 --2- 172.21.15.88:0/2769691925 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f68240664d0 0x7f6824068990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f683814dc30 tx=0x7f682c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:38.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.813+0000 7f6834ff9640 1 -- 172.21.15.88:0/2769691925 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f68280ebd40 con 0x7f683806d3f0 2023-10-16T20:05:38.968 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.965+0000 7f683e69c640 1 -- 172.21.15.88:0/2769691925 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6838071500 con 0x7f683806d3f0 2023-10-16T20:05:38.968 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.965+0000 7f6834ff9640 1 -- 172.21.15.88:0/2769691925 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f68280c4020 con 0x7f683806d3f0 2023-10-16T20:05:38.968 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:05:38.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.969+0000 7f683e69c640 1 -- 172.21.15.88:0/2769691925 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f68240664d0 msgr2=0x7f6824068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:38.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.969+0000 7f683e69c640 1 --2- 172.21.15.88:0/2769691925 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f68240664d0 0x7f6824068990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f683814dc30 tx=0x7f682c009040 comp rx=0 tx=0).stop 2023-10-16T20:05:38.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.969+0000 7f683e69c640 1 -- 172.21.15.88:0/2769691925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f683806d3f0 msgr2=0x7f68380cd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:38.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.969+0000 7f683e69c640 1 --2- 172.21.15.88:0/2769691925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f683806d3f0 0x7f68380cd1a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f68280ad040 tx=0x7f682801aca0 comp rx=0 tx=0).stop 2023-10-16T20:05:38.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.969+0000 7f683e69c640 1 -- 172.21.15.88:0/2769691925 shutdown_connections 2023-10-16T20:05:38.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.969+0000 7f683e69c640 1 --2- 172.21.15.88:0/2769691925 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f68240664d0 0x7f6824068990 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:38.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.969+0000 7f683e69c640 1 --2- 172.21.15.88:0/2769691925 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f68380b4260 0x7f68381431f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:38.972 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.969+0000 7f683e69c640 1 --2- 172.21.15.88:0/2769691925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68380b0da0 0x7f6838151200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:38.972 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.969+0000 7f683e69c640 1 --2- 172.21.15.88:0/2769691925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f683806d3f0 0x7f68380cd1a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:38.972 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.969+0000 7f683e69c640 1 -- 172.21.15.88:0/2769691925 >> 172.21.15.88:0/2769691925 conn(0x7f68380cb080 msgr2=0x7f68380afaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:38.972 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.969+0000 7f683e69c640 1 -- 172.21.15.88:0/2769691925 shutdown_connections 2023-10-16T20:05:38.972 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:38.969+0000 7f683e69c640 1 -- 172.21.15.88:0/2769691925 wait complete. 2023-10-16T20:05:38.989 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T20:05:39.130 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.129+0000 7f28ec09e640 1 Processor -- start 2023-10-16T20:05:39.131 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.129+0000 7f28ec09e640 1 -- start start 2023-10-16T20:05:39.132 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.129+0000 7f28ec09e640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f28e40b0c10 0x7f28e415aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:39.132 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.129+0000 7f28ec09e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f28e415b530 0x7f28e406c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:39.132 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.129+0000 7f28ec09e640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28e415bd20 con 0x7f28e4154f30 2023-10-16T20:05:39.132 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.129+0000 7f28ec09e640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28e4155310 con 0x7f28e40b0c10 2023-10-16T20:05:39.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.129+0000 7f28ec09e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e40702f0 con 0x7f28e415b530 2023-10-16T20:05:39.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.129+0000 7f28e9e3a640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f28e4154f30 0x7f28e40b0500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:54760/0 (socket says 172.21.15.88:54760) 2023-10-16T20:05:39.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.129+0000 7f28e9e3a640 1 -- 172.21.15.88:0/4088750840 learned_addr learned my addr 172.21.15.88:0/4088750840 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:39.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.129+0000 7f28e9639640 1 --2- 172.21.15.88:0/4088750840 >> v2:172.21.15.88:3300/0 conn(0x7f28e40b0c10 0x7f28e415aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:39.134 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.129+0000 7f28ea63b640 1 --2- 172.21.15.88:0/4088750840 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f28e415b530 0x7f28e406c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:39.134 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.129+0000 7f28e8e38640 1 -- 172.21.15.88:0/4088750840 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 81114292 0 0) 0x7f28e415bd20 con 0x7f28e4154f30 2023-10-16T20:05:39.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.129+0000 7f28e8e38640 1 -- 172.21.15.88:0/4088750840 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28d0003530 con 0x7f28e4154f30 2023-10-16T20:05:39.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.129+0000 7f28e9639640 1 -- 172.21.15.88:0/4088750840 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f28e415b530 msgr2=0x7f28e406c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:39.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.129+0000 7f28e9639640 1 --2- 172.21.15.88:0/4088750840 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f28e415b530 0x7f28e406c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:39.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.129+0000 7f28e9639640 1 -- 172.21.15.88:0/4088750840 >> v1:172.21.15.88:6789/0 conn(0x7f28e4154f30 legacy=0x7f28e40b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:39.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.129+0000 7f28e8e38640 1 -- 172.21.15.88:0/4088750840 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1012355677 0 0) 0x7f28d0003530 con 0x7f28e4154f30 2023-10-16T20:05:39.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.129+0000 7f28e9639640 1 -- 172.21.15.88:0/4088750840 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e40705d0 con 0x7f28e40b0c10 2023-10-16T20:05:39.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.133+0000 7f28e9639640 1 --2- 172.21.15.88:0/4088750840 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28e40b0c10 0x7f28e415aff0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f28d40097c0 tx=0x7f28d401c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82794dcd49e2b5a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:39.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.133+0000 7f28e8e38640 1 -- 172.21.15.88:0/4088750840 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e0001680 con 0x7f28e40b0c10 2023-10-16T20:05:39.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.133+0000 7f28e8e38640 1 -- 172.21.15.88:0/4088750840 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d40a7050 con 0x7f28e40b0c10 2023-10-16T20:05:39.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.133+0000 7f28e8e38640 1 -- 172.21.15.88:0/4088750840 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d40ab620 con 0x7f28e40b0c10 2023-10-16T20:05:39.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.133+0000 7f28ec09e640 1 -- 172.21.15.88:0/4088750840 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28e40b0c10 msgr2=0x7f28e415aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:39.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.133+0000 7f28ec09e640 1 --2- 172.21.15.88:0/4088750840 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28e40b0c10 0x7f28e415aff0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f28d40097c0 tx=0x7f28d401c370 comp rx=0 tx=0).stop 2023-10-16T20:05:39.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.133+0000 7f28ec09e640 1 -- 172.21.15.88:0/4088750840 shutdown_connections 2023-10-16T20:05:39.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.133+0000 7f28ec09e640 1 --2- 172.21.15.88:0/4088750840 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f28e415b530 0x7f28e406c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:39.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.133+0000 7f28ec09e640 1 --2- 172.21.15.88:0/4088750840 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28e40b0c10 0x7f28e415aff0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:39.139 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.133+0000 7f28ec09e640 1 -- 172.21.15.88:0/4088750840 >> 172.21.15.88:0/4088750840 conn(0x7f28e40cb080 msgr2=0x7f28e40cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:39.139 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.133+0000 7f28ec09e640 1 -- 172.21.15.88:0/4088750840 shutdown_connections 2023-10-16T20:05:39.139 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.133+0000 7f28ec09e640 1 -- 172.21.15.88:0/4088750840 wait complete. 2023-10-16T20:05:39.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.137+0000 7f28ec09e640 1 Processor -- start 2023-10-16T20:05:39.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.137+0000 7f28ec09e640 1 -- start start 2023-10-16T20:05:39.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.137+0000 7f28ec09e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f28e40b0c10 0x7f28e4153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:39.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.137+0000 7f28ec09e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28e4154f30 0x7f28e40aa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:39.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.137+0000 7f28ec09e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28e415b530 0x7f28e40aad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:39.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.137+0000 7f28ec09e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e40a8520 con 0x7f28e415b530 2023-10-16T20:05:39.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.137+0000 7f28ec09e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e40a7910 con 0x7f28e4154f30 2023-10-16T20:05:39.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.137+0000 7f28ec09e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e40a7bf0 con 0x7f28e40b0c10 2023-10-16T20:05:39.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.137+0000 7f28ea63b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28e415b530 0x7f28e40aad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:39.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.137+0000 7f28ea63b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28e415b530 0x7f28e40aad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47190/0 (socket says 172.21.15.88:47190) 2023-10-16T20:05:39.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.137+0000 7f28ea63b640 1 -- 172.21.15.88:0/3792775857 learned_addr learned my addr 172.21.15.88:0/3792775857 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:39.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.137+0000 7f28e9e3a640 1 --2- 172.21.15.88:0/3792775857 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f28e40b0c10 0x7f28e4153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:39.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.137+0000 7f28e9639640 1 --2- 172.21.15.88:0/3792775857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28e4154f30 0x7f28e40aa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:39.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.137+0000 7f28ea63b640 1 -- 172.21.15.88:0/3792775857 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f28e40b0c10 msgr2=0x7f28e4153130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:39.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.137+0000 7f28ea63b640 1 --2- 172.21.15.88:0/3792775857 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f28e40b0c10 0x7f28e4153130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:39.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.137+0000 7f28ea63b640 1 -- 172.21.15.88:0/3792775857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28e4154f30 msgr2=0x7f28e40aa7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:39.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.137+0000 7f28ea63b640 1 --2- 172.21.15.88:0/3792775857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28e4154f30 0x7f28e40aa7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:39.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.137+0000 7f28ea63b640 1 -- 172.21.15.88:0/3792775857 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d4009420 con 0x7f28e415b530 2023-10-16T20:05:39.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.137+0000 7f28e9639640 1 --2- 172.21.15.88:0/3792775857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28e4154f30 0x7f28e40aa7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:39.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.137+0000 7f28ea63b640 1 --2- 172.21.15.88:0/3792775857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28e415b530 0x7f28e40aad00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00efc0 tx=0x7f28dc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:39.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.137+0000 7f28da7fc640 1 -- 172.21.15.88:0/3792775857 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc009ba0 con 0x7f28e415b530 2023-10-16T20:05:39.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.137+0000 7f28da7fc640 1 -- 172.21.15.88:0/3792775857 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28dc00f040 con 0x7f28e415b530 2023-10-16T20:05:39.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.137+0000 7f28ec09e640 1 -- 172.21.15.88:0/3792775857 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28e40a7ed0 con 0x7f28e415b530 2023-10-16T20:05:39.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.137+0000 7f28da7fc640 1 -- 172.21.15.88:0/3792775857 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc0146c0 con 0x7f28e415b530 2023-10-16T20:05:39.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.137+0000 7f28ec09e640 1 -- 172.21.15.88:0/3792775857 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e4153670 con 0x7f28e415b530 2023-10-16T20:05:39.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.141+0000 7f28ec09e640 1 -- 172.21.15.88:0/3792775857 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28e404e9c0 con 0x7f28e415b530 2023-10-16T20:05:39.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.141+0000 7f28da7fc640 1 -- 172.21.15.88:0/3792775857 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f28dc014820 con 0x7f28e415b530 2023-10-16T20:05:39.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.141+0000 7f28da7fc640 1 --2- 172.21.15.88:0/3792775857 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f28c80664a0 0x7f28c8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:39.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.141+0000 7f28da7fc640 1 -- 172.21.15.88:0/3792775857 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f28dc084d60 con 0x7f28e415b530 2023-10-16T20:05:39.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.141+0000 7f28e9e3a640 1 --2- 172.21.15.88:0/3792775857 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f28c80664a0 0x7f28c8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:39.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.141+0000 7f28e9e3a640 1 --2- 172.21.15.88:0/3792775857 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f28c80664a0 0x7f28c8068960 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f28e0005890 tx=0x7f28e0001660 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:39.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.145+0000 7f28da7fc640 1 -- 172.21.15.88:0/3792775857 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f28dc051d80 con 0x7f28e415b530 2023-10-16T20:05:39.299 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.297+0000 7f28ec09e640 1 -- 172.21.15.88:0/3792775857 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f28e40a8660 con 0x7f28e415b530 2023-10-16T20:05:39.299 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.297+0000 7f28da7fc640 1 -- 172.21.15.88:0/3792775857 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f28dc0180a0 con 0x7f28e415b530 2023-10-16T20:05:39.300 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:05:39.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.297+0000 7f28ec09e640 1 -- 172.21.15.88:0/3792775857 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f28c80664a0 msgr2=0x7f28c8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:39.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.297+0000 7f28ec09e640 1 --2- 172.21.15.88:0/3792775857 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f28c80664a0 0x7f28c8068960 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f28e0005890 tx=0x7f28e0001660 comp rx=0 tx=0).stop 2023-10-16T20:05:39.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.297+0000 7f28ec09e640 1 -- 172.21.15.88:0/3792775857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28e415b530 msgr2=0x7f28e40aad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:39.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.297+0000 7f28ec09e640 1 --2- 172.21.15.88:0/3792775857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28e415b530 0x7f28e40aad00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00efc0 tx=0x7f28dc00c490 comp rx=0 tx=0).stop 2023-10-16T20:05:39.302 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.297+0000 7f28ec09e640 1 -- 172.21.15.88:0/3792775857 shutdown_connections 2023-10-16T20:05:39.302 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.297+0000 7f28ec09e640 1 --2- 172.21.15.88:0/3792775857 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f28c80664a0 0x7f28c8068960 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:39.302 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.297+0000 7f28ec09e640 1 --2- 172.21.15.88:0/3792775857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28e415b530 0x7f28e40aad00 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:39.302 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.297+0000 7f28ec09e640 1 --2- 172.21.15.88:0/3792775857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28e4154f30 0x7f28e40aa7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:39.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.297+0000 7f28ec09e640 1 --2- 172.21.15.88:0/3792775857 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f28e40b0c10 0x7f28e4153130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:39.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.297+0000 7f28ec09e640 1 -- 172.21.15.88:0/3792775857 >> 172.21.15.88:0/3792775857 conn(0x7f28e40cb080 msgr2=0x7f28e415bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:39.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.297+0000 7f28ec09e640 1 -- 172.21.15.88:0/3792775857 shutdown_connections 2023-10-16T20:05:39.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.297+0000 7f28ec09e640 1 -- 172.21.15.88:0/3792775857 wait complete. 2023-10-16T20:05:39.320 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T20:05:39.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.457+0000 7f0dbfdc4640 1 Processor -- start 2023-10-16T20:05:39.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.457+0000 7f0dbfdc4640 1 -- start start 2023-10-16T20:05:39.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.461+0000 7f0dbfdc4640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0db80b0da0 0x7f0db8159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:39.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.461+0000 7f0dbfdc4640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0db815a1f0 0x7f0db806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:39.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.461+0000 7f0dbfdc4640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0db815ad30 con 0x7f0db80b4260 2023-10-16T20:05:39.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.461+0000 7f0dbfdc4640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0db80b4640 con 0x7f0db80b0da0 2023-10-16T20:05:39.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.461+0000 7f0dbfdc4640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0db815a770 con 0x7f0db815a1f0 2023-10-16T20:05:39.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.461+0000 7f0dbe361640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0db815a1f0 0x7f0db806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:39.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.461+0000 7f0dbd35f640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0db80b0da0 0x7f0db8159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:39.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.461+0000 7f0dbdb60640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f0db80b4260 0x7f0db815dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:59256/0 (socket says 172.21.15.88:59256) 2023-10-16T20:05:39.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.461+0000 7f0dbe361640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0db815a1f0 0x7f0db806d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:59490/0 (socket says 172.21.15.88:59490) 2023-10-16T20:05:39.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.461+0000 7f0dbdb60640 1 -- 172.21.15.88:0/2909499681 learned_addr learned my addr 172.21.15.88:0/2909499681 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:39.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.461+0000 7f0dbe361640 1 -- 172.21.15.88:0/2909499681 >> v2:172.21.15.94:3300/0 conn(0x7f0db80b0da0 msgr2=0x7f0db8159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:39.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.461+0000 7f0dbe361640 1 --2- 172.21.15.88:0/2909499681 >> v2:172.21.15.94:3300/0 conn(0x7f0db80b0da0 0x7f0db8159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:39.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.461+0000 7f0dbe361640 1 -- 172.21.15.88:0/2909499681 >> v1:172.21.15.94:6789/0 conn(0x7f0db80b4260 legacy=0x7f0db815dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:39.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.461+0000 7f0dbe361640 1 -- 172.21.15.88:0/2909499681 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db815aa50 con 0x7f0db815a1f0 2023-10-16T20:05:39.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.465+0000 7f0dbe361640 1 --2- 172.21.15.88:0/2909499681 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0db815a1f0 0x7f0db806d3f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0db0009d90 tx=0x7f0db001c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b2a312d915e35e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:39.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.465+0000 7f0dbcb5e640 1 -- 172.21.15.88:0/2909499681 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db00a5780 con 0x7f0db815a1f0 2023-10-16T20:05:39.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.465+0000 7f0dbcb5e640 1 -- 172.21.15.88:0/2909499681 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db00a58e0 con 0x7f0db815a1f0 2023-10-16T20:05:39.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.465+0000 7f0dbcb5e640 1 -- 172.21.15.88:0/2909499681 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db00a5c70 con 0x7f0db815a1f0 2023-10-16T20:05:39.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.465+0000 7f0dbfdc4640 1 -- 172.21.15.88:0/2909499681 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0db815a1f0 msgr2=0x7f0db806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:39.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.465+0000 7f0dbfdc4640 1 --2- 172.21.15.88:0/2909499681 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0db815a1f0 0x7f0db806d3f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0db0009d90 tx=0x7f0db001c8c0 comp rx=0 tx=0).stop 2023-10-16T20:05:39.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.465+0000 7f0dbfdc4640 1 -- 172.21.15.88:0/2909499681 shutdown_connections 2023-10-16T20:05:39.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.465+0000 7f0dbfdc4640 1 --2- 172.21.15.88:0/2909499681 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0db815a1f0 0x7f0db806d3f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:39.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.465+0000 7f0dbfdc4640 1 --2- 172.21.15.88:0/2909499681 >> v2:172.21.15.94:3300/0 conn(0x7f0db80b0da0 0x7f0db8159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:39.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.465+0000 7f0dbfdc4640 1 -- 172.21.15.88:0/2909499681 >> 172.21.15.88:0/2909499681 conn(0x7f0db80cb080 msgr2=0x7f0db80cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:39.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.465+0000 7f0dbfdc4640 1 -- 172.21.15.88:0/2909499681 shutdown_connections 2023-10-16T20:05:39.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.465+0000 7f0dbfdc4640 1 -- 172.21.15.88:0/2909499681 wait complete. 2023-10-16T20:05:39.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.465+0000 7f0dbfdc4640 1 Processor -- start 2023-10-16T20:05:39.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.465+0000 7f0dbfdc4640 1 -- start start 2023-10-16T20:05:39.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.465+0000 7f0dbfdc4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0db80b0da0 0x7f0db8149a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:39.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.465+0000 7f0dbfdc4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0db80b4260 0x7f0db814d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:39.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.465+0000 7f0dbfdc4640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0db815a1f0 0x7f0db814d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:39.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.465+0000 7f0dbfdc4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db80ccfb0 con 0x7f0db80b4260 2023-10-16T20:05:39.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.465+0000 7f0dbfdc4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db8152730 con 0x7f0db80b0da0 2023-10-16T20:05:39.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.465+0000 7f0dbfdc4640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0db815bf60 con 0x7f0db815a1f0 2023-10-16T20:05:39.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.469+0000 7f0dbd35f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0db80b4260 0x7f0db814d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:39.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.469+0000 7f0dbd35f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0db80b4260 0x7f0db814d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47198/0 (socket says 172.21.15.88:47198) 2023-10-16T20:05:39.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.469+0000 7f0dbd35f640 1 -- 172.21.15.88:0/2909237900 learned_addr learned my addr 172.21.15.88:0/2909237900 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:39.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.469+0000 7f0dbe361640 1 --2- 172.21.15.88:0/2909237900 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0db815a1f0 0x7f0db814d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:39.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.469+0000 7f0dbdb60640 1 --2- 172.21.15.88:0/2909237900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0db80b0da0 0x7f0db8149a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:39.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.469+0000 7f0dbd35f640 1 -- 172.21.15.88:0/2909237900 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0db815a1f0 msgr2=0x7f0db814d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:39.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.469+0000 7f0dbd35f640 1 --2- 172.21.15.88:0/2909237900 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0db815a1f0 0x7f0db814d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:39.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.469+0000 7f0dbd35f640 1 -- 172.21.15.88:0/2909237900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0db80b0da0 msgr2=0x7f0db8149a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:39.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.469+0000 7f0dbd35f640 1 --2- 172.21.15.88:0/2909237900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0db80b0da0 0x7f0db8149a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:39.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.469+0000 7f0dbd35f640 1 -- 172.21.15.88:0/2909237900 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db0009a70 con 0x7f0db80b4260 2023-10-16T20:05:39.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.469+0000 7f0dbd35f640 1 --2- 172.21.15.88:0/2909237900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0db80b4260 0x7f0db814d450 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0da800e930 tx=0x7f0da800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:39.475 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.469+0000 7f0dae7fc640 1 -- 172.21.15.88:0/2909237900 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da8009690 con 0x7f0db80b4260 2023-10-16T20:05:39.475 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.469+0000 7f0dae7fc640 1 -- 172.21.15.88:0/2909237900 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da800ca00 con 0x7f0db80b4260 2023-10-16T20:05:39.475 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.469+0000 7f0dbfdc4640 1 -- 172.21.15.88:0/2909237900 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db8152450 con 0x7f0db80b4260 2023-10-16T20:05:39.475 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.469+0000 7f0dae7fc640 1 -- 172.21.15.88:0/2909237900 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da800cd70 con 0x7f0db80b4260 2023-10-16T20:05:39.475 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.469+0000 7f0dbfdc4640 1 -- 172.21.15.88:0/2909237900 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0db80cd4c0 con 0x7f0db80b4260 2023-10-16T20:05:39.476 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.469+0000 7f0dbfdc4640 1 -- 172.21.15.88:0/2909237900 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0db804e9c0 con 0x7f0db80b4260 2023-10-16T20:05:39.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.469+0000 7f0dae7fc640 1 -- 172.21.15.88:0/2909237900 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0da8013470 con 0x7f0db80b4260 2023-10-16T20:05:39.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.469+0000 7f0dae7fc640 1 --2- 172.21.15.88:0/2909237900 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0d9c0664a0 0x7f0d9c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:39.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.473+0000 7f0dae7fc640 1 -- 172.21.15.88:0/2909237900 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0da80843e0 con 0x7f0db80b4260 2023-10-16T20:05:39.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.473+0000 7f0dbdb60640 1 --2- 172.21.15.88:0/2909237900 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0d9c0664a0 0x7f0d9c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:39.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.473+0000 7f0dbdb60640 1 --2- 172.21.15.88:0/2909237900 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0d9c0664a0 0x7f0d9c068960 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0db4005890 tx=0x7f0db4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:39.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.477+0000 7f0dae7fc640 1 -- 172.21.15.88:0/2909237900 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0da8051610 con 0x7f0db80b4260 2023-10-16T20:05:39.624 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.621+0000 7f0dbfdc4640 1 -- 172.21.15.88:0/2909237900 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0db8071500 con 0x7f0db80b4260 2023-10-16T20:05:39.624 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.621+0000 7f0dae7fc640 1 -- 172.21.15.88:0/2909237900 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0da8054c30 con 0x7f0db80b4260 2023-10-16T20:05:39.625 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2023-10-16T20:05:39.626 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.625+0000 7f0dbfdc4640 1 -- 172.21.15.88:0/2909237900 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0d9c0664a0 msgr2=0x7f0d9c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:39.627 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.625+0000 7f0dbfdc4640 1 --2- 172.21.15.88:0/2909237900 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0d9c0664a0 0x7f0d9c068960 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0db4005890 tx=0x7f0db4009040 comp rx=0 tx=0).stop 2023-10-16T20:05:39.627 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.625+0000 7f0dbfdc4640 1 -- 172.21.15.88:0/2909237900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0db80b4260 msgr2=0x7f0db814d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:39.627 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.625+0000 7f0dbfdc4640 1 --2- 172.21.15.88:0/2909237900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0db80b4260 0x7f0db814d450 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0da800e930 tx=0x7f0da800edf0 comp rx=0 tx=0).stop 2023-10-16T20:05:39.627 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.625+0000 7f0dbfdc4640 1 -- 172.21.15.88:0/2909237900 shutdown_connections 2023-10-16T20:05:39.627 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.625+0000 7f0dbfdc4640 1 --2- 172.21.15.88:0/2909237900 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0d9c0664a0 0x7f0d9c068960 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:39.628 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.625+0000 7f0dbfdc4640 1 --2- 172.21.15.88:0/2909237900 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0db815a1f0 0x7f0db814d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:39.628 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.625+0000 7f0dbfdc4640 1 --2- 172.21.15.88:0/2909237900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0db80b4260 0x7f0db814d450 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:39.628 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.625+0000 7f0dbfdc4640 1 --2- 172.21.15.88:0/2909237900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0db80b0da0 0x7f0db8149a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:39.628 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.625+0000 7f0dbfdc4640 1 -- 172.21.15.88:0/2909237900 >> 172.21.15.88:0/2909237900 conn(0x7f0db80cb080 msgr2=0x7f0db80af790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:39.628 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.625+0000 7f0dbfdc4640 1 -- 172.21.15.88:0/2909237900 shutdown_connections 2023-10-16T20:05:39.628 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.625+0000 7f0dbfdc4640 1 -- 172.21.15.88:0/2909237900 wait complete. 2023-10-16T20:05:39.645 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T20:05:39.785 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.781+0000 7fc21e50f640 1 Processor -- start 2023-10-16T20:05:39.785 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.781+0000 7fc21e50f640 1 -- start start 2023-10-16T20:05:39.786 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.785+0000 7fc21e50f640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc2180ca0b0 0x7fc218161540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:39.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.785+0000 7fc21e50f640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fc218071360 0x7fc2180717b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:39.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.785+0000 7fc21e50f640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc218071cf0 con 0x7fc21815cf90 2023-10-16T20:05:39.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.785+0000 7fc21e50f640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc218161a80 con 0x7fc2180ca0b0 2023-10-16T20:05:39.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.785+0000 7fc21e50f640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2180ca840 con 0x7fc218071360 2023-10-16T20:05:39.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.785+0000 7fc2177fe640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc2180ca0b0 0x7fc218161540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:39.788 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.785+0000 7fc2177fe640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc2180ca0b0 0x7fc218161540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47200/0 (socket says 172.21.15.88:47200) 2023-10-16T20:05:39.788 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.785+0000 7fc2177fe640 1 -- 172.21.15.88:0/4041871875 learned_addr learned my addr 172.21.15.88:0/4041871875 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:39.789 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.785+0000 7fc21caac640 1 --2- 172.21.15.88:0/4041871875 >> v2:172.21.15.94:3300/0 conn(0x7fc218071360 0x7fc2180717b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:39.789 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.785+0000 7fc2177fe640 1 -- 172.21.15.88:0/4041871875 >> v2:172.21.15.94:3300/0 conn(0x7fc218071360 msgr2=0x7fc2180717b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:39.789 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.785+0000 7fc2177fe640 1 --2- 172.21.15.88:0/4041871875 >> v2:172.21.15.94:3300/0 conn(0x7fc218071360 0x7fc2180717b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:39.789 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.785+0000 7fc2177fe640 1 -- 172.21.15.88:0/4041871875 >> v1:172.21.15.94:6789/0 conn(0x7fc21815cf90 legacy=0x7fc2180cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:39.789 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.785+0000 7fc2177fe640 1 -- 172.21.15.88:0/4041871875 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2180cab20 con 0x7fc2180ca0b0 2023-10-16T20:05:39.790 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.785+0000 7fc21caac640 1 --2- 172.21.15.88:0/4041871875 >> v2:172.21.15.94:3300/0 conn(0x7fc218071360 0x7fc2180717b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:39.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.789+0000 7fc2177fe640 1 --2- 172.21.15.88:0/4041871875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2180ca0b0 0x7fc218161540 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc2180712f0 tx=0x7fc20c01c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28cc263ac293a98e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:39.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.789+0000 7fc216ffd640 1 -- 172.21.15.88:0/4041871875 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c0ac410 con 0x7fc2180ca0b0 2023-10-16T20:05:39.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.789+0000 7fc216ffd640 1 -- 172.21.15.88:0/4041871875 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20c0ac570 con 0x7fc2180ca0b0 2023-10-16T20:05:39.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.789+0000 7fc216ffd640 1 -- 172.21.15.88:0/4041871875 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c0ac900 con 0x7fc2180ca0b0 2023-10-16T20:05:39.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.789+0000 7fc21e50f640 1 -- 172.21.15.88:0/4041871875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2180ca0b0 msgr2=0x7fc218161540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:39.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.789+0000 7fc21e50f640 1 --2- 172.21.15.88:0/4041871875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2180ca0b0 0x7fc218161540 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc2180712f0 tx=0x7fc20c01c4c0 comp rx=0 tx=0).stop 2023-10-16T20:05:39.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.789+0000 7fc21e50f640 1 -- 172.21.15.88:0/4041871875 shutdown_connections 2023-10-16T20:05:39.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.789+0000 7fc21e50f640 1 --2- 172.21.15.88:0/4041871875 >> v2:172.21.15.94:3300/0 conn(0x7fc218071360 0x7fc2180717b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:39.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.789+0000 7fc21e50f640 1 --2- 172.21.15.88:0/4041871875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2180ca0b0 0x7fc218161540 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:39.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.789+0000 7fc21e50f640 1 -- 172.21.15.88:0/4041871875 >> 172.21.15.88:0/4041871875 conn(0x7fc2180cf710 msgr2=0x7fc2180cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:39.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.789+0000 7fc21e50f640 1 -- 172.21.15.88:0/4041871875 shutdown_connections 2023-10-16T20:05:39.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.789+0000 7fc21e50f640 1 -- 172.21.15.88:0/4041871875 wait complete. 2023-10-16T20:05:39.795 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.789+0000 7fc21e50f640 1 Processor -- start 2023-10-16T20:05:39.795 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.789+0000 7fc21e50f640 1 -- start start 2023-10-16T20:05:39.796 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.793+0000 7fc21e50f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc218071360 0x7fc21814e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:39.796 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.793+0000 7fc21e50f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2180ca0b0 0x7fc21815bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:39.796 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.793+0000 7fc21e50f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc21815cf90 0x7fc21814c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:39.796 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.793+0000 7fc21e50f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21815c700 con 0x7fc2180ca0b0 2023-10-16T20:05:39.796 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.793+0000 7fc21e50f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21814fcf0 con 0x7fc218071360 2023-10-16T20:05:39.797 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.793+0000 7fc21e50f640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21814f0e0 con 0x7fc21815cf90 2023-10-16T20:05:39.797 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.793+0000 7fc2177fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2180ca0b0 0x7fc21815bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:39.797 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.793+0000 7fc2177fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2180ca0b0 0x7fc21815bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47210/0 (socket says 172.21.15.88:47210) 2023-10-16T20:05:39.797 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.793+0000 7fc2177fe640 1 -- 172.21.15.88:0/1352935461 learned_addr learned my addr 172.21.15.88:0/1352935461 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:39.797 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.793+0000 7fc217fff640 1 --2- 172.21.15.88:0/1352935461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc218071360 0x7fc21814e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:39.798 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.793+0000 7fc21caac640 1 --2- 172.21.15.88:0/1352935461 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc21815cf90 0x7fc21814c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:39.798 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.793+0000 7fc2177fe640 1 -- 172.21.15.88:0/1352935461 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc21815cf90 msgr2=0x7fc21814c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:39.798 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.793+0000 7fc2177fe640 1 --2- 172.21.15.88:0/1352935461 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc21815cf90 0x7fc21814c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:39.798 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.793+0000 7fc2177fe640 1 -- 172.21.15.88:0/1352935461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc218071360 msgr2=0x7fc21814e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:39.798 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.793+0000 7fc2177fe640 1 --2- 172.21.15.88:0/1352935461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc218071360 0x7fc21814e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:39.799 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.793+0000 7fc2177fe640 1 -- 172.21.15.88:0/1352935461 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20c0094d0 con 0x7fc2180ca0b0 2023-10-16T20:05:39.799 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.793+0000 7fc2177fe640 1 --2- 172.21.15.88:0/1352935461 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2180ca0b0 0x7fc21815bae0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc20c0af040 tx=0x7fc20c019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:39.799 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.793+0000 7fc214ff9640 1 -- 172.21.15.88:0/1352935461 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c0ace20 con 0x7fc2180ca0b0 2023-10-16T20:05:39.799 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.793+0000 7fc214ff9640 1 -- 172.21.15.88:0/1352935461 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20c0b5630 con 0x7fc2180ca0b0 2023-10-16T20:05:39.799 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.793+0000 7fc214ff9640 1 -- 172.21.15.88:0/1352935461 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c0b59e0 con 0x7fc2180ca0b0 2023-10-16T20:05:39.800 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.793+0000 7fc21e50f640 1 -- 172.21.15.88:0/1352935461 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc21814f3c0 con 0x7fc2180ca0b0 2023-10-16T20:05:39.800 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.793+0000 7fc21e50f640 1 -- 172.21.15.88:0/1352935461 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc21814f980 con 0x7fc2180ca0b0 2023-10-16T20:05:39.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.793+0000 7fc21e50f640 1 -- 172.21.15.88:0/1352935461 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc21814fe60 con 0x7fc2180ca0b0 2023-10-16T20:05:39.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.793+0000 7fc214ff9640 1 -- 172.21.15.88:0/1352935461 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc20c0b3070 con 0x7fc2180ca0b0 2023-10-16T20:05:39.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.797+0000 7fc214ff9640 1 --2- 172.21.15.88:0/1352935461 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc2040664a0 0x7fc204068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:39.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.797+0000 7fc214ff9640 1 -- 172.21.15.88:0/1352935461 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc20c11e740 con 0x7fc2180ca0b0 2023-10-16T20:05:39.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.797+0000 7fc217fff640 1 --2- 172.21.15.88:0/1352935461 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc2040664a0 0x7fc204068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:39.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.797+0000 7fc217fff640 1 --2- 172.21.15.88:0/1352935461 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc2040664a0 0x7fc204068960 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc208004e30 tx=0x7fc208009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:39.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.801+0000 7fc214ff9640 1 -- 172.21.15.88:0/1352935461 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc20c0eb760 con 0x7fc2180ca0b0 2023-10-16T20:05:39.951 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.949+0000 7fc21e50f640 1 -- 172.21.15.88:0/1352935461 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc218071cf0 con 0x7fc2180ca0b0 2023-10-16T20:05:39.951 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.949+0000 7fc214ff9640 1 -- 172.21.15.88:0/1352935461 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc218071cf0 con 0x7fc2180ca0b0 2023-10-16T20:05:39.951 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:05:39.953 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.949+0000 7fc21e50f640 1 -- 172.21.15.88:0/1352935461 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc2040664a0 msgr2=0x7fc204068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:39.953 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.949+0000 7fc21e50f640 1 --2- 172.21.15.88:0/1352935461 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc2040664a0 0x7fc204068960 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc208004e30 tx=0x7fc208009040 comp rx=0 tx=0).stop 2023-10-16T20:05:39.953 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.949+0000 7fc21e50f640 1 -- 172.21.15.88:0/1352935461 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2180ca0b0 msgr2=0x7fc21815bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:39.953 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.949+0000 7fc21e50f640 1 --2- 172.21.15.88:0/1352935461 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2180ca0b0 0x7fc21815bae0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc20c0af040 tx=0x7fc20c019260 comp rx=0 tx=0).stop 2023-10-16T20:05:39.954 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.949+0000 7fc21e50f640 1 -- 172.21.15.88:0/1352935461 shutdown_connections 2023-10-16T20:05:39.954 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.949+0000 7fc21e50f640 1 --2- 172.21.15.88:0/1352935461 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc2040664a0 0x7fc204068960 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:39.954 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.949+0000 7fc21e50f640 1 --2- 172.21.15.88:0/1352935461 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc21815cf90 0x7fc21814c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:39.954 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.949+0000 7fc21e50f640 1 --2- 172.21.15.88:0/1352935461 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2180ca0b0 0x7fc21815bae0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:39.954 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.949+0000 7fc21e50f640 1 --2- 172.21.15.88:0/1352935461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc218071360 0x7fc21814e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:39.955 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.949+0000 7fc21e50f640 1 -- 172.21.15.88:0/1352935461 >> 172.21.15.88:0/1352935461 conn(0x7fc2180cf710 msgr2=0x7fc21815a590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:39.955 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.949+0000 7fc21e50f640 1 -- 172.21.15.88:0/1352935461 shutdown_connections 2023-10-16T20:05:39.955 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:39.949+0000 7fc21e50f640 1 -- 172.21.15.88:0/1352935461 wait complete. 2023-10-16T20:05:39.972 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:05:40.109 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.105+0000 7f26c0765640 1 Processor -- start 2023-10-16T20:05:40.109 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.105+0000 7f26c0765640 1 -- start start 2023-10-16T20:05:40.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.109+0000 7f26c0765640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f26b81599f0 0x7f26b80b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:40.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.109+0000 7f26c0765640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f26b80b0cb0 0x7f26b806a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:40.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.109+0000 7f26c0765640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b806a830 con 0x7f26b8069e90 2023-10-16T20:05:40.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.109+0000 7f26c0765640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26b80b1260 con 0x7f26b81599f0 2023-10-16T20:05:40.112 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.109+0000 7f26c0765640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b80ae960 con 0x7f26b80b0cb0 2023-10-16T20:05:40.112 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.109+0000 7f26be501640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f26b81599f0 0x7f26b80b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:40.112 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.109+0000 7f26be501640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f26b81599f0 0x7f26b80b2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47214/0 (socket says 172.21.15.88:47214) 2023-10-16T20:05:40.112 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.109+0000 7f26be501640 1 -- 172.21.15.88:0/3203454816 learned_addr learned my addr 172.21.15.88:0/3203454816 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:40.112 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.109+0000 7f26bed02640 1 --2- 172.21.15.88:0/3203454816 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f26b80b0cb0 0x7f26b806a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:40.113 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.109+0000 7f26be501640 1 -- 172.21.15.88:0/3203454816 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f26b80b0cb0 msgr2=0x7f26b806a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:40.113 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.109+0000 7f26be501640 1 --2- 172.21.15.88:0/3203454816 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f26b80b0cb0 0x7f26b806a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:40.113 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.109+0000 7f26be501640 1 -- 172.21.15.88:0/3203454816 >> v1:172.21.15.94:6789/0 conn(0x7f26b8069e90 legacy=0x7f26b80b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:40.113 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.109+0000 7f26be501640 1 -- 172.21.15.88:0/3203454816 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b80aec40 con 0x7f26b81599f0 2023-10-16T20:05:40.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.113+0000 7f26be501640 1 --2- 172.21.15.88:0/3203454816 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26b81599f0 0x7f26b80b2a30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f26b4009870 tx=0x7f26b401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d56cd721638fbf98 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:40.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.113+0000 7f26bd4ff640 1 -- 172.21.15.88:0/3203454816 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b40aa670 con 0x7f26b81599f0 2023-10-16T20:05:40.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.113+0000 7f26bd4ff640 1 -- 172.21.15.88:0/3203454816 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b40aa7d0 con 0x7f26b81599f0 2023-10-16T20:05:40.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.113+0000 7f26bd4ff640 1 -- 172.21.15.88:0/3203454816 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b40aab60 con 0x7f26b81599f0 2023-10-16T20:05:40.115 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.113+0000 7f26c0765640 1 -- 172.21.15.88:0/3203454816 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26b81599f0 msgr2=0x7f26b80b2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:40.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.113+0000 7f26c0765640 1 --2- 172.21.15.88:0/3203454816 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26b81599f0 0x7f26b80b2a30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f26b4009870 tx=0x7f26b401c510 comp rx=0 tx=0).stop 2023-10-16T20:05:40.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.113+0000 7f26c0765640 1 -- 172.21.15.88:0/3203454816 shutdown_connections 2023-10-16T20:05:40.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.113+0000 7f26c0765640 1 --2- 172.21.15.88:0/3203454816 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f26b80b0cb0 0x7f26b806a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:40.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.113+0000 7f26c0765640 1 --2- 172.21.15.88:0/3203454816 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26b81599f0 0x7f26b80b2a30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:40.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.113+0000 7f26c0765640 1 -- 172.21.15.88:0/3203454816 >> 172.21.15.88:0/3203454816 conn(0x7f26b80ce5e0 msgr2=0x7f26b80ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:40.117 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.113+0000 7f26c0765640 1 -- 172.21.15.88:0/3203454816 shutdown_connections 2023-10-16T20:05:40.117 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.113+0000 7f26c0765640 1 -- 172.21.15.88:0/3203454816 wait complete. 2023-10-16T20:05:40.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.113+0000 7f26c0765640 1 Processor -- start 2023-10-16T20:05:40.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.113+0000 7f26c0765640 1 -- start start 2023-10-16T20:05:40.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.117+0000 7f26c0765640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26b8069e90 0x7f26b80abaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:40.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.117+0000 7f26c0765640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26b80b0cb0 0x7f26b80abfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:40.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.117+0000 7f26c0765640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f26b81599f0 0x7f26b80ac520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:40.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.117+0000 7f26c0765640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b8167960 con 0x7f26b80b0cb0 2023-10-16T20:05:40.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.117+0000 7f26c0765640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b814a570 con 0x7f26b8069e90 2023-10-16T20:05:40.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.117+0000 7f26c0765640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b814b3d0 con 0x7f26b81599f0 2023-10-16T20:05:40.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.117+0000 7f26bdd00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26b80b0cb0 0x7f26b80abfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:40.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.117+0000 7f26bdd00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26b80b0cb0 0x7f26b80abfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47226/0 (socket says 172.21.15.88:47226) 2023-10-16T20:05:40.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.117+0000 7f26be501640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26b8069e90 0x7f26b80abaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:40.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.117+0000 7f26bdd00640 1 -- 172.21.15.88:0/148718267 learned_addr learned my addr 172.21.15.88:0/148718267 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:40.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.117+0000 7f26bed02640 1 --2- 172.21.15.88:0/148718267 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f26b81599f0 0x7f26b80ac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:40.122 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.117+0000 7f26bdd00640 1 -- 172.21.15.88:0/148718267 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f26b81599f0 msgr2=0x7f26b80ac520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:40.122 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.117+0000 7f26bdd00640 1 --2- 172.21.15.88:0/148718267 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f26b81599f0 0x7f26b80ac520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:40.122 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.117+0000 7f26bdd00640 1 -- 172.21.15.88:0/148718267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26b8069e90 msgr2=0x7f26b80abaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:40.122 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.117+0000 7f26bdd00640 1 --2- 172.21.15.88:0/148718267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26b8069e90 0x7f26b80abaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:40.122 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.117+0000 7f26bdd00640 1 -- 172.21.15.88:0/148718267 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b40094d0 con 0x7f26b80b0cb0 2023-10-16T20:05:40.123 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.117+0000 7f26bdd00640 1 --2- 172.21.15.88:0/148718267 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26b80b0cb0 0x7f26b80abfe0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f26a800c500 tx=0x7f26a800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:40.123 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.117+0000 7f26aeffd640 1 -- 172.21.15.88:0/148718267 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a8005220 con 0x7f26b80b0cb0 2023-10-16T20:05:40.123 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.117+0000 7f26aeffd640 1 -- 172.21.15.88:0/148718267 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a8005380 con 0x7f26b80b0cb0 2023-10-16T20:05:40.123 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.117+0000 7f26aeffd640 1 -- 172.21.15.88:0/148718267 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a8007270 con 0x7f26b80b0cb0 2023-10-16T20:05:40.123 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.117+0000 7f26c0765640 1 -- 172.21.15.88:0/148718267 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26b814a7c0 con 0x7f26b80b0cb0 2023-10-16T20:05:40.124 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.117+0000 7f26c0765640 1 -- 172.21.15.88:0/148718267 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b814ad80 con 0x7f26b80b0cb0 2023-10-16T20:05:40.125 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.117+0000 7f26c0765640 1 -- 172.21.15.88:0/148718267 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26b814b540 con 0x7f26b80b0cb0 2023-10-16T20:05:40.125 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.117+0000 7f26aeffd640 1 -- 172.21.15.88:0/148718267 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f26a80073d0 con 0x7f26b80b0cb0 2023-10-16T20:05:40.125 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.121+0000 7f26aeffd640 1 --2- 172.21.15.88:0/148718267 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f269c0664a0 0x7f269c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:40.125 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.121+0000 7f26aeffd640 1 -- 172.21.15.88:0/148718267 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f26a80839a0 con 0x7f26b80b0cb0 2023-10-16T20:05:40.125 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.121+0000 7f26be501640 1 --2- 172.21.15.88:0/148718267 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f269c0664a0 0x7f269c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:40.126 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.121+0000 7f26be501640 1 --2- 172.21.15.88:0/148718267 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f269c0664a0 0x7f269c068960 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f26b40ae000 tx=0x7f26b40b5040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:40.126 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.125+0000 7f26aeffd640 1 -- 172.21.15.88:0/148718267 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f26a8050b20 con 0x7f26b80b0cb0 2023-10-16T20:05:40.279 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.277+0000 7f26c0765640 1 -- 172.21.15.88:0/148718267 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26b804e9c0 con 0x7f26b80b0cb0 2023-10-16T20:05:40.280 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.277+0000 7f26aeffd640 1 -- 172.21.15.88:0/148718267 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18661 (secure 0 0 0) 0x7f26a8054140 con 0x7f26b80b0cb0 2023-10-16T20:05:40.281 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:40.281 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":29,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:05:38.202959+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T20:04:54.442487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T20:04:55.766641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T20:04:59.475830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":1}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:05:40.283 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.281+0000 7f26c0765640 1 -- 172.21.15.88:0/148718267 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f269c0664a0 msgr2=0x7f269c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:40.283 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.281+0000 7f26c0765640 1 --2- 172.21.15.88:0/148718267 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f269c0664a0 0x7f269c068960 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f26b40ae000 tx=0x7f26b40b5040 comp rx=0 tx=0).stop 2023-10-16T20:05:40.283 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.281+0000 7f26c0765640 1 -- 172.21.15.88:0/148718267 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26b80b0cb0 msgr2=0x7f26b80abfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:40.283 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.281+0000 7f26c0765640 1 --2- 172.21.15.88:0/148718267 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26b80b0cb0 0x7f26b80abfe0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f26a800c500 tx=0x7f26a800c9c0 comp rx=0 tx=0).stop 2023-10-16T20:05:40.283 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.281+0000 7f26c0765640 1 -- 172.21.15.88:0/148718267 shutdown_connections 2023-10-16T20:05:40.284 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.281+0000 7f26c0765640 1 --2- 172.21.15.88:0/148718267 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f269c0664a0 0x7f269c068960 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:40.284 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.281+0000 7f26c0765640 1 --2- 172.21.15.88:0/148718267 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f26b81599f0 0x7f26b80ac520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:40.284 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.281+0000 7f26c0765640 1 --2- 172.21.15.88:0/148718267 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26b80b0cb0 0x7f26b80abfe0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:40.284 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.281+0000 7f26c0765640 1 --2- 172.21.15.88:0/148718267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26b8069e90 0x7f26b80abaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:40.285 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.281+0000 7f26c0765640 1 -- 172.21.15.88:0/148718267 >> 172.21.15.88:0/148718267 conn(0x7f26b80ce5e0 msgr2=0x7f26b8071a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:40.285 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.281+0000 7f26c0765640 1 -- 172.21.15.88:0/148718267 shutdown_connections 2023-10-16T20:05:40.285 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.281+0000 7f26c0765640 1 -- 172.21.15.88:0/148718267 wait complete. 2023-10-16T20:05:40.302 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T20:04:54.442487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T20:04:55.766641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T20:04:59.475830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T20:05:40.303 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:05:40.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.441+0000 7f76f533e640 1 Processor -- start 2023-10-16T20:05:40.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.441+0000 7f76f533e640 1 -- start start 2023-10-16T20:05:40.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.441+0000 7f76f533e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76f00d1be0 0x7f76f00b4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:40.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.441+0000 7f76f533e640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f76f00b4a70 0x7f76f00b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:40.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.445+0000 7f76f533e640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76f00b7d20 con 0x7f76f015cb70 2023-10-16T20:05:40.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.445+0000 7f76f533e640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76f00b4f00 con 0x7f76f00b4a70 2023-10-16T20:05:40.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.445+0000 7f76f533e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f00b7110 con 0x7f76f00d1be0 2023-10-16T20:05:40.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.445+0000 7f76eeffd640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f76f015cb70 0x7f76f00b83d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:54768/0 (socket says 172.21.15.88:54768) 2023-10-16T20:05:40.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.445+0000 7f76eeffd640 1 -- 172.21.15.88:0/2240116617 learned_addr learned my addr 172.21.15.88:0/2240116617 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:40.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.445+0000 7f76ef7fe640 1 --2- 172.21.15.88:0/2240116617 >> v2:172.21.15.88:3300/0 conn(0x7f76f00b4a70 0x7f76f00b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:40.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.445+0000 7f76e7fff640 1 --2- 172.21.15.88:0/2240116617 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76f00d1be0 0x7f76f00b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:40.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.445+0000 7f76ee7fc640 1 -- 172.21.15.88:0/2240116617 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2474133863 0 0) 0x7f76f00b7d20 con 0x7f76f015cb70 2023-10-16T20:05:40.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.445+0000 7f76ee7fc640 1 -- 172.21.15.88:0/2240116617 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76dc003530 con 0x7f76f015cb70 2023-10-16T20:05:40.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.445+0000 7f76ef7fe640 1 -- 172.21.15.88:0/2240116617 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76f00d1be0 msgr2=0x7f76f00b4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:40.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.445+0000 7f76ef7fe640 1 --2- 172.21.15.88:0/2240116617 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76f00d1be0 0x7f76f00b4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:40.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.445+0000 7f76ef7fe640 1 -- 172.21.15.88:0/2240116617 >> v1:172.21.15.88:6789/0 conn(0x7f76f015cb70 legacy=0x7f76f00b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:40.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.445+0000 7f76ef7fe640 1 -- 172.21.15.88:0/2240116617 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f00b73f0 con 0x7f76f00b4a70 2023-10-16T20:05:40.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.445+0000 7f76ef7fe640 1 --2- 172.21.15.88:0/2240116617 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76f00b4a70 0x7f76f00b5a70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f76e8009d90 tx=0x7f76e801f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd64100f73276270 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:40.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.449+0000 7f76ee7fc640 1 -- 172.21.15.88:0/2240116617 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e80af600 con 0x7f76f00b4a70 2023-10-16T20:05:40.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.449+0000 7f76ee7fc640 1 -- 172.21.15.88:0/2240116617 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e80ab050 con 0x7f76f00b4a70 2023-10-16T20:05:40.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.449+0000 7f76ee7fc640 1 -- 172.21.15.88:0/2240116617 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e80af980 con 0x7f76f00b4a70 2023-10-16T20:05:40.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.449+0000 7f76f533e640 1 -- 172.21.15.88:0/2240116617 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76f00b4a70 msgr2=0x7f76f00b5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:40.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.449+0000 7f76f533e640 1 --2- 172.21.15.88:0/2240116617 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76f00b4a70 0x7f76f00b5a70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f76e8009d90 tx=0x7f76e801f130 comp rx=0 tx=0).stop 2023-10-16T20:05:40.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.449+0000 7f76f533e640 1 -- 172.21.15.88:0/2240116617 shutdown_connections 2023-10-16T20:05:40.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.449+0000 7f76f533e640 1 --2- 172.21.15.88:0/2240116617 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76f00b4a70 0x7f76f00b5a70 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:40.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.449+0000 7f76f533e640 1 --2- 172.21.15.88:0/2240116617 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76f00d1be0 0x7f76f00b4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:40.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.449+0000 7f76f533e640 1 -- 172.21.15.88:0/2240116617 >> 172.21.15.88:0/2240116617 conn(0x7f76f006b6e0 msgr2=0x7f76f006baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:40.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.449+0000 7f76f533e640 1 -- 172.21.15.88:0/2240116617 shutdown_connections 2023-10-16T20:05:40.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.449+0000 7f76f533e640 1 -- 172.21.15.88:0/2240116617 wait complete. 2023-10-16T20:05:40.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.449+0000 7f76f533e640 1 Processor -- start 2023-10-16T20:05:40.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.449+0000 7f76f533e640 1 -- start start 2023-10-16T20:05:40.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.449+0000 7f76f533e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76f00b4a70 0x7f76f015e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:40.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.449+0000 7f76f533e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76f00d1be0 0x7f76f00b5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:40.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.449+0000 7f76f533e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76f015cb70 0x7f76f014eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:40.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.449+0000 7f76f533e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f015f7b0 con 0x7f76f015cb70 2023-10-16T20:05:40.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.449+0000 7f76f533e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f015eba0 con 0x7f76f00d1be0 2023-10-16T20:05:40.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.449+0000 7f76f533e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f015ee80 con 0x7f76f00b4a70 2023-10-16T20:05:40.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.453+0000 7f76ef7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76f015cb70 0x7f76f014eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:40.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.453+0000 7f76ef7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76f015cb70 0x7f76f014eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47252/0 (socket says 172.21.15.88:47252) 2023-10-16T20:05:40.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.453+0000 7f76ef7fe640 1 -- 172.21.15.88:0/1356183616 learned_addr learned my addr 172.21.15.88:0/1356183616 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:40.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.453+0000 7f76eeffd640 1 --2- 172.21.15.88:0/1356183616 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76f00b4a70 0x7f76f015e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:40.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.453+0000 7f76e7fff640 1 --2- 172.21.15.88:0/1356183616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76f00d1be0 0x7f76f00b5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:40.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.453+0000 7f76ef7fe640 1 -- 172.21.15.88:0/1356183616 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76f00b4a70 msgr2=0x7f76f015e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:40.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.453+0000 7f76ef7fe640 1 --2- 172.21.15.88:0/1356183616 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76f00b4a70 0x7f76f015e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:40.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.453+0000 7f76ef7fe640 1 -- 172.21.15.88:0/1356183616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76f00d1be0 msgr2=0x7f76f00b5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:40.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.453+0000 7f76ef7fe640 1 --2- 172.21.15.88:0/1356183616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76f00d1be0 0x7f76f00b5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:40.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.453+0000 7f76ef7fe640 1 -- 172.21.15.88:0/1356183616 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e8009a70 con 0x7f76f015cb70 2023-10-16T20:05:40.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.453+0000 7f76e7fff640 1 --2- 172.21.15.88:0/1356183616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76f00d1be0 0x7f76f00b5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:40.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.453+0000 7f76ef7fe640 1 --2- 172.21.15.88:0/1356183616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76f015cb70 0x7f76f014eb70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f76e80b2040 tx=0x7f76e801bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:40.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.453+0000 7f76e77fe640 1 -- 172.21.15.88:0/1356183616 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e8009450 con 0x7f76f015cb70 2023-10-16T20:05:40.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.453+0000 7f76e77fe640 1 -- 172.21.15.88:0/1356183616 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e80b9410 con 0x7f76f015cb70 2023-10-16T20:05:40.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.453+0000 7f76f533e640 1 -- 172.21.15.88:0/1356183616 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f015f160 con 0x7f76f015cb70 2023-10-16T20:05:40.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.453+0000 7f76e77fe640 1 -- 172.21.15.88:0/1356183616 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e80b97c0 con 0x7f76f015cb70 2023-10-16T20:05:40.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.453+0000 7f76f533e640 1 -- 172.21.15.88:0/1356183616 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f015e8b0 con 0x7f76f015cb70 2023-10-16T20:05:40.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.453+0000 7f76f533e640 1 -- 172.21.15.88:0/1356183616 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76f015f920 con 0x7f76f015cb70 2023-10-16T20:05:40.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.453+0000 7f76e77fe640 1 -- 172.21.15.88:0/1356183616 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f76e80b6050 con 0x7f76f015cb70 2023-10-16T20:05:40.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.453+0000 7f76e77fe640 1 --2- 172.21.15.88:0/1356183616 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f76d00664a0 0x7f76d0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:40.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.453+0000 7f76e77fe640 1 -- 172.21.15.88:0/1356183616 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f76e8122740 con 0x7f76f015cb70 2023-10-16T20:05:40.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.453+0000 7f76eeffd640 1 --2- 172.21.15.88:0/1356183616 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f76d00664a0 0x7f76d0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:40.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.457+0000 7f76eeffd640 1 --2- 172.21.15.88:0/1356183616 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f76d00664a0 0x7f76d0068960 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f76e00011f0 tx=0x7f76e0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:40.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.457+0000 7f76e77fe640 1 -- 172.21.15.88:0/1356183616 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f76e80ef760 con 0x7f76f015cb70 2023-10-16T20:05:40.611 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.609+0000 7f76f533e640 1 -- 172.21.15.88:0/1356183616 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f76f004e9c0 con 0x7f76f015cb70 2023-10-16T20:05:40.611 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.609+0000 7f76e77fe640 1 -- 172.21.15.88:0/1356183616 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f76e80f2d80 con 0x7f76f015cb70 2023-10-16T20:05:40.612 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:05:40.614 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.609+0000 7f76f533e640 1 -- 172.21.15.88:0/1356183616 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f76d00664a0 msgr2=0x7f76d0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:40.614 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.609+0000 7f76f533e640 1 --2- 172.21.15.88:0/1356183616 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f76d00664a0 0x7f76d0068960 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f76e00011f0 tx=0x7f76e0009040 comp rx=0 tx=0).stop 2023-10-16T20:05:40.615 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.609+0000 7f76f533e640 1 -- 172.21.15.88:0/1356183616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76f015cb70 msgr2=0x7f76f014eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:40.615 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.609+0000 7f76f533e640 1 --2- 172.21.15.88:0/1356183616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76f015cb70 0x7f76f014eb70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f76e80b2040 tx=0x7f76e801bed0 comp rx=0 tx=0).stop 2023-10-16T20:05:40.615 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.609+0000 7f76f533e640 1 -- 172.21.15.88:0/1356183616 shutdown_connections 2023-10-16T20:05:40.616 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.609+0000 7f76f533e640 1 --2- 172.21.15.88:0/1356183616 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f76d00664a0 0x7f76d0068960 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:40.616 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.609+0000 7f76f533e640 1 --2- 172.21.15.88:0/1356183616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76f015cb70 0x7f76f014eb70 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:40.616 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.609+0000 7f76f533e640 1 --2- 172.21.15.88:0/1356183616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76f00d1be0 0x7f76f00b5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:40.617 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.609+0000 7f76f533e640 1 --2- 172.21.15.88:0/1356183616 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76f00b4a70 0x7f76f015e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:40.617 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.609+0000 7f76f533e640 1 -- 172.21.15.88:0/1356183616 >> 172.21.15.88:0/1356183616 conn(0x7f76f006b6e0 msgr2=0x7f76f00d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:40.617 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.609+0000 7f76f533e640 1 -- 172.21.15.88:0/1356183616 shutdown_connections 2023-10-16T20:05:40.617 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.609+0000 7f76f533e640 1 -- 172.21.15.88:0/1356183616 wait complete. 2023-10-16T20:05:40.632 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T20:05:40.774 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.769+0000 7fb8f189e640 1 Processor -- start 2023-10-16T20:05:40.774 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.773+0000 7fb8f189e640 1 -- start start 2023-10-16T20:05:40.776 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.773+0000 7fb8f189e640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb8ec0d1be0 0x7fb8ec0b4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:40.776 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.773+0000 7fb8f189e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb8ec0b4a70 0x7fb8ec0b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:40.776 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.773+0000 7fb8f189e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8ec0b7d20 con 0x7fb8ec15cb70 2023-10-16T20:05:40.776 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.773+0000 7fb8f189e640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8ec0b4ea0 con 0x7fb8ec0d1be0 2023-10-16T20:05:40.776 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.773+0000 7fb8f189e640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8ec0b7110 con 0x7fb8ec0b4a70 2023-10-16T20:05:40.777 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.773+0000 7fb8e27fc640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb8ec0d1be0 0x7fb8ec0b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:40.777 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.773+0000 7fb8e27fc640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb8ec0d1be0 0x7fb8ec0b4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47268/0 (socket says 172.21.15.88:47268) 2023-10-16T20:05:40.777 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.773+0000 7fb8e27fc640 1 -- 172.21.15.88:0/1216333821 learned_addr learned my addr 172.21.15.88:0/1216333821 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:40.777 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.773+0000 7fb8eb7fe640 1 --2- 172.21.15.88:0/1216333821 >> v2:172.21.15.94:3300/0 conn(0x7fb8ec0b4a70 0x7fb8ec0b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:40.777 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.773+0000 7fb8e27fc640 1 -- 172.21.15.88:0/1216333821 >> v2:172.21.15.94:3300/0 conn(0x7fb8ec0b4a70 msgr2=0x7fb8ec0b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:40.778 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.773+0000 7fb8e27fc640 1 --2- 172.21.15.88:0/1216333821 >> v2:172.21.15.94:3300/0 conn(0x7fb8ec0b4a70 0x7fb8ec0b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:40.778 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.773+0000 7fb8e27fc640 1 -- 172.21.15.88:0/1216333821 >> v1:172.21.15.94:6789/0 conn(0x7fb8ec15cb70 legacy=0x7fb8ec0b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:40.778 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.773+0000 7fb8e27fc640 1 -- 172.21.15.88:0/1216333821 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ec0b73f0 con 0x7fb8ec0d1be0 2023-10-16T20:05:40.778 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.773+0000 7fb8eb7fe640 1 --2- 172.21.15.88:0/1216333821 >> v2:172.21.15.94:3300/0 conn(0x7fb8ec0b4a70 0x7fb8ec0b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:40.780 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.777+0000 7fb8e27fc640 1 --2- 172.21.15.88:0/1216333821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8ec0d1be0 0x7fb8ec0b4530 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec0d03c0 tx=0x7fb8d401ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b726ceb9cd0ae70 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:40.780 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.777+0000 7fb8ea7fc640 1 -- 172.21.15.88:0/1216333821 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d40a89b0 con 0x7fb8ec0d1be0 2023-10-16T20:05:40.780 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.777+0000 7fb8ea7fc640 1 -- 172.21.15.88:0/1216333821 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d40a8b10 con 0x7fb8ec0d1be0 2023-10-16T20:05:40.780 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.777+0000 7fb8ea7fc640 1 -- 172.21.15.88:0/1216333821 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d40a8ea0 con 0x7fb8ec0d1be0 2023-10-16T20:05:40.781 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.777+0000 7fb8f189e640 1 -- 172.21.15.88:0/1216333821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8ec0d1be0 msgr2=0x7fb8ec0b4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:40.781 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.777+0000 7fb8f189e640 1 --2- 172.21.15.88:0/1216333821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8ec0d1be0 0x7fb8ec0b4530 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec0d03c0 tx=0x7fb8d401ecb0 comp rx=0 tx=0).stop 2023-10-16T20:05:40.781 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.777+0000 7fb8f189e640 1 -- 172.21.15.88:0/1216333821 shutdown_connections 2023-10-16T20:05:40.781 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.777+0000 7fb8f189e640 1 --2- 172.21.15.88:0/1216333821 >> v2:172.21.15.94:3300/0 conn(0x7fb8ec0b4a70 0x7fb8ec0b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:40.781 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.777+0000 7fb8f189e640 1 --2- 172.21.15.88:0/1216333821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8ec0d1be0 0x7fb8ec0b4530 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:40.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.777+0000 7fb8f189e640 1 -- 172.21.15.88:0/1216333821 >> 172.21.15.88:0/1216333821 conn(0x7fb8ec06b6e0 msgr2=0x7fb8ec06baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:40.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.777+0000 7fb8f189e640 1 -- 172.21.15.88:0/1216333821 shutdown_connections 2023-10-16T20:05:40.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.777+0000 7fb8f189e640 1 -- 172.21.15.88:0/1216333821 wait complete. 2023-10-16T20:05:40.785 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.777+0000 7fb8f189e640 1 Processor -- start 2023-10-16T20:05:40.785 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.777+0000 7fb8f189e640 1 -- start start 2023-10-16T20:05:40.785 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.781+0000 7fb8f189e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb8ec0b4a70 0x7fb8ec0c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:40.785 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.781+0000 7fb8f189e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8ec0d1be0 0x7fb8ec0c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:40.786 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.781+0000 7fb8f189e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8ec15cb70 0x7fb8ec0b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:40.786 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.781+0000 7fb8f189e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec0cbb20 con 0x7fb8ec15cb70 2023-10-16T20:05:40.786 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.781+0000 7fb8f189e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec0cb560 con 0x7fb8ec0d1be0 2023-10-16T20:05:40.786 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.781+0000 7fb8f189e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec0cb840 con 0x7fb8ec0b4a70 2023-10-16T20:05:40.786 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.781+0000 7fb8eb7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8ec15cb70 0x7fb8ec0b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:40.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.781+0000 7fb8eb7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8ec15cb70 0x7fb8ec0b39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47282/0 (socket says 172.21.15.88:47282) 2023-10-16T20:05:40.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.781+0000 7fb8eb7fe640 1 -- 172.21.15.88:0/803680555 learned_addr learned my addr 172.21.15.88:0/803680555 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:40.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.781+0000 7fb8eaffd640 1 --2- 172.21.15.88:0/803680555 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb8ec0b4a70 0x7fb8ec0c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:40.788 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.781+0000 7fb8e27fc640 1 --2- 172.21.15.88:0/803680555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8ec0d1be0 0x7fb8ec0c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:40.788 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.781+0000 7fb8eb7fe640 1 -- 172.21.15.88:0/803680555 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb8ec0b4a70 msgr2=0x7fb8ec0c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:40.788 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.781+0000 7fb8eb7fe640 1 --2- 172.21.15.88:0/803680555 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb8ec0b4a70 0x7fb8ec0c6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:40.789 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.781+0000 7fb8eb7fe640 1 -- 172.21.15.88:0/803680555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8ec0d1be0 msgr2=0x7fb8ec0c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:40.789 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.781+0000 7fb8eb7fe640 1 --2- 172.21.15.88:0/803680555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8ec0d1be0 0x7fb8ec0c82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:40.789 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.781+0000 7fb8eb7fe640 1 -- 172.21.15.88:0/803680555 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d40094d0 con 0x7fb8ec15cb70 2023-10-16T20:05:40.790 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.781+0000 7fb8eb7fe640 1 --2- 172.21.15.88:0/803680555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8ec15cb70 0x7fb8ec0b39f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb8e400ce40 tx=0x7fb8e400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:40.790 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.781+0000 7fb8e8ff9640 1 -- 172.21.15.88:0/803680555 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e400d840 con 0x7fb8ec15cb70 2023-10-16T20:05:40.790 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.781+0000 7fb8e8ff9640 1 -- 172.21.15.88:0/803680555 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e400d9a0 con 0x7fb8ec15cb70 2023-10-16T20:05:40.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.781+0000 7fb8e8ff9640 1 -- 172.21.15.88:0/803680555 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e400dd50 con 0x7fb8ec15cb70 2023-10-16T20:05:40.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.781+0000 7fb8f189e640 1 -- 172.21.15.88:0/803680555 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8ec0c6760 con 0x7fb8ec15cb70 2023-10-16T20:05:40.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.781+0000 7fb8f189e640 1 -- 172.21.15.88:0/803680555 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8ec0c74a0 con 0x7fb8ec15cb70 2023-10-16T20:05:40.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.781+0000 7fb8f189e640 1 -- 172.21.15.88:0/803680555 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8ec0c65c0 con 0x7fb8ec15cb70 2023-10-16T20:05:40.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.781+0000 7fb8e8ff9640 1 -- 172.21.15.88:0/803680555 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb8e401e3b0 con 0x7fb8ec15cb70 2023-10-16T20:05:40.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.785+0000 7fb8e8ff9640 1 --2- 172.21.15.88:0/803680555 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb8cc0664a0 0x7fb8cc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:40.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.785+0000 7fb8e8ff9640 1 -- 172.21.15.88:0/803680555 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb8e4084c50 con 0x7fb8ec15cb70 2023-10-16T20:05:40.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.785+0000 7fb8eaffd640 1 --2- 172.21.15.88:0/803680555 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb8cc0664a0 0x7fb8cc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:40.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.785+0000 7fb8eaffd640 1 --2- 172.21.15.88:0/803680555 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb8cc0664a0 0x7fb8cc068960 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec0c9600 tx=0x7fb8dc009060 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:40.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.789+0000 7fb8e8ff9640 1 -- 172.21.15.88:0/803680555 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb8ec0c65c0 con 0x7fb8ec15cb70 2023-10-16T20:05:40.939 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.937+0000 7fb8f189e640 1 -- 172.21.15.88:0/803680555 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb8ec0c65c0 con 0x7fb8ec15cb70 2023-10-16T20:05:40.940 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.937+0000 7fb8e8ff9640 1 -- 172.21.15.88:0/803680555 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb8ec0c65c0 con 0x7fb8ec15cb70 2023-10-16T20:05:40.940 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:05:40.941 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.937+0000 7fb8f189e640 1 -- 172.21.15.88:0/803680555 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb8cc0664a0 msgr2=0x7fb8cc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:40.942 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.937+0000 7fb8f189e640 1 --2- 172.21.15.88:0/803680555 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb8cc0664a0 0x7fb8cc068960 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec0c9600 tx=0x7fb8dc009060 comp rx=0 tx=0).stop 2023-10-16T20:05:40.942 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.937+0000 7fb8f189e640 1 -- 172.21.15.88:0/803680555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8ec15cb70 msgr2=0x7fb8ec0b39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:40.942 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.937+0000 7fb8f189e640 1 --2- 172.21.15.88:0/803680555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8ec15cb70 0x7fb8ec0b39f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb8e400ce40 tx=0x7fb8e400a710 comp rx=0 tx=0).stop 2023-10-16T20:05:40.942 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.937+0000 7fb8f189e640 1 -- 172.21.15.88:0/803680555 shutdown_connections 2023-10-16T20:05:40.942 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.937+0000 7fb8f189e640 1 --2- 172.21.15.88:0/803680555 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb8cc0664a0 0x7fb8cc068960 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:40.943 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.937+0000 7fb8f189e640 1 --2- 172.21.15.88:0/803680555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8ec15cb70 0x7fb8ec0b39f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:40.943 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.937+0000 7fb8f189e640 1 --2- 172.21.15.88:0/803680555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8ec0d1be0 0x7fb8ec0c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:40.943 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.937+0000 7fb8f189e640 1 --2- 172.21.15.88:0/803680555 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb8ec0b4a70 0x7fb8ec0c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:40.943 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.937+0000 7fb8f189e640 1 -- 172.21.15.88:0/803680555 >> 172.21.15.88:0/803680555 conn(0x7fb8ec06b6e0 msgr2=0x7fb8ec157f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:40.943 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.941+0000 7fb8f189e640 1 -- 172.21.15.88:0/803680555 shutdown_connections 2023-10-16T20:05:40.944 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:40.941+0000 7fb8f189e640 1 -- 172.21.15.88:0/803680555 wait complete. 2023-10-16T20:05:40.960 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T20:05:41.100 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.097+0000 7f9af2f65640 1 Processor -- start 2023-10-16T20:05:41.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.097+0000 7f9af2f65640 1 -- start start 2023-10-16T20:05:41.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.101+0000 7f9af2f65640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f9aec15cae0 0x7f9aec0cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:41.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.101+0000 7f9af2f65640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9aec15cec0 0x7f9aec0cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:41.104 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.101+0000 7f9af2f65640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aec0cec10 con 0x7f9aec0b22b0 2023-10-16T20:05:41.104 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.101+0000 7f9af2f65640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9aec0cce00 con 0x7f9aec15cae0 2023-10-16T20:05:41.104 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.101+0000 7f9af2f65640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aec1611b0 con 0x7f9aec15cec0 2023-10-16T20:05:41.104 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.101+0000 7f9af0d01640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f9aec15cae0 0x7f9aec0cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:41.104 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.101+0000 7f9af0d01640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f9aec15cae0 0x7f9aec0cf0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47286/0 (socket says 172.21.15.88:47286) 2023-10-16T20:05:41.105 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.101+0000 7f9af1502640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f9aec0b22b0 0x7f9aec06ff00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:54776/0 (socket says 172.21.15.88:54776) 2023-10-16T20:05:41.105 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.101+0000 7f9af0d01640 1 -- 172.21.15.88:0/532822934 learned_addr learned my addr 172.21.15.88:0/532822934 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:41.105 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.101+0000 7f9ae3fff640 1 --2- 172.21.15.88:0/532822934 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9aec15cec0 0x7f9aec0cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:41.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.101+0000 7f9af0d01640 1 -- 172.21.15.88:0/532822934 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9aec15cec0 msgr2=0x7f9aec0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:41.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.101+0000 7f9af0d01640 1 --2- 172.21.15.88:0/532822934 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9aec15cec0 0x7f9aec0cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:41.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.101+0000 7f9af0d01640 1 -- 172.21.15.88:0/532822934 >> v1:172.21.15.88:6789/0 conn(0x7f9aec0b22b0 legacy=0x7f9aec06ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:41.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.101+0000 7f9af0d01640 1 -- 172.21.15.88:0/532822934 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aec161490 con 0x7f9aec15cae0 2023-10-16T20:05:41.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.101+0000 7f9ae37fe640 1 -- 172.21.15.88:0/532822934 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2801581934 0 0) 0x7f9aec0cec10 con 0x7f9aec0b22b0 2023-10-16T20:05:41.108 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.101+0000 7f9af0d01640 1 --2- 172.21.15.88:0/532822934 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9aec15cae0 0x7f9aec0cf0a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8009870 tx=0x7f9ae801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=674b78317a55d24 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:41.109 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.101+0000 7f9ae37fe640 1 -- 172.21.15.88:0/532822934 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae80a9620 con 0x7f9aec15cae0 2023-10-16T20:05:41.109 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.101+0000 7f9ae37fe640 1 -- 172.21.15.88:0/532822934 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae80a9780 con 0x7f9aec15cae0 2023-10-16T20:05:41.109 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.105+0000 7f9ae37fe640 1 -- 172.21.15.88:0/532822934 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae80a9b10 con 0x7f9aec15cae0 2023-10-16T20:05:41.109 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.105+0000 7f9af2f65640 1 -- 172.21.15.88:0/532822934 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9aec15cae0 msgr2=0x7f9aec0cf0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:41.110 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.105+0000 7f9af2f65640 1 --2- 172.21.15.88:0/532822934 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9aec15cae0 0x7f9aec0cf0a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8009870 tx=0x7f9ae801c540 comp rx=0 tx=0).stop 2023-10-16T20:05:41.110 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.105+0000 7f9af2f65640 1 -- 172.21.15.88:0/532822934 shutdown_connections 2023-10-16T20:05:41.110 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.105+0000 7f9af2f65640 1 --2- 172.21.15.88:0/532822934 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9aec15cec0 0x7f9aec0cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:41.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.105+0000 7f9af2f65640 1 --2- 172.21.15.88:0/532822934 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9aec15cae0 0x7f9aec0cf0a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:41.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.105+0000 7f9af2f65640 1 -- 172.21.15.88:0/532822934 >> 172.21.15.88:0/532822934 conn(0x7f9aec0cd2b0 msgr2=0x7f9aec0cd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:41.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.105+0000 7f9af2f65640 1 -- 172.21.15.88:0/532822934 shutdown_connections 2023-10-16T20:05:41.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.105+0000 7f9af2f65640 1 -- 172.21.15.88:0/532822934 wait complete. 2023-10-16T20:05:41.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.105+0000 7f9af2f65640 1 Processor -- start 2023-10-16T20:05:41.112 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.105+0000 7f9af2f65640 1 -- start start 2023-10-16T20:05:41.115 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.105+0000 7f9af2f65640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9aec0b22b0 0x7f9aec15f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:41.115 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.105+0000 7f9af2f65640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9aec15cae0 0x7f9aec1534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:41.115 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.105+0000 7f9af2f65640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9aec15cec0 0x7f9aec1539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:41.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.105+0000 7f9af2f65640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aec1563c0 con 0x7f9aec15cae0 2023-10-16T20:05:41.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.105+0000 7f9af2f65640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aec1557b0 con 0x7f9aec0b22b0 2023-10-16T20:05:41.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.105+0000 7f9af2f65640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aec155a90 con 0x7f9aec15cec0 2023-10-16T20:05:41.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.105+0000 7f9ae3fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9aec15cae0 0x7f9aec1534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:41.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.105+0000 7f9ae3fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9aec15cae0 0x7f9aec1534b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47288/0 (socket says 172.21.15.88:47288) 2023-10-16T20:05:41.117 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.105+0000 7f9ae3fff640 1 -- 172.21.15.88:0/1406442053 learned_addr learned my addr 172.21.15.88:0/1406442053 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:41.117 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.109+0000 7f9af0d01640 1 --2- 172.21.15.88:0/1406442053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9aec0b22b0 0x7f9aec15f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:41.117 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.109+0000 7f9af1502640 1 --2- 172.21.15.88:0/1406442053 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9aec15cec0 0x7f9aec1539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:41.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.109+0000 7f9ae3fff640 1 -- 172.21.15.88:0/1406442053 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9aec15cec0 msgr2=0x7f9aec1539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:41.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.109+0000 7f9ae3fff640 1 --2- 172.21.15.88:0/1406442053 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9aec15cec0 0x7f9aec1539f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:41.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.109+0000 7f9ae3fff640 1 -- 172.21.15.88:0/1406442053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9aec0b22b0 msgr2=0x7f9aec15f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:41.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.109+0000 7f9ae3fff640 1 --2- 172.21.15.88:0/1406442053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9aec0b22b0 0x7f9aec15f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:41.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.109+0000 7f9ae3fff640 1 -- 172.21.15.88:0/1406442053 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae80094d0 con 0x7f9aec15cae0 2023-10-16T20:05:41.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.109+0000 7f9af1502640 1 --2- 172.21.15.88:0/1406442053 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9aec15cec0 0x7f9aec1539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:41.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.109+0000 7f9ae3fff640 1 --2- 172.21.15.88:0/1406442053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9aec15cae0 0x7f9aec1534b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00d5f0 tx=0x7f9adc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:41.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.109+0000 7f9ae17fa640 1 -- 172.21.15.88:0/1406442053 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc0125f0 con 0x7f9aec15cae0 2023-10-16T20:05:41.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.109+0000 7f9ae17fa640 1 -- 172.21.15.88:0/1406442053 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9adc012750 con 0x7f9aec15cae0 2023-10-16T20:05:41.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.109+0000 7f9ae17fa640 1 -- 172.21.15.88:0/1406442053 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc012ae0 con 0x7f9aec15cae0 2023-10-16T20:05:41.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.109+0000 7f9af2f65640 1 -- 172.21.15.88:0/1406442053 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aec155d70 con 0x7f9aec15cae0 2023-10-16T20:05:41.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.109+0000 7f9af2f65640 1 -- 172.21.15.88:0/1406442053 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aec15fcd0 con 0x7f9aec15cae0 2023-10-16T20:05:41.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.109+0000 7f9af2f65640 1 -- 172.21.15.88:0/1406442053 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aec156570 con 0x7f9aec15cae0 2023-10-16T20:05:41.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.109+0000 7f9ae17fa640 1 -- 172.21.15.88:0/1406442053 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9adc012c40 con 0x7f9aec15cae0 2023-10-16T20:05:41.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.109+0000 7f9ae17fa640 1 --2- 172.21.15.88:0/1406442053 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9ad80664a0 0x7f9ad8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:41.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.109+0000 7f9ae17fa640 1 -- 172.21.15.88:0/1406442053 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9adc0846d0 con 0x7f9aec15cae0 2023-10-16T20:05:41.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.109+0000 7f9af0d01640 1 --2- 172.21.15.88:0/1406442053 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9ad80664a0 0x7f9ad8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:41.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.113+0000 7f9af0d01640 1 --2- 172.21.15.88:0/1406442053 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9ad80664a0 0x7f9ad8068960 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80ae000 tx=0x7f9ae80b3040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:41.122 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.113+0000 7f9ae17fa640 1 -- 172.21.15.88:0/1406442053 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9aec156570 con 0x7f9aec15cae0 2023-10-16T20:05:41.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.265+0000 7f9af2f65640 1 -- 172.21.15.88:0/1406442053 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9aec04e9c0 con 0x7f9aec15cae0 2023-10-16T20:05:41.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.265+0000 7f9ae17fa640 1 -- 172.21.15.88:0/1406442053 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9adc0516f0 con 0x7f9aec15cae0 2023-10-16T20:05:41.268 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2023-10-16T20:05:41.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.265+0000 7f9af2f65640 1 -- 172.21.15.88:0/1406442053 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9ad80664a0 msgr2=0x7f9ad8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:41.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.265+0000 7f9af2f65640 1 --2- 172.21.15.88:0/1406442053 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9ad80664a0 0x7f9ad8068960 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80ae000 tx=0x7f9ae80b3040 comp rx=0 tx=0).stop 2023-10-16T20:05:41.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.265+0000 7f9af2f65640 1 -- 172.21.15.88:0/1406442053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9aec15cae0 msgr2=0x7f9aec1534b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:41.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.265+0000 7f9af2f65640 1 --2- 172.21.15.88:0/1406442053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9aec15cae0 0x7f9aec1534b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00d5f0 tx=0x7f9adc00dab0 comp rx=0 tx=0).stop 2023-10-16T20:05:41.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.265+0000 7f9af2f65640 1 -- 172.21.15.88:0/1406442053 shutdown_connections 2023-10-16T20:05:41.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.265+0000 7f9af2f65640 1 --2- 172.21.15.88:0/1406442053 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9ad80664a0 0x7f9ad8068960 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:41.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.265+0000 7f9af2f65640 1 --2- 172.21.15.88:0/1406442053 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9aec15cec0 0x7f9aec1539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:41.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.265+0000 7f9af2f65640 1 --2- 172.21.15.88:0/1406442053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9aec15cae0 0x7f9aec1534b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:41.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.265+0000 7f9af2f65640 1 --2- 172.21.15.88:0/1406442053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9aec0b22b0 0x7f9aec15f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:41.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.265+0000 7f9af2f65640 1 -- 172.21.15.88:0/1406442053 >> 172.21.15.88:0/1406442053 conn(0x7f9aec0cd2b0 msgr2=0x7f9aec06bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:41.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.269+0000 7f9af2f65640 1 -- 172.21.15.88:0/1406442053 shutdown_connections 2023-10-16T20:05:41.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.269+0000 7f9af2f65640 1 -- 172.21.15.88:0/1406442053 wait complete. 2023-10-16T20:05:41.289 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T20:05:41.431 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.429+0000 7f75ba9d0640 1 Processor -- start 2023-10-16T20:05:41.431 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.429+0000 7f75ba9d0640 1 -- start start 2023-10-16T20:05:41.433 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.429+0000 7f75ba9d0640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f75b415cb70 0x7f75b40b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:41.433 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.429+0000 7f75ba9d0640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f75b40b4530 0x7f75b40b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:41.433 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.429+0000 7f75ba9d0640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b40b4ed0 con 0x7f75b40d0bd0 2023-10-16T20:05:41.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.429+0000 7f75ba9d0640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b40b7d20 con 0x7f75b40b4530 2023-10-16T20:05:41.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.429+0000 7f75ba9d0640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b40b7110 con 0x7f75b415cb70 2023-10-16T20:05:41.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.429+0000 7f75b37fe640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f75b40b4530 0x7f75b40b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:41.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.429+0000 7f75b37fe640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f75b40b4530 0x7f75b40b4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47302/0 (socket says 172.21.15.88:47302) 2023-10-16T20:05:41.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.429+0000 7f75b37fe640 1 -- 172.21.15.88:0/4184683017 learned_addr learned my addr 172.21.15.88:0/4184683017 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:41.435 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.429+0000 7f75b3fff640 1 --2- 172.21.15.88:0/4184683017 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f75b415cb70 0x7f75b40b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:41.435 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.429+0000 7f75b37fe640 1 -- 172.21.15.88:0/4184683017 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f75b415cb70 msgr2=0x7f75b40b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:41.435 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.429+0000 7f75b37fe640 1 --2- 172.21.15.88:0/4184683017 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f75b415cb70 0x7f75b40b83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:41.435 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.429+0000 7f75b37fe640 1 -- 172.21.15.88:0/4184683017 >> v1:172.21.15.94:6789/0 conn(0x7f75b40d0bd0 legacy=0x7f75b40d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:41.435 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.429+0000 7f75b37fe640 1 -- 172.21.15.88:0/4184683017 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b40b73f0 con 0x7f75b40b4530 2023-10-16T20:05:41.437 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.433+0000 7f75b37fe640 1 --2- 172.21.15.88:0/4184683017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75b40b4530 0x7f75b40b4990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009870 tx=0x7f75a801ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc4b64d5a26a9a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:41.437 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.433+0000 7f75b2ffd640 1 -- 172.21.15.88:0/4184683017 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a80a8a30 con 0x7f75b40b4530 2023-10-16T20:05:41.437 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.433+0000 7f75b2ffd640 1 -- 172.21.15.88:0/4184683017 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a80a8b90 con 0x7f75b40b4530 2023-10-16T20:05:41.437 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.433+0000 7f75b2ffd640 1 -- 172.21.15.88:0/4184683017 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a80b0440 con 0x7f75b40b4530 2023-10-16T20:05:41.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.433+0000 7f75ba9d0640 1 -- 172.21.15.88:0/4184683017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75b40b4530 msgr2=0x7f75b40b4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:41.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.433+0000 7f75ba9d0640 1 --2- 172.21.15.88:0/4184683017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75b40b4530 0x7f75b40b4990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009870 tx=0x7f75a801ed30 comp rx=0 tx=0).stop 2023-10-16T20:05:41.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.433+0000 7f75ba9d0640 1 -- 172.21.15.88:0/4184683017 shutdown_connections 2023-10-16T20:05:41.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.433+0000 7f75ba9d0640 1 --2- 172.21.15.88:0/4184683017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75b40b4530 0x7f75b40b4990 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:41.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.433+0000 7f75ba9d0640 1 --2- 172.21.15.88:0/4184683017 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f75b415cb70 0x7f75b40b83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:41.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.433+0000 7f75ba9d0640 1 -- 172.21.15.88:0/4184683017 >> 172.21.15.88:0/4184683017 conn(0x7f75b406b6e0 msgr2=0x7f75b406baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:41.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.433+0000 7f75ba9d0640 1 -- 172.21.15.88:0/4184683017 shutdown_connections 2023-10-16T20:05:41.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.433+0000 7f75ba9d0640 1 -- 172.21.15.88:0/4184683017 wait complete. 2023-10-16T20:05:41.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.437+0000 7f75ba9d0640 1 Processor -- start 2023-10-16T20:05:41.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.437+0000 7f75ba9d0640 1 -- start start 2023-10-16T20:05:41.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.437+0000 7f75ba9d0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75b40b4530 0x7f75b41550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:41.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.437+0000 7f75ba9d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75b40d0bd0 0x7f75b415f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:41.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.437+0000 7f75ba9d0640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f75b415cb70 0x7f75b415f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:41.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.437+0000 7f75ba9d0640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b415dee0 con 0x7f75b40b4530 2023-10-16T20:05:41.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.437+0000 7f75ba9d0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b41f6ab0 con 0x7f75b40d0bd0 2023-10-16T20:05:41.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.437+0000 7f75ba9d0640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b41f67d0 con 0x7f75b415cb70 2023-10-16T20:05:41.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.437+0000 7f75b3fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75b40b4530 0x7f75b41550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:41.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.437+0000 7f75b3fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75b40b4530 0x7f75b41550d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47314/0 (socket says 172.21.15.88:47314) 2023-10-16T20:05:41.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.437+0000 7f75b3fff640 1 -- 172.21.15.88:0/1467463423 learned_addr learned my addr 172.21.15.88:0/1467463423 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:41.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.437+0000 7f75b37fe640 1 --2- 172.21.15.88:0/1467463423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75b40d0bd0 0x7f75b415f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:41.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.437+0000 7f75b8f6d640 1 --2- 172.21.15.88:0/1467463423 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f75b415cb70 0x7f75b415f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:41.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.437+0000 7f75b3fff640 1 -- 172.21.15.88:0/1467463423 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f75b415cb70 msgr2=0x7f75b415f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:41.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.437+0000 7f75b3fff640 1 --2- 172.21.15.88:0/1467463423 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f75b415cb70 0x7f75b415f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:41.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.437+0000 7f75b3fff640 1 -- 172.21.15.88:0/1467463423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75b40d0bd0 msgr2=0x7f75b415f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:41.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.437+0000 7f75b3fff640 1 --2- 172.21.15.88:0/1467463423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75b40d0bd0 0x7f75b415f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:41.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.437+0000 7f75b3fff640 1 -- 172.21.15.88:0/1467463423 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a80094d0 con 0x7f75b40b4530 2023-10-16T20:05:41.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.437+0000 7f75b8f6d640 1 --2- 172.21.15.88:0/1467463423 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f75b415cb70 0x7f75b415f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:41.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.437+0000 7f75b3fff640 1 --2- 172.21.15.88:0/1467463423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75b40b4530 0x7f75b41550d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f75a400d5f0 tx=0x7f75a400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:41.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.437+0000 7f75b0ff9640 1 -- 172.21.15.88:0/1467463423 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a40125f0 con 0x7f75b40b4530 2023-10-16T20:05:41.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.437+0000 7f75b0ff9640 1 -- 172.21.15.88:0/1467463423 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a4012750 con 0x7f75b40b4530 2023-10-16T20:05:41.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.437+0000 7f75b0ff9640 1 -- 172.21.15.88:0/1467463423 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a4012b00 con 0x7f75b40b4530 2023-10-16T20:05:41.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.437+0000 7f75ba9d0640 1 -- 172.21.15.88:0/1467463423 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b415e620 con 0x7f75b40b4530 2023-10-16T20:05:41.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.437+0000 7f75ba9d0640 1 -- 172.21.15.88:0/1467463423 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b415ebe0 con 0x7f75b40b4530 2023-10-16T20:05:41.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.441+0000 7f75ba9d0640 1 -- 172.21.15.88:0/1467463423 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b404e9c0 con 0x7f75b40b4530 2023-10-16T20:05:41.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.441+0000 7f75b0ff9640 1 -- 172.21.15.88:0/1467463423 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f75a4012c60 con 0x7f75b40b4530 2023-10-16T20:05:41.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.441+0000 7f75b0ff9640 1 --2- 172.21.15.88:0/1467463423 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f75a00664a0 0x7f75a0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:41.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.441+0000 7f75b0ff9640 1 -- 172.21.15.88:0/1467463423 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f75a4084700 con 0x7f75b40b4530 2023-10-16T20:05:41.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.441+0000 7f75b37fe640 1 --2- 172.21.15.88:0/1467463423 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f75a00664a0 0x7f75a0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:41.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.441+0000 7f75b37fe640 1 --2- 172.21.15.88:0/1467463423 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f75a00664a0 0x7f75a0068960 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f75a80b2000 tx=0x7f75a8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:41.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.445+0000 7f75b0ff9640 1 -- 172.21.15.88:0/1467463423 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f75a40517b0 con 0x7f75b40b4530 2023-10-16T20:05:41.602 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.601+0000 7f75ba9d0640 1 -- 172.21.15.88:0/1467463423 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f75b415edd0 con 0x7f75b40b4530 2023-10-16T20:05:41.603 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.601+0000 7f75b0ff9640 1 -- 172.21.15.88:0/1467463423 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f75a4054dd0 con 0x7f75b40b4530 2023-10-16T20:05:41.603 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:05:41.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.601+0000 7f75ba9d0640 1 -- 172.21.15.88:0/1467463423 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f75a00664a0 msgr2=0x7f75a0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:41.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.601+0000 7f75ba9d0640 1 --2- 172.21.15.88:0/1467463423 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f75a00664a0 0x7f75a0068960 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f75a80b2000 tx=0x7f75a8009040 comp rx=0 tx=0).stop 2023-10-16T20:05:41.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.601+0000 7f75ba9d0640 1 -- 172.21.15.88:0/1467463423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75b40b4530 msgr2=0x7f75b41550d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:41.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.601+0000 7f75ba9d0640 1 --2- 172.21.15.88:0/1467463423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75b40b4530 0x7f75b41550d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f75a400d5f0 tx=0x7f75a400dab0 comp rx=0 tx=0).stop 2023-10-16T20:05:41.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.601+0000 7f75ba9d0640 1 -- 172.21.15.88:0/1467463423 shutdown_connections 2023-10-16T20:05:41.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.601+0000 7f75ba9d0640 1 --2- 172.21.15.88:0/1467463423 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f75a00664a0 0x7f75a0068960 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:41.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.601+0000 7f75ba9d0640 1 --2- 172.21.15.88:0/1467463423 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f75b415cb70 0x7f75b415f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:41.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.601+0000 7f75ba9d0640 1 --2- 172.21.15.88:0/1467463423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75b40d0bd0 0x7f75b415f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:41.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.601+0000 7f75ba9d0640 1 --2- 172.21.15.88:0/1467463423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75b40b4530 0x7f75b41550d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:41.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.601+0000 7f75ba9d0640 1 -- 172.21.15.88:0/1467463423 >> 172.21.15.88:0/1467463423 conn(0x7f75b406b6e0 msgr2=0x7f75b40d2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:41.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.601+0000 7f75ba9d0640 1 -- 172.21.15.88:0/1467463423 shutdown_connections 2023-10-16T20:05:41.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.601+0000 7f75ba9d0640 1 -- 172.21.15.88:0/1467463423 wait complete. 2023-10-16T20:05:41.650 INFO:tasks.cephfs_test_runner:Starting test: test_forbidden_modification (tasks.cephfs.test_pool_perm.TestPoolPerm) That a client who does not have the capability for setting 2023-10-16T20:05:41.651 INFO:tasks.cephfs_test_runner:test_forbidden_modification (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-10-16T20:05:41.651 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:05:41.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.789+0000 7f14f89b8640 1 Processor -- start 2023-10-16T20:05:41.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.789+0000 7f14f89b8640 1 -- start start 2023-10-16T20:05:41.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.789+0000 7f14f89b8640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14f40d22e0 0x7f14f4161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:41.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.789+0000 7f14f89b8640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f14f40d26c0 0x7f14f415d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:41.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.789+0000 7f14f89b8640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14f40b87e0 con 0x7f14f415d820 2023-10-16T20:05:41.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.789+0000 7f14f89b8640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14f415dc40 con 0x7f14f40d26c0 2023-10-16T20:05:41.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.789+0000 7f14f89b8640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f415df10 con 0x7f14f40d22e0 2023-10-16T20:05:41.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.789+0000 7f14e9d9b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f14f40d26c0 0x7f14f415d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:41.795 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.789+0000 7f14f259c640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14f40d22e0 0x7f14f4161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:41.795 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.789+0000 7f14f2d9d640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f14f415d820 0x7f14f40cf2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:59306/0 (socket says 172.21.15.88:59306) 2023-10-16T20:05:41.795 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.789+0000 7f14f2d9d640 1 -- 172.21.15.88:0/4051694921 learned_addr learned my addr 172.21.15.88:0/4051694921 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:41.795 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.789+0000 7f14f1d9b640 1 -- 172.21.15.88:0/4051694921 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4111205135 0 0) 0x7f14f40b87e0 con 0x7f14f415d820 2023-10-16T20:05:41.795 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.789+0000 7f14f1d9b640 1 -- 172.21.15.88:0/4051694921 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14e0003530 con 0x7f14f415d820 2023-10-16T20:05:41.795 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.789+0000 7f14f259c640 1 -- 172.21.15.88:0/4051694921 >> v2:172.21.15.94:3300/0 conn(0x7f14f40d26c0 msgr2=0x7f14f415d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:41.796 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.789+0000 7f14f259c640 1 --2- 172.21.15.88:0/4051694921 >> v2:172.21.15.94:3300/0 conn(0x7f14f40d26c0 0x7f14f415d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:41.796 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.789+0000 7f14f259c640 1 -- 172.21.15.88:0/4051694921 >> v1:172.21.15.94:6789/0 conn(0x7f14f415d820 legacy=0x7f14f40cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:41.796 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.789+0000 7f14f259c640 1 -- 172.21.15.88:0/4051694921 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f415e1f0 con 0x7f14f40d22e0 2023-10-16T20:05:41.796 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.789+0000 7f14e9d9b640 1 --2- 172.21.15.88:0/4051694921 >> v2:172.21.15.94:3300/0 conn(0x7f14f40d26c0 0x7f14f415d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:05:41.797 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.793+0000 7f14f259c640 1 --2- 172.21.15.88:0/4051694921 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14f40d22e0 0x7f14f4161450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f14e40097c0 tx=0x7f14e401c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c2fc89d1978f498 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:41.798 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.793+0000 7f14f1d9b640 1 -- 172.21.15.88:0/4051694921 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e40ab600 con 0x7f14f40d22e0 2023-10-16T20:05:41.798 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.793+0000 7f14f1d9b640 1 -- 172.21.15.88:0/4051694921 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e40a7050 con 0x7f14f40d22e0 2023-10-16T20:05:41.798 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.793+0000 7f14f1d9b640 1 -- 172.21.15.88:0/4051694921 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e40ab980 con 0x7f14f40d22e0 2023-10-16T20:05:41.798 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.793+0000 7f14f89b8640 1 -- 172.21.15.88:0/4051694921 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14f40d22e0 msgr2=0x7f14f4161450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:41.798 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.793+0000 7f14f89b8640 1 --2- 172.21.15.88:0/4051694921 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14f40d22e0 0x7f14f4161450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f14e40097c0 tx=0x7f14e401c3c0 comp rx=0 tx=0).stop 2023-10-16T20:05:41.799 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.797+0000 7f14f89b8640 1 -- 172.21.15.88:0/4051694921 shutdown_connections 2023-10-16T20:05:41.799 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.797+0000 7f14f89b8640 1 --2- 172.21.15.88:0/4051694921 >> v2:172.21.15.94:3300/0 conn(0x7f14f40d26c0 0x7f14f415d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:41.799 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.797+0000 7f14f89b8640 1 --2- 172.21.15.88:0/4051694921 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14f40d22e0 0x7f14f4161450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:41.799 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.797+0000 7f14f89b8640 1 -- 172.21.15.88:0/4051694921 >> 172.21.15.88:0/4051694921 conn(0x7f14f40715c0 msgr2=0x7f14f40719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:41.799 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.797+0000 7f14f89b8640 1 -- 172.21.15.88:0/4051694921 shutdown_connections 2023-10-16T20:05:41.800 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.797+0000 7f14f89b8640 1 -- 172.21.15.88:0/4051694921 wait complete. 2023-10-16T20:05:41.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.797+0000 7f14f89b8640 1 Processor -- start 2023-10-16T20:05:41.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.797+0000 7f14f89b8640 1 -- start start 2023-10-16T20:05:41.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.797+0000 7f14f89b8640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14f40d22e0 0x7f14f414a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:41.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.797+0000 7f14f89b8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14f40d26c0 0x7f14f40b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:41.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.797+0000 7f14f89b8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14f415d820 0x7f14f40b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:41.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.797+0000 7f14f89b8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f40c9f50 con 0x7f14f415d820 2023-10-16T20:05:41.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.797+0000 7f14f89b8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f4152500 con 0x7f14f40d26c0 2023-10-16T20:05:41.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.797+0000 7f14f89b8640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f41527e0 con 0x7f14f40d22e0 2023-10-16T20:05:41.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.797+0000 7f14f2d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14f415d820 0x7f14f40b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:41.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.797+0000 7f14f2d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14f415d820 0x7f14f40b61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47324/0 (socket says 172.21.15.88:47324) 2023-10-16T20:05:41.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.797+0000 7f14e9d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14f40d26c0 0x7f14f40b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:41.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.797+0000 7f14f2d9d640 1 -- 172.21.15.88:0/4190532452 learned_addr learned my addr 172.21.15.88:0/4190532452 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:41.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.797+0000 7f14f259c640 1 --2- 172.21.15.88:0/4190532452 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14f40d22e0 0x7f14f414a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:41.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.797+0000 7f14f259c640 1 -- 172.21.15.88:0/4190532452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14f40d26c0 msgr2=0x7f14f40b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:41.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.797+0000 7f14f259c640 1 --2- 172.21.15.88:0/4190532452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14f40d26c0 0x7f14f40b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:41.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.801+0000 7f14f259c640 1 -- 172.21.15.88:0/4190532452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14f415d820 msgr2=0x7f14f40b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:41.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.801+0000 7f14f259c640 1 --2- 172.21.15.88:0/4190532452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14f415d820 0x7f14f40b61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:41.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.801+0000 7f14f259c640 1 -- 172.21.15.88:0/4190532452 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14dc009a70 con 0x7f14f40d22e0 2023-10-16T20:05:41.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.801+0000 7f14f2d9d640 1 --2- 172.21.15.88:0/4190532452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14f415d820 0x7f14f40b61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:05:41.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.801+0000 7f14f259c640 1 --2- 172.21.15.88:0/4190532452 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14f40d22e0 0x7f14f414a130 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f14f415e050 tx=0x7f14e401ac00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:41.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.801+0000 7f14eb7fe640 1 -- 172.21.15.88:0/4190532452 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e40b7450 con 0x7f14f40d22e0 2023-10-16T20:05:41.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.801+0000 7f14eb7fe640 1 -- 172.21.15.88:0/4190532452 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e40b75b0 con 0x7f14f40d22e0 2023-10-16T20:05:41.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.801+0000 7f14f89b8640 1 -- 172.21.15.88:0/4190532452 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14e4009420 con 0x7f14f40d22e0 2023-10-16T20:05:41.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.801+0000 7f14eb7fe640 1 -- 172.21.15.88:0/4190532452 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e40b7960 con 0x7f14f40d22e0 2023-10-16T20:05:41.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.801+0000 7f14f89b8640 1 -- 172.21.15.88:0/4190532452 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14f4152da0 con 0x7f14f40d22e0 2023-10-16T20:05:41.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.801+0000 7f14f89b8640 1 -- 172.21.15.88:0/4190532452 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14f40ca090 con 0x7f14f40d22e0 2023-10-16T20:05:41.808 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.801+0000 7f14eb7fe640 1 -- 172.21.15.88:0/4190532452 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f14e40b7ac0 con 0x7f14f40d22e0 2023-10-16T20:05:41.808 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.801+0000 7f14eb7fe640 1 --2- 172.21.15.88:0/4190532452 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f14d40664a0 0x7f14d4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:41.808 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.801+0000 7f14eb7fe640 1 -- 172.21.15.88:0/4190532452 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f14e411f240 con 0x7f14f40d22e0 2023-10-16T20:05:41.809 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.805+0000 7f14e9d9b640 1 --2- 172.21.15.88:0/4190532452 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f14d40664a0 0x7f14d4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:41.809 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.805+0000 7f14e9d9b640 1 --2- 172.21.15.88:0/4190532452 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f14d40664a0 0x7f14d4068960 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009fd0 tx=0x7f14dc00b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:41.809 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.805+0000 7f14eb7fe640 1 -- 172.21.15.88:0/4190532452 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f14e40ec390 con 0x7f14f40d22e0 2023-10-16T20:05:41.960 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.957+0000 7f14f89b8640 1 -- 172.21.15.88:0/4190532452 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14f4152920 con 0x7f14f40d22e0 2023-10-16T20:05:41.962 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.961+0000 7f14eb7fe640 1 -- 172.21.15.88:0/4190532452 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18661 (secure 0 0 0) 0x7f14f4152920 con 0x7f14f40d22e0 2023-10-16T20:05:41.963 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:41.963 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":29,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:05:38.202959+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T20:04:54.442487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T20:04:55.766641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T20:04:59.475830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":1}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:05:41.965 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.961+0000 7f14f89b8640 1 -- 172.21.15.88:0/4190532452 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f14d40664a0 msgr2=0x7f14d4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:41.965 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.961+0000 7f14f89b8640 1 --2- 172.21.15.88:0/4190532452 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f14d40664a0 0x7f14d4068960 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009fd0 tx=0x7f14dc00b040 comp rx=0 tx=0).stop 2023-10-16T20:05:41.965 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.961+0000 7f14f89b8640 1 -- 172.21.15.88:0/4190532452 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14f40d22e0 msgr2=0x7f14f414a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:41.965 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.961+0000 7f14f89b8640 1 --2- 172.21.15.88:0/4190532452 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14f40d22e0 0x7f14f414a130 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f14f415e050 tx=0x7f14e401ac00 comp rx=0 tx=0).stop 2023-10-16T20:05:41.965 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.961+0000 7f14f89b8640 1 -- 172.21.15.88:0/4190532452 shutdown_connections 2023-10-16T20:05:41.966 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.961+0000 7f14f89b8640 1 --2- 172.21.15.88:0/4190532452 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f14d40664a0 0x7f14d4068960 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:41.966 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.961+0000 7f14f89b8640 1 --2- 172.21.15.88:0/4190532452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14f415d820 0x7f14f40b61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:41.966 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.961+0000 7f14f89b8640 1 --2- 172.21.15.88:0/4190532452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14f40d26c0 0x7f14f40b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:41.966 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.961+0000 7f14f89b8640 1 --2- 172.21.15.88:0/4190532452 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14f40d22e0 0x7f14f414a130 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:41.967 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.961+0000 7f14f89b8640 1 -- 172.21.15.88:0/4190532452 >> 172.21.15.88:0/4190532452 conn(0x7f14f40715c0 msgr2=0x7f14f40cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:41.967 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.961+0000 7f14f89b8640 1 -- 172.21.15.88:0/4190532452 shutdown_connections 2023-10-16T20:05:41.967 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:41.961+0000 7f14f89b8640 1 -- 172.21.15.88:0/4190532452 wait complete. 2023-10-16T20:05:41.984 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T20:04:54.442487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T20:04:55.766641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T20:04:59.475830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T20:05:41.985 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:05:42.125 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.121+0000 7f784b71d640 1 Processor -- start 2023-10-16T20:05:42.125 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.121+0000 7f784b71d640 1 -- start start 2023-10-16T20:05:42.127 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.125+0000 7f784b71d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f784406a0f0 0x7f7844144260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:42.127 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.125+0000 7f784b71d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f78440c9be0 0x7f784406a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:42.127 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.125+0000 7f784b71d640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f784406b010 con 0x7f784406a4d0 2023-10-16T20:05:42.127 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.125+0000 7f784b71d640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f784406b2f0 con 0x7f784406a0f0 2023-10-16T20:05:42.127 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.125+0000 7f784b71d640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f784406b5d0 con 0x7f78440c9be0 2023-10-16T20:05:42.128 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.125+0000 7f78494b9640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f784406a0f0 0x7f7844144260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:42.128 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.125+0000 7f7848cb8640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f784406a4d0 0x7f78440c94d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:34624/0 (socket says 172.21.15.88:34624) 2023-10-16T20:05:42.128 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.125+0000 7f7848cb8640 1 -- 172.21.15.88:0/830990590 learned_addr learned my addr 172.21.15.88:0/830990590 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:42.129 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.125+0000 7f78494b9640 1 -- 172.21.15.88:0/830990590 >> v2:172.21.15.94:3300/0 conn(0x7f78440c9be0 msgr2=0x7f784406a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:42.129 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.125+0000 7f78494b9640 1 --2- 172.21.15.88:0/830990590 >> v2:172.21.15.94:3300/0 conn(0x7f78440c9be0 0x7f784406a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:42.129 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.125+0000 7f78494b9640 1 -- 172.21.15.88:0/830990590 >> v1:172.21.15.88:6789/0 conn(0x7f784406a4d0 legacy=0x7f78440c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:42.129 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.125+0000 7f78494b9640 1 -- 172.21.15.88:0/830990590 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78440ccbd0 con 0x7f784406a0f0 2023-10-16T20:05:42.130 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.125+0000 7f783bfff640 1 -- 172.21.15.88:0/830990590 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1361029869 0 0) 0x7f784406b010 con 0x7f784406a4d0 2023-10-16T20:05:42.130 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.129+0000 7f78494b9640 1 --2- 172.21.15.88:0/830990590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f784406a0f0 0x7f7844144260 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f78440b1a50 tx=0x7f784001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a155dbf3926d6bc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:42.131 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.129+0000 7f783bfff640 1 -- 172.21.15.88:0/830990590 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78400ab680 con 0x7f784406a0f0 2023-10-16T20:05:42.131 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.129+0000 7f783bfff640 1 -- 172.21.15.88:0/830990590 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78400ab7e0 con 0x7f784406a0f0 2023-10-16T20:05:42.131 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.129+0000 7f783bfff640 1 -- 172.21.15.88:0/830990590 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78400abb70 con 0x7f784406a0f0 2023-10-16T20:05:42.132 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.129+0000 7f784b71d640 1 -- 172.21.15.88:0/830990590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f784406a0f0 msgr2=0x7f7844144260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:42.132 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.129+0000 7f784b71d640 1 --2- 172.21.15.88:0/830990590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f784406a0f0 0x7f7844144260 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f78440b1a50 tx=0x7f784001c460 comp rx=0 tx=0).stop 2023-10-16T20:05:42.132 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.129+0000 7f784b71d640 1 -- 172.21.15.88:0/830990590 shutdown_connections 2023-10-16T20:05:42.132 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.129+0000 7f784b71d640 1 --2- 172.21.15.88:0/830990590 >> v2:172.21.15.94:3300/0 conn(0x7f78440c9be0 0x7f784406a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:42.132 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.129+0000 7f784b71d640 1 --2- 172.21.15.88:0/830990590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f784406a0f0 0x7f7844144260 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:42.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.129+0000 7f784b71d640 1 -- 172.21.15.88:0/830990590 >> 172.21.15.88:0/830990590 conn(0x7f78440b0970 msgr2=0x7f78440b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:42.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.129+0000 7f784b71d640 1 -- 172.21.15.88:0/830990590 shutdown_connections 2023-10-16T20:05:42.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.129+0000 7f784b71d640 1 -- 172.21.15.88:0/830990590 wait complete. 2023-10-16T20:05:42.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.129+0000 7f784b71d640 1 Processor -- start 2023-10-16T20:05:42.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.129+0000 7f784b71d640 1 -- start start 2023-10-16T20:05:42.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.129+0000 7f784b71d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f784406a0f0 0x7f7844155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:42.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.133+0000 7f784b71d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f784406a4d0 0x7f7844143220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:42.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.133+0000 7f784b71d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f78440c9be0 0x7f7844143760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:42.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.133+0000 7f784b71d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7844156520 con 0x7f784406a4d0 2023-10-16T20:05:42.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.133+0000 7f784b71d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7844150f30 con 0x7f784406a0f0 2023-10-16T20:05:42.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.133+0000 7f784b71d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7844155c80 con 0x7f78440c9be0 2023-10-16T20:05:42.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.133+0000 7f7848cb8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f784406a4d0 0x7f7844143220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:42.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.133+0000 7f7848cb8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f784406a4d0 0x7f7844143220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47928/0 (socket says 172.21.15.88:47928) 2023-10-16T20:05:42.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.133+0000 7f7848cb8640 1 -- 172.21.15.88:0/1497961887 learned_addr learned my addr 172.21.15.88:0/1497961887 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:42.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.133+0000 7f78494b9640 1 --2- 172.21.15.88:0/1497961887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f784406a0f0 0x7f7844155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:42.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.133+0000 7f7849cba640 1 --2- 172.21.15.88:0/1497961887 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f78440c9be0 0x7f7844143760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:42.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.133+0000 7f7848cb8640 1 -- 172.21.15.88:0/1497961887 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f78440c9be0 msgr2=0x7f7844143760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:42.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.133+0000 7f7848cb8640 1 --2- 172.21.15.88:0/1497961887 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f78440c9be0 0x7f7844143760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:42.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.133+0000 7f7848cb8640 1 -- 172.21.15.88:0/1497961887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f784406a0f0 msgr2=0x7f7844155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:42.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.133+0000 7f7848cb8640 1 --2- 172.21.15.88:0/1497961887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f784406a0f0 0x7f7844155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:42.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.133+0000 7f7848cb8640 1 -- 172.21.15.88:0/1497961887 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7840009420 con 0x7f784406a4d0 2023-10-16T20:05:42.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.133+0000 7f7848cb8640 1 --2- 172.21.15.88:0/1497961887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f784406a4d0 0x7f7844143220 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f783400c520 tx=0x7f783400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:42.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.133+0000 7f7839ffb640 1 -- 172.21.15.88:0/1497961887 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78340074e0 con 0x7f784406a4d0 2023-10-16T20:05:42.139 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.133+0000 7f7839ffb640 1 -- 172.21.15.88:0/1497961887 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7834007640 con 0x7f784406a4d0 2023-10-16T20:05:42.139 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.133+0000 7f7839ffb640 1 -- 172.21.15.88:0/1497961887 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78340049f0 con 0x7f784406a4d0 2023-10-16T20:05:42.139 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.133+0000 7f784b71d640 1 -- 172.21.15.88:0/1497961887 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7844155f60 con 0x7f784406a4d0 2023-10-16T20:05:42.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.133+0000 7f784b71d640 1 -- 172.21.15.88:0/1497961887 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78440b4c00 con 0x7f784406a4d0 2023-10-16T20:05:42.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.133+0000 7f784b71d640 1 -- 172.21.15.88:0/1497961887 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f784404e9c0 con 0x7f784406a4d0 2023-10-16T20:05:42.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.133+0000 7f7839ffb640 1 -- 172.21.15.88:0/1497961887 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7834004b50 con 0x7f784406a4d0 2023-10-16T20:05:42.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.137+0000 7f7839ffb640 1 --2- 172.21.15.88:0/1497961887 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f78280664a0 0x7f7828068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:42.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.137+0000 7f7839ffb640 1 -- 172.21.15.88:0/1497961887 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7834083380 con 0x7f784406a4d0 2023-10-16T20:05:42.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.137+0000 7f78494b9640 1 --2- 172.21.15.88:0/1497961887 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f78280664a0 0x7f7828068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:42.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.137+0000 7f78494b9640 1 --2- 172.21.15.88:0/1497961887 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f78280664a0 0x7f7828068960 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f78400af000 tx=0x7f78400a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:42.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.141+0000 7f7839ffb640 1 -- 172.21.15.88:0/1497961887 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7834050580 con 0x7f784406a4d0 2023-10-16T20:05:42.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.289+0000 7f784b71d640 1 -- 172.21.15.88:0/1497961887 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7844151070 con 0x7f784406a4d0 2023-10-16T20:05:42.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.289+0000 7f7839ffb640 1 -- 172.21.15.88:0/1497961887 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7834053ba0 con 0x7f784406a4d0 2023-10-16T20:05:42.293 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:05:42.295 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.293+0000 7f784b71d640 1 -- 172.21.15.88:0/1497961887 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f78280664a0 msgr2=0x7f7828068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:42.295 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.293+0000 7f784b71d640 1 --2- 172.21.15.88:0/1497961887 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f78280664a0 0x7f7828068960 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f78400af000 tx=0x7f78400a7000 comp rx=0 tx=0).stop 2023-10-16T20:05:42.295 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.293+0000 7f784b71d640 1 -- 172.21.15.88:0/1497961887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f784406a4d0 msgr2=0x7f7844143220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:42.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.293+0000 7f784b71d640 1 --2- 172.21.15.88:0/1497961887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f784406a4d0 0x7f7844143220 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f783400c520 tx=0x7f783400c9e0 comp rx=0 tx=0).stop 2023-10-16T20:05:42.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.293+0000 7f784b71d640 1 -- 172.21.15.88:0/1497961887 shutdown_connections 2023-10-16T20:05:42.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.293+0000 7f784b71d640 1 --2- 172.21.15.88:0/1497961887 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f78280664a0 0x7f7828068960 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:42.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.293+0000 7f784b71d640 1 --2- 172.21.15.88:0/1497961887 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f78440c9be0 0x7f7844143760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:42.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.293+0000 7f784b71d640 1 --2- 172.21.15.88:0/1497961887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f784406a4d0 0x7f7844143220 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:42.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.293+0000 7f784b71d640 1 --2- 172.21.15.88:0/1497961887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f784406a0f0 0x7f7844155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:42.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.293+0000 7f784b71d640 1 -- 172.21.15.88:0/1497961887 >> 172.21.15.88:0/1497961887 conn(0x7f78440b0970 msgr2=0x7f78440c2390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:42.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.293+0000 7f784b71d640 1 -- 172.21.15.88:0/1497961887 shutdown_connections 2023-10-16T20:05:42.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.293+0000 7f784b71d640 1 -- 172.21.15.88:0/1497961887 wait complete. 2023-10-16T20:05:42.314 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T20:05:42.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.453+0000 7f4a9759e640 1 Processor -- start 2023-10-16T20:05:42.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.453+0000 7f4a9759e640 1 -- start start 2023-10-16T20:05:42.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.453+0000 7f4a9759e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4a980cde80 0x7f4a980d3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:42.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.453+0000 7f4a9759e640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a980d0c40 con 0x7f4a980cf230 2023-10-16T20:05:42.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.453+0000 7f4a9759e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a980ce2d0 con 0x7f4a9815c9a0 2023-10-16T20:05:42.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.453+0000 7f4a9759e640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a980d37b0 con 0x7f4a980cde80 2023-10-16T20:05:42.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.453+0000 7f4a96d9d640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f4a980cf230 0x7f4a980cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:34630/0 (socket says 172.21.15.88:34630) 2023-10-16T20:05:42.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.453+0000 7f4a96d9d640 1 -- 172.21.15.88:0/1925851445 learned_addr learned my addr 172.21.15.88:0/1925851445 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:42.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.453+0000 7f4a95d9b640 1 --2- 172.21.15.88:0/1925851445 >> v2:172.21.15.94:3300/0 conn(0x7f4a980cde80 0x7f4a980d3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:42.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.453+0000 7f4a9559a640 1 -- 172.21.15.88:0/1925851445 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1938813811 0 0) 0x7f4a980d0c40 con 0x7f4a980cf230 2023-10-16T20:05:42.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.453+0000 7f4a9559a640 1 -- 172.21.15.88:0/1925851445 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a80003530 con 0x7f4a980cf230 2023-10-16T20:05:42.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.453+0000 7f4a95d9b640 1 -- 172.21.15.88:0/1925851445 >> v1:172.21.15.94:6789/0 conn(0x7f4a9815c9a0 legacy=0x7f4a980d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:42.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.453+0000 7f4a9559a640 1 -- 172.21.15.88:0/1925851445 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 81103719 0 0) 0x7f4a980ce2d0 con 0x7f4a9815c9a0 2023-10-16T20:05:42.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.453+0000 7f4a95d9b640 1 -- 172.21.15.88:0/1925851445 >> v1:172.21.15.88:6789/0 conn(0x7f4a980cf230 legacy=0x7f4a980cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:42.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.453+0000 7f4a95d9b640 1 -- 172.21.15.88:0/1925851445 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a980d0630 con 0x7f4a980cde80 2023-10-16T20:05:42.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.457+0000 7f4a95d9b640 1 --2- 172.21.15.88:0/1925851445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a980cde80 0x7f4a980d3270 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0097c0 tx=0x7f4a8c01c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a088122208aba4d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:42.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.457+0000 7f4a9559a640 1 -- 172.21.15.88:0/1925851445 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c0ab600 con 0x7f4a980cde80 2023-10-16T20:05:42.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.457+0000 7f4a9559a640 1 -- 172.21.15.88:0/1925851445 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a8c0a7050 con 0x7f4a980cde80 2023-10-16T20:05:42.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.457+0000 7f4a9559a640 1 -- 172.21.15.88:0/1925851445 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c0ab980 con 0x7f4a980cde80 2023-10-16T20:05:42.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.457+0000 7f4a9759e640 1 -- 172.21.15.88:0/1925851445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a980cde80 msgr2=0x7f4a980d3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:42.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.457+0000 7f4a9759e640 1 --2- 172.21.15.88:0/1925851445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a980cde80 0x7f4a980d3270 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0097c0 tx=0x7f4a8c01c3f0 comp rx=0 tx=0).stop 2023-10-16T20:05:42.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.457+0000 7f4a9759e640 1 -- 172.21.15.88:0/1925851445 shutdown_connections 2023-10-16T20:05:42.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.457+0000 7f4a9759e640 1 --2- 172.21.15.88:0/1925851445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a980cde80 0x7f4a980d3270 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:42.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.457+0000 7f4a9759e640 1 -- 172.21.15.88:0/1925851445 >> 172.21.15.88:0/1925851445 conn(0x7f4a9809bc30 msgr2=0x7f4a980d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:42.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.457+0000 7f4a9759e640 1 -- 172.21.15.88:0/1925851445 shutdown_connections 2023-10-16T20:05:42.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.457+0000 7f4a9759e640 1 -- 172.21.15.88:0/1925851445 wait complete. 2023-10-16T20:05:42.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.461+0000 7f4a9759e640 1 Processor -- start 2023-10-16T20:05:42.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.461+0000 7f4a9759e640 1 -- start start 2023-10-16T20:05:42.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.461+0000 7f4a9759e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4a980cde80 0x7f4a98158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:42.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.461+0000 7f4a9759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a980cf230 0x7f4a98156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:42.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.461+0000 7f4a9759e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a9815c9a0 0x7f4a98156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:42.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.461+0000 7f4a9759e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a980c8280 con 0x7f4a9815c9a0 2023-10-16T20:05:42.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.461+0000 7f4a9759e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a980c7950 con 0x7f4a980cf230 2023-10-16T20:05:42.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.461+0000 7f4a9759e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a980c7c30 con 0x7f4a980cde80 2023-10-16T20:05:42.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.461+0000 7f4a96d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a9815c9a0 0x7f4a98156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:42.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.461+0000 7f4a96d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a9815c9a0 0x7f4a98156770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47930/0 (socket says 172.21.15.88:47930) 2023-10-16T20:05:42.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.461+0000 7f4a96d9d640 1 -- 172.21.15.88:0/665200367 learned_addr learned my addr 172.21.15.88:0/665200367 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:42.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.461+0000 7f4a95d9b640 1 --2- 172.21.15.88:0/665200367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a980cf230 0x7f4a98156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:42.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.461+0000 7f4a9659c640 1 --2- 172.21.15.88:0/665200367 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4a980cde80 0x7f4a98158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:42.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.461+0000 7f4a96d9d640 1 -- 172.21.15.88:0/665200367 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4a980cde80 msgr2=0x7f4a98158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:42.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.461+0000 7f4a96d9d640 1 --2- 172.21.15.88:0/665200367 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4a980cde80 0x7f4a98158a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:42.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.461+0000 7f4a96d9d640 1 -- 172.21.15.88:0/665200367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a980cf230 msgr2=0x7f4a98156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:42.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.461+0000 7f4a96d9d640 1 --2- 172.21.15.88:0/665200367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a980cf230 0x7f4a98156230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:42.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.461+0000 7f4a96d9d640 1 -- 172.21.15.88:0/665200367 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8c009420 con 0x7f4a9815c9a0 2023-10-16T20:05:42.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.461+0000 7f4a95d9b640 1 --2- 172.21.15.88:0/665200367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a980cf230 0x7f4a98156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:42.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.461+0000 7f4a9659c640 1 --2- 172.21.15.88:0/665200367 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4a980cde80 0x7f4a98158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:42.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.461+0000 7f4a96d9d640 1 --2- 172.21.15.88:0/665200367 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a9815c9a0 0x7f4a98156770 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4a9000cb40 tx=0x7f4a9000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:42.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.461+0000 7f4a86ffd640 1 -- 172.21.15.88:0/665200367 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9000aca0 con 0x7f4a9815c9a0 2023-10-16T20:05:42.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.461+0000 7f4a86ffd640 1 -- 172.21.15.88:0/665200367 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a9000ae00 con 0x7f4a9815c9a0 2023-10-16T20:05:42.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.461+0000 7f4a86ffd640 1 -- 172.21.15.88:0/665200367 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a90012620 con 0x7f4a9815c9a0 2023-10-16T20:05:42.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.461+0000 7f4a9759e640 1 -- 172.21.15.88:0/665200367 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a9815cf00 con 0x7f4a9815c9a0 2023-10-16T20:05:42.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.461+0000 7f4a9759e640 1 -- 172.21.15.88:0/665200367 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a980b6d80 con 0x7f4a9815c9a0 2023-10-16T20:05:42.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.465+0000 7f4a9759e640 1 -- 172.21.15.88:0/665200367 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a980c8430 con 0x7f4a9815c9a0 2023-10-16T20:05:42.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.465+0000 7f4a86ffd640 1 -- 172.21.15.88:0/665200367 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4a90007070 con 0x7f4a9815c9a0 2023-10-16T20:05:42.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.465+0000 7f4a86ffd640 1 --2- 172.21.15.88:0/665200367 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4a780664a0 0x7f4a78068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:42.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.465+0000 7f4a86ffd640 1 -- 172.21.15.88:0/665200367 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4a900842b0 con 0x7f4a9815c9a0 2023-10-16T20:05:42.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.465+0000 7f4a9659c640 1 --2- 172.21.15.88:0/665200367 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4a780664a0 0x7f4a78068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:42.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.465+0000 7f4a9659c640 1 --2- 172.21.15.88:0/665200367 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4a780664a0 0x7f4a78068960 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4a88000fc0 tx=0x7f4a88009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:42.475 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.469+0000 7f4a86ffd640 1 -- 172.21.15.88:0/665200367 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4a980c8430 con 0x7f4a9815c9a0 2023-10-16T20:05:42.620 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.617+0000 7f4a9759e640 1 -- 172.21.15.88:0/665200367 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4a980674f0 con 0x7f4a9815c9a0 2023-10-16T20:05:42.621 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.617+0000 7f4a86ffd640 1 -- 172.21.15.88:0/665200367 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4a90051360 con 0x7f4a9815c9a0 2023-10-16T20:05:42.621 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:05:42.623 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.621+0000 7f4a9759e640 1 -- 172.21.15.88:0/665200367 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4a780664a0 msgr2=0x7f4a78068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:42.623 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.621+0000 7f4a9759e640 1 --2- 172.21.15.88:0/665200367 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4a780664a0 0x7f4a78068960 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4a88000fc0 tx=0x7f4a88009040 comp rx=0 tx=0).stop 2023-10-16T20:05:42.624 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.621+0000 7f4a9759e640 1 -- 172.21.15.88:0/665200367 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a9815c9a0 msgr2=0x7f4a98156770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:42.624 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.621+0000 7f4a9759e640 1 --2- 172.21.15.88:0/665200367 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a9815c9a0 0x7f4a98156770 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4a9000cb40 tx=0x7f4a9000a060 comp rx=0 tx=0).stop 2023-10-16T20:05:42.624 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.621+0000 7f4a9759e640 1 -- 172.21.15.88:0/665200367 shutdown_connections 2023-10-16T20:05:42.624 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.621+0000 7f4a9759e640 1 --2- 172.21.15.88:0/665200367 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4a780664a0 0x7f4a78068960 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:42.624 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.621+0000 7f4a9759e640 1 --2- 172.21.15.88:0/665200367 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a9815c9a0 0x7f4a98156770 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:42.625 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.621+0000 7f4a9759e640 1 --2- 172.21.15.88:0/665200367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a980cf230 0x7f4a98156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:42.625 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.621+0000 7f4a9759e640 1 --2- 172.21.15.88:0/665200367 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4a980cde80 0x7f4a98158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:42.625 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.621+0000 7f4a9759e640 1 -- 172.21.15.88:0/665200367 >> 172.21.15.88:0/665200367 conn(0x7f4a9809bc30 msgr2=0x7f4a980d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:42.625 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.621+0000 7f4a9759e640 1 -- 172.21.15.88:0/665200367 shutdown_connections 2023-10-16T20:05:42.625 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.621+0000 7f4a9759e640 1 -- 172.21.15.88:0/665200367 wait complete. 2023-10-16T20:05:42.642 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T20:05:42.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.781+0000 7f9ae5a6b640 1 Processor -- start 2023-10-16T20:05:42.783 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.781+0000 7f9ae5a6b640 1 -- start start 2023-10-16T20:05:42.784 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.781+0000 7f9ae5a6b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ae00aeb90 0x7f9ae0070250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:42.784 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.781+0000 7f9ae5a6b640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ae006c2e0 con 0x7f9ae0146290 2023-10-16T20:05:42.784 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.781+0000 7f9ae5a6b640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ae0160ec0 con 0x7f9ae01457f0 2023-10-16T20:05:42.784 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.781+0000 7f9ae5a6b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae0145bd0 con 0x7f9ae00aeb90 2023-10-16T20:05:42.785 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.781+0000 7f9adffff640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f9ae0146290 0x7f9ae006caa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:34636/0 (socket says 172.21.15.88:34636) 2023-10-16T20:05:42.785 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.781+0000 7f9adffff640 1 -- 172.21.15.88:0/3104735104 learned_addr learned my addr 172.21.15.88:0/3104735104 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:42.785 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.781+0000 7f9adeffd640 1 --2- 172.21.15.88:0/3104735104 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ae00aeb90 0x7f9ae0070250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:42.786 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.781+0000 7f9ade7fc640 1 -- 172.21.15.88:0/3104735104 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 429598933 0 0) 0x7f9ae006c2e0 con 0x7f9ae0146290 2023-10-16T20:05:42.786 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.781+0000 7f9ade7fc640 1 -- 172.21.15.88:0/3104735104 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ac8003530 con 0x7f9ae0146290 2023-10-16T20:05:42.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.781+0000 7f9ade7fc640 1 -- 172.21.15.88:0/3104735104 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1441325879 0 0) 0x7f9ae0160ec0 con 0x7f9ae01457f0 2023-10-16T20:05:42.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.781+0000 7f9ade7fc640 1 -- 172.21.15.88:0/3104735104 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ae006c2e0 con 0x7f9ae01457f0 2023-10-16T20:05:42.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.781+0000 7f9adeffd640 1 -- 172.21.15.88:0/3104735104 >> v1:172.21.15.94:6789/0 conn(0x7f9ae01457f0 legacy=0x7f9ae0160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:42.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.781+0000 7f9adeffd640 1 -- 172.21.15.88:0/3104735104 >> v1:172.21.15.88:6789/0 conn(0x7f9ae0146290 legacy=0x7f9ae006caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:42.788 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.781+0000 7f9adeffd640 1 -- 172.21.15.88:0/3104735104 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae01611a0 con 0x7f9ae00aeb90 2023-10-16T20:05:42.789 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.785+0000 7f9adeffd640 1 --2- 172.21.15.88:0/3104735104 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ae00aeb90 0x7f9ae0070250 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40097f0 tx=0x7f9ad401c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26eca1d45736ee81 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:42.789 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.785+0000 7f9ade7fc640 1 -- 172.21.15.88:0/3104735104 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad40ac600 con 0x7f9ae00aeb90 2023-10-16T20:05:42.789 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.785+0000 7f9ade7fc640 1 -- 172.21.15.88:0/3104735104 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad40a8050 con 0x7f9ae00aeb90 2023-10-16T20:05:42.790 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.785+0000 7f9ade7fc640 1 -- 172.21.15.88:0/3104735104 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad40ac980 con 0x7f9ae00aeb90 2023-10-16T20:05:42.790 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.785+0000 7f9ae5a6b640 1 -- 172.21.15.88:0/3104735104 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ae00aeb90 msgr2=0x7f9ae0070250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:42.790 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.785+0000 7f9ae5a6b640 1 --2- 172.21.15.88:0/3104735104 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ae00aeb90 0x7f9ae0070250 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40097f0 tx=0x7f9ad401c490 comp rx=0 tx=0).stop 2023-10-16T20:05:42.790 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.785+0000 7f9ae5a6b640 1 -- 172.21.15.88:0/3104735104 shutdown_connections 2023-10-16T20:05:42.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.785+0000 7f9ae5a6b640 1 --2- 172.21.15.88:0/3104735104 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ae00aeb90 0x7f9ae0070250 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:42.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.785+0000 7f9ae5a6b640 1 -- 172.21.15.88:0/3104735104 >> 172.21.15.88:0/3104735104 conn(0x7f9ae00cb080 msgr2=0x7f9ae00cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:42.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.785+0000 7f9ae5a6b640 1 -- 172.21.15.88:0/3104735104 shutdown_connections 2023-10-16T20:05:42.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.785+0000 7f9ae5a6b640 1 -- 172.21.15.88:0/3104735104 wait complete. 2023-10-16T20:05:42.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.789+0000 7f9ae5a6b640 1 Processor -- start 2023-10-16T20:05:42.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.789+0000 7f9ae5a6b640 1 -- start start 2023-10-16T20:05:42.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.789+0000 7f9ae5a6b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae00aeb90 0x7f9ae00c9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:42.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.789+0000 7f9ae5a6b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ae01457f0 0x7f9ae015f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:42.795 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.789+0000 7f9ae5a6b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ae0146290 0x7f9ae0143e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:42.795 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.789+0000 7f9ae5a6b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae0160370 con 0x7f9ae00aeb90 2023-10-16T20:05:42.795 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.789+0000 7f9ae5a6b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae015fad0 con 0x7f9ae0146290 2023-10-16T20:05:42.795 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.789+0000 7f9ae5a6b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae015fdb0 con 0x7f9ae01457f0 2023-10-16T20:05:42.795 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.789+0000 7f9adf7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae00aeb90 0x7f9ae00c9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:42.796 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.789+0000 7f9adf7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae00aeb90 0x7f9ae00c9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47942/0 (socket says 172.21.15.88:47942) 2023-10-16T20:05:42.796 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.789+0000 7f9adf7fe640 1 -- 172.21.15.88:0/2553635988 learned_addr learned my addr 172.21.15.88:0/2553635988 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:42.796 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.789+0000 7f9adeffd640 1 --2- 172.21.15.88:0/2553635988 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ae01457f0 0x7f9ae015f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:42.796 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.789+0000 7f9adffff640 1 --2- 172.21.15.88:0/2553635988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ae0146290 0x7f9ae0143e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:42.797 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.789+0000 7f9adf7fe640 1 -- 172.21.15.88:0/2553635988 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ae01457f0 msgr2=0x7f9ae015f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:42.797 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.789+0000 7f9adf7fe640 1 --2- 172.21.15.88:0/2553635988 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ae01457f0 0x7f9ae015f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:42.797 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.789+0000 7f9adf7fe640 1 -- 172.21.15.88:0/2553635988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ae0146290 msgr2=0x7f9ae0143e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:42.797 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.789+0000 7f9adf7fe640 1 --2- 172.21.15.88:0/2553635988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ae0146290 0x7f9ae0143e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:42.797 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.789+0000 7f9adf7fe640 1 -- 172.21.15.88:0/2553635988 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad4009420 con 0x7f9ae00aeb90 2023-10-16T20:05:42.798 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.789+0000 7f9adf7fe640 1 --2- 172.21.15.88:0/2553635988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae00aeb90 0x7f9ae00c9c00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9ad000c490 tx=0x7f9ad000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:42.798 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.789+0000 7f9ac7fff640 1 -- 172.21.15.88:0/2553635988 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad00052d0 con 0x7f9ae00aeb90 2023-10-16T20:05:42.798 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.789+0000 7f9ac7fff640 1 -- 172.21.15.88:0/2553635988 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad0005430 con 0x7f9ae00aeb90 2023-10-16T20:05:42.798 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.789+0000 7f9ac7fff640 1 -- 172.21.15.88:0/2553635988 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0012870 con 0x7f9ae00aeb90 2023-10-16T20:05:42.799 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.789+0000 7f9ae5a6b640 1 -- 172.21.15.88:0/2553635988 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae0160090 con 0x7f9ae00aeb90 2023-10-16T20:05:42.799 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.789+0000 7f9ae5a6b640 1 -- 172.21.15.88:0/2553635988 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae00ca140 con 0x7f9ae00aeb90 2023-10-16T20:05:42.799 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.789+0000 7f9ae5a6b640 1 -- 172.21.15.88:0/2553635988 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ae004e9c0 con 0x7f9ae00aeb90 2023-10-16T20:05:42.800 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.793+0000 7f9ac7fff640 1 -- 172.21.15.88:0/2553635988 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9ad00129d0 con 0x7f9ae00aeb90 2023-10-16T20:05:42.800 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.793+0000 7f9ac7fff640 1 --2- 172.21.15.88:0/2553635988 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9acc0664a0 0x7f9acc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:42.800 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.793+0000 7f9ac7fff640 1 -- 172.21.15.88:0/2553635988 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9ad0083e00 con 0x7f9ae00aeb90 2023-10-16T20:05:42.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.793+0000 7f9adeffd640 1 --2- 172.21.15.88:0/2553635988 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9acc0664a0 0x7f9acc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:42.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.793+0000 7f9adeffd640 1 --2- 172.21.15.88:0/2553635988 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9acc0664a0 0x7f9acc068960 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40ae000 tx=0x7f9ad40a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:42.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.797+0000 7f9ac7fff640 1 -- 172.21.15.88:0/2553635988 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9ad0050e20 con 0x7f9ae00aeb90 2023-10-16T20:05:42.950 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.949+0000 7f9ae5a6b640 1 -- 172.21.15.88:0/2553635988 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9ae006ac50 con 0x7f9ae00aeb90 2023-10-16T20:05:42.951 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.949+0000 7f9ac7fff640 1 -- 172.21.15.88:0/2553635988 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9ad0054440 con 0x7f9ae00aeb90 2023-10-16T20:05:42.951 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2023-10-16T20:05:42.953 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.949+0000 7f9ae5a6b640 1 -- 172.21.15.88:0/2553635988 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9acc0664a0 msgr2=0x7f9acc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:42.953 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.949+0000 7f9ae5a6b640 1 --2- 172.21.15.88:0/2553635988 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9acc0664a0 0x7f9acc068960 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40ae000 tx=0x7f9ad40a7000 comp rx=0 tx=0).stop 2023-10-16T20:05:42.953 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.949+0000 7f9ae5a6b640 1 -- 172.21.15.88:0/2553635988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae00aeb90 msgr2=0x7f9ae00c9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:42.953 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.949+0000 7f9ae5a6b640 1 --2- 172.21.15.88:0/2553635988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae00aeb90 0x7f9ae00c9c00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9ad000c490 tx=0x7f9ad000c950 comp rx=0 tx=0).stop 2023-10-16T20:05:42.953 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.949+0000 7f9ae5a6b640 1 -- 172.21.15.88:0/2553635988 shutdown_connections 2023-10-16T20:05:42.953 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.949+0000 7f9ae5a6b640 1 --2- 172.21.15.88:0/2553635988 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9acc0664a0 0x7f9acc068960 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:42.954 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.949+0000 7f9ae5a6b640 1 --2- 172.21.15.88:0/2553635988 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9ae01457f0 0x7f9ae015f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:42.954 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.949+0000 7f9ae5a6b640 1 --2- 172.21.15.88:0/2553635988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ae0146290 0x7f9ae0143e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:42.954 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.949+0000 7f9ae5a6b640 1 --2- 172.21.15.88:0/2553635988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae00aeb90 0x7f9ae00c9c00 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:42.954 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.949+0000 7f9ae5a6b640 1 -- 172.21.15.88:0/2553635988 >> 172.21.15.88:0/2553635988 conn(0x7f9ae00cb080 msgr2=0x7f9ae0069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:42.955 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.949+0000 7f9ae5a6b640 1 -- 172.21.15.88:0/2553635988 shutdown_connections 2023-10-16T20:05:42.955 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:42.949+0000 7f9ae5a6b640 1 -- 172.21.15.88:0/2553635988 wait complete. 2023-10-16T20:05:42.972 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T20:05:43.112 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.109+0000 7f90780c9640 1 Processor -- start 2023-10-16T20:05:43.112 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.109+0000 7f90780c9640 1 -- start start 2023-10-16T20:05:43.113 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.109+0000 7f90780c9640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f90700711f0 0x7f9070159f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:43.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.109+0000 7f90780c9640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f907006c2e0 0x7f907015aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:43.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.109+0000 7f90780c9640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f907006c790 con 0x7f90701500a0 2023-10-16T20:05:43.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.109+0000 7f90780c9640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9070150480 con 0x7f90700711f0 2023-10-16T20:05:43.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.109+0000 7f90780c9640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f907015a490 con 0x7f907006c2e0 2023-10-16T20:05:43.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.113+0000 7f9075e65640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f90701500a0 0x7f9070070ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:34652/0 (socket says 172.21.15.88:34652) 2023-10-16T20:05:43.115 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.113+0000 7f9075664640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f90700711f0 0x7f9070159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:43.115 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.113+0000 7f9075e65640 1 -- 172.21.15.88:0/2368369558 learned_addr learned my addr 172.21.15.88:0/2368369558 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:43.115 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.113+0000 7f9076666640 1 --2- 172.21.15.88:0/2368369558 >> v2:172.21.15.94:3300/0 conn(0x7f907006c2e0 0x7f907015aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:43.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.113+0000 7f9075664640 1 -- 172.21.15.88:0/2368369558 >> v2:172.21.15.94:3300/0 conn(0x7f907006c2e0 msgr2=0x7f907015aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:43.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.113+0000 7f9075664640 1 --2- 172.21.15.88:0/2368369558 >> v2:172.21.15.94:3300/0 conn(0x7f907006c2e0 0x7f907015aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:43.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.113+0000 7f9075664640 1 -- 172.21.15.88:0/2368369558 >> v1:172.21.15.88:6789/0 conn(0x7f90701500a0 legacy=0x7f9070070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:43.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.113+0000 7f9075664640 1 -- 172.21.15.88:0/2368369558 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f907015aa10 con 0x7f90700711f0 2023-10-16T20:05:43.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.113+0000 7f9076666640 1 --2- 172.21.15.88:0/2368369558 >> v2:172.21.15.94:3300/0 conn(0x7f907006c2e0 0x7f907015aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:43.117 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.113+0000 7f9074e63640 1 -- 172.21.15.88:0/2368369558 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1615789762 0 0) 0x7f907006c790 con 0x7f90701500a0 2023-10-16T20:05:43.117 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.113+0000 7f9075664640 1 --2- 172.21.15.88:0/2368369558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90700711f0 0x7f9070159f50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f90700b1060 tx=0x7f906001c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59374bad15eeb08 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:43.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.113+0000 7f9074e63640 1 -- 172.21.15.88:0/2368369558 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90600ab600 con 0x7f90700711f0 2023-10-16T20:05:43.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.113+0000 7f9074e63640 1 -- 172.21.15.88:0/2368369558 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90600a7050 con 0x7f90700711f0 2023-10-16T20:05:43.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.113+0000 7f9074e63640 1 -- 172.21.15.88:0/2368369558 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90600ab980 con 0x7f90700711f0 2023-10-16T20:05:43.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.117+0000 7f90780c9640 1 -- 172.21.15.88:0/2368369558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90700711f0 msgr2=0x7f9070159f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:43.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.117+0000 7f90780c9640 1 --2- 172.21.15.88:0/2368369558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90700711f0 0x7f9070159f50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f90700b1060 tx=0x7f906001c3b0 comp rx=0 tx=0).stop 2023-10-16T20:05:43.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.117+0000 7f90780c9640 1 -- 172.21.15.88:0/2368369558 shutdown_connections 2023-10-16T20:05:43.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.117+0000 7f90780c9640 1 --2- 172.21.15.88:0/2368369558 >> v2:172.21.15.94:3300/0 conn(0x7f907006c2e0 0x7f907015aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:43.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.117+0000 7f90780c9640 1 --2- 172.21.15.88:0/2368369558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90700711f0 0x7f9070159f50 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:43.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.117+0000 7f90780c9640 1 -- 172.21.15.88:0/2368369558 >> 172.21.15.88:0/2368369558 conn(0x7f90700cb080 msgr2=0x7f90700cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:43.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.117+0000 7f90780c9640 1 -- 172.21.15.88:0/2368369558 shutdown_connections 2023-10-16T20:05:43.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.117+0000 7f90780c9640 1 -- 172.21.15.88:0/2368369558 wait complete. 2023-10-16T20:05:43.122 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.117+0000 7f90780c9640 1 Processor -- start 2023-10-16T20:05:43.122 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.117+0000 7f90780c9640 1 -- start start 2023-10-16T20:05:43.123 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.117+0000 7f90780c9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f907006c2e0 0x7f9070144c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:43.123 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.117+0000 7f90780c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90700711f0 0x7f90701453f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:43.123 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.117+0000 7f90780c9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90701500a0 0x7f9070148e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:43.123 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.117+0000 7f90780c9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9070149930 con 0x7f90701500a0 2023-10-16T20:05:43.123 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.117+0000 7f90780c9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907014df30 con 0x7f90700711f0 2023-10-16T20:05:43.123 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.117+0000 7f90780c9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f907014e210 con 0x7f907006c2e0 2023-10-16T20:05:43.124 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.117+0000 7f9076666640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90701500a0 0x7f9070148e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:43.124 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.117+0000 7f9076666640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90701500a0 0x7f9070148e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47948/0 (socket says 172.21.15.88:47948) 2023-10-16T20:05:43.124 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.117+0000 7f9076666640 1 -- 172.21.15.88:0/2944251458 learned_addr learned my addr 172.21.15.88:0/2944251458 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:43.124 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.117+0000 7f9075664640 1 --2- 172.21.15.88:0/2944251458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90700711f0 0x7f90701453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:43.124 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.117+0000 7f9075e65640 1 --2- 172.21.15.88:0/2944251458 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f907006c2e0 0x7f9070144c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:43.125 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.117+0000 7f9076666640 1 -- 172.21.15.88:0/2944251458 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f907006c2e0 msgr2=0x7f9070144c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:43.125 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.117+0000 7f9076666640 1 --2- 172.21.15.88:0/2944251458 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f907006c2e0 0x7f9070144c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:43.125 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.117+0000 7f9076666640 1 -- 172.21.15.88:0/2944251458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90700711f0 msgr2=0x7f90701453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:43.125 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.117+0000 7f9076666640 1 --2- 172.21.15.88:0/2944251458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90700711f0 0x7f90701453f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:43.125 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.117+0000 7f9076666640 1 -- 172.21.15.88:0/2944251458 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9060009420 con 0x7f90701500a0 2023-10-16T20:05:43.126 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.117+0000 7f9075664640 1 --2- 172.21.15.88:0/2944251458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90700711f0 0x7f90701453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:43.126 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.117+0000 7f9076666640 1 --2- 172.21.15.88:0/2944251458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90701500a0 0x7f9070148e90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f906800cef0 tx=0x7f906800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:43.126 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.121+0000 7f90667fc640 1 -- 172.21.15.88:0/2944251458 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906800d840 con 0x7f90701500a0 2023-10-16T20:05:43.126 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.121+0000 7f90667fc640 1 -- 172.21.15.88:0/2944251458 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f906800d9c0 con 0x7f90701500a0 2023-10-16T20:05:43.126 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.121+0000 7f90667fc640 1 -- 172.21.15.88:0/2944251458 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906800dd70 con 0x7f90701500a0 2023-10-16T20:05:43.126 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.121+0000 7f90780c9640 1 -- 172.21.15.88:0/2944251458 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f907014e4f0 con 0x7f90701500a0 2023-10-16T20:05:43.127 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.121+0000 7f90780c9640 1 -- 172.21.15.88:0/2944251458 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9070145170 con 0x7f90701500a0 2023-10-16T20:05:43.127 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.121+0000 7f90780c9640 1 -- 172.21.15.88:0/2944251458 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f907014ea10 con 0x7f90701500a0 2023-10-16T20:05:43.128 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.121+0000 7f90667fc640 1 -- 172.21.15.88:0/2944251458 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f906801e3b0 con 0x7f90701500a0 2023-10-16T20:05:43.128 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.121+0000 7f90667fc640 1 --2- 172.21.15.88:0/2944251458 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f90540664a0 0x7f9054068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:43.128 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.121+0000 7f90667fc640 1 -- 172.21.15.88:0/2944251458 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9068084d10 con 0x7f90701500a0 2023-10-16T20:05:43.128 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.121+0000 7f9075e65640 1 --2- 172.21.15.88:0/2944251458 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f90540664a0 0x7f9054068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:43.128 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.121+0000 7f9075e65640 1 --2- 172.21.15.88:0/2944251458 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f90540664a0 0x7f9054068960 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f906c000f70 tx=0x7f906c007c10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:43.129 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.125+0000 7f90667fc640 1 -- 172.21.15.88:0/2944251458 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f907014ea10 con 0x7f90701500a0 2023-10-16T20:05:43.278 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.277+0000 7f90780c9640 1 -- 172.21.15.88:0/2944251458 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f907014ea10 con 0x7f90701500a0 2023-10-16T20:05:43.279 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.277+0000 7f90667fc640 1 -- 172.21.15.88:0/2944251458 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f907014ea10 con 0x7f90701500a0 2023-10-16T20:05:43.279 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:05:43.280 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.277+0000 7f90780c9640 1 -- 172.21.15.88:0/2944251458 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f90540664a0 msgr2=0x7f9054068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:43.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.277+0000 7f90780c9640 1 --2- 172.21.15.88:0/2944251458 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f90540664a0 0x7f9054068960 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f906c000f70 tx=0x7f906c007c10 comp rx=0 tx=0).stop 2023-10-16T20:05:43.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.277+0000 7f90780c9640 1 -- 172.21.15.88:0/2944251458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90701500a0 msgr2=0x7f9070148e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:43.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.277+0000 7f90780c9640 1 --2- 172.21.15.88:0/2944251458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90701500a0 0x7f9070148e90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f906800cef0 tx=0x7f906800a710 comp rx=0 tx=0).stop 2023-10-16T20:05:43.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.277+0000 7f90780c9640 1 -- 172.21.15.88:0/2944251458 shutdown_connections 2023-10-16T20:05:43.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.277+0000 7f90780c9640 1 --2- 172.21.15.88:0/2944251458 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f90540664a0 0x7f9054068960 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:43.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.277+0000 7f90780c9640 1 --2- 172.21.15.88:0/2944251458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90701500a0 0x7f9070148e90 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:43.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.277+0000 7f90780c9640 1 --2- 172.21.15.88:0/2944251458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90700711f0 0x7f90701453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:43.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.277+0000 7f90780c9640 1 --2- 172.21.15.88:0/2944251458 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f907006c2e0 0x7f9070144c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:43.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.277+0000 7f90780c9640 1 -- 172.21.15.88:0/2944251458 >> 172.21.15.88:0/2944251458 conn(0x7f90700cb080 msgr2=0x7f90700cc050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:43.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.277+0000 7f90780c9640 1 -- 172.21.15.88:0/2944251458 shutdown_connections 2023-10-16T20:05:43.283 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.277+0000 7f90780c9640 1 -- 172.21.15.88:0/2944251458 wait complete. 2023-10-16T20:05:43.299 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification' 2023-10-16T20:05:43.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.437+0000 7f3dc006b640 1 Processor -- start 2023-10-16T20:05:43.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.437+0000 7f3dc006b640 1 -- start start 2023-10-16T20:05:43.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.437+0000 7f3dc006b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3db80d22e0 0x7f3db8161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:43.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.437+0000 7f3dc006b640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f3db80d26c0 0x7f3db815d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:43.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.437+0000 7f3dc006b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3db815d820 0x7f3db80cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:43.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.437+0000 7f3dc006b640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3db80b87e0 con 0x7f3db80d26c0 2023-10-16T20:05:43.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.437+0000 7f3dc006b640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3db815dc50 con 0x7f3db80d22e0 2023-10-16T20:05:43.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.437+0000 7f3dc006b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db815df10 con 0x7f3db815d820 2023-10-16T20:05:43.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.437+0000 7f3dbd606640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f3db80d26c0 0x7f3db815d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:43.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.437+0000 7f3dbd606640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f3db80d26c0 0x7f3db815d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47954/0 (socket says 172.21.15.88:47954) 2023-10-16T20:05:43.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.437+0000 7f3dbd606640 1 -- 172.21.15.88:0/742709241 learned_addr learned my addr 172.21.15.88:0/742709241 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:43.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.437+0000 7f3dbe608640 1 --2- 172.21.15.88:0/742709241 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3db815d820 0x7f3db80cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:43.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.437+0000 7f3dbde07640 1 --2- 172.21.15.88:0/742709241 >> v2:172.21.15.94:3300/0 conn(0x7f3db80d22e0 0x7f3db8161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:43.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.437+0000 7f3dbd606640 1 -- 172.21.15.88:0/742709241 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3db815d820 msgr2=0x7f3db80cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:43.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.437+0000 7f3dbd606640 1 --2- 172.21.15.88:0/742709241 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3db815d820 0x7f3db80cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:43.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.437+0000 7f3dbd606640 1 -- 172.21.15.88:0/742709241 >> v2:172.21.15.94:3300/0 conn(0x7f3db80d22e0 msgr2=0x7f3db8161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:43.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.437+0000 7f3dbd606640 1 --2- 172.21.15.88:0/742709241 >> v2:172.21.15.94:3300/0 conn(0x7f3db80d22e0 0x7f3db8161450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:43.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.437+0000 7f3dbd606640 1 -- 172.21.15.88:0/742709241 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db815e1f0 con 0x7f3db80d26c0 2023-10-16T20:05:43.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.437+0000 7f3dbe608640 1 --2- 172.21.15.88:0/742709241 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3db815d820 0x7f3db80cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:43.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.437+0000 7f3dbde07640 1 --2- 172.21.15.88:0/742709241 >> v2:172.21.15.94:3300/0 conn(0x7f3db80d22e0 0x7f3db8161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:43.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.441+0000 7f3dbd606640 1 --2- 172.21.15.88:0/742709241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3db80d26c0 0x7f3db815d2e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3db808bb40 tx=0x7f3da801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df815d1dce28e29 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:43.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.441+0000 7f3dbce05640 1 -- 172.21.15.88:0/742709241 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da80ab600 con 0x7f3db80d26c0 2023-10-16T20:05:43.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.441+0000 7f3dbce05640 1 -- 172.21.15.88:0/742709241 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da80ab760 con 0x7f3db80d26c0 2023-10-16T20:05:43.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.441+0000 7f3dbce05640 1 -- 172.21.15.88:0/742709241 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da80abaf0 con 0x7f3db80d26c0 2023-10-16T20:05:43.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.441+0000 7f3dc006b640 1 -- 172.21.15.88:0/742709241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3db80d26c0 msgr2=0x7f3db815d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:43.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.441+0000 7f3dc006b640 1 --2- 172.21.15.88:0/742709241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3db80d26c0 0x7f3db815d2e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3db808bb40 tx=0x7f3da801c440 comp rx=0 tx=0).stop 2023-10-16T20:05:43.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.441+0000 7f3dc006b640 1 -- 172.21.15.88:0/742709241 shutdown_connections 2023-10-16T20:05:43.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.441+0000 7f3dc006b640 1 --2- 172.21.15.88:0/742709241 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3db815d820 0x7f3db80cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:43.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.441+0000 7f3dc006b640 1 --2- 172.21.15.88:0/742709241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3db80d26c0 0x7f3db815d2e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:43.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.441+0000 7f3dc006b640 1 --2- 172.21.15.88:0/742709241 >> v2:172.21.15.94:3300/0 conn(0x7f3db80d22e0 0x7f3db8161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:43.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.441+0000 7f3dc006b640 1 -- 172.21.15.88:0/742709241 >> 172.21.15.88:0/742709241 conn(0x7f3db80715c0 msgr2=0x7f3db80719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:43.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.441+0000 7f3dc006b640 1 -- 172.21.15.88:0/742709241 shutdown_connections 2023-10-16T20:05:43.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.441+0000 7f3dc006b640 1 -- 172.21.15.88:0/742709241 wait complete. 2023-10-16T20:05:43.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.445+0000 7f3dc006b640 1 Processor -- start 2023-10-16T20:05:43.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.445+0000 7f3dc006b640 1 -- start start 2023-10-16T20:05:43.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.445+0000 7f3dc006b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3db80d22e0 0x7f3db8149b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:43.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.445+0000 7f3dc006b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3db80d26c0 0x7f3db8156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:43.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.445+0000 7f3dc006b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3db815d820 0x7f3db814e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:43.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.445+0000 7f3dc006b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db80c7470 con 0x7f3db815d820 2023-10-16T20:05:43.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.445+0000 7f3dc006b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db80c7750 con 0x7f3db80d22e0 2023-10-16T20:05:43.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.445+0000 7f3dc006b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db80c7a30 con 0x7f3db80d26c0 2023-10-16T20:05:43.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.445+0000 7f3dbe608640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3db815d820 0x7f3db814e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:43.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.445+0000 7f3dbe608640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3db815d820 0x7f3db814e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47964/0 (socket says 172.21.15.88:47964) 2023-10-16T20:05:43.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.445+0000 7f3dbe608640 1 -- 172.21.15.88:0/3408976236 learned_addr learned my addr 172.21.15.88:0/3408976236 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:43.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.445+0000 7f3dbde07640 1 --2- 172.21.15.88:0/3408976236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3db80d22e0 0x7f3db8149b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:43.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.445+0000 7f3dbd606640 1 --2- 172.21.15.88:0/3408976236 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3db80d26c0 0x7f3db8156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:43.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.445+0000 7f3dbe608640 1 -- 172.21.15.88:0/3408976236 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3db80d26c0 msgr2=0x7f3db8156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:43.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.445+0000 7f3dbe608640 1 --2- 172.21.15.88:0/3408976236 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3db80d26c0 0x7f3db8156cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:43.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.445+0000 7f3dbe608640 1 -- 172.21.15.88:0/3408976236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3db80d22e0 msgr2=0x7f3db8149b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:43.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.445+0000 7f3dbe608640 1 --2- 172.21.15.88:0/3408976236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3db80d22e0 0x7f3db8149b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:43.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.445+0000 7f3dbe608640 1 -- 172.21.15.88:0/3408976236 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da80094d0 con 0x7f3db815d820 2023-10-16T20:05:43.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.445+0000 7f3dbe608640 1 --2- 172.21.15.88:0/3408976236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3db815d820 0x7f3db814e110 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3db0007df0 tx=0x7f3db000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:43.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.445+0000 7f3dae7fc640 1 -- 172.21.15.88:0/3408976236 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db00039a0 con 0x7f3db815d820 2023-10-16T20:05:43.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.445+0000 7f3dae7fc640 1 -- 172.21.15.88:0/3408976236 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db000f040 con 0x7f3db815d820 2023-10-16T20:05:43.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.445+0000 7f3dc006b640 1 -- 172.21.15.88:0/3408976236 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db814f370 con 0x7f3db815d820 2023-10-16T20:05:43.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.445+0000 7f3dae7fc640 1 -- 172.21.15.88:0/3408976236 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db0003d40 con 0x7f3db815d820 2023-10-16T20:05:43.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.445+0000 7f3dc006b640 1 -- 172.21.15.88:0/3408976236 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db814e650 con 0x7f3db815d820 2023-10-16T20:05:43.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.445+0000 7f3dc006b640 1 -- 172.21.15.88:0/3408976236 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db804e9c0 con 0x7f3db815d820 2023-10-16T20:05:43.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.449+0000 7f3dae7fc640 1 -- 172.21.15.88:0/3408976236 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3db0004030 con 0x7f3db815d820 2023-10-16T20:05:43.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.449+0000 7f3dae7fc640 1 --2- 172.21.15.88:0/3408976236 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3d9c0664a0 0x7f3d9c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:43.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.449+0000 7f3dae7fc640 1 -- 172.21.15.88:0/3408976236 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3db0084520 con 0x7f3db815d820 2023-10-16T20:05:43.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.449+0000 7f3dbde07640 1 --2- 172.21.15.88:0/3408976236 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3d9c0664a0 0x7f3d9c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:43.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.449+0000 7f3dbde07640 1 --2- 172.21.15.88:0/3408976236 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3d9c0664a0 0x7f3d9c068960 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3db80c6310 tx=0x7f3db4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:43.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.453+0000 7f3dae7fc640 1 -- 172.21.15.88:0/3408976236 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3db00515f0 con 0x7f3db815d820 2023-10-16T20:05:43.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:43.657+0000 7f3dc006b640 1 -- 172.21.15.88:0/3408976236 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]} v 0) v1 -- 0x7f3db814f820 con 0x7f3db815d820 2023-10-16T20:05:44.234 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.229+0000 7f3dae7fc640 1 -- 172.21.15.88:0/3408976236 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]}]=0 v67) v1 ==== 150+0+0 (secure 0 0 0) 0x7f3db0054c10 con 0x7f3db815d820 2023-10-16T20:05:44.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.233+0000 7f3dc006b640 1 -- 172.21.15.88:0/3408976236 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3d9c0664a0 msgr2=0x7f3d9c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:44.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.233+0000 7f3dc006b640 1 --2- 172.21.15.88:0/3408976236 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3d9c0664a0 0x7f3d9c068960 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3db80c6310 tx=0x7f3db4008040 comp rx=0 tx=0).stop 2023-10-16T20:05:44.237 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.233+0000 7f3dc006b640 1 -- 172.21.15.88:0/3408976236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3db815d820 msgr2=0x7f3db814e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:44.237 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.233+0000 7f3dc006b640 1 --2- 172.21.15.88:0/3408976236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3db815d820 0x7f3db814e110 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3db0007df0 tx=0x7f3db000d4a0 comp rx=0 tx=0).stop 2023-10-16T20:05:44.237 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.233+0000 7f3dc006b640 1 -- 172.21.15.88:0/3408976236 shutdown_connections 2023-10-16T20:05:44.237 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.233+0000 7f3dc006b640 1 --2- 172.21.15.88:0/3408976236 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3d9c0664a0 0x7f3d9c068960 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:44.237 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.233+0000 7f3dc006b640 1 --2- 172.21.15.88:0/3408976236 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3db80d26c0 0x7f3db8156cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:44.238 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.233+0000 7f3dc006b640 1 --2- 172.21.15.88:0/3408976236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3db815d820 0x7f3db814e110 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:44.238 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.233+0000 7f3dc006b640 1 --2- 172.21.15.88:0/3408976236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3db80d22e0 0x7f3db8149b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:44.238 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.233+0000 7f3dc006b640 1 -- 172.21.15.88:0/3408976236 >> 172.21.15.88:0/3408976236 conn(0x7f3db80715c0 msgr2=0x7f3db80b78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:44.238 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.233+0000 7f3dc006b640 1 -- 172.21.15.88:0/3408976236 shutdown_connections 2023-10-16T20:05:44.238 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.233+0000 7f3dc006b640 1 -- 172.21.15.88:0/3408976236 wait complete. 2023-10-16T20:05:44.259 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T20:05:44.403 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.401+0000 7f4ac2609640 1 Processor -- start 2023-10-16T20:05:44.404 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.401+0000 7f4ac2609640 1 -- start start 2023-10-16T20:05:44.404 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.401+0000 7f4ac2609640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4abc0b4260 0x7f4abc15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:44.405 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.401+0000 7f4ac2609640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4abc06d3f0 0x7f4abc06d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:44.405 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.401+0000 7f4ac2609640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4abc15ad30 con 0x7f4abc0b0da0 2023-10-16T20:05:44.405 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.401+0000 7f4ac2609640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4abc06dd60 con 0x7f4abc06d3f0 2023-10-16T20:05:44.405 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.401+0000 7f4ac2609640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4abc0b4640 con 0x7f4abc0b4260 2023-10-16T20:05:44.406 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.401+0000 7f4ac0ba6640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4abc06d3f0 0x7f4abc06d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:44.406 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.401+0000 7f4abbfff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4abc0b4260 0x7f4abc15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:44.406 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.401+0000 7f4abb7fe640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f4abc0b0da0 0x7f4abc159cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:34656/0 (socket says 172.21.15.88:34656) 2023-10-16T20:05:44.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.401+0000 7f4abb7fe640 1 -- 172.21.15.88:0/3300851291 learned_addr learned my addr 172.21.15.88:0/3300851291 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:44.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.401+0000 7f4abaffd640 1 -- 172.21.15.88:0/3300851291 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3773472099 0 0) 0x7f4abc15ad30 con 0x7f4abc0b0da0 2023-10-16T20:05:44.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.401+0000 7f4abbfff640 1 -- 172.21.15.88:0/3300851291 >> v2:172.21.15.88:3300/0 conn(0x7f4abc06d3f0 msgr2=0x7f4abc06d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:44.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.401+0000 7f4abbfff640 1 --2- 172.21.15.88:0/3300851291 >> v2:172.21.15.88:3300/0 conn(0x7f4abc06d3f0 0x7f4abc06d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:44.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.401+0000 7f4abbfff640 1 -- 172.21.15.88:0/3300851291 >> v1:172.21.15.88:6789/0 conn(0x7f4abc0b0da0 legacy=0x7f4abc159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:44.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.401+0000 7f4abbfff640 1 -- 172.21.15.88:0/3300851291 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4abc15a770 con 0x7f4abc0b4260 2023-10-16T20:05:44.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.401+0000 7f4ac0ba6640 1 --2- 172.21.15.88:0/3300851291 >> v2:172.21.15.88:3300/0 conn(0x7f4abc06d3f0 0x7f4abc06d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:05:44.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.405+0000 7f4abbfff640 1 --2- 172.21.15.88:0/3300851291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4abc0b4260 0x7f4abc15dc90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4abc0b1a80 tx=0x7f4aac01c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1edb7955ba88762a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:44.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.405+0000 7f4abaffd640 1 -- 172.21.15.88:0/3300851291 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aac0ac600 con 0x7f4abc0b4260 2023-10-16T20:05:44.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.405+0000 7f4abaffd640 1 -- 172.21.15.88:0/3300851291 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aac0a8050 con 0x7f4abc0b4260 2023-10-16T20:05:44.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.405+0000 7f4abaffd640 1 -- 172.21.15.88:0/3300851291 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aac0ac980 con 0x7f4abc0b4260 2023-10-16T20:05:44.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.409+0000 7f4ac2609640 1 -- 172.21.15.88:0/3300851291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4abc0b4260 msgr2=0x7f4abc15dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:44.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.409+0000 7f4ac2609640 1 --2- 172.21.15.88:0/3300851291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4abc0b4260 0x7f4abc15dc90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4abc0b1a80 tx=0x7f4aac01c3a0 comp rx=0 tx=0).stop 2023-10-16T20:05:44.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.409+0000 7f4ac2609640 1 -- 172.21.15.88:0/3300851291 shutdown_connections 2023-10-16T20:05:44.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.409+0000 7f4ac2609640 1 --2- 172.21.15.88:0/3300851291 >> v2:172.21.15.88:3300/0 conn(0x7f4abc06d3f0 0x7f4abc06d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:44.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.409+0000 7f4ac2609640 1 --2- 172.21.15.88:0/3300851291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4abc0b4260 0x7f4abc15dc90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:44.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.409+0000 7f4ac2609640 1 -- 172.21.15.88:0/3300851291 >> 172.21.15.88:0/3300851291 conn(0x7f4abc0cb080 msgr2=0x7f4abc0cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:44.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.409+0000 7f4ac2609640 1 -- 172.21.15.88:0/3300851291 shutdown_connections 2023-10-16T20:05:44.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.409+0000 7f4ac2609640 1 -- 172.21.15.88:0/3300851291 wait complete. 2023-10-16T20:05:44.414 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.409+0000 7f4ac2609640 1 Processor -- start 2023-10-16T20:05:44.414 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.409+0000 7f4ac2609640 1 -- start start 2023-10-16T20:05:44.414 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.409+0000 7f4ac2609640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4abc06d3f0 0x7f4abc150b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:44.414 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.409+0000 7f4ac2609640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4abc0b0da0 0x7f4abc150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:44.415 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.409+0000 7f4ac2609640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4abc0b4260 0x7f4abc1613f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:44.415 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.409+0000 7f4ac2609640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc15c9d0 con 0x7f4abc06d3f0 2023-10-16T20:05:44.415 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.409+0000 7f4ac2609640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc15c410 con 0x7f4abc0b0da0 2023-10-16T20:05:44.415 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.409+0000 7f4ac2609640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc15c6f0 con 0x7f4abc0b4260 2023-10-16T20:05:44.415 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.409+0000 7f4abbfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4abc06d3f0 0x7f4abc150b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:44.416 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.409+0000 7f4abbfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4abc06d3f0 0x7f4abc150b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47978/0 (socket says 172.21.15.88:47978) 2023-10-16T20:05:44.416 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.409+0000 7f4abbfff640 1 -- 172.21.15.88:0/2023547737 learned_addr learned my addr 172.21.15.88:0/2023547737 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:44.416 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.409+0000 7f4abb7fe640 1 --2- 172.21.15.88:0/2023547737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4abc0b0da0 0x7f4abc150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:44.416 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.409+0000 7f4ac0ba6640 1 --2- 172.21.15.88:0/2023547737 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4abc0b4260 0x7f4abc1613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:44.416 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.413+0000 7f4abbfff640 1 -- 172.21.15.88:0/2023547737 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4abc0b4260 msgr2=0x7f4abc1613f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:44.416 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.413+0000 7f4abbfff640 1 --2- 172.21.15.88:0/2023547737 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4abc0b4260 0x7f4abc1613f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:44.417 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.413+0000 7f4abbfff640 1 -- 172.21.15.88:0/2023547737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4abc0b0da0 msgr2=0x7f4abc150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:44.417 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.413+0000 7f4abbfff640 1 --2- 172.21.15.88:0/2023547737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4abc0b0da0 0x7f4abc150330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:44.417 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.413+0000 7f4abbfff640 1 -- 172.21.15.88:0/2023547737 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab4009a70 con 0x7f4abc06d3f0 2023-10-16T20:05:44.417 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.413+0000 7f4abb7fe640 1 --2- 172.21.15.88:0/2023547737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4abc0b0da0 0x7f4abc150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:44.417 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.413+0000 7f4abbfff640 1 --2- 172.21.15.88:0/2023547737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4abc06d3f0 0x7f4abc150b60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4aac019fd0 tx=0x7f4aac01abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:44.419 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.413+0000 7f4ab8ff9640 1 -- 172.21.15.88:0/2023547737 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aac006e20 con 0x7f4abc06d3f0 2023-10-16T20:05:44.419 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.413+0000 7f4ab8ff9640 1 -- 172.21.15.88:0/2023547737 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aac0b7450 con 0x7f4abc06d3f0 2023-10-16T20:05:44.419 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.413+0000 7f4ac2609640 1 -- 172.21.15.88:0/2023547737 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aac009420 con 0x7f4abc06d3f0 2023-10-16T20:05:44.419 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.413+0000 7f4ab8ff9640 1 -- 172.21.15.88:0/2023547737 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aac0b77e0 con 0x7f4abc06d3f0 2023-10-16T20:05:44.420 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.413+0000 7f4ac2609640 1 -- 172.21.15.88:0/2023547737 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4abc150870 con 0x7f4abc06d3f0 2023-10-16T20:05:44.420 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.413+0000 7f4ac2609640 1 -- 172.21.15.88:0/2023547737 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4abc15ed60 con 0x7f4abc06d3f0 2023-10-16T20:05:44.420 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.413+0000 7f4ab8ff9640 1 -- 172.21.15.88:0/2023547737 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4aac0b3070 con 0x7f4abc06d3f0 2023-10-16T20:05:44.420 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.413+0000 7f4ab8ff9640 1 --2- 172.21.15.88:0/2023547737 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4aa80664a0 0x7f4aa8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:44.420 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.413+0000 7f4ab8ff9640 1 -- 172.21.15.88:0/2023547737 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4aac11e890 con 0x7f4abc06d3f0 2023-10-16T20:05:44.421 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.413+0000 7f4abb7fe640 1 --2- 172.21.15.88:0/2023547737 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4aa80664a0 0x7f4aa8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:44.421 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.417+0000 7f4abb7fe640 1 --2- 172.21.15.88:0/2023547737 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4aa80664a0 0x7f4aa8068960 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4abc0b3f60 tx=0x7f4ab0007c20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:44.421 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.417+0000 7f4ab8ff9640 1 -- 172.21.15.88:0/2023547737 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4abc15ed60 con 0x7f4abc06d3f0 2023-10-16T20:05:44.571 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.569+0000 7f4ac2609640 1 -- 172.21.15.88:0/2023547737 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f4abc15ed60 con 0x7f4abc06d3f0 2023-10-16T20:05:44.575 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.573+0000 7f4ab8ff9640 1 -- 172.21.15.88:0/2023547737 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4abc15ed60 con 0x7f4abc06d3f0 2023-10-16T20:05:44.578 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.573+0000 7f4ac2609640 1 -- 172.21.15.88:0/2023547737 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4aa80664a0 msgr2=0x7f4aa8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:44.578 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.573+0000 7f4ac2609640 1 --2- 172.21.15.88:0/2023547737 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4aa80664a0 0x7f4aa8068960 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4abc0b3f60 tx=0x7f4ab0007c20 comp rx=0 tx=0).stop 2023-10-16T20:05:44.578 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.573+0000 7f4ac2609640 1 -- 172.21.15.88:0/2023547737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4abc06d3f0 msgr2=0x7f4abc150b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:44.578 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.573+0000 7f4ac2609640 1 --2- 172.21.15.88:0/2023547737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4abc06d3f0 0x7f4abc150b60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4aac019fd0 tx=0x7f4aac01abe0 comp rx=0 tx=0).stop 2023-10-16T20:05:44.578 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.573+0000 7f4ac2609640 1 -- 172.21.15.88:0/2023547737 shutdown_connections 2023-10-16T20:05:44.579 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.573+0000 7f4ac2609640 1 --2- 172.21.15.88:0/2023547737 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4aa80664a0 0x7f4aa8068960 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:44.579 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.573+0000 7f4ac2609640 1 --2- 172.21.15.88:0/2023547737 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4abc0b4260 0x7f4abc1613f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:44.579 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.573+0000 7f4ac2609640 1 --2- 172.21.15.88:0/2023547737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4abc0b0da0 0x7f4abc150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:44.579 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.573+0000 7f4ac2609640 1 --2- 172.21.15.88:0/2023547737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4abc06d3f0 0x7f4abc150b60 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:44.579 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.573+0000 7f4ac2609640 1 -- 172.21.15.88:0/2023547737 >> 172.21.15.88:0/2023547737 conn(0x7f4abc0cb080 msgr2=0x7f4abc152cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:44.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.573+0000 7f4ac2609640 1 -- 172.21.15.88:0/2023547737 shutdown_connections 2023-10-16T20:05:44.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:44.573+0000 7f4ac2609640 1 -- 172.21.15.88:0/2023547737 wait complete. 2023-10-16T20:05:44.597 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T20:05:44.598 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T20:05:44.716 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T20:05:44.794 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T20:05:44.795 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:05:44.795 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:05:44.804 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:05:44.805 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:05:44.854 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:05:45.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.037+0000 7fb67a446640 1 Processor -- start 2023-10-16T20:05:45.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.037+0000 7fb67a446640 1 -- start start 2023-10-16T20:05:45.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.037+0000 7fb67a446640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb67415d9f0 0x7fb674161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:45.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.037+0000 7fb67a446640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb6740d26c0 0x7fb6740cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:45.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.037+0000 7fb67a446640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6740b87e0 con 0x7fb6740d22e0 2023-10-16T20:05:45.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.037+0000 7fb67a446640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb67415df10 con 0x7fb6740d26c0 2023-10-16T20:05:45.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.037+0000 7fb67a446640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb67415e1f0 con 0x7fb67415d9f0 2023-10-16T20:05:45.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.037+0000 7fb6789e3640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb6740d26c0 0x7fb6740cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:45.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.037+0000 7fb6737fe640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb67415d9f0 0x7fb674161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:45.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.037+0000 7fb6789e3640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb6740d26c0 0x7fb6740cf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:57270/0 (socket says 172.21.15.88:57270) 2023-10-16T20:05:45.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.037+0000 7fb673fff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fb6740d22e0 0x7fb67415d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:47498/0 (socket says 172.21.15.88:47498) 2023-10-16T20:05:45.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.037+0000 7fb6789e3640 1 -- 172.21.15.88:0/1290438811 learned_addr learned my addr 172.21.15.88:0/1290438811 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:45.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.037+0000 7fb6789e3640 1 -- 172.21.15.88:0/1290438811 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb67415d9f0 msgr2=0x7fb674161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:45.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.037+0000 7fb6789e3640 1 --2- 172.21.15.88:0/1290438811 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb67415d9f0 0x7fb674161450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:45.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.037+0000 7fb6789e3640 1 -- 172.21.15.88:0/1290438811 >> v1:172.21.15.94:6789/0 conn(0x7fb6740d22e0 legacy=0x7fb67415d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:45.044 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.037+0000 7fb6789e3640 1 -- 172.21.15.88:0/1290438811 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb674160970 con 0x7fb6740d26c0 2023-10-16T20:05:45.044 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.041+0000 7fb6789e3640 1 --2- 172.21.15.88:0/1290438811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6740d26c0 0x7fb6740cf2e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb66c009e50 tx=0x7fb66c01c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3214376652322dcc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:45.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.041+0000 7fb672ffd640 1 -- 172.21.15.88:0/1290438811 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66c0a5810 con 0x7fb6740d26c0 2023-10-16T20:05:45.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.041+0000 7fb672ffd640 1 -- 172.21.15.88:0/1290438811 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66c0a5970 con 0x7fb6740d26c0 2023-10-16T20:05:45.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.041+0000 7fb672ffd640 1 -- 172.21.15.88:0/1290438811 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66c0a5d00 con 0x7fb6740d26c0 2023-10-16T20:05:45.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.041+0000 7fb67a446640 1 -- 172.21.15.88:0/1290438811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6740d26c0 msgr2=0x7fb6740cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:45.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.041+0000 7fb67a446640 1 --2- 172.21.15.88:0/1290438811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6740d26c0 0x7fb6740cf2e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb66c009e50 tx=0x7fb66c01c950 comp rx=0 tx=0).stop 2023-10-16T20:05:45.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.041+0000 7fb67a446640 1 -- 172.21.15.88:0/1290438811 shutdown_connections 2023-10-16T20:05:45.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.041+0000 7fb67a446640 1 --2- 172.21.15.88:0/1290438811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6740d26c0 0x7fb6740cf2e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:45.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.041+0000 7fb67a446640 1 --2- 172.21.15.88:0/1290438811 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb67415d9f0 0x7fb674161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:45.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.041+0000 7fb67a446640 1 -- 172.21.15.88:0/1290438811 >> 172.21.15.88:0/1290438811 conn(0x7fb6740715c0 msgr2=0x7fb6740719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:45.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.041+0000 7fb67a446640 1 -- 172.21.15.88:0/1290438811 shutdown_connections 2023-10-16T20:05:45.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.041+0000 7fb67a446640 1 -- 172.21.15.88:0/1290438811 wait complete. 2023-10-16T20:05:45.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.045+0000 7fb67a446640 1 Processor -- start 2023-10-16T20:05:45.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.045+0000 7fb67a446640 1 -- start start 2023-10-16T20:05:45.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.045+0000 7fb67a446640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6740d22e0 0x7fb67414a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:45.050 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.045+0000 7fb67a446640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6740d26c0 0x7fb67414e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:45.050 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.045+0000 7fb67a446640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb67415d9f0 0x7fb6740b7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:45.050 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.045+0000 7fb67a446640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67414eed0 con 0x7fb6740d26c0 2023-10-16T20:05:45.050 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.045+0000 7fb67a446640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67414f1b0 con 0x7fb6740d22e0 2023-10-16T20:05:45.051 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.045+0000 7fb67a446640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb67414f490 con 0x7fb67415d9f0 2023-10-16T20:05:45.051 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.045+0000 7fb6737fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6740d26c0 0x7fb67414e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:45.051 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.045+0000 7fb673fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6740d22e0 0x7fb67414a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:45.051 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.045+0000 7fb6737fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6740d26c0 0x7fb67414e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47984/0 (socket says 172.21.15.88:47984) 2023-10-16T20:05:45.051 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.045+0000 7fb6737fe640 1 -- 172.21.15.88:0/101833566 learned_addr learned my addr 172.21.15.88:0/101833566 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:45.052 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.045+0000 7fb6789e3640 1 --2- 172.21.15.88:0/101833566 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb67415d9f0 0x7fb6740b7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:45.052 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.045+0000 7fb6737fe640 1 -- 172.21.15.88:0/101833566 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb67415d9f0 msgr2=0x7fb6740b7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:45.052 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.045+0000 7fb6737fe640 1 --2- 172.21.15.88:0/101833566 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb67415d9f0 0x7fb6740b7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:45.052 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.045+0000 7fb6737fe640 1 -- 172.21.15.88:0/101833566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6740d22e0 msgr2=0x7fb67414a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:45.052 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.045+0000 7fb6737fe640 1 --2- 172.21.15.88:0/101833566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6740d22e0 0x7fb67414a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:45.053 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.045+0000 7fb6737fe640 1 -- 172.21.15.88:0/101833566 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb66c009b30 con 0x7fb6740d26c0 2023-10-16T20:05:45.054 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.045+0000 7fb6737fe640 1 --2- 172.21.15.88:0/101833566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6740d26c0 0x7fb67414e610 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb66800e840 tx=0x7fb66800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:45.054 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.049+0000 7fb670ff9640 1 -- 172.21.15.88:0/101833566 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb668009510 con 0x7fb6740d26c0 2023-10-16T20:05:45.055 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.049+0000 7fb670ff9640 1 -- 172.21.15.88:0/101833566 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66800ee90 con 0x7fb6740d26c0 2023-10-16T20:05:45.055 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.049+0000 7fb67a446640 1 -- 172.21.15.88:0/101833566 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb67414f770 con 0x7fb6740d26c0 2023-10-16T20:05:45.055 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.049+0000 7fb670ff9640 1 -- 172.21.15.88:0/101833566 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66800cb80 con 0x7fb6740d26c0 2023-10-16T20:05:45.055 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.049+0000 7fb67a446640 1 -- 172.21.15.88:0/101833566 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb67414eb50 con 0x7fb6740d26c0 2023-10-16T20:05:45.056 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.049+0000 7fb67a446640 1 -- 172.21.15.88:0/101833566 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb67404e9c0 con 0x7fb6740d26c0 2023-10-16T20:05:45.056 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.049+0000 7fb670ff9640 1 -- 172.21.15.88:0/101833566 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb66800cce0 con 0x7fb6740d26c0 2023-10-16T20:05:45.056 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.049+0000 7fb670ff9640 1 --2- 172.21.15.88:0/101833566 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb6600664a0 0x7fb660068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:45.056 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.049+0000 7fb670ff9640 1 -- 172.21.15.88:0/101833566 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb668084350 con 0x7fb6740d26c0 2023-10-16T20:05:45.056 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.049+0000 7fb673fff640 1 --2- 172.21.15.88:0/101833566 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb6600664a0 0x7fb660068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:45.057 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.049+0000 7fb673fff640 1 --2- 172.21.15.88:0/101833566 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb6600664a0 0x7fb660068960 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb664000f70 tx=0x7fb664007e10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:45.057 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.053+0000 7fb670ff9640 1 -- 172.21.15.88:0/101833566 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb668051420 con 0x7fb6740d26c0 2023-10-16T20:05:45.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.205+0000 7fb67a446640 1 -- 172.21.15.88:0/101833566 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6740713a0 con 0x7fb6740d26c0 2023-10-16T20:05:45.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.209+0000 7fb670ff9640 1 -- 172.21.15.88:0/101833566 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18661 (secure 0 0 0) 0x7fb668054a40 con 0x7fb6740d26c0 2023-10-16T20:05:45.211 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:45.211 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":29,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:05:38.202959+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T20:04:54.442487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T20:04:55.766641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T20:04:59.475830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":1}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:05:45.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.209+0000 7fb67a446640 1 -- 172.21.15.88:0/101833566 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb6600664a0 msgr2=0x7fb660068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:45.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.209+0000 7fb67a446640 1 --2- 172.21.15.88:0/101833566 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb6600664a0 0x7fb660068960 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb664000f70 tx=0x7fb664007e10 comp rx=0 tx=0).stop 2023-10-16T20:05:45.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.209+0000 7fb67a446640 1 -- 172.21.15.88:0/101833566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6740d26c0 msgr2=0x7fb67414e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:45.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.209+0000 7fb67a446640 1 --2- 172.21.15.88:0/101833566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6740d26c0 0x7fb67414e610 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb66800e840 tx=0x7fb66800ec40 comp rx=0 tx=0).stop 2023-10-16T20:05:45.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.209+0000 7fb67a446640 1 -- 172.21.15.88:0/101833566 shutdown_connections 2023-10-16T20:05:45.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.209+0000 7fb67a446640 1 --2- 172.21.15.88:0/101833566 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb6600664a0 0x7fb660068960 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:45.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.209+0000 7fb67a446640 1 --2- 172.21.15.88:0/101833566 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb67415d9f0 0x7fb6740b7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:45.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.209+0000 7fb67a446640 1 --2- 172.21.15.88:0/101833566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6740d26c0 0x7fb67414e610 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:45.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.209+0000 7fb67a446640 1 --2- 172.21.15.88:0/101833566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6740d22e0 0x7fb67414a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:45.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.209+0000 7fb67a446640 1 -- 172.21.15.88:0/101833566 >> 172.21.15.88:0/101833566 conn(0x7fb6740715c0 msgr2=0x7fb6740719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:45.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.209+0000 7fb67a446640 1 -- 172.21.15.88:0/101833566 shutdown_connections 2023-10-16T20:05:45.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.209+0000 7fb67a446640 1 -- 172.21.15.88:0/101833566 wait complete. 2023-10-16T20:05:45.232 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T20:04:54.442487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T20:04:55.766641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T20:04:59.475830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T20:05:45.233 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:05:45.373 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.369+0000 7f7679e9a640 1 Processor -- start 2023-10-16T20:05:45.374 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.369+0000 7f7679e9a640 1 -- start start 2023-10-16T20:05:45.375 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f7679e9a640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f767406d3f0 0x7f767406d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:45.375 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f7679e9a640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f767415ad30 con 0x7f76740b0da0 2023-10-16T20:05:45.375 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f7679e9a640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76740b4640 con 0x7f76740b4260 2023-10-16T20:05:45.375 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f7679e9a640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f767406dda0 con 0x7f767406d3f0 2023-10-16T20:05:45.376 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f766bfff640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f76740b0da0 0x7f7674159cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:34660/0 (socket says 172.21.15.88:34660) 2023-10-16T20:05:45.376 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f766bfff640 1 -- 172.21.15.88:0/1864531606 learned_addr learned my addr 172.21.15.88:0/1864531606 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:45.376 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f7673fff640 1 --2- 172.21.15.88:0/1864531606 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f767406d3f0 0x7f767406d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:45.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f7672ffd640 1 -- 172.21.15.88:0/1864531606 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 646476136 0 0) 0x7f767415ad30 con 0x7f76740b0da0 2023-10-16T20:05:45.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f7672ffd640 1 -- 172.21.15.88:0/1864531606 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7660005740 con 0x7f76740b0da0 2023-10-16T20:05:45.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f7672ffd640 1 -- 172.21.15.88:0/1864531606 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3576210608 0 0) 0x7f76740b4640 con 0x7f76740b4260 2023-10-16T20:05:45.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f7672ffd640 1 -- 172.21.15.88:0/1864531606 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f767415ad30 con 0x7f76740b4260 2023-10-16T20:05:45.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f7672ffd640 1 -- 172.21.15.88:0/1864531606 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1108837732 0 0) 0x7f7660005740 con 0x7f76740b0da0 2023-10-16T20:05:45.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f7672ffd640 1 -- 172.21.15.88:0/1864531606 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f767406d3f0 msgr2=0x7f767406d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:45.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f7672ffd640 1 --2- 172.21.15.88:0/1864531606 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f767406d3f0 0x7f767406d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:45.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f7672ffd640 1 -- 172.21.15.88:0/1864531606 >> v1:172.21.15.94:6789/0 conn(0x7f76740b4260 legacy=0x7f767415dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:45.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f7672ffd640 1 -- 172.21.15.88:0/1864531606 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767415a770 con 0x7f76740b0da0 2023-10-16T20:05:45.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f7672ffd640 1 -- 172.21.15.88:0/1864531606 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f765c001680 con 0x7f76740b0da0 2023-10-16T20:05:45.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f7672ffd640 1 -- 172.21.15.88:0/1864531606 >> v1:172.21.15.88:6789/0 conn(0x7f76740b0da0 legacy=0x7f7674159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:45.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f7672ffd640 1 --2- 172.21.15.88:0/1864531606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7660005d50 0x7f7660006150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:45.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f7672ffd640 1 --2- 172.21.15.88:0/1864531606 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76600069e0 0x7f7660006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:45.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f7672ffd640 1 --2- 172.21.15.88:0/1864531606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76600073a0 0x7f766000dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:45.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f7672ffd640 1 -- 172.21.15.88:0/1864531606 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767406dda0 con 0x7f76600073a0 2023-10-16T20:05:45.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f7672ffd640 1 -- 172.21.15.88:0/1864531606 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766000e210 con 0x7f7660005d50 2023-10-16T20:05:45.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f7672ffd640 1 -- 172.21.15.88:0/1864531606 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f766000e3d0 con 0x7f76600069e0 2023-10-16T20:05:45.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f7673fff640 1 --2- 172.21.15.88:0/1864531606 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f767406d3f0 0x7f767406d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:05:45.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f76737fe640 1 --2- 172.21.15.88:0/1864531606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7660005d50 0x7f7660006150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:45.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f7673fff640 1 --2- 172.21.15.88:0/1864531606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76600073a0 0x7f766000dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:45.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f766bfff640 1 --2- 172.21.15.88:0/1864531606 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76600069e0 0x7f7660006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:45.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f7673fff640 1 -- 172.21.15.88:0/1864531606 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76600069e0 msgr2=0x7f7660006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:45.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f7673fff640 1 --2- 172.21.15.88:0/1864531606 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f76600069e0 0x7f7660006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:45.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f7673fff640 1 -- 172.21.15.88:0/1864531606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7660005d50 msgr2=0x7f7660006150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:45.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f7673fff640 1 --2- 172.21.15.88:0/1864531606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7660005d50 0x7f7660006150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:45.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f7673fff640 1 -- 172.21.15.88:0/1864531606 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7660005740 con 0x7f76600073a0 2023-10-16T20:05:45.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.373+0000 7f76737fe640 1 --2- 172.21.15.88:0/1864531606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7660005d50 0x7f7660006150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:05:45.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.377+0000 7f7673fff640 1 --2- 172.21.15.88:0/1864531606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76600073a0 0x7f766000dba0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f767406d7d0 tx=0x7f766c01ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7b9f736a569be4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:45.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.377+0000 7f7672ffd640 1 -- 172.21.15.88:0/1864531606 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766c0a5d20 con 0x7f76600073a0 2023-10-16T20:05:45.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.377+0000 7f7672ffd640 1 -- 172.21.15.88:0/1864531606 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766c00f040 con 0x7f76600073a0 2023-10-16T20:05:45.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.377+0000 7f7672ffd640 1 -- 172.21.15.88:0/1864531606 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766c003bc0 con 0x7f76600073a0 2023-10-16T20:05:45.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.377+0000 7f7679e9a640 1 -- 172.21.15.88:0/1864531606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76600073a0 msgr2=0x7f766000dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:45.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.377+0000 7f7679e9a640 1 --2- 172.21.15.88:0/1864531606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76600073a0 0x7f766000dba0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f767406d7d0 tx=0x7f766c01ccd0 comp rx=0 tx=0).stop 2023-10-16T20:05:45.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.377+0000 7f7673fff640 1 -- 172.21.15.88:0/1864531606 reap_dead start 2023-10-16T20:05:45.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.377+0000 7f7679e9a640 1 -- 172.21.15.88:0/1864531606 shutdown_connections 2023-10-16T20:05:45.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.377+0000 7f7679e9a640 1 -- 172.21.15.88:0/1864531606 >> 172.21.15.88:0/1864531606 conn(0x7f76740cb080 msgr2=0x7f76740cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:45.388 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.377+0000 7f7679e9a640 1 -- 172.21.15.88:0/1864531606 shutdown_connections 2023-10-16T20:05:45.388 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.377+0000 7f7679e9a640 1 -- 172.21.15.88:0/1864531606 wait complete. 2023-10-16T20:05:45.388 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.381+0000 7f7679e9a640 1 Processor -- start 2023-10-16T20:05:45.388 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.381+0000 7f7679e9a640 1 -- start start 2023-10-16T20:05:45.388 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.381+0000 7f7679e9a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76600073a0 0x7f7674156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:45.389 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.381+0000 7f7679e9a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76741429e0 0x7f7674151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:45.389 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.381+0000 7f7679e9a640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f767414e2d0 0x7f76740afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:45.389 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.381+0000 7f7679e9a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7674152600 con 0x7f76600073a0 2023-10-16T20:05:45.389 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.381+0000 7f7679e9a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76741528e0 con 0x7f76741429e0 2023-10-16T20:05:45.389 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.381+0000 7f7679e9a640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7674152bc0 con 0x7f767414e2d0 2023-10-16T20:05:45.390 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.381+0000 7f76737fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76600073a0 0x7f7674156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:45.390 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.381+0000 7f76737fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76600073a0 0x7f7674156c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48004/0 (socket says 172.21.15.88:48004) 2023-10-16T20:05:45.390 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.381+0000 7f76737fe640 1 -- 172.21.15.88:0/143719277 learned_addr learned my addr 172.21.15.88:0/143719277 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:45.390 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.381+0000 7f766bfff640 1 --2- 172.21.15.88:0/143719277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76741429e0 0x7f7674151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:45.390 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.381+0000 7f7673fff640 1 --2- 172.21.15.88:0/143719277 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f767414e2d0 0x7f76740afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:45.390 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.381+0000 7f76737fe640 1 -- 172.21.15.88:0/143719277 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f767414e2d0 msgr2=0x7f76740afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:45.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.381+0000 7f76737fe640 1 --2- 172.21.15.88:0/143719277 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f767414e2d0 0x7f76740afaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:45.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.381+0000 7f76737fe640 1 -- 172.21.15.88:0/143719277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76741429e0 msgr2=0x7f7674151cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:45.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.381+0000 7f76737fe640 1 --2- 172.21.15.88:0/143719277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76741429e0 0x7f7674151cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:45.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.381+0000 7f76737fe640 1 -- 172.21.15.88:0/143719277 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7674152ea0 con 0x7f76600073a0 2023-10-16T20:05:45.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.381+0000 7f766bfff640 1 --2- 172.21.15.88:0/143719277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76741429e0 0x7f7674151cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:45.392 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.381+0000 7f76737fe640 1 --2- 172.21.15.88:0/143719277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76600073a0 0x7f7674156c70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7664012800 tx=0x7f7664012cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:45.392 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.381+0000 7f7670ff9640 1 -- 172.21.15.88:0/143719277 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7664010bd0 con 0x7f76600073a0 2023-10-16T20:05:45.392 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.381+0000 7f7670ff9640 1 -- 172.21.15.88:0/143719277 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7664010d30 con 0x7f76600073a0 2023-10-16T20:05:45.392 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.381+0000 7f7670ff9640 1 -- 172.21.15.88:0/143719277 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7664010ea0 con 0x7f76600073a0 2023-10-16T20:05:45.392 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.381+0000 7f7679e9a640 1 -- 172.21.15.88:0/143719277 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7674153180 con 0x7f76600073a0 2023-10-16T20:05:45.393 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.381+0000 7f7679e9a640 1 -- 172.21.15.88:0/143719277 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7674152210 con 0x7f76600073a0 2023-10-16T20:05:45.393 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.381+0000 7f7679e9a640 1 -- 172.21.15.88:0/143719277 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f767404e9c0 con 0x7f76600073a0 2023-10-16T20:05:45.394 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.385+0000 7f7670ff9640 1 -- 172.21.15.88:0/143719277 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7664016680 con 0x7f76600073a0 2023-10-16T20:05:45.394 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.385+0000 7f7670ff9640 1 --2- 172.21.15.88:0/143719277 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f76540664d0 0x7f7654068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:45.394 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.385+0000 7f7670ff9640 1 -- 172.21.15.88:0/143719277 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7664087b30 con 0x7f76600073a0 2023-10-16T20:05:45.394 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.385+0000 7f766bfff640 1 --2- 172.21.15.88:0/143719277 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f76540664d0 0x7f7654068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:45.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.385+0000 7f766bfff640 1 --2- 172.21.15.88:0/143719277 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f76540664d0 0x7f7654068990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7674158270 tx=0x7f765c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:45.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.389+0000 7f7670ff9640 1 -- 172.21.15.88:0/143719277 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7664054b50 con 0x7f76600073a0 2023-10-16T20:05:45.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.541+0000 7f7679e9a640 1 -- 172.21.15.88:0/143719277 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f76740ae590 con 0x7f76600073a0 2023-10-16T20:05:45.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.545+0000 7f7670ff9640 1 -- 172.21.15.88:0/143719277 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7664058170 con 0x7f76600073a0 2023-10-16T20:05:45.546 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:05:45.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.545+0000 7f7679e9a640 1 -- 172.21.15.88:0/143719277 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f76540664d0 msgr2=0x7f7654068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:45.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.545+0000 7f7679e9a640 1 --2- 172.21.15.88:0/143719277 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f76540664d0 0x7f7654068990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7674158270 tx=0x7f765c009040 comp rx=0 tx=0).stop 2023-10-16T20:05:45.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.545+0000 7f7679e9a640 1 -- 172.21.15.88:0/143719277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76600073a0 msgr2=0x7f7674156c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:45.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.545+0000 7f7679e9a640 1 --2- 172.21.15.88:0/143719277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76600073a0 0x7f7674156c70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7664012800 tx=0x7f7664012cc0 comp rx=0 tx=0).stop 2023-10-16T20:05:45.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.545+0000 7f7679e9a640 1 -- 172.21.15.88:0/143719277 shutdown_connections 2023-10-16T20:05:45.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.545+0000 7f7679e9a640 1 --2- 172.21.15.88:0/143719277 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f76540664d0 0x7f7654068990 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:45.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.545+0000 7f7679e9a640 1 --2- 172.21.15.88:0/143719277 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f767414e2d0 0x7f76740afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:45.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.545+0000 7f7679e9a640 1 --2- 172.21.15.88:0/143719277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76741429e0 0x7f7674151cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:45.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.545+0000 7f7679e9a640 1 --2- 172.21.15.88:0/143719277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76600073a0 0x7f7674156c70 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:45.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.545+0000 7f7679e9a640 1 -- 172.21.15.88:0/143719277 >> 172.21.15.88:0/143719277 conn(0x7f76740cb080 msgr2=0x7f76740b2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:45.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.545+0000 7f7679e9a640 1 -- 172.21.15.88:0/143719277 shutdown_connections 2023-10-16T20:05:45.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.545+0000 7f7679e9a640 1 -- 172.21.15.88:0/143719277 wait complete. 2023-10-16T20:05:45.567 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T20:05:45.707 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.705+0000 7f4bbce32640 1 Processor -- start 2023-10-16T20:05:45.707 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.705+0000 7f4bbce32640 1 -- start start 2023-10-16T20:05:45.708 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.705+0000 7f4bbce32640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4bb80b4260 0x7f4bb815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:45.708 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.705+0000 7f4bbce32640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bb815ad30 con 0x7f4bb806d3f0 2023-10-16T20:05:45.709 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.705+0000 7f4bbce32640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bb80b4640 con 0x7f4bb80b0da0 2023-10-16T20:05:45.709 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.705+0000 7f4bbce32640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bb815a770 con 0x7f4bb80b4260 2023-10-16T20:05:45.709 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.705+0000 7f4bb659c640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4bb80b4260 0x7f4bb815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:45.709 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.705+0000 7f4bb6d9d640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f4bb806d3f0 0x7f4bb80ae590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:34666/0 (socket says 172.21.15.88:34666) 2023-10-16T20:05:45.709 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.705+0000 7f4bb6d9d640 1 -- 172.21.15.88:0/1827471771 learned_addr learned my addr 172.21.15.88:0/1827471771 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:45.710 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.705+0000 7f4bb659c640 1 -- 172.21.15.88:0/1827471771 >> v1:172.21.15.94:6789/0 conn(0x7f4bb80b0da0 legacy=0x7f4bb8159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:45.710 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.705+0000 7f4bb659c640 1 -- 172.21.15.88:0/1827471771 >> v1:172.21.15.88:6789/0 conn(0x7f4bb806d3f0 legacy=0x7f4bb80ae590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:45.711 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.705+0000 7f4bb5d9b640 1 -- 172.21.15.88:0/1827471771 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 340078570 0 0) 0x7f4bb815ad30 con 0x7f4bb806d3f0 2023-10-16T20:05:45.711 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.705+0000 7f4bb659c640 1 -- 172.21.15.88:0/1827471771 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb815aa50 con 0x7f4bb80b4260 2023-10-16T20:05:45.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.709+0000 7f4bb659c640 1 --2- 172.21.15.88:0/1827471771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bb80b4260 0x7f4bb815dc90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80097c0 tx=0x7f4ba801c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=183f79bf7da68a61 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:45.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.709+0000 7f4bb5d9b640 1 -- 172.21.15.88:0/1827471771 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba80ab600 con 0x7f4bb80b4260 2023-10-16T20:05:45.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.709+0000 7f4bb5d9b640 1 -- 172.21.15.88:0/1827471771 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba80a7050 con 0x7f4bb80b4260 2023-10-16T20:05:45.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.709+0000 7f4bb5d9b640 1 -- 172.21.15.88:0/1827471771 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba80ab980 con 0x7f4bb80b4260 2023-10-16T20:05:45.713 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.709+0000 7f4bbce32640 1 -- 172.21.15.88:0/1827471771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bb80b4260 msgr2=0x7f4bb815dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:45.714 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.709+0000 7f4bbce32640 1 --2- 172.21.15.88:0/1827471771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bb80b4260 0x7f4bb815dc90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80097c0 tx=0x7f4ba801c380 comp rx=0 tx=0).stop 2023-10-16T20:05:45.714 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.709+0000 7f4bbce32640 1 -- 172.21.15.88:0/1827471771 shutdown_connections 2023-10-16T20:05:45.714 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.709+0000 7f4bbce32640 1 --2- 172.21.15.88:0/1827471771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bb80b4260 0x7f4bb815dc90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:45.714 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.709+0000 7f4bbce32640 1 -- 172.21.15.88:0/1827471771 >> 172.21.15.88:0/1827471771 conn(0x7f4bb80cb080 msgr2=0x7f4bb80cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:45.714 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.709+0000 7f4bbce32640 1 -- 172.21.15.88:0/1827471771 shutdown_connections 2023-10-16T20:05:45.715 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.709+0000 7f4bbce32640 1 -- 172.21.15.88:0/1827471771 wait complete. 2023-10-16T20:05:45.717 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.713+0000 7f4bbce32640 1 Processor -- start 2023-10-16T20:05:45.717 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.713+0000 7f4bbce32640 1 -- start start 2023-10-16T20:05:45.717 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.713+0000 7f4bbce32640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bb806d3f0 0x7f4bb814f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:45.717 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.713+0000 7f4bbce32640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4bb80b0da0 0x7f4bb814ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:45.718 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.713+0000 7f4bbce32640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bb80b4260 0x7f4bb80b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:45.718 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.713+0000 7f4bbce32640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb814f5f0 con 0x7f4bb80b4260 2023-10-16T20:05:45.718 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.713+0000 7f4bbce32640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb814b1d0 con 0x7f4bb806d3f0 2023-10-16T20:05:45.718 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.713+0000 7f4bbce32640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb815b430 con 0x7f4bb80b0da0 2023-10-16T20:05:45.719 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.713+0000 7f4bb6d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bb80b4260 0x7f4bb80b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:45.719 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.713+0000 7f4bb6d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bb80b4260 0x7f4bb80b28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48026/0 (socket says 172.21.15.88:48026) 2023-10-16T20:05:45.719 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.713+0000 7f4bb6d9d640 1 -- 172.21.15.88:0/4283592391 learned_addr learned my addr 172.21.15.88:0/4283592391 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:45.719 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.713+0000 7f4bb659c640 1 --2- 172.21.15.88:0/4283592391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bb806d3f0 0x7f4bb814f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:45.719 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.713+0000 7f4badd9b640 1 --2- 172.21.15.88:0/4283592391 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4bb80b0da0 0x7f4bb814ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:45.720 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.713+0000 7f4bb6d9d640 1 -- 172.21.15.88:0/4283592391 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4bb80b0da0 msgr2=0x7f4bb814ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:45.720 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.713+0000 7f4bb6d9d640 1 --2- 172.21.15.88:0/4283592391 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4bb80b0da0 0x7f4bb814ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:45.720 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.713+0000 7f4bb6d9d640 1 -- 172.21.15.88:0/4283592391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bb806d3f0 msgr2=0x7f4bb814f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:45.720 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.713+0000 7f4bb6d9d640 1 --2- 172.21.15.88:0/4283592391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bb806d3f0 0x7f4bb814f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:45.720 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.713+0000 7f4bb6d9d640 1 -- 172.21.15.88:0/4283592391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba8009420 con 0x7f4bb80b4260 2023-10-16T20:05:45.721 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.713+0000 7f4bb6d9d640 1 --2- 172.21.15.88:0/4283592391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bb80b4260 0x7f4bb80b28a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000c810 tx=0x7f4bb000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:45.721 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.713+0000 7f4baf7fe640 1 -- 172.21.15.88:0/4283592391 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb000a640 con 0x7f4bb80b4260 2023-10-16T20:05:45.721 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.713+0000 7f4baf7fe640 1 -- 172.21.15.88:0/4283592391 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb000a7c0 con 0x7f4bb80b4260 2023-10-16T20:05:45.721 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.713+0000 7f4bbce32640 1 -- 172.21.15.88:0/4283592391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bb815a540 con 0x7f4bb80b4260 2023-10-16T20:05:45.722 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.713+0000 7f4baf7fe640 1 -- 172.21.15.88:0/4283592391 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb000ab70 con 0x7f4bb80b4260 2023-10-16T20:05:45.722 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.713+0000 7f4bbce32640 1 -- 172.21.15.88:0/4283592391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bb815b9f0 con 0x7f4bb80b4260 2023-10-16T20:05:45.723 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.717+0000 7f4bbce32640 1 -- 172.21.15.88:0/4283592391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bb8154440 con 0x7f4bb80b4260 2023-10-16T20:05:45.723 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.717+0000 7f4baf7fe640 1 -- 172.21.15.88:0/4283592391 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4bb000acd0 con 0x7f4bb80b4260 2023-10-16T20:05:45.723 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.717+0000 7f4baf7fe640 1 --2- 172.21.15.88:0/4283592391 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4b980664a0 0x7f4b98068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:45.723 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.717+0000 7f4baf7fe640 1 -- 172.21.15.88:0/4283592391 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4bb0083a50 con 0x7f4bb80b4260 2023-10-16T20:05:45.724 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.717+0000 7f4bb659c640 1 --2- 172.21.15.88:0/4283592391 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4b980664a0 0x7f4b98068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:45.724 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.717+0000 7f4bb659c640 1 --2- 172.21.15.88:0/4283592391 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4b980664a0 0x7f4b98068960 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80ad000 tx=0x7f4ba80a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:45.724 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.721+0000 7f4baf7fe640 1 -- 172.21.15.88:0/4283592391 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4bb0050bc0 con 0x7f4bb80b4260 2023-10-16T20:05:45.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.869+0000 7f4bbce32640 1 -- 172.21.15.88:0/4283592391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4bb804e9c0 con 0x7f4bb80b4260 2023-10-16T20:05:45.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.869+0000 7f4baf7fe640 1 -- 172.21.15.88:0/4283592391 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4bb0018020 con 0x7f4bb80b4260 2023-10-16T20:05:45.873 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:05:45.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.873+0000 7f4bbce32640 1 -- 172.21.15.88:0/4283592391 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4b980664a0 msgr2=0x7f4b98068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:45.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.873+0000 7f4bbce32640 1 --2- 172.21.15.88:0/4283592391 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4b980664a0 0x7f4b98068960 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80ad000 tx=0x7f4ba80a6000 comp rx=0 tx=0).stop 2023-10-16T20:05:45.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.873+0000 7f4bbce32640 1 -- 172.21.15.88:0/4283592391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bb80b4260 msgr2=0x7f4bb80b28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:45.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.873+0000 7f4bbce32640 1 --2- 172.21.15.88:0/4283592391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bb80b4260 0x7f4bb80b28a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000c810 tx=0x7f4bb000ccd0 comp rx=0 tx=0).stop 2023-10-16T20:05:45.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.873+0000 7f4bbce32640 1 -- 172.21.15.88:0/4283592391 shutdown_connections 2023-10-16T20:05:45.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.873+0000 7f4bbce32640 1 --2- 172.21.15.88:0/4283592391 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4b980664a0 0x7f4b98068960 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:45.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.873+0000 7f4bbce32640 1 --2- 172.21.15.88:0/4283592391 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4bb80b0da0 0x7f4bb814ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:45.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.873+0000 7f4bbce32640 1 --2- 172.21.15.88:0/4283592391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bb80b4260 0x7f4bb80b28a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:45.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.873+0000 7f4bbce32640 1 --2- 172.21.15.88:0/4283592391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bb806d3f0 0x7f4bb814f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:45.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.873+0000 7f4bbce32640 1 -- 172.21.15.88:0/4283592391 >> 172.21.15.88:0/4283592391 conn(0x7f4bb80cb080 msgr2=0x7f4bb80ae590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:45.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.873+0000 7f4bbce32640 1 -- 172.21.15.88:0/4283592391 shutdown_connections 2023-10-16T20:05:45.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:45.873+0000 7f4bbce32640 1 -- 172.21.15.88:0/4283592391 wait complete. 2023-10-16T20:05:45.894 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T20:05:46.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.029+0000 7f3ba9565640 1 Processor -- start 2023-10-16T20:05:46.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.029+0000 7f3ba9565640 1 -- start start 2023-10-16T20:05:46.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.033+0000 7f3ba9565640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f3ba415c960 0x7f3ba40cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:46.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.033+0000 7f3ba9565640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3ba40ce450 0x7f3ba40cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:46.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.033+0000 7f3ba9565640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba40cfc20 con 0x7f3ba40cf7d0 2023-10-16T20:05:46.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.033+0000 7f3ba9565640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba40d05f0 con 0x7f3ba415c960 2023-10-16T20:05:46.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.033+0000 7f3ba9565640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba40d08d0 con 0x7f3ba40ce450 2023-10-16T20:05:46.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.033+0000 7f3ba2ffd640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f3ba415c960 0x7f3ba40cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:46.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.033+0000 7f3ba2ffd640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f3ba415c960 0x7f3ba40cdf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48030/0 (socket says 172.21.15.88:48030) 2023-10-16T20:05:46.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.033+0000 7f3ba2ffd640 1 -- 172.21.15.88:0/1523910186 learned_addr learned my addr 172.21.15.88:0/1523910186 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:46.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.033+0000 7f3ba2ffd640 1 -- 172.21.15.88:0/1523910186 >> v2:172.21.15.94:3300/0 conn(0x7f3ba40ce450 msgr2=0x7f3ba40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:46.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.033+0000 7f3b9a7fc640 1 --2- 172.21.15.88:0/1523910186 >> v2:172.21.15.94:3300/0 conn(0x7f3ba40ce450 0x7f3ba40cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:46.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.033+0000 7f3ba2ffd640 1 --2- 172.21.15.88:0/1523910186 >> v2:172.21.15.94:3300/0 conn(0x7f3ba40ce450 0x7f3ba40cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:46.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.033+0000 7f3ba2ffd640 1 -- 172.21.15.88:0/1523910186 >> v1:172.21.15.94:6789/0 conn(0x7f3ba40cf7d0 legacy=0x7f3ba4160550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:46.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.033+0000 7f3ba2ffd640 1 -- 172.21.15.88:0/1523910186 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba40d0bb0 con 0x7f3ba415c960 2023-10-16T20:05:46.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.033+0000 7f3b9a7fc640 1 --2- 172.21.15.88:0/1523910186 >> v2:172.21.15.94:3300/0 conn(0x7f3ba40ce450 0x7f3ba40cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:46.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.037+0000 7f3ba2ffd640 1 --2- 172.21.15.88:0/1523910186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ba415c960 0x7f3ba40cdf10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3ba408b980 tx=0x7f3b9401c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be855eab520d7ae server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:46.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.037+0000 7f3ba27fc640 1 -- 172.21.15.88:0/1523910186 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b940ab600 con 0x7f3ba415c960 2023-10-16T20:05:46.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.037+0000 7f3ba27fc640 1 -- 172.21.15.88:0/1523910186 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b940a7050 con 0x7f3ba415c960 2023-10-16T20:05:46.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.037+0000 7f3ba27fc640 1 -- 172.21.15.88:0/1523910186 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b940ab980 con 0x7f3ba415c960 2023-10-16T20:05:46.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.037+0000 7f3ba9565640 1 -- 172.21.15.88:0/1523910186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ba415c960 msgr2=0x7f3ba40cdf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:46.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.037+0000 7f3ba9565640 1 --2- 172.21.15.88:0/1523910186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ba415c960 0x7f3ba40cdf10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3ba408b980 tx=0x7f3b9401c4c0 comp rx=0 tx=0).stop 2023-10-16T20:05:46.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.037+0000 7f3ba9565640 1 -- 172.21.15.88:0/1523910186 shutdown_connections 2023-10-16T20:05:46.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.037+0000 7f3ba9565640 1 --2- 172.21.15.88:0/1523910186 >> v2:172.21.15.94:3300/0 conn(0x7f3ba40ce450 0x7f3ba40cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:46.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.037+0000 7f3ba9565640 1 --2- 172.21.15.88:0/1523910186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ba415c960 0x7f3ba40cdf10 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:46.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.037+0000 7f3ba9565640 1 -- 172.21.15.88:0/1523910186 >> 172.21.15.88:0/1523910186 conn(0x7f3ba409bbd0 msgr2=0x7f3ba40d2880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:46.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.037+0000 7f3ba9565640 1 -- 172.21.15.88:0/1523910186 shutdown_connections 2023-10-16T20:05:46.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.037+0000 7f3ba9565640 1 -- 172.21.15.88:0/1523910186 wait complete. 2023-10-16T20:05:46.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.037+0000 7f3ba9565640 1 Processor -- start 2023-10-16T20:05:46.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.037+0000 7f3ba9565640 1 -- start start 2023-10-16T20:05:46.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.041+0000 7f3ba9565640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ba40ce450 0x7f3ba40b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:46.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.041+0000 7f3ba9565640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ba40cf7d0 0x7f3ba406bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:46.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.041+0000 7f3ba9565640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ba415c960 0x7f3ba406c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:46.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.041+0000 7f3ba9565640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba40b4850 con 0x7f3ba40cf7d0 2023-10-16T20:05:46.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.041+0000 7f3ba9565640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba40b4b30 con 0x7f3ba40ce450 2023-10-16T20:05:46.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.041+0000 7f3ba9565640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba40b4e10 con 0x7f3ba415c960 2023-10-16T20:05:46.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.041+0000 7f3b9a7fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ba40cf7d0 0x7f3ba406bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:46.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.041+0000 7f3b9a7fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ba40cf7d0 0x7f3ba406bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48038/0 (socket says 172.21.15.88:48038) 2023-10-16T20:05:46.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.041+0000 7f3b9a7fc640 1 -- 172.21.15.88:0/1240413131 learned_addr learned my addr 172.21.15.88:0/1240413131 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:46.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.041+0000 7f3ba2ffd640 1 --2- 172.21.15.88:0/1240413131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ba40ce450 0x7f3ba40b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:46.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.041+0000 7f3ba37fe640 1 --2- 172.21.15.88:0/1240413131 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ba415c960 0x7f3ba406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:46.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.041+0000 7f3b9a7fc640 1 -- 172.21.15.88:0/1240413131 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ba415c960 msgr2=0x7f3ba406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:46.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.041+0000 7f3b9a7fc640 1 --2- 172.21.15.88:0/1240413131 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ba415c960 0x7f3ba406c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:46.048 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.041+0000 7f3b9a7fc640 1 -- 172.21.15.88:0/1240413131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ba40ce450 msgr2=0x7f3ba40b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:46.048 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.041+0000 7f3b9a7fc640 1 --2- 172.21.15.88:0/1240413131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ba40ce450 0x7f3ba40b3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:46.048 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.041+0000 7f3b9a7fc640 1 -- 172.21.15.88:0/1240413131 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b940094d0 con 0x7f3ba40cf7d0 2023-10-16T20:05:46.048 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.041+0000 7f3b9a7fc640 1 --2- 172.21.15.88:0/1240413131 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ba40cf7d0 0x7f3ba406bf50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c00ce00 tx=0x7f3b8c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:46.048 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.041+0000 7f3b9bfff640 1 -- 172.21.15.88:0/1240413131 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c00f890 con 0x7f3ba40cf7d0 2023-10-16T20:05:46.048 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.041+0000 7f3b9bfff640 1 -- 172.21.15.88:0/1240413131 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b8c00f9f0 con 0x7f3ba40cf7d0 2023-10-16T20:05:46.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.041+0000 7f3b9bfff640 1 -- 172.21.15.88:0/1240413131 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c00fdc0 con 0x7f3ba40cf7d0 2023-10-16T20:05:46.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.041+0000 7f3ba9565640 1 -- 172.21.15.88:0/1240413131 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba40d1810 con 0x7f3ba40cf7d0 2023-10-16T20:05:46.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.041+0000 7f3ba9565640 1 -- 172.21.15.88:0/1240413131 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba40b50f0 con 0x7f3ba40cf7d0 2023-10-16T20:05:46.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.041+0000 7f3ba9565640 1 -- 172.21.15.88:0/1240413131 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba40674f0 con 0x7f3ba40cf7d0 2023-10-16T20:05:46.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.041+0000 7f3b9bfff640 1 -- 172.21.15.88:0/1240413131 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3b8c00b040 con 0x7f3ba40cf7d0 2023-10-16T20:05:46.050 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.045+0000 7f3b9bfff640 1 --2- 172.21.15.88:0/1240413131 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3b840664a0 0x7f3b84068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:46.050 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.045+0000 7f3b9bfff640 1 -- 172.21.15.88:0/1240413131 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3b8c084b90 con 0x7f3ba40cf7d0 2023-10-16T20:05:46.051 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.045+0000 7f3ba2ffd640 1 --2- 172.21.15.88:0/1240413131 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3b840664a0 0x7f3b84068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:46.051 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.045+0000 7f3ba2ffd640 1 --2- 172.21.15.88:0/1240413131 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3b840664a0 0x7f3b84068960 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3b940ad000 tx=0x7f3b940a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:46.051 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.049+0000 7f3b9bfff640 1 -- 172.21.15.88:0/1240413131 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3b8c051be0 con 0x7f3ba40cf7d0 2023-10-16T20:05:46.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.197+0000 7f3ba9565640 1 -- 172.21.15.88:0/1240413131 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3ba40d2f90 con 0x7f3ba40cf7d0 2023-10-16T20:05:46.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.197+0000 7f3b9bfff640 1 -- 172.21.15.88:0/1240413131 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3b8c01d0a0 con 0x7f3ba40cf7d0 2023-10-16T20:05:46.201 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2023-10-16T20:05:46.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.201+0000 7f3ba9565640 1 -- 172.21.15.88:0/1240413131 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3b840664a0 msgr2=0x7f3b84068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:46.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.201+0000 7f3ba9565640 1 --2- 172.21.15.88:0/1240413131 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3b840664a0 0x7f3b84068960 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3b940ad000 tx=0x7f3b940a6000 comp rx=0 tx=0).stop 2023-10-16T20:05:46.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.201+0000 7f3ba9565640 1 -- 172.21.15.88:0/1240413131 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ba40cf7d0 msgr2=0x7f3ba406bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:46.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.201+0000 7f3ba9565640 1 --2- 172.21.15.88:0/1240413131 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ba40cf7d0 0x7f3ba406bf50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c00ce00 tx=0x7f3b8c00a650 comp rx=0 tx=0).stop 2023-10-16T20:05:46.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.201+0000 7f3ba9565640 1 -- 172.21.15.88:0/1240413131 shutdown_connections 2023-10-16T20:05:46.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.201+0000 7f3ba9565640 1 --2- 172.21.15.88:0/1240413131 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3b840664a0 0x7f3b84068960 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:46.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.201+0000 7f3ba9565640 1 --2- 172.21.15.88:0/1240413131 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ba415c960 0x7f3ba406c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:46.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.201+0000 7f3ba9565640 1 --2- 172.21.15.88:0/1240413131 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ba40cf7d0 0x7f3ba406bf50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:46.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.201+0000 7f3ba9565640 1 --2- 172.21.15.88:0/1240413131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ba40ce450 0x7f3ba40b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:46.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.201+0000 7f3ba9565640 1 -- 172.21.15.88:0/1240413131 >> 172.21.15.88:0/1240413131 conn(0x7f3ba409bbd0 msgr2=0x7f3ba40b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:46.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.201+0000 7f3ba9565640 1 -- 172.21.15.88:0/1240413131 shutdown_connections 2023-10-16T20:05:46.206 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.201+0000 7f3ba9565640 1 -- 172.21.15.88:0/1240413131 wait complete. 2023-10-16T20:05:46.223 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T20:05:46.363 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7c2365640 1 Processor -- start 2023-10-16T20:05:46.363 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7c2365640 1 -- start start 2023-10-16T20:05:46.364 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7c2365640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb7bc154f30 0x7fb7bc0b0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:46.365 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7c2365640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb7bc0b0a40 0x7fb7bc15aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:46.365 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7c2365640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7bc06d2f0 con 0x7fb7bc15b530 2023-10-16T20:05:46.365 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7c2365640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7bc155310 con 0x7fb7bc0b0a40 2023-10-16T20:05:46.365 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7c2365640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7bc0b0ec0 con 0x7fb7bc154f30 2023-10-16T20:05:46.365 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7c0902640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fb7bc15b530 0x7fb7bc0b1610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:34680/0 (socket says 172.21.15.88:34680) 2023-10-16T20:05:46.366 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7c0902640 1 -- 172.21.15.88:0/3032101207 learned_addr learned my addr 172.21.15.88:0/3032101207 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:46.366 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7bbfff640 1 --2- 172.21.15.88:0/3032101207 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb7bc154f30 0x7fb7bc0b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:46.366 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7bb7fe640 1 --2- 172.21.15.88:0/3032101207 >> v2:172.21.15.94:3300/0 conn(0x7fb7bc0b0a40 0x7fb7bc15aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:46.366 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7baffd640 1 -- 172.21.15.88:0/3032101207 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3153060317 0 0) 0x7fb7bc06d2f0 con 0x7fb7bc15b530 2023-10-16T20:05:46.367 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7baffd640 1 -- 172.21.15.88:0/3032101207 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7a4003530 con 0x7fb7bc15b530 2023-10-16T20:05:46.369 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7baffd640 1 -- 172.21.15.88:0/3032101207 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 815156514 0 0) 0x7fb7a4003530 con 0x7fb7bc15b530 2023-10-16T20:05:46.369 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7baffd640 1 -- 172.21.15.88:0/3032101207 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb7bc154f30 msgr2=0x7fb7bc0b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:46.369 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7baffd640 1 --2- 172.21.15.88:0/3032101207 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb7bc154f30 0x7fb7bc0b0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:46.370 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7baffd640 1 -- 172.21.15.88:0/3032101207 >> v2:172.21.15.94:3300/0 conn(0x7fb7bc0b0a40 msgr2=0x7fb7bc15aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:46.370 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7baffd640 1 --2- 172.21.15.88:0/3032101207 >> v2:172.21.15.94:3300/0 conn(0x7fb7bc0b0a40 0x7fb7bc15aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:46.370 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7baffd640 1 -- 172.21.15.88:0/3032101207 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7bc15bd20 con 0x7fb7bc15b530 2023-10-16T20:05:46.370 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7baffd640 1 -- 172.21.15.88:0/3032101207 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7fb7b4001a00 con 0x7fb7bc15b530 2023-10-16T20:05:46.370 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7baffd640 1 -- 172.21.15.88:0/3032101207 >> v1:172.21.15.88:6789/0 conn(0x7fb7bc15b530 legacy=0x7fb7bc0b1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:46.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7baffd640 1 --2- 172.21.15.88:0/3032101207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7a4003980 0x7fb7a4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:46.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7baffd640 1 --2- 172.21.15.88:0/3032101207 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb7bc154f30 0x7fb7a40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:46.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7baffd640 1 --2- 172.21.15.88:0/3032101207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7a4004d10 0x7fb7a4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:46.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7baffd640 1 -- 172.21.15.88:0/3032101207 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7bc155310 con 0x7fb7a4003980 2023-10-16T20:05:46.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7baffd640 1 -- 172.21.15.88:0/3032101207 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7bc0b0ec0 con 0x7fb7a4004d10 2023-10-16T20:05:46.372 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7baffd640 1 -- 172.21.15.88:0/3032101207 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a400b990 con 0x7fb7bc154f30 2023-10-16T20:05:46.372 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7bbfff640 1 --2- 172.21.15.88:0/3032101207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7a4003980 0x7fb7a4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:46.372 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7bb7fe640 1 --2- 172.21.15.88:0/3032101207 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb7bc154f30 0x7fb7a40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:46.372 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7c0902640 1 --2- 172.21.15.88:0/3032101207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7a4004d10 0x7fb7a4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:46.373 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7bbfff640 1 -- 172.21.15.88:0/3032101207 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb7bc154f30 msgr2=0x7fb7a40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:46.373 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7bbfff640 1 --2- 172.21.15.88:0/3032101207 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb7bc154f30 0x7fb7a40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:46.373 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7bbfff640 1 -- 172.21.15.88:0/3032101207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7a4004d10 msgr2=0x7fb7a4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:46.373 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7bbfff640 1 --2- 172.21.15.88:0/3032101207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7a4004d10 0x7fb7a4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:46.373 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7bbfff640 1 -- 172.21.15.88:0/3032101207 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a4003530 con 0x7fb7a4003980 2023-10-16T20:05:46.374 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.361+0000 7fb7bb7fe640 1 --2- 172.21.15.88:0/3032101207 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb7bc154f30 0x7fb7a40047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:46.374 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.365+0000 7fb7bbfff640 1 --2- 172.21.15.88:0/3032101207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7a4003980 0x7fb7a4003da0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac009ae0 tx=0x7fb7ac01ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac50cc83a8052674 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:46.374 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.365+0000 7fb7baffd640 1 -- 172.21.15.88:0/3032101207 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac0a6510 con 0x7fb7a4003980 2023-10-16T20:05:46.374 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.365+0000 7fb7baffd640 1 -- 172.21.15.88:0/3032101207 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7ac0a6670 con 0x7fb7a4003980 2023-10-16T20:05:46.374 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.365+0000 7fb7baffd640 1 -- 172.21.15.88:0/3032101207 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac0a6a20 con 0x7fb7a4003980 2023-10-16T20:05:46.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.365+0000 7fb7c2365640 1 -- 172.21.15.88:0/3032101207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7a4003980 msgr2=0x7fb7a4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:46.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.365+0000 7fb7c2365640 1 --2- 172.21.15.88:0/3032101207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7a4003980 0x7fb7a4003da0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac009ae0 tx=0x7fb7ac01ca20 comp rx=0 tx=0).stop 2023-10-16T20:05:46.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.365+0000 7fb7c0902640 1 -- 172.21.15.88:0/3032101207 reap_dead start 2023-10-16T20:05:46.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.365+0000 7fb7c2365640 1 -- 172.21.15.88:0/3032101207 shutdown_connections 2023-10-16T20:05:46.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.365+0000 7fb7c2365640 1 -- 172.21.15.88:0/3032101207 >> 172.21.15.88:0/3032101207 conn(0x7fb7bc0cb080 msgr2=0x7fb7bc0cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:46.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.365+0000 7fb7c2365640 1 -- 172.21.15.88:0/3032101207 shutdown_connections 2023-10-16T20:05:46.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.365+0000 7fb7c2365640 1 -- 172.21.15.88:0/3032101207 wait complete. 2023-10-16T20:05:46.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.369+0000 7fb7c2365640 1 Processor -- start 2023-10-16T20:05:46.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.369+0000 7fb7c2365640 1 -- start start 2023-10-16T20:05:46.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.369+0000 7fb7c2365640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7a4003980 0x7fb7bc14e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:46.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.369+0000 7fb7c2365640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb7bc1f4f90 0x7fb7bc1f5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:46.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.369+0000 7fb7c2365640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7bc0cc3e0 0x7fb7bc0cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:46.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.369+0000 7fb7c2365640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7bc14e870 con 0x7fb7a4003980 2023-10-16T20:05:46.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.369+0000 7fb7c2365640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7bc0ccda0 con 0x7fb7bc0cc3e0 2023-10-16T20:05:46.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.369+0000 7fb7c2365640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7bc151470 con 0x7fb7bc1f4f90 2023-10-16T20:05:46.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.369+0000 7fb7bbfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7a4003980 0x7fb7bc14e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:46.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.369+0000 7fb7bbfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7a4003980 0x7fb7bc14e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48058/0 (socket says 172.21.15.88:48058) 2023-10-16T20:05:46.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.369+0000 7fb7bbfff640 1 -- 172.21.15.88:0/3751770420 learned_addr learned my addr 172.21.15.88:0/3751770420 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:46.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.369+0000 7fb7bb7fe640 1 --2- 172.21.15.88:0/3751770420 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb7bc1f4f90 0x7fb7bc1f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:46.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.369+0000 7fb7c0902640 1 --2- 172.21.15.88:0/3751770420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7bc0cc3e0 0x7fb7bc0cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:46.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.369+0000 7fb7bbfff640 1 -- 172.21.15.88:0/3751770420 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb7bc1f4f90 msgr2=0x7fb7bc1f5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:46.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.369+0000 7fb7bbfff640 1 --2- 172.21.15.88:0/3751770420 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb7bc1f4f90 0x7fb7bc1f5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:46.384 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.369+0000 7fb7bbfff640 1 -- 172.21.15.88:0/3751770420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7bc0cc3e0 msgr2=0x7fb7bc0cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:46.384 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.369+0000 7fb7bbfff640 1 --2- 172.21.15.88:0/3751770420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7bc0cc3e0 0x7fb7bc0cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:46.384 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.369+0000 7fb7bbfff640 1 -- 172.21.15.88:0/3751770420 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7bc0aeb60 con 0x7fb7a4003980 2023-10-16T20:05:46.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.369+0000 7fb7bbfff640 1 --2- 172.21.15.88:0/3751770420 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7a4003980 0x7fb7bc14e330 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac01c850 tx=0x7fb7a4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:46.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.373+0000 7fb7b8ff9640 1 -- 172.21.15.88:0/3751770420 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac00f000 con 0x7fb7a4003980 2023-10-16T20:05:46.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.373+0000 7fb7b8ff9640 1 -- 172.21.15.88:0/3751770420 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7ac0a67e0 con 0x7fb7a4003980 2023-10-16T20:05:46.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.373+0000 7fb7b8ff9640 1 -- 172.21.15.88:0/3751770420 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac0a6b90 con 0x7fb7a4003980 2023-10-16T20:05:46.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.373+0000 7fb7c2365640 1 -- 172.21.15.88:0/3751770420 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7bc0aee40 con 0x7fb7a4003980 2023-10-16T20:05:46.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.373+0000 7fb7c2365640 1 -- 172.21.15.88:0/3751770420 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7bc0af400 con 0x7fb7a4003980 2023-10-16T20:05:46.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.373+0000 7fb7c2365640 1 -- 172.21.15.88:0/3751770420 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7bc04e9c0 con 0x7fb7a4003980 2023-10-16T20:05:46.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.373+0000 7fb7b8ff9640 1 -- 172.21.15.88:0/3751770420 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb7ac0a6cf0 con 0x7fb7a4003980 2023-10-16T20:05:46.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.373+0000 7fb7b8ff9640 1 --2- 172.21.15.88:0/3751770420 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb7a80664a0 0x7fb7a8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:46.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.373+0000 7fb7b8ff9640 1 -- 172.21.15.88:0/3751770420 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb7ac0aa070 con 0x7fb7a4003980 2023-10-16T20:05:46.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.373+0000 7fb7bb7fe640 1 --2- 172.21.15.88:0/3751770420 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb7a80664a0 0x7fb7a8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:46.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.377+0000 7fb7bb7fe640 1 --2- 172.21.15.88:0/3751770420 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb7a80664a0 0x7fb7a8068960 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc1f8490 tx=0x7fb7b0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:46.388 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.377+0000 7fb7b8ff9640 1 -- 172.21.15.88:0/3751770420 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb7ac0ebef0 con 0x7fb7a4003980 2023-10-16T20:05:46.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.529+0000 7fb7c2365640 1 -- 172.21.15.88:0/3751770420 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb7bc06d020 con 0x7fb7a4003980 2023-10-16T20:05:46.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.529+0000 7fb7b8ff9640 1 -- 172.21.15.88:0/3751770420 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb7ac0ef510 con 0x7fb7a4003980 2023-10-16T20:05:46.534 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:05:46.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.533+0000 7fb7c2365640 1 -- 172.21.15.88:0/3751770420 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb7a80664a0 msgr2=0x7fb7a8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:46.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.533+0000 7fb7c2365640 1 --2- 172.21.15.88:0/3751770420 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb7a80664a0 0x7fb7a8068960 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc1f8490 tx=0x7fb7b0008040 comp rx=0 tx=0).stop 2023-10-16T20:05:46.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.533+0000 7fb7c2365640 1 -- 172.21.15.88:0/3751770420 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7a4003980 msgr2=0x7fb7bc14e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:46.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.533+0000 7fb7c2365640 1 --2- 172.21.15.88:0/3751770420 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7a4003980 0x7fb7bc14e330 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac01c850 tx=0x7fb7a4005800 comp rx=0 tx=0).stop 2023-10-16T20:05:46.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.533+0000 7fb7c2365640 1 -- 172.21.15.88:0/3751770420 shutdown_connections 2023-10-16T20:05:46.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.533+0000 7fb7c2365640 1 --2- 172.21.15.88:0/3751770420 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb7a80664a0 0x7fb7a8068960 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:46.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.533+0000 7fb7c2365640 1 --2- 172.21.15.88:0/3751770420 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb7bc1f4f90 0x7fb7bc1f5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:46.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.533+0000 7fb7c2365640 1 --2- 172.21.15.88:0/3751770420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7bc0cc3e0 0x7fb7bc0cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:46.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.533+0000 7fb7c2365640 1 --2- 172.21.15.88:0/3751770420 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7a4003980 0x7fb7bc14e330 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:46.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.533+0000 7fb7c2365640 1 -- 172.21.15.88:0/3751770420 >> 172.21.15.88:0/3751770420 conn(0x7fb7bc0cb080 msgr2=0x7fb7bc154230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:46.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.533+0000 7fb7c2365640 1 -- 172.21.15.88:0/3751770420 shutdown_connections 2023-10-16T20:05:46.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.533+0000 7fb7c2365640 1 -- 172.21.15.88:0/3751770420 wait complete. 2023-10-16T20:05:46.555 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T20:05:46.693 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.689+0000 7fd9e6b1d640 1 Processor -- start 2023-10-16T20:05:46.694 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.689+0000 7fd9e6b1d640 1 -- start start 2023-10-16T20:05:46.695 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.693+0000 7fd9e6b1d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd9e00adf80 0x7fd9e015a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:46.695 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.693+0000 7fd9e6b1d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd9e015d480 0x7fd9e00afa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:46.696 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.693+0000 7fd9e6b1d640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9e015e400 con 0x7fd9e0069e90 2023-10-16T20:05:46.696 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.693+0000 7fd9e6b1d640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9e0159e40 con 0x7fd9e015d480 2023-10-16T20:05:46.696 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.693+0000 7fd9e6b1d640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9e015de40 con 0x7fd9e00adf80 2023-10-16T20:05:46.696 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.693+0000 7fd9e48b9640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fd9e0069e90 0x7fd9e006a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:34692/0 (socket says 172.21.15.88:34692) 2023-10-16T20:05:46.697 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.693+0000 7fd9e48b9640 1 -- 172.21.15.88:0/1401764397 learned_addr learned my addr 172.21.15.88:0/1401764397 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:46.697 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.693+0000 7fd9e50ba640 1 --2- 172.21.15.88:0/1401764397 >> v2:172.21.15.88:3300/0 conn(0x7fd9e015d480 0x7fd9e00afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:46.697 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.693+0000 7fd9d7fff640 1 --2- 172.21.15.88:0/1401764397 >> v2:172.21.15.94:3300/0 conn(0x7fd9e00adf80 0x7fd9e015a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:46.697 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.693+0000 7fd9e50ba640 1 -- 172.21.15.88:0/1401764397 >> v2:172.21.15.94:3300/0 conn(0x7fd9e00adf80 msgr2=0x7fd9e015a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:46.697 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.693+0000 7fd9e50ba640 1 --2- 172.21.15.88:0/1401764397 >> v2:172.21.15.94:3300/0 conn(0x7fd9e00adf80 0x7fd9e015a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:46.697 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.693+0000 7fd9e50ba640 1 -- 172.21.15.88:0/1401764397 >> v1:172.21.15.88:6789/0 conn(0x7fd9e0069e90 legacy=0x7fd9e006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:46.698 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.693+0000 7fd9d77fe640 1 -- 172.21.15.88:0/1401764397 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 499996986 0 0) 0x7fd9e015e400 con 0x7fd9e0069e90 2023-10-16T20:05:46.698 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.693+0000 7fd9e50ba640 1 -- 172.21.15.88:0/1401764397 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e015e120 con 0x7fd9e015d480 2023-10-16T20:05:46.698 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.693+0000 7fd9d7fff640 1 --2- 172.21.15.88:0/1401764397 >> v2:172.21.15.94:3300/0 conn(0x7fd9e00adf80 0x7fd9e015a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:46.699 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.697+0000 7fd9e50ba640 1 --2- 172.21.15.88:0/1401764397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9e015d480 0x7fd9e00afa20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd9e00725c0 tx=0x7fd9d801c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acee58e6f466a659 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:46.700 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.697+0000 7fd9d77fe640 1 -- 172.21.15.88:0/1401764397 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d80a5820 con 0x7fd9e015d480 2023-10-16T20:05:46.700 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.697+0000 7fd9d77fe640 1 -- 172.21.15.88:0/1401764397 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d80a5980 con 0x7fd9e015d480 2023-10-16T20:05:46.700 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.697+0000 7fd9d77fe640 1 -- 172.21.15.88:0/1401764397 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d80a5d10 con 0x7fd9e015d480 2023-10-16T20:05:46.700 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.697+0000 7fd9e6b1d640 1 -- 172.21.15.88:0/1401764397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9e015d480 msgr2=0x7fd9e00afa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:46.700 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.697+0000 7fd9e6b1d640 1 --2- 172.21.15.88:0/1401764397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9e015d480 0x7fd9e00afa20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd9e00725c0 tx=0x7fd9d801c850 comp rx=0 tx=0).stop 2023-10-16T20:05:46.701 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.697+0000 7fd9e6b1d640 1 -- 172.21.15.88:0/1401764397 shutdown_connections 2023-10-16T20:05:46.701 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.697+0000 7fd9e6b1d640 1 --2- 172.21.15.88:0/1401764397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9e015d480 0x7fd9e00afa20 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:46.701 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.697+0000 7fd9e6b1d640 1 --2- 172.21.15.88:0/1401764397 >> v2:172.21.15.94:3300/0 conn(0x7fd9e00adf80 0x7fd9e015a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:46.701 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.697+0000 7fd9e6b1d640 1 -- 172.21.15.88:0/1401764397 >> 172.21.15.88:0/1401764397 conn(0x7fd9e00ce5e0 msgr2=0x7fd9e00ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:46.701 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.697+0000 7fd9e6b1d640 1 -- 172.21.15.88:0/1401764397 shutdown_connections 2023-10-16T20:05:46.701 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.697+0000 7fd9e6b1d640 1 -- 172.21.15.88:0/1401764397 wait complete. 2023-10-16T20:05:46.703 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.697+0000 7fd9e6b1d640 1 Processor -- start 2023-10-16T20:05:46.703 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.697+0000 7fd9e6b1d640 1 -- start start 2023-10-16T20:05:46.703 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.701+0000 7fd9e6b1d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd9e0069e90 0x7fd9e013cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:46.704 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.701+0000 7fd9e6b1d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9e00adf80 0x7fd9e00cb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:46.704 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.701+0000 7fd9e6b1d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9e015d480 0x7fd9e00cb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:46.704 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.701+0000 7fd9e6b1d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e015bb40 con 0x7fd9e00adf80 2023-10-16T20:05:46.704 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.701+0000 7fd9e6b1d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e013d460 con 0x7fd9e015d480 2023-10-16T20:05:46.704 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.701+0000 7fd9e6b1d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e013d0d0 con 0x7fd9e0069e90 2023-10-16T20:05:46.705 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.701+0000 7fd9d7fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9e00adf80 0x7fd9e00cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:46.705 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.701+0000 7fd9d7fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9e00adf80 0x7fd9e00cb200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48088/0 (socket says 172.21.15.88:48088) 2023-10-16T20:05:46.705 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.701+0000 7fd9d7fff640 1 -- 172.21.15.88:0/699007710 learned_addr learned my addr 172.21.15.88:0/699007710 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:46.705 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.701+0000 7fd9e48b9640 1 --2- 172.21.15.88:0/699007710 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd9e0069e90 0x7fd9e013cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:46.705 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.701+0000 7fd9e50ba640 1 --2- 172.21.15.88:0/699007710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9e015d480 0x7fd9e00cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:46.706 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.701+0000 7fd9d7fff640 1 -- 172.21.15.88:0/699007710 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd9e0069e90 msgr2=0x7fd9e013cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:46.706 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.701+0000 7fd9d7fff640 1 --2- 172.21.15.88:0/699007710 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd9e0069e90 0x7fd9e013cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:46.706 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.701+0000 7fd9d7fff640 1 -- 172.21.15.88:0/699007710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9e015d480 msgr2=0x7fd9e00cb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:46.706 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.701+0000 7fd9d7fff640 1 --2- 172.21.15.88:0/699007710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9e015d480 0x7fd9e00cb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:46.706 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.701+0000 7fd9d7fff640 1 -- 172.21.15.88:0/699007710 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d8009a70 con 0x7fd9e00adf80 2023-10-16T20:05:46.707 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.701+0000 7fd9d7fff640 1 --2- 172.21.15.88:0/699007710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9e00adf80 0x7fd9e00cb200 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000c7f0 tx=0x7fd9d000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:46.708 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.701+0000 7fd9d57fa640 1 -- 172.21.15.88:0/699007710 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d000cea0 con 0x7fd9e00adf80 2023-10-16T20:05:46.708 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.701+0000 7fd9d57fa640 1 -- 172.21.15.88:0/699007710 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d0007a30 con 0x7fd9e00adf80 2023-10-16T20:05:46.709 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.701+0000 7fd9d57fa640 1 -- 172.21.15.88:0/699007710 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0007de0 con 0x7fd9e00adf80 2023-10-16T20:05:46.709 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.701+0000 7fd9e6b1d640 1 -- 172.21.15.88:0/699007710 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e013d6d0 con 0x7fd9e00adf80 2023-10-16T20:05:46.709 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.701+0000 7fd9e6b1d640 1 -- 172.21.15.88:0/699007710 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e013dc90 con 0x7fd9e00adf80 2023-10-16T20:05:46.709 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.701+0000 7fd9e6b1d640 1 -- 172.21.15.88:0/699007710 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e006d180 con 0x7fd9e00adf80 2023-10-16T20:05:46.709 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.701+0000 7fd9d57fa640 1 -- 172.21.15.88:0/699007710 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd9d00113f0 con 0x7fd9e00adf80 2023-10-16T20:05:46.710 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.705+0000 7fd9d57fa640 1 --2- 172.21.15.88:0/699007710 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd9cc0664a0 0x7fd9cc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:46.710 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.705+0000 7fd9d57fa640 1 -- 172.21.15.88:0/699007710 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd9d0084300 con 0x7fd9e00adf80 2023-10-16T20:05:46.710 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.705+0000 7fd9e48b9640 1 --2- 172.21.15.88:0/699007710 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd9cc0664a0 0x7fd9cc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:46.710 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.705+0000 7fd9e48b9640 1 --2- 172.21.15.88:0/699007710 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd9cc0664a0 0x7fd9cc068960 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc007fd0 tx=0x7fd9dc007d60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:46.711 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.709+0000 7fd9d57fa640 1 -- 172.21.15.88:0/699007710 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd9d00513d0 con 0x7fd9e00adf80 2023-10-16T20:05:46.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.865+0000 7fd9e6b1d640 1 -- 172.21.15.88:0/699007710 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd9e004e9c0 con 0x7fd9e00adf80 2023-10-16T20:05:46.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.869+0000 7fd9d57fa640 1 -- 172.21.15.88:0/699007710 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd9d00549f0 con 0x7fd9e00adf80 2023-10-16T20:05:46.870 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2023-10-16T20:05:46.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.869+0000 7fd9e6b1d640 1 -- 172.21.15.88:0/699007710 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd9cc0664a0 msgr2=0x7fd9cc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:46.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.869+0000 7fd9e6b1d640 1 --2- 172.21.15.88:0/699007710 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd9cc0664a0 0x7fd9cc068960 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc007fd0 tx=0x7fd9dc007d60 comp rx=0 tx=0).stop 2023-10-16T20:05:46.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.869+0000 7fd9e6b1d640 1 -- 172.21.15.88:0/699007710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9e00adf80 msgr2=0x7fd9e00cb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:46.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.869+0000 7fd9e6b1d640 1 --2- 172.21.15.88:0/699007710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9e00adf80 0x7fd9e00cb200 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000c7f0 tx=0x7fd9d000ccb0 comp rx=0 tx=0).stop 2023-10-16T20:05:46.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.869+0000 7fd9e6b1d640 1 -- 172.21.15.88:0/699007710 shutdown_connections 2023-10-16T20:05:46.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.869+0000 7fd9e6b1d640 1 --2- 172.21.15.88:0/699007710 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd9cc0664a0 0x7fd9cc068960 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:46.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.869+0000 7fd9e6b1d640 1 --2- 172.21.15.88:0/699007710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9e015d480 0x7fd9e00cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:46.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.869+0000 7fd9e6b1d640 1 --2- 172.21.15.88:0/699007710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9e00adf80 0x7fd9e00cb200 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:46.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.869+0000 7fd9e6b1d640 1 --2- 172.21.15.88:0/699007710 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd9e0069e90 0x7fd9e013cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:46.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.869+0000 7fd9e6b1d640 1 -- 172.21.15.88:0/699007710 >> 172.21.15.88:0/699007710 conn(0x7fd9e00ce5e0 msgr2=0x7fd9e00af530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:46.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.869+0000 7fd9e6b1d640 1 -- 172.21.15.88:0/699007710 shutdown_connections 2023-10-16T20:05:46.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:46.869+0000 7fd9e6b1d640 1 -- 172.21.15.88:0/699007710 wait complete. 2023-10-16T20:05:46.891 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:05:46.891 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:05:46.901 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:05:47.082 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.077+0000 7f6e94f1d640 1 Processor -- start 2023-10-16T20:05:47.082 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.077+0000 7f6e94f1d640 1 -- start start 2023-10-16T20:05:47.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.081+0000 7f6e94f1d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f6e9006b100 0x7f6e9006c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:47.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.081+0000 7f6e94f1d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6e900715c0 0x7f6e9006c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:47.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.081+0000 7f6e94f1d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f6e9006b4e0 0x7f6e9006cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:47.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.081+0000 7f6e94f1d640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e900ae720 con 0x7f6e9006b4e0 2023-10-16T20:05:47.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.081+0000 7f6e94f1d640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e900c9a10 con 0x7f6e9006b100 2023-10-16T20:05:47.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.081+0000 7f6e94f1d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9006d650 con 0x7f6e900715c0 2023-10-16T20:05:47.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.081+0000 7f6e8ed9d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f6e9006b4e0 0x7f6e9006cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:47.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.081+0000 7f6e8ed9d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f6e9006b4e0 0x7f6e9006cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48092/0 (socket says 172.21.15.88:48092) 2023-10-16T20:05:47.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.081+0000 7f6e8ed9d640 1 -- 172.21.15.88:0/1271785376 learned_addr learned my addr 172.21.15.88:0/1271785376 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:47.086 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.081+0000 7f6e8dd9b640 1 --2- 172.21.15.88:0/1271785376 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6e900715c0 0x7f6e9006c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:47.086 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.081+0000 7f6e8e59c640 1 --2- 172.21.15.88:0/1271785376 >> v2:172.21.15.94:3300/0 conn(0x7f6e9006b100 0x7f6e9006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:47.086 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.081+0000 7f6e8ed9d640 1 -- 172.21.15.88:0/1271785376 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6e900715c0 msgr2=0x7f6e9006c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:47.086 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.081+0000 7f6e8ed9d640 1 --2- 172.21.15.88:0/1271785376 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6e900715c0 0x7f6e9006c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:47.087 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.081+0000 7f6e8ed9d640 1 -- 172.21.15.88:0/1271785376 >> v2:172.21.15.94:3300/0 conn(0x7f6e9006b100 msgr2=0x7f6e9006c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:47.087 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.081+0000 7f6e8ed9d640 1 --2- 172.21.15.88:0/1271785376 >> v2:172.21.15.94:3300/0 conn(0x7f6e9006b100 0x7f6e9006c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:47.087 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.081+0000 7f6e8ed9d640 1 -- 172.21.15.88:0/1271785376 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e900c9420 con 0x7f6e9006b4e0 2023-10-16T20:05:47.087 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.081+0000 7f6e8e59c640 1 --2- 172.21.15.88:0/1271785376 >> v2:172.21.15.94:3300/0 conn(0x7f6e9006b100 0x7f6e9006c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:47.088 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.085+0000 7f6e8ed9d640 1 --2- 172.21.15.88:0/1271785376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e9006b4e0 0x7f6e9006cdf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6e900673f0 tx=0x7f6e8801c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2c3977a4347a19 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:47.089 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.085+0000 7f6e8d59a640 1 -- 172.21.15.88:0/1271785376 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e880a5850 con 0x7f6e9006b4e0 2023-10-16T20:05:47.089 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.085+0000 7f6e8d59a640 1 -- 172.21.15.88:0/1271785376 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e880a59b0 con 0x7f6e9006b4e0 2023-10-16T20:05:47.089 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.085+0000 7f6e8d59a640 1 -- 172.21.15.88:0/1271785376 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e880a5d40 con 0x7f6e9006b4e0 2023-10-16T20:05:47.089 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.085+0000 7f6e94f1d640 1 -- 172.21.15.88:0/1271785376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e9006b4e0 msgr2=0x7f6e9006cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:47.090 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.085+0000 7f6e94f1d640 1 --2- 172.21.15.88:0/1271785376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e9006b4e0 0x7f6e9006cdf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6e900673f0 tx=0x7f6e8801c9d0 comp rx=0 tx=0).stop 2023-10-16T20:05:47.090 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.085+0000 7f6e94f1d640 1 -- 172.21.15.88:0/1271785376 shutdown_connections 2023-10-16T20:05:47.090 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.085+0000 7f6e94f1d640 1 --2- 172.21.15.88:0/1271785376 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6e900715c0 0x7f6e9006c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:47.090 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.085+0000 7f6e94f1d640 1 --2- 172.21.15.88:0/1271785376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e9006b4e0 0x7f6e9006cdf0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:47.090 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.085+0000 7f6e94f1d640 1 --2- 172.21.15.88:0/1271785376 >> v2:172.21.15.94:3300/0 conn(0x7f6e9006b100 0x7f6e9006c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:47.091 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.085+0000 7f6e94f1d640 1 -- 172.21.15.88:0/1271785376 >> 172.21.15.88:0/1271785376 conn(0x7f6e90070aa0 msgr2=0x7f6e90070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:47.091 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.085+0000 7f6e94f1d640 1 -- 172.21.15.88:0/1271785376 shutdown_connections 2023-10-16T20:05:47.091 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.085+0000 7f6e94f1d640 1 -- 172.21.15.88:0/1271785376 wait complete. 2023-10-16T20:05:47.093 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.085+0000 7f6e94f1d640 1 Processor -- start 2023-10-16T20:05:47.093 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.085+0000 7f6e94f1d640 1 -- start start 2023-10-16T20:05:47.094 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.089+0000 7f6e94f1d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e9006b100 0x7f6e900b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:47.094 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.089+0000 7f6e94f1d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6e9006b4e0 0x7f6e900c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:47.094 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.089+0000 7f6e94f1d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e900715c0 0x7f6e9015d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:47.094 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.089+0000 7f6e94f1d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e900c3920 con 0x7f6e900715c0 2023-10-16T20:05:47.094 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.089+0000 7f6e94f1d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e900c3c00 con 0x7f6e9006b100 2023-10-16T20:05:47.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.089+0000 7f6e94f1d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e900b43c0 con 0x7f6e9006b4e0 2023-10-16T20:05:47.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.089+0000 7f6e8ed9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e900715c0 0x7f6e9015d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:47.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.089+0000 7f6e8ed9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e900715c0 0x7f6e9015d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48096/0 (socket says 172.21.15.88:48096) 2023-10-16T20:05:47.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.089+0000 7f6e8ed9d640 1 -- 172.21.15.88:0/53782718 learned_addr learned my addr 172.21.15.88:0/53782718 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:47.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.089+0000 7f6e8dd9b640 1 --2- 172.21.15.88:0/53782718 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6e9006b4e0 0x7f6e900c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:47.096 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.089+0000 7f6e8e59c640 1 --2- 172.21.15.88:0/53782718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e9006b100 0x7f6e900b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:47.096 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.089+0000 7f6e8ed9d640 1 -- 172.21.15.88:0/53782718 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6e9006b4e0 msgr2=0x7f6e900c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:47.096 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.089+0000 7f6e8ed9d640 1 --2- 172.21.15.88:0/53782718 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6e9006b4e0 0x7f6e900c84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:47.096 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.089+0000 7f6e8ed9d640 1 -- 172.21.15.88:0/53782718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e9006b100 msgr2=0x7f6e900b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:47.096 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.089+0000 7f6e8ed9d640 1 --2- 172.21.15.88:0/53782718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e9006b100 0x7f6e900b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:47.096 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.089+0000 7f6e8ed9d640 1 -- 172.21.15.88:0/53782718 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e88009b30 con 0x7f6e900715c0 2023-10-16T20:05:47.097 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.089+0000 7f6e8ed9d640 1 --2- 172.21.15.88:0/53782718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e900715c0 0x7f6e9015d250 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6e880af040 tx=0x7f6e88019770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:47.097 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.089+0000 7f6e7effd640 1 -- 172.21.15.88:0/53782718 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e880b8670 con 0x7f6e900715c0 2023-10-16T20:05:47.097 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.089+0000 7f6e7effd640 1 -- 172.21.15.88:0/53782718 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e880b8810 con 0x7f6e900715c0 2023-10-16T20:05:47.097 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.089+0000 7f6e94f1d640 1 -- 172.21.15.88:0/53782718 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e900b46a0 con 0x7f6e900715c0 2023-10-16T20:05:47.097 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.089+0000 7f6e7effd640 1 -- 172.21.15.88:0/53782718 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e880b8bc0 con 0x7f6e900715c0 2023-10-16T20:05:47.098 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.089+0000 7f6e94f1d640 1 -- 172.21.15.88:0/53782718 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e900b1c70 con 0x7f6e900715c0 2023-10-16T20:05:47.098 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.089+0000 7f6e94f1d640 1 -- 172.21.15.88:0/53782718 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e9004e9c0 con 0x7f6e900715c0 2023-10-16T20:05:47.099 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.089+0000 7f6e7effd640 1 -- 172.21.15.88:0/53782718 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6e880b8d20 con 0x7f6e900715c0 2023-10-16T20:05:47.099 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.093+0000 7f6e7effd640 1 --2- 172.21.15.88:0/53782718 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f6e700664a0 0x7f6e70068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:47.099 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.093+0000 7f6e7effd640 1 -- 172.21.15.88:0/53782718 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6e8811f690 con 0x7f6e900715c0 2023-10-16T20:05:47.099 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.093+0000 7f6e8e59c640 1 --2- 172.21.15.88:0/53782718 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f6e700664a0 0x7f6e70068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:47.099 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.093+0000 7f6e8e59c640 1 --2- 172.21.15.88:0/53782718 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f6e700664a0 0x7f6e70068960 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6e8000aaf0 tx=0x7f6e80008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:47.100 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.097+0000 7f6e7effd640 1 -- 172.21.15.88:0/53782718 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6e880ec760 con 0x7f6e900715c0 2023-10-16T20:05:47.249 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.245+0000 7f6e94f1d640 1 -- 172.21.15.88:0/53782718 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e900c3d40 con 0x7f6e900715c0 2023-10-16T20:05:47.251 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.249+0000 7f6e7effd640 1 -- 172.21.15.88:0/53782718 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18661 (secure 0 0 0) 0x7f6e880efd80 con 0x7f6e900715c0 2023-10-16T20:05:47.252 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:47.252 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":29,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:05:38.202959+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T20:04:54.442487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T20:04:55.766641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T20:04:59.475830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":1}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:05:47.253 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.249+0000 7f6e94f1d640 1 -- 172.21.15.88:0/53782718 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f6e700664a0 msgr2=0x7f6e70068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:47.254 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.249+0000 7f6e94f1d640 1 --2- 172.21.15.88:0/53782718 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f6e700664a0 0x7f6e70068960 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6e8000aaf0 tx=0x7f6e80008040 comp rx=0 tx=0).stop 2023-10-16T20:05:47.254 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.249+0000 7f6e94f1d640 1 -- 172.21.15.88:0/53782718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e900715c0 msgr2=0x7f6e9015d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:47.254 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.249+0000 7f6e94f1d640 1 --2- 172.21.15.88:0/53782718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e900715c0 0x7f6e9015d250 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6e880af040 tx=0x7f6e88019770 comp rx=0 tx=0).stop 2023-10-16T20:05:47.254 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.249+0000 7f6e94f1d640 1 -- 172.21.15.88:0/53782718 shutdown_connections 2023-10-16T20:05:47.254 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.249+0000 7f6e94f1d640 1 --2- 172.21.15.88:0/53782718 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f6e700664a0 0x7f6e70068960 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:47.255 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.249+0000 7f6e94f1d640 1 --2- 172.21.15.88:0/53782718 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6e9006b4e0 0x7f6e900c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:47.255 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.249+0000 7f6e94f1d640 1 --2- 172.21.15.88:0/53782718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e900715c0 0x7f6e9015d250 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:47.255 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.249+0000 7f6e94f1d640 1 --2- 172.21.15.88:0/53782718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e9006b100 0x7f6e900b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:47.255 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.249+0000 7f6e94f1d640 1 -- 172.21.15.88:0/53782718 >> 172.21.15.88:0/53782718 conn(0x7f6e90070aa0 msgr2=0x7f6e900afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:47.255 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.249+0000 7f6e94f1d640 1 -- 172.21.15.88:0/53782718 shutdown_connections 2023-10-16T20:05:47.255 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.249+0000 7f6e94f1d640 1 -- 172.21.15.88:0/53782718 wait complete. 2023-10-16T20:05:47.274 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T20:04:54.442487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T20:04:55.766641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T20:04:59.475830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T20:05:47.275 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:05:47.416 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.413+0000 7fb08eaea640 1 Processor -- start 2023-10-16T20:05:47.417 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.413+0000 7fb08eaea640 1 -- start start 2023-10-16T20:05:47.418 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.413+0000 7fb08eaea640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb0880d1be0 0x7fb0880b4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:47.419 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.413+0000 7fb08eaea640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb0880b4a70 0x7fb0880b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:47.419 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.413+0000 7fb08eaea640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0880b7d20 con 0x7fb08815cb70 2023-10-16T20:05:47.419 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.413+0000 7fb08eaea640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0880b4ea0 con 0x7fb0880d1be0 2023-10-16T20:05:47.419 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.413+0000 7fb08eaea640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0880b7110 con 0x7fb0880b4a70 2023-10-16T20:05:47.419 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.413+0000 7fb087fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb0880d1be0 0x7fb0880b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:47.420 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.413+0000 7fb087fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb0880d1be0 0x7fb0880b4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48108/0 (socket says 172.21.15.88:48108) 2023-10-16T20:05:47.420 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.417+0000 7fb087fff640 1 -- 172.21.15.88:0/1837619433 learned_addr learned my addr 172.21.15.88:0/1837619433 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:47.420 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.417+0000 7fb08d087640 1 --2- 172.21.15.88:0/1837619433 >> v2:172.21.15.94:3300/0 conn(0x7fb0880b4a70 0x7fb0880b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:47.420 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.417+0000 7fb087fff640 1 -- 172.21.15.88:0/1837619433 >> v2:172.21.15.94:3300/0 conn(0x7fb0880b4a70 msgr2=0x7fb0880b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:47.421 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.417+0000 7fb087fff640 1 --2- 172.21.15.88:0/1837619433 >> v2:172.21.15.94:3300/0 conn(0x7fb0880b4a70 0x7fb0880b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:47.421 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.417+0000 7fb087fff640 1 -- 172.21.15.88:0/1837619433 >> v1:172.21.15.94:6789/0 conn(0x7fb08815cb70 legacy=0x7fb0880b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:47.421 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.417+0000 7fb087fff640 1 -- 172.21.15.88:0/1837619433 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0880b73f0 con 0x7fb0880d1be0 2023-10-16T20:05:47.421 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.417+0000 7fb08d087640 1 --2- 172.21.15.88:0/1837619433 >> v2:172.21.15.94:3300/0 conn(0x7fb0880b4a70 0x7fb0880b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:47.423 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.417+0000 7fb087fff640 1 --2- 172.21.15.88:0/1837619433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0880d1be0 0x7fb0880b4530 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb0880d03c0 tx=0x7fb07401ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b93caef79918542 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:47.423 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.417+0000 7fb0877fe640 1 -- 172.21.15.88:0/1837619433 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0740a89b0 con 0x7fb0880d1be0 2023-10-16T20:05:47.423 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.417+0000 7fb0877fe640 1 -- 172.21.15.88:0/1837619433 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0740a8b10 con 0x7fb0880d1be0 2023-10-16T20:05:47.423 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.417+0000 7fb0877fe640 1 -- 172.21.15.88:0/1837619433 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0740a8ea0 con 0x7fb0880d1be0 2023-10-16T20:05:47.424 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.421+0000 7fb08eaea640 1 -- 172.21.15.88:0/1837619433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0880d1be0 msgr2=0x7fb0880b4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:47.424 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.421+0000 7fb08eaea640 1 --2- 172.21.15.88:0/1837619433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0880d1be0 0x7fb0880b4530 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb0880d03c0 tx=0x7fb07401ecb0 comp rx=0 tx=0).stop 2023-10-16T20:05:47.424 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.421+0000 7fb08eaea640 1 -- 172.21.15.88:0/1837619433 shutdown_connections 2023-10-16T20:05:47.424 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.421+0000 7fb08eaea640 1 --2- 172.21.15.88:0/1837619433 >> v2:172.21.15.94:3300/0 conn(0x7fb0880b4a70 0x7fb0880b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:47.424 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.421+0000 7fb08eaea640 1 --2- 172.21.15.88:0/1837619433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0880d1be0 0x7fb0880b4530 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:47.425 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.421+0000 7fb08eaea640 1 -- 172.21.15.88:0/1837619433 >> 172.21.15.88:0/1837619433 conn(0x7fb08806b6e0 msgr2=0x7fb08806baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:47.425 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.421+0000 7fb08eaea640 1 -- 172.21.15.88:0/1837619433 shutdown_connections 2023-10-16T20:05:47.425 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.421+0000 7fb08eaea640 1 -- 172.21.15.88:0/1837619433 wait complete. 2023-10-16T20:05:47.427 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.421+0000 7fb08eaea640 1 Processor -- start 2023-10-16T20:05:47.428 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.421+0000 7fb08eaea640 1 -- start start 2023-10-16T20:05:47.428 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.421+0000 7fb08eaea640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb0880b4a70 0x7fb0880c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:47.428 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.421+0000 7fb08eaea640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0880d1be0 0x7fb0880c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:47.428 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.421+0000 7fb08eaea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb08815cb70 0x7fb0880b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:47.428 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.421+0000 7fb08eaea640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0880cbb20 con 0x7fb0880d1be0 2023-10-16T20:05:47.429 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.421+0000 7fb08eaea640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0880cb560 con 0x7fb08815cb70 2023-10-16T20:05:47.429 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.421+0000 7fb08eaea640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0880cb840 con 0x7fb0880b4a70 2023-10-16T20:05:47.429 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.421+0000 7fb087fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0880d1be0 0x7fb0880c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:47.429 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.421+0000 7fb087fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0880d1be0 0x7fb0880c82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48122/0 (socket says 172.21.15.88:48122) 2023-10-16T20:05:47.429 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.421+0000 7fb087fff640 1 -- 172.21.15.88:0/1024356813 learned_addr learned my addr 172.21.15.88:0/1024356813 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:47.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.421+0000 7fb087fff640 1 -- 172.21.15.88:0/1024356813 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb0880b4a70 msgr2=0x7fb0880c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:47.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.421+0000 7fb087fff640 1 --2- 172.21.15.88:0/1024356813 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb0880b4a70 0x7fb0880c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:47.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.421+0000 7fb087fff640 1 -- 172.21.15.88:0/1024356813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb08815cb70 msgr2=0x7fb0880b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:47.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.421+0000 7fb087fff640 1 --2- 172.21.15.88:0/1024356813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb08815cb70 0x7fb0880b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:47.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.421+0000 7fb087fff640 1 -- 172.21.15.88:0/1024356813 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0740094d0 con 0x7fb0880d1be0 2023-10-16T20:05:47.431 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.425+0000 7fb087fff640 1 --2- 172.21.15.88:0/1024356813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0880d1be0 0x7fb0880c82d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb0740b3040 tx=0x7fb07401ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:47.431 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.425+0000 7fb0857fa640 1 -- 172.21.15.88:0/1024356813 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0740b0e80 con 0x7fb0880d1be0 2023-10-16T20:05:47.431 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.425+0000 7fb08eaea640 1 -- 172.21.15.88:0/1024356813 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0880c6760 con 0x7fb0880d1be0 2023-10-16T20:05:47.431 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.425+0000 7fb0857fa640 1 -- 172.21.15.88:0/1024356813 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0740bb480 con 0x7fb0880d1be0 2023-10-16T20:05:47.431 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.425+0000 7fb0857fa640 1 -- 172.21.15.88:0/1024356813 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0740bb830 con 0x7fb0880d1be0 2023-10-16T20:05:47.432 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.425+0000 7fb08eaea640 1 -- 172.21.15.88:0/1024356813 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0880c74a0 con 0x7fb0880d1be0 2023-10-16T20:05:47.432 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.425+0000 7fb08eaea640 1 -- 172.21.15.88:0/1024356813 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0880c65c0 con 0x7fb0880d1be0 2023-10-16T20:05:47.433 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.425+0000 7fb0857fa640 1 -- 172.21.15.88:0/1024356813 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb0740bb990 con 0x7fb0880d1be0 2023-10-16T20:05:47.433 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.425+0000 7fb0857fa640 1 --2- 172.21.15.88:0/1024356813 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb06c0664a0 0x7fb06c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:47.433 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.425+0000 7fb0857fa640 1 -- 172.21.15.88:0/1024356813 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb074122a60 con 0x7fb0880d1be0 2023-10-16T20:05:47.433 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.425+0000 7fb08c886640 1 --2- 172.21.15.88:0/1024356813 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb06c0664a0 0x7fb06c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:47.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.425+0000 7fb08c886640 1 --2- 172.21.15.88:0/1024356813 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb06c0664a0 0x7fb06c068960 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb0880c9600 tx=0x7fb07c006bd0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:47.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.429+0000 7fb0857fa640 1 -- 172.21.15.88:0/1024356813 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb0880c65c0 con 0x7fb0880d1be0 2023-10-16T20:05:47.584 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.581+0000 7fb08eaea640 1 -- 172.21.15.88:0/1024356813 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb0880c65c0 con 0x7fb0880d1be0 2023-10-16T20:05:47.585 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.581+0000 7fb0857fa640 1 -- 172.21.15.88:0/1024356813 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb0880c65c0 con 0x7fb0880d1be0 2023-10-16T20:05:47.585 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:05:47.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.585+0000 7fb08eaea640 1 -- 172.21.15.88:0/1024356813 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb06c0664a0 msgr2=0x7fb06c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:47.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.585+0000 7fb08eaea640 1 --2- 172.21.15.88:0/1024356813 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb06c0664a0 0x7fb06c068960 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb0880c9600 tx=0x7fb07c006bd0 comp rx=0 tx=0).stop 2023-10-16T20:05:47.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.585+0000 7fb08eaea640 1 -- 172.21.15.88:0/1024356813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0880d1be0 msgr2=0x7fb0880c82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:47.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.585+0000 7fb08eaea640 1 --2- 172.21.15.88:0/1024356813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0880d1be0 0x7fb0880c82d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb0740b3040 tx=0x7fb07401ba50 comp rx=0 tx=0).stop 2023-10-16T20:05:47.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.585+0000 7fb08eaea640 1 -- 172.21.15.88:0/1024356813 shutdown_connections 2023-10-16T20:05:47.588 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.585+0000 7fb08eaea640 1 --2- 172.21.15.88:0/1024356813 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb06c0664a0 0x7fb06c068960 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:47.588 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.585+0000 7fb08eaea640 1 --2- 172.21.15.88:0/1024356813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb08815cb70 0x7fb0880b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:47.588 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.585+0000 7fb08eaea640 1 --2- 172.21.15.88:0/1024356813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0880d1be0 0x7fb0880c82d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:47.588 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.585+0000 7fb08eaea640 1 --2- 172.21.15.88:0/1024356813 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb0880b4a70 0x7fb0880c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:47.588 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.585+0000 7fb08eaea640 1 -- 172.21.15.88:0/1024356813 >> 172.21.15.88:0/1024356813 conn(0x7fb08806b6e0 msgr2=0x7fb088157f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:47.589 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.585+0000 7fb08eaea640 1 -- 172.21.15.88:0/1024356813 shutdown_connections 2023-10-16T20:05:47.589 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.585+0000 7fb08eaea640 1 -- 172.21.15.88:0/1024356813 wait complete. 2023-10-16T20:05:47.605 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T20:05:47.745 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.741+0000 7fe3a6b72640 1 Processor -- start 2023-10-16T20:05:47.746 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.741+0000 7fe3a6b72640 1 -- start start 2023-10-16T20:05:47.746 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.745+0000 7fe3a6b72640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3a015a820 0x7fe3a00cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:47.747 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.745+0000 7fe3a6b72640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe3a00b2b00 0x7fe3a006c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:47.747 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.745+0000 7fe3a6b72640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a00ce4b0 con 0x7fe3a015ac00 2023-10-16T20:05:47.747 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.745+0000 7fe3a6b72640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a015e270 con 0x7fe3a00b2b00 2023-10-16T20:05:47.747 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.745+0000 7fe3a6b72640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a00ce700 con 0x7fe3a015a820 2023-10-16T20:05:47.748 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.745+0000 7fe3a490e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3a015a820 0x7fe3a00cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:47.748 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.745+0000 7fe3a510f640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe3a00b2b00 0x7fe3a006c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:47.748 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.745+0000 7fe397fff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fe3a015ac00 0x7fe3a00b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:47522/0 (socket says 172.21.15.88:47522) 2023-10-16T20:05:47.749 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.745+0000 7fe397fff640 1 -- 172.21.15.88:0/2438324072 learned_addr learned my addr 172.21.15.88:0/2438324072 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:47.749 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.745+0000 7fe3a490e640 1 -- 172.21.15.88:0/2438324072 >> v2:172.21.15.94:3300/0 conn(0x7fe3a00b2b00 msgr2=0x7fe3a006c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:47.749 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.745+0000 7fe3977fe640 1 -- 172.21.15.88:0/2438324072 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2368915399 0 0) 0x7fe3a00ce4b0 con 0x7fe3a015ac00 2023-10-16T20:05:47.749 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.745+0000 7fe3a490e640 1 --2- 172.21.15.88:0/2438324072 >> v2:172.21.15.94:3300/0 conn(0x7fe3a00b2b00 0x7fe3a006c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:47.749 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.745+0000 7fe3a490e640 1 -- 172.21.15.88:0/2438324072 >> v1:172.21.15.94:6789/0 conn(0x7fe3a015ac00 legacy=0x7fe3a00b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:47.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.745+0000 7fe3a490e640 1 -- 172.21.15.88:0/2438324072 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a00ce9e0 con 0x7fe3a015a820 2023-10-16T20:05:47.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.745+0000 7fe3a510f640 1 --2- 172.21.15.88:0/2438324072 >> v2:172.21.15.94:3300/0 conn(0x7fe3a00b2b00 0x7fe3a006c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:05:47.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.749+0000 7fe3a490e640 1 --2- 172.21.15.88:0/2438324072 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3a015a820 0x7fe3a00cf1e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe39c0097c0 tx=0x7fe39c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffb06d8d3112ebd0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:47.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.749+0000 7fe3977fe640 1 -- 172.21.15.88:0/2438324072 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a006cea0 con 0x7fe3a015a820 2023-10-16T20:05:47.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.749+0000 7fe3977fe640 1 -- 172.21.15.88:0/2438324072 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39c0ab670 con 0x7fe3a015a820 2023-10-16T20:05:47.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.749+0000 7fe3977fe640 1 -- 172.21.15.88:0/2438324072 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c0aba00 con 0x7fe3a015a820 2023-10-16T20:05:47.752 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.749+0000 7fe3a6b72640 1 -- 172.21.15.88:0/2438324072 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3a015a820 msgr2=0x7fe3a00cf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:47.752 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.749+0000 7fe3a6b72640 1 --2- 172.21.15.88:0/2438324072 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3a015a820 0x7fe3a00cf1e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe39c0097c0 tx=0x7fe39c01c450 comp rx=0 tx=0).stop 2023-10-16T20:05:47.752 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.749+0000 7fe3a6b72640 1 -- 172.21.15.88:0/2438324072 shutdown_connections 2023-10-16T20:05:47.753 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.749+0000 7fe3a6b72640 1 --2- 172.21.15.88:0/2438324072 >> v2:172.21.15.94:3300/0 conn(0x7fe3a00b2b00 0x7fe3a006c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:47.753 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.749+0000 7fe3a6b72640 1 --2- 172.21.15.88:0/2438324072 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3a015a820 0x7fe3a00cf1e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:47.753 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.749+0000 7fe3a6b72640 1 -- 172.21.15.88:0/2438324072 >> 172.21.15.88:0/2438324072 conn(0x7fe3a00cd3f0 msgr2=0x7fe3a00cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:47.753 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.749+0000 7fe3a6b72640 1 -- 172.21.15.88:0/2438324072 shutdown_connections 2023-10-16T20:05:47.753 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.749+0000 7fe3a6b72640 1 -- 172.21.15.88:0/2438324072 wait complete. 2023-10-16T20:05:47.755 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.749+0000 7fe3a6b72640 1 Processor -- start 2023-10-16T20:05:47.755 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.749+0000 7fe3a6b72640 1 -- start start 2023-10-16T20:05:47.755 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.753+0000 7fe3a6b72640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3a00b2b00 0x7fe3a0150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:47.755 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.753+0000 7fe3a6b72640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3a015a820 0x7fe3a00b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:47.756 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.753+0000 7fe3a6b72640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3a015ac00 0x7fe3a00b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:47.756 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.753+0000 7fe3a6b72640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a0158ba0 con 0x7fe3a015a820 2023-10-16T20:05:47.756 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.753+0000 7fe3a6b72640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a0158e80 con 0x7fe3a015ac00 2023-10-16T20:05:47.756 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.753+0000 7fe3a6b72640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a0159160 con 0x7fe3a00b2b00 2023-10-16T20:05:47.756 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.753+0000 7fe397fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3a015a820 0x7fe3a00b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:47.757 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.753+0000 7fe397fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3a015a820 0x7fe3a00b11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48138/0 (socket says 172.21.15.88:48138) 2023-10-16T20:05:47.757 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.753+0000 7fe397fff640 1 -- 172.21.15.88:0/1038119746 learned_addr learned my addr 172.21.15.88:0/1038119746 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:47.757 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.753+0000 7fe3a490e640 1 --2- 172.21.15.88:0/1038119746 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3a00b2b00 0x7fe3a0150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:47.757 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.753+0000 7fe3a510f640 1 --2- 172.21.15.88:0/1038119746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3a015ac00 0x7fe3a00b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:47.757 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.753+0000 7fe397fff640 1 -- 172.21.15.88:0/1038119746 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3a00b2b00 msgr2=0x7fe3a0150180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:47.758 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.753+0000 7fe397fff640 1 --2- 172.21.15.88:0/1038119746 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3a00b2b00 0x7fe3a0150180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:47.758 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.753+0000 7fe397fff640 1 -- 172.21.15.88:0/1038119746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3a015ac00 msgr2=0x7fe3a00b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:47.758 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.753+0000 7fe397fff640 1 --2- 172.21.15.88:0/1038119746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3a015ac00 0x7fe3a00b16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:47.758 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.753+0000 7fe397fff640 1 -- 172.21.15.88:0/1038119746 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe398009a70 con 0x7fe3a015a820 2023-10-16T20:05:47.760 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.753+0000 7fe397fff640 1 --2- 172.21.15.88:0/1038119746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3a015a820 0x7fe3a00b11a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe39000c480 tx=0x7fe39000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:47.760 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.753+0000 7fe3957fa640 1 -- 172.21.15.88:0/1038119746 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3900074e0 con 0x7fe3a015a820 2023-10-16T20:05:47.760 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.753+0000 7fe3957fa640 1 -- 172.21.15.88:0/1038119746 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe390007660 con 0x7fe3a015a820 2023-10-16T20:05:47.760 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.753+0000 7fe3a6b72640 1 -- 172.21.15.88:0/1038119746 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe39c009420 con 0x7fe3a015a820 2023-10-16T20:05:47.760 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.753+0000 7fe3957fa640 1 -- 172.21.15.88:0/1038119746 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe390004a30 con 0x7fe3a015a820 2023-10-16T20:05:47.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.753+0000 7fe3a6b72640 1 -- 172.21.15.88:0/1038119746 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a0159720 con 0x7fe3a015a820 2023-10-16T20:05:47.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.753+0000 7fe3a6b72640 1 -- 172.21.15.88:0/1038119746 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a0159940 con 0x7fe3a015a820 2023-10-16T20:05:47.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.753+0000 7fe3957fa640 1 -- 172.21.15.88:0/1038119746 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe390004b90 con 0x7fe3a015a820 2023-10-16T20:05:47.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.757+0000 7fe3957fa640 1 --2- 172.21.15.88:0/1038119746 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe38c0664a0 0x7fe38c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:47.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.757+0000 7fe3957fa640 1 -- 172.21.15.88:0/1038119746 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe3900836d0 con 0x7fe3a015a820 2023-10-16T20:05:47.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.757+0000 7fe3a490e640 1 --2- 172.21.15.88:0/1038119746 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe38c0664a0 0x7fe38c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:47.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.757+0000 7fe3a490e640 1 --2- 172.21.15.88:0/1038119746 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe38c0664a0 0x7fe38c068960 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe39c0af000 tx=0x7fe39c0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:47.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.761+0000 7fe3957fa640 1 -- 172.21.15.88:0/1038119746 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe390050830 con 0x7fe3a015a820 2023-10-16T20:05:47.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.909+0000 7fe3a6b72640 1 -- 172.21.15.88:0/1038119746 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe3a004e9c0 con 0x7fe3a015a820 2023-10-16T20:05:47.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.909+0000 7fe3957fa640 1 -- 172.21.15.88:0/1038119746 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe390053e50 con 0x7fe3a015a820 2023-10-16T20:05:47.911 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:05:47.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.909+0000 7fe3a6b72640 1 -- 172.21.15.88:0/1038119746 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe38c0664a0 msgr2=0x7fe38c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:47.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.909+0000 7fe3a6b72640 1 --2- 172.21.15.88:0/1038119746 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe38c0664a0 0x7fe38c068960 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe39c0af000 tx=0x7fe39c0a7000 comp rx=0 tx=0).stop 2023-10-16T20:05:47.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.909+0000 7fe3a6b72640 1 -- 172.21.15.88:0/1038119746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3a015a820 msgr2=0x7fe3a00b11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:47.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.909+0000 7fe3a6b72640 1 --2- 172.21.15.88:0/1038119746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3a015a820 0x7fe3a00b11a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe39000c480 tx=0x7fe39000c940 comp rx=0 tx=0).stop 2023-10-16T20:05:47.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.909+0000 7fe3a6b72640 1 -- 172.21.15.88:0/1038119746 shutdown_connections 2023-10-16T20:05:47.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.909+0000 7fe3a6b72640 1 --2- 172.21.15.88:0/1038119746 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe38c0664a0 0x7fe38c068960 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:47.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.909+0000 7fe3a6b72640 1 --2- 172.21.15.88:0/1038119746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3a015ac00 0x7fe3a00b16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:47.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.909+0000 7fe3a6b72640 1 --2- 172.21.15.88:0/1038119746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3a015a820 0x7fe3a00b11a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:47.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.909+0000 7fe3a6b72640 1 --2- 172.21.15.88:0/1038119746 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe3a00b2b00 0x7fe3a0150180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:47.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.909+0000 7fe3a6b72640 1 -- 172.21.15.88:0/1038119746 >> 172.21.15.88:0/1038119746 conn(0x7fe3a00cd3f0 msgr2=0x7fe3a00cc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:47.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.909+0000 7fe3a6b72640 1 -- 172.21.15.88:0/1038119746 shutdown_connections 2023-10-16T20:05:47.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:47.909+0000 7fe3a6b72640 1 -- 172.21.15.88:0/1038119746 wait complete. 2023-10-16T20:05:47.931 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T20:05:48.072 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.068+0000 7ff487f65640 1 Processor -- start 2023-10-16T20:05:48.072 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.068+0000 7ff487f65640 1 -- start start 2023-10-16T20:05:48.074 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.068+0000 7ff487f65640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff480164570 0x7ff4800af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:48.074 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.068+0000 7ff487f65640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff4800afe20 0x7ff4800addf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:48.075 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.068+0000 7ff487f65640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4800ae840 con 0x7ff4800ae330 2023-10-16T20:05:48.075 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.068+0000 7ff487f65640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4800aeb20 con 0x7ff4800afe20 2023-10-16T20:05:48.075 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.068+0000 7ff487f65640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4800aee00 con 0x7ff480164570 2023-10-16T20:05:48.075 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.068+0000 7ff485500640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff4800afe20 0x7ff4800addf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:48.075 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.068+0000 7ff485500640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff4800afe20 0x7ff4800addf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48148/0 (socket says 172.21.15.88:48148) 2023-10-16T20:05:48.076 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.068+0000 7ff485500640 1 -- 172.21.15.88:0/3771923528 learned_addr learned my addr 172.21.15.88:0/3771923528 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:48.076 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.068+0000 7ff485d01640 1 --2- 172.21.15.88:0/3771923528 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff480164570 0x7ff4800af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:48.076 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.068+0000 7ff485500640 1 -- 172.21.15.88:0/3771923528 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff480164570 msgr2=0x7ff4800af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:48.076 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.068+0000 7ff485500640 1 --2- 172.21.15.88:0/3771923528 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff480164570 0x7ff4800af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:48.076 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.068+0000 7ff485500640 1 -- 172.21.15.88:0/3771923528 >> v1:172.21.15.94:6789/0 conn(0x7ff4800ae330 legacy=0x7ff480070250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:48.076 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.072+0000 7ff485500640 1 -- 172.21.15.88:0/3771923528 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4800af0e0 con 0x7ff4800afe20 2023-10-16T20:05:48.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.072+0000 7ff485500640 1 --2- 172.21.15.88:0/3771923528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4800afe20 0x7ff4800addf0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff470009870 tx=0x7ff47001ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e5dd45d2acd5775 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:48.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.072+0000 7ff484cff640 1 -- 172.21.15.88:0/3771923528 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4700a8a30 con 0x7ff4800afe20 2023-10-16T20:05:48.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.072+0000 7ff484cff640 1 -- 172.21.15.88:0/3771923528 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4700a8b90 con 0x7ff4800afe20 2023-10-16T20:05:48.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.072+0000 7ff484cff640 1 -- 172.21.15.88:0/3771923528 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4700b0440 con 0x7ff4800afe20 2023-10-16T20:05:48.079 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.072+0000 7ff487f65640 1 -- 172.21.15.88:0/3771923528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4800afe20 msgr2=0x7ff4800addf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:48.079 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.072+0000 7ff487f65640 1 --2- 172.21.15.88:0/3771923528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4800afe20 0x7ff4800addf0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff470009870 tx=0x7ff47001ed30 comp rx=0 tx=0).stop 2023-10-16T20:05:48.079 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.072+0000 7ff487f65640 1 -- 172.21.15.88:0/3771923528 shutdown_connections 2023-10-16T20:05:48.079 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.072+0000 7ff487f65640 1 --2- 172.21.15.88:0/3771923528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4800afe20 0x7ff4800addf0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:48.079 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.072+0000 7ff487f65640 1 --2- 172.21.15.88:0/3771923528 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff480164570 0x7ff4800af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:48.079 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.072+0000 7ff487f65640 1 -- 172.21.15.88:0/3771923528 >> 172.21.15.88:0/3771923528 conn(0x7ff4800cb080 msgr2=0x7ff4800cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:48.080 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.072+0000 7ff487f65640 1 -- 172.21.15.88:0/3771923528 shutdown_connections 2023-10-16T20:05:48.080 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.076+0000 7ff487f65640 1 -- 172.21.15.88:0/3771923528 wait complete. 2023-10-16T20:05:48.082 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.076+0000 7ff487f65640 1 Processor -- start 2023-10-16T20:05:48.082 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.076+0000 7ff487f65640 1 -- start start 2023-10-16T20:05:48.082 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.076+0000 7ff487f65640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4800ae330 0x7ff48014db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:48.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.076+0000 7ff487f65640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4800afe20 0x7ff480162910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:48.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.076+0000 7ff487f65640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff480164570 0x7ff480162e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:48.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.076+0000 7ff487f65640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4800a9f90 con 0x7ff480164570 2023-10-16T20:05:48.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.076+0000 7ff487f65640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4800aa270 con 0x7ff4800afe20 2023-10-16T20:05:48.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.076+0000 7ff487f65640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4800aa550 con 0x7ff4800ae330 2023-10-16T20:05:48.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.076+0000 7ff486502640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff480164570 0x7ff480162e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:48.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.076+0000 7ff486502640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff480164570 0x7ff480162e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48168/0 (socket says 172.21.15.88:48168) 2023-10-16T20:05:48.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.076+0000 7ff486502640 1 -- 172.21.15.88:0/291702381 learned_addr learned my addr 172.21.15.88:0/291702381 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:48.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.076+0000 7ff485500640 1 --2- 172.21.15.88:0/291702381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4800afe20 0x7ff480162910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:48.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.076+0000 7ff485d01640 1 --2- 172.21.15.88:0/291702381 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4800ae330 0x7ff48014db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:48.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.076+0000 7ff486502640 1 -- 172.21.15.88:0/291702381 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4800ae330 msgr2=0x7ff48014db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:48.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.076+0000 7ff486502640 1 --2- 172.21.15.88:0/291702381 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4800ae330 0x7ff48014db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:48.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.076+0000 7ff486502640 1 -- 172.21.15.88:0/291702381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4800afe20 msgr2=0x7ff480162910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:48.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.076+0000 7ff486502640 1 --2- 172.21.15.88:0/291702381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4800afe20 0x7ff480162910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:48.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.076+0000 7ff486502640 1 -- 172.21.15.88:0/291702381 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4700094d0 con 0x7ff480164570 2023-10-16T20:05:48.086 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.076+0000 7ff486502640 1 --2- 172.21.15.88:0/291702381 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff480164570 0x7ff480162e50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff47800c770 tx=0x7ff47800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:48.086 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.076+0000 7ff4767fc640 1 -- 172.21.15.88:0/291702381 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff478003b80 con 0x7ff480164570 2023-10-16T20:05:48.086 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.076+0000 7ff4767fc640 1 -- 172.21.15.88:0/291702381 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff478003ce0 con 0x7ff480164570 2023-10-16T20:05:48.086 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.076+0000 7ff487f65640 1 -- 172.21.15.88:0/291702381 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4800aa830 con 0x7ff480164570 2023-10-16T20:05:48.086 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.076+0000 7ff4767fc640 1 -- 172.21.15.88:0/291702381 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47800aa90 con 0x7ff480164570 2023-10-16T20:05:48.087 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.076+0000 7ff487f65640 1 -- 172.21.15.88:0/291702381 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4800ab030 con 0x7ff480164570 2023-10-16T20:05:48.088 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.080+0000 7ff487f65640 1 -- 172.21.15.88:0/291702381 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4800aad50 con 0x7ff480164570 2023-10-16T20:05:48.088 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.080+0000 7ff4767fc640 1 -- 172.21.15.88:0/291702381 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff47800abf0 con 0x7ff480164570 2023-10-16T20:05:48.088 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.080+0000 7ff4767fc640 1 --2- 172.21.15.88:0/291702381 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff4640664a0 0x7ff464068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:48.088 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.080+0000 7ff4767fc640 1 -- 172.21.15.88:0/291702381 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff4780840f0 con 0x7ff480164570 2023-10-16T20:05:48.089 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.080+0000 7ff485d01640 1 --2- 172.21.15.88:0/291702381 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff4640664a0 0x7ff464068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:48.089 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.080+0000 7ff485d01640 1 --2- 172.21.15.88:0/291702381 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff4640664a0 0x7ff464068960 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff47c009ac0 tx=0x7ff47c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:48.089 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.084+0000 7ff4767fc640 1 -- 172.21.15.88:0/291702381 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff4800aad50 con 0x7ff480164570 2023-10-16T20:05:48.237 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.232+0000 7ff487f65640 1 -- 172.21.15.88:0/291702381 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff4800aad50 con 0x7ff480164570 2023-10-16T20:05:48.237 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.232+0000 7ff4767fc640 1 -- 172.21.15.88:0/291702381 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff4800aad50 con 0x7ff480164570 2023-10-16T20:05:48.238 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2023-10-16T20:05:48.239 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.236+0000 7ff487f65640 1 -- 172.21.15.88:0/291702381 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff4640664a0 msgr2=0x7ff464068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:48.239 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.236+0000 7ff487f65640 1 --2- 172.21.15.88:0/291702381 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff4640664a0 0x7ff464068960 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff47c009ac0 tx=0x7ff47c009190 comp rx=0 tx=0).stop 2023-10-16T20:05:48.239 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.236+0000 7ff487f65640 1 -- 172.21.15.88:0/291702381 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff480164570 msgr2=0x7ff480162e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:48.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.236+0000 7ff487f65640 1 --2- 172.21.15.88:0/291702381 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff480164570 0x7ff480162e50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff47800c770 tx=0x7ff47800cc30 comp rx=0 tx=0).stop 2023-10-16T20:05:48.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.236+0000 7ff487f65640 1 -- 172.21.15.88:0/291702381 shutdown_connections 2023-10-16T20:05:48.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.236+0000 7ff487f65640 1 --2- 172.21.15.88:0/291702381 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff4640664a0 0x7ff464068960 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:48.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.236+0000 7ff487f65640 1 --2- 172.21.15.88:0/291702381 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff480164570 0x7ff480162e50 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:48.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.236+0000 7ff487f65640 1 --2- 172.21.15.88:0/291702381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4800afe20 0x7ff480162910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:48.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.236+0000 7ff487f65640 1 --2- 172.21.15.88:0/291702381 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4800ae330 0x7ff48014db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:48.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.236+0000 7ff487f65640 1 -- 172.21.15.88:0/291702381 >> 172.21.15.88:0/291702381 conn(0x7ff4800cb080 msgr2=0x7ff4800cca40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:48.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.236+0000 7ff487f65640 1 -- 172.21.15.88:0/291702381 shutdown_connections 2023-10-16T20:05:48.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.236+0000 7ff487f65640 1 -- 172.21.15.88:0/291702381 wait complete. 2023-10-16T20:05:48.261 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T20:05:48.401 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.396+0000 7f9262267640 1 Processor -- start 2023-10-16T20:05:48.401 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.396+0000 7f9262267640 1 -- start start 2023-10-16T20:05:48.403 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.400+0000 7f9262267640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f925c0b1360 0x7f925c070ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:48.403 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.400+0000 7f9262267640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f925c0b0030 con 0x7f925c15a730 2023-10-16T20:05:48.403 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.400+0000 7f9262267640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f925c0b0310 con 0x7f925c0cb5f0 2023-10-16T20:05:48.404 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.400+0000 7f9262267640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f925c0b0cb0 con 0x7f925c0b1360 2023-10-16T20:05:48.404 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.400+0000 7f9261265640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f925c15a730 0x7f925c0caee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:34700/0 (socket says 172.21.15.88:34700) 2023-10-16T20:05:48.404 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.400+0000 7f9261265640 1 -- 172.21.15.88:0/3030701922 learned_addr learned my addr 172.21.15.88:0/3030701922 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:48.404 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.400+0000 7f9261a66640 1 --2- 172.21.15.88:0/3030701922 >> v2:172.21.15.94:3300/0 conn(0x7f925c0b1360 0x7f925c070ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:48.404 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.400+0000 7f9253fff640 1 -- 172.21.15.88:0/3030701922 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1196928302 0 0) 0x7f925c0b0030 con 0x7f925c15a730 2023-10-16T20:05:48.405 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.400+0000 7f9253fff640 1 -- 172.21.15.88:0/3030701922 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9248003530 con 0x7f925c15a730 2023-10-16T20:05:48.405 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.400+0000 7f9261a66640 1 -- 172.21.15.88:0/3030701922 >> v1:172.21.15.94:6789/0 conn(0x7f925c0cb5f0 legacy=0x7f925c15e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:48.405 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.400+0000 7f9261a66640 1 -- 172.21.15.88:0/3030701922 >> v1:172.21.15.88:6789/0 conn(0x7f925c15a730 legacy=0x7f925c0caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:48.405 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.400+0000 7f9261a66640 1 -- 172.21.15.88:0/3030701922 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f925c0b05f0 con 0x7f925c0b1360 2023-10-16T20:05:48.406 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.404+0000 7f9261a66640 1 --2- 172.21.15.88:0/3030701922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f925c0b1360 0x7f925c070ab0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9254009d90 tx=0x7f925401c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5b227390f953f39 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:48.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.404+0000 7f9253fff640 1 -- 172.21.15.88:0/3030701922 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92540a5880 con 0x7f925c0b1360 2023-10-16T20:05:48.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.404+0000 7f9253fff640 1 -- 172.21.15.88:0/3030701922 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92540a59e0 con 0x7f925c0b1360 2023-10-16T20:05:48.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.404+0000 7f9253fff640 1 -- 172.21.15.88:0/3030701922 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92540a5d70 con 0x7f925c0b1360 2023-10-16T20:05:48.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.404+0000 7f9262267640 1 -- 172.21.15.88:0/3030701922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f925c0b1360 msgr2=0x7f925c070ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:48.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.404+0000 7f9262267640 1 --2- 172.21.15.88:0/3030701922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f925c0b1360 0x7f925c070ab0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9254009d90 tx=0x7f925401c800 comp rx=0 tx=0).stop 2023-10-16T20:05:48.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.404+0000 7f9262267640 1 -- 172.21.15.88:0/3030701922 shutdown_connections 2023-10-16T20:05:48.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.404+0000 7f9262267640 1 --2- 172.21.15.88:0/3030701922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f925c0b1360 0x7f925c070ab0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:48.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.404+0000 7f9262267640 1 -- 172.21.15.88:0/3030701922 >> 172.21.15.88:0/3030701922 conn(0x7f925c06a4c0 msgr2=0x7f925c06c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:48.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.404+0000 7f9262267640 1 -- 172.21.15.88:0/3030701922 shutdown_connections 2023-10-16T20:05:48.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.404+0000 7f9262267640 1 -- 172.21.15.88:0/3030701922 wait complete. 2023-10-16T20:05:48.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.404+0000 7f9262267640 1 Processor -- start 2023-10-16T20:05:48.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.404+0000 7f9262267640 1 -- start start 2023-10-16T20:05:48.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.404+0000 7f9262267640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f925c0b1360 0x7f925c1f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:48.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.404+0000 7f9262267640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f925c0cb5f0 0x7f925c0adf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:48.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.404+0000 7f9262267640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f925c15a730 0x7f925c0ae460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:48.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.404+0000 7f9262267640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925c158520 con 0x7f925c0b1360 2023-10-16T20:05:48.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.404+0000 7f9262267640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925c157930 con 0x7f925c0cb5f0 2023-10-16T20:05:48.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.404+0000 7f9262267640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f925c157bf0 con 0x7f925c15a730 2023-10-16T20:05:48.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.408+0000 7f9261265640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f925c0b1360 0x7f925c1f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:48.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.408+0000 7f9261265640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f925c0b1360 0x7f925c1f5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48170/0 (socket says 172.21.15.88:48170) 2023-10-16T20:05:48.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.408+0000 7f9261265640 1 -- 172.21.15.88:0/1448093122 learned_addr learned my addr 172.21.15.88:0/1448093122 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:48.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.408+0000 7f9261a66640 1 --2- 172.21.15.88:0/1448093122 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f925c15a730 0x7f925c0ae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:48.414 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.408+0000 7f9260a64640 1 --2- 172.21.15.88:0/1448093122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f925c0cb5f0 0x7f925c0adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:48.414 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.408+0000 7f9261265640 1 -- 172.21.15.88:0/1448093122 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f925c15a730 msgr2=0x7f925c0ae460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:48.414 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.408+0000 7f9261265640 1 --2- 172.21.15.88:0/1448093122 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f925c15a730 0x7f925c0ae460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:48.414 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.408+0000 7f9261265640 1 -- 172.21.15.88:0/1448093122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f925c0cb5f0 msgr2=0x7f925c0adf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:48.414 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.408+0000 7f9261265640 1 --2- 172.21.15.88:0/1448093122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f925c0cb5f0 0x7f925c0adf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:48.415 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.408+0000 7f9261265640 1 -- 172.21.15.88:0/1448093122 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9254009a70 con 0x7f925c0b1360 2023-10-16T20:05:48.415 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.408+0000 7f9261265640 1 --2- 172.21.15.88:0/1448093122 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f925c0b1360 0x7f925c1f5b40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f925800c4d0 tx=0x7f925800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:48.415 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.408+0000 7f9251ffb640 1 -- 172.21.15.88:0/1448093122 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9258005cf0 con 0x7f925c0b1360 2023-10-16T20:05:48.415 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.408+0000 7f9251ffb640 1 -- 172.21.15.88:0/1448093122 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9258005e50 con 0x7f925c0b1360 2023-10-16T20:05:48.415 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.408+0000 7f9251ffb640 1 -- 172.21.15.88:0/1448093122 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9258007270 con 0x7f925c0b1360 2023-10-16T20:05:48.416 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.408+0000 7f9262267640 1 -- 172.21.15.88:0/1448093122 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f925c157ed0 con 0x7f925c0b1360 2023-10-16T20:05:48.417 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.408+0000 7f9262267640 1 -- 172.21.15.88:0/1448093122 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f925c1f6080 con 0x7f925c0b1360 2023-10-16T20:05:48.417 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.408+0000 7f9262267640 1 -- 172.21.15.88:0/1448093122 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f925c0674f0 con 0x7f925c0b1360 2023-10-16T20:05:48.417 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.408+0000 7f9251ffb640 1 -- 172.21.15.88:0/1448093122 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f92580073d0 con 0x7f925c0b1360 2023-10-16T20:05:48.417 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.412+0000 7f9251ffb640 1 --2- 172.21.15.88:0/1448093122 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f92400664d0 0x7f9240068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:48.417 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.412+0000 7f9251ffb640 1 -- 172.21.15.88:0/1448093122 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9258083700 con 0x7f925c0b1360 2023-10-16T20:05:48.418 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.412+0000 7f9260a64640 1 --2- 172.21.15.88:0/1448093122 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f92400664d0 0x7f9240068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:48.418 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.412+0000 7f9260a64640 1 --2- 172.21.15.88:0/1448093122 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f92400664d0 0x7f9240068990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f925c0b3f70 tx=0x7f924c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:48.418 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.412+0000 7f9251ffb640 1 -- 172.21.15.88:0/1448093122 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9258050870 con 0x7f925c0b1360 2023-10-16T20:05:48.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.564+0000 7f9262267640 1 -- 172.21.15.88:0/1448093122 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f925c0ce0d0 con 0x7f925c0b1360 2023-10-16T20:05:48.569 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.564+0000 7f9251ffb640 1 -- 172.21.15.88:0/1448093122 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9258053e90 con 0x7f925c0b1360 2023-10-16T20:05:48.569 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:05:48.571 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.568+0000 7f9262267640 1 -- 172.21.15.88:0/1448093122 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f92400664d0 msgr2=0x7f9240068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:48.571 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.568+0000 7f9262267640 1 --2- 172.21.15.88:0/1448093122 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f92400664d0 0x7f9240068990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f925c0b3f70 tx=0x7f924c009040 comp rx=0 tx=0).stop 2023-10-16T20:05:48.571 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.568+0000 7f9262267640 1 -- 172.21.15.88:0/1448093122 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f925c0b1360 msgr2=0x7f925c1f5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:48.571 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.568+0000 7f9262267640 1 --2- 172.21.15.88:0/1448093122 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f925c0b1360 0x7f925c1f5b40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f925800c4d0 tx=0x7f925800c990 comp rx=0 tx=0).stop 2023-10-16T20:05:48.572 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.568+0000 7f9262267640 1 -- 172.21.15.88:0/1448093122 shutdown_connections 2023-10-16T20:05:48.572 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.568+0000 7f9262267640 1 --2- 172.21.15.88:0/1448093122 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f92400664d0 0x7f9240068990 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:48.572 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.568+0000 7f9262267640 1 --2- 172.21.15.88:0/1448093122 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f925c15a730 0x7f925c0ae460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:48.572 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.568+0000 7f9262267640 1 --2- 172.21.15.88:0/1448093122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f925c0cb5f0 0x7f925c0adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:48.572 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.568+0000 7f9262267640 1 --2- 172.21.15.88:0/1448093122 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f925c0b1360 0x7f925c1f5b40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:48.573 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.568+0000 7f9262267640 1 -- 172.21.15.88:0/1448093122 >> 172.21.15.88:0/1448093122 conn(0x7f925c06a4c0 msgr2=0x7f925c0b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:48.573 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.568+0000 7f9262267640 1 -- 172.21.15.88:0/1448093122 shutdown_connections 2023-10-16T20:05:48.573 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.568+0000 7f9262267640 1 -- 172.21.15.88:0/1448093122 wait complete. 2023-10-16T20:05:48.591 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T20:05:48.728 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.724+0000 7febada9e640 1 Processor -- start 2023-10-16T20:05:48.729 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.724+0000 7febada9e640 1 -- start start 2023-10-16T20:05:48.730 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.724+0000 7febada9e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7feba8069e90 0x7feba806a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:48.730 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.724+0000 7febada9e640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feba815e400 con 0x7feba80adf80 2023-10-16T20:05:48.730 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.724+0000 7febada9e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feba806a7b0 con 0x7feba815d480 2023-10-16T20:05:48.730 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.724+0000 7febada9e640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feba8159e40 con 0x7feba8069e90 2023-10-16T20:05:48.731 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.728+0000 7feba6ffd640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7feba80adf80 0x7feba815a830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:34708/0 (socket says 172.21.15.88:34708) 2023-10-16T20:05:48.731 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.728+0000 7feba6ffd640 1 -- 172.21.15.88:0/140538142 learned_addr learned my addr 172.21.15.88:0/140538142 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:48.731 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.728+0000 7feba77fe640 1 --2- 172.21.15.88:0/140538142 >> v2:172.21.15.94:3300/0 conn(0x7feba8069e90 0x7feba806a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:48.732 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.728+0000 7feba67fc640 1 -- 172.21.15.88:0/140538142 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 497565639 0 0) 0x7feba815e400 con 0x7feba80adf80 2023-10-16T20:05:48.732 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.728+0000 7feba67fc640 1 -- 172.21.15.88:0/140538142 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb90003530 con 0x7feba80adf80 2023-10-16T20:05:48.732 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.728+0000 7feba77fe640 1 -- 172.21.15.88:0/140538142 >> v1:172.21.15.94:6789/0 conn(0x7feba815d480 legacy=0x7feba80afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:48.733 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.728+0000 7feba67fc640 1 -- 172.21.15.88:0/140538142 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1155181459 0 0) 0x7feba806a7b0 con 0x7feba815d480 2023-10-16T20:05:48.733 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.728+0000 7feba77fe640 1 -- 172.21.15.88:0/140538142 >> v1:172.21.15.88:6789/0 conn(0x7feba80adf80 legacy=0x7feba815a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:48.733 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.728+0000 7feba77fe640 1 -- 172.21.15.88:0/140538142 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba815de40 con 0x7feba8069e90 2023-10-16T20:05:48.734 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.728+0000 7feba77fe640 1 --2- 172.21.15.88:0/140538142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feba8069e90 0x7feba806a270 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7feb980097c0 tx=0x7feb9801c2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=404e041102a32879 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:48.734 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.728+0000 7feba67fc640 1 -- 172.21.15.88:0/140538142 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb980ab600 con 0x7feba8069e90 2023-10-16T20:05:48.734 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.728+0000 7feba67fc640 1 -- 172.21.15.88:0/140538142 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb980a7050 con 0x7feba8069e90 2023-10-16T20:05:48.734 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.728+0000 7feba67fc640 1 -- 172.21.15.88:0/140538142 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb980ab980 con 0x7feba8069e90 2023-10-16T20:05:48.735 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.732+0000 7febada9e640 1 -- 172.21.15.88:0/140538142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feba8069e90 msgr2=0x7feba806a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:48.735 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.732+0000 7febada9e640 1 --2- 172.21.15.88:0/140538142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feba8069e90 0x7feba806a270 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7feb980097c0 tx=0x7feb9801c2e0 comp rx=0 tx=0).stop 2023-10-16T20:05:48.736 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.732+0000 7febada9e640 1 -- 172.21.15.88:0/140538142 shutdown_connections 2023-10-16T20:05:48.736 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.732+0000 7febada9e640 1 --2- 172.21.15.88:0/140538142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feba8069e90 0x7feba806a270 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:48.736 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.732+0000 7febada9e640 1 -- 172.21.15.88:0/140538142 >> 172.21.15.88:0/140538142 conn(0x7feba80ce5e0 msgr2=0x7feba80ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:48.736 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.732+0000 7febada9e640 1 -- 172.21.15.88:0/140538142 shutdown_connections 2023-10-16T20:05:48.737 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.732+0000 7febada9e640 1 -- 172.21.15.88:0/140538142 wait complete. 2023-10-16T20:05:48.738 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.732+0000 7febada9e640 1 Processor -- start 2023-10-16T20:05:48.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.732+0000 7febada9e640 1 -- start start 2023-10-16T20:05:48.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.732+0000 7febada9e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feba8069e90 0x7feba813b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:48.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.732+0000 7febada9e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feba80adf80 0x7feba8147c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:48.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.732+0000 7febada9e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feba815d480 0x7feba814b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:48.740 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.732+0000 7febada9e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba8146920 con 0x7feba8069e90 2023-10-16T20:05:48.740 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.732+0000 7febada9e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba813bde0 con 0x7feba815d480 2023-10-16T20:05:48.740 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.732+0000 7febada9e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba8146080 con 0x7feba80adf80 2023-10-16T20:05:48.740 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.732+0000 7feba77fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feba8069e90 0x7feba813b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:48.740 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.732+0000 7feba77fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feba8069e90 0x7feba813b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48176/0 (socket says 172.21.15.88:48176) 2023-10-16T20:05:48.741 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.732+0000 7feba77fe640 1 -- 172.21.15.88:0/3292026717 learned_addr learned my addr 172.21.15.88:0/3292026717 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:48.741 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.732+0000 7feba6ffd640 1 --2- 172.21.15.88:0/3292026717 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feba80adf80 0x7feba8147c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:48.741 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.732+0000 7feba7fff640 1 --2- 172.21.15.88:0/3292026717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feba815d480 0x7feba814b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:48.741 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.732+0000 7feba77fe640 1 -- 172.21.15.88:0/3292026717 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feba80adf80 msgr2=0x7feba8147c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:48.741 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.732+0000 7feba77fe640 1 --2- 172.21.15.88:0/3292026717 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feba80adf80 0x7feba8147c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:48.742 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.732+0000 7feba77fe640 1 -- 172.21.15.88:0/3292026717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feba815d480 msgr2=0x7feba814b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:48.742 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.732+0000 7feba77fe640 1 --2- 172.21.15.88:0/3292026717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feba815d480 0x7feba814b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:48.742 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.732+0000 7feba77fe640 1 -- 172.21.15.88:0/3292026717 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb98009420 con 0x7feba8069e90 2023-10-16T20:05:48.742 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.732+0000 7feba7fff640 1 --2- 172.21.15.88:0/3292026717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feba815d480 0x7feba814b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:48.742 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.736+0000 7feba77fe640 1 --2- 172.21.15.88:0/3292026717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feba8069e90 0x7feba813b610 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7feb980097c0 tx=0x7feb9801ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:48.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.736+0000 7feb8ffff640 1 -- 172.21.15.88:0/3292026717 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb980abd80 con 0x7feba8069e90 2023-10-16T20:05:48.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.736+0000 7feb8ffff640 1 -- 172.21.15.88:0/3292026717 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb98004020 con 0x7feba8069e90 2023-10-16T20:05:48.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.736+0000 7febada9e640 1 -- 172.21.15.88:0/3292026717 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feba8146360 con 0x7feba8069e90 2023-10-16T20:05:48.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.736+0000 7feb8ffff640 1 -- 172.21.15.88:0/3292026717 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb980043d0 con 0x7feba8069e90 2023-10-16T20:05:48.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.736+0000 7febada9e640 1 -- 172.21.15.88:0/3292026717 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba813d6f0 con 0x7feba8069e90 2023-10-16T20:05:48.744 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.736+0000 7febada9e640 1 -- 172.21.15.88:0/3292026717 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feba813bc40 con 0x7feba8069e90 2023-10-16T20:05:48.745 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.736+0000 7feb8ffff640 1 -- 172.21.15.88:0/3292026717 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7feb98003620 con 0x7feba8069e90 2023-10-16T20:05:48.745 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.736+0000 7feb8ffff640 1 --2- 172.21.15.88:0/3292026717 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7feb940664a0 0x7feb94068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:48.745 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.736+0000 7feb8ffff640 1 -- 172.21.15.88:0/3292026717 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7feb9811e830 con 0x7feba8069e90 2023-10-16T20:05:48.745 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.736+0000 7feba6ffd640 1 --2- 172.21.15.88:0/3292026717 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7feb940664a0 0x7feb94068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:48.746 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.740+0000 7feba6ffd640 1 --2- 172.21.15.88:0/3292026717 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7feb940664a0 0x7feb94068960 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7feba814fdc0 tx=0x7feb9c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:48.746 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.740+0000 7feb8ffff640 1 -- 172.21.15.88:0/3292026717 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7feba813bc40 con 0x7feba8069e90 2023-10-16T20:05:48.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.904+0000 7febada9e640 1 -- 172.21.15.88:0/3292026717 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7feba813bc40 con 0x7feba8069e90 2023-10-16T20:05:48.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.904+0000 7feb8ffff640 1 -- 172.21.15.88:0/3292026717 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7feba813bc40 con 0x7feba8069e90 2023-10-16T20:05:48.908 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2023-10-16T20:05:48.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.904+0000 7febada9e640 1 -- 172.21.15.88:0/3292026717 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7feb940664a0 msgr2=0x7feb94068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:48.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.904+0000 7febada9e640 1 --2- 172.21.15.88:0/3292026717 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7feb940664a0 0x7feb94068960 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7feba814fdc0 tx=0x7feb9c009040 comp rx=0 tx=0).stop 2023-10-16T20:05:48.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.904+0000 7febada9e640 1 -- 172.21.15.88:0/3292026717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feba8069e90 msgr2=0x7feba813b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:48.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.904+0000 7febada9e640 1 --2- 172.21.15.88:0/3292026717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feba8069e90 0x7feba813b610 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7feb980097c0 tx=0x7feb9801ab20 comp rx=0 tx=0).stop 2023-10-16T20:05:48.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.904+0000 7febada9e640 1 -- 172.21.15.88:0/3292026717 shutdown_connections 2023-10-16T20:05:48.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.904+0000 7febada9e640 1 --2- 172.21.15.88:0/3292026717 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7feb940664a0 0x7feb94068960 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:48.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.904+0000 7febada9e640 1 --2- 172.21.15.88:0/3292026717 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feba80adf80 0x7feba8147c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:48.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.904+0000 7febada9e640 1 --2- 172.21.15.88:0/3292026717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feba815d480 0x7feba814b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:48.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.904+0000 7febada9e640 1 --2- 172.21.15.88:0/3292026717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feba8069e90 0x7feba813b610 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:48.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.904+0000 7febada9e640 1 -- 172.21.15.88:0/3292026717 >> 172.21.15.88:0/3292026717 conn(0x7feba80ce5e0 msgr2=0x7feba815ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:48.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.904+0000 7febada9e640 1 -- 172.21.15.88:0/3292026717 shutdown_connections 2023-10-16T20:05:48.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:48.904+0000 7febada9e640 1 -- 172.21.15.88:0/3292026717 wait complete. 2023-10-16T20:05:48.929 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi088.front.sepia.ceph.com... 2023-10-16T20:05:48.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:48.930 DEBUG:teuthology.orchestra.run.smithi088:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-16T20:05:48.951 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:05:48.951 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2023-10-16T20:05:48.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:48.952 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T20:05:49.003 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:05:49.003 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:05:49.013 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:05:49.014 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:05:49.069 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:05:49.206 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.200+0000 7fd9cd3d9640 1 Processor -- start 2023-10-16T20:05:49.206 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.204+0000 7fd9cd3d9640 1 -- start start 2023-10-16T20:05:49.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.204+0000 7fd9cd3d9640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd9c815d9f0 0x7fd9c8161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:49.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.204+0000 7fd9cd3d9640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd9c80d26c0 0x7fd9c80cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:49.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.204+0000 7fd9cd3d9640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9c80b87e0 con 0x7fd9c80d22e0 2023-10-16T20:05:49.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.204+0000 7fd9cd3d9640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9c815df10 con 0x7fd9c80d26c0 2023-10-16T20:05:49.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.204+0000 7fd9cd3d9640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9c815e1f0 con 0x7fd9c815d9f0 2023-10-16T20:05:49.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.204+0000 7fd9c77fe640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd9c80d26c0 0x7fd9c80cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:49.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.204+0000 7fd9c77fe640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd9c80d26c0 0x7fd9c80cf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48178/0 (socket says 172.21.15.88:48178) 2023-10-16T20:05:49.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.204+0000 7fd9c77fe640 1 -- 172.21.15.88:0/1264487192 learned_addr learned my addr 172.21.15.88:0/1264487192 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:49.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.204+0000 7fd9be7fc640 1 --2- 172.21.15.88:0/1264487192 >> v2:172.21.15.94:3300/0 conn(0x7fd9c815d9f0 0x7fd9c8161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:49.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.204+0000 7fd9c77fe640 1 -- 172.21.15.88:0/1264487192 >> v2:172.21.15.94:3300/0 conn(0x7fd9c815d9f0 msgr2=0x7fd9c8161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:49.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.204+0000 7fd9c77fe640 1 --2- 172.21.15.88:0/1264487192 >> v2:172.21.15.94:3300/0 conn(0x7fd9c815d9f0 0x7fd9c8161450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:49.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.204+0000 7fd9c77fe640 1 -- 172.21.15.88:0/1264487192 >> v1:172.21.15.94:6789/0 conn(0x7fd9c80d22e0 legacy=0x7fd9c815d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:49.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.204+0000 7fd9c77fe640 1 -- 172.21.15.88:0/1264487192 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c8160970 con 0x7fd9c80d26c0 2023-10-16T20:05:49.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.208+0000 7fd9c77fe640 1 --2- 172.21.15.88:0/1264487192 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9c80d26c0 0x7fd9c80cf2e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8072520 tx=0x7fd9c001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e4b1ea3b288086 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:49.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.208+0000 7fd9c67fc640 1 -- 172.21.15.88:0/1264487192 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c00ab640 con 0x7fd9c80d26c0 2023-10-16T20:05:49.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.208+0000 7fd9c67fc640 1 -- 172.21.15.88:0/1264487192 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c00a7070 con 0x7fd9c80d26c0 2023-10-16T20:05:49.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.208+0000 7fd9c67fc640 1 -- 172.21.15.88:0/1264487192 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c00ab9c0 con 0x7fd9c80d26c0 2023-10-16T20:05:49.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.208+0000 7fd9cd3d9640 1 -- 172.21.15.88:0/1264487192 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9c80d26c0 msgr2=0x7fd9c80cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:49.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.208+0000 7fd9cd3d9640 1 --2- 172.21.15.88:0/1264487192 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9c80d26c0 0x7fd9c80cf2e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8072520 tx=0x7fd9c001c440 comp rx=0 tx=0).stop 2023-10-16T20:05:49.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.208+0000 7fd9cd3d9640 1 -- 172.21.15.88:0/1264487192 shutdown_connections 2023-10-16T20:05:49.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.208+0000 7fd9cd3d9640 1 --2- 172.21.15.88:0/1264487192 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9c80d26c0 0x7fd9c80cf2e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:49.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.208+0000 7fd9cd3d9640 1 --2- 172.21.15.88:0/1264487192 >> v2:172.21.15.94:3300/0 conn(0x7fd9c815d9f0 0x7fd9c8161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:49.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.208+0000 7fd9cd3d9640 1 -- 172.21.15.88:0/1264487192 >> 172.21.15.88:0/1264487192 conn(0x7fd9c80715c0 msgr2=0x7fd9c80719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:49.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.208+0000 7fd9cd3d9640 1 -- 172.21.15.88:0/1264487192 shutdown_connections 2023-10-16T20:05:49.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.208+0000 7fd9cd3d9640 1 -- 172.21.15.88:0/1264487192 wait complete. 2023-10-16T20:05:49.217 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.208+0000 7fd9cd3d9640 1 Processor -- start 2023-10-16T20:05:49.217 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.208+0000 7fd9cd3d9640 1 -- start start 2023-10-16T20:05:49.217 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.212+0000 7fd9cd3d9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd9c80d22e0 0x7fd9c80c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:49.217 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.212+0000 7fd9cd3d9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9c80d26c0 0x7fd9c814aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:49.218 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.212+0000 7fd9cd3d9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9c815d9f0 0x7fd9c80c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:49.218 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.212+0000 7fd9cd3d9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c814b1e0 con 0x7fd9c80d26c0 2023-10-16T20:05:49.218 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.212+0000 7fd9cd3d9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c80c3820 con 0x7fd9c815d9f0 2023-10-16T20:05:49.218 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.212+0000 7fd9cd3d9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c80c9590 con 0x7fd9c80d22e0 2023-10-16T20:05:49.218 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.212+0000 7fd9be7fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9c80d26c0 0x7fd9c814aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:49.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.212+0000 7fd9be7fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9c80d26c0 0x7fd9c814aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48192/0 (socket says 172.21.15.88:48192) 2023-10-16T20:05:49.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.212+0000 7fd9be7fc640 1 -- 172.21.15.88:0/1264912782 learned_addr learned my addr 172.21.15.88:0/1264912782 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:49.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.212+0000 7fd9c77fe640 1 --2- 172.21.15.88:0/1264912782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9c815d9f0 0x7fd9c80c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:49.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.212+0000 7fd9c6ffd640 1 --2- 172.21.15.88:0/1264912782 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd9c80d22e0 0x7fd9c80c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:49.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.212+0000 7fd9be7fc640 1 -- 172.21.15.88:0/1264912782 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd9c80d22e0 msgr2=0x7fd9c80c73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:49.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.212+0000 7fd9be7fc640 1 --2- 172.21.15.88:0/1264912782 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd9c80d22e0 0x7fd9c80c73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:49.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.212+0000 7fd9be7fc640 1 -- 172.21.15.88:0/1264912782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9c815d9f0 msgr2=0x7fd9c80c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:49.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.212+0000 7fd9be7fc640 1 --2- 172.21.15.88:0/1264912782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9c815d9f0 0x7fd9c80c8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:49.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.212+0000 7fd9be7fc640 1 -- 172.21.15.88:0/1264912782 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c00094d0 con 0x7fd9c80d26c0 2023-10-16T20:05:49.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.212+0000 7fd9c77fe640 1 --2- 172.21.15.88:0/1264912782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9c815d9f0 0x7fd9c80c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:49.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.212+0000 7fd9be7fc640 1 --2- 172.21.15.88:0/1264912782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9c80d26c0 0x7fd9c814aca0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000efc0 tx=0x7fd9b000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:49.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.212+0000 7fd9bffff640 1 -- 172.21.15.88:0/1264912782 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0009e80 con 0x7fd9c80d26c0 2023-10-16T20:05:49.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.212+0000 7fd9bffff640 1 -- 172.21.15.88:0/1264912782 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b000f040 con 0x7fd9c80d26c0 2023-10-16T20:05:49.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.212+0000 7fd9bffff640 1 -- 172.21.15.88:0/1264912782 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0014680 con 0x7fd9c80d26c0 2023-10-16T20:05:49.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.212+0000 7fd9cd3d9640 1 -- 172.21.15.88:0/1264912782 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c80c92b0 con 0x7fd9c80d26c0 2023-10-16T20:05:49.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.212+0000 7fd9cd3d9640 1 -- 172.21.15.88:0/1264912782 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c80c34b0 con 0x7fd9c80d26c0 2023-10-16T20:05:49.222 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.212+0000 7fd9cd3d9640 1 -- 172.21.15.88:0/1264912782 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c804e9c0 con 0x7fd9c80d26c0 2023-10-16T20:05:49.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.216+0000 7fd9bffff640 1 -- 172.21.15.88:0/1264912782 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd9b00147e0 con 0x7fd9c80d26c0 2023-10-16T20:05:49.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.216+0000 7fd9bffff640 1 --2- 172.21.15.88:0/1264912782 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd9a80664a0 0x7fd9a8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:49.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.216+0000 7fd9bffff640 1 -- 172.21.15.88:0/1264912782 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd9b0084f70 con 0x7fd9c80d26c0 2023-10-16T20:05:49.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.216+0000 7fd9c6ffd640 1 --2- 172.21.15.88:0/1264912782 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd9a80664a0 0x7fd9a8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:49.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.216+0000 7fd9c6ffd640 1 --2- 172.21.15.88:0/1264912782 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd9a80664a0 0x7fd9a8068960 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8000f70 tx=0x7fd9b8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:49.224 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.220+0000 7fd9bffff640 1 -- 172.21.15.88:0/1264912782 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd9b00b6090 con 0x7fd9c80d26c0 2023-10-16T20:05:49.375 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.372+0000 7fd9cd3d9640 1 -- 172.21.15.88:0/1264912782 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9c80d0840 con 0x7fd9c80d26c0 2023-10-16T20:05:49.376 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.372+0000 7fd9bffff640 1 -- 172.21.15.88:0/1264912782 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18661 (secure 0 0 0) 0x7fd9b00520c0 con 0x7fd9c80d26c0 2023-10-16T20:05:49.377 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:49.377 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":29,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:05:38.202959+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T20:04:54.442487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T20:04:55.766641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T20:04:59.475830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":1}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:05:49.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.376+0000 7fd9cd3d9640 1 -- 172.21.15.88:0/1264912782 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd9a80664a0 msgr2=0x7fd9a8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:49.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.376+0000 7fd9cd3d9640 1 --2- 172.21.15.88:0/1264912782 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd9a80664a0 0x7fd9a8068960 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8000f70 tx=0x7fd9b8009040 comp rx=0 tx=0).stop 2023-10-16T20:05:49.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.376+0000 7fd9cd3d9640 1 -- 172.21.15.88:0/1264912782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9c80d26c0 msgr2=0x7fd9c814aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:49.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.376+0000 7fd9cd3d9640 1 --2- 172.21.15.88:0/1264912782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9c80d26c0 0x7fd9c814aca0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000efc0 tx=0x7fd9b000c490 comp rx=0 tx=0).stop 2023-10-16T20:05:49.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.376+0000 7fd9cd3d9640 1 -- 172.21.15.88:0/1264912782 shutdown_connections 2023-10-16T20:05:49.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.376+0000 7fd9cd3d9640 1 --2- 172.21.15.88:0/1264912782 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd9a80664a0 0x7fd9a8068960 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:49.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.376+0000 7fd9cd3d9640 1 --2- 172.21.15.88:0/1264912782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9c815d9f0 0x7fd9c80c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:49.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.376+0000 7fd9cd3d9640 1 --2- 172.21.15.88:0/1264912782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9c80d26c0 0x7fd9c814aca0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:49.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.376+0000 7fd9cd3d9640 1 --2- 172.21.15.88:0/1264912782 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd9c80d22e0 0x7fd9c80c73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:49.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.376+0000 7fd9cd3d9640 1 -- 172.21.15.88:0/1264912782 >> 172.21.15.88:0/1264912782 conn(0x7fd9c80715c0 msgr2=0x7fd9c80b7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:49.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.376+0000 7fd9cd3d9640 1 -- 172.21.15.88:0/1264912782 shutdown_connections 2023-10-16T20:05:49.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.376+0000 7fd9cd3d9640 1 -- 172.21.15.88:0/1264912782 wait complete. 2023-10-16T20:05:49.398 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T20:04:54.442487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T20:04:55.766641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T20:04:59.475830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T20:05:49.399 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:05:49.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.536+0000 7f0152d80640 1 Processor -- start 2023-10-16T20:05:49.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.540+0000 7f0152d80640 1 -- start start 2023-10-16T20:05:49.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.540+0000 7f0152d80640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f014c154f30 0x7f014c0b0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:49.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.540+0000 7f0152d80640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f014c0b0a40 0x7f014c15aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:49.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.540+0000 7f0152d80640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f014c06d2f0 con 0x7f014c15b530 2023-10-16T20:05:49.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.540+0000 7f0152d80640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f014c155310 con 0x7f014c0b0a40 2023-10-16T20:05:49.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.540+0000 7f0152d80640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f014c0b0ec0 con 0x7f014c154f30 2023-10-16T20:05:49.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.540+0000 7f015131d640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f014c15b530 0x7f014c0b1610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:34722/0 (socket says 172.21.15.88:34722) 2023-10-16T20:05:49.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.540+0000 7f015131d640 1 -- 172.21.15.88:0/1237321949 learned_addr learned my addr 172.21.15.88:0/1237321949 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:49.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.540+0000 7f0150b1c640 1 --2- 172.21.15.88:0/1237321949 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f014c154f30 0x7f014c0b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:49.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.540+0000 7f014bfff640 1 --2- 172.21.15.88:0/1237321949 >> v2:172.21.15.94:3300/0 conn(0x7f014c0b0a40 0x7f014c15aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:49.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.540+0000 7f014b7fe640 1 -- 172.21.15.88:0/1237321949 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3618322105 0 0) 0x7f014c06d2f0 con 0x7f014c15b530 2023-10-16T20:05:49.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.540+0000 7f014b7fe640 1 -- 172.21.15.88:0/1237321949 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f013c003530 con 0x7f014c15b530 2023-10-16T20:05:49.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.540+0000 7f0150b1c640 1 -- 172.21.15.88:0/1237321949 >> v2:172.21.15.94:3300/0 conn(0x7f014c0b0a40 msgr2=0x7f014c15aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:49.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.540+0000 7f0150b1c640 1 --2- 172.21.15.88:0/1237321949 >> v2:172.21.15.94:3300/0 conn(0x7f014c0b0a40 0x7f014c15aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:49.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.540+0000 7f0150b1c640 1 -- 172.21.15.88:0/1237321949 >> v1:172.21.15.88:6789/0 conn(0x7f014c15b530 legacy=0x7f014c0b1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:49.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.540+0000 7f0150b1c640 1 -- 172.21.15.88:0/1237321949 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f014c15bd20 con 0x7f014c154f30 2023-10-16T20:05:49.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.544+0000 7f0150b1c640 1 --2- 172.21.15.88:0/1237321949 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f014c154f30 0x7f014c0b0500 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f01400097c0 tx=0x7f014001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=293cbe78810765ee server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:49.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.544+0000 7f014b7fe640 1 -- 172.21.15.88:0/1237321949 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01400ab600 con 0x7f014c154f30 2023-10-16T20:05:49.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.544+0000 7f014b7fe640 1 -- 172.21.15.88:0/1237321949 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01400a7050 con 0x7f014c154f30 2023-10-16T20:05:49.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.544+0000 7f014b7fe640 1 -- 172.21.15.88:0/1237321949 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01400ab980 con 0x7f014c154f30 2023-10-16T20:05:49.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.544+0000 7f0152d80640 1 -- 172.21.15.88:0/1237321949 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f014c154f30 msgr2=0x7f014c0b0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:49.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.544+0000 7f0152d80640 1 --2- 172.21.15.88:0/1237321949 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f014c154f30 0x7f014c0b0500 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f01400097c0 tx=0x7f014001c450 comp rx=0 tx=0).stop 2023-10-16T20:05:49.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.544+0000 7f0152d80640 1 -- 172.21.15.88:0/1237321949 shutdown_connections 2023-10-16T20:05:49.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.544+0000 7f0152d80640 1 --2- 172.21.15.88:0/1237321949 >> v2:172.21.15.94:3300/0 conn(0x7f014c0b0a40 0x7f014c15aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:49.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.544+0000 7f0152d80640 1 --2- 172.21.15.88:0/1237321949 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f014c154f30 0x7f014c0b0500 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:49.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.544+0000 7f0152d80640 1 -- 172.21.15.88:0/1237321949 >> 172.21.15.88:0/1237321949 conn(0x7f014c0cb080 msgr2=0x7f014c0cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:49.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.544+0000 7f0152d80640 1 -- 172.21.15.88:0/1237321949 shutdown_connections 2023-10-16T20:05:49.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.544+0000 7f0152d80640 1 -- 172.21.15.88:0/1237321949 wait complete. 2023-10-16T20:05:49.552 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.544+0000 7f0152d80640 1 Processor -- start 2023-10-16T20:05:49.552 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.544+0000 7f0152d80640 1 -- start start 2023-10-16T20:05:49.553 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.548+0000 7f0152d80640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f014c0b0a40 0x7f014c134ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:49.553 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.548+0000 7f0152d80640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f014c154f30 0x7f014c1345a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:49.553 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.548+0000 7f0152d80640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f014c15b530 0x7f014c15c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:49.553 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.548+0000 7f0152d80640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014c14a800 con 0x7f014c154f30 2023-10-16T20:05:49.553 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.548+0000 7f0152d80640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014c15cbb0 con 0x7f014c0b0a40 2023-10-16T20:05:49.554 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.548+0000 7f0152d80640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f014c15ce90 con 0x7f014c15b530 2023-10-16T20:05:49.554 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.548+0000 7f014bfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f014c154f30 0x7f014c1345a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:49.554 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.548+0000 7f014bfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f014c154f30 0x7f014c1345a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48194/0 (socket says 172.21.15.88:48194) 2023-10-16T20:05:49.554 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.548+0000 7f014bfff640 1 -- 172.21.15.88:0/3433937254 learned_addr learned my addr 172.21.15.88:0/3433937254 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:49.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.548+0000 7f0150b1c640 1 --2- 172.21.15.88:0/3433937254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f014c0b0a40 0x7f014c134ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:49.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.548+0000 7f015131d640 1 --2- 172.21.15.88:0/3433937254 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f014c15b530 0x7f014c15c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:49.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.548+0000 7f014bfff640 1 -- 172.21.15.88:0/3433937254 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f014c15b530 msgr2=0x7f014c15c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:49.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.548+0000 7f014bfff640 1 --2- 172.21.15.88:0/3433937254 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f014c15b530 0x7f014c15c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:49.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.548+0000 7f014bfff640 1 -- 172.21.15.88:0/3433937254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f014c0b0a40 msgr2=0x7f014c134ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:49.556 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.548+0000 7f014bfff640 1 --2- 172.21.15.88:0/3433937254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f014c0b0a40 0x7f014c134ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:49.556 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.548+0000 7f014bfff640 1 -- 172.21.15.88:0/3433937254 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0140009420 con 0x7f014c154f30 2023-10-16T20:05:49.556 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.548+0000 7f015131d640 1 --2- 172.21.15.88:0/3433937254 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f014c15b530 0x7f014c15c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:49.556 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.548+0000 7f014bfff640 1 --2- 172.21.15.88:0/3433937254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f014c154f30 0x7f014c1345a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f013800e870 tx=0x7f013800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:49.556 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.548+0000 7f01497fa640 1 -- 172.21.15.88:0/3433937254 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0138009510 con 0x7f014c154f30 2023-10-16T20:05:49.556 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.548+0000 7f01497fa640 1 -- 172.21.15.88:0/3433937254 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013800eec0 con 0x7f014c154f30 2023-10-16T20:05:49.557 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.548+0000 7f0152d80640 1 -- 172.21.15.88:0/3433937254 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f014c15d170 con 0x7f014c154f30 2023-10-16T20:05:49.557 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.548+0000 7f01497fa640 1 -- 172.21.15.88:0/3433937254 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013800cb40 con 0x7f014c154f30 2023-10-16T20:05:49.557 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.548+0000 7f0152d80640 1 -- 172.21.15.88:0/3433937254 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f014c1492c0 con 0x7f014c154f30 2023-10-16T20:05:49.558 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.548+0000 7f0152d80640 1 -- 172.21.15.88:0/3433937254 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f014c04e9c0 con 0x7f014c154f30 2023-10-16T20:05:49.558 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.552+0000 7f01497fa640 1 -- 172.21.15.88:0/3433937254 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f013800cca0 con 0x7f014c154f30 2023-10-16T20:05:49.558 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.552+0000 7f01497fa640 1 --2- 172.21.15.88:0/3433937254 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f01300664d0 0x7f0130068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:49.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.552+0000 7f01497fa640 1 -- 172.21.15.88:0/3433937254 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0138084260 con 0x7f014c154f30 2023-10-16T20:05:49.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.552+0000 7f0150b1c640 1 --2- 172.21.15.88:0/3433937254 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f01300664d0 0x7f0130068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:49.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.552+0000 7f0150b1c640 1 --2- 172.21.15.88:0/3433937254 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f01300664d0 0x7f0130068990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f01400ad000 tx=0x7f01400a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:49.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.556+0000 7f01497fa640 1 -- 172.21.15.88:0/3433937254 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f01380513d0 con 0x7f014c154f30 2023-10-16T20:05:49.708 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.704+0000 7f0152d80640 1 -- 172.21.15.88:0/3433937254 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f014c070880 con 0x7f014c154f30 2023-10-16T20:05:49.709 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.704+0000 7f01497fa640 1 -- 172.21.15.88:0/3433937254 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f01380549f0 con 0x7f014c154f30 2023-10-16T20:05:49.709 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:05:49.711 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.708+0000 7f0152d80640 1 -- 172.21.15.88:0/3433937254 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f01300664d0 msgr2=0x7f0130068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:49.711 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.708+0000 7f0152d80640 1 --2- 172.21.15.88:0/3433937254 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f01300664d0 0x7f0130068990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f01400ad000 tx=0x7f01400a6000 comp rx=0 tx=0).stop 2023-10-16T20:05:49.711 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.708+0000 7f0152d80640 1 -- 172.21.15.88:0/3433937254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f014c154f30 msgr2=0x7f014c1345a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:49.711 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.708+0000 7f0152d80640 1 --2- 172.21.15.88:0/3433937254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f014c154f30 0x7f014c1345a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f013800e870 tx=0x7f013800ec70 comp rx=0 tx=0).stop 2023-10-16T20:05:49.711 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.708+0000 7f0152d80640 1 -- 172.21.15.88:0/3433937254 shutdown_connections 2023-10-16T20:05:49.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.708+0000 7f0152d80640 1 --2- 172.21.15.88:0/3433937254 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f01300664d0 0x7f0130068990 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:49.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.708+0000 7f0152d80640 1 --2- 172.21.15.88:0/3433937254 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f014c15b530 0x7f014c15c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:49.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.708+0000 7f0152d80640 1 --2- 172.21.15.88:0/3433937254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f014c154f30 0x7f014c1345a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:49.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.708+0000 7f0152d80640 1 --2- 172.21.15.88:0/3433937254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f014c0b0a40 0x7f014c134ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:49.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.708+0000 7f0152d80640 1 -- 172.21.15.88:0/3433937254 >> 172.21.15.88:0/3433937254 conn(0x7f014c0cb080 msgr2=0x7f014c151530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:49.713 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.708+0000 7f0152d80640 1 -- 172.21.15.88:0/3433937254 shutdown_connections 2023-10-16T20:05:49.713 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.708+0000 7f0152d80640 1 -- 172.21.15.88:0/3433937254 wait complete. 2023-10-16T20:05:49.730 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T20:05:49.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.868+0000 7f6590df8640 1 Processor -- start 2023-10-16T20:05:49.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.868+0000 7f6590df8640 1 -- start start 2023-10-16T20:05:49.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.868+0000 7f6590df8640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f658c15cae0 0x7f658c160730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:49.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.868+0000 7f6590df8640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f658c0b22b0 0x7f658c0b2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:49.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.868+0000 7f6590df8640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f658c0b2c40 con 0x7f658c0caec0 2023-10-16T20:05:49.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.868+0000 7f6590df8640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f658c06b370 con 0x7f658c0b22b0 2023-10-16T20:05:49.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.868+0000 7f6590df8640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f658c161660 con 0x7f658c15cae0 2023-10-16T20:05:49.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.868+0000 7f658ad9d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f658c0b22b0 0x7f658c0b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:49.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.872+0000 7f658ad9d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f658c0b22b0 0x7f658c0b2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48206/0 (socket says 172.21.15.88:48206) 2023-10-16T20:05:49.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.872+0000 7f658ad9d640 1 -- 172.21.15.88:0/1196840862 learned_addr learned my addr 172.21.15.88:0/1196840862 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:49.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.872+0000 7f6581d9b640 1 --1- 172.21.15.88:0/1196840862 >> v1:172.21.15.88:6789/0 conn(0x7f658c0caec0 0x7f658c0cedc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:34724/0 (socket says 172.21.15.88:34724) 2023-10-16T20:05:49.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.872+0000 7f658a59c640 1 --2- 172.21.15.88:0/1196840862 >> v2:172.21.15.94:3300/0 conn(0x7f658c15cae0 0x7f658c160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:49.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.872+0000 7f658ad9d640 1 -- 172.21.15.88:0/1196840862 >> v2:172.21.15.94:3300/0 conn(0x7f658c15cae0 msgr2=0x7f658c160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:49.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.872+0000 7f658ad9d640 1 --2- 172.21.15.88:0/1196840862 >> v2:172.21.15.94:3300/0 conn(0x7f658c15cae0 0x7f658c160730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:49.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.872+0000 7f658ad9d640 1 -- 172.21.15.88:0/1196840862 >> v1:172.21.15.88:6789/0 conn(0x7f658c0caec0 legacy=0x7f658c0cedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:49.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.872+0000 7f658ad9d640 1 -- 172.21.15.88:0/1196840862 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f658c160dc0 con 0x7f658c0b22b0 2023-10-16T20:05:49.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.872+0000 7f6589d9b640 1 -- 172.21.15.88:0/1196840862 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2824334050 0 0) 0x7f6574001120 con 0x7f658c0caec0 2023-10-16T20:05:49.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.872+0000 7f658a59c640 1 --2- 172.21.15.88:0/1196840862 >> v2:172.21.15.94:3300/0 conn(0x7f658c15cae0 0x7f658c160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:49.878 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.872+0000 7f658ad9d640 1 --2- 172.21.15.88:0/1196840862 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f658c0b22b0 0x7f658c0b2700 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f658c06ae00 tx=0x7f658401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d396f11d674065 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:49.878 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.872+0000 7f6589d9b640 1 -- 172.21.15.88:0/1196840862 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65840a5860 con 0x7f658c0b22b0 2023-10-16T20:05:49.879 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.872+0000 7f6589d9b640 1 -- 172.21.15.88:0/1196840862 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65840a59c0 con 0x7f658c0b22b0 2023-10-16T20:05:49.879 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.872+0000 7f6589d9b640 1 -- 172.21.15.88:0/1196840862 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65840a5d50 con 0x7f658c0b22b0 2023-10-16T20:05:49.879 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.872+0000 7f6590df8640 1 -- 172.21.15.88:0/1196840862 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f658c0b22b0 msgr2=0x7f658c0b2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:49.879 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.876+0000 7f6590df8640 1 --2- 172.21.15.88:0/1196840862 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f658c0b22b0 0x7f658c0b2700 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f658c06ae00 tx=0x7f658401c9a0 comp rx=0 tx=0).stop 2023-10-16T20:05:49.879 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.876+0000 7f6590df8640 1 -- 172.21.15.88:0/1196840862 shutdown_connections 2023-10-16T20:05:49.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.876+0000 7f6590df8640 1 --2- 172.21.15.88:0/1196840862 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f658c0b22b0 0x7f658c0b2700 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:49.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.876+0000 7f6590df8640 1 --2- 172.21.15.88:0/1196840862 >> v2:172.21.15.94:3300/0 conn(0x7f658c15cae0 0x7f658c160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:49.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.876+0000 7f6590df8640 1 -- 172.21.15.88:0/1196840862 >> 172.21.15.88:0/1196840862 conn(0x7f658c0cd2b0 msgr2=0x7f658c0cd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:49.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.876+0000 7f6590df8640 1 -- 172.21.15.88:0/1196840862 shutdown_connections 2023-10-16T20:05:49.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.876+0000 7f6590df8640 1 -- 172.21.15.88:0/1196840862 wait complete. 2023-10-16T20:05:49.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.876+0000 7f6590df8640 1 Processor -- start 2023-10-16T20:05:49.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.876+0000 7f6590df8640 1 -- start start 2023-10-16T20:05:49.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.876+0000 7f6590df8640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f658c0b22b0 0x7f658c15be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:49.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.876+0000 7f6590df8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f658c0caec0 0x7f658c0cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:49.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.876+0000 7f6590df8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f658c15cae0 0x7f658c0ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:49.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.876+0000 7f6590df8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658c0c6c60 con 0x7f658c15cae0 2023-10-16T20:05:49.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.876+0000 7f6590df8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658c15c3b0 con 0x7f658c0caec0 2023-10-16T20:05:49.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.876+0000 7f6590df8640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f658c153590 con 0x7f658c0b22b0 2023-10-16T20:05:49.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.876+0000 7f658ad9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f658c15cae0 0x7f658c0ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:49.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.876+0000 7f658ad9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f658c15cae0 0x7f658c0ccb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48218/0 (socket says 172.21.15.88:48218) 2023-10-16T20:05:49.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.876+0000 7f658ad9d640 1 -- 172.21.15.88:0/2019426187 learned_addr learned my addr 172.21.15.88:0/2019426187 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:49.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.876+0000 7f6581d9b640 1 --2- 172.21.15.88:0/2019426187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f658c0caec0 0x7f658c0cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:49.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.876+0000 7f658a59c640 1 --2- 172.21.15.88:0/2019426187 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f658c0b22b0 0x7f658c15be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:49.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.876+0000 7f658ad9d640 1 -- 172.21.15.88:0/2019426187 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f658c0b22b0 msgr2=0x7f658c15be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:49.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.876+0000 7f658ad9d640 1 --2- 172.21.15.88:0/2019426187 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f658c0b22b0 0x7f658c15be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:49.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.876+0000 7f658ad9d640 1 -- 172.21.15.88:0/2019426187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f658c0caec0 msgr2=0x7f658c0cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:49.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.876+0000 7f658ad9d640 1 --2- 172.21.15.88:0/2019426187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f658c0caec0 0x7f658c0cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:49.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.876+0000 7f658ad9d640 1 -- 172.21.15.88:0/2019426187 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6584009b30 con 0x7f658c15cae0 2023-10-16T20:05:49.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.880+0000 7f658ad9d640 1 --2- 172.21.15.88:0/2019426187 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f658c15cae0 0x7f658c0ccb50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f658401a5d0 tx=0x7f6584019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:49.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.880+0000 7f65837fe640 1 -- 172.21.15.88:0/2019426187 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65840b8680 con 0x7f658c15cae0 2023-10-16T20:05:49.887 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.880+0000 7f65837fe640 1 -- 172.21.15.88:0/2019426187 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65840b8820 con 0x7f658c15cae0 2023-10-16T20:05:49.887 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.880+0000 7f65837fe640 1 -- 172.21.15.88:0/2019426187 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65840b8bd0 con 0x7f658c15cae0 2023-10-16T20:05:49.887 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.880+0000 7f6590df8640 1 -- 172.21.15.88:0/2019426187 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f658c152980 con 0x7f658c15cae0 2023-10-16T20:05:49.887 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.880+0000 7f6590df8640 1 -- 172.21.15.88:0/2019426187 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f658c152f40 con 0x7f658c15cae0 2023-10-16T20:05:49.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.880+0000 7f6590df8640 1 -- 172.21.15.88:0/2019426187 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f658c0cc450 con 0x7f658c15cae0 2023-10-16T20:05:49.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.880+0000 7f65837fe640 1 -- 172.21.15.88:0/2019426187 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f65840b8d30 con 0x7f658c15cae0 2023-10-16T20:05:49.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.880+0000 7f65837fe640 1 --2- 172.21.15.88:0/2019426187 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f656c0664a0 0x7f656c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:49.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.880+0000 7f65837fe640 1 -- 172.21.15.88:0/2019426187 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f658411f940 con 0x7f658c15cae0 2023-10-16T20:05:49.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.880+0000 7f658a59c640 1 --2- 172.21.15.88:0/2019426187 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f656c0664a0 0x7f656c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:49.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.884+0000 7f658a59c640 1 --2- 172.21.15.88:0/2019426187 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f656c0664a0 0x7f656c068960 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f657c00c900 tx=0x7f657c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:49.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:49.884+0000 7f65837fe640 1 -- 172.21.15.88:0/2019426187 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f65840ec8e0 con 0x7f658c15cae0 2023-10-16T20:05:50.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.080+0000 7f6590df8640 1 -- 172.21.15.88:0/2019426187 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f658c153700 con 0x7f658c15cae0 2023-10-16T20:05:50.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.080+0000 7f65837fe640 1 -- 172.21.15.88:0/2019426187 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f65840eff00 con 0x7f658c15cae0 2023-10-16T20:05:50.085 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:05:50.088 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.084+0000 7f6590df8640 1 -- 172.21.15.88:0/2019426187 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f656c0664a0 msgr2=0x7f656c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:50.088 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.084+0000 7f6590df8640 1 --2- 172.21.15.88:0/2019426187 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f656c0664a0 0x7f656c068960 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f657c00c900 tx=0x7f657c008040 comp rx=0 tx=0).stop 2023-10-16T20:05:50.088 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.084+0000 7f6590df8640 1 -- 172.21.15.88:0/2019426187 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f658c15cae0 msgr2=0x7f658c0ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:50.089 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.084+0000 7f6590df8640 1 --2- 172.21.15.88:0/2019426187 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f658c15cae0 0x7f658c0ccb50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f658401a5d0 tx=0x7f6584019740 comp rx=0 tx=0).stop 2023-10-16T20:05:50.089 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.084+0000 7f6590df8640 1 -- 172.21.15.88:0/2019426187 shutdown_connections 2023-10-16T20:05:50.089 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.084+0000 7f6590df8640 1 --2- 172.21.15.88:0/2019426187 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f656c0664a0 0x7f656c068960 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:50.089 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.084+0000 7f6590df8640 1 --2- 172.21.15.88:0/2019426187 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f658c15cae0 0x7f658c0ccb50 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:50.089 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.084+0000 7f6590df8640 1 --2- 172.21.15.88:0/2019426187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f658c0caec0 0x7f658c0cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:50.090 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.084+0000 7f6590df8640 1 --2- 172.21.15.88:0/2019426187 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f658c0b22b0 0x7f658c15be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:50.090 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.084+0000 7f6590df8640 1 -- 172.21.15.88:0/2019426187 >> 172.21.15.88:0/2019426187 conn(0x7f658c0cd2b0 msgr2=0x7f658c0d0810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:50.090 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.084+0000 7f6590df8640 1 -- 172.21.15.88:0/2019426187 shutdown_connections 2023-10-16T20:05:50.090 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.084+0000 7f6590df8640 1 -- 172.21.15.88:0/2019426187 wait complete. 2023-10-16T20:05:50.115 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T20:05:50.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.252+0000 7fedf6068640 1 Processor -- start 2023-10-16T20:05:50.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.252+0000 7fedf6068640 1 -- start start 2023-10-16T20:05:50.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.256+0000 7fedf6068640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fedf015cb70 0x7fedf00b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:50.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.256+0000 7fedf6068640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fedf00b4530 0x7fedf00b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:50.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.256+0000 7fedf6068640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedf00b4ed0 con 0x7fedf00d0bd0 2023-10-16T20:05:50.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.256+0000 7fedf6068640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedf00b7d20 con 0x7fedf00b4530 2023-10-16T20:05:50.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.256+0000 7fedf6068640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf00b7110 con 0x7fedf015cb70 2023-10-16T20:05:50.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.256+0000 7fedeeffd640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fedf00b4530 0x7fedf00b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:50.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.256+0000 7fedeeffd640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fedf00b4530 0x7fedf00b4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48220/0 (socket says 172.21.15.88:48220) 2023-10-16T20:05:50.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.256+0000 7fedeeffd640 1 -- 172.21.15.88:0/3866130828 learned_addr learned my addr 172.21.15.88:0/3866130828 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:50.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.256+0000 7fedef7fe640 1 --2- 172.21.15.88:0/3866130828 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fedf015cb70 0x7fedf00b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:50.261 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.256+0000 7fedeeffd640 1 -- 172.21.15.88:0/3866130828 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fedf015cb70 msgr2=0x7fedf00b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:50.261 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.256+0000 7fedeeffd640 1 --2- 172.21.15.88:0/3866130828 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fedf015cb70 0x7fedf00b83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:50.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.256+0000 7fedeeffd640 1 -- 172.21.15.88:0/3866130828 >> v1:172.21.15.88:6789/0 conn(0x7fedf00d0bd0 legacy=0x7fedf00d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:50.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.256+0000 7fedeeffd640 1 -- 172.21.15.88:0/3866130828 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf00b73f0 con 0x7fedf00b4530 2023-10-16T20:05:50.263 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.260+0000 7fedeeffd640 1 --2- 172.21.15.88:0/3866130828 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedf00b4530 0x7fedf00b4990 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fede4009870 tx=0x7fede401ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77679fde0ac492e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:50.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.260+0000 7fedee7fc640 1 -- 172.21.15.88:0/3866130828 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede40a8a30 con 0x7fedf00b4530 2023-10-16T20:05:50.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.260+0000 7fedee7fc640 1 -- 172.21.15.88:0/3866130828 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede40a8b90 con 0x7fedf00b4530 2023-10-16T20:05:50.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.260+0000 7fedee7fc640 1 -- 172.21.15.88:0/3866130828 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede40b0440 con 0x7fedf00b4530 2023-10-16T20:05:50.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.260+0000 7fedf6068640 1 -- 172.21.15.88:0/3866130828 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedf00b4530 msgr2=0x7fedf00b4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:50.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.260+0000 7fedf6068640 1 --2- 172.21.15.88:0/3866130828 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedf00b4530 0x7fedf00b4990 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fede4009870 tx=0x7fede401ed30 comp rx=0 tx=0).stop 2023-10-16T20:05:50.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.260+0000 7fedf6068640 1 -- 172.21.15.88:0/3866130828 shutdown_connections 2023-10-16T20:05:50.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.260+0000 7fedf6068640 1 --2- 172.21.15.88:0/3866130828 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedf00b4530 0x7fedf00b4990 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:50.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.260+0000 7fedf6068640 1 --2- 172.21.15.88:0/3866130828 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fedf015cb70 0x7fedf00b83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:50.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.260+0000 7fedf6068640 1 -- 172.21.15.88:0/3866130828 >> 172.21.15.88:0/3866130828 conn(0x7fedf006b6e0 msgr2=0x7fedf006baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:50.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.260+0000 7fedf6068640 1 -- 172.21.15.88:0/3866130828 shutdown_connections 2023-10-16T20:05:50.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.260+0000 7fedf6068640 1 -- 172.21.15.88:0/3866130828 wait complete. 2023-10-16T20:05:50.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.264+0000 7fedf6068640 1 Processor -- start 2023-10-16T20:05:50.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.264+0000 7fedf6068640 1 -- start start 2023-10-16T20:05:50.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.264+0000 7fedf6068640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fedf00b4530 0x7fedf014d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:50.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.264+0000 7fedf6068640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedf00d0bd0 0x7fedf00c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:50.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.264+0000 7fedf6068640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedf015cb70 0x7fedf01503a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:50.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.264+0000 7fedf6068640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf00c6140 con 0x7fedf00d0bd0 2023-10-16T20:05:50.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.264+0000 7fedf6068640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf00c6420 con 0x7fedf015cb70 2023-10-16T20:05:50.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.264+0000 7fedf6068640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf00c6700 con 0x7fedf00b4530 2023-10-16T20:05:50.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.264+0000 7fedeeffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedf00d0bd0 0x7fedf00c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:50.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.264+0000 7fedeeffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedf00d0bd0 0x7fedf00c71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48236/0 (socket says 172.21.15.88:48236) 2023-10-16T20:05:50.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.264+0000 7fedeeffd640 1 -- 172.21.15.88:0/3066348845 learned_addr learned my addr 172.21.15.88:0/3066348845 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:50.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.264+0000 7fedef7fe640 1 --2- 172.21.15.88:0/3066348845 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fedf00b4530 0x7fedf014d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:50.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.264+0000 7fedeffff640 1 --2- 172.21.15.88:0/3066348845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedf015cb70 0x7fedf01503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:50.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.264+0000 7fedeeffd640 1 -- 172.21.15.88:0/3066348845 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fedf00b4530 msgr2=0x7fedf014d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:50.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.264+0000 7fedeeffd640 1 --2- 172.21.15.88:0/3066348845 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fedf00b4530 0x7fedf014d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:50.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.264+0000 7fedeeffd640 1 -- 172.21.15.88:0/3066348845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedf015cb70 msgr2=0x7fedf01503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:50.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.264+0000 7fedeeffd640 1 --2- 172.21.15.88:0/3066348845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedf015cb70 0x7fedf01503a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:50.273 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.264+0000 7fedeeffd640 1 -- 172.21.15.88:0/3066348845 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede40094d0 con 0x7fedf00d0bd0 2023-10-16T20:05:50.273 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.264+0000 7fedeeffd640 1 --2- 172.21.15.88:0/3066348845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedf00d0bd0 0x7fedf00c71e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fede40b3040 tx=0x7fede401bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:50.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.268+0000 7fedd7fff640 1 -- 172.21.15.88:0/3066348845 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede40b0dd0 con 0x7fedf00d0bd0 2023-10-16T20:05:50.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.268+0000 7fedd7fff640 1 -- 172.21.15.88:0/3066348845 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede40bb460 con 0x7fedf00d0bd0 2023-10-16T20:05:50.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.268+0000 7fedd7fff640 1 -- 172.21.15.88:0/3066348845 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede40bb810 con 0x7fedf00d0bd0 2023-10-16T20:05:50.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.268+0000 7fedf6068640 1 -- 172.21.15.88:0/3066348845 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf00c69e0 con 0x7fedf00d0bd0 2023-10-16T20:05:50.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.268+0000 7fedf6068640 1 -- 172.21.15.88:0/3066348845 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf014da70 con 0x7fedf00d0bd0 2023-10-16T20:05:50.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.268+0000 7fedf6068640 1 -- 172.21.15.88:0/3066348845 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedf004e9c0 con 0x7fedf00d0bd0 2023-10-16T20:05:50.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.268+0000 7fedd7fff640 1 -- 172.21.15.88:0/3066348845 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fede40b83f0 con 0x7fedf00d0bd0 2023-10-16T20:05:50.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.268+0000 7fedd7fff640 1 --2- 172.21.15.88:0/3066348845 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7feddc0664d0 0x7feddc068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:50.276 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.268+0000 7fedd7fff640 1 -- 172.21.15.88:0/3066348845 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fede4121a50 con 0x7fedf00d0bd0 2023-10-16T20:05:50.276 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.268+0000 7fedef7fe640 1 --2- 172.21.15.88:0/3066348845 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7feddc0664d0 0x7feddc068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:50.276 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.272+0000 7fedef7fe640 1 --2- 172.21.15.88:0/3066348845 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7feddc0664d0 0x7feddc068990 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fedf00ce7b0 tx=0x7fede0009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:50.277 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.272+0000 7fedd7fff640 1 -- 172.21.15.88:0/3066348845 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fede40eebb0 con 0x7fedf00d0bd0 2023-10-16T20:05:50.426 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.424+0000 7fedf6068640 1 -- 172.21.15.88:0/3066348845 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fedf00c6e90 con 0x7fedf00d0bd0 2023-10-16T20:05:50.427 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.424+0000 7fedd7fff640 1 -- 172.21.15.88:0/3066348845 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7fede40c00a0 con 0x7fedf00d0bd0 2023-10-16T20:05:50.428 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2023-10-16T20:05:50.429 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.424+0000 7fedf6068640 1 -- 172.21.15.88:0/3066348845 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7feddc0664d0 msgr2=0x7feddc068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:50.429 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.424+0000 7fedf6068640 1 --2- 172.21.15.88:0/3066348845 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7feddc0664d0 0x7feddc068990 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fedf00ce7b0 tx=0x7fede0009190 comp rx=0 tx=0).stop 2023-10-16T20:05:50.429 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.424+0000 7fedf6068640 1 -- 172.21.15.88:0/3066348845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedf00d0bd0 msgr2=0x7fedf00c71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:50.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.424+0000 7fedf6068640 1 --2- 172.21.15.88:0/3066348845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedf00d0bd0 0x7fedf00c71e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fede40b3040 tx=0x7fede401bad0 comp rx=0 tx=0).stop 2023-10-16T20:05:50.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.424+0000 7fedf6068640 1 -- 172.21.15.88:0/3066348845 shutdown_connections 2023-10-16T20:05:50.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.424+0000 7fedf6068640 1 --2- 172.21.15.88:0/3066348845 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7feddc0664d0 0x7feddc068990 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:50.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.424+0000 7fedf6068640 1 --2- 172.21.15.88:0/3066348845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedf015cb70 0x7fedf01503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:50.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.424+0000 7fedf6068640 1 --2- 172.21.15.88:0/3066348845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedf00d0bd0 0x7fedf00c71e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:50.431 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.424+0000 7fedf6068640 1 --2- 172.21.15.88:0/3066348845 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fedf00b4530 0x7fedf014d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:50.431 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.424+0000 7fedf6068640 1 -- 172.21.15.88:0/3066348845 >> 172.21.15.88:0/3066348845 conn(0x7fedf006b6e0 msgr2=0x7fedf00d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:50.431 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.424+0000 7fedf6068640 1 -- 172.21.15.88:0/3066348845 shutdown_connections 2023-10-16T20:05:50.431 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.424+0000 7fedf6068640 1 -- 172.21.15.88:0/3066348845 wait complete. 2023-10-16T20:05:50.449 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T20:05:50.592 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.588+0000 7f08fb59e640 1 Processor -- start 2023-10-16T20:05:50.592 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.588+0000 7f08fb59e640 1 -- start start 2023-10-16T20:05:50.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.588+0000 7f08fb59e640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f08fc15a730 0x7f08fc070ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:50.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.588+0000 7f08fb59e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f08fc0b1360 0x7f08fc0caee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:50.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.588+0000 7f08fb59e640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08fc0b0030 con 0x7f08fc0ca7e0 2023-10-16T20:05:50.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.588+0000 7f08fb59e640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08fc0b0cb0 con 0x7f08fc15a730 2023-10-16T20:05:50.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.588+0000 7f08fb59e640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08fc0b0310 con 0x7f08fc0b1360 2023-10-16T20:05:50.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.588+0000 7f08fa59c640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f08fc15a730 0x7f08fc070ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:50.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.588+0000 7f08fa59c640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f08fc15a730 0x7f08fc070ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48238/0 (socket says 172.21.15.88:48238) 2023-10-16T20:05:50.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.588+0000 7f08fa59c640 1 -- 172.21.15.88:0/2217375012 learned_addr learned my addr 172.21.15.88:0/2217375012 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:50.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.588+0000 7f08f1d9b640 1 --2- 172.21.15.88:0/2217375012 >> v2:172.21.15.94:3300/0 conn(0x7f08fc0b1360 0x7f08fc0caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:50.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.588+0000 7f08fa59c640 1 -- 172.21.15.88:0/2217375012 >> v2:172.21.15.94:3300/0 conn(0x7f08fc0b1360 msgr2=0x7f08fc0caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:50.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.588+0000 7f08fa59c640 1 --2- 172.21.15.88:0/2217375012 >> v2:172.21.15.94:3300/0 conn(0x7f08fc0b1360 0x7f08fc0caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:50.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.588+0000 7f08fa59c640 1 -- 172.21.15.88:0/2217375012 >> v1:172.21.15.88:6789/0 conn(0x7f08fc0ca7e0 legacy=0x7f08fc15e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:50.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.588+0000 7f08fa59c640 1 -- 172.21.15.88:0/2217375012 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08fc0b05f0 con 0x7f08fc15a730 2023-10-16T20:05:50.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.588+0000 7f08f1d9b640 1 --2- 172.21.15.88:0/2217375012 >> v2:172.21.15.94:3300/0 conn(0x7f08fc0b1360 0x7f08fc0caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:50.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.588+0000 7f08f9d9b640 1 -- 172.21.15.88:0/2217375012 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3114228394 0 0) 0x7f08fc0b0030 con 0x7f08fc0ca7e0 2023-10-16T20:05:50.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.592+0000 7f08fa59c640 1 --2- 172.21.15.88:0/2217375012 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08fc15a730 0x7f08fc070ab0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f08fc0adb40 tx=0x7f08ec01c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80353174c7b1383a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:50.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.592+0000 7f08f9d9b640 1 -- 172.21.15.88:0/2217375012 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec0a5700 con 0x7f08fc15a730 2023-10-16T20:05:50.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.592+0000 7f08f9d9b640 1 -- 172.21.15.88:0/2217375012 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ec0a5860 con 0x7f08fc15a730 2023-10-16T20:05:50.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.592+0000 7f08f9d9b640 1 -- 172.21.15.88:0/2217375012 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec0a5bf0 con 0x7f08fc15a730 2023-10-16T20:05:50.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.592+0000 7f08fb59e640 1 -- 172.21.15.88:0/2217375012 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08fc15a730 msgr2=0x7f08fc070ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:50.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.592+0000 7f08fb59e640 1 --2- 172.21.15.88:0/2217375012 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08fc15a730 0x7f08fc070ab0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f08fc0adb40 tx=0x7f08ec01c880 comp rx=0 tx=0).stop 2023-10-16T20:05:50.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.592+0000 7f08fb59e640 1 -- 172.21.15.88:0/2217375012 shutdown_connections 2023-10-16T20:05:50.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.592+0000 7f08fb59e640 1 --2- 172.21.15.88:0/2217375012 >> v2:172.21.15.94:3300/0 conn(0x7f08fc0b1360 0x7f08fc0caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:50.600 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.592+0000 7f08fb59e640 1 --2- 172.21.15.88:0/2217375012 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08fc15a730 0x7f08fc070ab0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:50.600 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.592+0000 7f08fb59e640 1 -- 172.21.15.88:0/2217375012 >> 172.21.15.88:0/2217375012 conn(0x7f08fc06a4c0 msgr2=0x7f08fc06c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:50.600 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.592+0000 7f08fb59e640 1 -- 172.21.15.88:0/2217375012 shutdown_connections 2023-10-16T20:05:50.600 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.592+0000 7f08fb59e640 1 -- 172.21.15.88:0/2217375012 wait complete. 2023-10-16T20:05:50.603 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.596+0000 7f08fb59e640 1 Processor -- start 2023-10-16T20:05:50.603 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.596+0000 7f08fb59e640 1 -- start start 2023-10-16T20:05:50.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.596+0000 7f08fb59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08fc0b1360 0x7f08fc157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:50.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.596+0000 7f08fb59e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f08fc0ca7e0 0x7f08fc15b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:50.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.596+0000 7f08fb59e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08fc15a730 0x7f08fc15ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:50.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.596+0000 7f08fb59e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc151310 con 0x7f08fc15a730 2023-10-16T20:05:50.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.596+0000 7f08fb59e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc1587d0 con 0x7f08fc0b1360 2023-10-16T20:05:50.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.596+0000 7f08fb59e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc1557b0 con 0x7f08fc0ca7e0 2023-10-16T20:05:50.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.596+0000 7f08fad9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08fc15a730 0x7f08fc15ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:50.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.596+0000 7f08fad9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08fc15a730 0x7f08fc15ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48248/0 (socket says 172.21.15.88:48248) 2023-10-16T20:05:50.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.596+0000 7f08fad9d640 1 -- 172.21.15.88:0/2319794341 learned_addr learned my addr 172.21.15.88:0/2319794341 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:50.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.596+0000 7f08fa59c640 1 --2- 172.21.15.88:0/2319794341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08fc0b1360 0x7f08fc157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:50.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.596+0000 7f08f1d9b640 1 --2- 172.21.15.88:0/2319794341 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f08fc0ca7e0 0x7f08fc15b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:50.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.596+0000 7f08fad9d640 1 -- 172.21.15.88:0/2319794341 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f08fc0ca7e0 msgr2=0x7f08fc15b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:50.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.596+0000 7f08fad9d640 1 --2- 172.21.15.88:0/2319794341 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f08fc0ca7e0 0x7f08fc15b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:50.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.596+0000 7f08fad9d640 1 -- 172.21.15.88:0/2319794341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08fc0b1360 msgr2=0x7f08fc157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:50.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.596+0000 7f08fad9d640 1 --2- 172.21.15.88:0/2319794341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08fc0b1360 0x7f08fc157950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:50.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.596+0000 7f08fad9d640 1 -- 172.21.15.88:0/2319794341 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ec009b30 con 0x7f08fc15a730 2023-10-16T20:05:50.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.596+0000 7f08fad9d640 1 --2- 172.21.15.88:0/2319794341 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08fc15a730 0x7f08fc15ccc0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f08f400c410 tx=0x7f08f400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:50.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.596+0000 7f08f37fe640 1 -- 172.21.15.88:0/2319794341 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f40054e0 con 0x7f08fc15a730 2023-10-16T20:05:50.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.596+0000 7f08f37fe640 1 -- 172.21.15.88:0/2319794341 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f4003600 con 0x7f08fc15a730 2023-10-16T20:05:50.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.596+0000 7f08fb59e640 1 -- 172.21.15.88:0/2319794341 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08fc158210 con 0x7f08fc15a730 2023-10-16T20:05:50.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.596+0000 7f08f37fe640 1 -- 172.21.15.88:0/2319794341 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f4003990 con 0x7f08fc15a730 2023-10-16T20:05:50.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.596+0000 7f08fb59e640 1 -- 172.21.15.88:0/2319794341 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08fc15b680 con 0x7f08fc15a730 2023-10-16T20:05:50.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.600+0000 7f08fb59e640 1 -- 172.21.15.88:0/2319794341 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08fc155960 con 0x7f08fc15a730 2023-10-16T20:05:50.610 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.600+0000 7f08f37fe640 1 -- 172.21.15.88:0/2319794341 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f08f4003af0 con 0x7f08fc15a730 2023-10-16T20:05:50.610 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.600+0000 7f08f37fe640 1 --2- 172.21.15.88:0/2319794341 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f08dc0664a0 0x7f08dc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:50.610 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.600+0000 7f08f37fe640 1 -- 172.21.15.88:0/2319794341 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f08f4083760 con 0x7f08fc15a730 2023-10-16T20:05:50.610 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.600+0000 7f08fa59c640 1 --2- 172.21.15.88:0/2319794341 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f08dc0664a0 0x7f08dc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:50.611 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.600+0000 7f08fa59c640 1 --2- 172.21.15.88:0/2319794341 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f08dc0664a0 0x7f08dc068960 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0ae000 tx=0x7f08ec0a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:50.611 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.604+0000 7f08f37fe640 1 -- 172.21.15.88:0/2319794341 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f08fc155960 con 0x7f08fc15a730 2023-10-16T20:05:50.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.756+0000 7f08fb59e640 1 -- 172.21.15.88:0/2319794341 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f08fc155920 con 0x7f08fc15a730 2023-10-16T20:05:50.760 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.756+0000 7f08f37fe640 1 -- 172.21.15.88:0/2319794341 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f08f4054410 con 0x7f08fc15a730 2023-10-16T20:05:50.760 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:05:50.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.756+0000 7f08fb59e640 1 -- 172.21.15.88:0/2319794341 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f08dc0664a0 msgr2=0x7f08dc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:50.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.756+0000 7f08fb59e640 1 --2- 172.21.15.88:0/2319794341 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f08dc0664a0 0x7f08dc068960 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0ae000 tx=0x7f08ec0a7070 comp rx=0 tx=0).stop 2023-10-16T20:05:50.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.756+0000 7f08fb59e640 1 -- 172.21.15.88:0/2319794341 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08fc15a730 msgr2=0x7f08fc15ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:50.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.756+0000 7f08fb59e640 1 --2- 172.21.15.88:0/2319794341 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08fc15a730 0x7f08fc15ccc0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f08f400c410 tx=0x7f08f400c8d0 comp rx=0 tx=0).stop 2023-10-16T20:05:50.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.756+0000 7f08fb59e640 1 -- 172.21.15.88:0/2319794341 shutdown_connections 2023-10-16T20:05:50.763 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.756+0000 7f08fb59e640 1 --2- 172.21.15.88:0/2319794341 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f08dc0664a0 0x7f08dc068960 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:50.763 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.756+0000 7f08fb59e640 1 --2- 172.21.15.88:0/2319794341 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f08fc0ca7e0 0x7f08fc15b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:50.763 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.756+0000 7f08fb59e640 1 --2- 172.21.15.88:0/2319794341 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08fc15a730 0x7f08fc15ccc0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:50.763 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.756+0000 7f08fb59e640 1 --2- 172.21.15.88:0/2319794341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08fc0b1360 0x7f08fc157950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:50.763 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.756+0000 7f08fb59e640 1 -- 172.21.15.88:0/2319794341 >> 172.21.15.88:0/2319794341 conn(0x7f08fc06a4c0 msgr2=0x7f08fc0af650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:50.764 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.756+0000 7f08fb59e640 1 -- 172.21.15.88:0/2319794341 shutdown_connections 2023-10-16T20:05:50.764 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.756+0000 7f08fb59e640 1 -- 172.21.15.88:0/2319794341 wait complete. 2023-10-16T20:05:50.782 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T20:05:50.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.916+0000 7f2308e09640 1 Processor -- start 2023-10-16T20:05:50.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.916+0000 7f2308e09640 1 -- start start 2023-10-16T20:05:50.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.916+0000 7f2308e09640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f230406a5a0 0x7f230406f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:50.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.916+0000 7f2308e09640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f23040cb8a0 0x7f23041607c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:50.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.916+0000 7f2308e09640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f230415d130 con 0x7f230415cd50 2023-10-16T20:05:50.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.920+0000 7f2308e09640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f230406fcc0 con 0x7f230406a5a0 2023-10-16T20:05:50.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.920+0000 7f2308e09640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f230406ed70 con 0x7f23040cb8a0 2023-10-16T20:05:50.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.920+0000 7f22f9d9b640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f230406a5a0 0x7f230406f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:50.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.920+0000 7f22f9d9b640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f230406a5a0 0x7f230406f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48262/0 (socket says 172.21.15.88:48262) 2023-10-16T20:05:50.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.920+0000 7f22f9d9b640 1 -- 172.21.15.88:0/197546866 learned_addr learned my addr 172.21.15.88:0/197546866 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:50.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.920+0000 7f2302d9d640 1 --2- 172.21.15.88:0/197546866 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f23040cb8a0 0x7f23041607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:50.925 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.920+0000 7f22f9d9b640 1 -- 172.21.15.88:0/197546866 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f23040cb8a0 msgr2=0x7f23041607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:50.925 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.920+0000 7f22f9d9b640 1 --2- 172.21.15.88:0/197546866 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f23040cb8a0 0x7f23041607c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:50.925 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.920+0000 7f22f9d9b640 1 -- 172.21.15.88:0/197546866 >> v1:172.21.15.94:6789/0 conn(0x7f230415cd50 legacy=0x7f2304069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:50.925 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.920+0000 7f22f9d9b640 1 -- 172.21.15.88:0/197546866 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f230406efc0 con 0x7f230406a5a0 2023-10-16T20:05:50.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.920+0000 7f22f9d9b640 1 --2- 172.21.15.88:0/197546866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f230406a5a0 0x7f230406f780 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f22ec009e50 tx=0x7f22ec01c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784918e42cdfe32e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:50.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.920+0000 7f2301d9b640 1 -- 172.21.15.88:0/197546866 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec0a57c0 con 0x7f230406a5a0 2023-10-16T20:05:50.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.920+0000 7f2301d9b640 1 -- 172.21.15.88:0/197546866 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ec0a5920 con 0x7f230406a5a0 2023-10-16T20:05:50.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.920+0000 7f2301d9b640 1 -- 172.21.15.88:0/197546866 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec0a5cb0 con 0x7f230406a5a0 2023-10-16T20:05:50.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.924+0000 7f2308e09640 1 -- 172.21.15.88:0/197546866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f230406a5a0 msgr2=0x7f230406f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:50.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.924+0000 7f2308e09640 1 --2- 172.21.15.88:0/197546866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f230406a5a0 0x7f230406f780 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f22ec009e50 tx=0x7f22ec01c9e0 comp rx=0 tx=0).stop 2023-10-16T20:05:50.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.924+0000 7f2308e09640 1 -- 172.21.15.88:0/197546866 shutdown_connections 2023-10-16T20:05:50.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.924+0000 7f2308e09640 1 --2- 172.21.15.88:0/197546866 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f23040cb8a0 0x7f23041607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:50.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.924+0000 7f2308e09640 1 --2- 172.21.15.88:0/197546866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f230406a5a0 0x7f230406f780 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:50.929 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.924+0000 7f2308e09640 1 -- 172.21.15.88:0/197546866 >> 172.21.15.88:0/197546866 conn(0x7f23040cc170 msgr2=0x7f23040cc580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:50.929 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.924+0000 7f2308e09640 1 -- 172.21.15.88:0/197546866 shutdown_connections 2023-10-16T20:05:50.929 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.924+0000 7f2308e09640 1 -- 172.21.15.88:0/197546866 wait complete. 2023-10-16T20:05:50.931 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.924+0000 7f2308e09640 1 Processor -- start 2023-10-16T20:05:50.932 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.924+0000 7f2308e09640 1 -- start start 2023-10-16T20:05:50.932 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.924+0000 7f2308e09640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f230406a5a0 0x7f2304155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:50.932 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.924+0000 7f2308e09640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f23040cb8a0 0x7f23040c3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:50.933 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.924+0000 7f2308e09640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f230415cd50 0x7f23040c71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:50.933 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.924+0000 7f2308e09640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23040b3370 con 0x7f230406a5a0 2023-10-16T20:05:50.933 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.924+0000 7f2308e09640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23040b2760 con 0x7f230415cd50 2023-10-16T20:05:50.933 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.924+0000 7f2308e09640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23040b2a40 con 0x7f23040cb8a0 2023-10-16T20:05:50.934 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.924+0000 7f230259c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f230406a5a0 0x7f2304155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:50.934 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.924+0000 7f230259c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f230406a5a0 0x7f2304155f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48264/0 (socket says 172.21.15.88:48264) 2023-10-16T20:05:50.934 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.924+0000 7f230259c640 1 -- 172.21.15.88:0/1763810811 learned_addr learned my addr 172.21.15.88:0/1763810811 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:50.934 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.928+0000 7f2302d9d640 1 --2- 172.21.15.88:0/1763810811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f230415cd50 0x7f23040c71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:50.935 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.928+0000 7f22f9d9b640 1 --2- 172.21.15.88:0/1763810811 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f23040cb8a0 0x7f23040c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:50.935 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.928+0000 7f230259c640 1 -- 172.21.15.88:0/1763810811 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f23040cb8a0 msgr2=0x7f23040c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:50.935 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.928+0000 7f230259c640 1 --2- 172.21.15.88:0/1763810811 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f23040cb8a0 0x7f23040c3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:50.935 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.928+0000 7f230259c640 1 -- 172.21.15.88:0/1763810811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f230415cd50 msgr2=0x7f23040c71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:50.935 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.928+0000 7f230259c640 1 --2- 172.21.15.88:0/1763810811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f230415cd50 0x7f23040c71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:50.936 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.928+0000 7f230259c640 1 -- 172.21.15.88:0/1763810811 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ec009b30 con 0x7f230406a5a0 2023-10-16T20:05:50.936 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.928+0000 7f2302d9d640 1 --2- 172.21.15.88:0/1763810811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f230415cd50 0x7f23040c71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:50.936 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.928+0000 7f230259c640 1 --2- 172.21.15.88:0/1763810811 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f230406a5a0 0x7f2304155f00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f22f400c3c0 tx=0x7f22f400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:50.936 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.928+0000 7f22fb7fe640 1 -- 172.21.15.88:0/1763810811 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f40052d0 con 0x7f230406a5a0 2023-10-16T20:05:50.936 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.928+0000 7f22fb7fe640 1 -- 172.21.15.88:0/1763810811 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f4005430 con 0x7f230406a5a0 2023-10-16T20:05:50.937 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.928+0000 7f22fb7fe640 1 -- 172.21.15.88:0/1763810811 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f4003840 con 0x7f230406a5a0 2023-10-16T20:05:50.937 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.928+0000 7f2308e09640 1 -- 172.21.15.88:0/1763810811 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23040b2d20 con 0x7f230406a5a0 2023-10-16T20:05:50.937 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.928+0000 7f2308e09640 1 -- 172.21.15.88:0/1763810811 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2304156440 con 0x7f230406a5a0 2023-10-16T20:05:50.938 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.928+0000 7f2308e09640 1 -- 172.21.15.88:0/1763810811 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f230404e9c0 con 0x7f230406a5a0 2023-10-16T20:05:50.939 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.928+0000 7f22fb7fe640 1 -- 172.21.15.88:0/1763810811 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f22f40039a0 con 0x7f230406a5a0 2023-10-16T20:05:50.939 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.932+0000 7f22fb7fe640 1 --2- 172.21.15.88:0/1763810811 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f22e40664d0 0x7f22e4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:50.939 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.932+0000 7f22fb7fe640 1 -- 172.21.15.88:0/1763810811 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f22f4083700 con 0x7f230406a5a0 2023-10-16T20:05:50.939 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.932+0000 7f22f9d9b640 1 --2- 172.21.15.88:0/1763810811 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f22e40664d0 0x7f22e4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:50.940 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.932+0000 7f22f9d9b640 1 --2- 172.21.15.88:0/1763810811 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f22e40664d0 0x7f22e4068990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f22ec0ae000 tx=0x7f22ec0a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:50.940 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:50.936+0000 7f22fb7fe640 1 -- 172.21.15.88:0/1763810811 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f22f4050850 con 0x7f230406a5a0 2023-10-16T20:05:51.100 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.096+0000 7f2308e09640 1 -- 172.21.15.88:0/1763810811 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f23040b5670 con 0x7f230406a5a0 2023-10-16T20:05:51.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.096+0000 7f22fb7fe640 1 -- 172.21.15.88:0/1763810811 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7f22f4053e70 con 0x7f230406a5a0 2023-10-16T20:05:51.101 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2023-10-16T20:05:51.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.100+0000 7f2308e09640 1 -- 172.21.15.88:0/1763810811 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f22e40664d0 msgr2=0x7f22e4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:51.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.100+0000 7f2308e09640 1 --2- 172.21.15.88:0/1763810811 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f22e40664d0 0x7f22e4068990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f22ec0ae000 tx=0x7f22ec0a7070 comp rx=0 tx=0).stop 2023-10-16T20:05:51.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.100+0000 7f2308e09640 1 -- 172.21.15.88:0/1763810811 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f230406a5a0 msgr2=0x7f2304155f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:51.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.100+0000 7f2308e09640 1 --2- 172.21.15.88:0/1763810811 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f230406a5a0 0x7f2304155f00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f22f400c3c0 tx=0x7f22f400c880 comp rx=0 tx=0).stop 2023-10-16T20:05:51.104 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.100+0000 7f2308e09640 1 -- 172.21.15.88:0/1763810811 shutdown_connections 2023-10-16T20:05:51.104 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.100+0000 7f2308e09640 1 --2- 172.21.15.88:0/1763810811 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f22e40664d0 0x7f22e4068990 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:51.104 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.100+0000 7f2308e09640 1 --2- 172.21.15.88:0/1763810811 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f23040cb8a0 0x7f23040c3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:51.104 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.100+0000 7f2308e09640 1 --2- 172.21.15.88:0/1763810811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f230415cd50 0x7f23040c71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:51.104 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.100+0000 7f2308e09640 1 --2- 172.21.15.88:0/1763810811 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f230406a5a0 0x7f2304155f00 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:51.105 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.100+0000 7f2308e09640 1 -- 172.21.15.88:0/1763810811 >> 172.21.15.88:0/1763810811 conn(0x7f23040cc170 msgr2=0x7f230415be10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:51.105 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.100+0000 7f2308e09640 1 -- 172.21.15.88:0/1763810811 shutdown_connections 2023-10-16T20:05:51.105 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.100+0000 7f2308e09640 1 -- 172.21.15.88:0/1763810811 wait complete. 2023-10-16T20:05:51.121 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:05:51.122 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:05:51.132 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:05:51.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.268+0000 7f8dfa50f640 1 Processor -- start 2023-10-16T20:05:51.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.268+0000 7f8dfa50f640 1 -- start start 2023-10-16T20:05:51.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.268+0000 7f8dfa50f640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f8df40d22e0 0x7f8df4161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:51.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.268+0000 7f8dfa50f640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8df415d9f0 0x7f8df40cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:51.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.268+0000 7f8dfa50f640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df40b87e0 con 0x7f8df40d26c0 2023-10-16T20:05:51.273 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.268+0000 7f8dfa50f640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8df415df10 con 0x7f8df40d22e0 2023-10-16T20:05:51.273 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.268+0000 7f8dfa50f640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8df415e1f0 con 0x7f8df415d9f0 2023-10-16T20:05:51.273 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.268+0000 7f8df3fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f8df40d22e0 0x7f8df4161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:51.273 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.268+0000 7f8df37fe640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f8df40d26c0 0x7f8df415d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:34762/0 (socket says 172.21.15.88:34762) 2023-10-16T20:05:51.273 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.268+0000 7f8df37fe640 1 -- 172.21.15.88:0/159304041 learned_addr learned my addr 172.21.15.88:0/159304041 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:51.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.268+0000 7f8df8aac640 1 --2- 172.21.15.88:0/159304041 >> v2:172.21.15.94:3300/0 conn(0x7f8df415d9f0 0x7f8df40cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:51.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.268+0000 7f8df3fff640 1 -- 172.21.15.88:0/159304041 >> v2:172.21.15.94:3300/0 conn(0x7f8df415d9f0 msgr2=0x7f8df40cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:51.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.268+0000 7f8df3fff640 1 --2- 172.21.15.88:0/159304041 >> v2:172.21.15.94:3300/0 conn(0x7f8df415d9f0 0x7f8df40cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:51.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.268+0000 7f8df3fff640 1 -- 172.21.15.88:0/159304041 >> v1:172.21.15.88:6789/0 conn(0x7f8df40d26c0 legacy=0x7f8df415d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:51.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.268+0000 7f8df3fff640 1 -- 172.21.15.88:0/159304041 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df4160970 con 0x7f8df40d22e0 2023-10-16T20:05:51.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.272+0000 7f8df3fff640 1 --2- 172.21.15.88:0/159304041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8df40d22e0 0x7f8df4161450 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8df40c66a0 tx=0x7f8de401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311de712ffe43a2c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:51.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.272+0000 7f8df2ffd640 1 -- 172.21.15.88:0/159304041 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de40ac600 con 0x7f8df40d22e0 2023-10-16T20:05:51.276 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.272+0000 7f8df2ffd640 1 -- 172.21.15.88:0/159304041 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de40a8050 con 0x7f8df40d22e0 2023-10-16T20:05:51.276 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.272+0000 7f8df2ffd640 1 -- 172.21.15.88:0/159304041 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de40ac980 con 0x7f8df40d22e0 2023-10-16T20:05:51.277 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.272+0000 7f8dfa50f640 1 -- 172.21.15.88:0/159304041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8df40d22e0 msgr2=0x7f8df4161450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:51.277 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.272+0000 7f8dfa50f640 1 --2- 172.21.15.88:0/159304041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8df40d22e0 0x7f8df4161450 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8df40c66a0 tx=0x7f8de401c3a0 comp rx=0 tx=0).stop 2023-10-16T20:05:51.277 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.272+0000 7f8dfa50f640 1 -- 172.21.15.88:0/159304041 shutdown_connections 2023-10-16T20:05:51.277 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.272+0000 7f8dfa50f640 1 --2- 172.21.15.88:0/159304041 >> v2:172.21.15.94:3300/0 conn(0x7f8df415d9f0 0x7f8df40cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:51.277 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.272+0000 7f8dfa50f640 1 --2- 172.21.15.88:0/159304041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8df40d22e0 0x7f8df4161450 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:51.278 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.272+0000 7f8dfa50f640 1 -- 172.21.15.88:0/159304041 >> 172.21.15.88:0/159304041 conn(0x7f8df40715c0 msgr2=0x7f8df40719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:51.278 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.272+0000 7f8dfa50f640 1 -- 172.21.15.88:0/159304041 shutdown_connections 2023-10-16T20:05:51.278 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.272+0000 7f8dfa50f640 1 -- 172.21.15.88:0/159304041 wait complete. 2023-10-16T20:05:51.280 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.272+0000 7f8dfa50f640 1 Processor -- start 2023-10-16T20:05:51.280 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.272+0000 7f8dfa50f640 1 -- start start 2023-10-16T20:05:51.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.276+0000 7f8dfa50f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8df40d22e0 0x7f8df41f7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:51.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.276+0000 7f8dfa50f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8df40d26c0 0x7f8df414e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:51.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.276+0000 7f8dfa50f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8df415d9f0 0x7f8df414edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:51.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.276+0000 7f8dfa50f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df4152450 con 0x7f8df415d9f0 2023-10-16T20:05:51.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.276+0000 7f8dfa50f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df4152730 con 0x7f8df40d22e0 2023-10-16T20:05:51.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.276+0000 7f8dfa50f640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df4152a10 con 0x7f8df40d26c0 2023-10-16T20:05:51.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.276+0000 7f8df8aac640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8df415d9f0 0x7f8df414edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:51.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.276+0000 7f8df8aac640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8df415d9f0 0x7f8df414edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48286/0 (socket says 172.21.15.88:48286) 2023-10-16T20:05:51.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.276+0000 7f8df8aac640 1 -- 172.21.15.88:0/3793810037 learned_addr learned my addr 172.21.15.88:0/3793810037 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:51.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.276+0000 7f8df3fff640 1 --2- 172.21.15.88:0/3793810037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8df40d22e0 0x7f8df41f7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:51.283 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.276+0000 7f8df37fe640 1 --2- 172.21.15.88:0/3793810037 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8df40d26c0 0x7f8df414e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:51.283 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.276+0000 7f8df8aac640 1 -- 172.21.15.88:0/3793810037 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8df40d26c0 msgr2=0x7f8df414e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:51.283 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.276+0000 7f8df8aac640 1 --2- 172.21.15.88:0/3793810037 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8df40d26c0 0x7f8df414e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:51.283 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.276+0000 7f8df8aac640 1 -- 172.21.15.88:0/3793810037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8df40d22e0 msgr2=0x7f8df41f7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:51.283 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.276+0000 7f8df8aac640 1 --2- 172.21.15.88:0/3793810037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8df40d22e0 0x7f8df41f7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:51.284 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.276+0000 7f8df8aac640 1 -- 172.21.15.88:0/3793810037 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de4009420 con 0x7f8df415d9f0 2023-10-16T20:05:51.284 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.276+0000 7f8df8aac640 1 --2- 172.21.15.88:0/3793810037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8df415d9f0 0x7f8df414edf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8dec00efc0 tx=0x7f8dec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:51.284 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.276+0000 7f8df0ff9640 1 -- 172.21.15.88:0/3793810037 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dec009ba0 con 0x7f8df415d9f0 2023-10-16T20:05:51.284 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.276+0000 7f8df0ff9640 1 -- 172.21.15.88:0/3793810037 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dec00f040 con 0x7f8df415d9f0 2023-10-16T20:05:51.284 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.276+0000 7f8df0ff9640 1 -- 172.21.15.88:0/3793810037 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dec0146c0 con 0x7f8df415d9f0 2023-10-16T20:05:51.284 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.276+0000 7f8dfa50f640 1 -- 172.21.15.88:0/3793810037 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df4152cf0 con 0x7f8df415d9f0 2023-10-16T20:05:51.285 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.276+0000 7f8dfa50f640 1 -- 172.21.15.88:0/3793810037 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df41534f0 con 0x7f8df415d9f0 2023-10-16T20:05:51.285 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.276+0000 7f8dfa50f640 1 -- 172.21.15.88:0/3793810037 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df404e9c0 con 0x7f8df415d9f0 2023-10-16T20:05:51.286 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.276+0000 7f8df0ff9640 1 -- 172.21.15.88:0/3793810037 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8dec014820 con 0x7f8df415d9f0 2023-10-16T20:05:51.286 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.280+0000 7f8df0ff9640 1 --2- 172.21.15.88:0/3793810037 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f8de00664a0 0x7f8de0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:51.286 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.280+0000 7f8df0ff9640 1 -- 172.21.15.88:0/3793810037 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8dec084cc0 con 0x7f8df415d9f0 2023-10-16T20:05:51.286 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.280+0000 7f8df3fff640 1 --2- 172.21.15.88:0/3793810037 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f8de00664a0 0x7f8de0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:51.287 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.280+0000 7f8df3fff640 1 --2- 172.21.15.88:0/3793810037 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f8de00664a0 0x7f8de0068960 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8de40ae000 tx=0x7f8de40a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:51.287 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.284+0000 7f8df0ff9640 1 -- 172.21.15.88:0/3793810037 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8dec051d80 con 0x7f8df415d9f0 2023-10-16T20:05:51.437 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.432+0000 7f8dfa50f640 1 -- 172.21.15.88:0/3793810037 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8df41531a0 con 0x7f8df415d9f0 2023-10-16T20:05:51.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.436+0000 7f8df0ff9640 1 -- 172.21.15.88:0/3793810037 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18661 (secure 0 0 0) 0x7f8dec018020 con 0x7f8df415d9f0 2023-10-16T20:05:51.439 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:51.439 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":29,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:05:38.202959+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T20:04:54.442487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T20:04:55.766641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T20:04:59.475830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":1}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:05:51.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.436+0000 7f8dfa50f640 1 -- 172.21.15.88:0/3793810037 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f8de00664a0 msgr2=0x7f8de0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:51.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.436+0000 7f8dfa50f640 1 --2- 172.21.15.88:0/3793810037 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f8de00664a0 0x7f8de0068960 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8de40ae000 tx=0x7f8de40a7000 comp rx=0 tx=0).stop 2023-10-16T20:05:51.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.436+0000 7f8dfa50f640 1 -- 172.21.15.88:0/3793810037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8df415d9f0 msgr2=0x7f8df414edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:51.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.436+0000 7f8dfa50f640 1 --2- 172.21.15.88:0/3793810037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8df415d9f0 0x7f8df414edf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8dec00efc0 tx=0x7f8dec00c490 comp rx=0 tx=0).stop 2023-10-16T20:05:51.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.436+0000 7f8dfa50f640 1 -- 172.21.15.88:0/3793810037 shutdown_connections 2023-10-16T20:05:51.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.436+0000 7f8dfa50f640 1 --2- 172.21.15.88:0/3793810037 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f8de00664a0 0x7f8de0068960 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:51.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.436+0000 7f8dfa50f640 1 --2- 172.21.15.88:0/3793810037 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8df40d26c0 0x7f8df414e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:51.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.436+0000 7f8dfa50f640 1 --2- 172.21.15.88:0/3793810037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8df415d9f0 0x7f8df414edf0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:51.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.436+0000 7f8dfa50f640 1 --2- 172.21.15.88:0/3793810037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8df40d22e0 0x7f8df41f7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:51.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.436+0000 7f8dfa50f640 1 -- 172.21.15.88:0/3793810037 >> 172.21.15.88:0/3793810037 conn(0x7f8df40715c0 msgr2=0x7f8df4153a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:51.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.436+0000 7f8dfa50f640 1 -- 172.21.15.88:0/3793810037 shutdown_connections 2023-10-16T20:05:51.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.436+0000 7f8dfa50f640 1 -- 172.21.15.88:0/3793810037 wait complete. 2023-10-16T20:05:51.460 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T20:04:54.442487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T20:04:55.766641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T20:04:59.475830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T20:05:51.461 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:05:51.601 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.596+0000 7f499790e640 1 Processor -- start 2023-10-16T20:05:51.602 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.596+0000 7f499790e640 1 -- start start 2023-10-16T20:05:51.602 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.600+0000 7f499790e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f49900b4260 0x7f499015dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:51.603 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.600+0000 7f499790e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49900b0da0 0x7f4990159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:51.603 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.600+0000 7f499790e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f499015ad30 con 0x7f499015a1f0 2023-10-16T20:05:51.603 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.600+0000 7f499790e640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49900b4640 con 0x7f49900b4260 2023-10-16T20:05:51.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.600+0000 7f499790e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499015a770 con 0x7f49900b0da0 2023-10-16T20:05:51.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.600+0000 7f4994ea9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49900b0da0 0x7f4990159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:51.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.600+0000 7f49956aa640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f49900b4260 0x7f499015dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:51.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.600+0000 7f4995eab640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f499015a1f0 0x7f499006d3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:47592/0 (socket says 172.21.15.88:47592) 2023-10-16T20:05:51.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.600+0000 7f4995eab640 1 -- 172.21.15.88:0/1730447005 learned_addr learned my addr 172.21.15.88:0/1730447005 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:51.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.600+0000 7f4987fff640 1 -- 172.21.15.88:0/1730447005 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2547290821 0 0) 0x7f499015ad30 con 0x7f499015a1f0 2023-10-16T20:05:51.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.600+0000 7f4987fff640 1 -- 172.21.15.88:0/1730447005 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f497c005740 con 0x7f499015a1f0 2023-10-16T20:05:51.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.600+0000 7f4994ea9640 1 -- 172.21.15.88:0/1730447005 >> v2:172.21.15.94:3300/0 conn(0x7f49900b4260 msgr2=0x7f499015dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:51.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.600+0000 7f4994ea9640 1 --2- 172.21.15.88:0/1730447005 >> v2:172.21.15.94:3300/0 conn(0x7f49900b4260 0x7f499015dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:51.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.600+0000 7f4994ea9640 1 -- 172.21.15.88:0/1730447005 >> v1:172.21.15.94:6789/0 conn(0x7f499015a1f0 legacy=0x7f499006d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:51.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.600+0000 7f4994ea9640 1 -- 172.21.15.88:0/1730447005 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499015aa50 con 0x7f49900b0da0 2023-10-16T20:05:51.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.600+0000 7f49956aa640 1 --2- 172.21.15.88:0/1730447005 >> v2:172.21.15.94:3300/0 conn(0x7f49900b4260 0x7f499015dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:05:51.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.604+0000 7f4994ea9640 1 --2- 172.21.15.88:0/1730447005 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49900b0da0 0x7f4990159cb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f49800097c0 tx=0x7f498001c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15d690831bfb75d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:51.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.604+0000 7f4987fff640 1 -- 172.21.15.88:0/1730447005 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49800ac620 con 0x7f49900b0da0 2023-10-16T20:05:51.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.604+0000 7f4987fff640 1 -- 172.21.15.88:0/1730447005 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49800ac780 con 0x7f49900b0da0 2023-10-16T20:05:51.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.604+0000 7f4987fff640 1 -- 172.21.15.88:0/1730447005 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49800acb10 con 0x7f49900b0da0 2023-10-16T20:05:51.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.604+0000 7f499790e640 1 -- 172.21.15.88:0/1730447005 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49900b0da0 msgr2=0x7f4990159cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:51.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.604+0000 7f499790e640 1 --2- 172.21.15.88:0/1730447005 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49900b0da0 0x7f4990159cb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f49800097c0 tx=0x7f498001c3c0 comp rx=0 tx=0).stop 2023-10-16T20:05:51.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.604+0000 7f499790e640 1 -- 172.21.15.88:0/1730447005 shutdown_connections 2023-10-16T20:05:51.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.604+0000 7f499790e640 1 --2- 172.21.15.88:0/1730447005 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49900b0da0 0x7f4990159cb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:51.610 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.604+0000 7f499790e640 1 --2- 172.21.15.88:0/1730447005 >> v2:172.21.15.94:3300/0 conn(0x7f49900b4260 0x7f499015dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:51.610 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.604+0000 7f499790e640 1 -- 172.21.15.88:0/1730447005 >> 172.21.15.88:0/1730447005 conn(0x7f49900cb080 msgr2=0x7f49900cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:51.610 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.604+0000 7f499790e640 1 -- 172.21.15.88:0/1730447005 shutdown_connections 2023-10-16T20:05:51.610 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.604+0000 7f499790e640 1 -- 172.21.15.88:0/1730447005 wait complete. 2023-10-16T20:05:51.612 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.604+0000 7f499790e640 1 Processor -- start 2023-10-16T20:05:51.612 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.604+0000 7f499790e640 1 -- start start 2023-10-16T20:05:51.613 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.608+0000 7f499790e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49900b0da0 0x7f499015b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:51.613 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.608+0000 7f499790e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49900b4260 0x7f4990156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:51.613 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.608+0000 7f499790e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f499015a1f0 0x7f49901542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:51.613 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.608+0000 7f499790e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499013ada0 con 0x7f49900b4260 2023-10-16T20:05:51.614 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.608+0000 7f499790e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499013a190 con 0x7f499015a1f0 2023-10-16T20:05:51.614 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.608+0000 7f499790e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499013a470 con 0x7f49900b0da0 2023-10-16T20:05:51.614 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.608+0000 7f4994ea9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49900b4260 0x7f4990156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:51.614 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.608+0000 7f4994ea9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49900b4260 0x7f4990156190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48302/0 (socket says 172.21.15.88:48302) 2023-10-16T20:05:51.614 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.608+0000 7f4994ea9640 1 -- 172.21.15.88:0/999048102 learned_addr learned my addr 172.21.15.88:0/999048102 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:51.615 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.608+0000 7f49956aa640 1 --2- 172.21.15.88:0/999048102 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49900b0da0 0x7f499015b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:51.615 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.608+0000 7f4995eab640 1 --2- 172.21.15.88:0/999048102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f499015a1f0 0x7f49901542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:51.615 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.608+0000 7f4994ea9640 1 -- 172.21.15.88:0/999048102 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49900b0da0 msgr2=0x7f499015b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:51.615 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.608+0000 7f4994ea9640 1 --2- 172.21.15.88:0/999048102 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49900b0da0 0x7f499015b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:51.615 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.608+0000 7f4994ea9640 1 -- 172.21.15.88:0/999048102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f499015a1f0 msgr2=0x7f49901542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:51.616 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.608+0000 7f4994ea9640 1 --2- 172.21.15.88:0/999048102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f499015a1f0 0x7f49901542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:51.616 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.608+0000 7f4994ea9640 1 -- 172.21.15.88:0/999048102 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498c009420 con 0x7f49900b4260 2023-10-16T20:05:51.616 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.608+0000 7f49956aa640 1 --2- 172.21.15.88:0/999048102 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49900b0da0 0x7f499015b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:05:51.616 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.608+0000 7f4994ea9640 1 --2- 172.21.15.88:0/999048102 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49900b4260 0x7f4990156190 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f49800af040 tx=0x7f4980019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:51.616 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.608+0000 7f4985ffb640 1 -- 172.21.15.88:0/999048102 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49800ac620 con 0x7f49900b4260 2023-10-16T20:05:51.617 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.608+0000 7f4985ffb640 1 -- 172.21.15.88:0/999048102 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49800b5410 con 0x7f49900b4260 2023-10-16T20:05:51.617 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.608+0000 7f4985ffb640 1 -- 172.21.15.88:0/999048102 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49800b57e0 con 0x7f49900b4260 2023-10-16T20:05:51.617 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.608+0000 7f499790e640 1 -- 172.21.15.88:0/999048102 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4980009420 con 0x7f49900b4260 2023-10-16T20:05:51.617 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.608+0000 7f499790e640 1 -- 172.21.15.88:0/999048102 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f499013aa30 con 0x7f49900b4260 2023-10-16T20:05:51.618 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.608+0000 7f499790e640 1 -- 172.21.15.88:0/999048102 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f499013a5b0 con 0x7f49900b4260 2023-10-16T20:05:51.618 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.608+0000 7f4985ffb640 1 -- 172.21.15.88:0/999048102 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f49800b3040 con 0x7f49900b4260 2023-10-16T20:05:51.618 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.612+0000 7f4985ffb640 1 --2- 172.21.15.88:0/999048102 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f49740664a0 0x7f4974068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:51.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.612+0000 7f4985ffb640 1 -- 172.21.15.88:0/999048102 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f498011e1f0 con 0x7f49900b4260 2023-10-16T20:05:51.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.612+0000 7f49956aa640 1 --2- 172.21.15.88:0/999048102 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f49740664a0 0x7f4974068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:51.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.612+0000 7f49956aa640 1 --2- 172.21.15.88:0/999048102 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f49740664a0 0x7f4974068960 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f498c009fd0 tx=0x7f498c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:51.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.616+0000 7f4985ffb640 1 -- 172.21.15.88:0/999048102 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f499013a5b0 con 0x7f49900b4260 2023-10-16T20:05:51.766 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.764+0000 7f499790e640 1 -- 172.21.15.88:0/999048102 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f499013a5b0 con 0x7f49900b4260 2023-10-16T20:05:51.767 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.764+0000 7f4985ffb640 1 -- 172.21.15.88:0/999048102 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f499013a5b0 con 0x7f49900b4260 2023-10-16T20:05:51.767 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:05:51.768 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.764+0000 7f499790e640 1 -- 172.21.15.88:0/999048102 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f49740664a0 msgr2=0x7f4974068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:51.769 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.764+0000 7f499790e640 1 --2- 172.21.15.88:0/999048102 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f49740664a0 0x7f4974068960 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f498c009fd0 tx=0x7f498c009190 comp rx=0 tx=0).stop 2023-10-16T20:05:51.769 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.764+0000 7f499790e640 1 -- 172.21.15.88:0/999048102 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49900b4260 msgr2=0x7f4990156190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:51.769 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.764+0000 7f499790e640 1 --2- 172.21.15.88:0/999048102 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49900b4260 0x7f4990156190 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f49800af040 tx=0x7f4980019160 comp rx=0 tx=0).stop 2023-10-16T20:05:51.769 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.764+0000 7f499790e640 1 -- 172.21.15.88:0/999048102 shutdown_connections 2023-10-16T20:05:51.770 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.764+0000 7f499790e640 1 --2- 172.21.15.88:0/999048102 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f49740664a0 0x7f4974068960 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:51.770 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.764+0000 7f499790e640 1 --2- 172.21.15.88:0/999048102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f499015a1f0 0x7f49901542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:51.770 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.764+0000 7f499790e640 1 --2- 172.21.15.88:0/999048102 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49900b4260 0x7f4990156190 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:51.770 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.764+0000 7f499790e640 1 --2- 172.21.15.88:0/999048102 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49900b0da0 0x7f499015b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:51.770 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.764+0000 7f499790e640 1 -- 172.21.15.88:0/999048102 >> 172.21.15.88:0/999048102 conn(0x7f49900cb080 msgr2=0x7f4990163ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:51.771 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.764+0000 7f499790e640 1 -- 172.21.15.88:0/999048102 shutdown_connections 2023-10-16T20:05:51.771 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.764+0000 7f499790e640 1 -- 172.21.15.88:0/999048102 wait complete. 2023-10-16T20:05:51.787 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T20:05:51.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.924+0000 7fa25fec9640 1 Processor -- start 2023-10-16T20:05:51.929 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.924+0000 7fa25fec9640 1 -- start start 2023-10-16T20:05:51.930 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.924+0000 7fa25fec9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2580addf0 0x7fa2580ae230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:51.930 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.924+0000 7fa25fec9640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2580afff0 con 0x7fa258164570 2023-10-16T20:05:51.930 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.924+0000 7fa25fec9640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2580ae840 con 0x7fa258070250 2023-10-16T20:05:51.931 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.924+0000 7fa25fec9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2580aeb20 con 0x7fa2580addf0 2023-10-16T20:05:51.931 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.924+0000 7fa25dc65640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fa258164570 0x7fa2580af8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:34776/0 (socket says 172.21.15.88:34776) 2023-10-16T20:05:51.931 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.924+0000 7fa25dc65640 1 -- 172.21.15.88:0/3634881978 learned_addr learned my addr 172.21.15.88:0/3634881978 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:51.931 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.924+0000 7fa25d464640 1 --2- 172.21.15.88:0/3634881978 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2580addf0 0x7fa2580ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:51.932 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.928+0000 7fa25cc63640 1 -- 172.21.15.88:0/3634881978 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2838489288 0 0) 0x7fa2580afff0 con 0x7fa258164570 2023-10-16T20:05:51.932 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.928+0000 7fa25cc63640 1 -- 172.21.15.88:0/3634881978 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa244003530 con 0x7fa258164570 2023-10-16T20:05:51.933 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.928+0000 7fa25cc63640 1 -- 172.21.15.88:0/3634881978 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2764867081 0 0) 0x7fa2580ae840 con 0x7fa258070250 2023-10-16T20:05:51.933 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.928+0000 7fa25cc63640 1 -- 172.21.15.88:0/3634881978 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2580afff0 con 0x7fa258070250 2023-10-16T20:05:51.933 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.928+0000 7fa25d464640 1 -- 172.21.15.88:0/3634881978 >> v1:172.21.15.94:6789/0 conn(0x7fa258070250 legacy=0x7fa2580b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:51.933 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.928+0000 7fa25d464640 1 -- 172.21.15.88:0/3634881978 >> v1:172.21.15.88:6789/0 conn(0x7fa258164570 legacy=0x7fa2580af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:51.933 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.928+0000 7fa25d464640 1 -- 172.21.15.88:0/3634881978 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2580aee00 con 0x7fa2580addf0 2023-10-16T20:05:51.935 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.928+0000 7fa25d464640 1 --2- 172.21.15.88:0/3634881978 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2580addf0 0x7fa2580ae230 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa2480097f0 tx=0x7fa24801ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2fed8d206a1ddae server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:51.935 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.928+0000 7fa25cc63640 1 -- 172.21.15.88:0/3634881978 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2480a89b0 con 0x7fa2580addf0 2023-10-16T20:05:51.935 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.928+0000 7fa25cc63640 1 -- 172.21.15.88:0/3634881978 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2480a8b10 con 0x7fa2580addf0 2023-10-16T20:05:51.936 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.928+0000 7fa25cc63640 1 -- 172.21.15.88:0/3634881978 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2480a8ea0 con 0x7fa2580addf0 2023-10-16T20:05:51.936 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.932+0000 7fa25fec9640 1 -- 172.21.15.88:0/3634881978 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2580addf0 msgr2=0x7fa2580ae230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:51.936 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.932+0000 7fa25fec9640 1 --2- 172.21.15.88:0/3634881978 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2580addf0 0x7fa2580ae230 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa2480097f0 tx=0x7fa24801ecb0 comp rx=0 tx=0).stop 2023-10-16T20:05:51.936 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.932+0000 7fa25fec9640 1 -- 172.21.15.88:0/3634881978 shutdown_connections 2023-10-16T20:05:51.936 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.932+0000 7fa25fec9640 1 --2- 172.21.15.88:0/3634881978 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2580addf0 0x7fa2580ae230 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:51.937 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.932+0000 7fa25fec9640 1 -- 172.21.15.88:0/3634881978 >> 172.21.15.88:0/3634881978 conn(0x7fa2580cb080 msgr2=0x7fa2580cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:51.937 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.932+0000 7fa25fec9640 1 -- 172.21.15.88:0/3634881978 shutdown_connections 2023-10-16T20:05:51.937 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.932+0000 7fa25fec9640 1 -- 172.21.15.88:0/3634881978 wait complete. 2023-10-16T20:05:51.939 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.932+0000 7fa25fec9640 1 Processor -- start 2023-10-16T20:05:51.940 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.932+0000 7fa25fec9640 1 -- start start 2023-10-16T20:05:51.940 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.932+0000 7fa25fec9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa258070250 0x7fa25814ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:51.940 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.932+0000 7fa25fec9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2580addf0 0x7fa2580b2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:51.940 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.932+0000 7fa25fec9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa258164570 0x7fa25815aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:51.941 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.932+0000 7fa25fec9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2580b2a40 con 0x7fa258070250 2023-10-16T20:05:51.941 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.932+0000 7fa25fec9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa258159a10 con 0x7fa258164570 2023-10-16T20:05:51.941 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.932+0000 7fa25fec9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa258159cb0 con 0x7fa2580addf0 2023-10-16T20:05:51.941 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.932+0000 7fa25dc65640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa258070250 0x7fa25814ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:51.941 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.932+0000 7fa25dc65640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa258070250 0x7fa25814ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48306/0 (socket says 172.21.15.88:48306) 2023-10-16T20:05:51.942 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.932+0000 7fa25dc65640 1 -- 172.21.15.88:0/129591658 learned_addr learned my addr 172.21.15.88:0/129591658 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:51.942 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.932+0000 7fa25d464640 1 --2- 172.21.15.88:0/129591658 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2580addf0 0x7fa2580b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:51.942 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.932+0000 7fa25e466640 1 --2- 172.21.15.88:0/129591658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa258164570 0x7fa25815aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:51.942 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.936+0000 7fa25dc65640 1 -- 172.21.15.88:0/129591658 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2580addf0 msgr2=0x7fa2580b2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:51.943 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.936+0000 7fa25dc65640 1 --2- 172.21.15.88:0/129591658 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2580addf0 0x7fa2580b2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:51.943 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.936+0000 7fa25dc65640 1 -- 172.21.15.88:0/129591658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa258164570 msgr2=0x7fa25815aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:51.943 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.936+0000 7fa25dc65640 1 --2- 172.21.15.88:0/129591658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa258164570 0x7fa25815aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:51.943 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.936+0000 7fa25dc65640 1 -- 172.21.15.88:0/129591658 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa248009420 con 0x7fa258070250 2023-10-16T20:05:51.943 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.936+0000 7fa25dc65640 1 --2- 172.21.15.88:0/129591658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa258070250 0x7fa25814ea30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa25400c4d0 tx=0x7fa25400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:51.944 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.936+0000 7fa24e7fc640 1 -- 172.21.15.88:0/129591658 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa254005cf0 con 0x7fa258070250 2023-10-16T20:05:51.944 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.936+0000 7fa24e7fc640 1 -- 172.21.15.88:0/129591658 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa254005e50 con 0x7fa258070250 2023-10-16T20:05:51.944 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.936+0000 7fa24e7fc640 1 -- 172.21.15.88:0/129591658 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2540072b0 con 0x7fa258070250 2023-10-16T20:05:51.944 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.936+0000 7fa25fec9640 1 -- 172.21.15.88:0/129591658 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa258159f50 con 0x7fa258070250 2023-10-16T20:05:51.944 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.936+0000 7fa25fec9640 1 -- 172.21.15.88:0/129591658 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa25815a510 con 0x7fa258070250 2023-10-16T20:05:51.946 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.936+0000 7fa25fec9640 1 -- 172.21.15.88:0/129591658 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2580670c0 con 0x7fa258070250 2023-10-16T20:05:51.946 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.936+0000 7fa24e7fc640 1 -- 172.21.15.88:0/129591658 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa254007410 con 0x7fa258070250 2023-10-16T20:05:51.946 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.936+0000 7fa24e7fc640 1 --2- 172.21.15.88:0/129591658 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa23c0664a0 0x7fa23c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:51.947 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.936+0000 7fa24e7fc640 1 -- 172.21.15.88:0/129591658 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa254083870 con 0x7fa258070250 2023-10-16T20:05:51.947 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.936+0000 7fa25d464640 1 --2- 172.21.15.88:0/129591658 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa23c0664a0 0x7fa23c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:51.947 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.940+0000 7fa25d464640 1 --2- 172.21.15.88:0/129591658 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa23c0664a0 0x7fa23c068960 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa2480b2000 tx=0x7fa248008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:51.947 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:51.940+0000 7fa24e7fc640 1 -- 172.21.15.88:0/129591658 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa254050930 con 0x7fa258070250 2023-10-16T20:05:52.094 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.092+0000 7fa25fec9640 1 -- 172.21.15.88:0/129591658 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa25804e9c0 con 0x7fa258070250 2023-10-16T20:05:52.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.092+0000 7fa24e7fc640 1 -- 172.21.15.88:0/129591658 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa254017020 con 0x7fa258070250 2023-10-16T20:05:52.095 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:05:52.097 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.092+0000 7fa25fec9640 1 -- 172.21.15.88:0/129591658 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa23c0664a0 msgr2=0x7fa23c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:52.097 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.092+0000 7fa25fec9640 1 --2- 172.21.15.88:0/129591658 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa23c0664a0 0x7fa23c068960 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa2480b2000 tx=0x7fa248008040 comp rx=0 tx=0).stop 2023-10-16T20:05:52.098 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.092+0000 7fa25fec9640 1 -- 172.21.15.88:0/129591658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa258070250 msgr2=0x7fa25814ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:52.098 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.092+0000 7fa25fec9640 1 --2- 172.21.15.88:0/129591658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa258070250 0x7fa25814ea30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa25400c4d0 tx=0x7fa25400c990 comp rx=0 tx=0).stop 2023-10-16T20:05:52.098 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.092+0000 7fa25fec9640 1 -- 172.21.15.88:0/129591658 shutdown_connections 2023-10-16T20:05:52.098 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.092+0000 7fa25fec9640 1 --2- 172.21.15.88:0/129591658 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa23c0664a0 0x7fa23c068960 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:52.098 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.092+0000 7fa25fec9640 1 --2- 172.21.15.88:0/129591658 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2580addf0 0x7fa2580b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:52.099 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.092+0000 7fa25fec9640 1 --2- 172.21.15.88:0/129591658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa258164570 0x7fa25815aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:52.099 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.092+0000 7fa25fec9640 1 --2- 172.21.15.88:0/129591658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa258070250 0x7fa25814ea30 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:52.099 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.092+0000 7fa25fec9640 1 -- 172.21.15.88:0/129591658 >> 172.21.15.88:0/129591658 conn(0x7fa2580cb080 msgr2=0x7fa258158280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:52.099 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.092+0000 7fa25fec9640 1 -- 172.21.15.88:0/129591658 shutdown_connections 2023-10-16T20:05:52.099 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.092+0000 7fa25fec9640 1 -- 172.21.15.88:0/129591658 wait complete. 2023-10-16T20:05:52.116 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T20:05:52.256 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.252+0000 7f192a4ea640 1 Processor -- start 2023-10-16T20:05:52.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.252+0000 7f192a4ea640 1 -- start start 2023-10-16T20:05:52.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.252+0000 7f192a4ea640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f192415a820 0x7f19240cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:52.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.252+0000 7f192a4ea640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f192415ac00 0x7f19240b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:52.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.252+0000 7f192a4ea640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f19240b2930 0x7f192406c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:52.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.252+0000 7f192a4ea640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19240b2da0 con 0x7f19240b2930 2023-10-16T20:05:52.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.252+0000 7f192a4ea640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19240ce4b0 con 0x7f192415ac00 2023-10-16T20:05:52.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.252+0000 7f192a4ea640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f192415e270 con 0x7f192415a820 2023-10-16T20:05:52.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.252+0000 7f1928a87640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f19240b2930 0x7f192406c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:52.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.252+0000 7f1928a87640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f19240b2930 0x7f192406c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50476/0 (socket says 172.21.15.88:50476) 2023-10-16T20:05:52.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.252+0000 7f1928a87640 1 -- 172.21.15.88:0/334110952 learned_addr learned my addr 172.21.15.88:0/334110952 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:52.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.252+0000 7f19237fe640 1 --2- 172.21.15.88:0/334110952 >> v2:172.21.15.94:3300/0 conn(0x7f192415ac00 0x7f19240b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:52.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.252+0000 7f1923fff640 1 --2- 172.21.15.88:0/334110952 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f192415a820 0x7f19240cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:52.261 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.256+0000 7f1928a87640 1 -- 172.21.15.88:0/334110952 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f192415a820 msgr2=0x7f19240cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:52.261 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.256+0000 7f1928a87640 1 --2- 172.21.15.88:0/334110952 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f192415a820 0x7f19240cf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:52.261 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.256+0000 7f1928a87640 1 -- 172.21.15.88:0/334110952 >> v2:172.21.15.94:3300/0 conn(0x7f192415ac00 msgr2=0x7f19240b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:52.261 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.256+0000 7f1928a87640 1 --2- 172.21.15.88:0/334110952 >> v2:172.21.15.94:3300/0 conn(0x7f192415ac00 0x7f19240b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:52.261 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.256+0000 7f1928a87640 1 -- 172.21.15.88:0/334110952 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19240ce700 con 0x7f19240b2930 2023-10-16T20:05:52.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.256+0000 7f1923fff640 1 --2- 172.21.15.88:0/334110952 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f192415a820 0x7f19240cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:52.263 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.256+0000 7f1928a87640 1 --2- 172.21.15.88:0/334110952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19240b2930 0x7f192406c340 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1924070ff0 tx=0x7f191c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65be0ee7aa927606 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:52.263 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.256+0000 7f1922ffd640 1 -- 172.21.15.88:0/334110952 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c0a5820 con 0x7f19240b2930 2023-10-16T20:05:52.263 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.256+0000 7f1922ffd640 1 -- 172.21.15.88:0/334110952 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191c0a5980 con 0x7f19240b2930 2023-10-16T20:05:52.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.256+0000 7f1922ffd640 1 -- 172.21.15.88:0/334110952 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c0a5d10 con 0x7f19240b2930 2023-10-16T20:05:52.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.256+0000 7f192a4ea640 1 -- 172.21.15.88:0/334110952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19240b2930 msgr2=0x7f192406c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:52.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.256+0000 7f192a4ea640 1 --2- 172.21.15.88:0/334110952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19240b2930 0x7f192406c340 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1924070ff0 tx=0x7f191c01c9a0 comp rx=0 tx=0).stop 2023-10-16T20:05:52.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.256+0000 7f192a4ea640 1 -- 172.21.15.88:0/334110952 shutdown_connections 2023-10-16T20:05:52.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.256+0000 7f192a4ea640 1 --2- 172.21.15.88:0/334110952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19240b2930 0x7f192406c340 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:52.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.256+0000 7f192a4ea640 1 --2- 172.21.15.88:0/334110952 >> v2:172.21.15.94:3300/0 conn(0x7f192415ac00 0x7f19240b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:52.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.256+0000 7f192a4ea640 1 --2- 172.21.15.88:0/334110952 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f192415a820 0x7f19240cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:52.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.256+0000 7f192a4ea640 1 -- 172.21.15.88:0/334110952 >> 172.21.15.88:0/334110952 conn(0x7f19240cd3f0 msgr2=0x7f19240cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:52.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.256+0000 7f192a4ea640 1 -- 172.21.15.88:0/334110952 shutdown_connections 2023-10-16T20:05:52.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.260+0000 7f192a4ea640 1 -- 172.21.15.88:0/334110952 wait complete. 2023-10-16T20:05:52.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.260+0000 7f192a4ea640 1 Processor -- start 2023-10-16T20:05:52.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.260+0000 7f192a4ea640 1 -- start start 2023-10-16T20:05:52.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.260+0000 7f192a4ea640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19240b2930 0x7f19240b5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:52.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.260+0000 7f192a4ea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f192415a820 0x7f19241fa760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:52.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.260+0000 7f192a4ea640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f192415ac00 0x7f19241552f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:52.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.260+0000 7f192a4ea640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19241f96a0 con 0x7f192415ac00 2023-10-16T20:05:52.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.260+0000 7f192a4ea640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19240b64c0 con 0x7f192415a820 2023-10-16T20:05:52.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.260+0000 7f192a4ea640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19240b3970 con 0x7f19240b2930 2023-10-16T20:05:52.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.260+0000 7f1928a87640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f192415ac00 0x7f19241552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:52.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.260+0000 7f1928a87640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f192415ac00 0x7f19241552f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50488/0 (socket says 172.21.15.88:50488) 2023-10-16T20:05:52.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.260+0000 7f1928a87640 1 -- 172.21.15.88:0/2198247243 learned_addr learned my addr 172.21.15.88:0/2198247243 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:52.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.260+0000 7f1923fff640 1 --2- 172.21.15.88:0/2198247243 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19240b2930 0x7f19240b5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:52.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.260+0000 7f19237fe640 1 --2- 172.21.15.88:0/2198247243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f192415a820 0x7f19241fa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:52.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.260+0000 7f1928a87640 1 -- 172.21.15.88:0/2198247243 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19240b2930 msgr2=0x7f19240b5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:52.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.260+0000 7f1928a87640 1 --2- 172.21.15.88:0/2198247243 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19240b2930 0x7f19240b5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:52.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.260+0000 7f1928a87640 1 -- 172.21.15.88:0/2198247243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f192415a820 msgr2=0x7f19241fa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:52.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.260+0000 7f1928a87640 1 --2- 172.21.15.88:0/2198247243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f192415a820 0x7f19241fa760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:52.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.260+0000 7f1928a87640 1 -- 172.21.15.88:0/2198247243 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f191c009b30 con 0x7f192415ac00 2023-10-16T20:05:52.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.264+0000 7f1928a87640 1 --2- 172.21.15.88:0/2198247243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f192415ac00 0x7f19241552f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f191c0af040 tx=0x7f191c019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:52.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.264+0000 7f1920ff9640 1 -- 172.21.15.88:0/2198247243 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c0b8670 con 0x7f192415ac00 2023-10-16T20:05:52.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.264+0000 7f1920ff9640 1 -- 172.21.15.88:0/2198247243 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191c0b87f0 con 0x7f192415ac00 2023-10-16T20:05:52.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.264+0000 7f1920ff9640 1 -- 172.21.15.88:0/2198247243 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c0b8ba0 con 0x7f192415ac00 2023-10-16T20:05:52.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.264+0000 7f192a4ea640 1 -- 172.21.15.88:0/2198247243 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19240b3c50 con 0x7f192415ac00 2023-10-16T20:05:52.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.264+0000 7f192a4ea640 1 -- 172.21.15.88:0/2198247243 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19240b4210 con 0x7f192415ac00 2023-10-16T20:05:52.273 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.264+0000 7f192a4ea640 1 -- 172.21.15.88:0/2198247243 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19241f9810 con 0x7f192415ac00 2023-10-16T20:05:52.273 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.264+0000 7f1920ff9640 1 -- 172.21.15.88:0/2198247243 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f191c0b8d00 con 0x7f192415ac00 2023-10-16T20:05:52.273 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.264+0000 7f1920ff9640 1 --2- 172.21.15.88:0/2198247243 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f19100664a0 0x7f1910068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:52.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.264+0000 7f1920ff9640 1 -- 172.21.15.88:0/2198247243 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f191c11f790 con 0x7f192415ac00 2023-10-16T20:05:52.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.268+0000 7f1923fff640 1 --2- 172.21.15.88:0/2198247243 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f19100664a0 0x7f1910068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:52.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.268+0000 7f1923fff640 1 --2- 172.21.15.88:0/2198247243 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f19100664a0 0x7f1910068960 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1914006b20 tx=0x7f1914008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:52.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.268+0000 7f1920ff9640 1 -- 172.21.15.88:0/2198247243 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f191c0ec980 con 0x7f192415ac00 2023-10-16T20:05:52.425 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.420+0000 7f192a4ea640 1 -- 172.21.15.88:0/2198247243 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f192404e9c0 con 0x7f192415ac00 2023-10-16T20:05:52.426 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.420+0000 7f1920ff9640 1 -- 172.21.15.88:0/2198247243 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f191c0effa0 con 0x7f192415ac00 2023-10-16T20:05:52.426 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2023-10-16T20:05:52.427 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.424+0000 7f192a4ea640 1 -- 172.21.15.88:0/2198247243 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f19100664a0 msgr2=0x7f1910068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:52.428 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.424+0000 7f192a4ea640 1 --2- 172.21.15.88:0/2198247243 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f19100664a0 0x7f1910068960 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1914006b20 tx=0x7f1914008040 comp rx=0 tx=0).stop 2023-10-16T20:05:52.428 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.424+0000 7f192a4ea640 1 -- 172.21.15.88:0/2198247243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f192415ac00 msgr2=0x7f19241552f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:52.428 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.424+0000 7f192a4ea640 1 --2- 172.21.15.88:0/2198247243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f192415ac00 0x7f19241552f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f191c0af040 tx=0x7f191c019740 comp rx=0 tx=0).stop 2023-10-16T20:05:52.428 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.424+0000 7f192a4ea640 1 -- 172.21.15.88:0/2198247243 shutdown_connections 2023-10-16T20:05:52.428 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.424+0000 7f192a4ea640 1 --2- 172.21.15.88:0/2198247243 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f19100664a0 0x7f1910068960 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:52.429 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.424+0000 7f192a4ea640 1 --2- 172.21.15.88:0/2198247243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f192415ac00 0x7f19241552f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:52.429 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.424+0000 7f192a4ea640 1 --2- 172.21.15.88:0/2198247243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f192415a820 0x7f19241fa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:52.429 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.424+0000 7f192a4ea640 1 --2- 172.21.15.88:0/2198247243 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19240b2930 0x7f19240b5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:52.429 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.424+0000 7f192a4ea640 1 -- 172.21.15.88:0/2198247243 >> 172.21.15.88:0/2198247243 conn(0x7f19240cd3f0 msgr2=0x7f192415b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:52.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.424+0000 7f192a4ea640 1 -- 172.21.15.88:0/2198247243 shutdown_connections 2023-10-16T20:05:52.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.424+0000 7f192a4ea640 1 -- 172.21.15.88:0/2198247243 wait complete. 2023-10-16T20:05:52.447 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T20:05:52.586 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.580+0000 7f56f0cc9640 1 Processor -- start 2023-10-16T20:05:52.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56f0cc9640 1 -- start start 2023-10-16T20:05:52.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56f0cc9640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f56ec0b0c10 0x7f56ec15aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:52.588 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56f0cc9640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56ec15bd20 con 0x7f56ec15b530 2023-10-16T20:05:52.588 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56f0cc9640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56ec0702f0 con 0x7f56ec154f30 2023-10-16T20:05:52.588 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56f0cc9640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56ec155310 con 0x7f56ec0b0c10 2023-10-16T20:05:52.588 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56ead9d640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f56ec15b530 0x7f56ec06c2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:53448/0 (socket says 172.21.15.88:53448) 2023-10-16T20:05:52.588 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56ead9d640 1 -- 172.21.15.88:0/1012022364 learned_addr learned my addr 172.21.15.88:0/1012022364 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:52.591 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56e1d9b640 1 --2- 172.21.15.88:0/1012022364 >> v2:172.21.15.94:3300/0 conn(0x7f56ec0b0c10 0x7f56ec15aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:52.591 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56e9d9b640 1 -- 172.21.15.88:0/1012022364 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2085077162 0 0) 0x7f56ec15bd20 con 0x7f56ec15b530 2023-10-16T20:05:52.591 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56e9d9b640 1 -- 172.21.15.88:0/1012022364 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56d8003530 con 0x7f56ec15b530 2023-10-16T20:05:52.591 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56e9d9b640 1 -- 172.21.15.88:0/1012022364 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1781242797 0 0) 0x7f56d8003530 con 0x7f56ec15b530 2023-10-16T20:05:52.591 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56e9d9b640 1 -- 172.21.15.88:0/1012022364 >> v2:172.21.15.94:3300/0 conn(0x7f56ec0b0c10 msgr2=0x7f56ec15aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:52.592 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56e9d9b640 1 --2- 172.21.15.88:0/1012022364 >> v2:172.21.15.94:3300/0 conn(0x7f56ec0b0c10 0x7f56ec15aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:52.592 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56e9d9b640 1 -- 172.21.15.88:0/1012022364 >> v1:172.21.15.94:6789/0 conn(0x7f56ec154f30 legacy=0x7f56ec0b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:52.592 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56e9d9b640 1 -- 172.21.15.88:0/1012022364 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ec0705d0 con 0x7f56ec15b530 2023-10-16T20:05:52.592 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56e9d9b640 1 -- 172.21.15.88:0/1012022364 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f56e4001680 con 0x7f56ec15b530 2023-10-16T20:05:52.592 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56e9d9b640 1 -- 172.21.15.88:0/1012022364 >> v1:172.21.15.88:6789/0 conn(0x7f56ec15b530 legacy=0x7f56ec06c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:52.592 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56e9d9b640 1 --2- 172.21.15.88:0/1012022364 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f56d8003980 0x7f56d8003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:52.593 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56e9d9b640 1 --2- 172.21.15.88:0/1012022364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56d8004380 0x7f56d8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:52.593 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56e9d9b640 1 --2- 172.21.15.88:0/1012022364 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56d8005050 0x7f56d800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:52.593 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56e9d9b640 1 -- 172.21.15.88:0/1012022364 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec155310 con 0x7f56d8005050 2023-10-16T20:05:52.593 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56e9d9b640 1 -- 172.21.15.88:0/1012022364 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d800bfb0 con 0x7f56d8004380 2023-10-16T20:05:52.593 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56e9d9b640 1 -- 172.21.15.88:0/1012022364 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d800c170 con 0x7f56d8003980 2023-10-16T20:05:52.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56ead9d640 1 --2- 172.21.15.88:0/1012022364 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56d8005050 0x7f56d800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:52.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56e1d9b640 1 --2- 172.21.15.88:0/1012022364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56d8004380 0x7f56d8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:52.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56ea59c640 1 --2- 172.21.15.88:0/1012022364 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f56d8003980 0x7f56d8003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:52.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56e1d9b640 1 -- 172.21.15.88:0/1012022364 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f56d8003980 msgr2=0x7f56d8003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:52.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56e1d9b640 1 --2- 172.21.15.88:0/1012022364 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f56d8003980 0x7f56d8003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:52.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56e1d9b640 1 -- 172.21.15.88:0/1012022364 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56d8005050 msgr2=0x7f56d800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:52.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56e1d9b640 1 --2- 172.21.15.88:0/1012022364 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56d8005050 0x7f56d800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:52.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56e1d9b640 1 -- 172.21.15.88:0/1012022364 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d8003530 con 0x7f56d8004380 2023-10-16T20:05:52.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56ead9d640 1 --2- 172.21.15.88:0/1012022364 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56d8005050 0x7f56d800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:05:52.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.584+0000 7f56ead9d640 1 -- 172.21.15.88:0/1012022364 reap_dead start 2023-10-16T20:05:52.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.588+0000 7f56e1d9b640 1 --2- 172.21.15.88:0/1012022364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56d8004380 0x7f56d8004850 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f56d4009ab0 tx=0x7f56d401ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b95dc07e8e9d2b65 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:52.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.588+0000 7f56e9d9b640 1 -- 172.21.15.88:0/1012022364 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d40a5510 con 0x7f56d8004380 2023-10-16T20:05:52.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.588+0000 7f56e9d9b640 1 -- 172.21.15.88:0/1012022364 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d40a5670 con 0x7f56d8004380 2023-10-16T20:05:52.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.588+0000 7f56e9d9b640 1 -- 172.21.15.88:0/1012022364 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d40a5a20 con 0x7f56d8004380 2023-10-16T20:05:52.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.588+0000 7f56f0cc9640 1 -- 172.21.15.88:0/1012022364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56d8004380 msgr2=0x7f56d8004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:52.601 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.588+0000 7f56f0cc9640 1 --2- 172.21.15.88:0/1012022364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56d8004380 0x7f56d8004850 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f56d4009ab0 tx=0x7f56d401ca30 comp rx=0 tx=0).stop 2023-10-16T20:05:52.601 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.588+0000 7f56f0cc9640 1 -- 172.21.15.88:0/1012022364 shutdown_connections 2023-10-16T20:05:52.601 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.588+0000 7f56f0cc9640 1 --2- 172.21.15.88:0/1012022364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56d8004380 0x7f56d8004850 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:52.602 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.588+0000 7f56f0cc9640 1 -- 172.21.15.88:0/1012022364 >> 172.21.15.88:0/1012022364 conn(0x7f56ec0cb080 msgr2=0x7f56ec0cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:52.602 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.588+0000 7f56f0cc9640 1 -- 172.21.15.88:0/1012022364 shutdown_connections 2023-10-16T20:05:52.602 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.588+0000 7f56f0cc9640 1 -- 172.21.15.88:0/1012022364 wait complete. 2023-10-16T20:05:52.602 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.592+0000 7f56f0cc9640 1 Processor -- start 2023-10-16T20:05:52.602 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.592+0000 7f56f0cc9640 1 -- start start 2023-10-16T20:05:52.603 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.592+0000 7f56f0cc9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56d8004380 0x7f56ec14bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:52.603 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.592+0000 7f56f0cc9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56ec0ac0e0 0x7f56ec0af470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:52.603 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.592+0000 7f56f0cc9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f56ec0af9b0 0x7f56ec1f5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:52.603 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.592+0000 7f56f0cc9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec0afe30 con 0x7f56ec0ac0e0 2023-10-16T20:05:52.603 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.592+0000 7f56f0cc9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec0a94a0 con 0x7f56d8004380 2023-10-16T20:05:52.603 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.592+0000 7f56f0cc9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec0ac560 con 0x7f56ec0af9b0 2023-10-16T20:05:52.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.592+0000 7f56e1d9b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56ec0ac0e0 0x7f56ec0af470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:52.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.592+0000 7f56e1d9b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56ec0ac0e0 0x7f56ec0af470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50508/0 (socket says 172.21.15.88:50508) 2023-10-16T20:05:52.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.592+0000 7f56e1d9b640 1 -- 172.21.15.88:0/181933455 learned_addr learned my addr 172.21.15.88:0/181933455 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:52.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.592+0000 7f56ea59c640 1 --2- 172.21.15.88:0/181933455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56d8004380 0x7f56ec14bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:52.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.592+0000 7f56ead9d640 1 --2- 172.21.15.88:0/181933455 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f56ec0af9b0 0x7f56ec1f5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:52.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.592+0000 7f56e1d9b640 1 -- 172.21.15.88:0/181933455 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f56ec0af9b0 msgr2=0x7f56ec1f5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:52.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.592+0000 7f56e1d9b640 1 --2- 172.21.15.88:0/181933455 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f56ec0af9b0 0x7f56ec1f5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:52.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.592+0000 7f56e1d9b640 1 -- 172.21.15.88:0/181933455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56d8004380 msgr2=0x7f56ec14bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:52.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.592+0000 7f56e1d9b640 1 --2- 172.21.15.88:0/181933455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56d8004380 0x7f56ec14bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:52.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.592+0000 7f56e1d9b640 1 -- 172.21.15.88:0/181933455 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ec0a8890 con 0x7f56ec0ac0e0 2023-10-16T20:05:52.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.592+0000 7f56ea59c640 1 --2- 172.21.15.88:0/181933455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56d8004380 0x7f56ec14bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:52.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.592+0000 7f56e1d9b640 1 --2- 172.21.15.88:0/181933455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56ec0ac0e0 0x7f56ec0af470 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f56d401c860 tx=0x7f56d40197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:52.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.592+0000 7f56e37fe640 1 -- 172.21.15.88:0/181933455 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d4004220 con 0x7f56ec0ac0e0 2023-10-16T20:05:52.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.592+0000 7f56e37fe640 1 -- 172.21.15.88:0/181933455 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d40043c0 con 0x7f56ec0ac0e0 2023-10-16T20:05:52.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.592+0000 7f56f0cc9640 1 -- 172.21.15.88:0/181933455 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56ec0a8b70 con 0x7f56ec0ac0e0 2023-10-16T20:05:52.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.592+0000 7f56e37fe640 1 -- 172.21.15.88:0/181933455 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d40b68b0 con 0x7f56ec0ac0e0 2023-10-16T20:05:52.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.592+0000 7f56f0cc9640 1 -- 172.21.15.88:0/181933455 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56ec0a9130 con 0x7f56ec0ac0e0 2023-10-16T20:05:52.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.596+0000 7f56f0cc9640 1 -- 172.21.15.88:0/181933455 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56ec04e9c0 con 0x7f56ec0ac0e0 2023-10-16T20:05:52.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.596+0000 7f56e37fe640 1 -- 172.21.15.88:0/181933455 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f56d40b4070 con 0x7f56ec0ac0e0 2023-10-16T20:05:52.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.596+0000 7f56e37fe640 1 --2- 172.21.15.88:0/181933455 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f56cc0664a0 0x7f56cc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:52.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.596+0000 7f56e37fe640 1 -- 172.21.15.88:0/181933455 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f56d411e090 con 0x7f56ec0ac0e0 2023-10-16T20:05:52.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.596+0000 7f56ea59c640 1 --2- 172.21.15.88:0/181933455 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f56cc0664a0 0x7f56cc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:52.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.596+0000 7f56ea59c640 1 --2- 172.21.15.88:0/181933455 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f56cc0664a0 0x7f56cc068960 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f56dc000f90 tx=0x7f56dc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:52.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.600+0000 7f56e37fe640 1 -- 172.21.15.88:0/181933455 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f56d40eb140 con 0x7f56ec0ac0e0 2023-10-16T20:05:52.754 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.748+0000 7f56f0cc9640 1 -- 172.21.15.88:0/181933455 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f56ec06b2a0 con 0x7f56ec0ac0e0 2023-10-16T20:05:52.755 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.752+0000 7f56e37fe640 1 -- 172.21.15.88:0/181933455 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f56d40ee760 con 0x7f56ec0ac0e0 2023-10-16T20:05:52.755 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:05:52.757 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.752+0000 7f56f0cc9640 1 -- 172.21.15.88:0/181933455 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f56cc0664a0 msgr2=0x7f56cc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:52.757 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.752+0000 7f56f0cc9640 1 --2- 172.21.15.88:0/181933455 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f56cc0664a0 0x7f56cc068960 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f56dc000f90 tx=0x7f56dc009040 comp rx=0 tx=0).stop 2023-10-16T20:05:52.757 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.752+0000 7f56f0cc9640 1 -- 172.21.15.88:0/181933455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56ec0ac0e0 msgr2=0x7f56ec0af470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:52.757 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.752+0000 7f56f0cc9640 1 --2- 172.21.15.88:0/181933455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56ec0ac0e0 0x7f56ec0af470 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f56d401c860 tx=0x7f56d40197d0 comp rx=0 tx=0).stop 2023-10-16T20:05:52.757 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.752+0000 7f56f0cc9640 1 -- 172.21.15.88:0/181933455 shutdown_connections 2023-10-16T20:05:52.758 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.752+0000 7f56f0cc9640 1 --2- 172.21.15.88:0/181933455 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f56cc0664a0 0x7f56cc068960 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:52.758 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.752+0000 7f56f0cc9640 1 --2- 172.21.15.88:0/181933455 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f56ec0af9b0 0x7f56ec1f5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:52.758 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.752+0000 7f56f0cc9640 1 --2- 172.21.15.88:0/181933455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56ec0ac0e0 0x7f56ec0af470 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:52.758 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.752+0000 7f56f0cc9640 1 --2- 172.21.15.88:0/181933455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56d8004380 0x7f56ec14bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:52.758 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.752+0000 7f56f0cc9640 1 -- 172.21.15.88:0/181933455 >> 172.21.15.88:0/181933455 conn(0x7f56ec0cb080 msgr2=0x7f56ec15d550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:52.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.752+0000 7f56f0cc9640 1 -- 172.21.15.88:0/181933455 shutdown_connections 2023-10-16T20:05:52.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.752+0000 7f56f0cc9640 1 -- 172.21.15.88:0/181933455 wait complete. 2023-10-16T20:05:52.776 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T20:05:52.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.908+0000 7f0677891640 1 Processor -- start 2023-10-16T20:05:52.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.908+0000 7f0677891640 1 -- start start 2023-10-16T20:05:52.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.908+0000 7f0677891640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f0670159a60 0x7f06700cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:52.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.912+0000 7f0677891640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f06700b1880 0x7f06700cae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:52.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.912+0000 7f0677891640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06700b2540 con 0x7f0670071d30 2023-10-16T20:05:52.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.912+0000 7f0677891640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0670072580 con 0x7f0670159a60 2023-10-16T20:05:52.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.912+0000 7f0677891640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f067006eef0 con 0x7f06700b1880 2023-10-16T20:05:52.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.912+0000 7f067562d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f0670159a60 0x7f06700cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:52.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.912+0000 7f0674e2c640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f0670071d30 0x7f06700b1170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:53450/0 (socket says 172.21.15.88:53450) 2023-10-16T20:05:52.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.912+0000 7f067562d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f0670159a60 0x7f06700cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50514/0 (socket says 172.21.15.88:50514) 2023-10-16T20:05:52.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.912+0000 7f0674e2c640 1 -- 172.21.15.88:0/864246439 learned_addr learned my addr 172.21.15.88:0/864246439 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:52.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.912+0000 7f0675e2e640 1 --2- 172.21.15.88:0/864246439 >> v2:172.21.15.94:3300/0 conn(0x7f06700b1880 0x7f06700cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:52.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.912+0000 7f067562d640 1 -- 172.21.15.88:0/864246439 >> v2:172.21.15.94:3300/0 conn(0x7f06700b1880 msgr2=0x7f06700cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:52.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.912+0000 7f067562d640 1 --2- 172.21.15.88:0/864246439 >> v2:172.21.15.94:3300/0 conn(0x7f06700b1880 0x7f06700cae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:52.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.912+0000 7f067562d640 1 -- 172.21.15.88:0/864246439 >> v1:172.21.15.88:6789/0 conn(0x7f0670071d30 legacy=0x7f06700b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:52.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.912+0000 7f067562d640 1 -- 172.21.15.88:0/864246439 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067006f1d0 con 0x7f0670159a60 2023-10-16T20:05:52.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.912+0000 7f0667fff640 1 -- 172.21.15.88:0/864246439 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2405639866 0 0) 0x7f06700b2540 con 0x7f0670071d30 2023-10-16T20:05:52.919 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.912+0000 7f067562d640 1 --2- 172.21.15.88:0/864246439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0670159a60 0x7f06700cef10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f06700712d0 tx=0x7f066c01ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d1e64fa9876117d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:52.919 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.912+0000 7f0667fff640 1 -- 172.21.15.88:0/864246439 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c0a8950 con 0x7f0670159a60 2023-10-16T20:05:52.919 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.912+0000 7f0667fff640 1 -- 172.21.15.88:0/864246439 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066c0a8ab0 con 0x7f0670159a60 2023-10-16T20:05:52.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.912+0000 7f0667fff640 1 -- 172.21.15.88:0/864246439 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c0a8e40 con 0x7f0670159a60 2023-10-16T20:05:52.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.916+0000 7f0677891640 1 -- 172.21.15.88:0/864246439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0670159a60 msgr2=0x7f06700cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:52.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.916+0000 7f0677891640 1 --2- 172.21.15.88:0/864246439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0670159a60 0x7f06700cef10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f06700712d0 tx=0x7f066c01ec50 comp rx=0 tx=0).stop 2023-10-16T20:05:52.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.916+0000 7f0677891640 1 -- 172.21.15.88:0/864246439 shutdown_connections 2023-10-16T20:05:52.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.916+0000 7f0677891640 1 --2- 172.21.15.88:0/864246439 >> v2:172.21.15.94:3300/0 conn(0x7f06700b1880 0x7f06700cae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:52.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.916+0000 7f0677891640 1 --2- 172.21.15.88:0/864246439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0670159a60 0x7f06700cef10 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:52.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.916+0000 7f0677891640 1 -- 172.21.15.88:0/864246439 >> 172.21.15.88:0/864246439 conn(0x7f06700cc170 msgr2=0x7f06700cc580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:52.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.916+0000 7f0677891640 1 -- 172.21.15.88:0/864246439 shutdown_connections 2023-10-16T20:05:52.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.916+0000 7f0677891640 1 -- 172.21.15.88:0/864246439 wait complete. 2023-10-16T20:05:52.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.916+0000 7f0677891640 1 Processor -- start 2023-10-16T20:05:52.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.916+0000 7f0677891640 1 -- start start 2023-10-16T20:05:52.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.916+0000 7f0677891640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0670071d30 0x7f06701efcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:52.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.916+0000 7f0677891640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06700b1880 0x7f067014ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:52.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.916+0000 7f0677891640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0670159a60 0x7f06701510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:52.925 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.916+0000 7f0677891640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06701515f0 con 0x7f0670159a60 2023-10-16T20:05:52.925 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.916+0000 7f0677891640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0670151890 con 0x7f06700b1880 2023-10-16T20:05:52.925 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.916+0000 7f0677891640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0670151b70 con 0x7f0670071d30 2023-10-16T20:05:52.925 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.916+0000 7f0675e2e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0670159a60 0x7f06701510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:52.925 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.916+0000 7f0675e2e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0670159a60 0x7f06701510b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50520/0 (socket says 172.21.15.88:50520) 2023-10-16T20:05:52.926 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.916+0000 7f0675e2e640 1 -- 172.21.15.88:0/557575250 learned_addr learned my addr 172.21.15.88:0/557575250 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:52.926 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.916+0000 7f067562d640 1 --2- 172.21.15.88:0/557575250 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0670071d30 0x7f06701efcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:52.926 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.920+0000 7f0675e2e640 1 -- 172.21.15.88:0/557575250 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0670071d30 msgr2=0x7f06701efcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:52.926 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.920+0000 7f0675e2e640 1 --2- 172.21.15.88:0/557575250 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0670071d30 0x7f06701efcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:52.926 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.920+0000 7f0675e2e640 1 -- 172.21.15.88:0/557575250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06700b1880 msgr2=0x7f067014ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:52.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.920+0000 7f0674e2c640 1 --2- 172.21.15.88:0/557575250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06700b1880 0x7f067014ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:52.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.920+0000 7f0675e2e640 1 --2- 172.21.15.88:0/557575250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06700b1880 0x7f067014ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:52.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.920+0000 7f0675e2e640 1 -- 172.21.15.88:0/557575250 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066c009420 con 0x7f0670159a60 2023-10-16T20:05:52.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.920+0000 7f067562d640 1 --2- 172.21.15.88:0/557575250 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0670071d30 0x7f06701efcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:52.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.920+0000 7f0674e2c640 1 --2- 172.21.15.88:0/557575250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06700b1880 0x7f067014ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:52.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.920+0000 7f0675e2e640 1 --2- 172.21.15.88:0/557575250 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0670159a60 0x7f06701510b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0668007e30 tx=0x7f066800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:52.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.920+0000 7f0665ffb640 1 -- 172.21.15.88:0/557575250 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0668014620 con 0x7f0670159a60 2023-10-16T20:05:52.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.920+0000 7f0665ffb640 1 -- 172.21.15.88:0/557575250 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066800ee90 con 0x7f0670159a60 2023-10-16T20:05:52.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.920+0000 7f0665ffb640 1 -- 172.21.15.88:0/557575250 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06680149c0 con 0x7f0670159a60 2023-10-16T20:05:52.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.920+0000 7f0677891640 1 -- 172.21.15.88:0/557575250 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0670151e50 con 0x7f0670159a60 2023-10-16T20:05:52.929 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.920+0000 7f0677891640 1 -- 172.21.15.88:0/557575250 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f067015b1f0 con 0x7f0670159a60 2023-10-16T20:05:52.930 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.920+0000 7f0677891640 1 -- 172.21.15.88:0/557575250 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f067004e9c0 con 0x7f0670159a60 2023-10-16T20:05:52.930 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.920+0000 7f0665ffb640 1 -- 172.21.15.88:0/557575250 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0668014b20 con 0x7f0670159a60 2023-10-16T20:05:52.930 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.920+0000 7f0665ffb640 1 --2- 172.21.15.88:0/557575250 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f06540664a0 0x7f0654068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:52.930 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.920+0000 7f0665ffb640 1 -- 172.21.15.88:0/557575250 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0668084bc0 con 0x7f0670159a60 2023-10-16T20:05:52.931 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.920+0000 7f067562d640 1 --2- 172.21.15.88:0/557575250 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f06540664a0 0x7f0654068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:52.931 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.924+0000 7f067562d640 1 --2- 172.21.15.88:0/557575250 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f06540664a0 0x7f0654068960 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f066c0b2000 tx=0x7f066c0aa070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:52.931 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:52.924+0000 7f0665ffb640 1 -- 172.21.15.88:0/557575250 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0668051c70 con 0x7f0670159a60 2023-10-16T20:05:53.088 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.084+0000 7f0677891640 1 -- 172.21.15.88:0/557575250 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f067006f760 con 0x7f0670159a60 2023-10-16T20:05:53.089 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.084+0000 7f0665ffb640 1 -- 172.21.15.88:0/557575250 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0668055290 con 0x7f0670159a60 2023-10-16T20:05:53.089 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2023-10-16T20:05:53.091 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.088+0000 7f0677891640 1 -- 172.21.15.88:0/557575250 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f06540664a0 msgr2=0x7f0654068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:53.091 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.088+0000 7f0677891640 1 --2- 172.21.15.88:0/557575250 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f06540664a0 0x7f0654068960 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f066c0b2000 tx=0x7f066c0aa070 comp rx=0 tx=0).stop 2023-10-16T20:05:53.092 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.088+0000 7f0677891640 1 -- 172.21.15.88:0/557575250 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0670159a60 msgr2=0x7f06701510b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:53.092 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.088+0000 7f0677891640 1 --2- 172.21.15.88:0/557575250 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0670159a60 0x7f06701510b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0668007e30 tx=0x7f066800e4b0 comp rx=0 tx=0).stop 2023-10-16T20:05:53.092 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.088+0000 7f0677891640 1 -- 172.21.15.88:0/557575250 shutdown_connections 2023-10-16T20:05:53.092 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.088+0000 7f0677891640 1 --2- 172.21.15.88:0/557575250 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f06540664a0 0x7f0654068960 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:53.092 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.088+0000 7f0677891640 1 --2- 172.21.15.88:0/557575250 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0670159a60 0x7f06701510b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:53.093 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.088+0000 7f0677891640 1 --2- 172.21.15.88:0/557575250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06700b1880 0x7f067014ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:53.093 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.088+0000 7f0677891640 1 --2- 172.21.15.88:0/557575250 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0670071d30 0x7f06701efcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:53.093 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.088+0000 7f0677891640 1 -- 172.21.15.88:0/557575250 >> 172.21.15.88:0/557575250 conn(0x7f06700cc170 msgr2=0x7f06700cfb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:53.093 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.088+0000 7f0677891640 1 -- 172.21.15.88:0/557575250 shutdown_connections 2023-10-16T20:05:53.093 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.088+0000 7f0677891640 1 -- 172.21.15.88:0/557575250 wait complete. 2023-10-16T20:05:53.110 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi094.front.sepia.ceph.com... 2023-10-16T20:05:53.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:53.111 DEBUG:teuthology.orchestra.run.smithi094:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-16T20:05:53.149 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:05:53.150 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2023-10-16T20:05:53.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:53.151 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-16T20:05:53.206 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:05:53.207 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:05:53.216 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:05:53.216 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:05:53.271 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:05:53.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.452+0000 7f958c846640 1 Processor -- start 2023-10-16T20:05:53.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.452+0000 7f958c846640 1 -- start start 2023-10-16T20:05:53.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.452+0000 7f958c846640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f95880d22e0 0x7f9588161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:53.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.452+0000 7f958c846640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f958815d9f0 0x7f95880cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:53.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.452+0000 7f958c846640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95880b87e0 con 0x7f95880d26c0 2023-10-16T20:05:53.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.452+0000 7f958c846640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f958815df10 con 0x7f95880d22e0 2023-10-16T20:05:53.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.452+0000 7f958c846640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f958815e1f0 con 0x7f958815d9f0 2023-10-16T20:05:53.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.452+0000 7f958659c640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f95880d22e0 0x7f9588161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:53.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.452+0000 7f958659c640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f95880d22e0 0x7f9588161450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50534/0 (socket says 172.21.15.88:50534) 2023-10-16T20:05:53.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.452+0000 7f958659c640 1 -- 172.21.15.88:0/1104308171 learned_addr learned my addr 172.21.15.88:0/1104308171 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:53.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.452+0000 7f9586d9d640 1 --2- 172.21.15.88:0/1104308171 >> v2:172.21.15.94:3300/0 conn(0x7f958815d9f0 0x7f95880cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:53.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.452+0000 7f958659c640 1 -- 172.21.15.88:0/1104308171 >> v2:172.21.15.94:3300/0 conn(0x7f958815d9f0 msgr2=0x7f95880cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:53.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.452+0000 7f958659c640 1 --2- 172.21.15.88:0/1104308171 >> v2:172.21.15.94:3300/0 conn(0x7f958815d9f0 0x7f95880cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:53.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.452+0000 7f958659c640 1 -- 172.21.15.88:0/1104308171 >> v1:172.21.15.94:6789/0 conn(0x7f95880d26c0 legacy=0x7f958815d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:53.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.452+0000 7f958659c640 1 -- 172.21.15.88:0/1104308171 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9588160970 con 0x7f95880d22e0 2023-10-16T20:05:53.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.456+0000 7f958659c640 1 --2- 172.21.15.88:0/1104308171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95880d22e0 0x7f9588161450 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f958808bb40 tx=0x7f957801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8543c2de327b990 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:53.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.456+0000 7f9585d9b640 1 -- 172.21.15.88:0/1104308171 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95780ac640 con 0x7f95880d22e0 2023-10-16T20:05:53.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.456+0000 7f9585d9b640 1 -- 172.21.15.88:0/1104308171 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95780ac7a0 con 0x7f95880d22e0 2023-10-16T20:05:53.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.456+0000 7f9585d9b640 1 -- 172.21.15.88:0/1104308171 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95780acb30 con 0x7f95880d22e0 2023-10-16T20:05:53.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.456+0000 7f958c846640 1 -- 172.21.15.88:0/1104308171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95880d22e0 msgr2=0x7f9588161450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:53.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.456+0000 7f958c846640 1 --2- 172.21.15.88:0/1104308171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95880d22e0 0x7f9588161450 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f958808bb40 tx=0x7f957801c3e0 comp rx=0 tx=0).stop 2023-10-16T20:05:53.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.456+0000 7f958c846640 1 -- 172.21.15.88:0/1104308171 shutdown_connections 2023-10-16T20:05:53.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.456+0000 7f958c846640 1 --2- 172.21.15.88:0/1104308171 >> v2:172.21.15.94:3300/0 conn(0x7f958815d9f0 0x7f95880cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:53.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.456+0000 7f958c846640 1 --2- 172.21.15.88:0/1104308171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95880d22e0 0x7f9588161450 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:53.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.456+0000 7f958c846640 1 -- 172.21.15.88:0/1104308171 >> 172.21.15.88:0/1104308171 conn(0x7f95880715c0 msgr2=0x7f95880719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:53.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.456+0000 7f958c846640 1 -- 172.21.15.88:0/1104308171 shutdown_connections 2023-10-16T20:05:53.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.456+0000 7f958c846640 1 -- 172.21.15.88:0/1104308171 wait complete. 2023-10-16T20:05:53.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.460+0000 7f958c846640 1 Processor -- start 2023-10-16T20:05:53.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.460+0000 7f958c846640 1 -- start start 2023-10-16T20:05:53.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.460+0000 7f958c846640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f95880d22e0 0x7f958814f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:53.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.460+0000 7f958c846640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95880d26c0 0x7f9588149e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:53.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.460+0000 7f958c846640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958815d9f0 0x7f958815f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:53.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.460+0000 7f958c846640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958814e5a0 con 0x7f95880d26c0 2023-10-16T20:05:53.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.460+0000 7f958c846640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958814e880 con 0x7f958815d9f0 2023-10-16T20:05:53.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.460+0000 7f958c846640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958814eb60 con 0x7f95880d22e0 2023-10-16T20:05:53.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.460+0000 7f957dd9b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95880d26c0 0x7f9588149e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:53.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.460+0000 7f9586d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958815d9f0 0x7f958815f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:53.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.460+0000 7f957dd9b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95880d26c0 0x7f9588149e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50550/0 (socket says 172.21.15.88:50550) 2023-10-16T20:05:53.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.460+0000 7f957dd9b640 1 -- 172.21.15.88:0/2328402450 learned_addr learned my addr 172.21.15.88:0/2328402450 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:53.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.460+0000 7f958659c640 1 --2- 172.21.15.88:0/2328402450 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f95880d22e0 0x7f958814f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:53.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.460+0000 7f957dd9b640 1 -- 172.21.15.88:0/2328402450 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f95880d22e0 msgr2=0x7f958814f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:53.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.460+0000 7f957dd9b640 1 --2- 172.21.15.88:0/2328402450 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f95880d22e0 0x7f958814f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:53.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.460+0000 7f957dd9b640 1 -- 172.21.15.88:0/2328402450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958815d9f0 msgr2=0x7f958815f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:53.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.460+0000 7f957dd9b640 1 --2- 172.21.15.88:0/2328402450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958815d9f0 0x7f958815f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:53.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.460+0000 7f957dd9b640 1 -- 172.21.15.88:0/2328402450 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95780094d0 con 0x7f95880d26c0 2023-10-16T20:05:53.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.460+0000 7f957dd9b640 1 --2- 172.21.15.88:0/2328402450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95880d26c0 0x7f9588149e00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f957000c450 tx=0x7f957000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:53.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.460+0000 7f957f7fe640 1 -- 172.21.15.88:0/2328402450 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9570012830 con 0x7f95880d26c0 2023-10-16T20:05:53.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.460+0000 7f957f7fe640 1 -- 172.21.15.88:0/2328402450 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95700129b0 con 0x7f95880d26c0 2023-10-16T20:05:53.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.460+0000 7f957f7fe640 1 -- 172.21.15.88:0/2328402450 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9570012d60 con 0x7f95880d26c0 2023-10-16T20:05:53.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.460+0000 7f958c846640 1 -- 172.21.15.88:0/2328402450 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f958814ee40 con 0x7f95880d26c0 2023-10-16T20:05:53.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.460+0000 7f958c846640 1 -- 172.21.15.88:0/2328402450 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95880c75e0 con 0x7f95880d26c0 2023-10-16T20:05:53.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.464+0000 7f958c846640 1 -- 172.21.15.88:0/2328402450 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f958804e9c0 con 0x7f95880d26c0 2023-10-16T20:05:53.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.464+0000 7f957f7fe640 1 -- 172.21.15.88:0/2328402450 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f957001c460 con 0x7f95880d26c0 2023-10-16T20:05:53.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.464+0000 7f957f7fe640 1 --2- 172.21.15.88:0/2328402450 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f95680664a0 0x7f9568068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:53.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.464+0000 7f957f7fe640 1 -- 172.21.15.88:0/2328402450 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9570083fa0 con 0x7f95880d26c0 2023-10-16T20:05:53.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.464+0000 7f958659c640 1 --2- 172.21.15.88:0/2328402450 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f95680664a0 0x7f9568068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:53.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.464+0000 7f958659c640 1 --2- 172.21.15.88:0/2328402450 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f95680664a0 0x7f9568068960 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f95780ae000 tx=0x7f9578008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:53.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.468+0000 7f957f7fe640 1 -- 172.21.15.88:0/2328402450 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f95700b5090 con 0x7f95880d26c0 2023-10-16T20:05:53.622 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.616+0000 7f958c846640 1 -- 172.21.15.88:0/2328402450 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f958815b760 con 0x7f95880d26c0 2023-10-16T20:05:53.623 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.620+0000 7f957f7fe640 1 -- 172.21.15.88:0/2328402450 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18661 (secure 0 0 0) 0x7f9570051110 con 0x7f95880d26c0 2023-10-16T20:05:53.623 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:53.624 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":29,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:05:38.202959+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T20:04:54.442487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T20:04:55.766641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T20:04:59.475830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":1}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:05:53.625 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.620+0000 7f958c846640 1 -- 172.21.15.88:0/2328402450 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f95680664a0 msgr2=0x7f9568068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:53.625 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.620+0000 7f958c846640 1 --2- 172.21.15.88:0/2328402450 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f95680664a0 0x7f9568068960 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f95780ae000 tx=0x7f9578008040 comp rx=0 tx=0).stop 2023-10-16T20:05:53.625 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.620+0000 7f958c846640 1 -- 172.21.15.88:0/2328402450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95880d26c0 msgr2=0x7f9588149e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:53.626 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.620+0000 7f958c846640 1 --2- 172.21.15.88:0/2328402450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95880d26c0 0x7f9588149e00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f957000c450 tx=0x7f957000c910 comp rx=0 tx=0).stop 2023-10-16T20:05:53.626 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.620+0000 7f958c846640 1 -- 172.21.15.88:0/2328402450 shutdown_connections 2023-10-16T20:05:53.626 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.620+0000 7f958c846640 1 --2- 172.21.15.88:0/2328402450 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f95680664a0 0x7f9568068960 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:53.626 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.620+0000 7f958c846640 1 --2- 172.21.15.88:0/2328402450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958815d9f0 0x7f958815f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:53.627 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.620+0000 7f958c846640 1 --2- 172.21.15.88:0/2328402450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95880d26c0 0x7f9588149e00 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:53.627 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.620+0000 7f958c846640 1 --2- 172.21.15.88:0/2328402450 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f95880d22e0 0x7f958814f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:53.627 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.620+0000 7f958c846640 1 -- 172.21.15.88:0/2328402450 >> 172.21.15.88:0/2328402450 conn(0x7f95880715c0 msgr2=0x7f958815af00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:53.627 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.620+0000 7f958c846640 1 -- 172.21.15.88:0/2328402450 shutdown_connections 2023-10-16T20:05:53.627 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.620+0000 7f958c846640 1 -- 172.21.15.88:0/2328402450 wait complete. 2023-10-16T20:05:53.645 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T20:04:54.442487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T20:04:55.766641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T20:04:59.475830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T20:05:53.646 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:05:53.786 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.780+0000 7ff59310f640 1 Processor -- start 2023-10-16T20:05:53.786 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.780+0000 7ff59310f640 1 -- start start 2023-10-16T20:05:53.790 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.784+0000 7ff59310f640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff58c1457f0 0x7ff58c160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:53.790 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.784+0000 7ff59310f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff58c070960 0x7ff58c06caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:53.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.784+0000 7ff59310f640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff58c06c2e0 con 0x7ff58c0aeb90 2023-10-16T20:05:53.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.784+0000 7ff59310f640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff58c145bd0 con 0x7ff58c1457f0 2023-10-16T20:05:53.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.784+0000 7ff59310f640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c160ec0 con 0x7ff58c070960 2023-10-16T20:05:53.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.784+0000 7ff5916ac640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff58c070960 0x7ff58c06caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:53.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.784+0000 7ff590eab640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff58c1457f0 0x7ff58c160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:53.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.784+0000 7ff5916ac640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff58c070960 0x7ff58c06caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:50370/0 (socket says 172.21.15.88:50370) 2023-10-16T20:05:53.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.784+0000 7ff583fff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7ff58c0aeb90 0x7ff58c070250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:49094/0 (socket says 172.21.15.88:49094) 2023-10-16T20:05:53.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.784+0000 7ff5916ac640 1 -- 172.21.15.88:0/2207103911 learned_addr learned my addr 172.21.15.88:0/2207103911 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:53.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.784+0000 7ff5916ac640 1 -- 172.21.15.88:0/2207103911 >> v2:172.21.15.94:3300/0 conn(0x7ff58c1457f0 msgr2=0x7ff58c160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:53.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.784+0000 7ff5916ac640 1 --2- 172.21.15.88:0/2207103911 >> v2:172.21.15.94:3300/0 conn(0x7ff58c1457f0 0x7ff58c160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:53.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.784+0000 7ff5916ac640 1 -- 172.21.15.88:0/2207103911 >> v1:172.21.15.94:6789/0 conn(0x7ff58c0aeb90 legacy=0x7ff58c070250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:53.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.784+0000 7ff5916ac640 1 -- 172.21.15.88:0/2207103911 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff58c1611a0 con 0x7ff58c070960 2023-10-16T20:05:53.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.788+0000 7ff5916ac640 1 --2- 172.21.15.88:0/2207103911 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff58c070960 0x7ff58c06caa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff584009e50 tx=0x7ff58401c9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a157f08ca3a1913 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:53.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.788+0000 7ff5837fe640 1 -- 172.21.15.88:0/2207103911 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5840a58a0 con 0x7ff58c070960 2023-10-16T20:05:53.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.788+0000 7ff5837fe640 1 -- 172.21.15.88:0/2207103911 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5840a5a00 con 0x7ff58c070960 2023-10-16T20:05:53.795 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.788+0000 7ff5837fe640 1 -- 172.21.15.88:0/2207103911 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5840a5d90 con 0x7ff58c070960 2023-10-16T20:05:53.795 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.788+0000 7ff59310f640 1 -- 172.21.15.88:0/2207103911 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff58c070960 msgr2=0x7ff58c06caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:53.795 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.788+0000 7ff59310f640 1 --2- 172.21.15.88:0/2207103911 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff58c070960 0x7ff58c06caa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff584009e50 tx=0x7ff58401c9e0 comp rx=0 tx=0).stop 2023-10-16T20:05:53.795 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.788+0000 7ff59310f640 1 -- 172.21.15.88:0/2207103911 shutdown_connections 2023-10-16T20:05:53.795 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.788+0000 7ff59310f640 1 --2- 172.21.15.88:0/2207103911 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff58c070960 0x7ff58c06caa0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:53.795 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.788+0000 7ff59310f640 1 --2- 172.21.15.88:0/2207103911 >> v2:172.21.15.94:3300/0 conn(0x7ff58c1457f0 0x7ff58c160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:53.796 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.788+0000 7ff59310f640 1 -- 172.21.15.88:0/2207103911 >> 172.21.15.88:0/2207103911 conn(0x7ff58c0cb080 msgr2=0x7ff58c0cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:53.796 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.788+0000 7ff59310f640 1 -- 172.21.15.88:0/2207103911 shutdown_connections 2023-10-16T20:05:53.796 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.788+0000 7ff59310f640 1 -- 172.21.15.88:0/2207103911 wait complete. 2023-10-16T20:05:53.798 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.792+0000 7ff59310f640 1 Processor -- start 2023-10-16T20:05:53.798 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.792+0000 7ff59310f640 1 -- start start 2023-10-16T20:05:53.799 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.792+0000 7ff59310f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff58c070960 0x7ff58c0afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:53.799 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.792+0000 7ff59310f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff58c0aeb90 0x7ff58c13d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:53.799 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.792+0000 7ff59310f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff58c1457f0 0x7ff58c142df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:53.799 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.792+0000 7ff59310f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c0b2c20 con 0x7ff58c0aeb90 2023-10-16T20:05:53.800 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.792+0000 7ff59310f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c13ef70 con 0x7ff58c070960 2023-10-16T20:05:53.800 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.792+0000 7ff59310f640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c0b2880 con 0x7ff58c1457f0 2023-10-16T20:05:53.800 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.792+0000 7ff583fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff58c0aeb90 0x7ff58c13d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:53.800 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.792+0000 7ff583fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff58c0aeb90 0x7ff58c13d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50556/0 (socket says 172.21.15.88:50556) 2023-10-16T20:05:53.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.792+0000 7ff583fff640 1 -- 172.21.15.88:0/3220872622 learned_addr learned my addr 172.21.15.88:0/3220872622 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:53.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.792+0000 7ff590eab640 1 --2- 172.21.15.88:0/3220872622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff58c070960 0x7ff58c0afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:53.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.792+0000 7ff5916ac640 1 --2- 172.21.15.88:0/3220872622 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff58c1457f0 0x7ff58c142df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:53.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.792+0000 7ff583fff640 1 -- 172.21.15.88:0/3220872622 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff58c1457f0 msgr2=0x7ff58c142df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:53.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.792+0000 7ff583fff640 1 --2- 172.21.15.88:0/3220872622 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff58c1457f0 0x7ff58c142df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:53.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.792+0000 7ff583fff640 1 -- 172.21.15.88:0/3220872622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff58c070960 msgr2=0x7ff58c0afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:53.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.792+0000 7ff583fff640 1 --2- 172.21.15.88:0/3220872622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff58c070960 0x7ff58c0afcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:53.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.792+0000 7ff583fff640 1 -- 172.21.15.88:0/3220872622 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff584009b30 con 0x7ff58c0aeb90 2023-10-16T20:05:53.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.792+0000 7ff583fff640 1 --2- 172.21.15.88:0/3220872622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff58c0aeb90 0x7ff58c13d6a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00c4c0 tx=0x7ff57c00c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:53.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.792+0000 7ff5817fa640 1 -- 172.21.15.88:0/3220872622 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c007480 con 0x7ff58c0aeb90 2023-10-16T20:05:53.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.792+0000 7ff5817fa640 1 -- 172.21.15.88:0/3220872622 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff57c0075e0 con 0x7ff58c0aeb90 2023-10-16T20:05:53.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.792+0000 7ff59310f640 1 -- 172.21.15.88:0/3220872622 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff58c143f90 con 0x7ff58c0aeb90 2023-10-16T20:05:53.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.792+0000 7ff5817fa640 1 -- 172.21.15.88:0/3220872622 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c004a10 con 0x7ff58c0aeb90 2023-10-16T20:05:53.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.792+0000 7ff59310f640 1 -- 172.21.15.88:0/3220872622 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff58c143cb0 con 0x7ff58c0aeb90 2023-10-16T20:05:53.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.796+0000 7ff59310f640 1 -- 172.21.15.88:0/3220872622 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff58c04e9c0 con 0x7ff58c0aeb90 2023-10-16T20:05:53.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.796+0000 7ff5817fa640 1 -- 172.21.15.88:0/3220872622 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff57c004b70 con 0x7ff58c0aeb90 2023-10-16T20:05:53.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.796+0000 7ff5817fa640 1 --2- 172.21.15.88:0/3220872622 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff5780664a0 0x7ff578068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:53.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.796+0000 7ff5817fa640 1 -- 172.21.15.88:0/3220872622 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff57c0834d0 con 0x7ff58c0aeb90 2023-10-16T20:05:53.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.796+0000 7ff590eab640 1 --2- 172.21.15.88:0/3220872622 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff5780664a0 0x7ff578068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:53.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.796+0000 7ff590eab640 1 --2- 172.21.15.88:0/3220872622 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff5780664a0 0x7ff578068960 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff588009b60 tx=0x7ff588009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:53.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.800+0000 7ff5817fa640 1 -- 172.21.15.88:0/3220872622 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff57c0506d0 con 0x7ff58c0aeb90 2023-10-16T20:05:53.951 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.948+0000 7ff59310f640 1 -- 172.21.15.88:0/3220872622 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff58c160bb0 con 0x7ff58c0aeb90 2023-10-16T20:05:53.952 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.948+0000 7ff5817fa640 1 -- 172.21.15.88:0/3220872622 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff57c05a030 con 0x7ff58c0aeb90 2023-10-16T20:05:53.952 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:05:53.954 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.948+0000 7ff59310f640 1 -- 172.21.15.88:0/3220872622 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff5780664a0 msgr2=0x7ff578068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:53.955 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.948+0000 7ff59310f640 1 --2- 172.21.15.88:0/3220872622 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff5780664a0 0x7ff578068960 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff588009b60 tx=0x7ff588009190 comp rx=0 tx=0).stop 2023-10-16T20:05:53.955 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.948+0000 7ff59310f640 1 -- 172.21.15.88:0/3220872622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff58c0aeb90 msgr2=0x7ff58c13d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:53.955 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.948+0000 7ff59310f640 1 --2- 172.21.15.88:0/3220872622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff58c0aeb90 0x7ff58c13d6a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00c4c0 tx=0x7ff57c00c980 comp rx=0 tx=0).stop 2023-10-16T20:05:53.955 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.948+0000 7ff59310f640 1 -- 172.21.15.88:0/3220872622 shutdown_connections 2023-10-16T20:05:53.955 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.948+0000 7ff59310f640 1 --2- 172.21.15.88:0/3220872622 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff5780664a0 0x7ff578068960 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:53.955 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.948+0000 7ff59310f640 1 --2- 172.21.15.88:0/3220872622 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff58c1457f0 0x7ff58c142df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:53.956 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.948+0000 7ff59310f640 1 --2- 172.21.15.88:0/3220872622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff58c0aeb90 0x7ff58c13d6a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:53.956 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.948+0000 7ff59310f640 1 --2- 172.21.15.88:0/3220872622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff58c070960 0x7ff58c0afcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:53.956 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.948+0000 7ff59310f640 1 -- 172.21.15.88:0/3220872622 >> 172.21.15.88:0/3220872622 conn(0x7ff58c0cb080 msgr2=0x7ff58c0cd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:53.956 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.948+0000 7ff59310f640 1 -- 172.21.15.88:0/3220872622 shutdown_connections 2023-10-16T20:05:53.957 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:53.948+0000 7ff59310f640 1 -- 172.21.15.88:0/3220872622 wait complete. 2023-10-16T20:05:53.973 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T20:05:54.113 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.108+0000 7fecfc5aa640 1 Processor -- start 2023-10-16T20:05:54.113 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.108+0000 7fecfc5aa640 1 -- start start 2023-10-16T20:05:54.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.112+0000 7fecfc5aa640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fecf40cd370 0x7fecf406a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:54.115 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.112+0000 7fecfc5aa640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecf406a3a0 con 0x7fecf415cac0 2023-10-16T20:05:54.115 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.112+0000 7fecfc5aa640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecf40ca650 con 0x7fecf415cea0 2023-10-16T20:05:54.115 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.112+0000 7fecfc5aa640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecf40cac10 con 0x7fecf40cd370 2023-10-16T20:05:54.115 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.112+0000 7fecfa346640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fecf415cac0 0x7fecf40ccc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:53456/0 (socket says 172.21.15.88:53456) 2023-10-16T20:05:54.115 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.112+0000 7fecfa346640 1 -- 172.21.15.88:0/1247840281 learned_addr learned my addr 172.21.15.88:0/1247840281 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:54.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.112+0000 7fecf9b45640 1 --2- 172.21.15.88:0/1247840281 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fecf40cd370 0x7fecf406a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:54.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.112+0000 7fecf9344640 1 -- 172.21.15.88:0/1247840281 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 713229765 0 0) 0x7fecf406a3a0 con 0x7fecf415cac0 2023-10-16T20:05:54.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.112+0000 7fecf9344640 1 -- 172.21.15.88:0/1247840281 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fece0003530 con 0x7fecf415cac0 2023-10-16T20:05:54.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.112+0000 7fecf9344640 1 -- 172.21.15.88:0/1247840281 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2654241145 0 0) 0x7fecf40ca650 con 0x7fecf415cea0 2023-10-16T20:05:54.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.112+0000 7fecf9344640 1 -- 172.21.15.88:0/1247840281 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecf406a3a0 con 0x7fecf415cea0 2023-10-16T20:05:54.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.112+0000 7fecf9344640 1 -- 172.21.15.88:0/1247840281 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 234368124 0 0) 0x7fece0003530 con 0x7fecf415cac0 2023-10-16T20:05:54.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.112+0000 7fecf9344640 1 -- 172.21.15.88:0/1247840281 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fecf40cd370 msgr2=0x7fecf406a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:54.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.112+0000 7fecf9344640 1 --2- 172.21.15.88:0/1247840281 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fecf40cd370 0x7fecf406a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:54.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.112+0000 7fecf9344640 1 -- 172.21.15.88:0/1247840281 >> v1:172.21.15.94:6789/0 conn(0x7fecf415cea0 legacy=0x7fecf406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:54.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.112+0000 7fecf9344640 1 -- 172.21.15.88:0/1247840281 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf40ca930 con 0x7fecf415cac0 2023-10-16T20:05:54.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.112+0000 7fecf9344640 1 -- 172.21.15.88:0/1247840281 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7fecf0001600 con 0x7fecf415cac0 2023-10-16T20:05:54.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.112+0000 7fecf9344640 1 -- 172.21.15.88:0/1247840281 >> v1:172.21.15.88:6789/0 conn(0x7fecf415cac0 legacy=0x7fecf40ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:54.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.112+0000 7fecf9344640 1 --2- 172.21.15.88:0/1247840281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fece0003b40 0x7fece0003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:54.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.112+0000 7fecf9344640 1 --2- 172.21.15.88:0/1247840281 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fecf40cd370 0x7fece00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:54.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.112+0000 7fecf9344640 1 --2- 172.21.15.88:0/1247840281 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fece0004d10 0x7fece0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:54.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.112+0000 7fecf9344640 1 -- 172.21.15.88:0/1247840281 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf40cac10 con 0x7fece0004d10 2023-10-16T20:05:54.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.112+0000 7fecf9344640 1 -- 172.21.15.88:0/1247840281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece000ba40 con 0x7fece0003b40 2023-10-16T20:05:54.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.112+0000 7fecf9344640 1 -- 172.21.15.88:0/1247840281 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece000bc00 con 0x7fecf40cd370 2023-10-16T20:05:54.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.112+0000 7fecfab47640 1 --2- 172.21.15.88:0/1247840281 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fece0004d10 0x7fece0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:54.122 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.112+0000 7fecfa346640 1 --2- 172.21.15.88:0/1247840281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fece0003b40 0x7fece0003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:54.122 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.112+0000 7fecf9b45640 1 --2- 172.21.15.88:0/1247840281 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fecf40cd370 0x7fece00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:54.122 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.112+0000 7fecfab47640 1 -- 172.21.15.88:0/1247840281 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fecf40cd370 msgr2=0x7fece00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:54.122 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.112+0000 7fecfab47640 1 --2- 172.21.15.88:0/1247840281 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fecf40cd370 0x7fece00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:54.123 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.112+0000 7fecfab47640 1 -- 172.21.15.88:0/1247840281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fece0003b40 msgr2=0x7fece0003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:54.123 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.112+0000 7fecfab47640 1 --2- 172.21.15.88:0/1247840281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fece0003b40 0x7fece0003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:54.123 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.112+0000 7fecfab47640 1 -- 172.21.15.88:0/1247840281 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece0003530 con 0x7fece0004d10 2023-10-16T20:05:54.126 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.116+0000 7fecfab47640 1 --2- 172.21.15.88:0/1247840281 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fece0004d10 0x7fece0005190 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fecf40cede0 tx=0x7fecec01c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3120e03d0640be53 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:54.126 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.116+0000 7fecf9344640 1 -- 172.21.15.88:0/1247840281 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec003b80 con 0x7fece0004d10 2023-10-16T20:05:54.127 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.116+0000 7fecf9344640 1 -- 172.21.15.88:0/1247840281 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecec003d00 con 0x7fece0004d10 2023-10-16T20:05:54.127 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.116+0000 7fecf9344640 1 -- 172.21.15.88:0/1247840281 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec005310 con 0x7fece0004d10 2023-10-16T20:05:54.127 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.116+0000 7fecfc5aa640 1 -- 172.21.15.88:0/1247840281 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fece0004d10 msgr2=0x7fece0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:54.127 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.116+0000 7fecfc5aa640 1 --2- 172.21.15.88:0/1247840281 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fece0004d10 0x7fece0005190 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fecf40cede0 tx=0x7fecec01c8c0 comp rx=0 tx=0).stop 2023-10-16T20:05:54.128 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.116+0000 7fecfab47640 1 -- 172.21.15.88:0/1247840281 reap_dead start 2023-10-16T20:05:54.128 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.116+0000 7fecfc5aa640 1 -- 172.21.15.88:0/1247840281 shutdown_connections 2023-10-16T20:05:54.128 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.116+0000 7fecfc5aa640 1 -- 172.21.15.88:0/1247840281 >> 172.21.15.88:0/1247840281 conn(0x7fecf40b0970 msgr2=0x7fecf40b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:54.128 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.116+0000 7fecfc5aa640 1 -- 172.21.15.88:0/1247840281 shutdown_connections 2023-10-16T20:05:54.128 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.116+0000 7fecfc5aa640 1 -- 172.21.15.88:0/1247840281 wait complete. 2023-10-16T20:05:54.129 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.116+0000 7fecfc5aa640 1 Processor -- start 2023-10-16T20:05:54.129 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.116+0000 7fecfc5aa640 1 -- start start 2023-10-16T20:05:54.129 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.120+0000 7fecfc5aa640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fece0004d10 0x7fecf41f5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:54.129 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.120+0000 7fecfc5aa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecf40c43d0 0x7fecf406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:54.129 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.120+0000 7fecfc5aa640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fecf40c4830 0x7fecf406dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:54.130 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.120+0000 7fecfc5aa640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf40c76c0 con 0x7fece0004d10 2023-10-16T20:05:54.130 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.120+0000 7fecfc5aa640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf40c79a0 con 0x7fecf40c43d0 2023-10-16T20:05:54.130 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.120+0000 7fecfc5aa640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecf40c7c80 con 0x7fecf40c4830 2023-10-16T20:05:54.130 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.120+0000 7fecfa346640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fece0004d10 0x7fecf41f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:54.130 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.120+0000 7fecfa346640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fece0004d10 0x7fecf41f5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50570/0 (socket says 172.21.15.88:50570) 2023-10-16T20:05:54.131 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.120+0000 7fecfa346640 1 -- 172.21.15.88:0/2708593757 learned_addr learned my addr 172.21.15.88:0/2708593757 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:54.131 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.120+0000 7fecf9b45640 1 --2- 172.21.15.88:0/2708593757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecf40c43d0 0x7fecf406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:54.131 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.120+0000 7fecfab47640 1 --2- 172.21.15.88:0/2708593757 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fecf40c4830 0x7fecf406dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:54.131 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.120+0000 7fecfa346640 1 -- 172.21.15.88:0/2708593757 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fecf40c4830 msgr2=0x7fecf406dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:54.131 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.120+0000 7fecfa346640 1 --2- 172.21.15.88:0/2708593757 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fecf40c4830 0x7fecf406dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:54.132 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.120+0000 7fecfa346640 1 -- 172.21.15.88:0/2708593757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecf40c43d0 msgr2=0x7fecf406d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:54.132 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.120+0000 7fecfa346640 1 --2- 172.21.15.88:0/2708593757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecf40c43d0 0x7fecf406d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:54.132 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.120+0000 7fecfa346640 1 -- 172.21.15.88:0/2708593757 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf40c7f60 con 0x7fece0004d10 2023-10-16T20:05:54.132 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.120+0000 7fecf9b45640 1 --2- 172.21.15.88:0/2708593757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecf40c43d0 0x7fecf406d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:54.132 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.120+0000 7fecfa346640 1 --2- 172.21.15.88:0/2708593757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fece0004d10 0x7fecf41f5230 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fecf0004f60 tx=0x7fecf0004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:54.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.120+0000 7feceaffd640 1 -- 172.21.15.88:0/2708593757 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf000f7b0 con 0x7fece0004d10 2023-10-16T20:05:54.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.120+0000 7feceaffd640 1 -- 172.21.15.88:0/2708593757 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf000f910 con 0x7fece0004d10 2023-10-16T20:05:54.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.120+0000 7feceaffd640 1 -- 172.21.15.88:0/2708593757 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf000fcc0 con 0x7fece0004d10 2023-10-16T20:05:54.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.120+0000 7fecfc5aa640 1 -- 172.21.15.88:0/2708593757 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecf41f5770 con 0x7fece0004d10 2023-10-16T20:05:54.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.120+0000 7fecfc5aa640 1 -- 172.21.15.88:0/2708593757 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecf41f30d0 con 0x7fece0004d10 2023-10-16T20:05:54.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.120+0000 7fecfc5aa640 1 -- 172.21.15.88:0/2708593757 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecf40c7800 con 0x7fece0004d10 2023-10-16T20:05:54.134 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.120+0000 7feceaffd640 1 -- 172.21.15.88:0/2708593757 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fecf001d460 con 0x7fece0004d10 2023-10-16T20:05:54.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.124+0000 7feceaffd640 1 --2- 172.21.15.88:0/2708593757 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fecd80664a0 0x7fecd8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:54.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.124+0000 7feceaffd640 1 -- 172.21.15.88:0/2708593757 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fecf0084350 con 0x7fece0004d10 2023-10-16T20:05:54.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.124+0000 7fecf9b45640 1 --2- 172.21.15.88:0/2708593757 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fecd80664a0 0x7fecd8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:54.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.124+0000 7fecf9b45640 1 --2- 172.21.15.88:0/2708593757 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fecd80664a0 0x7fecd8068960 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fecf40c52b0 tx=0x7fece4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:54.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.128+0000 7feceaffd640 1 -- 172.21.15.88:0/2708593757 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fecf40c7800 con 0x7fece0004d10 2023-10-16T20:05:54.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.276+0000 7fecfc5aa640 1 -- 172.21.15.88:0/2708593757 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fecf40c7800 con 0x7fece0004d10 2023-10-16T20:05:54.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.276+0000 7feceaffd640 1 -- 172.21.15.88:0/2708593757 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7fecf40c7800 con 0x7fece0004d10 2023-10-16T20:05:54.282 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:05:54.283 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.280+0000 7fecfc5aa640 1 -- 172.21.15.88:0/2708593757 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fecd80664a0 msgr2=0x7fecd8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:54.284 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.280+0000 7fecfc5aa640 1 --2- 172.21.15.88:0/2708593757 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fecd80664a0 0x7fecd8068960 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fecf40c52b0 tx=0x7fece4008040 comp rx=0 tx=0).stop 2023-10-16T20:05:54.284 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.280+0000 7fecfc5aa640 1 -- 172.21.15.88:0/2708593757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fece0004d10 msgr2=0x7fecf41f5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:54.284 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.280+0000 7fecfc5aa640 1 --2- 172.21.15.88:0/2708593757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fece0004d10 0x7fecf41f5230 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fecf0004f60 tx=0x7fecf0004f90 comp rx=0 tx=0).stop 2023-10-16T20:05:54.284 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.280+0000 7fecfc5aa640 1 -- 172.21.15.88:0/2708593757 shutdown_connections 2023-10-16T20:05:54.285 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.280+0000 7fecfc5aa640 1 --2- 172.21.15.88:0/2708593757 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fecd80664a0 0x7fecd8068960 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:54.285 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.280+0000 7fecfc5aa640 1 --2- 172.21.15.88:0/2708593757 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fecf40c4830 0x7fecf406dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:54.285 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.280+0000 7fecfc5aa640 1 --2- 172.21.15.88:0/2708593757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecf40c43d0 0x7fecf406d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:54.285 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.280+0000 7fecfc5aa640 1 --2- 172.21.15.88:0/2708593757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fece0004d10 0x7fecf41f5230 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:54.286 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.280+0000 7fecfc5aa640 1 -- 172.21.15.88:0/2708593757 >> 172.21.15.88:0/2708593757 conn(0x7fecf40b0970 msgr2=0x7fecf4070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:54.286 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.280+0000 7fecfc5aa640 1 -- 172.21.15.88:0/2708593757 shutdown_connections 2023-10-16T20:05:54.286 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.280+0000 7fecfc5aa640 1 -- 172.21.15.88:0/2708593757 wait complete. 2023-10-16T20:05:54.302 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T20:05:54.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.440+0000 7fedcf246640 1 Processor -- start 2023-10-16T20:05:54.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.440+0000 7fedcf246640 1 -- start start 2023-10-16T20:05:54.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.440+0000 7fedcf246640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fedc806d3f0 0x7fedc806d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:54.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.440+0000 7fedcf246640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc815ad30 con 0x7fedc80b4260 2023-10-16T20:05:54.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.440+0000 7fedcf246640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc806dd60 con 0x7fedc80b0da0 2023-10-16T20:05:54.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.440+0000 7fedcf246640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc80b4640 con 0x7fedc806d3f0 2023-10-16T20:05:54.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.440+0000 7fedcd7e3640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fedc806d3f0 0x7fedc806d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:54.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.440+0000 7fedccfe2640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fedc80b4260 0x7fedc815dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:53466/0 (socket says 172.21.15.88:53466) 2023-10-16T20:05:54.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.440+0000 7fedcd7e3640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fedc806d3f0 0x7fedc806d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50578/0 (socket says 172.21.15.88:50578) 2023-10-16T20:05:54.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.440+0000 7fedccfe2640 1 -- 172.21.15.88:0/2514173742 learned_addr learned my addr 172.21.15.88:0/2514173742 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:54.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.440+0000 7fedcd7e3640 1 -- 172.21.15.88:0/2514173742 >> v1:172.21.15.94:6789/0 conn(0x7fedc80b0da0 legacy=0x7fedc8159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:54.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.440+0000 7fedcd7e3640 1 -- 172.21.15.88:0/2514173742 >> v1:172.21.15.88:6789/0 conn(0x7fedc80b4260 legacy=0x7fedc815dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:54.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.440+0000 7fedcd7e3640 1 -- 172.21.15.88:0/2514173742 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc815a770 con 0x7fedc806d3f0 2023-10-16T20:05:54.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.440+0000 7fedc77fe640 1 -- 172.21.15.88:0/2514173742 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2639831562 0 0) 0x7fedc815ad30 con 0x7fedc80b4260 2023-10-16T20:05:54.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.444+0000 7fedcd7e3640 1 --2- 172.21.15.88:0/2514173742 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedc806d3f0 0x7fedc806d820 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009d90 tx=0x7fedc001c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4498d4a40b9b7c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:54.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.444+0000 7fedc77fe640 1 -- 172.21.15.88:0/2514173742 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc00a5870 con 0x7fedc806d3f0 2023-10-16T20:05:54.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.444+0000 7fedc77fe640 1 -- 172.21.15.88:0/2514173742 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc00a59d0 con 0x7fedc806d3f0 2023-10-16T20:05:54.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.444+0000 7fedc77fe640 1 -- 172.21.15.88:0/2514173742 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc00a5d60 con 0x7fedc806d3f0 2023-10-16T20:05:54.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.444+0000 7fedcf246640 1 -- 172.21.15.88:0/2514173742 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedc806d3f0 msgr2=0x7fedc806d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:54.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.444+0000 7fedcf246640 1 --2- 172.21.15.88:0/2514173742 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedc806d3f0 0x7fedc806d820 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009d90 tx=0x7fedc001c8c0 comp rx=0 tx=0).stop 2023-10-16T20:05:54.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.444+0000 7fedcf246640 1 -- 172.21.15.88:0/2514173742 shutdown_connections 2023-10-16T20:05:54.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.444+0000 7fedcf246640 1 --2- 172.21.15.88:0/2514173742 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedc806d3f0 0x7fedc806d820 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:54.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.444+0000 7fedcf246640 1 -- 172.21.15.88:0/2514173742 >> 172.21.15.88:0/2514173742 conn(0x7fedc80cb080 msgr2=0x7fedc80cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:54.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.444+0000 7fedcf246640 1 -- 172.21.15.88:0/2514173742 shutdown_connections 2023-10-16T20:05:54.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.444+0000 7fedcf246640 1 -- 172.21.15.88:0/2514173742 wait complete. 2023-10-16T20:05:54.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.448+0000 7fedcf246640 1 Processor -- start 2023-10-16T20:05:54.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.448+0000 7fedcf246640 1 -- start start 2023-10-16T20:05:54.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.448+0000 7fedcf246640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedc806d3f0 0x7fedc814b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:54.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.448+0000 7fedcf246640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedc80b0da0 0x7fedc814e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:54.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.448+0000 7fedcf246640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fedc80b4260 0x7fedc815c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:54.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.448+0000 7fedcf246640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc814ea00 con 0x7fedc806d3f0 2023-10-16T20:05:54.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.448+0000 7fedcf246640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc814ece0 con 0x7fedc80b0da0 2023-10-16T20:05:54.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.448+0000 7fedcf246640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc814efc0 con 0x7fedc80b4260 2023-10-16T20:05:54.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.448+0000 7fedccfe2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedc806d3f0 0x7fedc814b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:54.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.448+0000 7fedccfe2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedc806d3f0 0x7fedc814b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50584/0 (socket says 172.21.15.88:50584) 2023-10-16T20:05:54.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.448+0000 7fedccfe2640 1 -- 172.21.15.88:0/94752377 learned_addr learned my addr 172.21.15.88:0/94752377 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:54.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.448+0000 7fedc7fff640 1 --2- 172.21.15.88:0/94752377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedc80b0da0 0x7fedc814e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:54.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.448+0000 7fedcd7e3640 1 --2- 172.21.15.88:0/94752377 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fedc80b4260 0x7fedc815c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:54.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.448+0000 7fedccfe2640 1 -- 172.21.15.88:0/94752377 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fedc80b4260 msgr2=0x7fedc815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:54.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.448+0000 7fedccfe2640 1 --2- 172.21.15.88:0/94752377 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fedc80b4260 0x7fedc815c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:54.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.448+0000 7fedccfe2640 1 -- 172.21.15.88:0/94752377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedc80b0da0 msgr2=0x7fedc814e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:54.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.448+0000 7fedccfe2640 1 --2- 172.21.15.88:0/94752377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedc80b0da0 0x7fedc814e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:54.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.448+0000 7fedccfe2640 1 -- 172.21.15.88:0/94752377 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc0009a70 con 0x7fedc806d3f0 2023-10-16T20:05:54.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.448+0000 7fedccfe2640 1 --2- 172.21.15.88:0/94752377 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedc806d3f0 0x7fedc814b7c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fedbc00c410 tx=0x7fedbc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:54.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.448+0000 7fedc57fa640 1 -- 172.21.15.88:0/94752377 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc0074e0 con 0x7fedc806d3f0 2023-10-16T20:05:54.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.448+0000 7fedc57fa640 1 -- 172.21.15.88:0/94752377 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedbc007640 con 0x7fedc806d3f0 2023-10-16T20:05:54.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.448+0000 7fedc57fa640 1 -- 172.21.15.88:0/94752377 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc004a30 con 0x7fedc806d3f0 2023-10-16T20:05:54.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.448+0000 7fedcf246640 1 -- 172.21.15.88:0/94752377 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc815a540 con 0x7fedc806d3f0 2023-10-16T20:05:54.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.448+0000 7fedcf246640 1 -- 172.21.15.88:0/94752377 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc814f580 con 0x7fedc806d3f0 2023-10-16T20:05:54.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.452+0000 7fedcf246640 1 -- 172.21.15.88:0/94752377 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc814f7a0 con 0x7fedc806d3f0 2023-10-16T20:05:54.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.452+0000 7fedc57fa640 1 -- 172.21.15.88:0/94752377 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fedbc004b90 con 0x7fedc806d3f0 2023-10-16T20:05:54.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.452+0000 7fedc57fa640 1 --2- 172.21.15.88:0/94752377 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fedac0664a0 0x7fedac068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:54.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.452+0000 7fedc57fa640 1 -- 172.21.15.88:0/94752377 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fedbc083f70 con 0x7fedc806d3f0 2023-10-16T20:05:54.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.452+0000 7fedc7fff640 1 --2- 172.21.15.88:0/94752377 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fedac0664a0 0x7fedac068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:54.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.452+0000 7fedc7fff640 1 --2- 172.21.15.88:0/94752377 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fedac0664a0 0x7fedac068960 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fedc806c780 tx=0x7fedb4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:54.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.456+0000 7fedc57fa640 1 -- 172.21.15.88:0/94752377 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fedbc051040 con 0x7fedc806d3f0 2023-10-16T20:05:54.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.604+0000 7fedcf246640 1 -- 172.21.15.88:0/94752377 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fedc806d7d0 con 0x7fedc806d3f0 2023-10-16T20:05:54.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.604+0000 7fedc57fa640 1 -- 172.21.15.88:0/94752377 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7fedbc054660 con 0x7fedc806d3f0 2023-10-16T20:05:54.610 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2023-10-16T20:05:54.612 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.608+0000 7fedcf246640 1 -- 172.21.15.88:0/94752377 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fedac0664a0 msgr2=0x7fedac068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:54.612 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.608+0000 7fedcf246640 1 --2- 172.21.15.88:0/94752377 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fedac0664a0 0x7fedac068960 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fedc806c780 tx=0x7fedb4009040 comp rx=0 tx=0).stop 2023-10-16T20:05:54.612 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.608+0000 7fedcf246640 1 -- 172.21.15.88:0/94752377 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedc806d3f0 msgr2=0x7fedc814b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:54.612 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.608+0000 7fedcf246640 1 --2- 172.21.15.88:0/94752377 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedc806d3f0 0x7fedc814b7c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fedbc00c410 tx=0x7fedbc00c8d0 comp rx=0 tx=0).stop 2023-10-16T20:05:54.612 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.608+0000 7fedcf246640 1 -- 172.21.15.88:0/94752377 shutdown_connections 2023-10-16T20:05:54.613 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.608+0000 7fedcf246640 1 --2- 172.21.15.88:0/94752377 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fedac0664a0 0x7fedac068960 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:54.613 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.608+0000 7fedcf246640 1 --2- 172.21.15.88:0/94752377 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fedc80b4260 0x7fedc815c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:54.613 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.608+0000 7fedcf246640 1 --2- 172.21.15.88:0/94752377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedc80b0da0 0x7fedc814e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:54.613 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.608+0000 7fedcf246640 1 --2- 172.21.15.88:0/94752377 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedc806d3f0 0x7fedc814b7c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:54.613 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.608+0000 7fedcf246640 1 -- 172.21.15.88:0/94752377 >> 172.21.15.88:0/94752377 conn(0x7fedc80cb080 msgr2=0x7fedc81541d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:54.614 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.608+0000 7fedcf246640 1 -- 172.21.15.88:0/94752377 shutdown_connections 2023-10-16T20:05:54.614 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.608+0000 7fedcf246640 1 -- 172.21.15.88:0/94752377 wait complete. 2023-10-16T20:05:54.631 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T20:05:54.771 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.768+0000 7fe35f1b8640 1 Processor -- start 2023-10-16T20:05:54.772 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.768+0000 7fe35f1b8640 1 -- start start 2023-10-16T20:05:54.773 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.768+0000 7fe35f1b8640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fe35815cf90 0x7fe3580ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:54.773 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.768+0000 7fe35f1b8640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe358161540 0x7fe3580cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:54.773 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.768+0000 7fe35f1b8640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe358071360 0x7fe3580717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:54.774 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.768+0000 7fe35f1b8640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe358071d30 con 0x7fe35815cf90 2023-10-16T20:05:54.774 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.768+0000 7fe35f1b8640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3580ca5f0 con 0x7fe358161540 2023-10-16T20:05:54.774 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.768+0000 7fe35f1b8640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3580ca840 con 0x7fe358071360 2023-10-16T20:05:54.774 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.768+0000 7fe35cf54640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fe35815cf90 0x7fe3580ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:54.774 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.768+0000 7fe35cf54640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fe35815cf90 0x7fe3580ca0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50594/0 (socket says 172.21.15.88:50594) 2023-10-16T20:05:54.775 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.768+0000 7fe35cf54640 1 -- 172.21.15.88:0/1655781450 learned_addr learned my addr 172.21.15.88:0/1655781450 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:54.775 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.768+0000 7fe357fff640 1 --2- 172.21.15.88:0/1655781450 >> v2:172.21.15.94:3300/0 conn(0x7fe358161540 0x7fe3580cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:54.775 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.768+0000 7fe35d755640 1 --2- 172.21.15.88:0/1655781450 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe358071360 0x7fe3580717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:54.775 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.768+0000 7fe35cf54640 1 -- 172.21.15.88:0/1655781450 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe358071360 msgr2=0x7fe3580717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:54.775 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.768+0000 7fe35cf54640 1 --2- 172.21.15.88:0/1655781450 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe358071360 0x7fe3580717f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:54.775 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.768+0000 7fe35cf54640 1 -- 172.21.15.88:0/1655781450 >> v2:172.21.15.94:3300/0 conn(0x7fe358161540 msgr2=0x7fe3580cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:54.776 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.768+0000 7fe35cf54640 1 --2- 172.21.15.88:0/1655781450 >> v2:172.21.15.94:3300/0 conn(0x7fe358161540 0x7fe3580cb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:54.776 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.768+0000 7fe35cf54640 1 -- 172.21.15.88:0/1655781450 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3580cab20 con 0x7fe35815cf90 2023-10-16T20:05:54.776 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.768+0000 7fe35d755640 1 --2- 172.21.15.88:0/1655781450 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe358071360 0x7fe3580717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:54.777 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.772+0000 7fe35cf54640 1 --2- 172.21.15.88:0/1655781450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe35815cf90 0x7fe3580ca0b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe3580cd230 tx=0x7fe34c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca793db691a39a6e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:54.777 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.772+0000 7fe3577fe640 1 -- 172.21.15.88:0/1655781450 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c0aa5f0 con 0x7fe35815cf90 2023-10-16T20:05:54.778 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.772+0000 7fe3577fe640 1 -- 172.21.15.88:0/1655781450 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c0aa750 con 0x7fe35815cf90 2023-10-16T20:05:54.778 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.772+0000 7fe3577fe640 1 -- 172.21.15.88:0/1655781450 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c0aaae0 con 0x7fe35815cf90 2023-10-16T20:05:54.778 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.772+0000 7fe35f1b8640 1 -- 172.21.15.88:0/1655781450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe35815cf90 msgr2=0x7fe3580ca0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:54.778 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.772+0000 7fe35f1b8640 1 --2- 172.21.15.88:0/1655781450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe35815cf90 0x7fe3580ca0b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe3580cd230 tx=0x7fe34c01c4d0 comp rx=0 tx=0).stop 2023-10-16T20:05:54.779 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.772+0000 7fe35f1b8640 1 -- 172.21.15.88:0/1655781450 shutdown_connections 2023-10-16T20:05:54.779 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.772+0000 7fe35f1b8640 1 --2- 172.21.15.88:0/1655781450 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe358071360 0x7fe3580717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:54.779 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.772+0000 7fe35f1b8640 1 --2- 172.21.15.88:0/1655781450 >> v2:172.21.15.94:3300/0 conn(0x7fe358161540 0x7fe3580cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:54.779 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.772+0000 7fe35f1b8640 1 --2- 172.21.15.88:0/1655781450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe35815cf90 0x7fe3580ca0b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:54.779 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.772+0000 7fe35f1b8640 1 -- 172.21.15.88:0/1655781450 >> 172.21.15.88:0/1655781450 conn(0x7fe3580cf710 msgr2=0x7fe3580cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:54.779 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.772+0000 7fe35f1b8640 1 -- 172.21.15.88:0/1655781450 shutdown_connections 2023-10-16T20:05:54.780 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.772+0000 7fe35f1b8640 1 -- 172.21.15.88:0/1655781450 wait complete. 2023-10-16T20:05:54.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.776+0000 7fe35f1b8640 1 Processor -- start 2023-10-16T20:05:54.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.776+0000 7fe35f1b8640 1 -- start start 2023-10-16T20:05:54.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.776+0000 7fe35f1b8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe358071360 0x7fe358160010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:54.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.776+0000 7fe35f1b8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe35815cf90 0x7fe3580c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:54.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.776+0000 7fe35f1b8640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe358161540 0x7fe35814bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:54.783 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.776+0000 7fe35f1b8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3580c6210 con 0x7fe358071360 2023-10-16T20:05:54.783 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.776+0000 7fe35f1b8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3580c8c70 con 0x7fe35815cf90 2023-10-16T20:05:54.783 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.776+0000 7fe35f1b8640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3580c9180 con 0x7fe358161540 2023-10-16T20:05:54.783 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.776+0000 7fe35cf54640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe358071360 0x7fe358160010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:54.783 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.776+0000 7fe35cf54640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe358071360 0x7fe358160010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50606/0 (socket says 172.21.15.88:50606) 2023-10-16T20:05:54.784 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.776+0000 7fe35cf54640 1 -- 172.21.15.88:0/3670439329 learned_addr learned my addr 172.21.15.88:0/3670439329 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:54.784 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.776+0000 7fe35cf54640 1 -- 172.21.15.88:0/3670439329 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe358161540 msgr2=0x7fe35814bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:54.784 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.776+0000 7fe35cf54640 1 --2- 172.21.15.88:0/3670439329 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe358161540 0x7fe35814bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:54.784 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.776+0000 7fe35cf54640 1 -- 172.21.15.88:0/3670439329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe35815cf90 msgr2=0x7fe3580c8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:54.784 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.776+0000 7fe35cf54640 1 --2- 172.21.15.88:0/3670439329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe35815cf90 0x7fe3580c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:54.784 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.776+0000 7fe35cf54640 1 -- 172.21.15.88:0/3670439329 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34c0094d0 con 0x7fe358071360 2023-10-16T20:05:54.785 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.776+0000 7fe35cf54640 1 --2- 172.21.15.88:0/3670439329 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe358071360 0x7fe358160010 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe34c0ac040 tx=0x7fe34c01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:54.785 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.776+0000 7fe3557fa640 1 -- 172.21.15.88:0/3670439329 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c0bd5f0 con 0x7fe358071360 2023-10-16T20:05:54.785 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.776+0000 7fe3557fa640 1 -- 172.21.15.88:0/3670439329 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c0bd770 con 0x7fe358071360 2023-10-16T20:05:54.785 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.776+0000 7fe35f1b8640 1 -- 172.21.15.88:0/3670439329 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe35815dc70 con 0x7fe358071360 2023-10-16T20:05:54.785 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.776+0000 7fe3557fa640 1 -- 172.21.15.88:0/3670439329 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c0bdb20 con 0x7fe358071360 2023-10-16T20:05:54.786 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.776+0000 7fe35f1b8640 1 -- 172.21.15.88:0/3670439329 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe35815d620 con 0x7fe358071360 2023-10-16T20:05:54.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.780+0000 7fe35f1b8640 1 -- 172.21.15.88:0/3670439329 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe35804e9c0 con 0x7fe358071360 2023-10-16T20:05:54.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.780+0000 7fe3557fa640 1 -- 172.21.15.88:0/3670439329 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe34c0b3600 con 0x7fe358071360 2023-10-16T20:05:54.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.780+0000 7fe3557fa640 1 --2- 172.21.15.88:0/3670439329 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe33c0664a0 0x7fe33c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:54.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.780+0000 7fe3557fa640 1 -- 172.21.15.88:0/3670439329 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe34c0a7070 con 0x7fe358071360 2023-10-16T20:05:54.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.780+0000 7fe357fff640 1 --2- 172.21.15.88:0/3670439329 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe33c0664a0 0x7fe33c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:54.788 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.780+0000 7fe357fff640 1 --2- 172.21.15.88:0/3670439329 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe33c0664a0 0x7fe33c068960 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe344006aa0 tx=0x7fe344006a30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:54.788 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.784+0000 7fe3557fa640 1 -- 172.21.15.88:0/3670439329 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe34c0ec460 con 0x7fe358071360 2023-10-16T20:05:54.939 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.936+0000 7fe35f1b8640 1 -- 172.21.15.88:0/3670439329 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe3580c6520 con 0x7fe358071360 2023-10-16T20:05:54.940 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.936+0000 7fe3557fa640 1 -- 172.21.15.88:0/3670439329 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe34c0efa80 con 0x7fe358071360 2023-10-16T20:05:54.940 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:05:54.941 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.936+0000 7fe35f1b8640 1 -- 172.21.15.88:0/3670439329 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe33c0664a0 msgr2=0x7fe33c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:54.942 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.936+0000 7fe35f1b8640 1 --2- 172.21.15.88:0/3670439329 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe33c0664a0 0x7fe33c068960 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe344006aa0 tx=0x7fe344006a30 comp rx=0 tx=0).stop 2023-10-16T20:05:54.942 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.936+0000 7fe35f1b8640 1 -- 172.21.15.88:0/3670439329 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe358071360 msgr2=0x7fe358160010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:54.942 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.936+0000 7fe35f1b8640 1 --2- 172.21.15.88:0/3670439329 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe358071360 0x7fe358160010 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe34c0ac040 tx=0x7fe34c01ad10 comp rx=0 tx=0).stop 2023-10-16T20:05:54.942 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.936+0000 7fe35f1b8640 1 -- 172.21.15.88:0/3670439329 shutdown_connections 2023-10-16T20:05:54.942 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.936+0000 7fe35f1b8640 1 --2- 172.21.15.88:0/3670439329 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe33c0664a0 0x7fe33c068960 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:54.943 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.936+0000 7fe35f1b8640 1 --2- 172.21.15.88:0/3670439329 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe358161540 0x7fe35814bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:54.943 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.936+0000 7fe35f1b8640 1 --2- 172.21.15.88:0/3670439329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe35815cf90 0x7fe3580c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:54.943 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.936+0000 7fe35f1b8640 1 --2- 172.21.15.88:0/3670439329 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe358071360 0x7fe358160010 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:54.943 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.936+0000 7fe35f1b8640 1 -- 172.21.15.88:0/3670439329 >> 172.21.15.88:0/3670439329 conn(0x7fe3580cf710 msgr2=0x7fe358152040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:54.943 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.936+0000 7fe35f1b8640 1 -- 172.21.15.88:0/3670439329 shutdown_connections 2023-10-16T20:05:54.944 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:54.936+0000 7fe35f1b8640 1 -- 172.21.15.88:0/3670439329 wait complete. 2023-10-16T20:05:54.961 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T20:05:55.098 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.092+0000 7f13e0565640 1 Processor -- start 2023-10-16T20:05:55.099 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.096+0000 7f13e0565640 1 -- start start 2023-10-16T20:05:55.100 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.096+0000 7f13e0565640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f13d80adf80 0x7f13d815a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:55.100 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.096+0000 7f13e0565640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f13d815d480 0x7f13d80afa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:55.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.096+0000 7f13e0565640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d815e400 con 0x7f13d8069e90 2023-10-16T20:05:55.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.096+0000 7f13e0565640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13d8159e40 con 0x7f13d815d480 2023-10-16T20:05:55.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.096+0000 7f13e0565640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d815de40 con 0x7f13d80adf80 2023-10-16T20:05:55.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.096+0000 7f13de301640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f13d8069e90 0x7f13d806a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:53476/0 (socket says 172.21.15.88:53476) 2023-10-16T20:05:55.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.096+0000 7f13de301640 1 -- 172.21.15.88:0/283447823 learned_addr learned my addr 172.21.15.88:0/283447823 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:55.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.096+0000 7f13deb02640 1 --2- 172.21.15.88:0/283447823 >> v2:172.21.15.94:3300/0 conn(0x7f13d815d480 0x7f13d80afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:55.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.096+0000 7f13ddb00640 1 --2- 172.21.15.88:0/283447823 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f13d80adf80 0x7f13d815a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:55.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.096+0000 7f13dd2ff640 1 -- 172.21.15.88:0/283447823 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2356635945 0 0) 0x7f13d815e400 con 0x7f13d8069e90 2023-10-16T20:05:55.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.096+0000 7f13dd2ff640 1 -- 172.21.15.88:0/283447823 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13c4003530 con 0x7f13d8069e90 2023-10-16T20:05:55.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.096+0000 7f13deb02640 1 -- 172.21.15.88:0/283447823 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f13d80adf80 msgr2=0x7f13d815a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:55.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.096+0000 7f13dd2ff640 1 -- 172.21.15.88:0/283447823 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 206321483 0 0) 0x7f13c4003530 con 0x7f13d8069e90 2023-10-16T20:05:55.104 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.096+0000 7f13deb02640 1 --2- 172.21.15.88:0/283447823 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f13d80adf80 0x7f13d815a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:55.104 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.096+0000 7f13deb02640 1 -- 172.21.15.88:0/283447823 >> v1:172.21.15.88:6789/0 conn(0x7f13d8069e90 legacy=0x7f13d806a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:55.104 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.096+0000 7f13deb02640 1 -- 172.21.15.88:0/283447823 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d815e120 con 0x7f13d815d480 2023-10-16T20:05:55.105 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.100+0000 7f13deb02640 1 --2- 172.21.15.88:0/283447823 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13d815d480 0x7f13d80afa20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f13d0009d90 tx=0x7f13d001c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be2f398ea41da09c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:55.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.100+0000 7f13dd2ff640 1 -- 172.21.15.88:0/283447823 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d4001680 con 0x7f13d815d480 2023-10-16T20:05:55.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.100+0000 7f13dd2ff640 1 -- 172.21.15.88:0/283447823 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d00a5640 con 0x7f13d815d480 2023-10-16T20:05:55.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.100+0000 7f13dd2ff640 1 -- 172.21.15.88:0/283447823 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d00a59d0 con 0x7f13d815d480 2023-10-16T20:05:55.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.100+0000 7f13e0565640 1 -- 172.21.15.88:0/283447823 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13d815d480 msgr2=0x7f13d80afa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:55.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.100+0000 7f13e0565640 1 --2- 172.21.15.88:0/283447823 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13d815d480 0x7f13d80afa20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f13d0009d90 tx=0x7f13d001c840 comp rx=0 tx=0).stop 2023-10-16T20:05:55.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.100+0000 7f13e0565640 1 -- 172.21.15.88:0/283447823 shutdown_connections 2023-10-16T20:05:55.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.100+0000 7f13e0565640 1 --2- 172.21.15.88:0/283447823 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13d815d480 0x7f13d80afa20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:55.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.100+0000 7f13e0565640 1 --2- 172.21.15.88:0/283447823 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f13d80adf80 0x7f13d815a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:55.108 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.100+0000 7f13e0565640 1 -- 172.21.15.88:0/283447823 >> 172.21.15.88:0/283447823 conn(0x7f13d80ce5e0 msgr2=0x7f13d80ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:55.108 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.100+0000 7f13e0565640 1 -- 172.21.15.88:0/283447823 shutdown_connections 2023-10-16T20:05:55.108 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.100+0000 7f13e0565640 1 -- 172.21.15.88:0/283447823 wait complete. 2023-10-16T20:05:55.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.104+0000 7f13e0565640 1 Processor -- start 2023-10-16T20:05:55.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.104+0000 7f13e0565640 1 -- start start 2023-10-16T20:05:55.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.104+0000 7f13e0565640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f13d8069e90 0x7f13d8070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:55.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.104+0000 7f13e0565640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13d80adf80 0x7f13d8154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:55.112 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.104+0000 7f13e0565640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13d815d480 0x7f13d81545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:55.112 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.104+0000 7f13e0565640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d8071f30 con 0x7f13d80adf80 2023-10-16T20:05:55.112 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.104+0000 7f13e0565640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d8071320 con 0x7f13d815d480 2023-10-16T20:05:55.112 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.104+0000 7f13e0565640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d8071600 con 0x7f13d8069e90 2023-10-16T20:05:55.112 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.104+0000 7f13ddb00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13d80adf80 0x7f13d8154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:55.113 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.104+0000 7f13ddb00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13d80adf80 0x7f13d8154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50614/0 (socket says 172.21.15.88:50614) 2023-10-16T20:05:55.113 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.104+0000 7f13ddb00640 1 -- 172.21.15.88:0/3914750708 learned_addr learned my addr 172.21.15.88:0/3914750708 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:55.113 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.104+0000 7f13de301640 1 --2- 172.21.15.88:0/3914750708 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f13d8069e90 0x7f13d8070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:55.113 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.104+0000 7f13deb02640 1 --2- 172.21.15.88:0/3914750708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13d815d480 0x7f13d81545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:55.113 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.104+0000 7f13ddb00640 1 -- 172.21.15.88:0/3914750708 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f13d8069e90 msgr2=0x7f13d8070ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:55.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.104+0000 7f13ddb00640 1 --2- 172.21.15.88:0/3914750708 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f13d8069e90 0x7f13d8070ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:55.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.104+0000 7f13ddb00640 1 -- 172.21.15.88:0/3914750708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13d815d480 msgr2=0x7f13d81545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:55.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.104+0000 7f13ddb00640 1 --2- 172.21.15.88:0/3914750708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13d815d480 0x7f13d81545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:55.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.104+0000 7f13ddb00640 1 -- 172.21.15.88:0/3914750708 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d0009a70 con 0x7f13d80adf80 2023-10-16T20:05:55.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.104+0000 7f13deb02640 1 --2- 172.21.15.88:0/3914750708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13d815d480 0x7f13d81545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:55.115 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.104+0000 7f13de301640 1 --2- 172.21.15.88:0/3914750708 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f13d8069e90 0x7f13d8070ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:55.115 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.104+0000 7f13ddb00640 1 --2- 172.21.15.88:0/3914750708 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13d80adf80 0x7f13d8154070 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f13c800e930 tx=0x7f13c800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:55.115 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.104+0000 7f13ceffd640 1 -- 172.21.15.88:0/3914750708 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c8009690 con 0x7f13d80adf80 2023-10-16T20:05:55.115 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.104+0000 7f13ceffd640 1 -- 172.21.15.88:0/3914750708 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c800ca00 con 0x7f13d80adf80 2023-10-16T20:05:55.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.104+0000 7f13e0565640 1 -- 172.21.15.88:0/3914750708 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d80718e0 con 0x7f13d80adf80 2023-10-16T20:05:55.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.104+0000 7f13ceffd640 1 -- 172.21.15.88:0/3914750708 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c800cd70 con 0x7f13d80adf80 2023-10-16T20:05:55.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.104+0000 7f13e0565640 1 -- 172.21.15.88:0/3914750708 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d8071010 con 0x7f13d80adf80 2023-10-16T20:05:55.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.104+0000 7f13e0565640 1 -- 172.21.15.88:0/3914750708 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d815ad50 con 0x7f13d80adf80 2023-10-16T20:05:55.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.108+0000 7f13ceffd640 1 -- 172.21.15.88:0/3914750708 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f13c8013470 con 0x7f13d80adf80 2023-10-16T20:05:55.117 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.108+0000 7f13ceffd640 1 --2- 172.21.15.88:0/3914750708 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f13bc0664a0 0x7f13bc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:55.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.108+0000 7f13ceffd640 1 -- 172.21.15.88:0/3914750708 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f13c80845f0 con 0x7f13d80adf80 2023-10-16T20:05:55.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.108+0000 7f13de301640 1 --2- 172.21.15.88:0/3914750708 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f13bc0664a0 0x7f13bc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:55.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.108+0000 7f13de301640 1 --2- 172.21.15.88:0/3914750708 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f13bc0664a0 0x7f13bc068960 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f13d4001ec0 tx=0x7f13d4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:55.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.112+0000 7f13ceffd640 1 -- 172.21.15.88:0/3914750708 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f13d815ad50 con 0x7f13d80adf80 2023-10-16T20:05:55.276 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.272+0000 7f13e0565640 1 -- 172.21.15.88:0/3914750708 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f13d815ad50 con 0x7f13d80adf80 2023-10-16T20:05:55.277 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.272+0000 7f13ceffd640 1 -- 172.21.15.88:0/3914750708 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7f13d815ad50 con 0x7f13d80adf80 2023-10-16T20:05:55.277 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2023-10-16T20:05:55.280 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.276+0000 7f13e0565640 1 -- 172.21.15.88:0/3914750708 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f13bc0664a0 msgr2=0x7f13bc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:55.280 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.276+0000 7f13e0565640 1 --2- 172.21.15.88:0/3914750708 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f13bc0664a0 0x7f13bc068960 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f13d4001ec0 tx=0x7f13d4009040 comp rx=0 tx=0).stop 2023-10-16T20:05:55.280 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.276+0000 7f13e0565640 1 -- 172.21.15.88:0/3914750708 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13d80adf80 msgr2=0x7f13d8154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:55.280 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.276+0000 7f13e0565640 1 --2- 172.21.15.88:0/3914750708 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13d80adf80 0x7f13d8154070 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f13c800e930 tx=0x7f13c800edf0 comp rx=0 tx=0).stop 2023-10-16T20:05:55.280 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.276+0000 7f13e0565640 1 -- 172.21.15.88:0/3914750708 shutdown_connections 2023-10-16T20:05:55.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.276+0000 7f13e0565640 1 --2- 172.21.15.88:0/3914750708 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f13bc0664a0 0x7f13bc068960 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:55.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.276+0000 7f13e0565640 1 --2- 172.21.15.88:0/3914750708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13d815d480 0x7f13d81545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:55.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.276+0000 7f13e0565640 1 --2- 172.21.15.88:0/3914750708 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13d80adf80 0x7f13d8154070 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:55.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.276+0000 7f13e0565640 1 --2- 172.21.15.88:0/3914750708 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f13d8069e90 0x7f13d8070ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:55.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.276+0000 7f13e0565640 1 -- 172.21.15.88:0/3914750708 >> 172.21.15.88:0/3914750708 conn(0x7f13d80ce5e0 msgr2=0x7f13d80b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:55.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.276+0000 7f13e0565640 1 -- 172.21.15.88:0/3914750708 shutdown_connections 2023-10-16T20:05:55.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.276+0000 7f13e0565640 1 -- 172.21.15.88:0/3914750708 wait complete. 2023-10-16T20:05:55.300 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:05:55.300 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:05:55.309 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:05:55.492 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.488+0000 7f5c2ce9a640 1 Processor -- start 2023-10-16T20:05:55.492 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.488+0000 7f5c2ce9a640 1 -- start start 2023-10-16T20:05:55.493 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.488+0000 7f5c2ce9a640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5c2815d9f0 0x7f5c28161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:55.494 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.488+0000 7f5c2ce9a640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5c280d26c0 0x7f5c280cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:55.494 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.488+0000 7f5c2ce9a640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c280b87e0 con 0x7f5c280d22e0 2023-10-16T20:05:55.494 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.488+0000 7f5c2ce9a640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c2815df10 con 0x7f5c280d26c0 2023-10-16T20:05:55.494 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.488+0000 7f5c2ce9a640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c2815e1f0 con 0x7f5c2815d9f0 2023-10-16T20:05:55.495 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.488+0000 7f5c26d9d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5c280d26c0 0x7f5c280cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:55.495 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.488+0000 7f5c26d9d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5c280d26c0 0x7f5c280cf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50624/0 (socket says 172.21.15.88:50624) 2023-10-16T20:05:55.495 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.488+0000 7f5c26d9d640 1 -- 172.21.15.88:0/4265001681 learned_addr learned my addr 172.21.15.88:0/4265001681 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:55.495 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.488+0000 7f5c1dd9b640 1 --2- 172.21.15.88:0/4265001681 >> v2:172.21.15.94:3300/0 conn(0x7f5c2815d9f0 0x7f5c28161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:55.496 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.488+0000 7f5c26d9d640 1 -- 172.21.15.88:0/4265001681 >> v2:172.21.15.94:3300/0 conn(0x7f5c2815d9f0 msgr2=0x7f5c28161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:55.496 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.488+0000 7f5c26d9d640 1 --2- 172.21.15.88:0/4265001681 >> v2:172.21.15.94:3300/0 conn(0x7f5c2815d9f0 0x7f5c28161450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:55.496 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.488+0000 7f5c26d9d640 1 -- 172.21.15.88:0/4265001681 >> v1:172.21.15.94:6789/0 conn(0x7f5c280d22e0 legacy=0x7f5c2815d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:55.496 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.488+0000 7f5c26d9d640 1 -- 172.21.15.88:0/4265001681 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c28160970 con 0x7f5c280d26c0 2023-10-16T20:05:55.498 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.492+0000 7f5c26d9d640 1 --2- 172.21.15.88:0/4265001681 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c280d26c0 0x7f5c280cf2e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5c28072520 tx=0x7f5c2001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a752756e0feff49e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:55.498 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.492+0000 7f5c25d9b640 1 -- 172.21.15.88:0/4265001681 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c200a57d0 con 0x7f5c280d26c0 2023-10-16T20:05:55.498 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.492+0000 7f5c25d9b640 1 -- 172.21.15.88:0/4265001681 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c200a5930 con 0x7f5c280d26c0 2023-10-16T20:05:55.498 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.492+0000 7f5c25d9b640 1 -- 172.21.15.88:0/4265001681 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c200a5cc0 con 0x7f5c280d26c0 2023-10-16T20:05:55.499 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.492+0000 7f5c2ce9a640 1 -- 172.21.15.88:0/4265001681 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c280d26c0 msgr2=0x7f5c280cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:55.499 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.492+0000 7f5c2ce9a640 1 --2- 172.21.15.88:0/4265001681 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c280d26c0 0x7f5c280cf2e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5c28072520 tx=0x7f5c2001c910 comp rx=0 tx=0).stop 2023-10-16T20:05:55.499 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.492+0000 7f5c2ce9a640 1 -- 172.21.15.88:0/4265001681 shutdown_connections 2023-10-16T20:05:55.499 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.492+0000 7f5c2ce9a640 1 --2- 172.21.15.88:0/4265001681 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c280d26c0 0x7f5c280cf2e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:55.499 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.492+0000 7f5c2ce9a640 1 --2- 172.21.15.88:0/4265001681 >> v2:172.21.15.94:3300/0 conn(0x7f5c2815d9f0 0x7f5c28161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:55.500 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.492+0000 7f5c2ce9a640 1 -- 172.21.15.88:0/4265001681 >> 172.21.15.88:0/4265001681 conn(0x7f5c280715c0 msgr2=0x7f5c280719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:55.500 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.492+0000 7f5c2ce9a640 1 -- 172.21.15.88:0/4265001681 shutdown_connections 2023-10-16T20:05:55.500 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.492+0000 7f5c2ce9a640 1 -- 172.21.15.88:0/4265001681 wait complete. 2023-10-16T20:05:55.502 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.496+0000 7f5c2ce9a640 1 Processor -- start 2023-10-16T20:05:55.502 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.496+0000 7f5c2ce9a640 1 -- start start 2023-10-16T20:05:55.502 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.496+0000 7f5c2ce9a640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c280d22e0 0x7f5c280c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:55.503 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.496+0000 7f5c2ce9a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c280d26c0 0x7f5c281f8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:55.503 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.496+0000 7f5c2ce9a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c2815d9f0 0x7f5c281f9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:55.503 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.496+0000 7f5c2ce9a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c281f8880 con 0x7f5c2815d9f0 2023-10-16T20:05:55.503 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.496+0000 7f5c2ce9a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c281f86a0 con 0x7f5c280d26c0 2023-10-16T20:05:55.504 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.496+0000 7f5c2ce9a640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c281f8e00 con 0x7f5c280d22e0 2023-10-16T20:05:55.504 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.496+0000 7f5c26d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c2815d9f0 0x7f5c281f9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:55.504 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.496+0000 7f5c26d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c2815d9f0 0x7f5c281f9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50640/0 (socket says 172.21.15.88:50640) 2023-10-16T20:05:55.504 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.496+0000 7f5c26d9d640 1 -- 172.21.15.88:0/525442368 learned_addr learned my addr 172.21.15.88:0/525442368 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:55.505 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.496+0000 7f5c1dd9b640 1 --2- 172.21.15.88:0/525442368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c280d26c0 0x7f5c281f8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:55.505 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.496+0000 7f5c2659c640 1 --2- 172.21.15.88:0/525442368 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c280d22e0 0x7f5c280c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:55.505 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.496+0000 7f5c26d9d640 1 -- 172.21.15.88:0/525442368 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c280d22e0 msgr2=0x7f5c280c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:55.505 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.496+0000 7f5c26d9d640 1 --2- 172.21.15.88:0/525442368 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c280d22e0 0x7f5c280c8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:55.505 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.496+0000 7f5c26d9d640 1 -- 172.21.15.88:0/525442368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c280d26c0 msgr2=0x7f5c281f8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:55.506 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.496+0000 7f5c26d9d640 1 --2- 172.21.15.88:0/525442368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c280d26c0 0x7f5c281f8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:55.506 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.496+0000 7f5c26d9d640 1 -- 172.21.15.88:0/525442368 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c20009b30 con 0x7f5c2815d9f0 2023-10-16T20:05:55.506 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.496+0000 7f5c26d9d640 1 --2- 172.21.15.88:0/525442368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c2815d9f0 0x7f5c281f9290 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5c20019520 tx=0x7f5c200196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:55.506 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.496+0000 7f5c1f7fe640 1 -- 172.21.15.88:0/525442368 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c200b8670 con 0x7f5c2815d9f0 2023-10-16T20:05:55.506 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.496+0000 7f5c1f7fe640 1 -- 172.21.15.88:0/525442368 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c200b87d0 con 0x7f5c2815d9f0 2023-10-16T20:05:55.507 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.496+0000 7f5c1f7fe640 1 -- 172.21.15.88:0/525442368 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c200b8b60 con 0x7f5c2815d9f0 2023-10-16T20:05:55.507 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.496+0000 7f5c2ce9a640 1 -- 172.21.15.88:0/525442368 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c281f97d0 con 0x7f5c2815d9f0 2023-10-16T20:05:55.507 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.496+0000 7f5c2ce9a640 1 -- 172.21.15.88:0/525442368 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c280c9320 con 0x7f5c2815d9f0 2023-10-16T20:05:55.508 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.500+0000 7f5c2ce9a640 1 -- 172.21.15.88:0/525442368 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c280c9510 con 0x7f5c2815d9f0 2023-10-16T20:05:55.508 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.500+0000 7f5c1f7fe640 1 -- 172.21.15.88:0/525442368 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5c200b8cc0 con 0x7f5c2815d9f0 2023-10-16T20:05:55.509 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.500+0000 7f5c1f7fe640 1 --2- 172.21.15.88:0/525442368 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5c080664a0 0x7f5c08068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:55.509 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.500+0000 7f5c1f7fe640 1 -- 172.21.15.88:0/525442368 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5c2011f8e0 con 0x7f5c2815d9f0 2023-10-16T20:05:55.509 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.500+0000 7f5c2659c640 1 --2- 172.21.15.88:0/525442368 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5c080664a0 0x7f5c08068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:55.509 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.500+0000 7f5c2659c640 1 --2- 172.21.15.88:0/525442368 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5c080664a0 0x7f5c08068960 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5c28167c30 tx=0x7f5c10009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:55.510 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.504+0000 7f5c1f7fe640 1 -- 172.21.15.88:0/525442368 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5c200ec900 con 0x7f5c2815d9f0 2023-10-16T20:05:55.661 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.656+0000 7f5c2ce9a640 1 -- 172.21.15.88:0/525442368 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c281f8f70 con 0x7f5c2815d9f0 2023-10-16T20:05:55.662 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.656+0000 7f5c1f7fe640 1 -- 172.21.15.88:0/525442368 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18661 (secure 0 0 0) 0x7f5c200eff20 con 0x7f5c2815d9f0 2023-10-16T20:05:55.662 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:55.663 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":29,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:05:38.202959+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T20:04:54.442487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T20:04:55.766641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T20:04:59.475830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":1}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:05:55.664 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.660+0000 7f5c2ce9a640 1 -- 172.21.15.88:0/525442368 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5c080664a0 msgr2=0x7f5c08068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:55.664 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.660+0000 7f5c2ce9a640 1 --2- 172.21.15.88:0/525442368 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5c080664a0 0x7f5c08068960 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5c28167c30 tx=0x7f5c10009040 comp rx=0 tx=0).stop 2023-10-16T20:05:55.664 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.660+0000 7f5c2ce9a640 1 -- 172.21.15.88:0/525442368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c2815d9f0 msgr2=0x7f5c281f9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:55.665 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.660+0000 7f5c2ce9a640 1 --2- 172.21.15.88:0/525442368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c2815d9f0 0x7f5c281f9290 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5c20019520 tx=0x7f5c200196b0 comp rx=0 tx=0).stop 2023-10-16T20:05:55.665 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.660+0000 7f5c2ce9a640 1 -- 172.21.15.88:0/525442368 shutdown_connections 2023-10-16T20:05:55.665 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.660+0000 7f5c2ce9a640 1 --2- 172.21.15.88:0/525442368 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5c080664a0 0x7f5c08068960 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:55.665 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.660+0000 7f5c2ce9a640 1 --2- 172.21.15.88:0/525442368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c2815d9f0 0x7f5c281f9290 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:55.665 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.660+0000 7f5c2ce9a640 1 --2- 172.21.15.88:0/525442368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c280d26c0 0x7f5c281f8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:55.666 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.660+0000 7f5c2ce9a640 1 --2- 172.21.15.88:0/525442368 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c280d22e0 0x7f5c280c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:55.666 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.660+0000 7f5c2ce9a640 1 -- 172.21.15.88:0/525442368 >> 172.21.15.88:0/525442368 conn(0x7f5c280715c0 msgr2=0x7f5c280b78c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:55.666 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.660+0000 7f5c2ce9a640 1 -- 172.21.15.88:0/525442368 shutdown_connections 2023-10-16T20:05:55.666 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.660+0000 7f5c2ce9a640 1 -- 172.21.15.88:0/525442368 wait complete. 2023-10-16T20:05:55.683 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T20:04:54.442487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T20:04:55.766641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T20:04:59.475830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T20:05:55.684 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:05:55.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.820+0000 7f916ed2a640 1 Processor -- start 2023-10-16T20:05:55.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.820+0000 7f916ed2a640 1 -- start start 2023-10-16T20:05:55.826 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.820+0000 7f916ed2a640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f91680b4260 0x7f916815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:55.826 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.820+0000 7f916ed2a640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f91680b0da0 0x7f9168159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:55.827 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.820+0000 7f916ed2a640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f916815ad30 con 0x7f916815a1f0 2023-10-16T20:05:55.827 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.820+0000 7f916ed2a640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91680b4640 con 0x7f91680b0da0 2023-10-16T20:05:55.827 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.820+0000 7f916ed2a640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f916815a770 con 0x7f91680b4260 2023-10-16T20:05:55.827 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.820+0000 7f9167fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f91680b0da0 0x7f9168159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:55.827 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.820+0000 7f9167fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f91680b0da0 0x7f9168159cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50654/0 (socket says 172.21.15.88:50654) 2023-10-16T20:05:55.828 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.820+0000 7f9167fff640 1 -- 172.21.15.88:0/569708550 learned_addr learned my addr 172.21.15.88:0/569708550 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:55.828 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.820+0000 7f916d2c7640 1 --1- 172.21.15.88:0/569708550 >> v1:172.21.15.88:6789/0 conn(0x7f916815a1f0 0x7f916806d3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:53484/0 (socket says 172.21.15.88:53484) 2023-10-16T20:05:55.828 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.820+0000 7f916cac6640 1 --2- 172.21.15.88:0/569708550 >> v2:172.21.15.94:3300/0 conn(0x7f91680b4260 0x7f916815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:55.828 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.820+0000 7f9167fff640 1 -- 172.21.15.88:0/569708550 >> v2:172.21.15.94:3300/0 conn(0x7f91680b4260 msgr2=0x7f916815dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:55.829 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.820+0000 7f9167fff640 1 --2- 172.21.15.88:0/569708550 >> v2:172.21.15.94:3300/0 conn(0x7f91680b4260 0x7f916815dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:55.829 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.820+0000 7f9167fff640 1 -- 172.21.15.88:0/569708550 >> v1:172.21.15.88:6789/0 conn(0x7f916815a1f0 legacy=0x7f916806d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:55.829 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.820+0000 7f9167fff640 1 -- 172.21.15.88:0/569708550 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916815aa50 con 0x7f91680b0da0 2023-10-16T20:05:55.829 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.820+0000 7f91677fe640 1 -- 172.21.15.88:0/569708550 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2374836292 0 0) 0x7f916815ad30 con 0x7f916815a1f0 2023-10-16T20:05:55.830 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.824+0000 7f9167fff640 1 --2- 172.21.15.88:0/569708550 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91680b0da0 0x7f9168159cb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f91680b1a80 tx=0x7f915401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858e8d7e8ddf97 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:55.831 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.824+0000 7f91677fe640 1 -- 172.21.15.88:0/569708550 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91540ab640 con 0x7f91680b0da0 2023-10-16T20:05:55.831 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.824+0000 7f91677fe640 1 -- 172.21.15.88:0/569708550 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91540a7070 con 0x7f91680b0da0 2023-10-16T20:05:55.831 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.824+0000 7f91677fe640 1 -- 172.21.15.88:0/569708550 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91540ab9c0 con 0x7f91680b0da0 2023-10-16T20:05:55.831 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.824+0000 7f916ed2a640 1 -- 172.21.15.88:0/569708550 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91680b0da0 msgr2=0x7f9168159cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:55.832 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.824+0000 7f916ed2a640 1 --2- 172.21.15.88:0/569708550 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91680b0da0 0x7f9168159cb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f91680b1a80 tx=0x7f915401c440 comp rx=0 tx=0).stop 2023-10-16T20:05:55.832 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.824+0000 7f916ed2a640 1 -- 172.21.15.88:0/569708550 shutdown_connections 2023-10-16T20:05:55.832 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.824+0000 7f916ed2a640 1 --2- 172.21.15.88:0/569708550 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91680b0da0 0x7f9168159cb0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:55.832 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.824+0000 7f916ed2a640 1 --2- 172.21.15.88:0/569708550 >> v2:172.21.15.94:3300/0 conn(0x7f91680b4260 0x7f916815dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:55.833 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.824+0000 7f916ed2a640 1 -- 172.21.15.88:0/569708550 >> 172.21.15.88:0/569708550 conn(0x7f91680cb080 msgr2=0x7f91680cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:55.833 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.824+0000 7f916ed2a640 1 -- 172.21.15.88:0/569708550 shutdown_connections 2023-10-16T20:05:55.833 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.824+0000 7f916ed2a640 1 -- 172.21.15.88:0/569708550 wait complete. 2023-10-16T20:05:55.835 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.828+0000 7f916ed2a640 1 Processor -- start 2023-10-16T20:05:55.836 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.828+0000 7f916ed2a640 1 -- start start 2023-10-16T20:05:55.836 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.828+0000 7f916ed2a640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f91680b0da0 0x7f916814f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:55.836 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.828+0000 7f916ed2a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91680b4260 0x7f916815b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:55.836 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.828+0000 7f916ed2a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f916815a1f0 0x7f9168153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:55.837 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.828+0000 7f916ed2a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91681508e0 con 0x7f916815a1f0 2023-10-16T20:05:55.837 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.828+0000 7f916ed2a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916815cb70 con 0x7f91680b4260 2023-10-16T20:05:55.837 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.828+0000 7f916ed2a640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916815c2d0 con 0x7f91680b0da0 2023-10-16T20:05:55.837 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.828+0000 7f916d2c7640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f916815a1f0 0x7f9168153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:55.837 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.828+0000 7f916d2c7640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f916815a1f0 0x7f9168153eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50668/0 (socket says 172.21.15.88:50668) 2023-10-16T20:05:55.838 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.828+0000 7f916d2c7640 1 -- 172.21.15.88:0/3606843316 learned_addr learned my addr 172.21.15.88:0/3606843316 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:55.838 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.828+0000 7f9167fff640 1 --2- 172.21.15.88:0/3606843316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91680b4260 0x7f916815b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:55.838 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.828+0000 7f916cac6640 1 --2- 172.21.15.88:0/3606843316 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f91680b0da0 0x7f916814f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:55.838 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.828+0000 7f916d2c7640 1 -- 172.21.15.88:0/3606843316 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f91680b0da0 msgr2=0x7f916814f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:55.839 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.828+0000 7f916d2c7640 1 --2- 172.21.15.88:0/3606843316 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f91680b0da0 0x7f916814f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:55.839 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.828+0000 7f916d2c7640 1 -- 172.21.15.88:0/3606843316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91680b4260 msgr2=0x7f916815b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:55.839 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.828+0000 7f916d2c7640 1 --2- 172.21.15.88:0/3606843316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91680b4260 0x7f916815b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:55.839 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.828+0000 7f916d2c7640 1 -- 172.21.15.88:0/3606843316 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91540094d0 con 0x7f916815a1f0 2023-10-16T20:05:55.839 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.828+0000 7f916d2c7640 1 --2- 172.21.15.88:0/3606843316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f916815a1f0 0x7f9168153eb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f916000c780 tx=0x7f916000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:55.840 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.828+0000 7f91657fa640 1 -- 172.21.15.88:0/3606843316 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916000ce90 con 0x7f916815a1f0 2023-10-16T20:05:55.840 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.828+0000 7f91657fa640 1 -- 172.21.15.88:0/3606843316 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f916000a640 con 0x7f916815a1f0 2023-10-16T20:05:55.840 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.828+0000 7f916ed2a640 1 -- 172.21.15.88:0/3606843316 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f916815c5b0 con 0x7f916815a1f0 2023-10-16T20:05:55.840 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.828+0000 7f91657fa640 1 -- 172.21.15.88:0/3606843316 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916000a9d0 con 0x7f916815a1f0 2023-10-16T20:05:55.840 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.828+0000 7f916ed2a640 1 -- 172.21.15.88:0/3606843316 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9168157110 con 0x7f916815a1f0 2023-10-16T20:05:55.841 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.832+0000 7f916ed2a640 1 -- 172.21.15.88:0/3606843316 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91681506b0 con 0x7f916815a1f0 2023-10-16T20:05:55.841 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.832+0000 7f91657fa640 1 -- 172.21.15.88:0/3606843316 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f916000ab30 con 0x7f916815a1f0 2023-10-16T20:05:55.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.832+0000 7f91657fa640 1 --2- 172.21.15.88:0/3606843316 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f914c0664d0 0x7f914c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:55.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.832+0000 7f91657fa640 1 -- 172.21.15.88:0/3606843316 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9160083950 con 0x7f916815a1f0 2023-10-16T20:05:55.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.832+0000 7f916cac6640 1 --2- 172.21.15.88:0/3606843316 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f914c0664d0 0x7f914c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:55.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.832+0000 7f916cac6640 1 --2- 172.21.15.88:0/3606843316 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f914c0664d0 0x7f914c068990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f915c007a60 tx=0x7f915c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:55.843 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.836+0000 7f91657fa640 1 -- 172.21.15.88:0/3606843316 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f91681506b0 con 0x7f916815a1f0 2023-10-16T20:05:55.989 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.984+0000 7f916ed2a640 1 -- 172.21.15.88:0/3606843316 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f91681506b0 con 0x7f916815a1f0 2023-10-16T20:05:55.990 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.984+0000 7f91657fa640 1 -- 172.21.15.88:0/3606843316 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f91681506b0 con 0x7f916815a1f0 2023-10-16T20:05:55.990 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:05:55.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.988+0000 7f916ed2a640 1 -- 172.21.15.88:0/3606843316 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f914c0664d0 msgr2=0x7f914c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:55.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.988+0000 7f916ed2a640 1 --2- 172.21.15.88:0/3606843316 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f914c0664d0 0x7f914c068990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f915c007a60 tx=0x7f915c008040 comp rx=0 tx=0).stop 2023-10-16T20:05:55.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.988+0000 7f916ed2a640 1 -- 172.21.15.88:0/3606843316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f916815a1f0 msgr2=0x7f9168153eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:55.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.988+0000 7f916ed2a640 1 --2- 172.21.15.88:0/3606843316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f916815a1f0 0x7f9168153eb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f916000c780 tx=0x7f916000cc40 comp rx=0 tx=0).stop 2023-10-16T20:05:55.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.988+0000 7f916ed2a640 1 -- 172.21.15.88:0/3606843316 shutdown_connections 2023-10-16T20:05:55.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.988+0000 7f916ed2a640 1 --2- 172.21.15.88:0/3606843316 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f914c0664d0 0x7f914c068990 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:55.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.988+0000 7f916ed2a640 1 --2- 172.21.15.88:0/3606843316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f916815a1f0 0x7f9168153eb0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:55.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.988+0000 7f916ed2a640 1 --2- 172.21.15.88:0/3606843316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91680b4260 0x7f916815b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:55.994 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.988+0000 7f916ed2a640 1 --2- 172.21.15.88:0/3606843316 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f91680b0da0 0x7f916814f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:55.994 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.988+0000 7f916ed2a640 1 -- 172.21.15.88:0/3606843316 >> 172.21.15.88:0/3606843316 conn(0x7f91680cb080 msgr2=0x7f9168155fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:55.994 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.988+0000 7f916ed2a640 1 -- 172.21.15.88:0/3606843316 shutdown_connections 2023-10-16T20:05:55.994 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:55.988+0000 7f916ed2a640 1 -- 172.21.15.88:0/3606843316 wait complete. 2023-10-16T20:05:56.011 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T20:05:56.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.148+0000 7ff9c0205640 1 Processor -- start 2023-10-16T20:05:56.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.148+0000 7ff9c0205640 1 -- start start 2023-10-16T20:05:56.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.148+0000 7ff9c0205640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff9b8164570 0x7ff9b80af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:56.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.148+0000 7ff9c0205640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff9b8070250 0x7ff9b80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:56.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.148+0000 7ff9c0205640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b80ae840 con 0x7ff9b80afe20 2023-10-16T20:05:56.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.148+0000 7ff9c0205640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b8070c00 con 0x7ff9b8070250 2023-10-16T20:05:56.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.148+0000 7ff9c0205640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b80aeb20 con 0x7ff9b8164570 2023-10-16T20:05:56.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.148+0000 7ff9be7a2640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff9b8070250 0x7ff9b80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:56.155 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.148+0000 7ff9bd7a0640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7ff9b80afe20 0x7ff9b80addf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:53490/0 (socket says 172.21.15.88:53490) 2023-10-16T20:05:56.155 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.148+0000 7ff9bd7a0640 1 -- 172.21.15.88:0/1343485053 learned_addr learned my addr 172.21.15.88:0/1343485053 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:56.155 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.148+0000 7ff9bdfa1640 1 --2- 172.21.15.88:0/1343485053 >> v2:172.21.15.94:3300/0 conn(0x7ff9b8164570 0x7ff9b80af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:56.155 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.148+0000 7ff9be7a2640 1 -- 172.21.15.88:0/1343485053 >> v2:172.21.15.94:3300/0 conn(0x7ff9b8164570 msgr2=0x7ff9b80af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:56.155 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.148+0000 7ff9be7a2640 1 --2- 172.21.15.88:0/1343485053 >> v2:172.21.15.94:3300/0 conn(0x7ff9b8164570 0x7ff9b80af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:56.155 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.148+0000 7ff9be7a2640 1 -- 172.21.15.88:0/1343485053 >> v1:172.21.15.88:6789/0 conn(0x7ff9b80afe20 legacy=0x7ff9b80addf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:56.156 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.148+0000 7ff9be7a2640 1 -- 172.21.15.88:0/1343485053 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b80aee00 con 0x7ff9b8070250 2023-10-16T20:05:56.156 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.148+0000 7ff9bcf9f640 1 -- 172.21.15.88:0/1343485053 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1841791206 0 0) 0x7ff9b80ae840 con 0x7ff9b80afe20 2023-10-16T20:05:56.156 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.148+0000 7ff9bdfa1640 1 --2- 172.21.15.88:0/1343485053 >> v2:172.21.15.94:3300/0 conn(0x7ff9b8164570 0x7ff9b80af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:56.157 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.152+0000 7ff9be7a2640 1 --2- 172.21.15.88:0/1343485053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9b8070250 0x7ff9b80706c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff9b806e5f0 tx=0x7ff9b001f0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ede002be64ad4c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:56.158 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.152+0000 7ff9bcf9f640 1 -- 172.21.15.88:0/1343485053 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b00b0440 con 0x7ff9b8070250 2023-10-16T20:05:56.158 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.152+0000 7ff9bcf9f640 1 -- 172.21.15.88:0/1343485053 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b00b05a0 con 0x7ff9b8070250 2023-10-16T20:05:56.158 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.152+0000 7ff9bcf9f640 1 -- 172.21.15.88:0/1343485053 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b00b0930 con 0x7ff9b8070250 2023-10-16T20:05:56.158 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.152+0000 7ff9c0205640 1 -- 172.21.15.88:0/1343485053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9b8070250 msgr2=0x7ff9b80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:56.159 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.152+0000 7ff9c0205640 1 --2- 172.21.15.88:0/1343485053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9b8070250 0x7ff9b80706c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff9b806e5f0 tx=0x7ff9b001f0f0 comp rx=0 tx=0).stop 2023-10-16T20:05:56.159 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.152+0000 7ff9c0205640 1 -- 172.21.15.88:0/1343485053 shutdown_connections 2023-10-16T20:05:56.159 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.152+0000 7ff9c0205640 1 --2- 172.21.15.88:0/1343485053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9b8070250 0x7ff9b80706c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:56.159 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.152+0000 7ff9c0205640 1 --2- 172.21.15.88:0/1343485053 >> v2:172.21.15.94:3300/0 conn(0x7ff9b8164570 0x7ff9b80af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:56.159 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.152+0000 7ff9c0205640 1 -- 172.21.15.88:0/1343485053 >> 172.21.15.88:0/1343485053 conn(0x7ff9b80cb080 msgr2=0x7ff9b80cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:56.160 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.152+0000 7ff9c0205640 1 -- 172.21.15.88:0/1343485053 shutdown_connections 2023-10-16T20:05:56.160 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.152+0000 7ff9c0205640 1 -- 172.21.15.88:0/1343485053 wait complete. 2023-10-16T20:05:56.162 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.156+0000 7ff9c0205640 1 Processor -- start 2023-10-16T20:05:56.162 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.156+0000 7ff9c0205640 1 -- start start 2023-10-16T20:05:56.162 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.156+0000 7ff9c0205640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff9b8070250 0x7ff9b815e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:56.162 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.156+0000 7ff9c0205640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9b80afe20 0x7ff9b814ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:56.162 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.156+0000 7ff9c0205640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9b8164570 0x7ff9b815f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:56.163 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.156+0000 7ff9c0205640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b81492a0 con 0x7ff9b80afe20 2023-10-16T20:05:56.163 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.156+0000 7ff9c0205640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b8149e20 con 0x7ff9b8164570 2023-10-16T20:05:56.163 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.156+0000 7ff9c0205640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b815ee30 con 0x7ff9b8070250 2023-10-16T20:05:56.163 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.156+0000 7ff9bd7a0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9b80afe20 0x7ff9b814ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:56.163 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.156+0000 7ff9bd7a0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9b80afe20 0x7ff9b814ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50692/0 (socket says 172.21.15.88:50692) 2023-10-16T20:05:56.164 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.156+0000 7ff9bd7a0640 1 -- 172.21.15.88:0/3871149285 learned_addr learned my addr 172.21.15.88:0/3871149285 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:56.164 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.156+0000 7ff9be7a2640 1 --2- 172.21.15.88:0/3871149285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9b8164570 0x7ff9b815f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:56.164 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.156+0000 7ff9bdfa1640 1 --2- 172.21.15.88:0/3871149285 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff9b8070250 0x7ff9b815e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:56.164 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.156+0000 7ff9bd7a0640 1 -- 172.21.15.88:0/3871149285 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff9b8070250 msgr2=0x7ff9b815e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:56.164 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.156+0000 7ff9bd7a0640 1 --2- 172.21.15.88:0/3871149285 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff9b8070250 0x7ff9b815e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:56.165 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.156+0000 7ff9bd7a0640 1 -- 172.21.15.88:0/3871149285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9b8164570 msgr2=0x7ff9b815f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:56.165 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.156+0000 7ff9bd7a0640 1 --2- 172.21.15.88:0/3871149285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9b8164570 0x7ff9b815f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:56.165 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.156+0000 7ff9bd7a0640 1 -- 172.21.15.88:0/3871149285 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b0009a70 con 0x7ff9b80afe20 2023-10-16T20:05:56.165 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.156+0000 7ff9be7a2640 1 --2- 172.21.15.88:0/3871149285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9b8164570 0x7ff9b815f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:05:56.165 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.156+0000 7ff9bd7a0640 1 --2- 172.21.15.88:0/3871149285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9b80afe20 0x7ff9b814ee60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff9a800c550 tx=0x7ff9a800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:56.166 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.156+0000 7ff9ae7fc640 1 -- 172.21.15.88:0/3871149285 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a80074e0 con 0x7ff9b80afe20 2023-10-16T20:05:56.166 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.156+0000 7ff9ae7fc640 1 -- 172.21.15.88:0/3871149285 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a8007640 con 0x7ff9b80afe20 2023-10-16T20:05:56.166 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.156+0000 7ff9ae7fc640 1 -- 172.21.15.88:0/3871149285 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a80049f0 con 0x7ff9b80afe20 2023-10-16T20:05:56.166 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.156+0000 7ff9c0205640 1 -- 172.21.15.88:0/3871149285 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b81494f0 con 0x7ff9b80afe20 2023-10-16T20:05:56.166 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.156+0000 7ff9c0205640 1 -- 172.21.15.88:0/3871149285 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b8149ab0 con 0x7ff9b80afe20 2023-10-16T20:05:56.167 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.160+0000 7ff9c0205640 1 -- 172.21.15.88:0/3871149285 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b815be90 con 0x7ff9b80afe20 2023-10-16T20:05:56.167 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.160+0000 7ff9ae7fc640 1 -- 172.21.15.88:0/3871149285 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff9a8004b50 con 0x7ff9b80afe20 2023-10-16T20:05:56.168 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.160+0000 7ff9ae7fc640 1 --2- 172.21.15.88:0/3871149285 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff99c0664a0 0x7ff99c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:56.168 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.160+0000 7ff9ae7fc640 1 -- 172.21.15.88:0/3871149285 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff9a8083550 con 0x7ff9b80afe20 2023-10-16T20:05:56.168 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.160+0000 7ff9bdfa1640 1 --2- 172.21.15.88:0/3871149285 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff99c0664a0 0x7ff99c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:56.168 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.160+0000 7ff9bdfa1640 1 --2- 172.21.15.88:0/3871149285 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff99c0664a0 0x7ff99c068960 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4006e90 tx=0x7ff9b4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:56.169 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.164+0000 7ff9ae7fc640 1 -- 172.21.15.88:0/3871149285 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff9a8050620 con 0x7ff9b80afe20 2023-10-16T20:05:56.271 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T20:05:56.276 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T20:05:56.321 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.316+0000 7ff9c0205640 1 -- 172.21.15.88:0/3871149285 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff9b804e9c0 con 0x7ff9b80afe20 2023-10-16T20:05:56.322 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.316+0000 7ff9ae7fc640 1 -- 172.21.15.88:0/3871149285 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff9a8018020 con 0x7ff9b80afe20 2023-10-16T20:05:56.322 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:05:56.324 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.320+0000 7ff9c0205640 1 -- 172.21.15.88:0/3871149285 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff99c0664a0 msgr2=0x7ff99c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:56.324 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.320+0000 7ff9c0205640 1 --2- 172.21.15.88:0/3871149285 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff99c0664a0 0x7ff99c068960 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4006e90 tx=0x7ff9b4008040 comp rx=0 tx=0).stop 2023-10-16T20:05:56.324 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.320+0000 7ff9c0205640 1 -- 172.21.15.88:0/3871149285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9b80afe20 msgr2=0x7ff9b814ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:56.325 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.320+0000 7ff9c0205640 1 --2- 172.21.15.88:0/3871149285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9b80afe20 0x7ff9b814ee60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff9a800c550 tx=0x7ff9a800ca10 comp rx=0 tx=0).stop 2023-10-16T20:05:56.325 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.320+0000 7ff9c0205640 1 -- 172.21.15.88:0/3871149285 shutdown_connections 2023-10-16T20:05:56.325 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.320+0000 7ff9c0205640 1 --2- 172.21.15.88:0/3871149285 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff99c0664a0 0x7ff99c068960 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:56.325 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.320+0000 7ff9c0205640 1 --2- 172.21.15.88:0/3871149285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9b8164570 0x7ff9b815f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:56.326 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.320+0000 7ff9c0205640 1 --2- 172.21.15.88:0/3871149285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9b80afe20 0x7ff9b814ee60 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:56.326 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.320+0000 7ff9c0205640 1 --2- 172.21.15.88:0/3871149285 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff9b8070250 0x7ff9b815e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:56.326 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.320+0000 7ff9c0205640 1 -- 172.21.15.88:0/3871149285 >> 172.21.15.88:0/3871149285 conn(0x7ff9b80cb080 msgr2=0x7ff9b80cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:56.326 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.320+0000 7ff9c0205640 1 -- 172.21.15.88:0/3871149285 shutdown_connections 2023-10-16T20:05:56.326 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.320+0000 7ff9c0205640 1 -- 172.21.15.88:0/3871149285 wait complete. 2023-10-16T20:05:56.343 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T20:05:56.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.524+0000 7f867ad94640 1 Processor -- start 2023-10-16T20:05:56.531 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.528+0000 7f867ad94640 1 -- start start 2023-10-16T20:05:56.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.528+0000 7f867ad94640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8674154f30 0x7f86740b0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:56.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.528+0000 7f867ad94640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f86740b0a40 0x7f867415aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:56.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.528+0000 7f867ad94640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f867406d2f0 con 0x7f867415b530 2023-10-16T20:05:56.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.528+0000 7f867ad94640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b0ef0 con 0x7f86740b0a40 2023-10-16T20:05:56.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.528+0000 7f867ad94640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8674155310 con 0x7f8674154f30 2023-10-16T20:05:56.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.528+0000 7f866bfff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f86740b0a40 0x7f867415aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:56.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.528+0000 7f866bfff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f86740b0a40 0x7f867415aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50704/0 (socket says 172.21.15.88:50704) 2023-10-16T20:05:56.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.528+0000 7f866bfff640 1 -- 172.21.15.88:0/92430952 learned_addr learned my addr 172.21.15.88:0/92430952 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:56.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.528+0000 7f8679331640 1 --1- 172.21.15.88:0/92430952 >> v1:172.21.15.88:6789/0 conn(0x7f867415b530 0x7f86740b1610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:53502/0 (socket says 172.21.15.88:53502) 2023-10-16T20:05:56.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.528+0000 7f8678b30640 1 --2- 172.21.15.88:0/92430952 >> v2:172.21.15.94:3300/0 conn(0x7f8674154f30 0x7f86740b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:56.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.528+0000 7f866bfff640 1 -- 172.21.15.88:0/92430952 >> v2:172.21.15.94:3300/0 conn(0x7f8674154f30 msgr2=0x7f86740b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:56.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.528+0000 7f866bfff640 1 --2- 172.21.15.88:0/92430952 >> v2:172.21.15.94:3300/0 conn(0x7f8674154f30 0x7f86740b0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:56.535 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.528+0000 7f866bfff640 1 -- 172.21.15.88:0/92430952 >> v1:172.21.15.88:6789/0 conn(0x7f867415b530 legacy=0x7f86740b1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:56.535 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.528+0000 7f866bfff640 1 -- 172.21.15.88:0/92430952 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867415bd20 con 0x7f86740b0a40 2023-10-16T20:05:56.535 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.528+0000 7f866b7fe640 1 -- 172.21.15.88:0/92430952 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1484099635 0 0) 0x7f867406d2f0 con 0x7f867415b530 2023-10-16T20:05:56.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.532+0000 7f866bfff640 1 --2- 172.21.15.88:0/92430952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86740b0a40 0x7f867415aff0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f86740ce250 tx=0x7f866401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b982c6e10b956b84 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:56.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.532+0000 7f866b7fe640 1 -- 172.21.15.88:0/92430952 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86640ab5f0 con 0x7f86740b0a40 2023-10-16T20:05:56.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.532+0000 7f866b7fe640 1 -- 172.21.15.88:0/92430952 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86640ab750 con 0x7f86740b0a40 2023-10-16T20:05:56.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.532+0000 7f866b7fe640 1 -- 172.21.15.88:0/92430952 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86640abae0 con 0x7f86740b0a40 2023-10-16T20:05:56.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.532+0000 7f867ad94640 1 -- 172.21.15.88:0/92430952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86740b0a40 msgr2=0x7f867415aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:56.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.532+0000 7f867ad94640 1 --2- 172.21.15.88:0/92430952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86740b0a40 0x7f867415aff0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f86740ce250 tx=0x7f866401c4d0 comp rx=0 tx=0).stop 2023-10-16T20:05:56.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.532+0000 7f867ad94640 1 -- 172.21.15.88:0/92430952 shutdown_connections 2023-10-16T20:05:56.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.532+0000 7f867ad94640 1 --2- 172.21.15.88:0/92430952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86740b0a40 0x7f867415aff0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:56.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.532+0000 7f867ad94640 1 --2- 172.21.15.88:0/92430952 >> v2:172.21.15.94:3300/0 conn(0x7f8674154f30 0x7f86740b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:56.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.532+0000 7f867ad94640 1 -- 172.21.15.88:0/92430952 >> 172.21.15.88:0/92430952 conn(0x7f86740cb080 msgr2=0x7f86740cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:56.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.532+0000 7f867ad94640 1 -- 172.21.15.88:0/92430952 shutdown_connections 2023-10-16T20:05:56.539 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.532+0000 7f867ad94640 1 -- 172.21.15.88:0/92430952 wait complete. 2023-10-16T20:05:56.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.532+0000 7f867ad94640 1 Processor -- start 2023-10-16T20:05:56.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.532+0000 7f867ad94640 1 -- start start 2023-10-16T20:05:56.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.536+0000 7f867ad94640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86740b0a40 0x7f86741f4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:56.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.536+0000 7f867ad94640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8674154f30 0x7f86740a7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:56.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.536+0000 7f867ad94640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f867415b530 0x7f86740a79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:56.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.536+0000 7f867ad94640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86741f51b0 con 0x7f86740b0a40 2023-10-16T20:05:56.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.536+0000 7f867ad94640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86741f5490 con 0x7f867415b530 2023-10-16T20:05:56.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.536+0000 7f867ad94640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86741f5770 con 0x7f8674154f30 2023-10-16T20:05:56.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.536+0000 7f8678b30640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86740b0a40 0x7f86741f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:56.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.536+0000 7f8678b30640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86740b0a40 0x7f86741f4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50712/0 (socket says 172.21.15.88:50712) 2023-10-16T20:05:56.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.536+0000 7f8678b30640 1 -- 172.21.15.88:0/953234626 learned_addr learned my addr 172.21.15.88:0/953234626 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:56.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.536+0000 7f866bfff640 1 --2- 172.21.15.88:0/953234626 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8674154f30 0x7f86740a7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:56.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.536+0000 7f8679331640 1 --2- 172.21.15.88:0/953234626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f867415b530 0x7f86740a79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:56.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.536+0000 7f8678b30640 1 -- 172.21.15.88:0/953234626 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8674154f30 msgr2=0x7f86740a7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:56.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.536+0000 7f8678b30640 1 --2- 172.21.15.88:0/953234626 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8674154f30 0x7f86740a7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:56.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.536+0000 7f8678b30640 1 -- 172.21.15.88:0/953234626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f867415b530 msgr2=0x7f86740a79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:56.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.536+0000 7f8678b30640 1 --2- 172.21.15.88:0/953234626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f867415b530 0x7f86740a79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:56.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.536+0000 7f8678b30640 1 -- 172.21.15.88:0/953234626 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86640094d0 con 0x7f86740b0a40 2023-10-16T20:05:56.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.536+0000 7f8679331640 1 --2- 172.21.15.88:0/953234626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f867415b530 0x7f86740a79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:56.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.536+0000 7f8678b30640 1 --2- 172.21.15.88:0/953234626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86740b0a40 0x7f86741f4760 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f867000d5f0 tx=0x7f867000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:56.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.536+0000 7f86697fa640 1 -- 172.21.15.88:0/953234626 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86700125f0 con 0x7f86740b0a40 2023-10-16T20:05:56.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.536+0000 7f86697fa640 1 -- 172.21.15.88:0/953234626 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8670012750 con 0x7f86740b0a40 2023-10-16T20:05:56.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.536+0000 7f86697fa640 1 -- 172.21.15.88:0/953234626 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8670012ae0 con 0x7f86740b0a40 2023-10-16T20:05:56.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.536+0000 7f867ad94640 1 -- 172.21.15.88:0/953234626 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86741f5a50 con 0x7f86740b0a40 2023-10-16T20:05:56.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.536+0000 7f867ad94640 1 -- 172.21.15.88:0/953234626 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f867415ce70 con 0x7f86740b0a40 2023-10-16T20:05:56.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.536+0000 7f867ad94640 1 -- 172.21.15.88:0/953234626 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f867404e9c0 con 0x7f86740b0a40 2023-10-16T20:05:56.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.540+0000 7f86697fa640 1 -- 172.21.15.88:0/953234626 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8670012c40 con 0x7f86740b0a40 2023-10-16T20:05:56.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.540+0000 7f86697fa640 1 --2- 172.21.15.88:0/953234626 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f86600664a0 0x7f8660068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:56.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.540+0000 7f86697fa640 1 -- 172.21.15.88:0/953234626 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8670084780 con 0x7f86740b0a40 2023-10-16T20:05:56.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.540+0000 7f866bfff640 1 --2- 172.21.15.88:0/953234626 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f86600664a0 0x7f8660068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:56.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.540+0000 7f866bfff640 1 --2- 172.21.15.88:0/953234626 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f86600664a0 0x7f8660068960 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f86640af000 tx=0x7f86640b6040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:56.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.544+0000 7f86697fa640 1 -- 172.21.15.88:0/953234626 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f86700517a0 con 0x7f86740b0a40 2023-10-16T20:05:56.695 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.692+0000 7f867ad94640 1 -- 172.21.15.88:0/953234626 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f86740b4540 con 0x7f86740b0a40 2023-10-16T20:05:56.696 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.692+0000 7f86697fa640 1 -- 172.21.15.88:0/953234626 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8670007380 con 0x7f86740b0a40 2023-10-16T20:05:56.696 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2023-10-16T20:05:56.698 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.692+0000 7f867ad94640 1 -- 172.21.15.88:0/953234626 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f86600664a0 msgr2=0x7f8660068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:56.698 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.692+0000 7f867ad94640 1 --2- 172.21.15.88:0/953234626 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f86600664a0 0x7f8660068960 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f86640af000 tx=0x7f86640b6040 comp rx=0 tx=0).stop 2023-10-16T20:05:56.698 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.692+0000 7f867ad94640 1 -- 172.21.15.88:0/953234626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86740b0a40 msgr2=0x7f86741f4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:56.698 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.692+0000 7f867ad94640 1 --2- 172.21.15.88:0/953234626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86740b0a40 0x7f86741f4760 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f867000d5f0 tx=0x7f867000dab0 comp rx=0 tx=0).stop 2023-10-16T20:05:56.699 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.692+0000 7f867ad94640 1 -- 172.21.15.88:0/953234626 shutdown_connections 2023-10-16T20:05:56.699 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.692+0000 7f867ad94640 1 --2- 172.21.15.88:0/953234626 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f86600664a0 0x7f8660068960 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:56.699 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.692+0000 7f867ad94640 1 --2- 172.21.15.88:0/953234626 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8674154f30 0x7f86740a7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:56.699 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.692+0000 7f867ad94640 1 --2- 172.21.15.88:0/953234626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f867415b530 0x7f86740a79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:56.699 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.692+0000 7f867ad94640 1 --2- 172.21.15.88:0/953234626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86740b0a40 0x7f86741f4760 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:56.700 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.692+0000 7f867ad94640 1 -- 172.21.15.88:0/953234626 >> 172.21.15.88:0/953234626 conn(0x7f86740cb080 msgr2=0x7f86740b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:56.700 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.692+0000 7f867ad94640 1 -- 172.21.15.88:0/953234626 shutdown_connections 2023-10-16T20:05:56.700 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.692+0000 7f867ad94640 1 -- 172.21.15.88:0/953234626 wait complete. 2023-10-16T20:05:56.716 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T20:05:56.857 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.852+0000 7f851e2bb640 1 Processor -- start 2023-10-16T20:05:56.857 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.852+0000 7f851e2bb640 1 -- start start 2023-10-16T20:05:56.858 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.852+0000 7f851e2bb640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8518159f50 0x7f851806c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:56.858 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.852+0000 7f851e2bb640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f851815a400 con 0x7f85180711f0 2023-10-16T20:05:56.858 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.852+0000 7f851e2bb640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f851815aa10 con 0x7f85181500a0 2023-10-16T20:05:56.859 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.852+0000 7f851e2bb640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8518150480 con 0x7f8518159f50 2023-10-16T20:05:56.859 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.852+0000 7f85177fe640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f85180711f0 0x7f851815aef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:53516/0 (socket says 172.21.15.88:53516) 2023-10-16T20:05:56.859 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.852+0000 7f85177fe640 1 -- 172.21.15.88:0/323944001 learned_addr learned my addr 172.21.15.88:0/323944001 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:56.861 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.856+0000 7f851c858640 1 --2- 172.21.15.88:0/323944001 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8518159f50 0x7f851806c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:56.861 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.856+0000 7f8516ffd640 1 -- 172.21.15.88:0/323944001 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1538291121 0 0) 0x7f851815a400 con 0x7f85180711f0 2023-10-16T20:05:56.862 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.856+0000 7f8516ffd640 1 -- 172.21.15.88:0/323944001 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8500005740 con 0x7f85180711f0 2023-10-16T20:05:56.862 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.856+0000 7f8516ffd640 1 -- 172.21.15.88:0/323944001 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2700662898 0 0) 0x7f8500005740 con 0x7f85180711f0 2023-10-16T20:05:56.862 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.856+0000 7f8516ffd640 1 -- 172.21.15.88:0/323944001 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8518159f50 msgr2=0x7f851806c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:56.862 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.856+0000 7f8516ffd640 1 --2- 172.21.15.88:0/323944001 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8518159f50 0x7f851806c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:56.863 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.856+0000 7f8516ffd640 1 -- 172.21.15.88:0/323944001 >> v1:172.21.15.94:6789/0 conn(0x7f85181500a0 legacy=0x7f8518070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:56.863 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.856+0000 7f8516ffd640 1 -- 172.21.15.88:0/323944001 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851806ca10 con 0x7f85180711f0 2023-10-16T20:05:56.863 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.856+0000 7f8516ffd640 1 -- 172.21.15.88:0/323944001 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f850c001680 con 0x7f85180711f0 2023-10-16T20:05:56.863 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.856+0000 7f8516ffd640 1 -- 172.21.15.88:0/323944001 >> v1:172.21.15.88:6789/0 conn(0x7f85180711f0 legacy=0x7f851815aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:56.863 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.856+0000 7f8516ffd640 1 --2- 172.21.15.88:0/323944001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8500005bc0 0x7f8500005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:56.864 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.856+0000 7f8516ffd640 1 --2- 172.21.15.88:0/323944001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85000069e0 0x7f8500006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:56.864 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.856+0000 7f8516ffd640 1 --2- 172.21.15.88:0/323944001 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8518159f50 0x7f85000073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:56.864 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.856+0000 7f8516ffd640 1 -- 172.21.15.88:0/323944001 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8518150480 con 0x7f8500005bc0 2023-10-16T20:05:56.864 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.856+0000 7f8516ffd640 1 -- 172.21.15.88:0/323944001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850000dc50 con 0x7f85000069e0 2023-10-16T20:05:56.864 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.856+0000 7f8516ffd640 1 -- 172.21.15.88:0/323944001 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850000de10 con 0x7f8518159f50 2023-10-16T20:05:56.865 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.856+0000 7f8517fff640 1 --2- 172.21.15.88:0/323944001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8500005bc0 0x7f8500005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:56.865 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.856+0000 7f85177fe640 1 --2- 172.21.15.88:0/323944001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85000069e0 0x7f8500006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:56.865 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.856+0000 7f851c858640 1 --2- 172.21.15.88:0/323944001 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8518159f50 0x7f85000073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:56.865 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.856+0000 7f8517fff640 1 -- 172.21.15.88:0/323944001 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8518159f50 msgr2=0x7f85000073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:56.865 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.856+0000 7f8517fff640 1 --2- 172.21.15.88:0/323944001 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8518159f50 0x7f85000073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:56.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.856+0000 7f8517fff640 1 -- 172.21.15.88:0/323944001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85000069e0 msgr2=0x7f8500006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:56.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.856+0000 7f8517fff640 1 --2- 172.21.15.88:0/323944001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85000069e0 0x7f8500006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:56.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.856+0000 7f8517fff640 1 -- 172.21.15.88:0/323944001 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8500005740 con 0x7f8500005bc0 2023-10-16T20:05:56.869 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.860+0000 7f8517fff640 1 --2- 172.21.15.88:0/323944001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8500005bc0 0x7f8500005fe0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8508001750 tx=0x7f850801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88346d4533143c33 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:56.869 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.860+0000 7f8516ffd640 1 -- 172.21.15.88:0/323944001 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85080052d0 con 0x7f8500005bc0 2023-10-16T20:05:56.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.860+0000 7f8516ffd640 1 -- 172.21.15.88:0/323944001 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85080a7050 con 0x7f8500005bc0 2023-10-16T20:05:56.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.860+0000 7f8516ffd640 1 -- 172.21.15.88:0/323944001 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8508003600 con 0x7f8500005bc0 2023-10-16T20:05:56.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.860+0000 7f851e2bb640 1 -- 172.21.15.88:0/323944001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8500005bc0 msgr2=0x7f8500005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:56.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.860+0000 7f851e2bb640 1 --2- 172.21.15.88:0/323944001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8500005bc0 0x7f8500005fe0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8508001750 tx=0x7f850801c3a0 comp rx=0 tx=0).stop 2023-10-16T20:05:56.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.860+0000 7f851c858640 1 -- 172.21.15.88:0/323944001 reap_dead start 2023-10-16T20:05:56.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.860+0000 7f851e2bb640 1 -- 172.21.15.88:0/323944001 shutdown_connections 2023-10-16T20:05:56.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.860+0000 7f851e2bb640 1 -- 172.21.15.88:0/323944001 >> 172.21.15.88:0/323944001 conn(0x7f85180cb080 msgr2=0x7f85180cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:56.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.860+0000 7f851e2bb640 1 -- 172.21.15.88:0/323944001 shutdown_connections 2023-10-16T20:05:56.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.860+0000 7f851e2bb640 1 -- 172.21.15.88:0/323944001 wait complete. 2023-10-16T20:05:56.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.860+0000 7f851e2bb640 1 Processor -- start 2023-10-16T20:05:56.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.860+0000 7f851e2bb640 1 -- start start 2023-10-16T20:05:56.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.864+0000 7f851e2bb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8500005bc0 0x7f85181565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:56.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.864+0000 7f851e2bb640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8518146d70 0x7f85181f8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:56.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.864+0000 7f851e2bb640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85180b20b0 0x7f85181478e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:56.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.864+0000 7f851e2bb640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85181507d0 con 0x7f85180b20b0 2023-10-16T20:05:56.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.864+0000 7f851e2bb640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8518160b90 con 0x7f8500005bc0 2023-10-16T20:05:56.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.864+0000 7f851e2bb640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85181471f0 con 0x7f8518146d70 2023-10-16T20:05:56.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.864+0000 7f8517fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8500005bc0 0x7f85181565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:56.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.864+0000 7f851c858640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85180b20b0 0x7f85181478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:56.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.864+0000 7f8517fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8500005bc0 0x7f85181565f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:56954/0 (socket says 172.21.15.88:56954) 2023-10-16T20:05:56.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.864+0000 7f8517fff640 1 -- 172.21.15.88:0/627949050 learned_addr learned my addr 172.21.15.88:0/627949050 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:56.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.864+0000 7f85177fe640 1 --2- 172.21.15.88:0/627949050 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8518146d70 0x7f85181f8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:56.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.864+0000 7f851c858640 1 -- 172.21.15.88:0/627949050 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8518146d70 msgr2=0x7f85181f8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:56.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.864+0000 7f851c858640 1 --2- 172.21.15.88:0/627949050 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8518146d70 0x7f85181f8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:56.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.864+0000 7f851c858640 1 -- 172.21.15.88:0/627949050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8500005bc0 msgr2=0x7f85181565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:56.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.864+0000 7f851c858640 1 --2- 172.21.15.88:0/627949050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8500005bc0 0x7f85181565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:56.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.864+0000 7f851c858640 1 -- 172.21.15.88:0/627949050 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85181f9260 con 0x7f85180b20b0 2023-10-16T20:05:56.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.864+0000 7f851c858640 1 --2- 172.21.15.88:0/627949050 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85180b20b0 0x7f85181478e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f851806f930 tx=0x7f851000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:56.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.864+0000 7f8514ff9640 1 -- 172.21.15.88:0/627949050 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851000e040 con 0x7f85180b20b0 2023-10-16T20:05:56.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.864+0000 7f8514ff9640 1 -- 172.21.15.88:0/627949050 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851000f040 con 0x7f85180b20b0 2023-10-16T20:05:56.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.864+0000 7f851e2bb640 1 -- 172.21.15.88:0/627949050 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85181f9540 con 0x7f85180b20b0 2023-10-16T20:05:56.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.864+0000 7f8514ff9640 1 -- 172.21.15.88:0/627949050 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510015670 con 0x7f85180b20b0 2023-10-16T20:05:56.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.864+0000 7f851e2bb640 1 -- 172.21.15.88:0/627949050 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8518160e30 con 0x7f85180b20b0 2023-10-16T20:05:56.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.864+0000 7f851e2bb640 1 -- 172.21.15.88:0/627949050 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8518161020 con 0x7f85180b20b0 2023-10-16T20:05:56.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.864+0000 7f8514ff9640 1 -- 172.21.15.88:0/627949050 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f85100157d0 con 0x7f85180b20b0 2023-10-16T20:05:56.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.868+0000 7f8514ff9640 1 --2- 172.21.15.88:0/627949050 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f85040664d0 0x7f8504068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:56.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.868+0000 7f8514ff9640 1 -- 172.21.15.88:0/627949050 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8510084850 con 0x7f85180b20b0 2023-10-16T20:05:56.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.868+0000 7f8517fff640 1 --2- 172.21.15.88:0/627949050 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f85040664d0 0x7f8504068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:56.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.868+0000 7f8517fff640 1 --2- 172.21.15.88:0/627949050 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f85040664d0 0x7f8504068990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8518162210 tx=0x7f85080011a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:56.878 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:56.872+0000 7f8514ff9640 1 -- 172.21.15.88:0/627949050 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8510051920 con 0x7f85180b20b0 2023-10-16T20:05:57.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.016+0000 7f851e2bb640 1 -- 172.21.15.88:0/627949050 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f851804e9c0 con 0x7f85180b20b0 2023-10-16T20:05:57.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.016+0000 7f8514ff9640 1 -- 172.21.15.88:0/627949050 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8510054f40 con 0x7f85180b20b0 2023-10-16T20:05:57.022 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:05:57.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.020+0000 7f851e2bb640 1 -- 172.21.15.88:0/627949050 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f85040664d0 msgr2=0x7f8504068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:57.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.020+0000 7f851e2bb640 1 --2- 172.21.15.88:0/627949050 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f85040664d0 0x7f8504068990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8518162210 tx=0x7f85080011a0 comp rx=0 tx=0).stop 2023-10-16T20:05:57.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.020+0000 7f851e2bb640 1 -- 172.21.15.88:0/627949050 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85180b20b0 msgr2=0x7f85181478e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:57.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.020+0000 7f851e2bb640 1 --2- 172.21.15.88:0/627949050 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85180b20b0 0x7f85181478e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f851806f930 tx=0x7f851000f580 comp rx=0 tx=0).stop 2023-10-16T20:05:57.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.020+0000 7f851e2bb640 1 -- 172.21.15.88:0/627949050 shutdown_connections 2023-10-16T20:05:57.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.020+0000 7f851e2bb640 1 --2- 172.21.15.88:0/627949050 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f85040664d0 0x7f8504068990 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:57.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.020+0000 7f851e2bb640 1 --2- 172.21.15.88:0/627949050 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8518146d70 0x7f85181f8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:57.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.020+0000 7f851e2bb640 1 --2- 172.21.15.88:0/627949050 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85180b20b0 0x7f85181478e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:57.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.020+0000 7f851e2bb640 1 --2- 172.21.15.88:0/627949050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8500005bc0 0x7f85181565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:57.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.020+0000 7f851e2bb640 1 -- 172.21.15.88:0/627949050 >> 172.21.15.88:0/627949050 conn(0x7f85180cb080 msgr2=0x7f85180cc170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:57.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.020+0000 7f851e2bb640 1 -- 172.21.15.88:0/627949050 shutdown_connections 2023-10-16T20:05:57.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.020+0000 7f851e2bb640 1 -- 172.21.15.88:0/627949050 wait complete. 2023-10-16T20:05:57.043 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T20:05:57.181 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.176+0000 7f13b2194640 1 Processor -- start 2023-10-16T20:05:57.182 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.176+0000 7f13b2194640 1 -- start start 2023-10-16T20:05:57.183 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.180+0000 7f13b2194640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f13ac0cd910 0x7f13ac15a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:57.184 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.180+0000 7f13b2194640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f13ac0af610 0x7f13ac0afc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:57.184 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.180+0000 7f13b2194640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f13ac0b01a0 0x7f13ac15cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:57.184 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.180+0000 7f13b2194640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13ac0b0600 con 0x7f13ac0af610 2023-10-16T20:05:57.185 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.180+0000 7f13b2194640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13ac15a630 con 0x7f13ac0cd910 2023-10-16T20:05:57.185 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.180+0000 7f13b2194640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac0cdcf0 con 0x7f13ac0b01a0 2023-10-16T20:05:57.185 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.180+0000 7f13aaffd640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f13ac0af610 0x7f13ac0afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:57.185 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.180+0000 7f13aaffd640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f13ac0af610 0x7f13ac0afc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50740/0 (socket says 172.21.15.88:50740) 2023-10-16T20:05:57.185 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.180+0000 7f13aaffd640 1 -- 172.21.15.88:0/2504943777 learned_addr learned my addr 172.21.15.88:0/2504943777 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:57.186 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.180+0000 7f13ab7fe640 1 --2- 172.21.15.88:0/2504943777 >> v2:172.21.15.94:3300/0 conn(0x7f13ac0cd910 0x7f13ac15a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:57.186 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.180+0000 7f13abfff640 1 --2- 172.21.15.88:0/2504943777 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f13ac0b01a0 0x7f13ac15cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:57.186 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.180+0000 7f13aaffd640 1 -- 172.21.15.88:0/2504943777 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f13ac0b01a0 msgr2=0x7f13ac15cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:57.186 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.180+0000 7f13aaffd640 1 --2- 172.21.15.88:0/2504943777 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f13ac0b01a0 0x7f13ac15cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:57.187 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.180+0000 7f13aaffd640 1 -- 172.21.15.88:0/2504943777 >> v2:172.21.15.94:3300/0 conn(0x7f13ac0cd910 msgr2=0x7f13ac15a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:57.187 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.180+0000 7f13aaffd640 1 --2- 172.21.15.88:0/2504943777 >> v2:172.21.15.94:3300/0 conn(0x7f13ac0cd910 0x7f13ac15a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:57.187 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.180+0000 7f13aaffd640 1 -- 172.21.15.88:0/2504943777 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ac0ae570 con 0x7f13ac0af610 2023-10-16T20:05:57.187 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.180+0000 7f13abfff640 1 --2- 172.21.15.88:0/2504943777 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f13ac0b01a0 0x7f13ac15cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:57.189 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.180+0000 7f13aaffd640 1 --2- 172.21.15.88:0/2504943777 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13ac0af610 0x7f13ac0afc60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f13ac0723d0 tx=0x7f13a001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f0d34821be519f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:57.189 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.184+0000 7f13aa7fc640 1 -- 172.21.15.88:0/2504943777 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a00ad680 con 0x7f13ac0af610 2023-10-16T20:05:57.189 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.184+0000 7f13aa7fc640 1 -- 172.21.15.88:0/2504943777 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a00ad7e0 con 0x7f13ac0af610 2023-10-16T20:05:57.189 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.184+0000 7f13aa7fc640 1 -- 172.21.15.88:0/2504943777 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a00adb70 con 0x7f13ac0af610 2023-10-16T20:05:57.190 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.184+0000 7f13b2194640 1 -- 172.21.15.88:0/2504943777 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13ac0af610 msgr2=0x7f13ac0afc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:57.190 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.184+0000 7f13b2194640 1 --2- 172.21.15.88:0/2504943777 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13ac0af610 0x7f13ac0afc60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f13ac0723d0 tx=0x7f13a001c440 comp rx=0 tx=0).stop 2023-10-16T20:05:57.190 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.184+0000 7f13b2194640 1 -- 172.21.15.88:0/2504943777 shutdown_connections 2023-10-16T20:05:57.190 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.184+0000 7f13b2194640 1 --2- 172.21.15.88:0/2504943777 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f13ac0b01a0 0x7f13ac15cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:57.191 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.184+0000 7f13b2194640 1 --2- 172.21.15.88:0/2504943777 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13ac0af610 0x7f13ac0afc60 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:57.191 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.184+0000 7f13b2194640 1 --2- 172.21.15.88:0/2504943777 >> v2:172.21.15.94:3300/0 conn(0x7f13ac0cd910 0x7f13ac15a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:57.191 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.184+0000 7f13b2194640 1 -- 172.21.15.88:0/2504943777 >> 172.21.15.88:0/2504943777 conn(0x7f13ac0ce5e0 msgr2=0x7f13ac0ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:57.191 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.184+0000 7f13b2194640 1 -- 172.21.15.88:0/2504943777 shutdown_connections 2023-10-16T20:05:57.191 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.184+0000 7f13b2194640 1 -- 172.21.15.88:0/2504943777 wait complete. 2023-10-16T20:05:57.192 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.184+0000 7f13b2194640 1 Processor -- start 2023-10-16T20:05:57.192 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.184+0000 7f13b2194640 1 -- start start 2023-10-16T20:05:57.194 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.184+0000 7f13b2194640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f13ac0af610 0x7f13ac153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:57.195 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.184+0000 7f13b2194640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13ac0b01a0 0x7f13ac135b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:57.195 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.184+0000 7f13b2194640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13ac0cd910 0x7f13ac1360b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:57.195 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.184+0000 7f13b2194640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac14dce0 con 0x7f13ac0cd910 2023-10-16T20:05:57.195 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.184+0000 7f13b2194640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac14a950 con 0x7f13ac0b01a0 2023-10-16T20:05:57.196 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.184+0000 7f13b2194640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac14ac30 con 0x7f13ac0af610 2023-10-16T20:05:57.196 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.184+0000 7f13abfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13ac0cd910 0x7f13ac1360b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:57.196 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.184+0000 7f13abfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13ac0cd910 0x7f13ac1360b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50750/0 (socket says 172.21.15.88:50750) 2023-10-16T20:05:57.196 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.188+0000 7f13abfff640 1 -- 172.21.15.88:0/2409592520 learned_addr learned my addr 172.21.15.88:0/2409592520 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:57.196 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.188+0000 7f13ab7fe640 1 --2- 172.21.15.88:0/2409592520 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f13ac0af610 0x7f13ac153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:57.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.188+0000 7f13aaffd640 1 --2- 172.21.15.88:0/2409592520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13ac0b01a0 0x7f13ac135b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:57.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.188+0000 7f13abfff640 1 -- 172.21.15.88:0/2409592520 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f13ac0af610 msgr2=0x7f13ac153c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:57.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.188+0000 7f13abfff640 1 --2- 172.21.15.88:0/2409592520 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f13ac0af610 0x7f13ac153c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:57.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.188+0000 7f13abfff640 1 -- 172.21.15.88:0/2409592520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13ac0b01a0 msgr2=0x7f13ac135b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:57.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.188+0000 7f13abfff640 1 --2- 172.21.15.88:0/2409592520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13ac0b01a0 0x7f13ac135b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:57.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.188+0000 7f13abfff640 1 -- 172.21.15.88:0/2409592520 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a00094d0 con 0x7f13ac0cd910 2023-10-16T20:05:57.198 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.188+0000 7f13abfff640 1 --2- 172.21.15.88:0/2409592520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13ac0cd910 0x7f13ac1360b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f13a400eef0 tx=0x7f13a400c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:57.198 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.188+0000 7f1393fff640 1 -- 172.21.15.88:0/2409592520 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a400f840 con 0x7f13ac0cd910 2023-10-16T20:05:57.198 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.188+0000 7f1393fff640 1 -- 172.21.15.88:0/2409592520 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a400f9a0 con 0x7f13ac0cd910 2023-10-16T20:05:57.198 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.188+0000 7f1393fff640 1 -- 172.21.15.88:0/2409592520 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a400fd50 con 0x7f13ac0cd910 2023-10-16T20:05:57.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.188+0000 7f13b2194640 1 -- 172.21.15.88:0/2409592520 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13ac14af10 con 0x7f13ac0cd910 2023-10-16T20:05:57.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.188+0000 7f13b2194640 1 -- 172.21.15.88:0/2409592520 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13ac14b4d0 con 0x7f13ac0cd910 2023-10-16T20:05:57.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.188+0000 7f13b2194640 1 -- 172.21.15.88:0/2409592520 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13ac06d180 con 0x7f13ac0cd910 2023-10-16T20:05:57.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.188+0000 7f1393fff640 1 -- 172.21.15.88:0/2409592520 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f13a40203b0 con 0x7f13ac0cd910 2023-10-16T20:05:57.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.188+0000 7f1393fff640 1 --2- 172.21.15.88:0/2409592520 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f13980664a0 0x7f1398068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:57.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.188+0000 7f1393fff640 1 -- 172.21.15.88:0/2409592520 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f13a4086ba0 con 0x7f13ac0cd910 2023-10-16T20:05:57.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.188+0000 7f13ab7fe640 1 --2- 172.21.15.88:0/2409592520 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f13980664a0 0x7f1398068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:57.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.192+0000 7f13ab7fe640 1 --2- 172.21.15.88:0/2409592520 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f13980664a0 0x7f1398068960 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f139c0079c0 tx=0x7f139c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:57.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.192+0000 7f1393fff640 1 -- 172.21.15.88:0/2409592520 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f13a4053d00 con 0x7f13ac0cd910 2023-10-16T20:05:57.361 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.356+0000 7f13b2194640 1 -- 172.21.15.88:0/2409592520 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f13ac04e9c0 con 0x7f13ac0cd910 2023-10-16T20:05:57.361 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.356+0000 7f1393fff640 1 -- 172.21.15.88:0/2409592520 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7f13a401c020 con 0x7f13ac0cd910 2023-10-16T20:05:57.362 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2023-10-16T20:05:57.363 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.360+0000 7f13b2194640 1 -- 172.21.15.88:0/2409592520 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f13980664a0 msgr2=0x7f1398068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:57.364 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.360+0000 7f13b2194640 1 --2- 172.21.15.88:0/2409592520 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f13980664a0 0x7f1398068960 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f139c0079c0 tx=0x7f139c008040 comp rx=0 tx=0).stop 2023-10-16T20:05:57.364 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.360+0000 7f13b2194640 1 -- 172.21.15.88:0/2409592520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13ac0cd910 msgr2=0x7f13ac1360b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:57.364 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.360+0000 7f13b2194640 1 --2- 172.21.15.88:0/2409592520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13ac0cd910 0x7f13ac1360b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f13a400eef0 tx=0x7f13a400c710 comp rx=0 tx=0).stop 2023-10-16T20:05:57.364 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.360+0000 7f13b2194640 1 -- 172.21.15.88:0/2409592520 shutdown_connections 2023-10-16T20:05:57.364 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.360+0000 7f13b2194640 1 --2- 172.21.15.88:0/2409592520 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f13980664a0 0x7f1398068960 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:57.365 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.360+0000 7f13b2194640 1 --2- 172.21.15.88:0/2409592520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13ac0cd910 0x7f13ac1360b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:57.365 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.360+0000 7f13b2194640 1 --2- 172.21.15.88:0/2409592520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13ac0b01a0 0x7f13ac135b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:57.365 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.360+0000 7f13b2194640 1 --2- 172.21.15.88:0/2409592520 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f13ac0af610 0x7f13ac153c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:57.365 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.360+0000 7f13b2194640 1 -- 172.21.15.88:0/2409592520 >> 172.21.15.88:0/2409592520 conn(0x7f13ac0ce5e0 msgr2=0x7f13ac159cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:57.365 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.360+0000 7f13b2194640 1 -- 172.21.15.88:0/2409592520 shutdown_connections 2023-10-16T20:05:57.365 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.360+0000 7f13b2194640 1 -- 172.21.15.88:0/2409592520 wait complete. 2023-10-16T20:05:57.383 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi088.front.sepia.ceph.com... 2023-10-16T20:05:57.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:57.383 DEBUG:teuthology.orchestra.run.smithi088:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-10-16T20:05:57.405 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:05:57.405 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2023-10-16T20:05:57.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:05:57.406 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-16T20:05:57.455 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:05:57.456 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:05:57.466 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:05:57.467 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:05:57.516 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:05:57.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.652+0000 7f9800be4640 1 Processor -- start 2023-10-16T20:05:57.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.652+0000 7f9800be4640 1 -- start start 2023-10-16T20:05:57.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.652+0000 7f9800be4640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f97fc0cde80 0x7f97fc0d3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:57.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.652+0000 7f9800be4640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f97fc0cf230 0x7f97fc0cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:57.657 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.652+0000 7f9800be4640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97fc0d0c40 con 0x7f97fc15c9a0 2023-10-16T20:05:57.657 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.652+0000 7f9800be4640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97fc0d37b0 con 0x7f97fc0cde80 2023-10-16T20:05:57.657 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.652+0000 7f9800be4640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc0ce7e0 con 0x7f97fc0cf230 2023-10-16T20:05:57.658 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.652+0000 7f97fbfff640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f97fc0cf230 0x7f97fc0cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:57.658 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.652+0000 7f97fb7fe640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f97fc15c9a0 0x7f97fc0d10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:49150/0 (socket says 172.21.15.88:49150) 2023-10-16T20:05:57.658 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.652+0000 7f97f3fff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f97fc0cde80 0x7f97fc0d3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:57.659 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.652+0000 7f97fb7fe640 1 -- 172.21.15.88:0/3444447757 learned_addr learned my addr 172.21.15.88:0/3444447757 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:57.659 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.652+0000 7f97f3fff640 1 -- 172.21.15.88:0/3444447757 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f97fc0cf230 msgr2=0x7f97fc0cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:57.659 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.652+0000 7f97f3fff640 1 --2- 172.21.15.88:0/3444447757 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f97fc0cf230 0x7f97fc0cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:57.659 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.652+0000 7f97f3fff640 1 -- 172.21.15.88:0/3444447757 >> v1:172.21.15.94:6789/0 conn(0x7f97fc15c9a0 legacy=0x7f97fc0d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:57.659 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.652+0000 7f97f3fff640 1 -- 172.21.15.88:0/3444447757 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97fc0ce2d0 con 0x7f97fc0cde80 2023-10-16T20:05:57.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.652+0000 7f97fbfff640 1 --2- 172.21.15.88:0/3444447757 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f97fc0cf230 0x7f97fc0cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:05:57.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.656+0000 7f97f3fff640 1 --2- 172.21.15.88:0/3444447757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97fc0cde80 0x7f97fc0d3270 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f97e40097c0 tx=0x7f97e401c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2671d47b5ca30749 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:57.661 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.656+0000 7f97faffd640 1 -- 172.21.15.88:0/3444447757 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e40ad640 con 0x7f97fc0cde80 2023-10-16T20:05:57.661 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.656+0000 7f97faffd640 1 -- 172.21.15.88:0/3444447757 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e40ad7a0 con 0x7f97fc0cde80 2023-10-16T20:05:57.661 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.656+0000 7f97faffd640 1 -- 172.21.15.88:0/3444447757 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e40adb30 con 0x7f97fc0cde80 2023-10-16T20:05:57.662 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.656+0000 7f9800be4640 1 -- 172.21.15.88:0/3444447757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97fc0cde80 msgr2=0x7f97fc0d3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:57.662 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.656+0000 7f9800be4640 1 --2- 172.21.15.88:0/3444447757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97fc0cde80 0x7f97fc0d3270 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f97e40097c0 tx=0x7f97e401c480 comp rx=0 tx=0).stop 2023-10-16T20:05:57.662 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.656+0000 7f9800be4640 1 -- 172.21.15.88:0/3444447757 shutdown_connections 2023-10-16T20:05:57.663 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.656+0000 7f9800be4640 1 --2- 172.21.15.88:0/3444447757 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f97fc0cf230 0x7f97fc0cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:57.663 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.656+0000 7f9800be4640 1 --2- 172.21.15.88:0/3444447757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97fc0cde80 0x7f97fc0d3270 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:57.663 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.656+0000 7f9800be4640 1 -- 172.21.15.88:0/3444447757 >> 172.21.15.88:0/3444447757 conn(0x7f97fc09bc30 msgr2=0x7f97fc0d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:57.663 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.656+0000 7f9800be4640 1 -- 172.21.15.88:0/3444447757 shutdown_connections 2023-10-16T20:05:57.664 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.656+0000 7f9800be4640 1 -- 172.21.15.88:0/3444447757 wait complete. 2023-10-16T20:05:57.666 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.660+0000 7f9800be4640 1 Processor -- start 2023-10-16T20:05:57.666 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.660+0000 7f9800be4640 1 -- start start 2023-10-16T20:05:57.666 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.660+0000 7f9800be4640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f97fc0cde80 0x7f97fc152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:57.666 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.660+0000 7f9800be4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97fc0cf230 0x7f97fc0cc350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:57.666 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.660+0000 7f9800be4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97fc15c9a0 0x7f97fc0c6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:57.667 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.660+0000 7f9800be4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc0cb770 con 0x7f97fc15c9a0 2023-10-16T20:05:57.667 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.660+0000 7f9800be4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc0cbf80 con 0x7f97fc0cf230 2023-10-16T20:05:57.667 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.660+0000 7f9800be4640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc1525a0 con 0x7f97fc0cde80 2023-10-16T20:05:57.667 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.660+0000 7f97fbfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97fc15c9a0 0x7f97fc0c6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:57.668 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.660+0000 7f97fbfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97fc15c9a0 0x7f97fc0c6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50756/0 (socket says 172.21.15.88:50756) 2023-10-16T20:05:57.668 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.660+0000 7f97fbfff640 1 -- 172.21.15.88:0/382385437 learned_addr learned my addr 172.21.15.88:0/382385437 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:57.668 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.660+0000 7f97f3fff640 1 --2- 172.21.15.88:0/382385437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97fc0cf230 0x7f97fc0cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:57.668 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.660+0000 7f97fb7fe640 1 --2- 172.21.15.88:0/382385437 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f97fc0cde80 0x7f97fc152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:57.668 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.660+0000 7f97fbfff640 1 -- 172.21.15.88:0/382385437 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f97fc0cde80 msgr2=0x7f97fc152060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:57.669 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.660+0000 7f97fbfff640 1 --2- 172.21.15.88:0/382385437 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f97fc0cde80 0x7f97fc152060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:57.669 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.660+0000 7f97fbfff640 1 -- 172.21.15.88:0/382385437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97fc0cf230 msgr2=0x7f97fc0cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:57.669 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.660+0000 7f97fbfff640 1 --2- 172.21.15.88:0/382385437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97fc0cf230 0x7f97fc0cc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:57.669 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.660+0000 7f97fbfff640 1 -- 172.21.15.88:0/382385437 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f4009a70 con 0x7f97fc15c9a0 2023-10-16T20:05:57.669 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.660+0000 7f97f3fff640 1 --2- 172.21.15.88:0/382385437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97fc0cf230 0x7f97fc0cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:05:57.670 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.660+0000 7f97fbfff640 1 --2- 172.21.15.88:0/382385437 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97fc15c9a0 0x7f97fc0c6d20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f97f400efc0 tx=0x7f97f4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:57.670 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.660+0000 7f97f8ff9640 1 -- 172.21.15.88:0/382385437 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f4009d70 con 0x7f97fc15c9a0 2023-10-16T20:05:57.670 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.660+0000 7f97f8ff9640 1 -- 172.21.15.88:0/382385437 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f400f040 con 0x7f97fc15c9a0 2023-10-16T20:05:57.670 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.660+0000 7f9800be4640 1 -- 172.21.15.88:0/382385437 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97e4009420 con 0x7f97fc15c9a0 2023-10-16T20:05:57.670 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.660+0000 7f97f8ff9640 1 -- 172.21.15.88:0/382385437 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f40039a0 con 0x7f97fc15c9a0 2023-10-16T20:05:57.671 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.660+0000 7f9800be4640 1 -- 172.21.15.88:0/382385437 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97fc0c7260 con 0x7f97fc15c9a0 2023-10-16T20:05:57.672 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.660+0000 7f9800be4640 1 -- 172.21.15.88:0/382385437 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97fc0b0710 con 0x7f97fc15c9a0 2023-10-16T20:05:57.672 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.664+0000 7f97f8ff9640 1 -- 172.21.15.88:0/382385437 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f97f40144a0 con 0x7f97fc15c9a0 2023-10-16T20:05:57.672 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.664+0000 7f97f8ff9640 1 --2- 172.21.15.88:0/382385437 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f97dc0664a0 0x7f97dc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:57.672 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.664+0000 7f97f8ff9640 1 -- 172.21.15.88:0/382385437 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f97f4084d20 con 0x7f97fc15c9a0 2023-10-16T20:05:57.673 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.664+0000 7f97fb7fe640 1 --2- 172.21.15.88:0/382385437 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f97dc0664a0 0x7f97dc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:57.673 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.664+0000 7f97fb7fe640 1 --2- 172.21.15.88:0/382385437 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f97dc0664a0 0x7f97dc068960 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f97ec0058e0 tx=0x7f97ec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:57.673 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.668+0000 7f97f8ff9640 1 -- 172.21.15.88:0/382385437 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f97f4089070 con 0x7f97fc15c9a0 2023-10-16T20:05:57.826 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.820+0000 7f9800be4640 1 -- 172.21.15.88:0/382385437 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97fc0674f0 con 0x7f97fc15c9a0 2023-10-16T20:05:57.827 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.824+0000 7f97f8ff9640 1 -- 172.21.15.88:0/382385437 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18661 (secure 0 0 0) 0x7f97f4018090 con 0x7f97fc15c9a0 2023-10-16T20:05:57.827 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:57.828 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":29,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:05:38.202959+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T20:04:54.442487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T20:04:55.766641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T20:04:59.475830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":1}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:05:57.829 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.824+0000 7f9800be4640 1 -- 172.21.15.88:0/382385437 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f97dc0664a0 msgr2=0x7f97dc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:57.830 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.824+0000 7f9800be4640 1 --2- 172.21.15.88:0/382385437 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f97dc0664a0 0x7f97dc068960 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f97ec0058e0 tx=0x7f97ec009040 comp rx=0 tx=0).stop 2023-10-16T20:05:57.830 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.824+0000 7f9800be4640 1 -- 172.21.15.88:0/382385437 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97fc15c9a0 msgr2=0x7f97fc0c6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:57.830 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.824+0000 7f9800be4640 1 --2- 172.21.15.88:0/382385437 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97fc15c9a0 0x7f97fc0c6d20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f97f400efc0 tx=0x7f97f4009f90 comp rx=0 tx=0).stop 2023-10-16T20:05:57.830 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.824+0000 7f9800be4640 1 -- 172.21.15.88:0/382385437 shutdown_connections 2023-10-16T20:05:57.830 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.824+0000 7f9800be4640 1 --2- 172.21.15.88:0/382385437 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f97dc0664a0 0x7f97dc068960 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:57.831 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.824+0000 7f9800be4640 1 --2- 172.21.15.88:0/382385437 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97fc15c9a0 0x7f97fc0c6d20 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:57.831 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.824+0000 7f9800be4640 1 --2- 172.21.15.88:0/382385437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97fc0cf230 0x7f97fc0cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:57.831 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.824+0000 7f9800be4640 1 --2- 172.21.15.88:0/382385437 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f97fc0cde80 0x7f97fc152060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:57.831 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.824+0000 7f9800be4640 1 -- 172.21.15.88:0/382385437 >> 172.21.15.88:0/382385437 conn(0x7f97fc09bc30 msgr2=0x7f97fc160520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:57.831 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.824+0000 7f9800be4640 1 -- 172.21.15.88:0/382385437 shutdown_connections 2023-10-16T20:05:57.832 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.824+0000 7f9800be4640 1 -- 172.21.15.88:0/382385437 wait complete. 2023-10-16T20:05:57.848 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T20:04:54.442487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T20:04:55.766641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T20:04:59.475830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T20:05:57.849 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:05:57.989 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.984+0000 7f879eaad640 1 Processor -- start 2023-10-16T20:05:57.990 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.984+0000 7f879eaad640 1 -- start start 2023-10-16T20:05:57.991 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.988+0000 7f879eaad640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f879815c9a0 0x7f87980cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:57.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.988+0000 7f879eaad640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f87980cf230 0x7f87980cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:57.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.988+0000 7f879eaad640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87980d0c40 con 0x7f87980d3270 2023-10-16T20:05:57.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.988+0000 7f879eaad640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87980ce7e0 con 0x7f879815c9a0 2023-10-16T20:05:57.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.988+0000 7f879eaad640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87980d0630 con 0x7f87980cf230 2023-10-16T20:05:57.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.988+0000 7f879daab640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f879815c9a0 0x7f87980cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:57.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.988+0000 7f879daab640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f879815c9a0 0x7f87980cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50762/0 (socket says 172.21.15.88:50762) 2023-10-16T20:05:57.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.988+0000 7f879daab640 1 -- 172.21.15.88:0/59036488 learned_addr learned my addr 172.21.15.88:0/59036488 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:57.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.988+0000 7f879d2aa640 1 --1- 172.21.15.88:0/59036488 >> v1:172.21.15.88:6789/0 conn(0x7f87980d3270 0x7f87980d10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:53520/0 (socket says 172.21.15.88:53520) 2023-10-16T20:05:57.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.988+0000 7f879e2ac640 1 --2- 172.21.15.88:0/59036488 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f87980cf230 0x7f87980cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:57.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.988+0000 7f879daab640 1 -- 172.21.15.88:0/59036488 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f87980cf230 msgr2=0x7f87980cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:57.994 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.988+0000 7f879daab640 1 --2- 172.21.15.88:0/59036488 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f87980cf230 0x7f87980cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:57.994 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.988+0000 7f879daab640 1 -- 172.21.15.88:0/59036488 >> v1:172.21.15.88:6789/0 conn(0x7f87980d3270 legacy=0x7f87980d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:57.994 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.988+0000 7f879daab640 1 -- 172.21.15.88:0/59036488 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87980ce520 con 0x7f879815c9a0 2023-10-16T20:05:57.994 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.988+0000 7f879caa9640 1 -- 172.21.15.88:0/59036488 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2365468263 0 0) 0x7f87980d0c40 con 0x7f87980d3270 2023-10-16T20:05:57.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.988+0000 7f879daab640 1 --2- 172.21.15.88:0/59036488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f879815c9a0 0x7f87980cde80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8794006fd0 tx=0x7f879401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7494757fab370a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:57.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.992+0000 7f879caa9640 1 -- 172.21.15.88:0/59036488 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87940aa680 con 0x7f879815c9a0 2023-10-16T20:05:57.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.992+0000 7f879caa9640 1 -- 172.21.15.88:0/59036488 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87940aa7e0 con 0x7f879815c9a0 2023-10-16T20:05:57.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.992+0000 7f879caa9640 1 -- 172.21.15.88:0/59036488 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87940aab70 con 0x7f879815c9a0 2023-10-16T20:05:57.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.992+0000 7f879eaad640 1 -- 172.21.15.88:0/59036488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f879815c9a0 msgr2=0x7f87980cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:57.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.992+0000 7f879eaad640 1 --2- 172.21.15.88:0/59036488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f879815c9a0 0x7f87980cde80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8794006fd0 tx=0x7f879401c520 comp rx=0 tx=0).stop 2023-10-16T20:05:57.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.992+0000 7f879eaad640 1 -- 172.21.15.88:0/59036488 shutdown_connections 2023-10-16T20:05:57.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.992+0000 7f879eaad640 1 --2- 172.21.15.88:0/59036488 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f87980cf230 0x7f87980cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:57.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.992+0000 7f879eaad640 1 --2- 172.21.15.88:0/59036488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f879815c9a0 0x7f87980cde80 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:57.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.992+0000 7f879eaad640 1 -- 172.21.15.88:0/59036488 >> 172.21.15.88:0/59036488 conn(0x7f879809bc30 msgr2=0x7f87980d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:57.998 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.992+0000 7f879eaad640 1 -- 172.21.15.88:0/59036488 shutdown_connections 2023-10-16T20:05:57.998 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.992+0000 7f879eaad640 1 -- 172.21.15.88:0/59036488 wait complete. 2023-10-16T20:05:58.000 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.992+0000 7f879eaad640 1 Processor -- start 2023-10-16T20:05:58.000 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.992+0000 7f879eaad640 1 -- start start 2023-10-16T20:05:58.000 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.992+0000 7f879eaad640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87980cf230 0x7f87980c63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:58.001 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.992+0000 7f879eaad640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87980d3270 0x7f8798157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:58.001 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.992+0000 7f879eaad640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f879815c9a0 0x7f87980c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:58.001 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.992+0000 7f879eaad640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87981552d0 con 0x7f87980d3270 2023-10-16T20:05:58.001 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.992+0000 7f879eaad640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87980c6930 con 0x7f87980cf230 2023-10-16T20:05:58.001 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.992+0000 7f879eaad640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87980c7770 con 0x7f879815c9a0 2023-10-16T20:05:58.002 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.992+0000 7f879d2aa640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87980d3270 0x7f8798157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:58.002 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.992+0000 7f879d2aa640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87980d3270 0x7f8798157910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50776/0 (socket says 172.21.15.88:50776) 2023-10-16T20:05:58.002 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.992+0000 7f879d2aa640 1 -- 172.21.15.88:0/2105500324 learned_addr learned my addr 172.21.15.88:0/2105500324 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:58.002 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.992+0000 7f879e2ac640 1 --2- 172.21.15.88:0/2105500324 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f879815c9a0 0x7f87980c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:58.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.992+0000 7f879daab640 1 --2- 172.21.15.88:0/2105500324 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87980cf230 0x7f87980c63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:58.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.996+0000 7f879d2aa640 1 -- 172.21.15.88:0/2105500324 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f879815c9a0 msgr2=0x7f87980c7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:58.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.996+0000 7f879d2aa640 1 --2- 172.21.15.88:0/2105500324 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f879815c9a0 0x7f87980c7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:58.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.996+0000 7f879d2aa640 1 -- 172.21.15.88:0/2105500324 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87980cf230 msgr2=0x7f87980c63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:58.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.996+0000 7f879d2aa640 1 --2- 172.21.15.88:0/2105500324 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87980cf230 0x7f87980c63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:58.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.996+0000 7f879d2aa640 1 -- 172.21.15.88:0/2105500324 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8794009480 con 0x7f87980d3270 2023-10-16T20:05:58.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.996+0000 7f879d2aa640 1 --2- 172.21.15.88:0/2105500324 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87980d3270 0x7f8798157910 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f87880011f0 tx=0x7f878800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:58.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.996+0000 7f878e7fc640 1 -- 172.21.15.88:0/2105500324 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87880074e0 con 0x7f87980d3270 2023-10-16T20:05:58.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.996+0000 7f878e7fc640 1 -- 172.21.15.88:0/2105500324 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8788007640 con 0x7f87980d3270 2023-10-16T20:05:58.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.996+0000 7f879eaad640 1 -- 172.21.15.88:0/2105500324 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87980c6bc0 con 0x7f87980d3270 2023-10-16T20:05:58.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.996+0000 7f878e7fc640 1 -- 172.21.15.88:0/2105500324 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8788004a10 con 0x7f87980d3270 2023-10-16T20:05:58.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.996+0000 7f879eaad640 1 -- 172.21.15.88:0/2105500324 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87980c7400 con 0x7f87980d3270 2023-10-16T20:05:58.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.996+0000 7f879eaad640 1 -- 172.21.15.88:0/2105500324 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87980d1780 con 0x7f87980d3270 2023-10-16T20:05:58.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.996+0000 7f878e7fc640 1 -- 172.21.15.88:0/2105500324 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8788004b70 con 0x7f87980d3270 2023-10-16T20:05:58.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.996+0000 7f878e7fc640 1 --2- 172.21.15.88:0/2105500324 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f877c0664a0 0x7f877c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:58.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.996+0000 7f878e7fc640 1 -- 172.21.15.88:0/2105500324 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8788083730 con 0x7f87980d3270 2023-10-16T20:05:58.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:57.996+0000 7f879daab640 1 --2- 172.21.15.88:0/2105500324 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f877c0664a0 0x7f877c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:58.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.000+0000 7f879daab640 1 --2- 172.21.15.88:0/2105500324 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f877c0664a0 0x7f877c068960 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f87940ae000 tx=0x7f87940b5040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:58.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.000+0000 7f878e7fc640 1 -- 172.21.15.88:0/2105500324 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8788050890 con 0x7f87980d3270 2023-10-16T20:05:58.155 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.152+0000 7f879eaad640 1 -- 172.21.15.88:0/2105500324 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f87980674f0 con 0x7f87980d3270 2023-10-16T20:05:58.156 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.152+0000 7f878e7fc640 1 -- 172.21.15.88:0/2105500324 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8788018020 con 0x7f87980d3270 2023-10-16T20:05:58.156 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:05:58.158 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.152+0000 7f879eaad640 1 -- 172.21.15.88:0/2105500324 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f877c0664a0 msgr2=0x7f877c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:58.158 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.152+0000 7f879eaad640 1 --2- 172.21.15.88:0/2105500324 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f877c0664a0 0x7f877c068960 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f87940ae000 tx=0x7f87940b5040 comp rx=0 tx=0).stop 2023-10-16T20:05:58.158 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.152+0000 7f879eaad640 1 -- 172.21.15.88:0/2105500324 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87980d3270 msgr2=0x7f8798157910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:58.158 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.152+0000 7f879eaad640 1 --2- 172.21.15.88:0/2105500324 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87980d3270 0x7f8798157910 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f87880011f0 tx=0x7f878800c9b0 comp rx=0 tx=0).stop 2023-10-16T20:05:58.159 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.152+0000 7f879eaad640 1 -- 172.21.15.88:0/2105500324 shutdown_connections 2023-10-16T20:05:58.159 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.152+0000 7f879eaad640 1 --2- 172.21.15.88:0/2105500324 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f877c0664a0 0x7f877c068960 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:58.159 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.152+0000 7f879eaad640 1 --2- 172.21.15.88:0/2105500324 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f879815c9a0 0x7f87980c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:58.159 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.152+0000 7f879eaad640 1 --2- 172.21.15.88:0/2105500324 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87980d3270 0x7f8798157910 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:58.159 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.152+0000 7f879eaad640 1 --2- 172.21.15.88:0/2105500324 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87980cf230 0x7f87980c63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:58.160 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.152+0000 7f879eaad640 1 -- 172.21.15.88:0/2105500324 >> 172.21.15.88:0/2105500324 conn(0x7f879809bc30 msgr2=0x7f8798155510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:58.160 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.152+0000 7f879eaad640 1 -- 172.21.15.88:0/2105500324 shutdown_connections 2023-10-16T20:05:58.160 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.152+0000 7f879eaad640 1 -- 172.21.15.88:0/2105500324 wait complete. 2023-10-16T20:05:58.177 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T20:05:58.318 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.312+0000 7f68939d9640 1 Processor -- start 2023-10-16T20:05:58.318 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.312+0000 7f68939d9640 1 -- start start 2023-10-16T20:05:58.320 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.316+0000 7f68939d9640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f688c0b0da0 0x7f688c159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:58.320 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.316+0000 7f68939d9640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f688c15a1f0 0x7f688c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:58.321 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.316+0000 7f68939d9640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f688c15ad30 con 0x7f688c0b4260 2023-10-16T20:05:58.321 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.316+0000 7f68939d9640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f688c0b4640 con 0x7f688c0b0da0 2023-10-16T20:05:58.321 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.316+0000 7f68939d9640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f688c15a770 con 0x7f688c15a1f0 2023-10-16T20:05:58.321 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.316+0000 7f6891775640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f688c0b4260 0x7f688c15dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:53534/0 (socket says 172.21.15.88:53534) 2023-10-16T20:05:58.321 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.316+0000 7f6891775640 1 -- 172.21.15.88:0/1358709303 learned_addr learned my addr 172.21.15.88:0/1358709303 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:58.322 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.316+0000 7f6890f74640 1 --2- 172.21.15.88:0/1358709303 >> v2:172.21.15.88:3300/0 conn(0x7f688c0b0da0 0x7f688c159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:58.322 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.316+0000 7f6891f76640 1 --2- 172.21.15.88:0/1358709303 >> v2:172.21.15.94:3300/0 conn(0x7f688c15a1f0 0x7f688c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:58.322 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.316+0000 7f6883fff640 1 -- 172.21.15.88:0/1358709303 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1102620902 0 0) 0x7f688c15ad30 con 0x7f688c0b4260 2023-10-16T20:05:58.322 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.316+0000 7f6883fff640 1 -- 172.21.15.88:0/1358709303 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6878003530 con 0x7f688c0b4260 2023-10-16T20:05:58.323 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.316+0000 7f6890f74640 1 -- 172.21.15.88:0/1358709303 >> v2:172.21.15.94:3300/0 conn(0x7f688c15a1f0 msgr2=0x7f688c06d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:58.323 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.316+0000 7f6890f74640 1 --2- 172.21.15.88:0/1358709303 >> v2:172.21.15.94:3300/0 conn(0x7f688c15a1f0 0x7f688c06d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:58.323 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.316+0000 7f6890f74640 1 -- 172.21.15.88:0/1358709303 >> v1:172.21.15.88:6789/0 conn(0x7f688c0b4260 legacy=0x7f688c15dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:58.323 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.316+0000 7f6890f74640 1 -- 172.21.15.88:0/1358709303 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f688c15aa50 con 0x7f688c0b0da0 2023-10-16T20:05:58.325 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.320+0000 7f6890f74640 1 --2- 172.21.15.88:0/1358709303 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f688c0b0da0 0x7f688c159cb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f688c0b1230 tx=0x7f687c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1adadb91c2488a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:58.325 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.320+0000 7f6883fff640 1 -- 172.21.15.88:0/1358709303 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c0ab600 con 0x7f688c0b0da0 2023-10-16T20:05:58.325 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.320+0000 7f6883fff640 1 -- 172.21.15.88:0/1358709303 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f687c0a7050 con 0x7f688c0b0da0 2023-10-16T20:05:58.325 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.320+0000 7f6883fff640 1 -- 172.21.15.88:0/1358709303 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c0ab980 con 0x7f688c0b0da0 2023-10-16T20:05:58.325 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.320+0000 7f68939d9640 1 -- 172.21.15.88:0/1358709303 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f688c0b0da0 msgr2=0x7f688c159cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:58.326 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.320+0000 7f68939d9640 1 --2- 172.21.15.88:0/1358709303 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f688c0b0da0 0x7f688c159cb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f688c0b1230 tx=0x7f687c01c3d0 comp rx=0 tx=0).stop 2023-10-16T20:05:58.326 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.320+0000 7f68939d9640 1 -- 172.21.15.88:0/1358709303 shutdown_connections 2023-10-16T20:05:58.326 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.320+0000 7f68939d9640 1 --2- 172.21.15.88:0/1358709303 >> v2:172.21.15.94:3300/0 conn(0x7f688c15a1f0 0x7f688c06d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:58.326 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.320+0000 7f68939d9640 1 --2- 172.21.15.88:0/1358709303 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f688c0b0da0 0x7f688c159cb0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:58.327 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.320+0000 7f68939d9640 1 -- 172.21.15.88:0/1358709303 >> 172.21.15.88:0/1358709303 conn(0x7f688c0cb080 msgr2=0x7f688c0cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:58.327 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.320+0000 7f68939d9640 1 -- 172.21.15.88:0/1358709303 shutdown_connections 2023-10-16T20:05:58.327 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.320+0000 7f68939d9640 1 -- 172.21.15.88:0/1358709303 wait complete. 2023-10-16T20:05:58.329 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.320+0000 7f68939d9640 1 Processor -- start 2023-10-16T20:05:58.330 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.320+0000 7f68939d9640 1 -- start start 2023-10-16T20:05:58.330 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.324+0000 7f68939d9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f688c0b0da0 0x7f688c145180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:58.330 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.324+0000 7f68939d9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f688c0b4260 0x7f688c1588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:58.330 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.324+0000 7f68939d9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f688c15a1f0 0x7f688c070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:58.330 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.324+0000 7f68939d9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688c1f6880 con 0x7f688c0b4260 2023-10-16T20:05:58.331 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.324+0000 7f68939d9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688c1f59f0 con 0x7f688c15a1f0 2023-10-16T20:05:58.331 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.324+0000 7f68939d9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f688c1f5c70 con 0x7f688c0b0da0 2023-10-16T20:05:58.331 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.324+0000 7f6890f74640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f688c0b4260 0x7f688c1588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:58.331 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.324+0000 7f6890f74640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f688c0b4260 0x7f688c1588a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50782/0 (socket says 172.21.15.88:50782) 2023-10-16T20:05:58.331 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.324+0000 7f6890f74640 1 -- 172.21.15.88:0/911433395 learned_addr learned my addr 172.21.15.88:0/911433395 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:58.332 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.324+0000 7f6891f76640 1 --2- 172.21.15.88:0/911433395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f688c15a1f0 0x7f688c070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:58.332 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.324+0000 7f6891775640 1 --2- 172.21.15.88:0/911433395 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f688c0b0da0 0x7f688c145180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:58.332 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.324+0000 7f6890f74640 1 -- 172.21.15.88:0/911433395 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f688c0b0da0 msgr2=0x7f688c145180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:58.332 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.324+0000 7f6890f74640 1 --2- 172.21.15.88:0/911433395 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f688c0b0da0 0x7f688c145180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:58.333 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.324+0000 7f6890f74640 1 -- 172.21.15.88:0/911433395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f688c15a1f0 msgr2=0x7f688c070150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:58.333 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.324+0000 7f6890f74640 1 --2- 172.21.15.88:0/911433395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f688c15a1f0 0x7f688c070150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:58.333 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.324+0000 7f6890f74640 1 -- 172.21.15.88:0/911433395 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687c009420 con 0x7f688c0b4260 2023-10-16T20:05:58.333 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.324+0000 7f6890f74640 1 --2- 172.21.15.88:0/911433395 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f688c0b4260 0x7f688c1588a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f687c01a000 tx=0x7f687c019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:58.333 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.324+0000 7f6881ffb640 1 -- 172.21.15.88:0/911433395 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c0abe70 con 0x7f688c0b4260 2023-10-16T20:05:58.334 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.324+0000 7f6881ffb640 1 -- 172.21.15.88:0/911433395 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f687c0b4430 con 0x7f688c0b4260 2023-10-16T20:05:58.334 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.324+0000 7f6881ffb640 1 -- 172.21.15.88:0/911433395 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c0b4800 con 0x7f688c0b4260 2023-10-16T20:05:58.334 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.324+0000 7f68939d9640 1 -- 172.21.15.88:0/911433395 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f688c1f5f50 con 0x7f688c0b4260 2023-10-16T20:05:58.334 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.324+0000 7f68939d9640 1 -- 172.21.15.88:0/911433395 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f688c1f6510 con 0x7f688c0b4260 2023-10-16T20:05:58.334 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.324+0000 7f68939d9640 1 -- 172.21.15.88:0/911433395 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f688c04e9c0 con 0x7f688c0b4260 2023-10-16T20:05:58.335 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.328+0000 7f6881ffb640 1 -- 172.21.15.88:0/911433395 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f687c0b2040 con 0x7f688c0b4260 2023-10-16T20:05:58.336 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.328+0000 7f6881ffb640 1 --2- 172.21.15.88:0/911433395 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f68700664a0 0x7f6870068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:58.336 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.328+0000 7f6881ffb640 1 -- 172.21.15.88:0/911433395 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f687c11dfe0 con 0x7f688c0b4260 2023-10-16T20:05:58.336 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.328+0000 7f6891775640 1 --2- 172.21.15.88:0/911433395 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f68700664a0 0x7f6870068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:58.336 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.328+0000 7f6891775640 1 --2- 172.21.15.88:0/911433395 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f68700664a0 0x7f6870068960 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f68880011f0 tx=0x7f6888009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:58.336 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.332+0000 7f6881ffb640 1 -- 172.21.15.88:0/911433395 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f687c0eafb0 con 0x7f688c0b4260 2023-10-16T20:05:58.487 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.484+0000 7f68939d9640 1 -- 172.21.15.88:0/911433395 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f688c071500 con 0x7f688c0b4260 2023-10-16T20:05:58.487 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.484+0000 7f6881ffb640 1 -- 172.21.15.88:0/911433395 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f687c0ee5d0 con 0x7f688c0b4260 2023-10-16T20:05:58.488 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:05:58.489 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.484+0000 7f68939d9640 1 -- 172.21.15.88:0/911433395 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f68700664a0 msgr2=0x7f6870068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:58.490 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.484+0000 7f68939d9640 1 --2- 172.21.15.88:0/911433395 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f68700664a0 0x7f6870068960 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f68880011f0 tx=0x7f6888009040 comp rx=0 tx=0).stop 2023-10-16T20:05:58.490 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.484+0000 7f68939d9640 1 -- 172.21.15.88:0/911433395 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f688c0b4260 msgr2=0x7f688c1588a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:58.490 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.484+0000 7f68939d9640 1 --2- 172.21.15.88:0/911433395 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f688c0b4260 0x7f688c1588a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f687c01a000 tx=0x7f687c019170 comp rx=0 tx=0).stop 2023-10-16T20:05:58.490 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.484+0000 7f68939d9640 1 -- 172.21.15.88:0/911433395 shutdown_connections 2023-10-16T20:05:58.490 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.484+0000 7f68939d9640 1 --2- 172.21.15.88:0/911433395 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f68700664a0 0x7f6870068960 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:58.491 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.484+0000 7f68939d9640 1 --2- 172.21.15.88:0/911433395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f688c15a1f0 0x7f688c070150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:58.491 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.484+0000 7f68939d9640 1 --2- 172.21.15.88:0/911433395 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f688c0b4260 0x7f688c1588a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:58.491 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.484+0000 7f68939d9640 1 --2- 172.21.15.88:0/911433395 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f688c0b0da0 0x7f688c145180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:58.491 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.484+0000 7f68939d9640 1 -- 172.21.15.88:0/911433395 >> 172.21.15.88:0/911433395 conn(0x7f688c0cb080 msgr2=0x7f688c0af0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:58.491 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.484+0000 7f68939d9640 1 -- 172.21.15.88:0/911433395 shutdown_connections 2023-10-16T20:05:58.492 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.484+0000 7f68939d9640 1 -- 172.21.15.88:0/911433395 wait complete. 2023-10-16T20:05:58.508 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T20:05:58.648 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.644+0000 7f5263054640 1 Processor -- start 2023-10-16T20:05:58.648 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.644+0000 7f5263054640 1 -- start start 2023-10-16T20:05:58.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.644+0000 7f5263054640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f525c15cb70 0x7f525c0b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:58.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.644+0000 7f5263054640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f525c0b7d20 con 0x7f525c0b6a80 2023-10-16T20:05:58.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.644+0000 7f5263054640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f525c0b6eb0 con 0x7f525c0b4530 2023-10-16T20:05:58.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.644+0000 7f5263054640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f525c0b7110 con 0x7f525c15cb70 2023-10-16T20:05:58.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.644+0000 7f5260df0640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f525c15cb70 0x7f525c0b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:58.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.644+0000 7f5260df0640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f525c15cb70 0x7f525c0b83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50792/0 (socket says 172.21.15.88:50792) 2023-10-16T20:05:58.651 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.644+0000 7f52615f1640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f525c0b6a80 0x7f525c0d12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:53536/0 (socket says 172.21.15.88:53536) 2023-10-16T20:05:58.651 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.644+0000 7f5260df0640 1 -- 172.21.15.88:0/214454143 learned_addr learned my addr 172.21.15.88:0/214454143 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:58.651 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.644+0000 7f5260df0640 1 -- 172.21.15.88:0/214454143 >> v1:172.21.15.94:6789/0 conn(0x7f525c0b4530 legacy=0x7f525c0d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:58.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.644+0000 7f5260df0640 1 -- 172.21.15.88:0/214454143 >> v1:172.21.15.88:6789/0 conn(0x7f525c0b6a80 legacy=0x7f525c0d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:58.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.644+0000 7f5260df0640 1 -- 172.21.15.88:0/214454143 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525c0b73f0 con 0x7f525c15cb70 2023-10-16T20:05:58.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.644+0000 7f52537fe640 1 -- 172.21.15.88:0/214454143 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2487055980 0 0) 0x7f525c0b7d20 con 0x7f525c0b6a80 2023-10-16T20:05:58.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.648+0000 7f5260df0640 1 --2- 172.21.15.88:0/214454143 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f525c15cb70 0x7f525c0b83d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5258009870 tx=0x7f525801ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332e94e321a7ea6a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:58.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.648+0000 7f52537fe640 1 -- 172.21.15.88:0/214454143 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52580a8a30 con 0x7f525c15cb70 2023-10-16T20:05:58.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.648+0000 7f52537fe640 1 -- 172.21.15.88:0/214454143 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52580a8b90 con 0x7f525c15cb70 2023-10-16T20:05:58.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.648+0000 7f52537fe640 1 -- 172.21.15.88:0/214454143 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52580b0440 con 0x7f525c15cb70 2023-10-16T20:05:58.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.648+0000 7f5263054640 1 -- 172.21.15.88:0/214454143 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f525c15cb70 msgr2=0x7f525c0b83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:58.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.648+0000 7f5263054640 1 --2- 172.21.15.88:0/214454143 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f525c15cb70 0x7f525c0b83d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5258009870 tx=0x7f525801ed30 comp rx=0 tx=0).stop 2023-10-16T20:05:58.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.648+0000 7f5263054640 1 -- 172.21.15.88:0/214454143 shutdown_connections 2023-10-16T20:05:58.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.648+0000 7f5263054640 1 --2- 172.21.15.88:0/214454143 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f525c15cb70 0x7f525c0b83d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:58.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.648+0000 7f5263054640 1 -- 172.21.15.88:0/214454143 >> 172.21.15.88:0/214454143 conn(0x7f525c06b6e0 msgr2=0x7f525c06baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:58.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.648+0000 7f5263054640 1 -- 172.21.15.88:0/214454143 shutdown_connections 2023-10-16T20:05:58.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.648+0000 7f5263054640 1 -- 172.21.15.88:0/214454143 wait complete. 2023-10-16T20:05:58.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.652+0000 7f5263054640 1 Processor -- start 2023-10-16T20:05:58.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.652+0000 7f5263054640 1 -- start start 2023-10-16T20:05:58.658 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.652+0000 7f5263054640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f525c0b4530 0x7f525c0c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:58.658 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.652+0000 7f5263054640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f525c0b6a80 0x7f525c0b3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:58.658 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.652+0000 7f5263054640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f525c15cb70 0x7f525c0b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:58.659 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.652+0000 7f5263054640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525c1f5550 con 0x7f525c0b6a80 2023-10-16T20:05:58.659 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.652+0000 7f5263054640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525c0c8780 con 0x7f525c0b4530 2023-10-16T20:05:58.659 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.652+0000 7f5263054640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f525c1f57a0 con 0x7f525c15cb70 2023-10-16T20:05:58.659 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.652+0000 7f5253fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f525c0b6a80 0x7f525c0b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:58.659 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.652+0000 7f5253fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f525c0b6a80 0x7f525c0b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50806/0 (socket says 172.21.15.88:50806) 2023-10-16T20:05:58.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.652+0000 7f5253fff640 1 -- 172.21.15.88:0/2805395521 learned_addr learned my addr 172.21.15.88:0/2805395521 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:58.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.652+0000 7f5260df0640 1 --2- 172.21.15.88:0/2805395521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f525c0b4530 0x7f525c0c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:58.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.652+0000 7f52615f1640 1 --2- 172.21.15.88:0/2805395521 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f525c15cb70 0x7f525c0b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:58.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.652+0000 7f5253fff640 1 -- 172.21.15.88:0/2805395521 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f525c15cb70 msgr2=0x7f525c0b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:58.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.652+0000 7f5253fff640 1 --2- 172.21.15.88:0/2805395521 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f525c15cb70 0x7f525c0b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:58.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.652+0000 7f5253fff640 1 -- 172.21.15.88:0/2805395521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f525c0b4530 msgr2=0x7f525c0c8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:58.661 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.652+0000 7f5253fff640 1 --2- 172.21.15.88:0/2805395521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f525c0b4530 0x7f525c0c8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:58.661 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.652+0000 7f5253fff640 1 -- 172.21.15.88:0/2805395521 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52580094d0 con 0x7f525c0b6a80 2023-10-16T20:05:58.661 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.652+0000 7f5253fff640 1 --2- 172.21.15.88:0/2805395521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f525c0b6a80 0x7f525c0b3560 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f524c00c500 tx=0x7f524c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:58.661 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.652+0000 7f52517fa640 1 -- 172.21.15.88:0/2805395521 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c005220 con 0x7f525c0b6a80 2023-10-16T20:05:58.661 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.652+0000 7f52517fa640 1 -- 172.21.15.88:0/2805395521 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524c005380 con 0x7f525c0b6a80 2023-10-16T20:05:58.662 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.652+0000 7f52517fa640 1 -- 172.21.15.88:0/2805395521 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c007270 con 0x7f525c0b6a80 2023-10-16T20:05:58.662 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.652+0000 7f5263054640 1 -- 172.21.15.88:0/2805395521 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f525c1f5a80 con 0x7f525c0b6a80 2023-10-16T20:05:58.662 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.652+0000 7f5263054640 1 -- 172.21.15.88:0/2805395521 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f525c1f6040 con 0x7f525c0b6a80 2023-10-16T20:05:58.663 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.656+0000 7f5263054640 1 -- 172.21.15.88:0/2805395521 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f525c04e9c0 con 0x7f525c0b6a80 2023-10-16T20:05:58.663 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.656+0000 7f52517fa640 1 -- 172.21.15.88:0/2805395521 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f524c0073d0 con 0x7f525c0b6a80 2023-10-16T20:05:58.663 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.656+0000 7f52517fa640 1 --2- 172.21.15.88:0/2805395521 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f52480664a0 0x7f5248068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:58.663 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.656+0000 7f52517fa640 1 -- 172.21.15.88:0/2805395521 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f524c083b00 con 0x7f525c0b6a80 2023-10-16T20:05:58.664 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.656+0000 7f5260df0640 1 --2- 172.21.15.88:0/2805395521 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f52480664a0 0x7f5248068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:58.664 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.656+0000 7f5260df0640 1 --2- 172.21.15.88:0/2805395521 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f52480664a0 0x7f5248068960 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f52580b2000 tx=0x7f5258009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:58.665 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.660+0000 7f52517fa640 1 -- 172.21.15.88:0/2805395521 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f524c085e60 con 0x7f525c0b6a80 2023-10-16T20:05:58.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.808+0000 7f5263054640 1 -- 172.21.15.88:0/2805395521 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f525c0b7980 con 0x7f525c0b6a80 2023-10-16T20:05:58.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.808+0000 7f52517fa640 1 -- 172.21.15.88:0/2805395521 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f524c050b30 con 0x7f525c0b6a80 2023-10-16T20:05:58.815 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2023-10-16T20:05:58.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.812+0000 7f5263054640 1 -- 172.21.15.88:0/2805395521 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f52480664a0 msgr2=0x7f5248068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:58.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.812+0000 7f5263054640 1 --2- 172.21.15.88:0/2805395521 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f52480664a0 0x7f5248068960 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f52580b2000 tx=0x7f5258009040 comp rx=0 tx=0).stop 2023-10-16T20:05:58.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.812+0000 7f5263054640 1 -- 172.21.15.88:0/2805395521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f525c0b6a80 msgr2=0x7f525c0b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:58.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.812+0000 7f5263054640 1 --2- 172.21.15.88:0/2805395521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f525c0b6a80 0x7f525c0b3560 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f524c00c500 tx=0x7f524c00c9c0 comp rx=0 tx=0).stop 2023-10-16T20:05:58.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.812+0000 7f5263054640 1 -- 172.21.15.88:0/2805395521 shutdown_connections 2023-10-16T20:05:58.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.812+0000 7f5263054640 1 --2- 172.21.15.88:0/2805395521 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f52480664a0 0x7f5248068960 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:58.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.812+0000 7f5263054640 1 --2- 172.21.15.88:0/2805395521 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f525c15cb70 0x7f525c0b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:58.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.812+0000 7f5263054640 1 --2- 172.21.15.88:0/2805395521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f525c0b6a80 0x7f525c0b3560 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:58.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.812+0000 7f5263054640 1 --2- 172.21.15.88:0/2805395521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f525c0b4530 0x7f525c0c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:58.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.812+0000 7f5263054640 1 -- 172.21.15.88:0/2805395521 >> 172.21.15.88:0/2805395521 conn(0x7f525c06b6e0 msgr2=0x7f525c0d2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:58.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.812+0000 7f5263054640 1 -- 172.21.15.88:0/2805395521 shutdown_connections 2023-10-16T20:05:58.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.812+0000 7f5263054640 1 -- 172.21.15.88:0/2805395521 wait complete. 2023-10-16T20:05:58.835 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T20:05:58.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.972+0000 7f4ab8bd9640 1 Processor -- start 2023-10-16T20:05:58.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.972+0000 7f4ab8bd9640 1 -- start start 2023-10-16T20:05:58.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.972+0000 7f4ab8bd9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4ab40aeb90 0x7f4ab4070250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:58.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.972+0000 7f4ab8bd9640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4ab4146290 0x7f4ab4070790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:58.978 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.972+0000 7f4ab8bd9640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ab406c2e0 con 0x7f4ab41457f0 2023-10-16T20:05:58.978 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.972+0000 7f4ab8bd9640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ab4145bd0 con 0x7f4ab4146290 2023-10-16T20:05:58.978 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.972+0000 7f4ab8bd9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab406dab0 con 0x7f4ab40aeb90 2023-10-16T20:05:58.978 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.972+0000 7f4ab259c640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f4ab41457f0 0x7f4ab4160670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:53552/0 (socket says 172.21.15.88:53552) 2023-10-16T20:05:58.978 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.972+0000 7f4ab259c640 1 -- 172.21.15.88:0/3034885057 learned_addr learned my addr 172.21.15.88:0/3034885057 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:58.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.972+0000 7f4ab2d9d640 1 --2- 172.21.15.88:0/3034885057 >> v2:172.21.15.88:3300/0 conn(0x7f4ab4146290 0x7f4ab4070790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:58.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.972+0000 7f4aa9d9b640 1 --2- 172.21.15.88:0/3034885057 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4ab40aeb90 0x7f4ab4070250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:58.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.972+0000 7f4ab1d9b640 1 -- 172.21.15.88:0/3034885057 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 583387209 0 0) 0x7f4ab406c2e0 con 0x7f4ab41457f0 2023-10-16T20:05:58.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.972+0000 7f4ab1d9b640 1 -- 172.21.15.88:0/3034885057 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4aa0003530 con 0x7f4ab41457f0 2023-10-16T20:05:58.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.972+0000 7f4ab2d9d640 1 -- 172.21.15.88:0/3034885057 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4ab40aeb90 msgr2=0x7f4ab4070250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:58.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.972+0000 7f4ab2d9d640 1 --2- 172.21.15.88:0/3034885057 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4ab40aeb90 0x7f4ab4070250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:58.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.972+0000 7f4ab2d9d640 1 -- 172.21.15.88:0/3034885057 >> v1:172.21.15.88:6789/0 conn(0x7f4ab41457f0 legacy=0x7f4ab4160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:58.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.972+0000 7f4ab2d9d640 1 -- 172.21.15.88:0/3034885057 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab4160ec0 con 0x7f4ab4146290 2023-10-16T20:05:58.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.976+0000 7f4ab2d9d640 1 --2- 172.21.15.88:0/3034885057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ab4146290 0x7f4ab4070790 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4aac009d90 tx=0x7f4aac01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b14555a3b8e5917 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:58.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.976+0000 7f4ab1d9b640 1 -- 172.21.15.88:0/3034885057 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aac0a59e0 con 0x7f4ab4146290 2023-10-16T20:05:58.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.976+0000 7f4ab1d9b640 1 -- 172.21.15.88:0/3034885057 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aac0a5b40 con 0x7f4ab4146290 2023-10-16T20:05:58.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.976+0000 7f4ab1d9b640 1 -- 172.21.15.88:0/3034885057 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aac0ad440 con 0x7f4ab4146290 2023-10-16T20:05:58.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.976+0000 7f4ab8bd9640 1 -- 172.21.15.88:0/3034885057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ab4146290 msgr2=0x7f4ab4070790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:58.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.976+0000 7f4ab8bd9640 1 --2- 172.21.15.88:0/3034885057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ab4146290 0x7f4ab4070790 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4aac009d90 tx=0x7f4aac01c920 comp rx=0 tx=0).stop 2023-10-16T20:05:58.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.976+0000 7f4ab8bd9640 1 -- 172.21.15.88:0/3034885057 shutdown_connections 2023-10-16T20:05:58.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.976+0000 7f4ab8bd9640 1 --2- 172.21.15.88:0/3034885057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ab4146290 0x7f4ab4070790 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:58.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.976+0000 7f4ab8bd9640 1 --2- 172.21.15.88:0/3034885057 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4ab40aeb90 0x7f4ab4070250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:58.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.976+0000 7f4ab8bd9640 1 -- 172.21.15.88:0/3034885057 >> 172.21.15.88:0/3034885057 conn(0x7f4ab40cb080 msgr2=0x7f4ab40cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:58.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.976+0000 7f4ab8bd9640 1 -- 172.21.15.88:0/3034885057 shutdown_connections 2023-10-16T20:05:58.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.976+0000 7f4ab8bd9640 1 -- 172.21.15.88:0/3034885057 wait complete. 2023-10-16T20:05:58.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.980+0000 7f4ab8bd9640 1 Processor -- start 2023-10-16T20:05:58.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.980+0000 7f4ab8bd9640 1 -- start start 2023-10-16T20:05:58.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.980+0000 7f4ab8bd9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ab40aeb90 0x7f4ab413f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:58.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.980+0000 7f4ab8bd9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ab41457f0 0x7f4ab40b1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:58.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.980+0000 7f4ab8bd9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4ab4146290 0x7f4ab415e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:58.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.980+0000 7f4ab8bd9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab415f570 con 0x7f4ab40aeb90 2023-10-16T20:05:58.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.980+0000 7f4ab8bd9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab4142f90 con 0x7f4ab41457f0 2023-10-16T20:05:58.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.980+0000 7f4ab8bd9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab4143270 con 0x7f4ab4146290 2023-10-16T20:05:58.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.980+0000 7f4ab259c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ab40aeb90 0x7f4ab413f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:58.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.980+0000 7f4ab259c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ab40aeb90 0x7f4ab413f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50822/0 (socket says 172.21.15.88:50822) 2023-10-16T20:05:58.988 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.980+0000 7f4ab259c640 1 -- 172.21.15.88:0/3834857870 learned_addr learned my addr 172.21.15.88:0/3834857870 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:58.988 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.980+0000 7f4ab2d9d640 1 --2- 172.21.15.88:0/3834857870 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4ab4146290 0x7f4ab415e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:58.988 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.980+0000 7f4aa9d9b640 1 --2- 172.21.15.88:0/3834857870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ab41457f0 0x7f4ab40b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:58.988 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.980+0000 7f4ab259c640 1 -- 172.21.15.88:0/3834857870 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4ab4146290 msgr2=0x7f4ab415e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:58.988 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.980+0000 7f4ab259c640 1 --2- 172.21.15.88:0/3834857870 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4ab4146290 0x7f4ab415e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:58.989 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.980+0000 7f4ab259c640 1 -- 172.21.15.88:0/3834857870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ab41457f0 msgr2=0x7f4ab40b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:58.989 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.980+0000 7f4ab259c640 1 --2- 172.21.15.88:0/3834857870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ab41457f0 0x7f4ab40b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:58.989 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.980+0000 7f4ab259c640 1 -- 172.21.15.88:0/3834857870 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aac009a70 con 0x7f4ab40aeb90 2023-10-16T20:05:58.989 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.980+0000 7f4ab259c640 1 --2- 172.21.15.88:0/3834857870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ab40aeb90 0x7f4ab413f020 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400c4d0 tx=0x7f4aa400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:58.989 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.980+0000 7f4aab7fe640 1 -- 172.21.15.88:0/3834857870 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4005cf0 con 0x7f4ab40aeb90 2023-10-16T20:05:58.989 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.980+0000 7f4aab7fe640 1 -- 172.21.15.88:0/3834857870 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa4005e50 con 0x7f4ab40aeb90 2023-10-16T20:05:58.990 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.980+0000 7f4aab7fe640 1 -- 172.21.15.88:0/3834857870 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4007270 con 0x7f4ab40aeb90 2023-10-16T20:05:58.990 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.980+0000 7f4ab8bd9640 1 -- 172.21.15.88:0/3834857870 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ab4143550 con 0x7f4ab40aeb90 2023-10-16T20:05:58.990 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.980+0000 7f4ab8bd9640 1 -- 172.21.15.88:0/3834857870 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ab4143b10 con 0x7f4ab40aeb90 2023-10-16T20:05:58.991 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.980+0000 7f4ab8bd9640 1 -- 172.21.15.88:0/3834857870 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ab404e9c0 con 0x7f4ab40aeb90 2023-10-16T20:05:58.991 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.984+0000 7f4aab7fe640 1 -- 172.21.15.88:0/3834857870 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4aa40073d0 con 0x7f4ab40aeb90 2023-10-16T20:05:58.991 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.984+0000 7f4aab7fe640 1 --2- 172.21.15.88:0/3834857870 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4a940664a0 0x7f4a94068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:58.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.984+0000 7f4aab7fe640 1 -- 172.21.15.88:0/3834857870 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4aa4083750 con 0x7f4ab40aeb90 2023-10-16T20:05:58.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.984+0000 7f4aa9d9b640 1 --2- 172.21.15.88:0/3834857870 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4a940664a0 0x7f4a94068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:58.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.984+0000 7f4aa9d9b640 1 --2- 172.21.15.88:0/3834857870 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4a940664a0 0x7f4a94068960 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40b0ee0 tx=0x7f4a9c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:58.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:58.988+0000 7f4aab7fe640 1 -- 172.21.15.88:0/3834857870 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4aa4050820 con 0x7f4ab40aeb90 2023-10-16T20:05:59.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.136+0000 7f4ab8bd9640 1 -- 172.21.15.88:0/3834857870 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4ab4145fd0 con 0x7f4ab40aeb90 2023-10-16T20:05:59.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.136+0000 7f4aab7fe640 1 -- 172.21.15.88:0/3834857870 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4aa4053e40 con 0x7f4ab40aeb90 2023-10-16T20:05:59.141 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:05:59.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.136+0000 7f4ab8bd9640 1 -- 172.21.15.88:0/3834857870 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4a940664a0 msgr2=0x7f4a94068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:59.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.136+0000 7f4ab8bd9640 1 --2- 172.21.15.88:0/3834857870 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4a940664a0 0x7f4a94068960 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40b0ee0 tx=0x7f4a9c008040 comp rx=0 tx=0).stop 2023-10-16T20:05:59.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.136+0000 7f4ab8bd9640 1 -- 172.21.15.88:0/3834857870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ab40aeb90 msgr2=0x7f4ab413f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:59.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.136+0000 7f4ab8bd9640 1 --2- 172.21.15.88:0/3834857870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ab40aeb90 0x7f4ab413f020 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400c4d0 tx=0x7f4aa400c990 comp rx=0 tx=0).stop 2023-10-16T20:05:59.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.136+0000 7f4ab8bd9640 1 -- 172.21.15.88:0/3834857870 shutdown_connections 2023-10-16T20:05:59.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.136+0000 7f4ab8bd9640 1 --2- 172.21.15.88:0/3834857870 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4a940664a0 0x7f4a94068960 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:59.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.136+0000 7f4ab8bd9640 1 --2- 172.21.15.88:0/3834857870 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4ab4146290 0x7f4ab415e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:59.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.136+0000 7f4ab8bd9640 1 --2- 172.21.15.88:0/3834857870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ab41457f0 0x7f4ab40b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:59.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.136+0000 7f4ab8bd9640 1 --2- 172.21.15.88:0/3834857870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ab40aeb90 0x7f4ab413f020 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:59.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.136+0000 7f4ab8bd9640 1 -- 172.21.15.88:0/3834857870 >> 172.21.15.88:0/3834857870 conn(0x7f4ab40cb080 msgr2=0x7f4ab406a830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:59.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.136+0000 7f4ab8bd9640 1 -- 172.21.15.88:0/3834857870 shutdown_connections 2023-10-16T20:05:59.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.136+0000 7f4ab8bd9640 1 -- 172.21.15.88:0/3834857870 wait complete. 2023-10-16T20:05:59.161 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T20:05:59.299 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.296+0000 7f1409980640 1 Processor -- start 2023-10-16T20:05:59.341 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.296+0000 7f1409980640 1 -- start start 2023-10-16T20:05:59.345 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.296+0000 7f1409980640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1404153ea0 0x7f14040d1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:59.346 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.296+0000 7f1409980640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14040cd970 0x7f14040cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:59.346 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.296+0000 7f1409980640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14040b5510 con 0x7f14040d2660 2023-10-16T20:05:59.346 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.296+0000 7f1409980640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f140415d9a0 con 0x7f1404153ea0 2023-10-16T20:05:59.346 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.296+0000 7f1409980640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14040b4990 con 0x7f14040cd970 2023-10-16T20:05:59.346 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.296+0000 7f1402ffd640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1404153ea0 0x7f14040d1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:59.347 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.296+0000 7f1402ffd640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1404153ea0 0x7f14040d1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50824/0 (socket says 172.21.15.88:50824) 2023-10-16T20:05:59.347 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.296+0000 7f1402ffd640 1 -- 172.21.15.88:0/729668676 learned_addr learned my addr 172.21.15.88:0/729668676 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:59.347 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.296+0000 7f14037fe640 1 --2- 172.21.15.88:0/729668676 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14040cd970 0x7f14040cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:59.349 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.296+0000 7f1402ffd640 1 -- 172.21.15.88:0/729668676 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14040cd970 msgr2=0x7f14040cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:59.349 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.296+0000 7f1402ffd640 1 --2- 172.21.15.88:0/729668676 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14040cd970 0x7f14040cddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:59.349 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.296+0000 7f1402ffd640 1 -- 172.21.15.88:0/729668676 >> v1:172.21.15.94:6789/0 conn(0x7f14040d2660 legacy=0x7f140415d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:59.350 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.296+0000 7f1402ffd640 1 -- 172.21.15.88:0/729668676 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14040b4c70 con 0x7f1404153ea0 2023-10-16T20:05:59.350 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.300+0000 7f1402ffd640 1 --2- 172.21.15.88:0/729668676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1404153ea0 0x7f14040d1d10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009870 tx=0x7f13f401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fecbc1916af7400d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:59.350 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.304+0000 7f14027fc640 1 -- 172.21.15.88:0/729668676 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f40aa670 con 0x7f1404153ea0 2023-10-16T20:05:59.350 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.304+0000 7f14027fc640 1 -- 172.21.15.88:0/729668676 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f40aa7d0 con 0x7f1404153ea0 2023-10-16T20:05:59.350 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.304+0000 7f14027fc640 1 -- 172.21.15.88:0/729668676 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f40aab60 con 0x7f1404153ea0 2023-10-16T20:05:59.351 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.304+0000 7f1409980640 1 -- 172.21.15.88:0/729668676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1404153ea0 msgr2=0x7f14040d1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:59.351 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.304+0000 7f1409980640 1 --2- 172.21.15.88:0/729668676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1404153ea0 0x7f14040d1d10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009870 tx=0x7f13f401c510 comp rx=0 tx=0).stop 2023-10-16T20:05:59.351 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.304+0000 7f1409980640 1 -- 172.21.15.88:0/729668676 shutdown_connections 2023-10-16T20:05:59.351 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.304+0000 7f1409980640 1 --2- 172.21.15.88:0/729668676 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14040cd970 0x7f14040cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:59.351 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.304+0000 7f1409980640 1 --2- 172.21.15.88:0/729668676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1404153ea0 0x7f14040d1d10 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:59.352 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.304+0000 7f1409980640 1 -- 172.21.15.88:0/729668676 >> 172.21.15.88:0/729668676 conn(0x7f14040715c0 msgr2=0x7f14040719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:59.352 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.304+0000 7f1409980640 1 -- 172.21.15.88:0/729668676 shutdown_connections 2023-10-16T20:05:59.352 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.304+0000 7f1409980640 1 -- 172.21.15.88:0/729668676 wait complete. 2023-10-16T20:05:59.352 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.304+0000 7f1409980640 1 Processor -- start 2023-10-16T20:05:59.352 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.304+0000 7f1409980640 1 -- start start 2023-10-16T20:05:59.352 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.308+0000 7f1409980640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14040cd970 0x7f14040cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:59.353 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.308+0000 7f1409980640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14040d2660 0x7f14040c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:59.353 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.308+0000 7f1409980640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1404153ea0 0x7f14040b7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:59.353 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.308+0000 7f1409980640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14040b60e0 con 0x7f14040cd970 2023-10-16T20:05:59.353 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.308+0000 7f1409980640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14040b63c0 con 0x7f1404153ea0 2023-10-16T20:05:59.353 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.308+0000 7f1409980640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14040b66a0 con 0x7f14040d2660 2023-10-16T20:05:59.354 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.308+0000 7f1402ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14040cd970 0x7f14040cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:59.354 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.308+0000 7f1402ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14040cd970 0x7f14040cc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50832/0 (socket says 172.21.15.88:50832) 2023-10-16T20:05:59.354 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.308+0000 7f1402ffd640 1 -- 172.21.15.88:0/3105537947 learned_addr learned my addr 172.21.15.88:0/3105537947 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:59.354 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.308+0000 7f13fbfff640 1 --2- 172.21.15.88:0/3105537947 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14040d2660 0x7f14040c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:59.354 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.308+0000 7f14037fe640 1 --2- 172.21.15.88:0/3105537947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1404153ea0 0x7f14040b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:59.354 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.308+0000 7f1402ffd640 1 -- 172.21.15.88:0/3105537947 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14040d2660 msgr2=0x7f14040c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:59.355 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.308+0000 7f1402ffd640 1 --2- 172.21.15.88:0/3105537947 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14040d2660 0x7f14040c89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:59.355 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.308+0000 7f1402ffd640 1 -- 172.21.15.88:0/3105537947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1404153ea0 msgr2=0x7f14040b7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:59.355 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.308+0000 7f1402ffd640 1 --2- 172.21.15.88:0/3105537947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1404153ea0 0x7f14040b7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:59.355 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.308+0000 7f1402ffd640 1 -- 172.21.15.88:0/3105537947 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f40094d0 con 0x7f14040cd970 2023-10-16T20:05:59.355 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.308+0000 7f14037fe640 1 --2- 172.21.15.88:0/3105537947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1404153ea0 0x7f14040b7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:59.356 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.308+0000 7f13fbfff640 1 --2- 172.21.15.88:0/3105537947 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14040d2660 0x7f14040c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:05:59.356 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.308+0000 7f1402ffd640 1 --2- 172.21.15.88:0/3105537947 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14040cd970 0x7f14040cc660 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f13f401a140 tx=0x7f13f401ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:59.356 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.308+0000 7f1400ff9640 1 -- 172.21.15.88:0/3105537947 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f40b3800 con 0x7f14040cd970 2023-10-16T20:05:59.356 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.308+0000 7f1400ff9640 1 -- 172.21.15.88:0/3105537947 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f40b3980 con 0x7f14040cd970 2023-10-16T20:05:59.356 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.308+0000 7f1400ff9640 1 -- 172.21.15.88:0/3105537947 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f40b3d30 con 0x7f14040cd970 2023-10-16T20:05:59.356 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.308+0000 7f1409980640 1 -- 172.21.15.88:0/3105537947 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14040b6980 con 0x7f14040cd970 2023-10-16T20:05:59.357 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.308+0000 7f1409980640 1 -- 172.21.15.88:0/3105537947 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14040b5d20 con 0x7f14040cd970 2023-10-16T20:05:59.357 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.308+0000 7f1409980640 1 -- 172.21.15.88:0/3105537947 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14040b67e0 con 0x7f14040cd970 2023-10-16T20:05:59.357 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.308+0000 7f1400ff9640 1 -- 172.21.15.88:0/3105537947 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f13f40b2380 con 0x7f14040cd970 2023-10-16T20:05:59.357 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.312+0000 7f1400ff9640 1 --2- 172.21.15.88:0/3105537947 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f13e40664a0 0x7f13e4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:59.357 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.312+0000 7f1400ff9640 1 -- 172.21.15.88:0/3105537947 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f13f411ead0 con 0x7f14040cd970 2023-10-16T20:05:59.358 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.312+0000 7f13fbfff640 1 --2- 172.21.15.88:0/3105537947 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f13e40664a0 0x7f13e4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:59.358 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.312+0000 7f13fbfff640 1 --2- 172.21.15.88:0/3105537947 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f13e40664a0 0x7f13e4068960 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f14040ce5f0 tx=0x7f13ec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:59.358 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.316+0000 7f1400ff9640 1 -- 172.21.15.88:0/3105537947 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f14040b67e0 con 0x7f14040cd970 2023-10-16T20:05:59.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.476+0000 7f1409980640 1 -- 172.21.15.88:0/3105537947 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f14040b67e0 con 0x7f14040cd970 2023-10-16T20:05:59.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.476+0000 7f1400ff9640 1 -- 172.21.15.88:0/3105537947 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7f14040b67e0 con 0x7f14040cd970 2023-10-16T20:05:59.480 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2023-10-16T20:05:59.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.476+0000 7f1409980640 1 -- 172.21.15.88:0/3105537947 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f13e40664a0 msgr2=0x7f13e4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:59.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.476+0000 7f1409980640 1 --2- 172.21.15.88:0/3105537947 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f13e40664a0 0x7f13e4068960 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f14040ce5f0 tx=0x7f13ec009040 comp rx=0 tx=0).stop 2023-10-16T20:05:59.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.476+0000 7f1409980640 1 -- 172.21.15.88:0/3105537947 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14040cd970 msgr2=0x7f14040cc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:59.482 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.476+0000 7f1409980640 1 --2- 172.21.15.88:0/3105537947 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14040cd970 0x7f14040cc660 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f13f401a140 tx=0x7f13f401ad50 comp rx=0 tx=0).stop 2023-10-16T20:05:59.482 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.476+0000 7f1409980640 1 -- 172.21.15.88:0/3105537947 shutdown_connections 2023-10-16T20:05:59.482 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.476+0000 7f1409980640 1 --2- 172.21.15.88:0/3105537947 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f13e40664a0 0x7f13e4068960 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:59.482 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.476+0000 7f1409980640 1 --2- 172.21.15.88:0/3105537947 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14040d2660 0x7f14040c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:59.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.476+0000 7f1409980640 1 --2- 172.21.15.88:0/3105537947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1404153ea0 0x7f14040b7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:59.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.476+0000 7f1409980640 1 --2- 172.21.15.88:0/3105537947 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14040cd970 0x7f14040cc660 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:59.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.476+0000 7f1409980640 1 -- 172.21.15.88:0/3105537947 >> 172.21.15.88:0/3105537947 conn(0x7f14040715c0 msgr2=0x7f14040d2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:59.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.476+0000 7f1409980640 1 -- 172.21.15.88:0/3105537947 shutdown_connections 2023-10-16T20:05:59.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.476+0000 7f1409980640 1 -- 172.21.15.88:0/3105537947 wait complete. 2023-10-16T20:05:59.500 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:05:59.501 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:05:59.511 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:05:59.648 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.644+0000 7f9c30580640 1 Processor -- start 2023-10-16T20:05:59.648 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.644+0000 7f9c30580640 1 -- start start 2023-10-16T20:05:59.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.644+0000 7f9c30580640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f9c280d22e0 0x7f9c28161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:59.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.644+0000 7f9c30580640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c280b87e0 con 0x7f9c280d26c0 2023-10-16T20:05:59.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.644+0000 7f9c30580640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c2815df10 con 0x7f9c2815d9f0 2023-10-16T20:05:59.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.644+0000 7f9c30580640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c2815e1f0 con 0x7f9c280d22e0 2023-10-16T20:05:59.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.644+0000 7f9c2db1b640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f9c280d26c0 0x7f9c2815d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:53568/0 (socket says 172.21.15.88:53568) 2023-10-16T20:05:59.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.644+0000 7f9c2db1b640 1 -- 172.21.15.88:0/2281230414 learned_addr learned my addr 172.21.15.88:0/2281230414 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:59.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.644+0000 7f9c2e31c640 1 --2- 172.21.15.88:0/2281230414 >> v2:172.21.15.94:3300/0 conn(0x7f9c280d22e0 0x7f9c28161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:59.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.644+0000 7f9c2d31a640 1 -- 172.21.15.88:0/2281230414 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1618093349 0 0) 0x7f9c280b87e0 con 0x7f9c280d26c0 2023-10-16T20:05:59.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.644+0000 7f9c2d31a640 1 -- 172.21.15.88:0/2281230414 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c14003530 con 0x7f9c280d26c0 2023-10-16T20:05:59.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.644+0000 7f9c2d31a640 1 -- 172.21.15.88:0/2281230414 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3660726756 0 0) 0x7f9c14003530 con 0x7f9c280d26c0 2023-10-16T20:05:59.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.644+0000 7f9c2d31a640 1 -- 172.21.15.88:0/2281230414 >> v2:172.21.15.94:3300/0 conn(0x7f9c280d22e0 msgr2=0x7f9c28161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:59.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.644+0000 7f9c2d31a640 1 --2- 172.21.15.88:0/2281230414 >> v2:172.21.15.94:3300/0 conn(0x7f9c280d22e0 0x7f9c28161450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:59.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.644+0000 7f9c2d31a640 1 -- 172.21.15.88:0/2281230414 >> v1:172.21.15.94:6789/0 conn(0x7f9c2815d9f0 legacy=0x7f9c280cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:59.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.644+0000 7f9c2d31a640 1 -- 172.21.15.88:0/2281230414 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c28160970 con 0x7f9c280d26c0 2023-10-16T20:05:59.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.644+0000 7f9c2d31a640 1 -- 172.21.15.88:0/2281230414 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f9c180016a0 con 0x7f9c280d26c0 2023-10-16T20:05:59.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.644+0000 7f9c2d31a640 1 -- 172.21.15.88:0/2281230414 >> v1:172.21.15.88:6789/0 conn(0x7f9c280d26c0 legacy=0x7f9c2815d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:59.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.644+0000 7f9c2d31a640 1 --2- 172.21.15.88:0/2281230414 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c14003980 0x7f9c14003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:59.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.644+0000 7f9c2d31a640 1 --2- 172.21.15.88:0/2281230414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c140047d0 0x7f9c14004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:59.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.644+0000 7f9c2d31a640 1 --2- 172.21.15.88:0/2281230414 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9c14005190 0x7f9c1400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:59.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.648+0000 7f9c2d31a640 1 -- 172.21.15.88:0/2281230414 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2815e1f0 con 0x7f9c14003980 2023-10-16T20:05:59.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.648+0000 7f9c2d31a640 1 -- 172.21.15.88:0/2281230414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1400c130 con 0x7f9c140047d0 2023-10-16T20:05:59.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.648+0000 7f9c2d31a640 1 -- 172.21.15.88:0/2281230414 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1400c2f0 con 0x7f9c14005190 2023-10-16T20:05:59.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.648+0000 7f9c2e31c640 1 --2- 172.21.15.88:0/2281230414 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c14003980 0x7f9c14003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:59.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.648+0000 7f9c2db1b640 1 --2- 172.21.15.88:0/2281230414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c140047d0 0x7f9c14004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:59.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.648+0000 7f9c2eb1d640 1 --2- 172.21.15.88:0/2281230414 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9c14005190 0x7f9c1400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:59.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.648+0000 7f9c2e31c640 1 -- 172.21.15.88:0/2281230414 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9c14005190 msgr2=0x7f9c1400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:59.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.648+0000 7f9c2e31c640 1 --2- 172.21.15.88:0/2281230414 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9c14005190 0x7f9c1400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:59.657 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.648+0000 7f9c2e31c640 1 -- 172.21.15.88:0/2281230414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c140047d0 msgr2=0x7f9c14004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:59.657 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.648+0000 7f9c2e31c640 1 --2- 172.21.15.88:0/2281230414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c140047d0 0x7f9c14004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:59.657 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.648+0000 7f9c2e31c640 1 -- 172.21.15.88:0/2281230414 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c14003530 con 0x7f9c14003980 2023-10-16T20:05:59.657 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.648+0000 7f9c2eb1d640 1 -- 172.21.15.88:0/2281230414 reap_dead start 2023-10-16T20:05:59.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.648+0000 7f9c2e31c640 1 --2- 172.21.15.88:0/2281230414 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c14003980 0x7f9c14003da0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009ae0 tx=0x7f9c2401ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2431306021f9c42c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:59.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.648+0000 7f9c2d31a640 1 -- 172.21.15.88:0/2281230414 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c240a6510 con 0x7f9c14003980 2023-10-16T20:05:59.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.648+0000 7f9c2d31a640 1 -- 172.21.15.88:0/2281230414 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c240a6670 con 0x7f9c14003980 2023-10-16T20:05:59.661 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.648+0000 7f9c2d31a640 1 -- 172.21.15.88:0/2281230414 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c240a6a20 con 0x7f9c14003980 2023-10-16T20:05:59.661 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.652+0000 7f9c30580640 1 -- 172.21.15.88:0/2281230414 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c14003980 msgr2=0x7f9c14003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:59.661 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.652+0000 7f9c30580640 1 --2- 172.21.15.88:0/2281230414 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c14003980 0x7f9c14003da0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009ae0 tx=0x7f9c2401ca20 comp rx=0 tx=0).stop 2023-10-16T20:05:59.661 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.652+0000 7f9c30580640 1 -- 172.21.15.88:0/2281230414 shutdown_connections 2023-10-16T20:05:59.661 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.652+0000 7f9c30580640 1 --2- 172.21.15.88:0/2281230414 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c14003980 0x7f9c14003da0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:59.662 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.652+0000 7f9c30580640 1 -- 172.21.15.88:0/2281230414 >> 172.21.15.88:0/2281230414 conn(0x7f9c280715c0 msgr2=0x7f9c280719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:59.662 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.652+0000 7f9c30580640 1 -- 172.21.15.88:0/2281230414 shutdown_connections 2023-10-16T20:05:59.662 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.652+0000 7f9c30580640 1 -- 172.21.15.88:0/2281230414 wait complete. 2023-10-16T20:05:59.662 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.652+0000 7f9c30580640 1 Processor -- start 2023-10-16T20:05:59.662 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.652+0000 7f9c30580640 1 -- start start 2023-10-16T20:05:59.663 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.652+0000 7f9c30580640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c14003980 0x7f9c280c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:59.663 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.652+0000 7f9c30580640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9c280c8690 0x7f9c2814e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:59.663 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.652+0000 7f9c30580640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c2814e9b0 0x7f9c280c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:59.663 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.652+0000 7f9c30580640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c280c8b10 con 0x7f9c14003980 2023-10-16T20:05:59.663 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.652+0000 7f9c30580640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c281f9c70 con 0x7f9c2814e9b0 2023-10-16T20:05:59.664 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.652+0000 7f9c30580640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c281f9060 con 0x7f9c280c8690 2023-10-16T20:05:59.664 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.652+0000 7f9c2e31c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c14003980 0x7f9c280c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:59.664 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.652+0000 7f9c2e31c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c14003980 0x7f9c280c8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50836/0 (socket says 172.21.15.88:50836) 2023-10-16T20:05:59.664 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.652+0000 7f9c2e31c640 1 -- 172.21.15.88:0/1513467173 learned_addr learned my addr 172.21.15.88:0/1513467173 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:59.664 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.652+0000 7f9c2db1b640 1 --2- 172.21.15.88:0/1513467173 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9c280c8690 0x7f9c2814e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:59.665 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.652+0000 7f9c2eb1d640 1 --2- 172.21.15.88:0/1513467173 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c2814e9b0 0x7f9c280c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:59.665 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.652+0000 7f9c2e31c640 1 -- 172.21.15.88:0/1513467173 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9c280c8690 msgr2=0x7f9c2814e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:59.665 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.652+0000 7f9c2e31c640 1 --2- 172.21.15.88:0/1513467173 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9c280c8690 0x7f9c2814e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:59.665 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.652+0000 7f9c2e31c640 1 -- 172.21.15.88:0/1513467173 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c2814e9b0 msgr2=0x7f9c280c9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:59.665 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.652+0000 7f9c2e31c640 1 --2- 172.21.15.88:0/1513467173 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c2814e9b0 0x7f9c280c9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:59.665 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.652+0000 7f9c2e31c640 1 -- 172.21.15.88:0/1513467173 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c281f9340 con 0x7f9c14003980 2023-10-16T20:05:59.666 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.656+0000 7f9c2e31c640 1 --2- 172.21.15.88:0/1513467173 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c14003980 0x7f9c280c8150 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9c240b0040 tx=0x7f9c240197c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:59.666 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.656+0000 7f9c1effd640 1 -- 172.21.15.88:0/1513467173 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c240a67e0 con 0x7f9c14003980 2023-10-16T20:05:59.666 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.656+0000 7f9c1effd640 1 -- 172.21.15.88:0/1513467173 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c240a6940 con 0x7f9c14003980 2023-10-16T20:05:59.666 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.656+0000 7f9c30580640 1 -- 172.21.15.88:0/1513467173 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c281f9620 con 0x7f9c14003980 2023-10-16T20:05:59.666 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.656+0000 7f9c1effd640 1 -- 172.21.15.88:0/1513467173 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c240a6cb0 con 0x7f9c14003980 2023-10-16T20:05:59.667 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.656+0000 7f9c30580640 1 -- 172.21.15.88:0/1513467173 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c280cb0c0 con 0x7f9c14003980 2023-10-16T20:05:59.667 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.656+0000 7f9c30580640 1 -- 172.21.15.88:0/1513467173 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c2804e9c0 con 0x7f9c14003980 2023-10-16T20:05:59.668 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.656+0000 7f9c1effd640 1 -- 172.21.15.88:0/1513467173 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9c24006040 con 0x7f9c14003980 2023-10-16T20:05:59.668 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.656+0000 7f9c1effd640 1 --2- 172.21.15.88:0/1513467173 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9c0c0664a0 0x7f9c0c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:59.668 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.656+0000 7f9c1effd640 1 -- 172.21.15.88:0/1513467173 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9c240aa070 con 0x7f9c14003980 2023-10-16T20:05:59.669 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.656+0000 7f9c2db1b640 1 --2- 172.21.15.88:0/1513467173 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9c0c0664a0 0x7f9c0c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:59.669 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.660+0000 7f9c2db1b640 1 --2- 172.21.15.88:0/1513467173 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9c0c0664a0 0x7f9c0c068960 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9c280c96d0 tx=0x7f9c18000f90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:59.669 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.660+0000 7f9c1effd640 1 -- 172.21.15.88:0/1513467173 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9c240ebb20 con 0x7f9c14003980 2023-10-16T20:05:59.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.812+0000 7f9c30580640 1 -- 172.21.15.88:0/1513467173 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c2815e330 con 0x7f9c14003980 2023-10-16T20:05:59.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.812+0000 7f9c1effd640 1 -- 172.21.15.88:0/1513467173 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18661 (secure 0 0 0) 0x7f9c240ef140 con 0x7f9c14003980 2023-10-16T20:05:59.817 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:05:59.817 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":29,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:05:38.202959+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T20:04:54.442487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T20:04:55.766641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T20:04:59.475830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":1}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:05:59.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.812+0000 7f9c30580640 1 -- 172.21.15.88:0/1513467173 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9c0c0664a0 msgr2=0x7f9c0c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:59.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.812+0000 7f9c30580640 1 --2- 172.21.15.88:0/1513467173 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9c0c0664a0 0x7f9c0c068960 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9c280c96d0 tx=0x7f9c18000f90 comp rx=0 tx=0).stop 2023-10-16T20:05:59.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.812+0000 7f9c30580640 1 -- 172.21.15.88:0/1513467173 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c14003980 msgr2=0x7f9c280c8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:59.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.812+0000 7f9c30580640 1 --2- 172.21.15.88:0/1513467173 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c14003980 0x7f9c280c8150 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9c240b0040 tx=0x7f9c240197c0 comp rx=0 tx=0).stop 2023-10-16T20:05:59.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.816+0000 7f9c30580640 1 -- 172.21.15.88:0/1513467173 shutdown_connections 2023-10-16T20:05:59.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.816+0000 7f9c30580640 1 --2- 172.21.15.88:0/1513467173 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9c0c0664a0 0x7f9c0c068960 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:59.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.816+0000 7f9c30580640 1 --2- 172.21.15.88:0/1513467173 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9c280c8690 0x7f9c2814e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:59.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.816+0000 7f9c30580640 1 --2- 172.21.15.88:0/1513467173 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c2814e9b0 0x7f9c280c9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:59.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.816+0000 7f9c30580640 1 --2- 172.21.15.88:0/1513467173 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c14003980 0x7f9c280c8150 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:59.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.816+0000 7f9c30580640 1 -- 172.21.15.88:0/1513467173 >> 172.21.15.88:0/1513467173 conn(0x7f9c280715c0 msgr2=0x7f9c280ce160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:59.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.816+0000 7f9c30580640 1 -- 172.21.15.88:0/1513467173 shutdown_connections 2023-10-16T20:05:59.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.816+0000 7f9c30580640 1 -- 172.21.15.88:0/1513467173 wait complete. 2023-10-16T20:05:59.838 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T20:04:54.442487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T20:04:55.766641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T20:04:59.475830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T20:05:59.839 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:05:59.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff4961d9640 1 Processor -- start 2023-10-16T20:05:59.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff4961d9640 1 -- start start 2023-10-16T20:05:59.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff4961d9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4900d3110 0x7ff4900b5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:59.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff4961d9640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4900cda20 con 0x7ff4900d0760 2023-10-16T20:05:59.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff4961d9640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4900cdc80 con 0x7ff49015c930 2023-10-16T20:05:59.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff4961d9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4900cdf60 con 0x7ff4900d3110 2023-10-16T20:05:59.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff48effd640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7ff4900d0760 0x7ff4900d2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:53578/0 (socket says 172.21.15.88:53578) 2023-10-16T20:05:59.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff48effd640 1 -- 172.21.15.88:0/3552439803 learned_addr learned my addr 172.21.15.88:0/3552439803 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:05:59.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff48ffff640 1 --2- 172.21.15.88:0/3552439803 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4900d3110 0x7ff4900b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:59.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff48e7fc640 1 -- 172.21.15.88:0/3552439803 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3552559768 0 0) 0x7ff4900cda20 con 0x7ff4900d0760 2023-10-16T20:05:59.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff48e7fc640 1 -- 172.21.15.88:0/3552439803 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff478003530 con 0x7ff4900d0760 2023-10-16T20:05:59.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff48e7fc640 1 -- 172.21.15.88:0/3552439803 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1050143369 0 0) 0x7ff4900cdc80 con 0x7ff49015c930 2023-10-16T20:05:59.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff48e7fc640 1 -- 172.21.15.88:0/3552439803 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4900cda20 con 0x7ff49015c930 2023-10-16T20:05:59.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff48e7fc640 1 -- 172.21.15.88:0/3552439803 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 805787726 0 0) 0x7ff478003530 con 0x7ff4900d0760 2023-10-16T20:05:59.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff48e7fc640 1 -- 172.21.15.88:0/3552439803 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4900d3110 msgr2=0x7ff4900b5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:59.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff48e7fc640 1 --2- 172.21.15.88:0/3552439803 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4900d3110 0x7ff4900b5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:59.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff48e7fc640 1 -- 172.21.15.88:0/3552439803 >> v1:172.21.15.94:6789/0 conn(0x7ff49015c930 legacy=0x7ff4900cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:59.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff48e7fc640 1 -- 172.21.15.88:0/3552439803 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4900b4530 con 0x7ff4900d0760 2023-10-16T20:05:59.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff48e7fc640 1 -- 172.21.15.88:0/3552439803 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7ff4840016a0 con 0x7ff4900d0760 2023-10-16T20:05:59.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff48e7fc640 1 -- 172.21.15.88:0/3552439803 >> v1:172.21.15.88:6789/0 conn(0x7ff4900d0760 legacy=0x7ff4900d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:59.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff48e7fc640 1 --2- 172.21.15.88:0/3552439803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff478003b40 0x7ff478003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:59.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff48e7fc640 1 --2- 172.21.15.88:0/3552439803 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4780047d0 0x7ff478004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:59.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff48e7fc640 1 --2- 172.21.15.88:0/3552439803 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4900d3110 0x7ff478005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:59.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff48e7fc640 1 -- 172.21.15.88:0/3552439803 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4900cdf60 con 0x7ff4780047d0 2023-10-16T20:05:59.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff48e7fc640 1 -- 172.21.15.88:0/3552439803 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47800ba70 con 0x7ff478003b40 2023-10-16T20:05:59.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff48e7fc640 1 -- 172.21.15.88:0/3552439803 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47800bc30 con 0x7ff4900d3110 2023-10-16T20:05:59.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff48effd640 1 --2- 172.21.15.88:0/3552439803 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4780047d0 0x7ff478004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:59.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff48f7fe640 1 --2- 172.21.15.88:0/3552439803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff478003b40 0x7ff478003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:59.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff48ffff640 1 --2- 172.21.15.88:0/3552439803 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4900d3110 0x7ff478005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:59.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff48effd640 1 -- 172.21.15.88:0/3552439803 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4900d3110 msgr2=0x7ff478005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:59.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff48effd640 1 --2- 172.21.15.88:0/3552439803 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4900d3110 0x7ff478005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:59.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff48effd640 1 -- 172.21.15.88:0/3552439803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff478003b40 msgr2=0x7ff478003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:05:59.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff48effd640 1 --2- 172.21.15.88:0/3552439803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff478003b40 0x7ff478003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:59.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.976+0000 7ff48effd640 1 -- 172.21.15.88:0/3552439803 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff478003530 con 0x7ff4780047d0 2023-10-16T20:05:59.990 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.980+0000 7ff48effd640 1 --2- 172.21.15.88:0/3552439803 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4780047d0 0x7ff478004c50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff484001ec0 tx=0x7ff48401cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e0f41cb9f1c081 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:59.991 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.980+0000 7ff48e7fc640 1 -- 172.21.15.88:0/3552439803 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4840058e0 con 0x7ff4780047d0 2023-10-16T20:05:59.991 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.980+0000 7ff48e7fc640 1 -- 172.21.15.88:0/3552439803 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff484005a40 con 0x7ff4780047d0 2023-10-16T20:05:59.991 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.980+0000 7ff48e7fc640 1 -- 172.21.15.88:0/3552439803 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff484005df0 con 0x7ff4780047d0 2023-10-16T20:05:59.991 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.980+0000 7ff4961d9640 1 -- 172.21.15.88:0/3552439803 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4780047d0 msgr2=0x7ff478004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:59.991 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.980+0000 7ff4961d9640 1 --2- 172.21.15.88:0/3552439803 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4780047d0 0x7ff478004c50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff484001ec0 tx=0x7ff48401cb40 comp rx=0 tx=0).stop 2023-10-16T20:05:59.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.980+0000 7ff48ffff640 1 -- 172.21.15.88:0/3552439803 reap_dead start 2023-10-16T20:05:59.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.980+0000 7ff4961d9640 1 -- 172.21.15.88:0/3552439803 shutdown_connections 2023-10-16T20:05:59.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.980+0000 7ff4961d9640 1 -- 172.21.15.88:0/3552439803 >> 172.21.15.88:0/3552439803 conn(0x7ff4900719c0 msgr2=0x7ff490071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:05:59.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.980+0000 7ff4961d9640 1 -- 172.21.15.88:0/3552439803 shutdown_connections 2023-10-16T20:05:59.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.980+0000 7ff4961d9640 1 -- 172.21.15.88:0/3552439803 wait complete. 2023-10-16T20:05:59.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.984+0000 7ff4961d9640 1 Processor -- start 2023-10-16T20:05:59.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.984+0000 7ff4961d9640 1 -- start start 2023-10-16T20:05:59.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.984+0000 7ff4961d9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4780047d0 0x7ff490159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:59.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.984+0000 7ff4961d9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4900c9050 0x7ff4900cc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:59.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.984+0000 7ff4961d9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4900ca8b0 0x7ff4900cad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:59.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.984+0000 7ff4961d9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4900cc680 con 0x7ff4780047d0 2023-10-16T20:05:59.994 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.984+0000 7ff4961d9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4900c9b80 con 0x7ff4900c9050 2023-10-16T20:05:59.994 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.984+0000 7ff4961d9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4900ca420 con 0x7ff4900ca8b0 2023-10-16T20:05:59.994 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.984+0000 7ff48f7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4780047d0 0x7ff490159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:59.994 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.984+0000 7ff48f7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4780047d0 0x7ff490159960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50848/0 (socket says 172.21.15.88:50848) 2023-10-16T20:05:59.994 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.984+0000 7ff48f7fe640 1 -- 172.21.15.88:0/2986384037 learned_addr learned my addr 172.21.15.88:0/2986384037 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:05:59.995 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.984+0000 7ff48ffff640 1 --2- 172.21.15.88:0/2986384037 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4900ca8b0 0x7ff4900cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:59.995 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.984+0000 7ff48effd640 1 --2- 172.21.15.88:0/2986384037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4900c9050 0x7ff4900cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:59.995 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.984+0000 7ff48f7fe640 1 -- 172.21.15.88:0/2986384037 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4900ca8b0 msgr2=0x7ff4900cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:59.995 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.984+0000 7ff48f7fe640 1 --2- 172.21.15.88:0/2986384037 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4900ca8b0 0x7ff4900cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:59.995 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.984+0000 7ff48f7fe640 1 -- 172.21.15.88:0/2986384037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4900c9050 msgr2=0x7ff4900cc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:05:59.995 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.984+0000 7ff48f7fe640 1 --2- 172.21.15.88:0/2986384037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4900c9050 0x7ff4900cc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:05:59.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.984+0000 7ff48f7fe640 1 -- 172.21.15.88:0/2986384037 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4900c94d0 con 0x7ff4780047d0 2023-10-16T20:05:59.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.984+0000 7ff48effd640 1 --2- 172.21.15.88:0/2986384037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4900c9050 0x7ff4900cc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:59.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.984+0000 7ff48ffff640 1 --2- 172.21.15.88:0/2986384037 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4900ca8b0 0x7ff4900cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:05:59.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.984+0000 7ff48f7fe640 1 --2- 172.21.15.88:0/2986384037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4780047d0 0x7ff490159960 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff480012800 tx=0x7ff480012cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:59.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.984+0000 7ff477fff640 1 -- 172.21.15.88:0/2986384037 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff480006ff0 con 0x7ff4780047d0 2023-10-16T20:05:59.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.984+0000 7ff477fff640 1 -- 172.21.15.88:0/2986384037 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff480010b90 con 0x7ff4780047d0 2023-10-16T20:05:59.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.984+0000 7ff4961d9640 1 -- 172.21.15.88:0/2986384037 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4900cc870 con 0x7ff4780047d0 2023-10-16T20:05:59.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.984+0000 7ff477fff640 1 -- 172.21.15.88:0/2986384037 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff480016480 con 0x7ff4780047d0 2023-10-16T20:05:59.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.984+0000 7ff4961d9640 1 -- 172.21.15.88:0/2986384037 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4900ca0b0 con 0x7ff4780047d0 2023-10-16T20:05:59.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.984+0000 7ff4961d9640 1 -- 172.21.15.88:0/2986384037 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff49004e9c0 con 0x7ff4780047d0 2023-10-16T20:05:59.998 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.988+0000 7ff477fff640 1 -- 172.21.15.88:0/2986384037 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff4800165e0 con 0x7ff4780047d0 2023-10-16T20:05:59.998 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.988+0000 7ff477fff640 1 --2- 172.21.15.88:0/2986384037 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff47c0664a0 0x7ff47c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:05:59.999 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.988+0000 7ff477fff640 1 -- 172.21.15.88:0/2986384037 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff480087ac0 con 0x7ff4780047d0 2023-10-16T20:05:59.999 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.988+0000 7ff48effd640 1 --2- 172.21.15.88:0/2986384037 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff47c0664a0 0x7ff47c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:05:59.999 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.988+0000 7ff48effd640 1 --2- 172.21.15.88:0/2986384037 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff47c0664a0 0x7ff47c068960 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff4840aa000 tx=0x7ff484001a20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:05:59.999 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:05:59.992+0000 7ff477fff640 1 -- 172.21.15.88:0/2986384037 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff480054ae0 con 0x7ff4780047d0 2023-10-16T20:06:00.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.140+0000 7ff4961d9640 1 -- 172.21.15.88:0/2986384037 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff4900b4310 con 0x7ff4780047d0 2023-10-16T20:06:00.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.140+0000 7ff477fff640 1 -- 172.21.15.88:0/2986384037 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff480058100 con 0x7ff4780047d0 2023-10-16T20:06:00.144 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:06:00.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.140+0000 7ff4961d9640 1 -- 172.21.15.88:0/2986384037 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff47c0664a0 msgr2=0x7ff47c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:00.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.140+0000 7ff4961d9640 1 --2- 172.21.15.88:0/2986384037 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff47c0664a0 0x7ff47c068960 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff4840aa000 tx=0x7ff484001a20 comp rx=0 tx=0).stop 2023-10-16T20:06:00.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.140+0000 7ff4961d9640 1 -- 172.21.15.88:0/2986384037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4780047d0 msgr2=0x7ff490159960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:00.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.140+0000 7ff4961d9640 1 --2- 172.21.15.88:0/2986384037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4780047d0 0x7ff490159960 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff480012800 tx=0x7ff480012cc0 comp rx=0 tx=0).stop 2023-10-16T20:06:00.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.140+0000 7ff4961d9640 1 -- 172.21.15.88:0/2986384037 shutdown_connections 2023-10-16T20:06:00.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.140+0000 7ff4961d9640 1 --2- 172.21.15.88:0/2986384037 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff47c0664a0 0x7ff47c068960 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:00.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.140+0000 7ff4961d9640 1 --2- 172.21.15.88:0/2986384037 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4900ca8b0 0x7ff4900cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:00.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.140+0000 7ff4961d9640 1 --2- 172.21.15.88:0/2986384037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4900c9050 0x7ff4900cc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:00.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.140+0000 7ff4961d9640 1 --2- 172.21.15.88:0/2986384037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4780047d0 0x7ff490159960 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:00.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.140+0000 7ff4961d9640 1 -- 172.21.15.88:0/2986384037 >> 172.21.15.88:0/2986384037 conn(0x7ff4900719c0 msgr2=0x7ff49015f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:00.149 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.140+0000 7ff4961d9640 1 -- 172.21.15.88:0/2986384037 shutdown_connections 2023-10-16T20:06:00.149 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.140+0000 7ff4961d9640 1 -- 172.21.15.88:0/2986384037 wait complete. 2023-10-16T20:06:00.165 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T20:06:00.306 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.300+0000 7f7a5546b640 1 Processor -- start 2023-10-16T20:06:00.307 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.300+0000 7f7a5546b640 1 -- start start 2023-10-16T20:06:00.308 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.304+0000 7f7a5546b640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f7a500ce0b0 0x7f7a5015de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:00.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.304+0000 7f7a5546b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7a50157510 0x7f7a50160860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:00.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.304+0000 7f7a5546b640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a50161470 con 0x7f7a50157130 2023-10-16T20:06:00.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.304+0000 7f7a5546b640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a500b88a0 con 0x7f7a500ce0b0 2023-10-16T20:06:00.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.304+0000 7f7a5546b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a50161750 con 0x7f7a50157510 2023-10-16T20:06:00.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.304+0000 7f7a4e7fc640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f7a500ce0b0 0x7f7a5015de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:00.310 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.304+0000 7f7a4e7fc640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f7a500ce0b0 0x7f7a5015de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50850/0 (socket says 172.21.15.88:50850) 2023-10-16T20:06:00.310 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.304+0000 7f7a4effd640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f7a50157130 0x7f7a500cd9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:53594/0 (socket says 172.21.15.88:53594) 2023-10-16T20:06:00.310 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.304+0000 7f7a4e7fc640 1 -- 172.21.15.88:0/2918311245 learned_addr learned my addr 172.21.15.88:0/2918311245 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:00.310 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.304+0000 7f7a4f7fe640 1 --2- 172.21.15.88:0/2918311245 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7a50157510 0x7f7a50160860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:00.310 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.304+0000 7f7a4e7fc640 1 -- 172.21.15.88:0/2918311245 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7a50157510 msgr2=0x7f7a50160860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:00.311 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.304+0000 7f7a4e7fc640 1 --2- 172.21.15.88:0/2918311245 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7a50157510 0x7f7a50160860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:00.311 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.304+0000 7f7a4e7fc640 1 -- 172.21.15.88:0/2918311245 >> v1:172.21.15.88:6789/0 conn(0x7f7a50157130 legacy=0x7f7a500cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:00.311 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.304+0000 7f7a4e7fc640 1 -- 172.21.15.88:0/2918311245 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5015d380 con 0x7f7a500ce0b0 2023-10-16T20:06:00.311 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.304+0000 7f7a4f7fe640 1 --2- 172.21.15.88:0/2918311245 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7a50157510 0x7f7a50160860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:06:00.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.308+0000 7f7a4e7fc640 1 --2- 172.21.15.88:0/2918311245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a500ce0b0 0x7f7a5015de60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7a44009870 tx=0x7f7a4401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f67fe35eb62d9e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:00.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.308+0000 7f7a4dffb640 1 -- 172.21.15.88:0/2918311245 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a440a95f0 con 0x7f7a500ce0b0 2023-10-16T20:06:00.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.308+0000 7f7a4dffb640 1 -- 172.21.15.88:0/2918311245 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a440a9750 con 0x7f7a500ce0b0 2023-10-16T20:06:00.313 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.308+0000 7f7a4dffb640 1 -- 172.21.15.88:0/2918311245 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a440a9ae0 con 0x7f7a500ce0b0 2023-10-16T20:06:00.314 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.308+0000 7f7a5546b640 1 -- 172.21.15.88:0/2918311245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a500ce0b0 msgr2=0x7f7a5015de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:00.314 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.308+0000 7f7a5546b640 1 --2- 172.21.15.88:0/2918311245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a500ce0b0 0x7f7a5015de60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7a44009870 tx=0x7f7a4401c510 comp rx=0 tx=0).stop 2023-10-16T20:06:00.314 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.308+0000 7f7a5546b640 1 -- 172.21.15.88:0/2918311245 shutdown_connections 2023-10-16T20:06:00.314 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.308+0000 7f7a5546b640 1 --2- 172.21.15.88:0/2918311245 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7a50157510 0x7f7a50160860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:00.314 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.308+0000 7f7a5546b640 1 --2- 172.21.15.88:0/2918311245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a500ce0b0 0x7f7a5015de60 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:00.315 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.308+0000 7f7a5546b640 1 -- 172.21.15.88:0/2918311245 >> 172.21.15.88:0/2918311245 conn(0x7f7a500715c0 msgr2=0x7f7a500719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:00.315 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.308+0000 7f7a5546b640 1 -- 172.21.15.88:0/2918311245 shutdown_connections 2023-10-16T20:06:00.315 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.308+0000 7f7a5546b640 1 -- 172.21.15.88:0/2918311245 wait complete. 2023-10-16T20:06:00.317 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.312+0000 7f7a5546b640 1 Processor -- start 2023-10-16T20:06:00.317 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.312+0000 7f7a5546b640 1 -- start start 2023-10-16T20:06:00.318 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.312+0000 7f7a5546b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a500ce0b0 0x7f7a500c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:00.318 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.312+0000 7f7a5546b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7a50157130 0x7f7a5014f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:00.318 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.312+0000 7f7a5546b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a50157510 0x7f7a500c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:00.318 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.312+0000 7f7a5546b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a500c7920 con 0x7f7a50157510 2023-10-16T20:06:00.318 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.312+0000 7f7a5546b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a500b3540 con 0x7f7a500ce0b0 2023-10-16T20:06:00.319 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.312+0000 7f7a5546b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a500b2ca0 con 0x7f7a50157130 2023-10-16T20:06:00.319 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.312+0000 7f7a4f7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a50157510 0x7f7a500c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:00.319 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.312+0000 7f7a4f7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a50157510 0x7f7a500c8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50856/0 (socket says 172.21.15.88:50856) 2023-10-16T20:06:00.319 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.312+0000 7f7a4f7fe640 1 -- 172.21.15.88:0/1967115594 learned_addr learned my addr 172.21.15.88:0/1967115594 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:00.320 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.312+0000 7f7a4effd640 1 --2- 172.21.15.88:0/1967115594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a500ce0b0 0x7f7a500c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:00.320 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.312+0000 7f7a4e7fc640 1 --2- 172.21.15.88:0/1967115594 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7a50157130 0x7f7a5014f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:00.320 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.312+0000 7f7a4f7fe640 1 -- 172.21.15.88:0/1967115594 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7a50157130 msgr2=0x7f7a5014f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:00.320 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.312+0000 7f7a4f7fe640 1 --2- 172.21.15.88:0/1967115594 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7a50157130 0x7f7a5014f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:00.320 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.312+0000 7f7a4f7fe640 1 -- 172.21.15.88:0/1967115594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a500ce0b0 msgr2=0x7f7a500c9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:00.320 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.312+0000 7f7a4f7fe640 1 --2- 172.21.15.88:0/1967115594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a500ce0b0 0x7f7a500c9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:00.321 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.312+0000 7f7a4f7fe640 1 -- 172.21.15.88:0/1967115594 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a48009a70 con 0x7f7a50157510 2023-10-16T20:06:00.321 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.312+0000 7f7a4f7fe640 1 --2- 172.21.15.88:0/1967115594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a50157510 0x7f7a500c8f00 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009e10 tx=0x7f7a4800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:00.321 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.312+0000 7f7a377fe640 1 -- 172.21.15.88:0/1967115594 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a48009cb0 con 0x7f7a50157510 2023-10-16T20:06:00.321 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.312+0000 7f7a377fe640 1 -- 172.21.15.88:0/1967115594 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4800f040 con 0x7f7a50157510 2023-10-16T20:06:00.321 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.312+0000 7f7a377fe640 1 -- 172.21.15.88:0/1967115594 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a480146a0 con 0x7f7a50157510 2023-10-16T20:06:00.322 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.312+0000 7f7a5546b640 1 -- 172.21.15.88:0/1967115594 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a440094d0 con 0x7f7a50157510 2023-10-16T20:06:00.322 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.312+0000 7f7a5546b640 1 -- 172.21.15.88:0/1967115594 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a500b3260 con 0x7f7a50157510 2023-10-16T20:06:00.323 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.312+0000 7f7a5546b640 1 -- 172.21.15.88:0/1967115594 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a500c7a90 con 0x7f7a50157510 2023-10-16T20:06:00.323 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.316+0000 7f7a377fe640 1 -- 172.21.15.88:0/1967115594 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7a48014800 con 0x7f7a50157510 2023-10-16T20:06:00.323 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.316+0000 7f7a377fe640 1 --2- 172.21.15.88:0/1967115594 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7a3c0664a0 0x7f7a3c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:00.323 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.316+0000 7f7a377fe640 1 -- 172.21.15.88:0/1967115594 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7a480850d0 con 0x7f7a50157510 2023-10-16T20:06:00.323 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.316+0000 7f7a4effd640 1 --2- 172.21.15.88:0/1967115594 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7a3c0664a0 0x7f7a3c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:00.324 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.316+0000 7f7a4effd640 1 --2- 172.21.15.88:0/1967115594 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7a3c0664a0 0x7f7a3c068960 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7a40000f70 tx=0x7f7a40007e50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:00.324 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.320+0000 7f7a377fe640 1 -- 172.21.15.88:0/1967115594 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7a480521a0 con 0x7f7a50157510 2023-10-16T20:06:00.476 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.472+0000 7f7a5546b640 1 -- 172.21.15.88:0/1967115594 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7a5004e9c0 con 0x7f7a50157510 2023-10-16T20:06:00.476 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.472+0000 7f7a377fe640 1 -- 172.21.15.88:0/1967115594 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7a480557c0 con 0x7f7a50157510 2023-10-16T20:06:00.477 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:06:00.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.472+0000 7f7a5546b640 1 -- 172.21.15.88:0/1967115594 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7a3c0664a0 msgr2=0x7f7a3c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:00.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.472+0000 7f7a5546b640 1 --2- 172.21.15.88:0/1967115594 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7a3c0664a0 0x7f7a3c068960 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7a40000f70 tx=0x7f7a40007e50 comp rx=0 tx=0).stop 2023-10-16T20:06:00.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.472+0000 7f7a5546b640 1 -- 172.21.15.88:0/1967115594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a50157510 msgr2=0x7f7a500c8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:00.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.472+0000 7f7a5546b640 1 --2- 172.21.15.88:0/1967115594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a50157510 0x7f7a500c8f00 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009e10 tx=0x7f7a4800c490 comp rx=0 tx=0).stop 2023-10-16T20:06:00.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.472+0000 7f7a5546b640 1 -- 172.21.15.88:0/1967115594 shutdown_connections 2023-10-16T20:06:00.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.472+0000 7f7a5546b640 1 --2- 172.21.15.88:0/1967115594 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7a3c0664a0 0x7f7a3c068960 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:00.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.472+0000 7f7a5546b640 1 --2- 172.21.15.88:0/1967115594 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7a50157130 0x7f7a5014f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:00.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.472+0000 7f7a5546b640 1 --2- 172.21.15.88:0/1967115594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a50157510 0x7f7a500c8f00 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:00.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.472+0000 7f7a5546b640 1 --2- 172.21.15.88:0/1967115594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a500ce0b0 0x7f7a500c9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:00.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.472+0000 7f7a5546b640 1 -- 172.21.15.88:0/1967115594 >> 172.21.15.88:0/1967115594 conn(0x7f7a500715c0 msgr2=0x7f7a50152db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:00.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.472+0000 7f7a5546b640 1 -- 172.21.15.88:0/1967115594 shutdown_connections 2023-10-16T20:06:00.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.472+0000 7f7a5546b640 1 -- 172.21.15.88:0/1967115594 wait complete. 2023-10-16T20:06:00.497 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T20:06:00.636 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.632+0000 7f0721b3e640 1 Processor -- start 2023-10-16T20:06:00.636 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.632+0000 7f0721b3e640 1 -- start start 2023-10-16T20:06:00.638 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.632+0000 7f0721b3e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f071c06c2e0 0x7f071c164e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:00.638 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.632+0000 7f0721b3e640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f071c15d8b0 con 0x7f071c1500a0 2023-10-16T20:06:00.638 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.632+0000 7f0721b3e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f071c15aa10 con 0x7f071c159f50 2023-10-16T20:06:00.638 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.632+0000 7f0721b3e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f071c150480 con 0x7f071c06c2e0 2023-10-16T20:06:00.639 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.632+0000 7f071b7fe640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f071c1500a0 0x7f071c15ca90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:53604/0 (socket says 172.21.15.88:53604) 2023-10-16T20:06:00.639 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.632+0000 7f071b7fe640 1 -- 172.21.15.88:0/1159043364 learned_addr learned my addr 172.21.15.88:0/1159043364 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:00.639 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.632+0000 7f071bfff640 1 --2- 172.21.15.88:0/1159043364 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f071c06c2e0 0x7f071c164e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:00.640 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.632+0000 7f071a7fc640 1 -- 172.21.15.88:0/1159043364 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2969561406 0 0) 0x7f071c15d8b0 con 0x7f071c1500a0 2023-10-16T20:06:00.640 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.632+0000 7f071a7fc640 1 -- 172.21.15.88:0/1159043364 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0704003530 con 0x7f071c1500a0 2023-10-16T20:06:00.640 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.632+0000 7f071a7fc640 1 -- 172.21.15.88:0/1159043364 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2493169164 0 0) 0x7f071c15aa10 con 0x7f071c159f50 2023-10-16T20:06:00.641 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.632+0000 7f071a7fc640 1 -- 172.21.15.88:0/1159043364 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f071c15d8b0 con 0x7f071c159f50 2023-10-16T20:06:00.641 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.632+0000 7f071bfff640 1 -- 172.21.15.88:0/1159043364 >> v1:172.21.15.94:6789/0 conn(0x7f071c159f50 legacy=0x7f071c15d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:00.641 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.632+0000 7f071bfff640 1 -- 172.21.15.88:0/1159043364 >> v1:172.21.15.88:6789/0 conn(0x7f071c1500a0 legacy=0x7f071c15ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:00.641 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.632+0000 7f071bfff640 1 -- 172.21.15.88:0/1159043364 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071c15a3c0 con 0x7f071c06c2e0 2023-10-16T20:06:00.642 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.636+0000 7f071bfff640 1 --2- 172.21.15.88:0/1159043364 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f071c06c2e0 0x7f071c164e30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0714009df0 tx=0x7f071401c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a229c58040264a87 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:00.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.636+0000 7f071a7fc640 1 -- 172.21.15.88:0/1159043364 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07140a5920 con 0x7f071c06c2e0 2023-10-16T20:06:00.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.636+0000 7f071a7fc640 1 -- 172.21.15.88:0/1159043364 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07140a5a80 con 0x7f071c06c2e0 2023-10-16T20:06:00.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.636+0000 7f071a7fc640 1 -- 172.21.15.88:0/1159043364 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07140a5e10 con 0x7f071c06c2e0 2023-10-16T20:06:00.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.636+0000 7f0721b3e640 1 -- 172.21.15.88:0/1159043364 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f071c06c2e0 msgr2=0x7f071c164e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:00.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.636+0000 7f0721b3e640 1 --2- 172.21.15.88:0/1159043364 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f071c06c2e0 0x7f071c164e30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0714009df0 tx=0x7f071401c970 comp rx=0 tx=0).stop 2023-10-16T20:06:00.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.636+0000 7f0721b3e640 1 -- 172.21.15.88:0/1159043364 shutdown_connections 2023-10-16T20:06:00.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.636+0000 7f0721b3e640 1 --2- 172.21.15.88:0/1159043364 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f071c06c2e0 0x7f071c164e30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:00.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.636+0000 7f0721b3e640 1 -- 172.21.15.88:0/1159043364 >> 172.21.15.88:0/1159043364 conn(0x7f071c0cb080 msgr2=0x7f071c0cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:00.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.640+0000 7f0721b3e640 1 -- 172.21.15.88:0/1159043364 shutdown_connections 2023-10-16T20:06:00.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.640+0000 7f0721b3e640 1 -- 172.21.15.88:0/1159043364 wait complete. 2023-10-16T20:06:00.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.640+0000 7f0721b3e640 1 Processor -- start 2023-10-16T20:06:00.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.640+0000 7f0721b3e640 1 -- start start 2023-10-16T20:06:00.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.640+0000 7f0721b3e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f071c06c2e0 0x7f071c144750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:00.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.640+0000 7f0721b3e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f071c1500a0 0x7f071c157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:00.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.640+0000 7f0721b3e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f071c159f50 0x7f071c1577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:00.648 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.640+0000 7f0721b3e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c15c1a0 con 0x7f071c1500a0 2023-10-16T20:06:00.648 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.640+0000 7f0721b3e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c14c200 con 0x7f071c06c2e0 2023-10-16T20:06:00.648 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.640+0000 7f0721b3e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f071c14c4e0 con 0x7f071c159f50 2023-10-16T20:06:00.648 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.640+0000 7f071affd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f071c1500a0 0x7f071c157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:00.648 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.640+0000 7f071b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f071c06c2e0 0x7f071c144750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:00.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.640+0000 7f071affd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f071c1500a0 0x7f071c157270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50866/0 (socket says 172.21.15.88:50866) 2023-10-16T20:06:00.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.640+0000 7f071b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f071c06c2e0 0x7f071c144750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:57122/0 (socket says 172.21.15.88:57122) 2023-10-16T20:06:00.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.640+0000 7f071affd640 1 -- 172.21.15.88:0/1487749348 learned_addr learned my addr 172.21.15.88:0/1487749348 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:00.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.640+0000 7f071bfff640 1 --2- 172.21.15.88:0/1487749348 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f071c159f50 0x7f071c1577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:00.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.640+0000 7f071b7fe640 1 -- 172.21.15.88:0/1487749348 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f071c159f50 msgr2=0x7f071c1577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:00.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.640+0000 7f071b7fe640 1 --2- 172.21.15.88:0/1487749348 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f071c159f50 0x7f071c1577b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:00.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.640+0000 7f071b7fe640 1 -- 172.21.15.88:0/1487749348 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f071c1500a0 msgr2=0x7f071c157270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:00.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.640+0000 7f071b7fe640 1 --2- 172.21.15.88:0/1487749348 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f071c1500a0 0x7f071c157270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:00.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.640+0000 7f071b7fe640 1 -- 172.21.15.88:0/1487749348 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0714009a70 con 0x7f071c06c2e0 2023-10-16T20:06:00.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.640+0000 7f071affd640 1 --2- 172.21.15.88:0/1487749348 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f071c1500a0 0x7f071c157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:06:00.651 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.640+0000 7f071b7fe640 1 --2- 172.21.15.88:0/1487749348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f071c06c2e0 0x7f071c144750 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f070c00c430 tx=0x7f070c00c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:00.651 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.644+0000 7f0703fff640 1 -- 172.21.15.88:0/1487749348 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c005cf0 con 0x7f071c06c2e0 2023-10-16T20:06:00.651 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.644+0000 7f0703fff640 1 -- 172.21.15.88:0/1487749348 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070c005e50 con 0x7f071c06c2e0 2023-10-16T20:06:00.651 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.644+0000 7f0703fff640 1 -- 172.21.15.88:0/1487749348 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c0072d0 con 0x7f071c06c2e0 2023-10-16T20:06:00.651 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.644+0000 7f0721b3e640 1 -- 172.21.15.88:0/1487749348 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f071c14c7c0 con 0x7f071c06c2e0 2023-10-16T20:06:00.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.644+0000 7f0721b3e640 1 -- 172.21.15.88:0/1487749348 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f071c14cd80 con 0x7f071c06c2e0 2023-10-16T20:06:00.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.644+0000 7f0721b3e640 1 -- 172.21.15.88:0/1487749348 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f071c04e9c0 con 0x7f071c06c2e0 2023-10-16T20:06:00.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.644+0000 7f0703fff640 1 -- 172.21.15.88:0/1487749348 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f070c007430 con 0x7f071c06c2e0 2023-10-16T20:06:00.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.644+0000 7f0703fff640 1 --2- 172.21.15.88:0/1487749348 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f07080664a0 0x7f0708068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:00.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.644+0000 7f0703fff640 1 -- 172.21.15.88:0/1487749348 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f070c083810 con 0x7f071c06c2e0 2023-10-16T20:06:00.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.644+0000 7f071affd640 1 --2- 172.21.15.88:0/1487749348 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f07080664a0 0x7f0708068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:00.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.644+0000 7f071affd640 1 --2- 172.21.15.88:0/1487749348 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f07080664a0 0x7f0708068960 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f071c0711d0 tx=0x7f0710009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:00.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.648+0000 7f0703fff640 1 -- 172.21.15.88:0/1487749348 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f070c0509a0 con 0x7f071c06c2e0 2023-10-16T20:06:00.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.800+0000 7f0721b3e640 1 -- 172.21.15.88:0/1487749348 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f071c06d430 con 0x7f071c06c2e0 2023-10-16T20:06:00.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.800+0000 7f0703fff640 1 -- 172.21.15.88:0/1487749348 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f070c053fc0 con 0x7f071c06c2e0 2023-10-16T20:06:00.804 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2023-10-16T20:06:00.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.800+0000 7f0721b3e640 1 -- 172.21.15.88:0/1487749348 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f07080664a0 msgr2=0x7f0708068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:00.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.800+0000 7f0721b3e640 1 --2- 172.21.15.88:0/1487749348 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f07080664a0 0x7f0708068960 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f071c0711d0 tx=0x7f0710009040 comp rx=0 tx=0).stop 2023-10-16T20:06:00.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.800+0000 7f0721b3e640 1 -- 172.21.15.88:0/1487749348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f071c06c2e0 msgr2=0x7f071c144750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:00.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.800+0000 7f0721b3e640 1 --2- 172.21.15.88:0/1487749348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f071c06c2e0 0x7f071c144750 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f070c00c430 tx=0x7f070c00c8f0 comp rx=0 tx=0).stop 2023-10-16T20:06:00.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.800+0000 7f0721b3e640 1 -- 172.21.15.88:0/1487749348 shutdown_connections 2023-10-16T20:06:00.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.800+0000 7f0721b3e640 1 --2- 172.21.15.88:0/1487749348 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f07080664a0 0x7f0708068960 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:00.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.800+0000 7f0721b3e640 1 --2- 172.21.15.88:0/1487749348 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f071c159f50 0x7f071c1577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:00.808 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.800+0000 7f0721b3e640 1 --2- 172.21.15.88:0/1487749348 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f071c1500a0 0x7f071c157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:00.808 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.800+0000 7f0721b3e640 1 --2- 172.21.15.88:0/1487749348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f071c06c2e0 0x7f071c144750 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:00.808 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.800+0000 7f0721b3e640 1 -- 172.21.15.88:0/1487749348 >> 172.21.15.88:0/1487749348 conn(0x7f071c0cb080 msgr2=0x7f071c0cce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:00.808 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.800+0000 7f0721b3e640 1 -- 172.21.15.88:0/1487749348 shutdown_connections 2023-10-16T20:06:00.808 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.800+0000 7f0721b3e640 1 -- 172.21.15.88:0/1487749348 wait complete. 2023-10-16T20:06:00.825 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T20:06:00.965 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.960+0000 7f1fde5a7640 1 Processor -- start 2023-10-16T20:06:00.966 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.960+0000 7f1fde5a7640 1 -- start start 2023-10-16T20:06:00.967 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.964+0000 7f1fde5a7640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1fd815c9a0 0x7f1fd80cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:00.967 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.964+0000 7f1fde5a7640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fd80d0c40 con 0x7f1fd80cf230 2023-10-16T20:06:00.967 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.964+0000 7f1fde5a7640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fd80d0630 con 0x7f1fd80d3270 2023-10-16T20:06:00.967 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.964+0000 7f1fde5a7640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fd80ce7e0 con 0x7f1fd815c9a0 2023-10-16T20:06:00.967 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.964+0000 7f1fddda6640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f1fd80cf230 0x7f1fd80cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:53612/0 (socket says 172.21.15.88:53612) 2023-10-16T20:06:00.968 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.964+0000 7f1fddda6640 1 -- 172.21.15.88:0/1695338941 learned_addr learned my addr 172.21.15.88:0/1695338941 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:00.969 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.964+0000 7f1fdd5a5640 1 --2- 172.21.15.88:0/1695338941 >> v2:172.21.15.94:3300/0 conn(0x7f1fd815c9a0 0x7f1fd80cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:00.969 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.964+0000 7f1fcffff640 1 -- 172.21.15.88:0/1695338941 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2814278679 0 0) 0x7f1fd80d0c40 con 0x7f1fd80cf230 2023-10-16T20:06:00.969 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.964+0000 7f1fcffff640 1 -- 172.21.15.88:0/1695338941 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fc4003530 con 0x7f1fd80cf230 2023-10-16T20:06:00.969 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.964+0000 7f1fcffff640 1 -- 172.21.15.88:0/1695338941 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 678542390 0 0) 0x7f1fd80d0630 con 0x7f1fd80d3270 2023-10-16T20:06:00.969 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.964+0000 7f1fcffff640 1 -- 172.21.15.88:0/1695338941 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fd80d0c40 con 0x7f1fd80d3270 2023-10-16T20:06:00.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.964+0000 7f1fdd5a5640 1 -- 172.21.15.88:0/1695338941 >> v1:172.21.15.94:6789/0 conn(0x7f1fd80d3270 legacy=0x7f1fd80d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:00.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.964+0000 7f1fdd5a5640 1 -- 172.21.15.88:0/1695338941 >> v1:172.21.15.88:6789/0 conn(0x7f1fd80cf230 legacy=0x7f1fd80cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:00.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.964+0000 7f1fdd5a5640 1 -- 172.21.15.88:0/1695338941 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd80ce520 con 0x7f1fd815c9a0 2023-10-16T20:06:00.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.968+0000 7f1fdd5a5640 1 --2- 172.21.15.88:0/1695338941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fd815c9a0 0x7f1fd80cde80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1fd40097a0 tx=0x7f1fd401c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f108975d91f462e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:00.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.968+0000 7f1fcffff640 1 -- 172.21.15.88:0/1695338941 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd40ab600 con 0x7f1fd815c9a0 2023-10-16T20:06:00.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.968+0000 7f1fcffff640 1 -- 172.21.15.88:0/1695338941 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd40a7050 con 0x7f1fd815c9a0 2023-10-16T20:06:00.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.968+0000 7f1fcffff640 1 -- 172.21.15.88:0/1695338941 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd40ab980 con 0x7f1fd815c9a0 2023-10-16T20:06:00.972 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.968+0000 7f1fde5a7640 1 -- 172.21.15.88:0/1695338941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fd815c9a0 msgr2=0x7f1fd80cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:00.972 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.968+0000 7f1fde5a7640 1 --2- 172.21.15.88:0/1695338941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fd815c9a0 0x7f1fd80cde80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1fd40097a0 tx=0x7f1fd401c3a0 comp rx=0 tx=0).stop 2023-10-16T20:06:00.972 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.968+0000 7f1fde5a7640 1 -- 172.21.15.88:0/1695338941 shutdown_connections 2023-10-16T20:06:00.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.968+0000 7f1fde5a7640 1 --2- 172.21.15.88:0/1695338941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fd815c9a0 0x7f1fd80cde80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:00.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.968+0000 7f1fde5a7640 1 -- 172.21.15.88:0/1695338941 >> 172.21.15.88:0/1695338941 conn(0x7f1fd809bc30 msgr2=0x7f1fd80d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:00.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.968+0000 7f1fde5a7640 1 -- 172.21.15.88:0/1695338941 shutdown_connections 2023-10-16T20:06:00.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.968+0000 7f1fde5a7640 1 -- 172.21.15.88:0/1695338941 wait complete. 2023-10-16T20:06:00.974 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.968+0000 7f1fde5a7640 1 Processor -- start 2023-10-16T20:06:00.974 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.968+0000 7f1fde5a7640 1 -- start start 2023-10-16T20:06:00.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.968+0000 7f1fde5a7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fd80cf230 0x7f1fd80cad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:00.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.968+0000 7f1fde5a7640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fd80d3270 0x7f1fd8153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:00.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.968+0000 7f1fde5a7640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1fd815c9a0 0x7f1fd80cc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:00.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.968+0000 7f1fde5a7640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd8154430 con 0x7f1fd80d3270 2023-10-16T20:06:00.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.968+0000 7f1fde5a7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd80cb2e0 con 0x7f1fd80cf230 2023-10-16T20:06:00.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.968+0000 7f1fde5a7640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd80cb5c0 con 0x7f1fd815c9a0 2023-10-16T20:06:00.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.972+0000 7f1fdcda4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fd80d3270 0x7f1fd8153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:00.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.972+0000 7f1fdcda4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fd80d3270 0x7f1fd8153ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50872/0 (socket says 172.21.15.88:50872) 2023-10-16T20:06:00.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.972+0000 7f1fdcda4640 1 -- 172.21.15.88:0/3168699819 learned_addr learned my addr 172.21.15.88:0/3168699819 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:00.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.972+0000 7f1fddda6640 1 --2- 172.21.15.88:0/3168699819 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1fd815c9a0 0x7f1fd80cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:00.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.972+0000 7f1fdd5a5640 1 --2- 172.21.15.88:0/3168699819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fd80cf230 0x7f1fd80cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:00.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.972+0000 7f1fdcda4640 1 -- 172.21.15.88:0/3168699819 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1fd815c9a0 msgr2=0x7f1fd80cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:00.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.972+0000 7f1fdcda4640 1 --2- 172.21.15.88:0/3168699819 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1fd815c9a0 0x7f1fd80cc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:00.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.972+0000 7f1fdcda4640 1 -- 172.21.15.88:0/3168699819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fd80cf230 msgr2=0x7f1fd80cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:00.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.972+0000 7f1fdcda4640 1 --2- 172.21.15.88:0/3168699819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fd80cf230 0x7f1fd80cad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:00.978 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.972+0000 7f1fdcda4640 1 -- 172.21.15.88:0/3168699819 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd40093b0 con 0x7f1fd80d3270 2023-10-16T20:06:00.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.972+0000 7f1fdcda4640 1 --2- 172.21.15.88:0/3168699819 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fd80d3270 0x7f1fd8153ef0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800c490 tx=0x7f1fc800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:00.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.972+0000 7f1fcdffb640 1 -- 172.21.15.88:0/3168699819 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc80052d0 con 0x7f1fd80d3270 2023-10-16T20:06:00.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.972+0000 7f1fcdffb640 1 -- 172.21.15.88:0/3168699819 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc8005430 con 0x7f1fd80d3270 2023-10-16T20:06:00.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.972+0000 7f1fde5a7640 1 -- 172.21.15.88:0/3168699819 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd815cf00 con 0x7f1fd80d3270 2023-10-16T20:06:00.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.972+0000 7f1fcdffb640 1 -- 172.21.15.88:0/3168699819 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8012870 con 0x7f1fd80d3270 2023-10-16T20:06:00.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.972+0000 7f1fde5a7640 1 -- 172.21.15.88:0/3168699819 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd80cbb80 con 0x7f1fd80d3270 2023-10-16T20:06:00.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.972+0000 7f1fde5a7640 1 -- 172.21.15.88:0/3168699819 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fd80674f0 con 0x7f1fd80d3270 2023-10-16T20:06:00.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.972+0000 7f1fcdffb640 1 -- 172.21.15.88:0/3168699819 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1fc80129d0 con 0x7f1fd80d3270 2023-10-16T20:06:00.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.972+0000 7f1fcdffb640 1 --2- 172.21.15.88:0/3168699819 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1fbc0664a0 0x7f1fbc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:00.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.976+0000 7f1fcdffb640 1 -- 172.21.15.88:0/3168699819 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1fc8083d20 con 0x7f1fd80d3270 2023-10-16T20:06:00.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.976+0000 7f1fdd5a5640 1 --2- 172.21.15.88:0/3168699819 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1fbc0664a0 0x7f1fbc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:00.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.976+0000 7f1fdd5a5640 1 --2- 172.21.15.88:0/3168699819 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1fbc0664a0 0x7f1fbc068960 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80c63a0 tx=0x7f1fd40a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:00.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:00.976+0000 7f1fcdffb640 1 -- 172.21.15.88:0/3168699819 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1fc8050df0 con 0x7f1fd80d3270 2023-10-16T20:06:01.131 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.128+0000 7f1fde5a7640 1 -- 172.21.15.88:0/3168699819 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1fd80cff40 con 0x7f1fd80d3270 2023-10-16T20:06:01.132 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.128+0000 7f1fcdffb640 1 -- 172.21.15.88:0/3168699819 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1fc8054410 con 0x7f1fd80d3270 2023-10-16T20:06:01.132 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:06:01.134 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.128+0000 7f1fde5a7640 1 -- 172.21.15.88:0/3168699819 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1fbc0664a0 msgr2=0x7f1fbc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:01.134 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.128+0000 7f1fde5a7640 1 --2- 172.21.15.88:0/3168699819 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1fbc0664a0 0x7f1fbc068960 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80c63a0 tx=0x7f1fd40a6000 comp rx=0 tx=0).stop 2023-10-16T20:06:01.134 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.128+0000 7f1fde5a7640 1 -- 172.21.15.88:0/3168699819 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fd80d3270 msgr2=0x7f1fd8153ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:01.134 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.128+0000 7f1fde5a7640 1 --2- 172.21.15.88:0/3168699819 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fd80d3270 0x7f1fd8153ef0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800c490 tx=0x7f1fc800c950 comp rx=0 tx=0).stop 2023-10-16T20:06:01.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.128+0000 7f1fde5a7640 1 -- 172.21.15.88:0/3168699819 shutdown_connections 2023-10-16T20:06:01.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.128+0000 7f1fde5a7640 1 --2- 172.21.15.88:0/3168699819 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1fbc0664a0 0x7f1fbc068960 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:01.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.128+0000 7f1fde5a7640 1 --2- 172.21.15.88:0/3168699819 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1fd815c9a0 0x7f1fd80cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:01.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.128+0000 7f1fde5a7640 1 --2- 172.21.15.88:0/3168699819 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fd80d3270 0x7f1fd8153ef0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:01.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.128+0000 7f1fde5a7640 1 --2- 172.21.15.88:0/3168699819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fd80cf230 0x7f1fd80cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:01.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.128+0000 7f1fde5a7640 1 -- 172.21.15.88:0/3168699819 >> 172.21.15.88:0/3168699819 conn(0x7f1fd809bc30 msgr2=0x7f1fd80cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:01.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.128+0000 7f1fde5a7640 1 -- 172.21.15.88:0/3168699819 shutdown_connections 2023-10-16T20:06:01.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.128+0000 7f1fde5a7640 1 -- 172.21.15.88:0/3168699819 wait complete. 2023-10-16T20:06:01.153 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T20:06:01.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.288+0000 7f1e04246640 1 Processor -- start 2023-10-16T20:06:01.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.288+0000 7f1e04246640 1 -- start start 2023-10-16T20:06:01.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.288+0000 7f1e04246640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1dfc069e90 0x7f1dfc06a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:01.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.288+0000 7f1e04246640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1dfc0adf80 0x7f1dfc15a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:01.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.288+0000 7f1e04246640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dfc15e400 con 0x7f1dfc15d480 2023-10-16T20:06:01.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.288+0000 7f1e04246640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dfc159e40 con 0x7f1dfc069e90 2023-10-16T20:06:01.295 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.288+0000 7f1e04246640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc06a7b0 con 0x7f1dfc0adf80 2023-10-16T20:06:01.295 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.288+0000 7f1e027e3640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f1dfc15d480 0x7f1dfc0afa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:53622/0 (socket says 172.21.15.88:53622) 2023-10-16T20:06:01.295 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.288+0000 7f1e027e3640 1 -- 172.21.15.88:0/3247115030 learned_addr learned my addr 172.21.15.88:0/3247115030 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:01.295 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.288+0000 7f1e01fe2640 1 --2- 172.21.15.88:0/3247115030 >> v2:172.21.15.94:3300/0 conn(0x7f1dfc069e90 0x7f1dfc06a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:01.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.288+0000 7f1e017e1640 1 --2- 172.21.15.88:0/3247115030 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1dfc0adf80 0x7f1dfc15a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:01.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.288+0000 7f1e00fe0640 1 -- 172.21.15.88:0/3247115030 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3029843172 0 0) 0x7f1dfc15e400 con 0x7f1dfc15d480 2023-10-16T20:06:01.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.288+0000 7f1e00fe0640 1 -- 172.21.15.88:0/3247115030 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1de8003530 con 0x7f1dfc15d480 2023-10-16T20:06:01.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.288+0000 7f1e01fe2640 1 -- 172.21.15.88:0/3247115030 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1dfc0adf80 msgr2=0x7f1dfc15a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:01.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.288+0000 7f1e01fe2640 1 --2- 172.21.15.88:0/3247115030 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1dfc0adf80 0x7f1dfc15a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:01.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.288+0000 7f1e01fe2640 1 -- 172.21.15.88:0/3247115030 >> v1:172.21.15.88:6789/0 conn(0x7f1dfc15d480 legacy=0x7f1dfc0afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:01.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.288+0000 7f1e01fe2640 1 -- 172.21.15.88:0/3247115030 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dfc15de40 con 0x7f1dfc069e90 2023-10-16T20:06:01.298 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.292+0000 7f1e01fe2640 1 --2- 172.21.15.88:0/3247115030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dfc069e90 0x7f1dfc06a270 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1df80097c0 tx=0x7f1df801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be72ef035d8ab20e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:01.299 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.292+0000 7f1e00fe0640 1 -- 172.21.15.88:0/3247115030 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df80ab650 con 0x7f1dfc069e90 2023-10-16T20:06:01.299 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.292+0000 7f1e00fe0640 1 -- 172.21.15.88:0/3247115030 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df80ab7b0 con 0x7f1dfc069e90 2023-10-16T20:06:01.299 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.292+0000 7f1e00fe0640 1 -- 172.21.15.88:0/3247115030 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df80abb40 con 0x7f1dfc069e90 2023-10-16T20:06:01.299 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.292+0000 7f1e04246640 1 -- 172.21.15.88:0/3247115030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dfc069e90 msgr2=0x7f1dfc06a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:01.300 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.292+0000 7f1e04246640 1 --2- 172.21.15.88:0/3247115030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dfc069e90 0x7f1dfc06a270 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1df80097c0 tx=0x7f1df801c450 comp rx=0 tx=0).stop 2023-10-16T20:06:01.300 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.292+0000 7f1e04246640 1 -- 172.21.15.88:0/3247115030 shutdown_connections 2023-10-16T20:06:01.300 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.292+0000 7f1e04246640 1 --2- 172.21.15.88:0/3247115030 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1dfc0adf80 0x7f1dfc15a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:01.300 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.292+0000 7f1e04246640 1 --2- 172.21.15.88:0/3247115030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dfc069e90 0x7f1dfc06a270 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:01.300 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.292+0000 7f1e04246640 1 -- 172.21.15.88:0/3247115030 >> 172.21.15.88:0/3247115030 conn(0x7f1dfc0ce5e0 msgr2=0x7f1dfc0ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:01.300 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.292+0000 7f1e04246640 1 -- 172.21.15.88:0/3247115030 shutdown_connections 2023-10-16T20:06:01.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.292+0000 7f1e04246640 1 -- 172.21.15.88:0/3247115030 wait complete. 2023-10-16T20:06:01.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.296+0000 7f1e04246640 1 Processor -- start 2023-10-16T20:06:01.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.296+0000 7f1e04246640 1 -- start start 2023-10-16T20:06:01.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.296+0000 7f1e04246640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dfc069e90 0x7f1dfc159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:01.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.296+0000 7f1e04246640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1dfc0adf80 0x7f1dfc13ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:01.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.296+0000 7f1e04246640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1dfc15d480 0x7f1dfc0b0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:01.304 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.296+0000 7f1e04246640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc14c1c0 con 0x7f1dfc15d480 2023-10-16T20:06:01.304 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.296+0000 7f1e04246640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc14c4a0 con 0x7f1dfc069e90 2023-10-16T20:06:01.304 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.296+0000 7f1e04246640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc14c780 con 0x7f1dfc0adf80 2023-10-16T20:06:01.304 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.296+0000 7f1e027e3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1dfc15d480 0x7f1dfc0b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:01.304 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.296+0000 7f1e027e3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1dfc15d480 0x7f1dfc0b0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50882/0 (socket says 172.21.15.88:50882) 2023-10-16T20:06:01.305 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.296+0000 7f1e027e3640 1 -- 172.21.15.88:0/2391019368 learned_addr learned my addr 172.21.15.88:0/2391019368 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:01.305 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.296+0000 7f1e01fe2640 1 --2- 172.21.15.88:0/2391019368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dfc069e90 0x7f1dfc159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:01.305 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.296+0000 7f1e017e1640 1 --2- 172.21.15.88:0/2391019368 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1dfc0adf80 0x7f1dfc13ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:01.305 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.296+0000 7f1e027e3640 1 -- 172.21.15.88:0/2391019368 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1dfc0adf80 msgr2=0x7f1dfc13ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:01.305 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.296+0000 7f1e027e3640 1 --2- 172.21.15.88:0/2391019368 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1dfc0adf80 0x7f1dfc13ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:01.306 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.296+0000 7f1e027e3640 1 -- 172.21.15.88:0/2391019368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dfc069e90 msgr2=0x7f1dfc159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:01.306 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.296+0000 7f1e027e3640 1 --2- 172.21.15.88:0/2391019368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dfc069e90 0x7f1dfc159480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:01.306 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.296+0000 7f1e027e3640 1 -- 172.21.15.88:0/2391019368 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df8009420 con 0x7f1dfc15d480 2023-10-16T20:06:01.306 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.296+0000 7f1e01fe2640 1 --2- 172.21.15.88:0/2391019368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dfc069e90 0x7f1dfc159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:06:01.306 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.296+0000 7f1e027e3640 1 --2- 172.21.15.88:0/2391019368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1dfc15d480 0x7f1dfc0b0750 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1df400caa0 tx=0x7f1df400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:01.307 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.296+0000 7f1df27fc640 1 -- 172.21.15.88:0/2391019368 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df400ac00 con 0x7f1dfc15d480 2023-10-16T20:06:01.307 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.296+0000 7f1df27fc640 1 -- 172.21.15.88:0/2391019368 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df400ad60 con 0x7f1dfc15d480 2023-10-16T20:06:01.307 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.296+0000 7f1df27fc640 1 -- 172.21.15.88:0/2391019368 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df4012620 con 0x7f1dfc15d480 2023-10-16T20:06:01.307 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.296+0000 7f1e04246640 1 -- 172.21.15.88:0/2391019368 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dfc158c80 con 0x7f1dfc15d480 2023-10-16T20:06:01.307 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.296+0000 7f1e04246640 1 -- 172.21.15.88:0/2391019368 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dfc13f4b0 con 0x7f1dfc15d480 2023-10-16T20:06:01.308 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.300+0000 7f1e04246640 1 -- 172.21.15.88:0/2391019368 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dfc1591a0 con 0x7f1dfc15d480 2023-10-16T20:06:01.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.300+0000 7f1df27fc640 1 -- 172.21.15.88:0/2391019368 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1df4007070 con 0x7f1dfc15d480 2023-10-16T20:06:01.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.300+0000 7f1df27fc640 1 --2- 172.21.15.88:0/2391019368 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1de00664d0 0x7f1de0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:01.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.300+0000 7f1df27fc640 1 -- 172.21.15.88:0/2391019368 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1df4084170 con 0x7f1dfc15d480 2023-10-16T20:06:01.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.300+0000 7f1e01fe2640 1 --2- 172.21.15.88:0/2391019368 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1de00664d0 0x7f1de0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:01.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.300+0000 7f1e01fe2640 1 --2- 172.21.15.88:0/2391019368 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1de00664d0 0x7f1de0068990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1df80af000 tx=0x7f1df80a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:01.310 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.304+0000 7f1df27fc640 1 -- 172.21.15.88:0/2391019368 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1dfc1591a0 con 0x7f1dfc15d480 2023-10-16T20:06:01.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.468+0000 7f1e04246640 1 -- 172.21.15.88:0/2391019368 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1dfc06d180 con 0x7f1dfc15d480 2023-10-16T20:06:01.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.468+0000 7f1df27fc640 1 -- 172.21.15.88:0/2391019368 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1df4051240 con 0x7f1dfc15d480 2023-10-16T20:06:01.475 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2023-10-16T20:06:01.476 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.472+0000 7f1e04246640 1 -- 172.21.15.88:0/2391019368 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1de00664d0 msgr2=0x7f1de0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:01.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.472+0000 7f1e04246640 1 --2- 172.21.15.88:0/2391019368 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1de00664d0 0x7f1de0068990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1df80af000 tx=0x7f1df80a7000 comp rx=0 tx=0).stop 2023-10-16T20:06:01.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.472+0000 7f1e04246640 1 -- 172.21.15.88:0/2391019368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1dfc15d480 msgr2=0x7f1dfc0b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:01.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.472+0000 7f1e04246640 1 --2- 172.21.15.88:0/2391019368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1dfc15d480 0x7f1dfc0b0750 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1df400caa0 tx=0x7f1df400cf60 comp rx=0 tx=0).stop 2023-10-16T20:06:01.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.472+0000 7f1e04246640 1 -- 172.21.15.88:0/2391019368 shutdown_connections 2023-10-16T20:06:01.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.472+0000 7f1e04246640 1 --2- 172.21.15.88:0/2391019368 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1de00664d0 0x7f1de0068990 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:01.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.472+0000 7f1e04246640 1 --2- 172.21.15.88:0/2391019368 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1dfc0adf80 0x7f1dfc13ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:01.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.472+0000 7f1e04246640 1 --2- 172.21.15.88:0/2391019368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1dfc15d480 0x7f1dfc0b0750 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:01.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.472+0000 7f1e04246640 1 --2- 172.21.15.88:0/2391019368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dfc069e90 0x7f1dfc159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:01.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.472+0000 7f1e04246640 1 -- 172.21.15.88:0/2391019368 >> 172.21.15.88:0/2391019368 conn(0x7f1dfc0ce5e0 msgr2=0x7f1dfc070130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:01.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.472+0000 7f1e04246640 1 -- 172.21.15.88:0/2391019368 shutdown_connections 2023-10-16T20:06:01.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.472+0000 7f1e04246640 1 -- 172.21.15.88:0/2391019368 wait complete. 2023-10-16T20:06:01.495 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi094.front.sepia.ceph.com... 2023-10-16T20:06:01.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:01.496 DEBUG:teuthology.orchestra.run.smithi094:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-10-16T20:06:01.517 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:06:01.518 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2023-10-16T20:06:01.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:01.518 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-16T20:06:01.568 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T20:06:01.568 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T20:06:01.705 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.700+0000 7ff013c9e640 1 Processor -- start 2023-10-16T20:06:01.706 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.700+0000 7ff013c9e640 1 -- start start 2023-10-16T20:06:01.707 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.704+0000 7ff013c9e640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff00c160040 0x7ff00c071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:01.707 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.704+0000 7ff013c9e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff00c160420 0x7ff00c069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:01.707 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.704+0000 7ff013c9e640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff00c06a820 con 0x7ff00c06a3d0 2023-10-16T20:06:01.708 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.704+0000 7ff013c9e640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff00c0cd7d0 con 0x7ff00c160040 2023-10-16T20:06:01.708 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.704+0000 7ff013c9e640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff00c0cdab0 con 0x7ff00c160420 2023-10-16T20:06:01.708 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.704+0000 7ff011a3a640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff00c160040 0x7ff00c071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:01.708 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.704+0000 7ff011a3a640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff00c160040 0x7ff00c071f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50896/0 (socket says 172.21.15.88:50896) 2023-10-16T20:06:01.708 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.704+0000 7ff01223b640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7ff00c06a3d0 0x7ff00c1608b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:53628/0 (socket says 172.21.15.88:53628) 2023-10-16T20:06:01.709 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.704+0000 7ff011a3a640 1 -- 172.21.15.88:0/756935890 learned_addr learned my addr 172.21.15.88:0/756935890 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:01.709 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.704+0000 7ff011a3a640 1 -- 172.21.15.88:0/756935890 >> v2:172.21.15.94:3300/0 conn(0x7ff00c160420 msgr2=0x7ff00c069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:01.709 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.704+0000 7ff011a3a640 1 --2- 172.21.15.88:0/756935890 >> v2:172.21.15.94:3300/0 conn(0x7ff00c160420 0x7ff00c069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:01.710 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.704+0000 7ff011a3a640 1 -- 172.21.15.88:0/756935890 >> v1:172.21.15.88:6789/0 conn(0x7ff00c06a3d0 legacy=0x7ff00c1608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:01.710 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.704+0000 7ff011a3a640 1 -- 172.21.15.88:0/756935890 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff00c0cdd90 con 0x7ff00c160040 2023-10-16T20:06:01.711 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.704+0000 7ff011a3a640 1 --2- 172.21.15.88:0/756935890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff00c160040 0x7ff00c071f10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff00c0c9d70 tx=0x7ff00801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84b7dd26738e72c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:01.711 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.708+0000 7ff010a38640 1 -- 172.21.15.88:0/756935890 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0080aa5f0 con 0x7ff00c160040 2023-10-16T20:06:01.711 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.708+0000 7ff010a38640 1 -- 172.21.15.88:0/756935890 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0080aa750 con 0x7ff00c160040 2023-10-16T20:06:01.711 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.708+0000 7ff010a38640 1 -- 172.21.15.88:0/756935890 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0080aaae0 con 0x7ff00c160040 2023-10-16T20:06:01.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.708+0000 7ff013c9e640 1 -- 172.21.15.88:0/756935890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff00c160040 msgr2=0x7ff00c071f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:01.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.708+0000 7ff013c9e640 1 --2- 172.21.15.88:0/756935890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff00c160040 0x7ff00c071f10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff00c0c9d70 tx=0x7ff00801c4d0 comp rx=0 tx=0).stop 2023-10-16T20:06:01.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.708+0000 7ff013c9e640 1 -- 172.21.15.88:0/756935890 shutdown_connections 2023-10-16T20:06:01.713 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.708+0000 7ff013c9e640 1 --2- 172.21.15.88:0/756935890 >> v2:172.21.15.94:3300/0 conn(0x7ff00c160420 0x7ff00c069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:01.713 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.708+0000 7ff013c9e640 1 --2- 172.21.15.88:0/756935890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff00c160040 0x7ff00c071f10 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:01.713 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.708+0000 7ff013c9e640 1 -- 172.21.15.88:0/756935890 >> 172.21.15.88:0/756935890 conn(0x7ff00c06d340 msgr2=0x7ff00c06d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:01.713 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.708+0000 7ff013c9e640 1 -- 172.21.15.88:0/756935890 shutdown_connections 2023-10-16T20:06:01.713 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.708+0000 7ff013c9e640 1 -- 172.21.15.88:0/756935890 wait complete. 2023-10-16T20:06:01.715 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.708+0000 7ff013c9e640 1 Processor -- start 2023-10-16T20:06:01.715 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.708+0000 7ff013c9e640 1 -- start start 2023-10-16T20:06:01.716 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.708+0000 7ff013c9e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff00c06a3d0 0x7ff00c15f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:01.716 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.708+0000 7ff013c9e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff00c160040 0x7ff00c0b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:01.716 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.708+0000 7ff013c9e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff00c160420 0x7ff00c0c4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:01.716 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.708+0000 7ff013c9e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00c155d40 con 0x7ff00c160420 2023-10-16T20:06:01.717 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.708+0000 7ff013c9e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00c154900 con 0x7ff00c06a3d0 2023-10-16T20:06:01.717 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.708+0000 7ff013c9e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff00c154be0 con 0x7ff00c160040 2023-10-16T20:06:01.717 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.708+0000 7ff01223b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff00c160420 0x7ff00c0c4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:01.717 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.708+0000 7ff01223b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff00c160420 0x7ff00c0c4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50898/0 (socket says 172.21.15.88:50898) 2023-10-16T20:06:01.717 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.708+0000 7ff01223b640 1 -- 172.21.15.88:0/3244423227 learned_addr learned my addr 172.21.15.88:0/3244423227 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:01.718 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.712+0000 7ff011a3a640 1 --2- 172.21.15.88:0/3244423227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff00c06a3d0 0x7ff00c15f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:01.718 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.712+0000 7ff011239640 1 --2- 172.21.15.88:0/3244423227 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff00c160040 0x7ff00c0b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:01.718 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.712+0000 7ff01223b640 1 -- 172.21.15.88:0/3244423227 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff00c160040 msgr2=0x7ff00c0b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:01.718 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.712+0000 7ff01223b640 1 --2- 172.21.15.88:0/3244423227 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff00c160040 0x7ff00c0b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:01.718 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.712+0000 7ff01223b640 1 -- 172.21.15.88:0/3244423227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff00c06a3d0 msgr2=0x7ff00c15f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:01.719 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.712+0000 7ff01223b640 1 --2- 172.21.15.88:0/3244423227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff00c06a3d0 0x7ff00c15f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:01.719 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.712+0000 7ff01223b640 1 -- 172.21.15.88:0/3244423227 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0080094d0 con 0x7ff00c160420 2023-10-16T20:06:01.719 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.712+0000 7ff01223b640 1 --2- 172.21.15.88:0/3244423227 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff00c160420 0x7ff00c0c4fd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff00400c7a0 tx=0x7ff00400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:01.719 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.712+0000 7ff0027fc640 1 -- 172.21.15.88:0/3244423227 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00400a640 con 0x7ff00c160420 2023-10-16T20:06:01.719 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.712+0000 7ff0027fc640 1 -- 172.21.15.88:0/3244423227 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff00400a7a0 con 0x7ff00c160420 2023-10-16T20:06:01.719 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.712+0000 7ff013c9e640 1 -- 172.21.15.88:0/3244423227 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff00c154ec0 con 0x7ff00c160420 2023-10-16T20:06:01.720 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.712+0000 7ff0027fc640 1 -- 172.21.15.88:0/3244423227 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00400ab10 con 0x7ff00c160420 2023-10-16T20:06:01.720 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.712+0000 7ff013c9e640 1 -- 172.21.15.88:0/3244423227 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff00c155480 con 0x7ff00c160420 2023-10-16T20:06:01.721 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.712+0000 7ff013c9e640 1 -- 172.21.15.88:0/3244423227 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff00c04e9c0 con 0x7ff00c160420 2023-10-16T20:06:01.721 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.712+0000 7ff0027fc640 1 -- 172.21.15.88:0/3244423227 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff00400ac70 con 0x7ff00c160420 2023-10-16T20:06:01.721 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.712+0000 7ff0027fc640 1 --2- 172.21.15.88:0/3244423227 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7feff00664a0 0x7feff0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:01.721 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.716+0000 7ff0027fc640 1 -- 172.21.15.88:0/3244423227 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff0040837f0 con 0x7ff00c160420 2023-10-16T20:06:01.721 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.716+0000 7ff011a3a640 1 --2- 172.21.15.88:0/3244423227 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7feff00664a0 0x7feff0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:01.722 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.716+0000 7ff011a3a640 1 --2- 172.21.15.88:0/3244423227 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7feff00664a0 0x7feff0068960 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff0080ae000 tx=0x7ff0080b5040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:01.722 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.716+0000 7ff0027fc640 1 -- 172.21.15.88:0/3244423227 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff004050950 con 0x7ff00c160420 2023-10-16T20:06:01.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.900+0000 7ff013c9e640 1 -- 172.21.15.88:0/3244423227 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff00c06a820 con 0x7ff00c160420 2023-10-16T20:06:01.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.900+0000 7ff0027fc640 1 -- 172.21.15.88:0/3244423227 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4906 (secure 0 0 0) 0x7ff00c06a820 con 0x7ff00c160420 2023-10-16T20:06:01.905 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:06:01.905 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3803627113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3803627113},{"type":"v1","addr":"172.21.15.94:6834","nonce":3803627113}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3758093239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":3758093239},{"type":"v1","addr":"172.21.15.88:6837","nonce":3758093239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/3836772385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3836772385},{"type":"v1","addr":"172.21.15.88:6836","nonce":3836772385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T20:04:56.759719+0000","modified":"2023-10-16T20:05:14.569618+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6835/555575750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":555575750},{"type":"v1","addr":"172.21.15.94:6835","nonce":555575750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T20:06:01.905 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 9 2023-10-16T20:06:01.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.900+0000 7ff013c9e640 1 -- 172.21.15.88:0/3244423227 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7feff00664a0 msgr2=0x7feff0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:01.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.900+0000 7ff013c9e640 1 --2- 172.21.15.88:0/3244423227 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7feff00664a0 0x7feff0068960 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff0080ae000 tx=0x7ff0080b5040 comp rx=0 tx=0).stop 2023-10-16T20:06:01.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.900+0000 7ff013c9e640 1 -- 172.21.15.88:0/3244423227 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff00c160420 msgr2=0x7ff00c0c4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:01.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.900+0000 7ff013c9e640 1 --2- 172.21.15.88:0/3244423227 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff00c160420 0x7ff00c0c4fd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff00400c7a0 tx=0x7ff00400cc60 comp rx=0 tx=0).stop 2023-10-16T20:06:01.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.900+0000 7ff013c9e640 1 -- 172.21.15.88:0/3244423227 shutdown_connections 2023-10-16T20:06:01.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.900+0000 7ff013c9e640 1 --2- 172.21.15.88:0/3244423227 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7feff00664a0 0x7feff0068960 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:01.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.900+0000 7ff013c9e640 1 --2- 172.21.15.88:0/3244423227 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff00c160040 0x7ff00c0b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:01.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.900+0000 7ff013c9e640 1 --2- 172.21.15.88:0/3244423227 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff00c160420 0x7ff00c0c4fd0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:01.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.900+0000 7ff013c9e640 1 --2- 172.21.15.88:0/3244423227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff00c06a3d0 0x7ff00c15f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:01.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.900+0000 7ff013c9e640 1 -- 172.21.15.88:0/3244423227 >> 172.21.15.88:0/3244423227 conn(0x7ff00c06d340 msgr2=0x7ff00c0afb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:01.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.900+0000 7ff013c9e640 1 -- 172.21.15.88:0/3244423227 shutdown_connections 2023-10-16T20:06:01.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:01.900+0000 7ff013c9e640 1 -- 172.21.15.88:0/3244423227 wait complete. 2023-10-16T20:06:01.926 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:06:02.063 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.060+0000 7f3229cc9640 1 Processor -- start 2023-10-16T20:06:02.064 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.060+0000 7f3229cc9640 1 -- start start 2023-10-16T20:06:02.065 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.060+0000 7f3229cc9640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3224069e90 0x7f322406a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:02.066 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.060+0000 7f3229cc9640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f32240b05a0 0x7f32240b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:02.066 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.060+0000 7f3229cc9640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f322406a830 con 0x7f32241599f0 2023-10-16T20:06:02.066 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.060+0000 7f3229cc9640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32240b1260 con 0x7f32240b05a0 2023-10-16T20:06:02.066 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.060+0000 7f3229cc9640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32240b3140 con 0x7f3224069e90 2023-10-16T20:06:02.066 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.060+0000 7f3223fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f32240b05a0 0x7f32240b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:02.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.060+0000 7f3223fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f32240b05a0 0x7f32240b0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39860/0 (socket says 172.21.15.88:39860) 2023-10-16T20:06:02.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.060+0000 7f3223fff640 1 -- 172.21.15.88:0/2334461688 learned_addr learned my addr 172.21.15.88:0/2334461688 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:02.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.060+0000 7f321bfff640 1 --2- 172.21.15.88:0/2334461688 >> v2:172.21.15.94:3300/0 conn(0x7f3224069e90 0x7f322406a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:02.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.060+0000 7f3223fff640 1 -- 172.21.15.88:0/2334461688 >> v2:172.21.15.94:3300/0 conn(0x7f3224069e90 msgr2=0x7f322406a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:02.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.060+0000 7f3223fff640 1 --2- 172.21.15.88:0/2334461688 >> v2:172.21.15.94:3300/0 conn(0x7f3224069e90 0x7f322406a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:02.068 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.060+0000 7f3223fff640 1 -- 172.21.15.88:0/2334461688 >> v1:172.21.15.88:6789/0 conn(0x7f32241599f0 legacy=0x7f32240b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:02.068 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.060+0000 7f3223fff640 1 -- 172.21.15.88:0/2334461688 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32240ae960 con 0x7f32240b05a0 2023-10-16T20:06:02.069 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.064+0000 7f3223fff640 1 --2- 172.21.15.88:0/2334461688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32240b05a0 0x7f32240b0a30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f32240ae500 tx=0x7f321c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2d3b57de74a3d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:02.069 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.064+0000 7f3222ffd640 1 -- 172.21.15.88:0/2334461688 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c0a5860 con 0x7f32240b05a0 2023-10-16T20:06:02.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.064+0000 7f3222ffd640 1 -- 172.21.15.88:0/2334461688 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321c0a59c0 con 0x7f32240b05a0 2023-10-16T20:06:02.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.064+0000 7f3222ffd640 1 -- 172.21.15.88:0/2334461688 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c0a5d50 con 0x7f32240b05a0 2023-10-16T20:06:02.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.064+0000 7f3229cc9640 1 -- 172.21.15.88:0/2334461688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32240b05a0 msgr2=0x7f32240b0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:02.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.064+0000 7f3229cc9640 1 --2- 172.21.15.88:0/2334461688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32240b05a0 0x7f32240b0a30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f32240ae500 tx=0x7f321c01c9a0 comp rx=0 tx=0).stop 2023-10-16T20:06:02.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.064+0000 7f3229cc9640 1 -- 172.21.15.88:0/2334461688 shutdown_connections 2023-10-16T20:06:02.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.064+0000 7f3229cc9640 1 --2- 172.21.15.88:0/2334461688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32240b05a0 0x7f32240b0a30 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:02.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.064+0000 7f3229cc9640 1 --2- 172.21.15.88:0/2334461688 >> v2:172.21.15.94:3300/0 conn(0x7f3224069e90 0x7f322406a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:02.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.064+0000 7f3229cc9640 1 -- 172.21.15.88:0/2334461688 >> 172.21.15.88:0/2334461688 conn(0x7f32240ce5e0 msgr2=0x7f32240ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:02.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.064+0000 7f3229cc9640 1 -- 172.21.15.88:0/2334461688 shutdown_connections 2023-10-16T20:06:02.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.064+0000 7f3229cc9640 1 -- 172.21.15.88:0/2334461688 wait complete. 2023-10-16T20:06:02.073 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.068+0000 7f3229cc9640 1 Processor -- start 2023-10-16T20:06:02.074 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.068+0000 7f3229cc9640 1 -- start start 2023-10-16T20:06:02.074 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.068+0000 7f3229cc9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3224069e90 0x7f3224143db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:02.074 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.068+0000 7f3229cc9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32240b05a0 0x7f3224154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:02.074 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.068+0000 7f3229cc9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32241599f0 0x7f32240b3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:02.075 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.068+0000 7f3229cc9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3224071020 con 0x7f32241599f0 2023-10-16T20:06:02.075 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.068+0000 7f3229cc9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3224071300 con 0x7f32240b05a0 2023-10-16T20:06:02.075 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.068+0000 7f3229cc9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32240715e0 con 0x7f3224069e90 2023-10-16T20:06:02.075 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.068+0000 7f3223fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32241599f0 0x7f32240b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:02.075 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.068+0000 7f3223fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32241599f0 0x7f32240b3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39872/0 (socket says 172.21.15.88:39872) 2023-10-16T20:06:02.076 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.068+0000 7f3223fff640 1 -- 172.21.15.88:0/3487435873 learned_addr learned my addr 172.21.15.88:0/3487435873 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:02.076 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.068+0000 7f3223fff640 1 -- 172.21.15.88:0/3487435873 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3224069e90 msgr2=0x7f3224143db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:02.076 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.068+0000 7f3223fff640 1 --2- 172.21.15.88:0/3487435873 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3224069e90 0x7f3224143db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:02.076 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.068+0000 7f3223fff640 1 -- 172.21.15.88:0/3487435873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32240b05a0 msgr2=0x7f3224154700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:02.076 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.068+0000 7f321bfff640 1 --2- 172.21.15.88:0/3487435873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32240b05a0 0x7f3224154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:02.077 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.068+0000 7f3223fff640 1 --2- 172.21.15.88:0/3487435873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32240b05a0 0x7f3224154700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:02.077 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.068+0000 7f3223fff640 1 -- 172.21.15.88:0/3487435873 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321c009b30 con 0x7f32241599f0 2023-10-16T20:06:02.077 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.068+0000 7f321bfff640 1 --2- 172.21.15.88:0/3487435873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32240b05a0 0x7f3224154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:06:02.077 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.068+0000 7f3223fff640 1 --2- 172.21.15.88:0/3487435873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32241599f0 0x7f32240b3280 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f321c0af040 tx=0x7f321c019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:02.077 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.068+0000 7f3220ff9640 1 -- 172.21.15.88:0/3487435873 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c0b8670 con 0x7f32241599f0 2023-10-16T20:06:02.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.068+0000 7f3220ff9640 1 -- 172.21.15.88:0/3487435873 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321c0b8810 con 0x7f32241599f0 2023-10-16T20:06:02.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.068+0000 7f3229cc9640 1 -- 172.21.15.88:0/3487435873 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32240718c0 con 0x7f32241599f0 2023-10-16T20:06:02.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.068+0000 7f3220ff9640 1 -- 172.21.15.88:0/3487435873 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c0b8bc0 con 0x7f32241599f0 2023-10-16T20:06:02.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.068+0000 7f3229cc9640 1 -- 172.21.15.88:0/3487435873 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32241442f0 con 0x7f32241599f0 2023-10-16T20:06:02.079 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.068+0000 7f3229cc9640 1 -- 172.21.15.88:0/3487435873 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f322404e9c0 con 0x7f32241599f0 2023-10-16T20:06:02.079 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.072+0000 7f3220ff9640 1 -- 172.21.15.88:0/3487435873 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f321c0b8d20 con 0x7f32241599f0 2023-10-16T20:06:02.079 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.072+0000 7f3220ff9640 1 --2- 172.21.15.88:0/3487435873 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f32040664a0 0x7f3204068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:02.080 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.072+0000 7f3220ff9640 1 -- 172.21.15.88:0/3487435873 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f321c11fa80 con 0x7f32241599f0 2023-10-16T20:06:02.080 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.072+0000 7f32237fe640 1 --2- 172.21.15.88:0/3487435873 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f32040664a0 0x7f3204068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:02.080 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.072+0000 7f32237fe640 1 --2- 172.21.15.88:0/3487435873 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f32040664a0 0x7f3204068960 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3214000f70 tx=0x7f32140079c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:02.080 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.076+0000 7f3220ff9640 1 -- 172.21.15.88:0/3487435873 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f321c0ecc80 con 0x7f32241599f0 2023-10-16T20:06:02.230 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.224+0000 7f3229cc9640 1 -- 172.21.15.88:0/3487435873 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3224071d70 con 0x7f32241599f0 2023-10-16T20:06:02.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.228+0000 7f3220ff9640 1 -- 172.21.15.88:0/3487435873 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18661 (secure 0 0 0) 0x7f321c0f02a0 con 0x7f32241599f0 2023-10-16T20:06:02.232 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:06:02.232 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":29,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:05:38.202959+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T20:04:54.442487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T20:04:55.766641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T20:04:59.475830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":1}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:06:02.233 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.228+0000 7f3229cc9640 1 -- 172.21.15.88:0/3487435873 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f32040664a0 msgr2=0x7f3204068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:02.234 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.228+0000 7f3229cc9640 1 --2- 172.21.15.88:0/3487435873 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f32040664a0 0x7f3204068960 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3214000f70 tx=0x7f32140079c0 comp rx=0 tx=0).stop 2023-10-16T20:06:02.234 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.228+0000 7f3229cc9640 1 -- 172.21.15.88:0/3487435873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32241599f0 msgr2=0x7f32240b3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:02.234 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.228+0000 7f3229cc9640 1 --2- 172.21.15.88:0/3487435873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32241599f0 0x7f32240b3280 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f321c0af040 tx=0x7f321c019740 comp rx=0 tx=0).stop 2023-10-16T20:06:02.234 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.228+0000 7f3229cc9640 1 -- 172.21.15.88:0/3487435873 shutdown_connections 2023-10-16T20:06:02.234 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.228+0000 7f3229cc9640 1 --2- 172.21.15.88:0/3487435873 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f32040664a0 0x7f3204068960 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:02.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.228+0000 7f3229cc9640 1 --2- 172.21.15.88:0/3487435873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32241599f0 0x7f32240b3280 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:02.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.228+0000 7f3229cc9640 1 --2- 172.21.15.88:0/3487435873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32240b05a0 0x7f3224154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:02.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.228+0000 7f3229cc9640 1 --2- 172.21.15.88:0/3487435873 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3224069e90 0x7f3224143db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:02.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.228+0000 7f3229cc9640 1 -- 172.21.15.88:0/3487435873 >> 172.21.15.88:0/3487435873 conn(0x7f32240ce5e0 msgr2=0x7f32240cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:02.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.228+0000 7f3229cc9640 1 -- 172.21.15.88:0/3487435873 shutdown_connections 2023-10-16T20:06:02.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.228+0000 7f3229cc9640 1 -- 172.21.15.88:0/3487435873 wait complete. 2023-10-16T20:06:02.252 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T20:04:54.442487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T20:04:55.766641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T20:04:59.475830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T20:06:02.253 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:06:02.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd1c2ea640 1 Processor -- start 2023-10-16T20:06:02.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd1c2ea640 1 -- start start 2023-10-16T20:06:02.396 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd1c2ea640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdd1415ac00 0x7fdd140b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:02.396 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd1c2ea640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fdd140b2930 0x7fdd1406c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:02.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd1c2ea640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd140ce4b0 con 0x7fdd1415a820 2023-10-16T20:06:02.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd1c2ea640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd140b2da0 con 0x7fdd140b2930 2023-10-16T20:06:02.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd1c2ea640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd140cf8f0 con 0x7fdd1415ac00 2023-10-16T20:06:02.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd1a086640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fdd1415a820 0x7fdd140cf1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:40494/0 (socket says 172.21.15.88:40494) 2023-10-16T20:06:02.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd1a086640 1 -- 172.21.15.88:0/3772832952 learned_addr learned my addr 172.21.15.88:0/3772832952 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:02.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd1a887640 1 --2- 172.21.15.88:0/3772832952 >> v2:172.21.15.94:3300/0 conn(0x7fdd140b2930 0x7fdd1406c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:02.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd19885640 1 --2- 172.21.15.88:0/3772832952 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdd1415ac00 0x7fdd140b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:02.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd19084640 1 -- 172.21.15.88:0/3772832952 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2761556811 0 0) 0x7fdd140ce4b0 con 0x7fdd1415a820 2023-10-16T20:06:02.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd19084640 1 -- 172.21.15.88:0/3772832952 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd00003530 con 0x7fdd1415a820 2023-10-16T20:06:02.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd19084640 1 -- 172.21.15.88:0/3772832952 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 165135384 0 0) 0x7fdd00003530 con 0x7fdd1415a820 2023-10-16T20:06:02.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd19084640 1 -- 172.21.15.88:0/3772832952 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdd1415ac00 msgr2=0x7fdd140b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:02.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd19084640 1 --2- 172.21.15.88:0/3772832952 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdd1415ac00 0x7fdd140b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:02.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd19084640 1 -- 172.21.15.88:0/3772832952 >> v2:172.21.15.94:3300/0 conn(0x7fdd140b2930 msgr2=0x7fdd1406c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:02.401 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd19084640 1 --2- 172.21.15.88:0/3772832952 >> v2:172.21.15.94:3300/0 conn(0x7fdd140b2930 0x7fdd1406c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:02.401 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd19084640 1 -- 172.21.15.88:0/3772832952 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd1415e270 con 0x7fdd1415a820 2023-10-16T20:06:02.401 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd19084640 1 -- 172.21.15.88:0/3772832952 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7fdd10001680 con 0x7fdd1415a820 2023-10-16T20:06:02.401 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd19084640 1 -- 172.21.15.88:0/3772832952 >> v1:172.21.15.88:6789/0 conn(0x7fdd1415a820 legacy=0x7fdd140cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:02.401 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd19084640 1 --2- 172.21.15.88:0/3772832952 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdd1415ac00 0x7fdd00003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:02.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd19084640 1 --2- 172.21.15.88:0/3772832952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd00003f00 0x7fdd000047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:02.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd19084640 1 --2- 172.21.15.88:0/3772832952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd00004d10 0x7fdd00005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:02.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd19084640 1 -- 172.21.15.88:0/3772832952 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd140b2da0 con 0x7fdd00003f00 2023-10-16T20:06:02.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd19084640 1 -- 172.21.15.88:0/3772832952 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd140cf8f0 con 0x7fdd00004d10 2023-10-16T20:06:02.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd19084640 1 -- 172.21.15.88:0/3772832952 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0000b990 con 0x7fdd1415ac00 2023-10-16T20:06:02.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd19885640 1 --2- 172.21.15.88:0/3772832952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd00003f00 0x7fdd000047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:02.403 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd1a086640 1 --2- 172.21.15.88:0/3772832952 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdd1415ac00 0x7fdd00003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:02.403 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd1a887640 1 --2- 172.21.15.88:0/3772832952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd00004d10 0x7fdd00005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:02.403 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd19885640 1 -- 172.21.15.88:0/3772832952 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdd1415ac00 msgr2=0x7fdd00003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:02.403 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd19885640 1 --2- 172.21.15.88:0/3772832952 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdd1415ac00 0x7fdd00003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:02.403 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd19885640 1 -- 172.21.15.88:0/3772832952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd00004d10 msgr2=0x7fdd00005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:02.403 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd19885640 1 --2- 172.21.15.88:0/3772832952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd00004d10 0x7fdd00005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:02.404 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.392+0000 7fdd19885640 1 -- 172.21.15.88:0/3772832952 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd00003530 con 0x7fdd00003f00 2023-10-16T20:06:02.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.396+0000 7fdd19885640 1 --2- 172.21.15.88:0/3772832952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd00003f00 0x7fdd000047d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009ae0 tx=0x7fdd0401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e6ede6887ec8f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:02.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.396+0000 7fdd19084640 1 -- 172.21.15.88:0/3772832952 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd040a5640 con 0x7fdd00003f00 2023-10-16T20:06:02.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.396+0000 7fdd19084640 1 -- 172.21.15.88:0/3772832952 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd040a57a0 con 0x7fdd00003f00 2023-10-16T20:06:02.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.396+0000 7fdd19084640 1 -- 172.21.15.88:0/3772832952 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd040a5b50 con 0x7fdd00003f00 2023-10-16T20:06:02.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.396+0000 7fdd1c2ea640 1 -- 172.21.15.88:0/3772832952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd00003f00 msgr2=0x7fdd000047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:02.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.396+0000 7fdd1c2ea640 1 --2- 172.21.15.88:0/3772832952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd00003f00 0x7fdd000047d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009ae0 tx=0x7fdd0401c9f0 comp rx=0 tx=0).stop 2023-10-16T20:06:02.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.396+0000 7fdd1a887640 1 -- 172.21.15.88:0/3772832952 reap_dead start 2023-10-16T20:06:02.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.396+0000 7fdd1c2ea640 1 -- 172.21.15.88:0/3772832952 shutdown_connections 2023-10-16T20:06:02.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.396+0000 7fdd1c2ea640 1 -- 172.21.15.88:0/3772832952 >> 172.21.15.88:0/3772832952 conn(0x7fdd140cd3f0 msgr2=0x7fdd140cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:02.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.396+0000 7fdd1c2ea640 1 -- 172.21.15.88:0/3772832952 shutdown_connections 2023-10-16T20:06:02.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.396+0000 7fdd1c2ea640 1 -- 172.21.15.88:0/3772832952 wait complete. 2023-10-16T20:06:02.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.400+0000 7fdd1c2ea640 1 Processor -- start 2023-10-16T20:06:02.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.400+0000 7fdd1c2ea640 1 -- start start 2023-10-16T20:06:02.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.400+0000 7fdd1c2ea640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdd00003f00 0x7fdd14159080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:02.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.400+0000 7fdd1c2ea640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd1415ca40 0x7fdd1415cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:02.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.400+0000 7fdd1c2ea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd140b3cf0 0x7fdd140b4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:02.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.400+0000 7fdd1c2ea640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd140b46b0 con 0x7fdd1415ca40 2023-10-16T20:06:02.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.400+0000 7fdd1c2ea640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd140b5fa0 con 0x7fdd140b3cf0 2023-10-16T20:06:02.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.400+0000 7fdd1c2ea640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd140c8770 con 0x7fdd00003f00 2023-10-16T20:06:02.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.400+0000 7fdd19885640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd1415ca40 0x7fdd1415cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:02.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.400+0000 7fdd19885640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd1415ca40 0x7fdd1415cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39892/0 (socket says 172.21.15.88:39892) 2023-10-16T20:06:02.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.400+0000 7fdd19885640 1 -- 172.21.15.88:0/4235567453 learned_addr learned my addr 172.21.15.88:0/4235567453 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:02.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.400+0000 7fdd1a086640 1 --2- 172.21.15.88:0/4235567453 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdd00003f00 0x7fdd14159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:02.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.400+0000 7fdd1a887640 1 --2- 172.21.15.88:0/4235567453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd140b3cf0 0x7fdd140b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:02.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.400+0000 7fdd19885640 1 -- 172.21.15.88:0/4235567453 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdd00003f00 msgr2=0x7fdd14159080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:02.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.400+0000 7fdd19885640 1 --2- 172.21.15.88:0/4235567453 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdd00003f00 0x7fdd14159080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:02.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.400+0000 7fdd19885640 1 -- 172.21.15.88:0/4235567453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd140b3cf0 msgr2=0x7fdd140b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:02.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.400+0000 7fdd19885640 1 --2- 172.21.15.88:0/4235567453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd140b3cf0 0x7fdd140b4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:02.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.400+0000 7fdd19885640 1 -- 172.21.15.88:0/4235567453 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd1415d5b0 con 0x7fdd1415ca40 2023-10-16T20:06:02.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.400+0000 7fdd19885640 1 --2- 172.21.15.88:0/4235567453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd1415ca40 0x7fdd1415cea0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009ae0 tx=0x7fdd00005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:02.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.400+0000 7fdd0affd640 1 -- 172.21.15.88:0/4235567453 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd040a5640 con 0x7fdd1415ca40 2023-10-16T20:06:02.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.400+0000 7fdd0affd640 1 -- 172.21.15.88:0/4235567453 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd04004220 con 0x7fdd1415ca40 2023-10-16T20:06:02.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.400+0000 7fdd0affd640 1 -- 172.21.15.88:0/4235567453 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd04003620 con 0x7fdd1415ca40 2023-10-16T20:06:02.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.400+0000 7fdd1c2ea640 1 -- 172.21.15.88:0/4235567453 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd1415d890 con 0x7fdd1415ca40 2023-10-16T20:06:02.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.400+0000 7fdd1c2ea640 1 -- 172.21.15.88:0/4235567453 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd140c8d10 con 0x7fdd1415ca40 2023-10-16T20:06:02.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.404+0000 7fdd1c2ea640 1 -- 172.21.15.88:0/4235567453 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd1404e9c0 con 0x7fdd1415ca40 2023-10-16T20:06:02.414 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.404+0000 7fdd0affd640 1 -- 172.21.15.88:0/4235567453 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdd040b4040 con 0x7fdd1415ca40 2023-10-16T20:06:02.414 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.404+0000 7fdd0affd640 1 --2- 172.21.15.88:0/4235567453 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fdcf80664d0 0x7fdcf8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:02.415 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.404+0000 7fdd0affd640 1 -- 172.21.15.88:0/4235567453 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fdd0411e380 con 0x7fdd1415ca40 2023-10-16T20:06:02.415 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.404+0000 7fdd1a086640 1 --2- 172.21.15.88:0/4235567453 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fdcf80664d0 0x7fdcf8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:02.415 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.404+0000 7fdd1a086640 1 --2- 172.21.15.88:0/4235567453 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fdcf80664d0 0x7fdcf8068990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdd10005180 tx=0x7fdd10007900 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:02.415 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.408+0000 7fdd0affd640 1 -- 172.21.15.88:0/4235567453 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdd040eb4e0 con 0x7fdd1415ca40 2023-10-16T20:06:02.564 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.560+0000 7fdd1c2ea640 1 -- 172.21.15.88:0/4235567453 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdd1406b110 con 0x7fdd1415ca40 2023-10-16T20:06:02.565 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.560+0000 7fdd0affd640 1 -- 172.21.15.88:0/4235567453 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdd040eeb00 con 0x7fdd1415ca40 2023-10-16T20:06:02.565 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:06:02.567 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.560+0000 7fdd1c2ea640 1 -- 172.21.15.88:0/4235567453 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fdcf80664d0 msgr2=0x7fdcf8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:02.567 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.560+0000 7fdd1c2ea640 1 --2- 172.21.15.88:0/4235567453 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fdcf80664d0 0x7fdcf8068990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdd10005180 tx=0x7fdd10007900 comp rx=0 tx=0).stop 2023-10-16T20:06:02.567 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.560+0000 7fdd1c2ea640 1 -- 172.21.15.88:0/4235567453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd1415ca40 msgr2=0x7fdd1415cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:02.567 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.560+0000 7fdd1c2ea640 1 --2- 172.21.15.88:0/4235567453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd1415ca40 0x7fdd1415cea0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009ae0 tx=0x7fdd00005800 comp rx=0 tx=0).stop 2023-10-16T20:06:02.567 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.560+0000 7fdd1c2ea640 1 -- 172.21.15.88:0/4235567453 shutdown_connections 2023-10-16T20:06:02.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.560+0000 7fdd1c2ea640 1 --2- 172.21.15.88:0/4235567453 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fdcf80664d0 0x7fdcf8068990 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:02.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.560+0000 7fdd1c2ea640 1 --2- 172.21.15.88:0/4235567453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd140b3cf0 0x7fdd140b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:02.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.560+0000 7fdd1c2ea640 1 --2- 172.21.15.88:0/4235567453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd1415ca40 0x7fdd1415cea0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:02.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.560+0000 7fdd1c2ea640 1 --2- 172.21.15.88:0/4235567453 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdd00003f00 0x7fdd14159080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:02.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.564+0000 7fdd1c2ea640 1 -- 172.21.15.88:0/4235567453 >> 172.21.15.88:0/4235567453 conn(0x7fdd140cd3f0 msgr2=0x7fdd140b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:02.569 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.564+0000 7fdd1c2ea640 1 -- 172.21.15.88:0/4235567453 shutdown_connections 2023-10-16T20:06:02.569 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.564+0000 7fdd1c2ea640 1 -- 172.21.15.88:0/4235567453 wait complete. 2023-10-16T20:06:02.586 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T20:06:02.725 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.720+0000 7f638d7c4640 1 Processor -- start 2023-10-16T20:06:02.726 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.720+0000 7f638d7c4640 1 -- start start 2023-10-16T20:06:02.727 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f638d7c4640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f63880d1be0 0x7f63880b4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:02.727 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f638d7c4640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63880b4a70 0x7f63880b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:02.727 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f638d7c4640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63880b7d20 con 0x7f638815cb70 2023-10-16T20:06:02.728 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f638d7c4640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63880b4ee0 con 0x7f63880d1be0 2023-10-16T20:06:02.728 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f638d7c4640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63880b7110 con 0x7f63880b4a70 2023-10-16T20:06:02.728 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f6386ffd640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f638815cb70 0x7f63880b83d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:40510/0 (socket says 172.21.15.88:40510) 2023-10-16T20:06:02.728 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f6386ffd640 1 -- 172.21.15.88:0/2681808029 learned_addr learned my addr 172.21.15.88:0/2681808029 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:02.728 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f63877fe640 1 --2- 172.21.15.88:0/2681808029 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63880b4a70 0x7f63880b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:02.729 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f637ffff640 1 --2- 172.21.15.88:0/2681808029 >> v2:172.21.15.94:3300/0 conn(0x7f63880d1be0 0x7f63880b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:02.729 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f63867fc640 1 -- 172.21.15.88:0/2681808029 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1240422042 0 0) 0x7f63880b7d20 con 0x7f638815cb70 2023-10-16T20:06:02.729 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f63867fc640 1 -- 172.21.15.88:0/2681808029 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6374003530 con 0x7f638815cb70 2023-10-16T20:06:02.730 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f63867fc640 1 -- 172.21.15.88:0/2681808029 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1264221455 0 0) 0x7f6374003530 con 0x7f638815cb70 2023-10-16T20:06:02.731 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f63867fc640 1 -- 172.21.15.88:0/2681808029 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63880b4a70 msgr2=0x7f63880b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:02.731 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f63867fc640 1 --2- 172.21.15.88:0/2681808029 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63880b4a70 0x7f63880b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:02.731 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f63867fc640 1 -- 172.21.15.88:0/2681808029 >> v2:172.21.15.94:3300/0 conn(0x7f63880d1be0 msgr2=0x7f63880b4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:02.731 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f63867fc640 1 --2- 172.21.15.88:0/2681808029 >> v2:172.21.15.94:3300/0 conn(0x7f63880d1be0 0x7f63880b4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:02.732 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f63867fc640 1 -- 172.21.15.88:0/2681808029 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63880b73f0 con 0x7f638815cb70 2023-10-16T20:06:02.732 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f63867fc640 1 -- 172.21.15.88:0/2681808029 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f6378001680 con 0x7f638815cb70 2023-10-16T20:06:02.732 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f63877fe640 1 --2- 172.21.15.88:0/2681808029 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63880b4a70 0x7f63880b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:06:02.732 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f63867fc640 1 -- 172.21.15.88:0/2681808029 >> v1:172.21.15.88:6789/0 conn(0x7f638815cb70 legacy=0x7f63880b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:02.732 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f63867fc640 1 --2- 172.21.15.88:0/2681808029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6374003980 0x7f6374003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:02.732 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f63867fc640 1 --2- 172.21.15.88:0/2681808029 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63880b4a70 0x7f63740047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:02.733 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f63867fc640 1 --2- 172.21.15.88:0/2681808029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6374004d10 0x7f6374005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:02.733 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f63867fc640 1 -- 172.21.15.88:0/2681808029 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63880b4ee0 con 0x7f6374004d10 2023-10-16T20:06:02.733 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f63867fc640 1 -- 172.21.15.88:0/2681808029 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63880b7110 con 0x7f6374003980 2023-10-16T20:06:02.733 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f63867fc640 1 -- 172.21.15.88:0/2681808029 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f637400b990 con 0x7f63880b4a70 2023-10-16T20:06:02.733 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f63877fe640 1 --2- 172.21.15.88:0/2681808029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6374004d10 0x7f6374005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:02.733 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f6386ffd640 1 --2- 172.21.15.88:0/2681808029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6374003980 0x7f6374003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:02.734 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f637ffff640 1 --2- 172.21.15.88:0/2681808029 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63880b4a70 0x7f63740047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:02.734 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f6386ffd640 1 -- 172.21.15.88:0/2681808029 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63880b4a70 msgr2=0x7f63740047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:02.734 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f6386ffd640 1 --2- 172.21.15.88:0/2681808029 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63880b4a70 0x7f63740047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:02.734 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f6386ffd640 1 -- 172.21.15.88:0/2681808029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6374004d10 msgr2=0x7f6374005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:02.734 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f6386ffd640 1 --2- 172.21.15.88:0/2681808029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6374004d10 0x7f6374005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:02.735 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f6386ffd640 1 -- 172.21.15.88:0/2681808029 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6380009a70 con 0x7f6374003980 2023-10-16T20:06:02.735 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.724+0000 7f63877fe640 1 --2- 172.21.15.88:0/2681808029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6374004d10 0x7f6374005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:06:02.737 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.728+0000 7f6386ffd640 1 --2- 172.21.15.88:0/2681808029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6374003980 0x7f6374003da0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f637800c240 tx=0x7f637801cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=482c55c10cb3dd6b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:02.738 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.728+0000 7f63867fc640 1 -- 172.21.15.88:0/2681808029 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63780a5960 con 0x7f6374003980 2023-10-16T20:06:02.738 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.728+0000 7f63867fc640 1 -- 172.21.15.88:0/2681808029 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63780a5ac0 con 0x7f6374003980 2023-10-16T20:06:02.738 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.728+0000 7f63867fc640 1 -- 172.21.15.88:0/2681808029 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63780a5e90 con 0x7f6374003980 2023-10-16T20:06:02.738 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.728+0000 7f638d7c4640 1 -- 172.21.15.88:0/2681808029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6374003980 msgr2=0x7f6374003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:02.738 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.728+0000 7f638d7c4640 1 --2- 172.21.15.88:0/2681808029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6374003980 0x7f6374003da0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f637800c240 tx=0x7f637801cc60 comp rx=0 tx=0).stop 2023-10-16T20:06:02.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.728+0000 7f63877fe640 1 -- 172.21.15.88:0/2681808029 reap_dead start 2023-10-16T20:06:02.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.728+0000 7f638d7c4640 1 -- 172.21.15.88:0/2681808029 shutdown_connections 2023-10-16T20:06:02.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.728+0000 7f638d7c4640 1 -- 172.21.15.88:0/2681808029 >> 172.21.15.88:0/2681808029 conn(0x7f638806b6e0 msgr2=0x7f638806baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:02.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.728+0000 7f638d7c4640 1 -- 172.21.15.88:0/2681808029 shutdown_connections 2023-10-16T20:06:02.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.728+0000 7f638d7c4640 1 -- 172.21.15.88:0/2681808029 wait complete. 2023-10-16T20:06:02.740 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.728+0000 7f638d7c4640 1 Processor -- start 2023-10-16T20:06:02.740 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.728+0000 7f638d7c4640 1 -- start start 2023-10-16T20:06:02.740 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.732+0000 7f638d7c4640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6374003980 0x7f638814d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:02.740 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.732+0000 7f638d7c4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f638815dba0 0x7f638815e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:02.740 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.732+0000 7f638d7c4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63880c6510 0x7f63880c6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:02.741 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.732+0000 7f638d7c4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638815e560 con 0x7f638815dba0 2023-10-16T20:06:02.741 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.732+0000 7f638d7c4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638814dfb0 con 0x7f63880c6510 2023-10-16T20:06:02.741 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.732+0000 7f638d7c4640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63880c7220 con 0x7f6374003980 2023-10-16T20:06:02.741 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.732+0000 7f637ffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f638815dba0 0x7f638815e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:02.741 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.732+0000 7f637ffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f638815dba0 0x7f638815e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39912/0 (socket says 172.21.15.88:39912) 2023-10-16T20:06:02.741 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.732+0000 7f637ffff640 1 -- 172.21.15.88:0/904870754 learned_addr learned my addr 172.21.15.88:0/904870754 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:02.742 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.732+0000 7f63877fe640 1 --2- 172.21.15.88:0/904870754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63880c6510 0x7f63880c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:02.742 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.732+0000 7f6386ffd640 1 --2- 172.21.15.88:0/904870754 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6374003980 0x7f638814d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:02.742 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.732+0000 7f637ffff640 1 -- 172.21.15.88:0/904870754 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6374003980 msgr2=0x7f638814d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:02.742 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.732+0000 7f637ffff640 1 --2- 172.21.15.88:0/904870754 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6374003980 0x7f638814d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:02.742 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.732+0000 7f637ffff640 1 -- 172.21.15.88:0/904870754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63880c6510 msgr2=0x7f63880c6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:02.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.732+0000 7f637ffff640 1 --2- 172.21.15.88:0/904870754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63880c6510 0x7f63880c6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:02.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.732+0000 7f637ffff640 1 -- 172.21.15.88:0/904870754 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63880c7500 con 0x7f638815dba0 2023-10-16T20:06:02.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.732+0000 7f6386ffd640 1 --2- 172.21.15.88:0/904870754 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6374003980 0x7f638814d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:02.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.732+0000 7f63877fe640 1 --2- 172.21.15.88:0/904870754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63880c6510 0x7f63880c6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:02.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.732+0000 7f637ffff640 1 --2- 172.21.15.88:0/904870754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f638815dba0 0x7f638815e020 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f63700119d0 tx=0x7f6370011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:02.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.732+0000 7f637f7fe640 1 -- 172.21.15.88:0/904870754 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637000fb50 con 0x7f638815dba0 2023-10-16T20:06:02.744 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.732+0000 7f637f7fe640 1 -- 172.21.15.88:0/904870754 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637000fcb0 con 0x7f638815dba0 2023-10-16T20:06:02.744 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.732+0000 7f637f7fe640 1 -- 172.21.15.88:0/904870754 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6370014440 con 0x7f638815dba0 2023-10-16T20:06:02.744 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.732+0000 7f638d7c4640 1 -- 172.21.15.88:0/904870754 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63880c77e0 con 0x7f638815dba0 2023-10-16T20:06:02.744 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.732+0000 7f638d7c4640 1 -- 172.21.15.88:0/904870754 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63880c7da0 con 0x7f638815dba0 2023-10-16T20:06:02.744 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.732+0000 7f638d7c4640 1 -- 172.21.15.88:0/904870754 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f638804e9c0 con 0x7f638815dba0 2023-10-16T20:06:02.745 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.732+0000 7f637f7fe640 1 -- 172.21.15.88:0/904870754 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f63700145a0 con 0x7f638815dba0 2023-10-16T20:06:02.745 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.736+0000 7f637f7fe640 1 --2- 172.21.15.88:0/904870754 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f63680664a0 0x7f6368068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:02.746 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.736+0000 7f637f7fe640 1 -- 172.21.15.88:0/904870754 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6370084a90 con 0x7f638815dba0 2023-10-16T20:06:02.746 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.736+0000 7f6386ffd640 1 --2- 172.21.15.88:0/904870754 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f63680664a0 0x7f6368068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:02.746 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.736+0000 7f6386ffd640 1 --2- 172.21.15.88:0/904870754 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f63680664a0 0x7f6368068960 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f63780aa000 tx=0x7f6378001a80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:02.746 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.740+0000 7f637f7fe640 1 -- 172.21.15.88:0/904870754 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6370051c00 con 0x7f638815dba0 2023-10-16T20:06:02.891 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.888+0000 7f638d7c4640 1 -- 172.21.15.88:0/904870754 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f63880b3000 con 0x7f638815dba0 2023-10-16T20:06:02.892 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.888+0000 7f637f7fe640 1 -- 172.21.15.88:0/904870754 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6370022020 con 0x7f638815dba0 2023-10-16T20:06:02.892 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:06:02.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.888+0000 7f638d7c4640 1 -- 172.21.15.88:0/904870754 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f63680664a0 msgr2=0x7f6368068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:02.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.888+0000 7f638d7c4640 1 --2- 172.21.15.88:0/904870754 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f63680664a0 0x7f6368068960 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f63780aa000 tx=0x7f6378001a80 comp rx=0 tx=0).stop 2023-10-16T20:06:02.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.888+0000 7f638d7c4640 1 -- 172.21.15.88:0/904870754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f638815dba0 msgr2=0x7f638815e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:02.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.888+0000 7f638d7c4640 1 --2- 172.21.15.88:0/904870754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f638815dba0 0x7f638815e020 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f63700119d0 tx=0x7f6370011e90 comp rx=0 tx=0).stop 2023-10-16T20:06:02.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.888+0000 7f638d7c4640 1 -- 172.21.15.88:0/904870754 shutdown_connections 2023-10-16T20:06:02.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.888+0000 7f638d7c4640 1 --2- 172.21.15.88:0/904870754 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f63680664a0 0x7f6368068960 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:02.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.888+0000 7f638d7c4640 1 --2- 172.21.15.88:0/904870754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63880c6510 0x7f63880c6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:02.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.888+0000 7f638d7c4640 1 --2- 172.21.15.88:0/904870754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f638815dba0 0x7f638815e020 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:02.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.888+0000 7f638d7c4640 1 --2- 172.21.15.88:0/904870754 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6374003980 0x7f638814d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:02.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.888+0000 7f638d7c4640 1 -- 172.21.15.88:0/904870754 >> 172.21.15.88:0/904870754 conn(0x7f638806b6e0 msgr2=0x7f63880d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:02.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.888+0000 7f638d7c4640 1 -- 172.21.15.88:0/904870754 shutdown_connections 2023-10-16T20:06:02.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:02.888+0000 7f638d7c4640 1 -- 172.21.15.88:0/904870754 wait complete. 2023-10-16T20:06:02.913 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T20:06:03.052 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.048+0000 7f6a58a09640 1 Processor -- start 2023-10-16T20:06:03.053 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.048+0000 7f6a58a09640 1 -- start start 2023-10-16T20:06:03.054 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.048+0000 7f6a58a09640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a540b0da0 0x7f6a54159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:03.054 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.048+0000 7f6a58a09640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f6a5415a1f0 0x7f6a5406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:03.054 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.048+0000 7f6a58a09640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a5415ad30 con 0x7f6a540b4260 2023-10-16T20:06:03.055 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.048+0000 7f6a58a09640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a540b4640 con 0x7f6a5415a1f0 2023-10-16T20:06:03.055 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.048+0000 7f6a58a09640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5415a770 con 0x7f6a540b0da0 2023-10-16T20:06:03.055 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.048+0000 7f6a5259c640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f6a540b4260 0x7f6a5415dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:40520/0 (socket says 172.21.15.88:40520) 2023-10-16T20:06:03.055 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.048+0000 7f6a5259c640 1 -- 172.21.15.88:0/3875575480 learned_addr learned my addr 172.21.15.88:0/3875575480 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:03.057 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.048+0000 7f6a49d9b640 1 --2- 172.21.15.88:0/3875575480 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a540b0da0 0x7f6a54159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:03.057 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.052+0000 7f6a52d9d640 1 --2- 172.21.15.88:0/3875575480 >> v2:172.21.15.94:3300/0 conn(0x7f6a5415a1f0 0x7f6a5406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:03.057 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.052+0000 7f6a51d9b640 1 -- 172.21.15.88:0/3875575480 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2463838485 0 0) 0x7f6a5415ad30 con 0x7f6a540b4260 2023-10-16T20:06:03.057 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.052+0000 7f6a51d9b640 1 -- 172.21.15.88:0/3875575480 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a40003530 con 0x7f6a540b4260 2023-10-16T20:06:03.058 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.052+0000 7f6a51d9b640 1 -- 172.21.15.88:0/3875575480 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 460272411 0 0) 0x7f6a40003530 con 0x7f6a540b4260 2023-10-16T20:06:03.058 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.052+0000 7f6a51d9b640 1 -- 172.21.15.88:0/3875575480 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a540b0da0 msgr2=0x7f6a54159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:03.058 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.052+0000 7f6a51d9b640 1 --2- 172.21.15.88:0/3875575480 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a540b0da0 0x7f6a54159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:03.058 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.052+0000 7f6a51d9b640 1 -- 172.21.15.88:0/3875575480 >> v2:172.21.15.94:3300/0 conn(0x7f6a5415a1f0 msgr2=0x7f6a5406d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:03.058 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.052+0000 7f6a51d9b640 1 --2- 172.21.15.88:0/3875575480 >> v2:172.21.15.94:3300/0 conn(0x7f6a5415a1f0 0x7f6a5406d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:03.059 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.052+0000 7f6a51d9b640 1 -- 172.21.15.88:0/3875575480 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5415aa50 con 0x7f6a540b4260 2023-10-16T20:06:03.059 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.052+0000 7f6a51d9b640 1 -- 172.21.15.88:0/3875575480 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f6a44001a00 con 0x7f6a540b4260 2023-10-16T20:06:03.059 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.052+0000 7f6a51d9b640 1 -- 172.21.15.88:0/3875575480 >> v1:172.21.15.88:6789/0 conn(0x7f6a540b4260 legacy=0x7f6a5415dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:03.059 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.052+0000 7f6a51d9b640 1 --2- 172.21.15.88:0/3875575480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a40003980 0x7f6a40003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:03.059 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.052+0000 7f6a51d9b640 1 --2- 172.21.15.88:0/3875575480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a400047d0 0x7f6a40004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:03.060 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.052+0000 7f6a51d9b640 1 --2- 172.21.15.88:0/3875575480 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a540b0da0 0x7f6a40005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:03.060 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.052+0000 7f6a51d9b640 1 -- 172.21.15.88:0/3875575480 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a540b4640 con 0x7f6a400047d0 2023-10-16T20:06:03.060 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.052+0000 7f6a51d9b640 1 -- 172.21.15.88:0/3875575480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5415a770 con 0x7f6a40003980 2023-10-16T20:06:03.060 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.052+0000 7f6a51d9b640 1 -- 172.21.15.88:0/3875575480 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4000b9c0 con 0x7f6a540b0da0 2023-10-16T20:06:03.060 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.052+0000 7f6a49d9b640 1 --2- 172.21.15.88:0/3875575480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a400047d0 0x7f6a40004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:03.061 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.052+0000 7f6a52d9d640 1 --2- 172.21.15.88:0/3875575480 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a540b0da0 0x7f6a40005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:03.061 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.052+0000 7f6a5259c640 1 --2- 172.21.15.88:0/3875575480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a40003980 0x7f6a40003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:03.061 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.052+0000 7f6a49d9b640 1 -- 172.21.15.88:0/3875575480 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a540b0da0 msgr2=0x7f6a40005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:03.061 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.052+0000 7f6a49d9b640 1 --2- 172.21.15.88:0/3875575480 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a540b0da0 0x7f6a40005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:03.061 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.052+0000 7f6a49d9b640 1 -- 172.21.15.88:0/3875575480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a40003980 msgr2=0x7f6a40003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:03.062 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.052+0000 7f6a49d9b640 1 --2- 172.21.15.88:0/3875575480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a40003980 0x7f6a40003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:03.062 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.052+0000 7f6a49d9b640 1 -- 172.21.15.88:0/3875575480 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a40003530 con 0x7f6a400047d0 2023-10-16T20:06:03.065 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.056+0000 7f6a49d9b640 1 --2- 172.21.15.88:0/3875575480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a400047d0 0x7f6a40004c50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c009ae0 tx=0x7f6a3c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5219288c9b06f8ec server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:03.065 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.056+0000 7f6a51d9b640 1 -- 172.21.15.88:0/3875575480 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c0a9450 con 0x7f6a400047d0 2023-10-16T20:06:03.065 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.056+0000 7f6a51d9b640 1 -- 172.21.15.88:0/3875575480 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3c0a95b0 con 0x7f6a400047d0 2023-10-16T20:06:03.066 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.056+0000 7f6a51d9b640 1 -- 172.21.15.88:0/3875575480 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c0a9960 con 0x7f6a400047d0 2023-10-16T20:06:03.066 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.056+0000 7f6a58a09640 1 -- 172.21.15.88:0/3875575480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a400047d0 msgr2=0x7f6a40004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:03.066 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.056+0000 7f6a58a09640 1 --2- 172.21.15.88:0/3875575480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a400047d0 0x7f6a40004c50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c009ae0 tx=0x7f6a3c01c9f0 comp rx=0 tx=0).stop 2023-10-16T20:06:03.066 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.056+0000 7f6a52d9d640 1 -- 172.21.15.88:0/3875575480 reap_dead start 2023-10-16T20:06:03.066 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.056+0000 7f6a58a09640 1 -- 172.21.15.88:0/3875575480 shutdown_connections 2023-10-16T20:06:03.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.056+0000 7f6a58a09640 1 -- 172.21.15.88:0/3875575480 >> 172.21.15.88:0/3875575480 conn(0x7f6a540cb080 msgr2=0x7f6a540cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:03.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.056+0000 7f6a58a09640 1 -- 172.21.15.88:0/3875575480 shutdown_connections 2023-10-16T20:06:03.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.056+0000 7f6a58a09640 1 -- 172.21.15.88:0/3875575480 wait complete. 2023-10-16T20:06:03.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.056+0000 7f6a58a09640 1 Processor -- start 2023-10-16T20:06:03.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.056+0000 7f6a58a09640 1 -- start start 2023-10-16T20:06:03.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.060+0000 7f6a58a09640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a400047d0 0x7f6a5406c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:03.068 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.060+0000 7f6a58a09640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a541f53d0 0x7f6a541f5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:03.068 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.060+0000 7f6a58a09640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a540acda0 0x7f6a540ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:03.068 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.060+0000 7f6a58a09640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a541f9920 con 0x7f6a540acda0 2023-10-16T20:06:03.068 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.060+0000 7f6a58a09640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a540ad760 con 0x7f6a541f53d0 2023-10-16T20:06:03.068 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.060+0000 7f6a58a09640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a540abfe0 con 0x7f6a400047d0 2023-10-16T20:06:03.069 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.060+0000 7f6a52d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a540acda0 0x7f6a540ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:03.069 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.060+0000 7f6a52d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a540acda0 0x7f6a540ad220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39932/0 (socket says 172.21.15.88:39932) 2023-10-16T20:06:03.069 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.060+0000 7f6a52d9d640 1 -- 172.21.15.88:0/2125481796 learned_addr learned my addr 172.21.15.88:0/2125481796 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:03.069 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.060+0000 7f6a5259c640 1 --2- 172.21.15.88:0/2125481796 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a400047d0 0x7f6a5406c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:03.069 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.060+0000 7f6a49d9b640 1 --2- 172.21.15.88:0/2125481796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a541f53d0 0x7f6a541f5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:03.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.060+0000 7f6a52d9d640 1 -- 172.21.15.88:0/2125481796 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a400047d0 msgr2=0x7f6a5406c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:03.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.060+0000 7f6a52d9d640 1 --2- 172.21.15.88:0/2125481796 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a400047d0 0x7f6a5406c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:03.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.060+0000 7f6a52d9d640 1 -- 172.21.15.88:0/2125481796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a541f53d0 msgr2=0x7f6a541f5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:03.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.060+0000 7f6a52d9d640 1 --2- 172.21.15.88:0/2125481796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a541f53d0 0x7f6a541f5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:03.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.060+0000 7f6a52d9d640 1 -- 172.21.15.88:0/2125481796 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a540ac2c0 con 0x7f6a540acda0 2023-10-16T20:06:03.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.060+0000 7f6a49d9b640 1 --2- 172.21.15.88:0/2125481796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a541f53d0 0x7f6a541f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:03.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.060+0000 7f6a52d9d640 1 --2- 172.21.15.88:0/2125481796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a540acda0 0x7f6a540ad220 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6a540b1fc0 tx=0x7f6a4c011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:03.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.060+0000 7f6a4b7fe640 1 -- 172.21.15.88:0/2125481796 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c004240 con 0x7f6a540acda0 2023-10-16T20:06:03.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.060+0000 7f6a4b7fe640 1 -- 172.21.15.88:0/2125481796 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4c0043a0 con 0x7f6a540acda0 2023-10-16T20:06:03.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.060+0000 7f6a58a09640 1 -- 172.21.15.88:0/2125481796 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a540ac5a0 con 0x7f6a540acda0 2023-10-16T20:06:03.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.060+0000 7f6a4b7fe640 1 -- 172.21.15.88:0/2125481796 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c00f900 con 0x7f6a540acda0 2023-10-16T20:06:03.072 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.060+0000 7f6a58a09640 1 -- 172.21.15.88:0/2125481796 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a5415faf0 con 0x7f6a540acda0 2023-10-16T20:06:03.072 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.060+0000 7f6a58a09640 1 -- 172.21.15.88:0/2125481796 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a540caef0 con 0x7f6a540acda0 2023-10-16T20:06:03.073 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.060+0000 7f6a4b7fe640 1 -- 172.21.15.88:0/2125481796 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6a4c00fa60 con 0x7f6a540acda0 2023-10-16T20:06:03.073 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.064+0000 7f6a4b7fe640 1 --2- 172.21.15.88:0/2125481796 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f6a340664a0 0x7f6a34068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:03.073 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.064+0000 7f6a4b7fe640 1 -- 172.21.15.88:0/2125481796 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6a4c0847b0 con 0x7f6a540acda0 2023-10-16T20:06:03.073 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.064+0000 7f6a5259c640 1 --2- 172.21.15.88:0/2125481796 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f6a340664a0 0x7f6a34068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:03.073 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.064+0000 7f6a5259c640 1 --2- 172.21.15.88:0/2125481796 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f6a340664a0 0x7f6a34068960 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6a541f9aa0 tx=0x7f6a44006040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:03.074 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.068+0000 7f6a4b7fe640 1 -- 172.21.15.88:0/2125481796 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6a4c051860 con 0x7f6a540acda0 2023-10-16T20:06:03.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.216+0000 7f6a58a09640 1 -- 172.21.15.88:0/2125481796 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6a5404e9c0 con 0x7f6a540acda0 2023-10-16T20:06:03.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.216+0000 7f6a4b7fe640 1 -- 172.21.15.88:0/2125481796 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6a4c054e80 con 0x7f6a540acda0 2023-10-16T20:06:03.220 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2023-10-16T20:06:03.222 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.216+0000 7f6a58a09640 1 -- 172.21.15.88:0/2125481796 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f6a340664a0 msgr2=0x7f6a34068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:03.222 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.216+0000 7f6a58a09640 1 --2- 172.21.15.88:0/2125481796 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f6a340664a0 0x7f6a34068960 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6a541f9aa0 tx=0x7f6a44006040 comp rx=0 tx=0).stop 2023-10-16T20:06:03.222 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.216+0000 7f6a58a09640 1 -- 172.21.15.88:0/2125481796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a540acda0 msgr2=0x7f6a540ad220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:03.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.216+0000 7f6a58a09640 1 --2- 172.21.15.88:0/2125481796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a540acda0 0x7f6a540ad220 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6a540b1fc0 tx=0x7f6a4c011cf0 comp rx=0 tx=0).stop 2023-10-16T20:06:03.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.216+0000 7f6a58a09640 1 -- 172.21.15.88:0/2125481796 shutdown_connections 2023-10-16T20:06:03.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.216+0000 7f6a58a09640 1 --2- 172.21.15.88:0/2125481796 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f6a340664a0 0x7f6a34068960 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:03.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.216+0000 7f6a58a09640 1 --2- 172.21.15.88:0/2125481796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a540acda0 0x7f6a540ad220 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:03.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.216+0000 7f6a58a09640 1 --2- 172.21.15.88:0/2125481796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a541f53d0 0x7f6a541f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:03.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.216+0000 7f6a58a09640 1 --2- 172.21.15.88:0/2125481796 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a400047d0 0x7f6a5406c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:03.224 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.216+0000 7f6a58a09640 1 -- 172.21.15.88:0/2125481796 >> 172.21.15.88:0/2125481796 conn(0x7f6a540cb080 msgr2=0x7f6a54158130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:03.224 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.216+0000 7f6a58a09640 1 -- 172.21.15.88:0/2125481796 shutdown_connections 2023-10-16T20:06:03.224 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.216+0000 7f6a58a09640 1 -- 172.21.15.88:0/2125481796 wait complete. 2023-10-16T20:06:03.241 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T20:06:03.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.376+0000 7f37b2a46640 1 Processor -- start 2023-10-16T20:06:03.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.376+0000 7f37b2a46640 1 -- start start 2023-10-16T20:06:03.384 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.380+0000 7f37b2a46640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f37ac0b4260 0x7f37ac15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:03.384 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.380+0000 7f37b2a46640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37ac0b0da0 0x7f37ac159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:03.384 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.380+0000 7f37b2a46640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ac15ad30 con 0x7f37ac15a1f0 2023-10-16T20:06:03.384 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.380+0000 7f37b2a46640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37ac0b4640 con 0x7f37ac0b4260 2023-10-16T20:06:03.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.380+0000 7f37b2a46640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac15a770 con 0x7f37ac0b0da0 2023-10-16T20:06:03.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.380+0000 7f37abfff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f37ac0b4260 0x7f37ac15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:03.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.380+0000 7f37ab7fe640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37ac0b0da0 0x7f37ac159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:03.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.380+0000 7f37abfff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f37ac0b4260 0x7f37ac15dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:58560/0 (socket says 172.21.15.88:58560) 2023-10-16T20:06:03.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.380+0000 7f37ab7fe640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37ac0b0da0 0x7f37ac159cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:41310/0 (socket says 172.21.15.88:41310) 2023-10-16T20:06:03.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.380+0000 7f37abfff640 1 -- 172.21.15.88:0/2003110848 learned_addr learned my addr 172.21.15.88:0/2003110848 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:03.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.380+0000 7f37b0fe3640 1 --1- 172.21.15.88:0/2003110848 >> v1:172.21.15.94:6789/0 conn(0x7f37ac15a1f0 0x7f37ac06d3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:57132/0 (socket says 172.21.15.88:57132) 2023-10-16T20:06:03.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.380+0000 7f37abfff640 1 -- 172.21.15.88:0/2003110848 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37ac0b0da0 msgr2=0x7f37ac159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:03.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.380+0000 7f37abfff640 1 --2- 172.21.15.88:0/2003110848 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37ac0b0da0 0x7f37ac159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:03.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.380+0000 7f37abfff640 1 -- 172.21.15.88:0/2003110848 >> v1:172.21.15.94:6789/0 conn(0x7f37ac15a1f0 legacy=0x7f37ac06d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:03.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.380+0000 7f37abfff640 1 -- 172.21.15.88:0/2003110848 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac15aa50 con 0x7f37ac0b4260 2023-10-16T20:06:03.388 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.384+0000 7f37abfff640 1 --2- 172.21.15.88:0/2003110848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37ac0b4260 0x7f37ac15dc90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f379c009870 tx=0x7f379c01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8600ba229b77ca78 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:03.388 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.384+0000 7f37aaffd640 1 -- 172.21.15.88:0/2003110848 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c0ab680 con 0x7f37ac0b4260 2023-10-16T20:06:03.388 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.384+0000 7f37aaffd640 1 -- 172.21.15.88:0/2003110848 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379c0ab7e0 con 0x7f37ac0b4260 2023-10-16T20:06:03.388 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.384+0000 7f37aaffd640 1 -- 172.21.15.88:0/2003110848 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c0abb70 con 0x7f37ac0b4260 2023-10-16T20:06:03.389 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.384+0000 7f37b2a46640 1 -- 172.21.15.88:0/2003110848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37ac0b4260 msgr2=0x7f37ac15dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:03.389 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.384+0000 7f37b2a46640 1 --2- 172.21.15.88:0/2003110848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37ac0b4260 0x7f37ac15dc90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f379c009870 tx=0x7f379c01c480 comp rx=0 tx=0).stop 2023-10-16T20:06:03.389 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.384+0000 7f37b2a46640 1 -- 172.21.15.88:0/2003110848 shutdown_connections 2023-10-16T20:06:03.390 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.384+0000 7f37b2a46640 1 --2- 172.21.15.88:0/2003110848 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37ac0b0da0 0x7f37ac159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:03.390 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.384+0000 7f37b2a46640 1 --2- 172.21.15.88:0/2003110848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37ac0b4260 0x7f37ac15dc90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:03.390 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.384+0000 7f37b2a46640 1 -- 172.21.15.88:0/2003110848 >> 172.21.15.88:0/2003110848 conn(0x7f37ac0cb080 msgr2=0x7f37ac0cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:03.390 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.384+0000 7f37b2a46640 1 -- 172.21.15.88:0/2003110848 shutdown_connections 2023-10-16T20:06:03.390 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.384+0000 7f37b2a46640 1 -- 172.21.15.88:0/2003110848 wait complete. 2023-10-16T20:06:03.393 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.384+0000 7f37b2a46640 1 Processor -- start 2023-10-16T20:06:03.393 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.384+0000 7f37b2a46640 1 -- start start 2023-10-16T20:06:03.393 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.388+0000 7f37b2a46640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37ac0b0da0 0x7f37ac14f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:03.393 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.388+0000 7f37b2a46640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37ac0b4260 0x7f37ac0aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:03.393 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.388+0000 7f37b2a46640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37ac15a1f0 0x7f37ac0af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:03.394 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.388+0000 7f37b2a46640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac14ca20 con 0x7f37ac0b0da0 2023-10-16T20:06:03.394 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.388+0000 7f37b2a46640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac153e60 con 0x7f37ac15a1f0 2023-10-16T20:06:03.394 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.388+0000 7f37b2a46640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac154140 con 0x7f37ac0b4260 2023-10-16T20:06:03.394 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.388+0000 7f37abfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37ac0b0da0 0x7f37ac14f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:03.394 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.388+0000 7f37abfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37ac0b0da0 0x7f37ac14f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39946/0 (socket says 172.21.15.88:39946) 2023-10-16T20:06:03.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.388+0000 7f37abfff640 1 -- 172.21.15.88:0/3067874241 learned_addr learned my addr 172.21.15.88:0/3067874241 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:03.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.388+0000 7f37b0fe3640 1 --2- 172.21.15.88:0/3067874241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37ac15a1f0 0x7f37ac0af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:03.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.388+0000 7f37ab7fe640 1 --2- 172.21.15.88:0/3067874241 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37ac0b4260 0x7f37ac0aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:03.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.388+0000 7f37abfff640 1 -- 172.21.15.88:0/3067874241 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37ac0b4260 msgr2=0x7f37ac0aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:03.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.388+0000 7f37abfff640 1 --2- 172.21.15.88:0/3067874241 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37ac0b4260 0x7f37ac0aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:03.396 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.388+0000 7f37abfff640 1 -- 172.21.15.88:0/3067874241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37ac15a1f0 msgr2=0x7f37ac0af2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:03.396 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.388+0000 7f37abfff640 1 --2- 172.21.15.88:0/3067874241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37ac15a1f0 0x7f37ac0af2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:03.396 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.388+0000 7f37abfff640 1 -- 172.21.15.88:0/3067874241 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379c0094d0 con 0x7f37ac0b0da0 2023-10-16T20:06:03.396 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.388+0000 7f37abfff640 1 --2- 172.21.15.88:0/3067874241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37ac0b0da0 0x7f37ac14f080 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f379c0ad040 tx=0x7f379c01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:03.396 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.388+0000 7f37a8ff9640 1 -- 172.21.15.88:0/3067874241 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c0ab680 con 0x7f37ac0b0da0 2023-10-16T20:06:03.396 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.388+0000 7f37a8ff9640 1 -- 172.21.15.88:0/3067874241 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379c0b4800 con 0x7f37ac0b0da0 2023-10-16T20:06:03.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.388+0000 7f37b2a46640 1 -- 172.21.15.88:0/3067874241 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37ac154420 con 0x7f37ac0b0da0 2023-10-16T20:06:03.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.388+0000 7f37a8ff9640 1 -- 172.21.15.88:0/3067874241 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c0b4bb0 con 0x7f37ac0b0da0 2023-10-16T20:06:03.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.388+0000 7f37b2a46640 1 -- 172.21.15.88:0/3067874241 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ac1643c0 con 0x7f37ac0b0da0 2023-10-16T20:06:03.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.388+0000 7f37b2a46640 1 -- 172.21.15.88:0/3067874241 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37ac04e9c0 con 0x7f37ac0b0da0 2023-10-16T20:06:03.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.392+0000 7f37a8ff9640 1 -- 172.21.15.88:0/3067874241 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f379c0b4d10 con 0x7f37ac0b0da0 2023-10-16T20:06:03.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.392+0000 7f37a8ff9640 1 --2- 172.21.15.88:0/3067874241 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f37980664a0 0x7f3798068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:03.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.392+0000 7f37a8ff9640 1 -- 172.21.15.88:0/3067874241 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f379c11efd0 con 0x7f37ac0b0da0 2023-10-16T20:06:03.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.392+0000 7f37ab7fe640 1 --2- 172.21.15.88:0/3067874241 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f37980664a0 0x7f3798068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:03.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.392+0000 7f37ab7fe640 1 --2- 172.21.15.88:0/3067874241 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f37980664a0 0x7f3798068960 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f37ac15c990 tx=0x7f37a0009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:03.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.396+0000 7f37a8ff9640 1 -- 172.21.15.88:0/3067874241 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f379c0ec0a0 con 0x7f37ac0b0da0 2023-10-16T20:06:03.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.544+0000 7f37b2a46640 1 -- 172.21.15.88:0/3067874241 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f37ac15cbc0 con 0x7f37ac0b0da0 2023-10-16T20:06:03.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.544+0000 7f37a8ff9640 1 -- 172.21.15.88:0/3067874241 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f379c0ef6c0 con 0x7f37ac0b0da0 2023-10-16T20:06:03.551 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:06:03.552 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.548+0000 7f37b2a46640 1 -- 172.21.15.88:0/3067874241 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f37980664a0 msgr2=0x7f3798068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:03.553 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.548+0000 7f37b2a46640 1 --2- 172.21.15.88:0/3067874241 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f37980664a0 0x7f3798068960 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f37ac15c990 tx=0x7f37a0009190 comp rx=0 tx=0).stop 2023-10-16T20:06:03.553 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.548+0000 7f37b2a46640 1 -- 172.21.15.88:0/3067874241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37ac0b0da0 msgr2=0x7f37ac14f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:03.553 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.548+0000 7f37b2a46640 1 --2- 172.21.15.88:0/3067874241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37ac0b0da0 0x7f37ac14f080 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f379c0ad040 tx=0x7f379c01acc0 comp rx=0 tx=0).stop 2023-10-16T20:06:03.553 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.548+0000 7f37b2a46640 1 -- 172.21.15.88:0/3067874241 shutdown_connections 2023-10-16T20:06:03.553 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.548+0000 7f37b2a46640 1 --2- 172.21.15.88:0/3067874241 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f37980664a0 0x7f3798068960 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:03.553 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.548+0000 7f37b2a46640 1 --2- 172.21.15.88:0/3067874241 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37ac0b4260 0x7f37ac0aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:03.554 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.548+0000 7f37b2a46640 1 --2- 172.21.15.88:0/3067874241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37ac15a1f0 0x7f37ac0af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:03.554 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.548+0000 7f37b2a46640 1 --2- 172.21.15.88:0/3067874241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37ac0b0da0 0x7f37ac14f080 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:03.554 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.548+0000 7f37b2a46640 1 -- 172.21.15.88:0/3067874241 >> 172.21.15.88:0/3067874241 conn(0x7f37ac0cb080 msgr2=0x7f37ac0adbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:03.554 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.548+0000 7f37b2a46640 1 -- 172.21.15.88:0/3067874241 shutdown_connections 2023-10-16T20:06:03.554 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.548+0000 7f37b2a46640 1 -- 172.21.15.88:0/3067874241 wait complete. 2023-10-16T20:06:03.572 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T20:06:03.710 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.704+0000 7fc7f506b640 1 Processor -- start 2023-10-16T20:06:03.710 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.704+0000 7fc7f506b640 1 -- start start 2023-10-16T20:06:03.711 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.708+0000 7fc7f506b640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc7f015cb80 0x7fc7f00b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:03.711 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.708+0000 7fc7f506b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fc7f00b28b0 0x7fc7f00ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:03.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.708+0000 7fc7f506b640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f006b490 con 0x7fc7f00cea00 2023-10-16T20:06:03.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.708+0000 7fc7f506b640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f00b2d20 con 0x7fc7f015cb80 2023-10-16T20:06:03.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.708+0000 7fc7f506b640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f00cf180 con 0x7fc7f00b28b0 2023-10-16T20:06:03.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.708+0000 7fc7eed9d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc7f015cb80 0x7fc7f00b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:03.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.708+0000 7fc7ef59e640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fc7f00cea00 0x7fc7f006ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:40536/0 (socket says 172.21.15.88:40536) 2023-10-16T20:06:03.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.708+0000 7fc7ef59e640 1 -- 172.21.15.88:0/8175366 learned_addr learned my addr 172.21.15.88:0/8175366 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:03.713 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.708+0000 7fc7eed9d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc7f015cb80 0x7fc7f00b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39954/0 (socket says 172.21.15.88:39954) 2023-10-16T20:06:03.713 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.708+0000 7fc7e659c640 1 --2- 172.21.15.88:0/8175366 >> v2:172.21.15.94:3300/0 conn(0x7fc7f00b28b0 0x7fc7f00ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:03.713 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.708+0000 7fc7eed9d640 1 -- 172.21.15.88:0/8175366 >> v2:172.21.15.94:3300/0 conn(0x7fc7f00b28b0 msgr2=0x7fc7f00ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:03.714 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.708+0000 7fc7eed9d640 1 --2- 172.21.15.88:0/8175366 >> v2:172.21.15.94:3300/0 conn(0x7fc7f00b28b0 0x7fc7f00ce4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:03.714 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.708+0000 7fc7ee59c640 1 -- 172.21.15.88:0/8175366 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3805269595 0 0) 0x7fc7f006b490 con 0x7fc7f00cea00 2023-10-16T20:06:03.714 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.708+0000 7fc7eed9d640 1 -- 172.21.15.88:0/8175366 >> v1:172.21.15.88:6789/0 conn(0x7fc7f00cea00 legacy=0x7fc7f006ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:03.714 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.708+0000 7fc7eed9d640 1 -- 172.21.15.88:0/8175366 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f006b700 con 0x7fc7f015cb80 2023-10-16T20:06:03.714 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.708+0000 7fc7e659c640 1 --2- 172.21.15.88:0/8175366 >> v2:172.21.15.94:3300/0 conn(0x7fc7f00b28b0 0x7fc7f00ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:06:03.716 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.708+0000 7fc7eed9d640 1 --2- 172.21.15.88:0/8175366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7f015cb80 0x7fc7f00b2370 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc7f00c4190 tx=0x7fc7e001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6f591ce6cb52e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:03.716 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.712+0000 7fc7ee59c640 1 -- 172.21.15.88:0/8175366 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e00a5850 con 0x7fc7f015cb80 2023-10-16T20:06:03.716 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.712+0000 7fc7ee59c640 1 -- 172.21.15.88:0/8175366 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e00a59b0 con 0x7fc7f015cb80 2023-10-16T20:06:03.716 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.712+0000 7fc7ee59c640 1 -- 172.21.15.88:0/8175366 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e00a5d40 con 0x7fc7f015cb80 2023-10-16T20:06:03.716 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.712+0000 7fc7f506b640 1 -- 172.21.15.88:0/8175366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7f015cb80 msgr2=0x7fc7f00b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:03.716 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.712+0000 7fc7f506b640 1 --2- 172.21.15.88:0/8175366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7f015cb80 0x7fc7f00b2370 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc7f00c4190 tx=0x7fc7e001c8e0 comp rx=0 tx=0).stop 2023-10-16T20:06:03.717 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.712+0000 7fc7f506b640 1 -- 172.21.15.88:0/8175366 shutdown_connections 2023-10-16T20:06:03.717 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.712+0000 7fc7f506b640 1 --2- 172.21.15.88:0/8175366 >> v2:172.21.15.94:3300/0 conn(0x7fc7f00b28b0 0x7fc7f00ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:03.717 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.712+0000 7fc7f506b640 1 --2- 172.21.15.88:0/8175366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7f015cb80 0x7fc7f00b2370 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:03.717 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.712+0000 7fc7f506b640 1 -- 172.21.15.88:0/8175366 >> 172.21.15.88:0/8175366 conn(0x7fc7f00cd370 msgr2=0x7fc7f00cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:03.717 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.712+0000 7fc7f506b640 1 -- 172.21.15.88:0/8175366 shutdown_connections 2023-10-16T20:06:03.718 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.712+0000 7fc7f506b640 1 -- 172.21.15.88:0/8175366 wait complete. 2023-10-16T20:06:03.720 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.712+0000 7fc7f506b640 1 Processor -- start 2023-10-16T20:06:03.720 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.712+0000 7fc7f506b640 1 -- start start 2023-10-16T20:06:03.720 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.712+0000 7fc7f506b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7f00b28b0 0x7fc7f014dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:03.720 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.712+0000 7fc7f506b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc7f00cea00 0x7fc7f0153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:03.720 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.712+0000 7fc7f506b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7f015cb80 0x7fc7f014c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:03.720 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.712+0000 7fc7f506b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f015a040 con 0x7fc7f00b28b0 2023-10-16T20:06:03.721 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.712+0000 7fc7f506b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f014e4e0 con 0x7fc7f015cb80 2023-10-16T20:06:03.721 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.712+0000 7fc7f506b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f0159730 con 0x7fc7f00cea00 2023-10-16T20:06:03.721 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.716+0000 7fc7eed9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7f00b28b0 0x7fc7f014dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:03.721 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.716+0000 7fc7eed9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7f00b28b0 0x7fc7f014dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39966/0 (socket says 172.21.15.88:39966) 2023-10-16T20:06:03.721 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.716+0000 7fc7eed9d640 1 -- 172.21.15.88:0/1131424076 learned_addr learned my addr 172.21.15.88:0/1131424076 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:03.722 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.716+0000 7fc7ef59e640 1 --2- 172.21.15.88:0/1131424076 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7f015cb80 0x7fc7f014c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:03.722 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.716+0000 7fc7e659c640 1 --2- 172.21.15.88:0/1131424076 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc7f00cea00 0x7fc7f0153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:03.722 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.716+0000 7fc7eed9d640 1 -- 172.21.15.88:0/1131424076 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc7f00cea00 msgr2=0x7fc7f0153ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:03.722 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.716+0000 7fc7eed9d640 1 --2- 172.21.15.88:0/1131424076 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc7f00cea00 0x7fc7f0153ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:03.722 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.716+0000 7fc7eed9d640 1 -- 172.21.15.88:0/1131424076 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7f015cb80 msgr2=0x7fc7f014c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:03.723 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.716+0000 7fc7eed9d640 1 --2- 172.21.15.88:0/1131424076 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7f015cb80 0x7fc7f014c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:03.723 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.716+0000 7fc7eed9d640 1 -- 172.21.15.88:0/1131424076 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e0009a70 con 0x7fc7f00b28b0 2023-10-16T20:06:03.723 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.716+0000 7fc7eed9d640 1 --2- 172.21.15.88:0/1131424076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7f00b28b0 0x7fc7f014dfa0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc7e00b0040 tx=0x7fc7e0019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:03.723 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.716+0000 7fc7e7fff640 1 -- 172.21.15.88:0/1131424076 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e00b9860 con 0x7fc7f00b28b0 2023-10-16T20:06:03.723 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.716+0000 7fc7e7fff640 1 -- 172.21.15.88:0/1131424076 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e00b99c0 con 0x7fc7f00b28b0 2023-10-16T20:06:03.724 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.716+0000 7fc7e7fff640 1 -- 172.21.15.88:0/1131424076 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e00b9d50 con 0x7fc7f00b28b0 2023-10-16T20:06:03.724 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.716+0000 7fc7f506b640 1 -- 172.21.15.88:0/1131424076 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f014e710 con 0x7fc7f00b28b0 2023-10-16T20:06:03.724 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.716+0000 7fc7f506b640 1 -- 172.21.15.88:0/1131424076 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f0159c60 con 0x7fc7f00b28b0 2023-10-16T20:06:03.725 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.716+0000 7fc7f506b640 1 -- 172.21.15.88:0/1131424076 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7f004e9c0 con 0x7fc7f00b28b0 2023-10-16T20:06:03.725 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.716+0000 7fc7e7fff640 1 -- 172.21.15.88:0/1131424076 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc7e00ad440 con 0x7fc7f00b28b0 2023-10-16T20:06:03.725 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.716+0000 7fc7e7fff640 1 --2- 172.21.15.88:0/1131424076 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc7d00664a0 0x7fc7d0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:03.725 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.716+0000 7fc7e7fff640 1 -- 172.21.15.88:0/1131424076 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc7e011f700 con 0x7fc7f00b28b0 2023-10-16T20:06:03.726 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.720+0000 7fc7e659c640 1 --2- 172.21.15.88:0/1131424076 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc7d00664a0 0x7fc7d0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:03.726 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.720+0000 7fc7e659c640 1 --2- 172.21.15.88:0/1131424076 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc7d00664a0 0x7fc7d0068960 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc7f014eff0 tx=0x7fc7d8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:03.726 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.720+0000 7fc7e7fff640 1 -- 172.21.15.88:0/1131424076 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc7e00ec8d0 con 0x7fc7f00b28b0 2023-10-16T20:06:03.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.900+0000 7fc7f506b640 1 -- 172.21.15.88:0/1131424076 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7f01556e0 con 0x7fc7f00b28b0 2023-10-16T20:06:03.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.900+0000 7fc7e7fff640 1 -- 172.21.15.88:0/1131424076 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4906 (secure 0 0 0) 0x7fc7e00efef0 con 0x7fc7f00b28b0 2023-10-16T20:06:03.907 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:06:03.907 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3803627113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3803627113},{"type":"v1","addr":"172.21.15.94:6834","nonce":3803627113}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3758093239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":3758093239},{"type":"v1","addr":"172.21.15.88:6837","nonce":3758093239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/3836772385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3836772385},{"type":"v1","addr":"172.21.15.88:6836","nonce":3836772385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T20:04:56.759719+0000","modified":"2023-10-16T20:05:14.569618+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6835/555575750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":555575750},{"type":"v1","addr":"172.21.15.94:6835","nonce":555575750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T20:06:03.908 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 9 2023-10-16T20:06:03.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.904+0000 7fc7f506b640 1 -- 172.21.15.88:0/1131424076 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc7d00664a0 msgr2=0x7fc7d0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:03.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.904+0000 7fc7f506b640 1 --2- 172.21.15.88:0/1131424076 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc7d00664a0 0x7fc7d0068960 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc7f014eff0 tx=0x7fc7d8008040 comp rx=0 tx=0).stop 2023-10-16T20:06:03.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.904+0000 7fc7f506b640 1 -- 172.21.15.88:0/1131424076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7f00b28b0 msgr2=0x7fc7f014dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:03.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.904+0000 7fc7f506b640 1 --2- 172.21.15.88:0/1131424076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7f00b28b0 0x7fc7f014dfa0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc7e00b0040 tx=0x7fc7e0019680 comp rx=0 tx=0).stop 2023-10-16T20:06:03.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.904+0000 7fc7f506b640 1 -- 172.21.15.88:0/1131424076 shutdown_connections 2023-10-16T20:06:03.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.904+0000 7fc7f506b640 1 --2- 172.21.15.88:0/1131424076 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc7d00664a0 0x7fc7d0068960 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:03.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.904+0000 7fc7f506b640 1 --2- 172.21.15.88:0/1131424076 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc7f00cea00 0x7fc7f0153ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:03.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.904+0000 7fc7f506b640 1 --2- 172.21.15.88:0/1131424076 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7f015cb80 0x7fc7f014c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:03.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.904+0000 7fc7f506b640 1 --2- 172.21.15.88:0/1131424076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7f00b28b0 0x7fc7f014dfa0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:03.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.904+0000 7fc7f506b640 1 -- 172.21.15.88:0/1131424076 >> 172.21.15.88:0/1131424076 conn(0x7fc7f00cd370 msgr2=0x7fc7f0158600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:03.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.904+0000 7fc7f506b640 1 -- 172.21.15.88:0/1131424076 shutdown_connections 2023-10-16T20:06:03.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:03.904+0000 7fc7f506b640 1 -- 172.21.15.88:0/1131424076 wait complete. 2023-10-16T20:06:03.927 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:06:04.068 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.060+0000 7f14184ea640 1 Processor -- start 2023-10-16T20:06:04.068 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.060+0000 7f14184ea640 1 -- start start 2023-10-16T20:06:04.068 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.060+0000 7f14184ea640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f141015d9f0 0x7f1410161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:04.069 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.060+0000 7f14184ea640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14100b87e0 con 0x7f14100d22e0 2023-10-16T20:06:04.069 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.060+0000 7f14184ea640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f141015df10 con 0x7f14100d26c0 2023-10-16T20:06:04.069 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.060+0000 7f14184ea640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f141015e1f0 con 0x7f141015d9f0 2023-10-16T20:06:04.069 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.060+0000 7f1416286640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f14100d22e0 0x7f141015d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:40548/0 (socket says 172.21.15.88:40548) 2023-10-16T20:06:04.069 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.060+0000 7f1416286640 1 -- 172.21.15.88:0/8720802 learned_addr learned my addr 172.21.15.88:0/8720802 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:04.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.060+0000 7f1415a85640 1 --2- 172.21.15.88:0/8720802 >> v2:172.21.15.94:3300/0 conn(0x7f141015d9f0 0x7f1410161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:04.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.060+0000 7f1415284640 1 -- 172.21.15.88:0/8720802 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2365974448 0 0) 0x7f14100b87e0 con 0x7f14100d22e0 2023-10-16T20:06:04.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.060+0000 7f1415284640 1 -- 172.21.15.88:0/8720802 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13fc003530 con 0x7f14100d22e0 2023-10-16T20:06:04.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.060+0000 7f1415284640 1 -- 172.21.15.88:0/8720802 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3876453754 0 0) 0x7f141015df10 con 0x7f14100d26c0 2023-10-16T20:06:04.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.060+0000 7f1415284640 1 -- 172.21.15.88:0/8720802 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14100b87e0 con 0x7f14100d26c0 2023-10-16T20:06:04.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.060+0000 7f1415284640 1 -- 172.21.15.88:0/8720802 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1177264027 0 0) 0x7f13fc003530 con 0x7f14100d22e0 2023-10-16T20:06:04.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.060+0000 7f1415284640 1 -- 172.21.15.88:0/8720802 >> v2:172.21.15.94:3300/0 conn(0x7f141015d9f0 msgr2=0x7f1410161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:04.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.060+0000 7f1415284640 1 --2- 172.21.15.88:0/8720802 >> v2:172.21.15.94:3300/0 conn(0x7f141015d9f0 0x7f1410161450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:04.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.060+0000 7f1415284640 1 -- 172.21.15.88:0/8720802 >> v1:172.21.15.94:6789/0 conn(0x7f14100d26c0 legacy=0x7f14100cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:04.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.060+0000 7f1415284640 1 -- 172.21.15.88:0/8720802 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1410160970 con 0x7f14100d22e0 2023-10-16T20:06:04.072 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.060+0000 7f1415284640 1 -- 172.21.15.88:0/8720802 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f140c001680 con 0x7f14100d22e0 2023-10-16T20:06:04.072 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.060+0000 7f1415284640 1 -- 172.21.15.88:0/8720802 >> v1:172.21.15.88:6789/0 conn(0x7f14100d22e0 legacy=0x7f141015d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:04.072 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.060+0000 7f1415284640 1 --2- 172.21.15.88:0/8720802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13fc003b10 0x7f13fc003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:04.072 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.060+0000 7f1415284640 1 --2- 172.21.15.88:0/8720802 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f13fc0047d0 0x7f13fc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:04.073 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.060+0000 7f1415284640 1 --2- 172.21.15.88:0/8720802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13fc005190 0x7f13fc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:04.073 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.060+0000 7f1415284640 1 -- 172.21.15.88:0/8720802 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141015e1f0 con 0x7f13fc005190 2023-10-16T20:06:04.073 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.060+0000 7f1415284640 1 -- 172.21.15.88:0/8720802 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc00c0c0 con 0x7f13fc003b10 2023-10-16T20:06:04.073 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.060+0000 7f1415284640 1 -- 172.21.15.88:0/8720802 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc00c280 con 0x7f13fc0047d0 2023-10-16T20:06:04.073 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.060+0000 7f1415a85640 1 --2- 172.21.15.88:0/8720802 >> v2:172.21.15.94:3300/0 conn(0x7f141015d9f0 0x7f1410161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:06:04.074 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.060+0000 7f1416286640 1 --2- 172.21.15.88:0/8720802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13fc003b10 0x7f13fc003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:04.074 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.060+0000 7f1416a87640 1 --2- 172.21.15.88:0/8720802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13fc005190 0x7f13fc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:04.074 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.064+0000 7f1415a85640 1 --2- 172.21.15.88:0/8720802 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f13fc0047d0 0x7f13fc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:04.074 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.064+0000 7f1416a87640 1 -- 172.21.15.88:0/8720802 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f13fc0047d0 msgr2=0x7f13fc004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:04.075 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.064+0000 7f1416a87640 1 --2- 172.21.15.88:0/8720802 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f13fc0047d0 0x7f13fc004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:04.075 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.064+0000 7f1416a87640 1 -- 172.21.15.88:0/8720802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13fc003b10 msgr2=0x7f13fc003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:04.075 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.064+0000 7f1416a87640 1 --2- 172.21.15.88:0/8720802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13fc003b10 0x7f13fc003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:04.075 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.064+0000 7f1416a87640 1 -- 172.21.15.88:0/8720802 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13fc003530 con 0x7f13fc005190 2023-10-16T20:06:04.076 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.064+0000 7f1416286640 1 --2- 172.21.15.88:0/8720802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13fc003b10 0x7f13fc003f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:06:04.079 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.064+0000 7f1416a87640 1 -- 172.21.15.88:0/8720802 reap_dead start 2023-10-16T20:06:04.079 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.064+0000 7f1416a87640 1 --2- 172.21.15.88:0/8720802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13fc005190 0x7f13fc00b990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1408001fd0 tx=0x7f140801f050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f0147a6fe0b8981 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:04.080 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.064+0000 7f1415284640 1 -- 172.21.15.88:0/8720802 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14080a8c70 con 0x7f13fc005190 2023-10-16T20:06:04.080 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.064+0000 7f1415284640 1 -- 172.21.15.88:0/8720802 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14080a8dd0 con 0x7f13fc005190 2023-10-16T20:06:04.080 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.064+0000 7f1415284640 1 -- 172.21.15.88:0/8720802 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1408003bc0 con 0x7f13fc005190 2023-10-16T20:06:04.080 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.064+0000 7f14184ea640 1 -- 172.21.15.88:0/8720802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13fc005190 msgr2=0x7f13fc00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:04.080 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.064+0000 7f14184ea640 1 --2- 172.21.15.88:0/8720802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13fc005190 0x7f13fc00b990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1408001fd0 tx=0x7f140801f050 comp rx=0 tx=0).stop 2023-10-16T20:06:04.081 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.068+0000 7f14184ea640 1 -- 172.21.15.88:0/8720802 shutdown_connections 2023-10-16T20:06:04.081 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.068+0000 7f14184ea640 1 --2- 172.21.15.88:0/8720802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13fc005190 0x7f13fc00b990 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:04.081 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.068+0000 7f14184ea640 1 -- 172.21.15.88:0/8720802 >> 172.21.15.88:0/8720802 conn(0x7f14100715c0 msgr2=0x7f14100719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:04.081 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.068+0000 7f14184ea640 1 -- 172.21.15.88:0/8720802 shutdown_connections 2023-10-16T20:06:04.082 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.068+0000 7f14184ea640 1 -- 172.21.15.88:0/8720802 wait complete. 2023-10-16T20:06:04.082 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.068+0000 7f14184ea640 1 Processor -- start 2023-10-16T20:06:04.082 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.068+0000 7f14184ea640 1 -- start start 2023-10-16T20:06:04.082 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.068+0000 7f14184ea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13fc005190 0x7f14100b71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:04.082 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.068+0000 7f14184ea640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14100b5c90 0x7f14100b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:04.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.068+0000 7f14184ea640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14100b6650 0x7f14100c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:04.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.068+0000 7f14184ea640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14100ccf10 con 0x7f14100b5c90 2023-10-16T20:06:04.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.068+0000 7f14184ea640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14100b6ad0 con 0x7f13fc005190 2023-10-16T20:06:04.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.068+0000 7f14184ea640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14100ccc30 con 0x7f14100b6650 2023-10-16T20:06:04.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.068+0000 7f1415a85640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14100b5c90 0x7f14100b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:04.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.068+0000 7f1415a85640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14100b5c90 0x7f14100b6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39988/0 (socket says 172.21.15.88:39988) 2023-10-16T20:06:04.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.068+0000 7f1415a85640 1 -- 172.21.15.88:0/124834756 learned_addr learned my addr 172.21.15.88:0/124834756 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:04.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.068+0000 7f1416286640 1 --2- 172.21.15.88:0/124834756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13fc005190 0x7f14100b71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:04.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.068+0000 7f1416a87640 1 --2- 172.21.15.88:0/124834756 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14100b6650 0x7f14100c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:04.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.068+0000 7f1415a85640 1 -- 172.21.15.88:0/124834756 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14100b6650 msgr2=0x7f14100c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:04.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.068+0000 7f1415a85640 1 --2- 172.21.15.88:0/124834756 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14100b6650 0x7f14100c7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:04.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.068+0000 7f1415a85640 1 -- 172.21.15.88:0/124834756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13fc005190 msgr2=0x7f14100b71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:04.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.068+0000 7f1415a85640 1 --2- 172.21.15.88:0/124834756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13fc005190 0x7f14100b71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:04.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.068+0000 7f1415a85640 1 -- 172.21.15.88:0/124834756 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14100c6840 con 0x7f14100b5c90 2023-10-16T20:06:04.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.072+0000 7f1415a85640 1 --2- 172.21.15.88:0/124834756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14100b5c90 0x7f14100b6110 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1400011a50 tx=0x7f1400011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:04.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.072+0000 7f1406ffd640 1 -- 172.21.15.88:0/124834756 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140000fc50 con 0x7f14100b5c90 2023-10-16T20:06:04.086 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.072+0000 7f1406ffd640 1 -- 172.21.15.88:0/124834756 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140000fdb0 con 0x7f14100b5c90 2023-10-16T20:06:04.086 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.072+0000 7f1406ffd640 1 -- 172.21.15.88:0/124834756 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1400004270 con 0x7f14100b5c90 2023-10-16T20:06:04.086 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.072+0000 7f14184ea640 1 -- 172.21.15.88:0/124834756 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14100c6b20 con 0x7f14100b5c90 2023-10-16T20:06:04.086 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.072+0000 7f14184ea640 1 -- 172.21.15.88:0/124834756 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14100b7700 con 0x7f14100b5c90 2023-10-16T20:06:04.086 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.072+0000 7f14184ea640 1 -- 172.21.15.88:0/124834756 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f141004e9c0 con 0x7f14100b5c90 2023-10-16T20:06:04.087 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.072+0000 7f1406ffd640 1 -- 172.21.15.88:0/124834756 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f14000043d0 con 0x7f14100b5c90 2023-10-16T20:06:04.087 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.072+0000 7f1406ffd640 1 --2- 172.21.15.88:0/124834756 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f13f40664a0 0x7f13f4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:04.087 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.072+0000 7f1406ffd640 1 -- 172.21.15.88:0/124834756 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f14000848e0 con 0x7f14100b5c90 2023-10-16T20:06:04.087 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.072+0000 7f1416286640 1 --2- 172.21.15.88:0/124834756 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f13f40664a0 0x7f13f4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:04.087 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.076+0000 7f1416286640 1 --2- 172.21.15.88:0/124834756 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f13f40664a0 0x7f13f4068960 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f140c001210 tx=0x7f140c001e30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:04.088 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.076+0000 7f1406ffd640 1 -- 172.21.15.88:0/124834756 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1400051a40 con 0x7f14100b5c90 2023-10-16T20:06:04.232 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.228+0000 7f14184ea640 1 -- 172.21.15.88:0/124834756 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14100b8290 con 0x7f14100b5c90 2023-10-16T20:06:04.233 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.228+0000 7f1406ffd640 1 -- 172.21.15.88:0/124834756 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18661 (secure 0 0 0) 0x7f1400022090 con 0x7f14100b5c90 2023-10-16T20:06:04.233 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:06:04.233 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":29,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:05:38.202959+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T20:04:54.442487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T20:04:55.766641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T20:04:59.475830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":1}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:06:04.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.232+0000 7f14184ea640 1 -- 172.21.15.88:0/124834756 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f13f40664a0 msgr2=0x7f13f4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:04.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.232+0000 7f14184ea640 1 --2- 172.21.15.88:0/124834756 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f13f40664a0 0x7f13f4068960 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f140c001210 tx=0x7f140c001e30 comp rx=0 tx=0).stop 2023-10-16T20:06:04.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.232+0000 7f14184ea640 1 -- 172.21.15.88:0/124834756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14100b5c90 msgr2=0x7f14100b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:04.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.232+0000 7f14184ea640 1 --2- 172.21.15.88:0/124834756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14100b5c90 0x7f14100b6110 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1400011a50 tx=0x7f1400011f10 comp rx=0 tx=0).stop 2023-10-16T20:06:04.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.232+0000 7f14184ea640 1 -- 172.21.15.88:0/124834756 shutdown_connections 2023-10-16T20:06:04.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.232+0000 7f14184ea640 1 --2- 172.21.15.88:0/124834756 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f13f40664a0 0x7f13f4068960 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:04.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.232+0000 7f14184ea640 1 --2- 172.21.15.88:0/124834756 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f14100b6650 0x7f14100c7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:04.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.232+0000 7f14184ea640 1 --2- 172.21.15.88:0/124834756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14100b5c90 0x7f14100b6110 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:04.237 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.232+0000 7f14184ea640 1 --2- 172.21.15.88:0/124834756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13fc005190 0x7f14100b71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:04.237 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.232+0000 7f14184ea640 1 -- 172.21.15.88:0/124834756 >> 172.21.15.88:0/124834756 conn(0x7f14100715c0 msgr2=0x7f141015fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:04.237 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.232+0000 7f14184ea640 1 -- 172.21.15.88:0/124834756 shutdown_connections 2023-10-16T20:06:04.237 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.232+0000 7f14184ea640 1 -- 172.21.15.88:0/124834756 wait complete. 2023-10-16T20:06:04.254 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T20:06:04.255 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T20:06:04.394 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.388+0000 7f2c83fd9640 1 Processor -- start 2023-10-16T20:06:04.394 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.388+0000 7f2c83fd9640 1 -- start start 2023-10-16T20:06:04.396 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.392+0000 7f2c83fd9640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2c7c0cebd0 0x7f2c7c167ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:04.396 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.392+0000 7f2c83fd9640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c7c06b490 con 0x7f2c7c0b2a80 2023-10-16T20:06:04.396 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.392+0000 7f2c83fd9640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c7c0cf180 con 0x7f2c7c15cb80 2023-10-16T20:06:04.396 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.392+0000 7f2c83fd9640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c7c06b700 con 0x7f2c7c0cebd0 2023-10-16T20:06:04.396 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.392+0000 7f2c81574640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f2c7c0b2a80 0x7f2c7c0ce4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:40562/0 (socket says 172.21.15.88:40562) 2023-10-16T20:06:04.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.392+0000 7f2c81574640 1 -- 172.21.15.88:0/4174551797 learned_addr learned my addr 172.21.15.88:0/4174551797 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:04.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.392+0000 7f2c82576640 1 --2- 172.21.15.88:0/4174551797 >> v2:172.21.15.94:3300/0 conn(0x7f2c7c0cebd0 0x7f2c7c167ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:04.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.392+0000 7f2c80d73640 1 -- 172.21.15.88:0/4174551797 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 691077137 0 0) 0x7f2c7c06b490 con 0x7f2c7c0b2a80 2023-10-16T20:06:04.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.392+0000 7f2c80d73640 1 -- 172.21.15.88:0/4174551797 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c68003530 con 0x7f2c7c0b2a80 2023-10-16T20:06:04.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.392+0000 7f2c80d73640 1 -- 172.21.15.88:0/4174551797 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 858730385 0 0) 0x7f2c7c0cf180 con 0x7f2c7c15cb80 2023-10-16T20:06:04.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.392+0000 7f2c80d73640 1 -- 172.21.15.88:0/4174551797 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c7c06b490 con 0x7f2c7c15cb80 2023-10-16T20:06:04.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.392+0000 7f2c82576640 1 -- 172.21.15.88:0/4174551797 >> v1:172.21.15.94:6789/0 conn(0x7f2c7c15cb80 legacy=0x7f2c7c0b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:04.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.392+0000 7f2c82576640 1 -- 172.21.15.88:0/4174551797 >> v1:172.21.15.88:6789/0 conn(0x7f2c7c0b2a80 legacy=0x7f2c7c0ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:04.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.392+0000 7f2c82576640 1 -- 172.21.15.88:0/4174551797 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c7c06b9e0 con 0x7f2c7c0cebd0 2023-10-16T20:06:04.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.396+0000 7f2c82576640 1 --2- 172.21.15.88:0/4174551797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c7c0cebd0 0x7f2c7c167ab0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009d90 tx=0x7f2c7401c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b40e86d17b395c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:04.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.396+0000 7f2c80d73640 1 -- 172.21.15.88:0/4174551797 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c740a58b0 con 0x7f2c7c0cebd0 2023-10-16T20:06:04.401 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.396+0000 7f2c80d73640 1 -- 172.21.15.88:0/4174551797 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c740a5a10 con 0x7f2c7c0cebd0 2023-10-16T20:06:04.401 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.396+0000 7f2c80d73640 1 -- 172.21.15.88:0/4174551797 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c740a5da0 con 0x7f2c7c0cebd0 2023-10-16T20:06:04.401 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.396+0000 7f2c83fd9640 1 -- 172.21.15.88:0/4174551797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c7c0cebd0 msgr2=0x7f2c7c167ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:04.401 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.396+0000 7f2c83fd9640 1 --2- 172.21.15.88:0/4174551797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c7c0cebd0 0x7f2c7c167ab0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009d90 tx=0x7f2c7401c8c0 comp rx=0 tx=0).stop 2023-10-16T20:06:04.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.396+0000 7f2c83fd9640 1 -- 172.21.15.88:0/4174551797 shutdown_connections 2023-10-16T20:06:04.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.396+0000 7f2c83fd9640 1 --2- 172.21.15.88:0/4174551797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c7c0cebd0 0x7f2c7c167ab0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:04.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.396+0000 7f2c83fd9640 1 -- 172.21.15.88:0/4174551797 >> 172.21.15.88:0/4174551797 conn(0x7f2c7c0cd370 msgr2=0x7f2c7c0cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:04.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.396+0000 7f2c83fd9640 1 -- 172.21.15.88:0/4174551797 shutdown_connections 2023-10-16T20:06:04.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.396+0000 7f2c83fd9640 1 -- 172.21.15.88:0/4174551797 wait complete. 2023-10-16T20:06:04.405 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.396+0000 7f2c83fd9640 1 Processor -- start 2023-10-16T20:06:04.405 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.396+0000 7f2c83fd9640 1 -- start start 2023-10-16T20:06:04.405 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.400+0000 7f2c83fd9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c7c0b2a80 0x7f2c7c15a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:04.406 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.400+0000 7f2c83fd9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2c7c0cebd0 0x7f2c7c15f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:04.406 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.400+0000 7f2c83fd9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c7c15cb80 0x7f2c7c0c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:04.406 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.400+0000 7f2c83fd9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c15f550 con 0x7f2c7c0b2a80 2023-10-16T20:06:04.406 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.400+0000 7f2c83fd9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c0ca3a0 con 0x7f2c7c15cb80 2023-10-16T20:06:04.406 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.400+0000 7f2c83fd9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c15b000 con 0x7f2c7c0cebd0 2023-10-16T20:06:04.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.400+0000 7f2c81d75640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c7c0b2a80 0x7f2c7c15a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:04.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.400+0000 7f2c81d75640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c7c0b2a80 0x7f2c7c15a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39994/0 (socket says 172.21.15.88:39994) 2023-10-16T20:06:04.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.400+0000 7f2c81d75640 1 -- 172.21.15.88:0/1175097407 learned_addr learned my addr 172.21.15.88:0/1175097407 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:04.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.400+0000 7f2c82576640 1 --2- 172.21.15.88:0/1175097407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c7c15cb80 0x7f2c7c0c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:04.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.400+0000 7f2c81574640 1 --2- 172.21.15.88:0/1175097407 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2c7c0cebd0 0x7f2c7c15f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:04.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.400+0000 7f2c81d75640 1 -- 172.21.15.88:0/1175097407 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2c7c0cebd0 msgr2=0x7f2c7c15f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:04.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.400+0000 7f2c81d75640 1 --2- 172.21.15.88:0/1175097407 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2c7c0cebd0 0x7f2c7c15f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:04.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.400+0000 7f2c81d75640 1 -- 172.21.15.88:0/1175097407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c7c15cb80 msgr2=0x7f2c7c0c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:04.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.400+0000 7f2c81d75640 1 --2- 172.21.15.88:0/1175097407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c7c15cb80 0x7f2c7c0c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:04.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.400+0000 7f2c81d75640 1 -- 172.21.15.88:0/1175097407 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c74009a70 con 0x7f2c7c0b2a80 2023-10-16T20:06:04.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.400+0000 7f2c81d75640 1 --2- 172.21.15.88:0/1175097407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c7c0b2a80 0x7f2c7c15a940 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800c430 tx=0x7f2c7800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:04.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.400+0000 7f2c727fc640 1 -- 172.21.15.88:0/1175097407 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c780052d0 con 0x7f2c7c0b2a80 2023-10-16T20:06:04.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.400+0000 7f2c727fc640 1 -- 172.21.15.88:0/1175097407 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c78005450 con 0x7f2c7c0b2a80 2023-10-16T20:06:04.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.400+0000 7f2c83fd9640 1 -- 172.21.15.88:0/1175097407 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c7c15b2e0 con 0x7f2c7c0b2a80 2023-10-16T20:06:04.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.400+0000 7f2c727fc640 1 -- 172.21.15.88:0/1175097407 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c78003840 con 0x7f2c7c0b2a80 2023-10-16T20:06:04.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.400+0000 7f2c83fd9640 1 -- 172.21.15.88:0/1175097407 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c7c15b8a0 con 0x7f2c7c0b2a80 2023-10-16T20:06:04.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.400+0000 7f2c83fd9640 1 -- 172.21.15.88:0/1175097407 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c7c04e9c0 con 0x7f2c7c0b2a80 2023-10-16T20:06:04.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.400+0000 7f2c727fc640 1 -- 172.21.15.88:0/1175097407 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2c780039a0 con 0x7f2c7c0b2a80 2023-10-16T20:06:04.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.404+0000 7f2c727fc640 1 --2- 172.21.15.88:0/1175097407 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2c600664a0 0x7f2c60068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:04.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.404+0000 7f2c727fc640 1 -- 172.21.15.88:0/1175097407 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2c78083850 con 0x7f2c7c0b2a80 2023-10-16T20:06:04.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.404+0000 7f2c81574640 1 --2- 172.21.15.88:0/1175097407 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2c600664a0 0x7f2c60068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:04.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.404+0000 7f2c81574640 1 --2- 172.21.15.88:0/1175097407 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2c600664a0 0x7f2c60068960 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c15f930 tx=0x7f2c6c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:04.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.404+0000 7f2c727fc640 1 -- 172.21.15.88:0/1175097407 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2c78050920 con 0x7f2c7c0b2a80 2023-10-16T20:06:04.591 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.588+0000 7f2c83fd9640 1 -- 172.21.15.88:0/1175097407 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c7c0b6870 con 0x7f2c7c0b2a80 2023-10-16T20:06:04.592 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.588+0000 7f2c727fc640 1 -- 172.21.15.88:0/1175097407 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4906 (secure 0 0 0) 0x7f2c78053f40 con 0x7f2c7c0b2a80 2023-10-16T20:06:04.593 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:06:04.593 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3803627113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3803627113},{"type":"v1","addr":"172.21.15.94:6834","nonce":3803627113}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3758093239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":3758093239},{"type":"v1","addr":"172.21.15.88:6837","nonce":3758093239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/3836772385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3836772385},{"type":"v1","addr":"172.21.15.88:6836","nonce":3836772385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T20:04:56.759719+0000","modified":"2023-10-16T20:05:14.569618+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6835/555575750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":555575750},{"type":"v1","addr":"172.21.15.94:6835","nonce":555575750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T20:06:04.594 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 9 2023-10-16T20:06:04.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.588+0000 7f2c83fd9640 1 -- 172.21.15.88:0/1175097407 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2c600664a0 msgr2=0x7f2c60068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:04.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.588+0000 7f2c83fd9640 1 --2- 172.21.15.88:0/1175097407 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2c600664a0 0x7f2c60068960 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c15f930 tx=0x7f2c6c009040 comp rx=0 tx=0).stop 2023-10-16T20:06:04.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.588+0000 7f2c83fd9640 1 -- 172.21.15.88:0/1175097407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c7c0b2a80 msgr2=0x7f2c7c15a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:04.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.588+0000 7f2c83fd9640 1 --2- 172.21.15.88:0/1175097407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c7c0b2a80 0x7f2c7c15a940 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800c430 tx=0x7f2c7800c8f0 comp rx=0 tx=0).stop 2023-10-16T20:06:04.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.588+0000 7f2c83fd9640 1 -- 172.21.15.88:0/1175097407 shutdown_connections 2023-10-16T20:06:04.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.588+0000 7f2c83fd9640 1 --2- 172.21.15.88:0/1175097407 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2c600664a0 0x7f2c60068960 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:04.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.588+0000 7f2c83fd9640 1 --2- 172.21.15.88:0/1175097407 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2c7c0cebd0 0x7f2c7c15f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:04.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.588+0000 7f2c83fd9640 1 --2- 172.21.15.88:0/1175097407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c7c15cb80 0x7f2c7c0c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:04.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.588+0000 7f2c83fd9640 1 --2- 172.21.15.88:0/1175097407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c7c0b2a80 0x7f2c7c15a940 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:04.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.588+0000 7f2c83fd9640 1 -- 172.21.15.88:0/1175097407 >> 172.21.15.88:0/1175097407 conn(0x7f2c7c0cd370 msgr2=0x7f2c7c0ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:04.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.588+0000 7f2c83fd9640 1 -- 172.21.15.88:0/1175097407 shutdown_connections 2023-10-16T20:06:04.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.588+0000 7f2c83fd9640 1 -- 172.21.15.88:0/1175097407 wait complete. 2023-10-16T20:06:04.613 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T20:06:04.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.748+0000 7ff1b7805640 1 Processor -- start 2023-10-16T20:06:04.752 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.748+0000 7ff1b7805640 1 -- start start 2023-10-16T20:06:04.753 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.748+0000 7ff1b7805640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff1b006a5a0 0x7ff1b00cc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:04.754 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.748+0000 7ff1b7805640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1b013fb10 0x7ff1b015d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:04.754 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.748+0000 7ff1b7805640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1b00abaa0 con 0x7ff1b013f730 2023-10-16T20:06:04.754 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.748+0000 7ff1b7805640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1b00ab890 con 0x7ff1b006a5a0 2023-10-16T20:06:04.754 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.748+0000 7ff1b7805640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b00abd80 con 0x7ff1b013fb10 2023-10-16T20:06:04.754 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.748+0000 7ff1b4da0640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff1b006a5a0 0x7ff1b00cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:04.755 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.748+0000 7ff1b4da0640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff1b006a5a0 0x7ff1b00cc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40002/0 (socket says 172.21.15.88:40002) 2023-10-16T20:06:04.755 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.748+0000 7ff1b4da0640 1 -- 172.21.15.88:0/2881635643 learned_addr learned my addr 172.21.15.88:0/2881635643 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:04.755 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.748+0000 7ff1b5da2640 1 --2- 172.21.15.88:0/2881635643 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1b013fb10 0x7ff1b015d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:04.755 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.748+0000 7ff1b4da0640 1 -- 172.21.15.88:0/2881635643 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1b013fb10 msgr2=0x7ff1b015d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:04.755 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.748+0000 7ff1b4da0640 1 --2- 172.21.15.88:0/2881635643 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1b013fb10 0x7ff1b015d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:04.755 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.748+0000 7ff1b4da0640 1 -- 172.21.15.88:0/2881635643 >> v1:172.21.15.94:6789/0 conn(0x7ff1b013f730 legacy=0x7ff1b0069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:04.756 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.748+0000 7ff1b4da0640 1 -- 172.21.15.88:0/2881635643 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b00ac060 con 0x7ff1b006a5a0 2023-10-16T20:06:04.757 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.752+0000 7ff1b4da0640 1 --2- 172.21.15.88:0/2881635643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1b006a5a0 0x7ff1b00cc120 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0009870 tx=0x7ff1a001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c933d681b079f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:04.757 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.752+0000 7ff1a7fff640 1 -- 172.21.15.88:0/2881635643 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a00a95f0 con 0x7ff1b006a5a0 2023-10-16T20:06:04.757 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.752+0000 7ff1a7fff640 1 -- 172.21.15.88:0/2881635643 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a00a9750 con 0x7ff1b006a5a0 2023-10-16T20:06:04.757 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.752+0000 7ff1a7fff640 1 -- 172.21.15.88:0/2881635643 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a00a9ae0 con 0x7ff1b006a5a0 2023-10-16T20:06:04.758 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.752+0000 7ff1b7805640 1 -- 172.21.15.88:0/2881635643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1b006a5a0 msgr2=0x7ff1b00cc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:04.758 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.752+0000 7ff1b7805640 1 --2- 172.21.15.88:0/2881635643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1b006a5a0 0x7ff1b00cc120 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0009870 tx=0x7ff1a001c510 comp rx=0 tx=0).stop 2023-10-16T20:06:04.758 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.752+0000 7ff1b7805640 1 -- 172.21.15.88:0/2881635643 shutdown_connections 2023-10-16T20:06:04.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.752+0000 7ff1b7805640 1 --2- 172.21.15.88:0/2881635643 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1b013fb10 0x7ff1b015d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:04.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.752+0000 7ff1b7805640 1 --2- 172.21.15.88:0/2881635643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1b006a5a0 0x7ff1b00cc120 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:04.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.752+0000 7ff1b7805640 1 -- 172.21.15.88:0/2881635643 >> 172.21.15.88:0/2881635643 conn(0x7ff1b00ce5e0 msgr2=0x7ff1b00ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:04.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.752+0000 7ff1b7805640 1 -- 172.21.15.88:0/2881635643 shutdown_connections 2023-10-16T20:06:04.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.752+0000 7ff1b7805640 1 -- 172.21.15.88:0/2881635643 wait complete. 2023-10-16T20:06:04.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.756+0000 7ff1b7805640 1 Processor -- start 2023-10-16T20:06:04.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.756+0000 7ff1b7805640 1 -- start start 2023-10-16T20:06:04.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.756+0000 7ff1b7805640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1b006a5a0 0x7ff1b0152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:04.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.756+0000 7ff1b7805640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1b013f730 0x7ff1b013d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:04.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.756+0000 7ff1b7805640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1b013fb10 0x7ff1b0137b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:04.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.756+0000 7ff1b7805640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b0153370 con 0x7ff1b013fb10 2023-10-16T20:06:04.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.756+0000 7ff1b7805640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b00b1f80 con 0x7ff1b006a5a0 2023-10-16T20:06:04.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.756+0000 7ff1b7805640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b013e340 con 0x7ff1b013f730 2023-10-16T20:06:04.763 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.756+0000 7ff1b5da2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1b013fb10 0x7ff1b0137b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:04.763 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.756+0000 7ff1b5da2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1b013fb10 0x7ff1b0137b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40012/0 (socket says 172.21.15.88:40012) 2023-10-16T20:06:04.763 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.756+0000 7ff1b5da2640 1 -- 172.21.15.88:0/3077052372 learned_addr learned my addr 172.21.15.88:0/3077052372 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:04.763 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.756+0000 7ff1b55a1640 1 --2- 172.21.15.88:0/3077052372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1b006a5a0 0x7ff1b0152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:04.763 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.756+0000 7ff1b4da0640 1 --2- 172.21.15.88:0/3077052372 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1b013f730 0x7ff1b013d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:04.763 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.756+0000 7ff1b5da2640 1 -- 172.21.15.88:0/3077052372 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1b013f730 msgr2=0x7ff1b013d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:04.764 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.756+0000 7ff1b5da2640 1 --2- 172.21.15.88:0/3077052372 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1b013f730 0x7ff1b013d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:04.764 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.756+0000 7ff1b5da2640 1 -- 172.21.15.88:0/3077052372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1b006a5a0 msgr2=0x7ff1b0152e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:04.764 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.756+0000 7ff1b5da2640 1 --2- 172.21.15.88:0/3077052372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1b006a5a0 0x7ff1b0152e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:04.764 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.756+0000 7ff1b5da2640 1 -- 172.21.15.88:0/3077052372 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a00094d0 con 0x7ff1b013fb10 2023-10-16T20:06:04.764 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.756+0000 7ff1b5da2640 1 --2- 172.21.15.88:0/3077052372 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1b013fb10 0x7ff1b0137b00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8009dd0 tx=0x7ff1a800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:04.766 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.756+0000 7ff1a5ffb640 1 -- 172.21.15.88:0/3077052372 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8014480 con 0x7ff1b013fb10 2023-10-16T20:06:04.766 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.756+0000 7ff1a5ffb640 1 -- 172.21.15.88:0/3077052372 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a800f040 con 0x7ff1b013fb10 2023-10-16T20:06:04.766 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.756+0000 7ff1a5ffb640 1 -- 172.21.15.88:0/3077052372 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8014840 con 0x7ff1b013fb10 2023-10-16T20:06:04.766 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.756+0000 7ff1b7805640 1 -- 172.21.15.88:0/3077052372 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b00b21b0 con 0x7ff1b013fb10 2023-10-16T20:06:04.766 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.756+0000 7ff1b7805640 1 -- 172.21.15.88:0/3077052372 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b013dd80 con 0x7ff1b013fb10 2023-10-16T20:06:04.767 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.760+0000 7ff1b7805640 1 -- 172.21.15.88:0/3077052372 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1b004e9c0 con 0x7ff1b013fb10 2023-10-16T20:06:04.767 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.760+0000 7ff1a5ffb640 1 -- 172.21.15.88:0/3077052372 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff1a80149a0 con 0x7ff1b013fb10 2023-10-16T20:06:04.767 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.760+0000 7ff1a5ffb640 1 --2- 172.21.15.88:0/3077052372 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff1940664a0 0x7ff194068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:04.767 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.760+0000 7ff1a5ffb640 1 -- 172.21.15.88:0/3077052372 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff1a8084cd0 con 0x7ff1b013fb10 2023-10-16T20:06:04.767 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.760+0000 7ff1b55a1640 1 --2- 172.21.15.88:0/3077052372 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff1940664a0 0x7ff194068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:04.767 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.760+0000 7ff1b55a1640 1 --2- 172.21.15.88:0/3077052372 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff1940664a0 0x7ff194068960 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac007fd0 tx=0x7ff1ac009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:04.769 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.764+0000 7ff1a5ffb640 1 -- 172.21.15.88:0/3077052372 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff1a8051e30 con 0x7ff1b013fb10 2023-10-16T20:06:04.948 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.944+0000 7ff1b7805640 1 -- 172.21.15.88:0/3077052372 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1b013c970 con 0x7ff1b013fb10 2023-10-16T20:06:04.949 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.944+0000 7ff1a5ffb640 1 -- 172.21.15.88:0/3077052372 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4906 (secure 0 0 0) 0x7ff1a8018090 con 0x7ff1b013fb10 2023-10-16T20:06:04.949 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:06:04.949 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3803627113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3803627113},{"type":"v1","addr":"172.21.15.94:6834","nonce":3803627113}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3758093239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":3758093239},{"type":"v1","addr":"172.21.15.88:6837","nonce":3758093239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/3836772385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3836772385},{"type":"v1","addr":"172.21.15.88:6836","nonce":3836772385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T20:04:56.759719+0000","modified":"2023-10-16T20:05:14.569618+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6835/555575750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":555575750},{"type":"v1","addr":"172.21.15.94:6835","nonce":555575750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T20:06:04.949 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 9 2023-10-16T20:06:04.951 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.944+0000 7ff1b7805640 1 -- 172.21.15.88:0/3077052372 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff1940664a0 msgr2=0x7ff194068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:04.951 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.944+0000 7ff1b7805640 1 --2- 172.21.15.88:0/3077052372 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff1940664a0 0x7ff194068960 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac007fd0 tx=0x7ff1ac009040 comp rx=0 tx=0).stop 2023-10-16T20:06:04.951 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.944+0000 7ff1b7805640 1 -- 172.21.15.88:0/3077052372 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1b013fb10 msgr2=0x7ff1b0137b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:04.951 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.944+0000 7ff1b7805640 1 --2- 172.21.15.88:0/3077052372 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1b013fb10 0x7ff1b0137b00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8009dd0 tx=0x7ff1a800c490 comp rx=0 tx=0).stop 2023-10-16T20:06:04.951 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.948+0000 7ff1b7805640 1 -- 172.21.15.88:0/3077052372 shutdown_connections 2023-10-16T20:06:04.952 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.948+0000 7ff1b7805640 1 --2- 172.21.15.88:0/3077052372 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff1940664a0 0x7ff194068960 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:04.952 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.948+0000 7ff1b7805640 1 --2- 172.21.15.88:0/3077052372 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1b013f730 0x7ff1b013d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:04.952 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.948+0000 7ff1b7805640 1 --2- 172.21.15.88:0/3077052372 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1b013fb10 0x7ff1b0137b00 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:04.952 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.948+0000 7ff1b7805640 1 --2- 172.21.15.88:0/3077052372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1b006a5a0 0x7ff1b0152e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:04.952 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.948+0000 7ff1b7805640 1 -- 172.21.15.88:0/3077052372 >> 172.21.15.88:0/3077052372 conn(0x7ff1b00ce5e0 msgr2=0x7ff1b013c260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:04.953 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.948+0000 7ff1b7805640 1 -- 172.21.15.88:0/3077052372 shutdown_connections 2023-10-16T20:06:04.953 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:04.948+0000 7ff1b7805640 1 -- 172.21.15.88:0/3077052372 wait complete. 2023-10-16T20:06:04.970 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:06:05.108 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.104+0000 7fd0eee32640 1 Processor -- start 2023-10-16T20:06:05.108 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.104+0000 7fd0eee32640 1 -- start start 2023-10-16T20:06:05.109 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.104+0000 7fd0eee32640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd0e815d9f0 0x7fd0e8161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:05.109 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.104+0000 7fd0eee32640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0e80b87e0 con 0x7fd0e80d26c0 2023-10-16T20:06:05.109 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.104+0000 7fd0eee32640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0e815df10 con 0x7fd0e80d22e0 2023-10-16T20:06:05.109 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.104+0000 7fd0eee32640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0e815e1f0 con 0x7fd0e815d9f0 2023-10-16T20:06:05.110 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.104+0000 7fd0ed3cf640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fd0e80d26c0 0x7fd0e80cf2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:40564/0 (socket says 172.21.15.88:40564) 2023-10-16T20:06:05.110 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.104+0000 7fd0ed3cf640 1 -- 172.21.15.88:0/2353281869 learned_addr learned my addr 172.21.15.88:0/2353281869 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:05.110 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.104+0000 7fd0dffff640 1 --2- 172.21.15.88:0/2353281869 >> v2:172.21.15.94:3300/0 conn(0x7fd0e815d9f0 0x7fd0e8161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:05.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.104+0000 7fd0df7fe640 1 -- 172.21.15.88:0/2353281869 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3586650176 0 0) 0x7fd0e80b87e0 con 0x7fd0e80d26c0 2023-10-16T20:06:05.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.104+0000 7fd0df7fe640 1 -- 172.21.15.88:0/2353281869 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0d0003530 con 0x7fd0e80d26c0 2023-10-16T20:06:05.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.104+0000 7fd0df7fe640 1 -- 172.21.15.88:0/2353281869 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1661398528 0 0) 0x7fd0e815df10 con 0x7fd0e80d22e0 2023-10-16T20:06:05.112 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.104+0000 7fd0df7fe640 1 -- 172.21.15.88:0/2353281869 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0e80b87e0 con 0x7fd0e80d22e0 2023-10-16T20:06:05.112 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.104+0000 7fd0dffff640 1 -- 172.21.15.88:0/2353281869 >> v1:172.21.15.94:6789/0 conn(0x7fd0e80d22e0 legacy=0x7fd0e815d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:05.112 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.104+0000 7fd0dffff640 1 -- 172.21.15.88:0/2353281869 >> v1:172.21.15.88:6789/0 conn(0x7fd0e80d26c0 legacy=0x7fd0e80cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:05.112 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.104+0000 7fd0df7fe640 1 -- 172.21.15.88:0/2353281869 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2429472464 0 0) 0x7fd0d0003530 con 0x7fd0e80d26c0 2023-10-16T20:06:05.112 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.104+0000 7fd0dffff640 1 -- 172.21.15.88:0/2353281869 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e8160970 con 0x7fd0e815d9f0 2023-10-16T20:06:05.113 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.108+0000 7fd0dffff640 1 --2- 172.21.15.88:0/2353281869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0e815d9f0 0x7fd0e8161450 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd0d80097c0 tx=0x7fd0d801c2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd7fa3dad7c1e1ff server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:05.113 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.108+0000 7fd0df7fe640 1 -- 172.21.15.88:0/2353281869 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0001a00 con 0x7fd0e815d9f0 2023-10-16T20:06:05.113 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.108+0000 7fd0df7fe640 1 -- 172.21.15.88:0/2353281869 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d80a7050 con 0x7fd0e815d9f0 2023-10-16T20:06:05.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.108+0000 7fd0df7fe640 1 -- 172.21.15.88:0/2353281869 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d80ab400 con 0x7fd0e815d9f0 2023-10-16T20:06:05.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.108+0000 7fd0eee32640 1 -- 172.21.15.88:0/2353281869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0e815d9f0 msgr2=0x7fd0e8161450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:05.115 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.108+0000 7fd0eee32640 1 --2- 172.21.15.88:0/2353281869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0e815d9f0 0x7fd0e8161450 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd0d80097c0 tx=0x7fd0d801c2e0 comp rx=0 tx=0).stop 2023-10-16T20:06:05.115 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.108+0000 7fd0eee32640 1 -- 172.21.15.88:0/2353281869 shutdown_connections 2023-10-16T20:06:05.115 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.108+0000 7fd0eee32640 1 --2- 172.21.15.88:0/2353281869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0e815d9f0 0x7fd0e8161450 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:05.115 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.108+0000 7fd0eee32640 1 -- 172.21.15.88:0/2353281869 >> 172.21.15.88:0/2353281869 conn(0x7fd0e80715c0 msgr2=0x7fd0e80719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:05.115 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.108+0000 7fd0eee32640 1 -- 172.21.15.88:0/2353281869 shutdown_connections 2023-10-16T20:06:05.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.108+0000 7fd0eee32640 1 -- 172.21.15.88:0/2353281869 wait complete. 2023-10-16T20:06:05.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.112+0000 7fd0eee32640 1 Processor -- start 2023-10-16T20:06:05.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.112+0000 7fd0eee32640 1 -- start start 2023-10-16T20:06:05.117 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.112+0000 7fd0eee32640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0e80d22e0 0x7fd0e8149220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:05.117 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.112+0000 7fd0eee32640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0e80d26c0 0x7fd0e8152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:05.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.112+0000 7fd0eee32640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0e815d9f0 0x7fd0e8151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:05.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.112+0000 7fd0eee32640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e80c6e60 con 0x7fd0e815d9f0 2023-10-16T20:06:05.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.112+0000 7fd0eee32640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e8151f50 con 0x7fd0e80d22e0 2023-10-16T20:06:05.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.112+0000 7fd0eee32640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e80c64e0 con 0x7fd0e80d26c0 2023-10-16T20:06:05.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.112+0000 7fd0ed3cf640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0e815d9f0 0x7fd0e8151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:05.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.112+0000 7fd0ed3cf640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0e815d9f0 0x7fd0e8151a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40016/0 (socket says 172.21.15.88:40016) 2023-10-16T20:06:05.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.112+0000 7fd0ed3cf640 1 -- 172.21.15.88:0/2808571561 learned_addr learned my addr 172.21.15.88:0/2808571561 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:05.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.112+0000 7fd0dffff640 1 --2- 172.21.15.88:0/2808571561 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0e80d26c0 0x7fd0e8152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:05.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.112+0000 7fd0ecbce640 1 --2- 172.21.15.88:0/2808571561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0e80d22e0 0x7fd0e8149220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:05.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.112+0000 7fd0ed3cf640 1 -- 172.21.15.88:0/2808571561 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0e80d26c0 msgr2=0x7fd0e8152110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:05.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.112+0000 7fd0ed3cf640 1 --2- 172.21.15.88:0/2808571561 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0e80d26c0 0x7fd0e8152110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:05.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.112+0000 7fd0ed3cf640 1 -- 172.21.15.88:0/2808571561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0e80d22e0 msgr2=0x7fd0e8149220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:05.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.112+0000 7fd0ed3cf640 1 --2- 172.21.15.88:0/2808571561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0e80d22e0 0x7fd0e8149220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:05.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.112+0000 7fd0ed3cf640 1 -- 172.21.15.88:0/2808571561 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d8009420 con 0x7fd0e815d9f0 2023-10-16T20:06:05.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.112+0000 7fd0ed3cf640 1 --2- 172.21.15.88:0/2808571561 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0e815d9f0 0x7fd0e8151a10 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0001fb0 tx=0x7fd0e000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:05.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.112+0000 7fd0dd7fa640 1 -- 172.21.15.88:0/2808571561 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0012400 con 0x7fd0e815d9f0 2023-10-16T20:06:05.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.112+0000 7fd0dd7fa640 1 -- 172.21.15.88:0/2808571561 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0e0012560 con 0x7fd0e815d9f0 2023-10-16T20:06:05.122 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.112+0000 7fd0eee32640 1 -- 172.21.15.88:0/2808571561 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e81527f0 con 0x7fd0e815d9f0 2023-10-16T20:06:05.122 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.112+0000 7fd0dd7fa640 1 -- 172.21.15.88:0/2808571561 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0012910 con 0x7fd0e815d9f0 2023-10-16T20:06:05.122 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.112+0000 7fd0eee32640 1 -- 172.21.15.88:0/2808571561 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e80c6a80 con 0x7fd0e815d9f0 2023-10-16T20:06:05.122 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.116+0000 7fd0eee32640 1 -- 172.21.15.88:0/2808571561 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0e80c6fa0 con 0x7fd0e815d9f0 2023-10-16T20:06:05.122 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.116+0000 7fd0dd7fa640 1 -- 172.21.15.88:0/2808571561 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd0e0012a70 con 0x7fd0e815d9f0 2023-10-16T20:06:05.123 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.116+0000 7fd0dd7fa640 1 --2- 172.21.15.88:0/2808571561 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd0d40664d0 0x7fd0d4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:05.123 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.116+0000 7fd0dd7fa640 1 -- 172.21.15.88:0/2808571561 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd0e00848f0 con 0x7fd0e815d9f0 2023-10-16T20:06:05.123 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.116+0000 7fd0ecbce640 1 --2- 172.21.15.88:0/2808571561 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd0d40664d0 0x7fd0d4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:05.123 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.116+0000 7fd0ecbce640 1 --2- 172.21.15.88:0/2808571561 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd0d40664d0 0x7fd0d4068990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40011f0 tx=0x7fd0e4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:05.125 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.120+0000 7fd0dd7fa640 1 -- 172.21.15.88:0/2808571561 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd0e80c6fa0 con 0x7fd0e815d9f0 2023-10-16T20:06:05.277 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.272+0000 7fd0eee32640 1 -- 172.21.15.88:0/2808571561 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0e80c6fa0 con 0x7fd0e815d9f0 2023-10-16T20:06:05.278 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.272+0000 7fd0dd7fa640 1 -- 172.21.15.88:0/2808571561 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18661 (secure 0 0 0) 0x7fd0e80c6fa0 con 0x7fd0e815d9f0 2023-10-16T20:06:05.279 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:06:05.279 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":29,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:05:38.202959+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T20:04:54.442487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T20:04:55.766641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T20:04:59.475830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":1}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:06:05.280 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.276+0000 7fd0eee32640 1 -- 172.21.15.88:0/2808571561 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd0d40664d0 msgr2=0x7fd0d4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:05.280 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.276+0000 7fd0eee32640 1 --2- 172.21.15.88:0/2808571561 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd0d40664d0 0x7fd0d4068990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40011f0 tx=0x7fd0e4009040 comp rx=0 tx=0).stop 2023-10-16T20:06:05.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.276+0000 7fd0eee32640 1 -- 172.21.15.88:0/2808571561 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0e815d9f0 msgr2=0x7fd0e8151a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:05.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.276+0000 7fd0eee32640 1 --2- 172.21.15.88:0/2808571561 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0e815d9f0 0x7fd0e8151a10 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0001fb0 tx=0x7fd0e000a430 comp rx=0 tx=0).stop 2023-10-16T20:06:05.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.276+0000 7fd0eee32640 1 -- 172.21.15.88:0/2808571561 shutdown_connections 2023-10-16T20:06:05.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.276+0000 7fd0eee32640 1 --2- 172.21.15.88:0/2808571561 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd0d40664d0 0x7fd0d4068990 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:05.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.276+0000 7fd0eee32640 1 --2- 172.21.15.88:0/2808571561 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0e80d26c0 0x7fd0e8152110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:05.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.276+0000 7fd0eee32640 1 --2- 172.21.15.88:0/2808571561 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0e815d9f0 0x7fd0e8151a10 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:05.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.276+0000 7fd0eee32640 1 --2- 172.21.15.88:0/2808571561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0e80d22e0 0x7fd0e8149220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:05.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.276+0000 7fd0eee32640 1 -- 172.21.15.88:0/2808571561 >> 172.21.15.88:0/2808571561 conn(0x7fd0e80715c0 msgr2=0x7fd0e80719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:05.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.276+0000 7fd0eee32640 1 -- 172.21.15.88:0/2808571561 shutdown_connections 2023-10-16T20:06:05.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.276+0000 7fd0eee32640 1 -- 172.21.15.88:0/2808571561 wait complete. 2023-10-16T20:06:05.300 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T20:06:05.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.435+0000 7f17c10dc640 1 Processor -- start 2023-10-16T20:06:05.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.435+0000 7f17c10dc640 1 -- start start 2023-10-16T20:06:05.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.435+0000 7f17c10dc640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f17bc06d1c0 0x7f17bc0cc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:05.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.435+0000 7f17c10dc640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f17bc0c9b70 0x7f17bc0ca020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:05.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.435+0000 7f17c10dc640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17bc0c96e0 con 0x7f17bc15c3a0 2023-10-16T20:06:05.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.435+0000 7f17c10dc640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17bc0ce2c0 con 0x7f17bc0c9b70 2023-10-16T20:06:05.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.435+0000 7f17c10dc640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17bc0cda20 con 0x7f17bc06d1c0 2023-10-16T20:06:05.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.435+0000 7f17c08db640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f17bc0c9b70 0x7f17bc0ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:05.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.435+0000 7f17bbfff640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f17bc15c3a0 0x7f17bc0700a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:40570/0 (socket says 172.21.15.88:40570) 2023-10-16T20:06:05.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.435+0000 7f17c08db640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f17bc0c9b70 0x7f17bc0ca020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40022/0 (socket says 172.21.15.88:40022) 2023-10-16T20:06:05.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.435+0000 7f17bbfff640 1 -- 172.21.15.88:0/910584517 learned_addr learned my addr 172.21.15.88:0/910584517 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:05.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.435+0000 7f17bb7fe640 1 --2- 172.21.15.88:0/910584517 >> v2:172.21.15.94:3300/0 conn(0x7f17bc06d1c0 0x7f17bc0cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:05.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.435+0000 7f17c08db640 1 -- 172.21.15.88:0/910584517 >> v2:172.21.15.94:3300/0 conn(0x7f17bc06d1c0 msgr2=0x7f17bc0cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:05.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.435+0000 7f17c08db640 1 --2- 172.21.15.88:0/910584517 >> v2:172.21.15.94:3300/0 conn(0x7f17bc06d1c0 0x7f17bc0cc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:05.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.435+0000 7f17c08db640 1 -- 172.21.15.88:0/910584517 >> v1:172.21.15.88:6789/0 conn(0x7f17bc15c3a0 legacy=0x7f17bc0700a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:05.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.435+0000 7f17c08db640 1 -- 172.21.15.88:0/910584517 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17bc0cdd00 con 0x7f17bc0c9b70 2023-10-16T20:06:05.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.439+0000 7f17c08db640 1 --2- 172.21.15.88:0/910584517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17bc0c9b70 0x7f17bc0ca020 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f17bc0c54d0 tx=0x7f17b401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec35926eb16ea3de server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:05.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.439+0000 7f17baffd640 1 -- 172.21.15.88:0/910584517 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b40a66e0 con 0x7f17bc0c9b70 2023-10-16T20:06:05.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.439+0000 7f17baffd640 1 -- 172.21.15.88:0/910584517 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b40a6840 con 0x7f17bc0c9b70 2023-10-16T20:06:05.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.439+0000 7f17baffd640 1 -- 172.21.15.88:0/910584517 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b40a6bd0 con 0x7f17bc0c9b70 2023-10-16T20:06:05.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.439+0000 7f17c10dc640 1 -- 172.21.15.88:0/910584517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17bc0c9b70 msgr2=0x7f17bc0ca020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:05.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.439+0000 7f17c10dc640 1 --2- 172.21.15.88:0/910584517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17bc0c9b70 0x7f17bc0ca020 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f17bc0c54d0 tx=0x7f17b401c910 comp rx=0 tx=0).stop 2023-10-16T20:06:05.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.439+0000 7f17c10dc640 1 -- 172.21.15.88:0/910584517 shutdown_connections 2023-10-16T20:06:05.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.439+0000 7f17c10dc640 1 --2- 172.21.15.88:0/910584517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17bc0c9b70 0x7f17bc0ca020 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:05.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.439+0000 7f17c10dc640 1 --2- 172.21.15.88:0/910584517 >> v2:172.21.15.94:3300/0 conn(0x7f17bc06d1c0 0x7f17bc0cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:05.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.439+0000 7f17c10dc640 1 -- 172.21.15.88:0/910584517 >> 172.21.15.88:0/910584517 conn(0x7f17bc0ce850 msgr2=0x7f17bc0cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:05.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.439+0000 7f17c10dc640 1 -- 172.21.15.88:0/910584517 shutdown_connections 2023-10-16T20:06:05.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.439+0000 7f17c10dc640 1 -- 172.21.15.88:0/910584517 wait complete. 2023-10-16T20:06:05.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.443+0000 7f17c10dc640 1 Processor -- start 2023-10-16T20:06:05.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.443+0000 7f17c10dc640 1 -- start start 2023-10-16T20:06:05.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.443+0000 7f17c10dc640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f17bc06d1c0 0x7f17bc0c1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:05.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.443+0000 7f17c10dc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17bc0c9b70 0x7f17bc0ab6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:05.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.443+0000 7f17c10dc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17bc15c3a0 0x7f17bc0abc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:05.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.443+0000 7f17c10dc640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17bc1566b0 con 0x7f17bc15c3a0 2023-10-16T20:06:05.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.443+0000 7f17c10dc640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17bc0aecf0 con 0x7f17bc0c9b70 2023-10-16T20:06:05.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.443+0000 7f17c10dc640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17bc0aefd0 con 0x7f17bc06d1c0 2023-10-16T20:06:05.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.443+0000 7f17c08db640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17bc15c3a0 0x7f17bc0abc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:05.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.443+0000 7f17c08db640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17bc15c3a0 0x7f17bc0abc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40038/0 (socket says 172.21.15.88:40038) 2023-10-16T20:06:05.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.443+0000 7f17c08db640 1 -- 172.21.15.88:0/1712705172 learned_addr learned my addr 172.21.15.88:0/1712705172 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:05.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.443+0000 7f17bb7fe640 1 --2- 172.21.15.88:0/1712705172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17bc0c9b70 0x7f17bc0ab6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:05.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.443+0000 7f17bbfff640 1 --2- 172.21.15.88:0/1712705172 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f17bc06d1c0 0x7f17bc0c1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:05.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.443+0000 7f17c08db640 1 -- 172.21.15.88:0/1712705172 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f17bc06d1c0 msgr2=0x7f17bc0c1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:05.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.443+0000 7f17c08db640 1 --2- 172.21.15.88:0/1712705172 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f17bc06d1c0 0x7f17bc0c1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:05.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.443+0000 7f17c08db640 1 -- 172.21.15.88:0/1712705172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17bc0c9b70 msgr2=0x7f17bc0ab6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:05.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.443+0000 7f17c08db640 1 --2- 172.21.15.88:0/1712705172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17bc0c9b70 0x7f17bc0ab6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:05.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.443+0000 7f17c08db640 1 -- 172.21.15.88:0/1712705172 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b4009a70 con 0x7f17bc15c3a0 2023-10-16T20:06:05.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.443+0000 7f17c08db640 1 --2- 172.21.15.88:0/1712705172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17bc15c3a0 0x7f17bc0abc10 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f17b401a540 tx=0x7f17b40196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:05.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.443+0000 7f17b8ff9640 1 -- 172.21.15.88:0/1712705172 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b40b8630 con 0x7f17bc15c3a0 2023-10-16T20:06:05.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.443+0000 7f17b8ff9640 1 -- 172.21.15.88:0/1712705172 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b40b8790 con 0x7f17bc15c3a0 2023-10-16T20:06:05.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.443+0000 7f17b8ff9640 1 -- 172.21.15.88:0/1712705172 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b40b8b40 con 0x7f17bc15c3a0 2023-10-16T20:06:05.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.443+0000 7f17c10dc640 1 -- 172.21.15.88:0/1712705172 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17bc070930 con 0x7f17bc15c3a0 2023-10-16T20:06:05.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.443+0000 7f17c10dc640 1 -- 172.21.15.88:0/1712705172 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17bc154c40 con 0x7f17bc15c3a0 2023-10-16T20:06:05.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.447+0000 7f17c10dc640 1 -- 172.21.15.88:0/1712705172 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17bc0674f0 con 0x7f17bc15c3a0 2023-10-16T20:06:05.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.447+0000 7f17b8ff9640 1 -- 172.21.15.88:0/1712705172 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f17b40b8ca0 con 0x7f17bc15c3a0 2023-10-16T20:06:05.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.447+0000 7f17b8ff9640 1 --2- 172.21.15.88:0/1712705172 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f17a80664a0 0x7f17a8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:05.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.447+0000 7f17b8ff9640 1 -- 172.21.15.88:0/1712705172 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f17b40aa070 con 0x7f17bc15c3a0 2023-10-16T20:06:05.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.447+0000 7f17bbfff640 1 --2- 172.21.15.88:0/1712705172 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f17a80664a0 0x7f17a8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:05.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.447+0000 7f17bbfff640 1 --2- 172.21.15.88:0/1712705172 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f17a80664a0 0x7f17a8068960 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f17ac005890 tx=0x7f17ac009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:05.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.451+0000 7f17b8ff9640 1 -- 172.21.15.88:0/1712705172 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f17b40ec880 con 0x7f17bc15c3a0 2023-10-16T20:06:05.640 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.635+0000 7f17c10dc640 1 -- 172.21.15.88:0/1712705172 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f17bc0af480 con 0x7f17bc15c3a0 2023-10-16T20:06:05.640 INFO:tasks.ceph.mon.a.smithi088.stderr:2023-10-16T20:06:05.635+0000 7f41ee15b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T20:06:05.642 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.635+0000 7f17b8ff9640 1 -- 172.21.15.88:0/1712705172 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7f17b40efea0 con 0x7f17bc15c3a0 2023-10-16T20:06:05.642 INFO:teuthology.orchestra.run.smithi088.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T20:06:05.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.639+0000 7f17c10dc640 1 -- 172.21.15.88:0/1712705172 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f17a80664a0 msgr2=0x7f17a8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:05.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.639+0000 7f17c10dc640 1 --2- 172.21.15.88:0/1712705172 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f17a80664a0 0x7f17a8068960 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f17ac005890 tx=0x7f17ac009040 comp rx=0 tx=0).stop 2023-10-16T20:06:05.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.639+0000 7f17c10dc640 1 -- 172.21.15.88:0/1712705172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17bc15c3a0 msgr2=0x7f17bc0abc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:05.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.639+0000 7f17c10dc640 1 --2- 172.21.15.88:0/1712705172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17bc15c3a0 0x7f17bc0abc10 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f17b401a540 tx=0x7f17b40196b0 comp rx=0 tx=0).stop 2023-10-16T20:06:05.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.639+0000 7f17c10dc640 1 -- 172.21.15.88:0/1712705172 shutdown_connections 2023-10-16T20:06:05.646 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.639+0000 7f17c10dc640 1 --2- 172.21.15.88:0/1712705172 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f17a80664a0 0x7f17a8068960 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:05.646 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.639+0000 7f17c10dc640 1 --2- 172.21.15.88:0/1712705172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17bc15c3a0 0x7f17bc0abc10 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:05.646 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.639+0000 7f17c10dc640 1 --2- 172.21.15.88:0/1712705172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17bc0c9b70 0x7f17bc0ab6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:05.646 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.639+0000 7f17c10dc640 1 --2- 172.21.15.88:0/1712705172 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f17bc06d1c0 0x7f17bc0c1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:05.646 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.639+0000 7f17c10dc640 1 -- 172.21.15.88:0/1712705172 >> 172.21.15.88:0/1712705172 conn(0x7f17bc0ce850 msgr2=0x7f17bc0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:05.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.639+0000 7f17c10dc640 1 -- 172.21.15.88:0/1712705172 shutdown_connections 2023-10-16T20:06:05.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.639+0000 7f17c10dc640 1 -- 172.21.15.88:0/1712705172 wait complete. 2023-10-16T20:06:05.655 INFO:tasks.ceph.mds.b.smithi094.stderr: -4190> 2023-10-16T20:05:00.798+0000 7f1da1992640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T20:06:05.665 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T20:06:05.703 INFO:tasks.ceph.mds.b.smithi094.stdout:starting mds.b at 2023-10-16T20:06:05.809 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.803+0000 7f0817b72640 1 Processor -- start 2023-10-16T20:06:05.809 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.803+0000 7f0817b72640 1 -- start start 2023-10-16T20:06:05.810 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.803+0000 7f0817b72640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f081006d3f0 0x7f081006d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:05.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.803+0000 7f0817b72640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081015ad30 con 0x7f08100b4260 2023-10-16T20:06:05.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.803+0000 7f0817b72640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100b4640 con 0x7f08100b0da0 2023-10-16T20:06:05.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.803+0000 7f0817b72640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f081006dda0 con 0x7f081006d3f0 2023-10-16T20:06:05.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.803+0000 7f081590e640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f08100b4260 0x7f081015dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:40584/0 (socket says 172.21.15.88:40584) 2023-10-16T20:06:05.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.803+0000 7f081590e640 1 -- 172.21.15.88:0/643207509 learned_addr learned my addr 172.21.15.88:0/643207509 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:05.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.803+0000 7f081610f640 1 --2- 172.21.15.88:0/643207509 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f081006d3f0 0x7f081006d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:05.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.803+0000 7f081490c640 1 -- 172.21.15.88:0/643207509 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 801736334 0 0) 0x7f081015ad30 con 0x7f08100b4260 2023-10-16T20:06:05.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.807+0000 7f081490c640 1 -- 172.21.15.88:0/643207509 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07fc005740 con 0x7f08100b4260 2023-10-16T20:06:05.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.807+0000 7f081490c640 1 -- 172.21.15.88:0/643207509 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1928156518 0 0) 0x7f07fc005740 con 0x7f08100b4260 2023-10-16T20:06:05.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.807+0000 7f081490c640 1 -- 172.21.15.88:0/643207509 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f081006d3f0 msgr2=0x7f081006d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:05.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.807+0000 7f081490c640 1 --2- 172.21.15.88:0/643207509 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f081006d3f0 0x7f081006d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:05.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.807+0000 7f081490c640 1 -- 172.21.15.88:0/643207509 >> v1:172.21.15.94:6789/0 conn(0x7f08100b0da0 legacy=0x7f0810159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:05.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.807+0000 7f081490c640 1 -- 172.21.15.88:0/643207509 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081015a770 con 0x7f08100b4260 2023-10-16T20:06:05.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.807+0000 7f081490c640 1 -- 172.21.15.88:0/643207509 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f080c001680 con 0x7f08100b4260 2023-10-16T20:06:05.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.807+0000 7f081490c640 1 -- 172.21.15.88:0/643207509 >> v1:172.21.15.88:6789/0 conn(0x7f08100b4260 legacy=0x7f081015dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:05.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.807+0000 7f081490c640 1 --2- 172.21.15.88:0/643207509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07fc005bc0 0x7f07fc005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:05.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.807+0000 7f081490c640 1 --2- 172.21.15.88:0/643207509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07fc0069e0 0x7f07fc006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:05.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.807+0000 7f081490c640 1 --2- 172.21.15.88:0/643207509 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f081006d3f0 0x7f07fc0073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:05.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.807+0000 7f081490c640 1 -- 172.21.15.88:0/643207509 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081006dda0 con 0x7f07fc005bc0 2023-10-16T20:06:05.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.807+0000 7f081490c640 1 -- 172.21.15.88:0/643207509 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc00dcd0 con 0x7f07fc0069e0 2023-10-16T20:06:05.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.807+0000 7f081490c640 1 -- 172.21.15.88:0/643207509 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc00de90 con 0x7f081006d3f0 2023-10-16T20:06:05.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.807+0000 7f081590e640 1 --2- 172.21.15.88:0/643207509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07fc005bc0 0x7f07fc005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:05.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.807+0000 7f081510d640 1 --2- 172.21.15.88:0/643207509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07fc0069e0 0x7f07fc006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:05.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.807+0000 7f081610f640 1 --2- 172.21.15.88:0/643207509 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f081006d3f0 0x7f07fc0073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:05.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.807+0000 7f081590e640 1 -- 172.21.15.88:0/643207509 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f081006d3f0 msgr2=0x7f07fc0073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:05.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.807+0000 7f081590e640 1 --2- 172.21.15.88:0/643207509 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f081006d3f0 0x7f07fc0073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:05.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.807+0000 7f081590e640 1 -- 172.21.15.88:0/643207509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07fc0069e0 msgr2=0x7f07fc006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:05.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.807+0000 7f081590e640 1 --2- 172.21.15.88:0/643207509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07fc0069e0 0x7f07fc006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:05.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.807+0000 7f081590e640 1 -- 172.21.15.88:0/643207509 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07fc005740 con 0x7f07fc005bc0 2023-10-16T20:06:05.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.807+0000 7f081590e640 1 --2- 172.21.15.88:0/643207509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07fc005bc0 0x7f07fc005fe0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f080c00c240 tx=0x7f080c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6f93715ab682d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:05.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.811+0000 7f081490c640 1 -- 172.21.15.88:0/643207509 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c005ae0 con 0x7f07fc005bc0 2023-10-16T20:06:05.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.811+0000 7f081490c640 1 -- 172.21.15.88:0/643207509 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f080c005c40 con 0x7f07fc005bc0 2023-10-16T20:06:05.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.811+0000 7f081490c640 1 -- 172.21.15.88:0/643207509 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c007070 con 0x7f07fc005bc0 2023-10-16T20:06:05.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.811+0000 7f0817b72640 1 -- 172.21.15.88:0/643207509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07fc005bc0 msgr2=0x7f07fc005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:05.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.811+0000 7f0817b72640 1 --2- 172.21.15.88:0/643207509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07fc005bc0 0x7f07fc005fe0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f080c00c240 tx=0x7f080c01cc40 comp rx=0 tx=0).stop 2023-10-16T20:06:05.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.811+0000 7f081610f640 1 -- 172.21.15.88:0/643207509 reap_dead start 2023-10-16T20:06:05.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.811+0000 7f0817b72640 1 -- 172.21.15.88:0/643207509 shutdown_connections 2023-10-16T20:06:05.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.811+0000 7f0817b72640 1 -- 172.21.15.88:0/643207509 >> 172.21.15.88:0/643207509 conn(0x7f08100cb080 msgr2=0x7f08100cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:05.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.811+0000 7f0817b72640 1 -- 172.21.15.88:0/643207509 shutdown_connections 2023-10-16T20:06:05.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.811+0000 7f0817b72640 1 -- 172.21.15.88:0/643207509 wait complete. 2023-10-16T20:06:05.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.811+0000 7f0817b72640 1 Processor -- start 2023-10-16T20:06:05.822 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.811+0000 7f0817b72640 1 -- start start 2023-10-16T20:06:05.822 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.811+0000 7f0817b72640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07fc005bc0 0x7f0810157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:05.822 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.811+0000 7f0817b72640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f081015c8e0 0x7f0810148ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:05.822 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.811+0000 7f0817b72640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0810151560 0x7f0810151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:05.822 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.811+0000 7f0817b72640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0810151f50 con 0x7f0810151560 2023-10-16T20:06:05.823 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.811+0000 7f0817b72640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081015cd60 con 0x7f07fc005bc0 2023-10-16T20:06:05.823 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.811+0000 7f0817b72640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0810149020 con 0x7f081015c8e0 2023-10-16T20:06:05.823 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.811+0000 7f081610f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0810151560 0x7f0810151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:05.823 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.811+0000 7f081590e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07fc005bc0 0x7f0810157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:05.823 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.811+0000 7f081610f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0810151560 0x7f0810151a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40058/0 (socket says 172.21.15.88:40058) 2023-10-16T20:06:05.823 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.811+0000 7f081610f640 1 -- 172.21.15.88:0/592194043 learned_addr learned my addr 172.21.15.88:0/592194043 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:05.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.811+0000 7f081510d640 1 --2- 172.21.15.88:0/592194043 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f081015c8e0 0x7f0810148ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:05.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.815+0000 7f081610f640 1 -- 172.21.15.88:0/592194043 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f081015c8e0 msgr2=0x7f0810148ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:05.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.815+0000 7f081610f640 1 --2- 172.21.15.88:0/592194043 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f081015c8e0 0x7f0810148ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:05.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.815+0000 7f081610f640 1 -- 172.21.15.88:0/592194043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07fc005bc0 msgr2=0x7f0810157890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:05.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.815+0000 7f081610f640 1 --2- 172.21.15.88:0/592194043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07fc005bc0 0x7f0810157890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:05.825 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.815+0000 7f081610f640 1 -- 172.21.15.88:0/592194043 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081015a540 con 0x7f0810151560 2023-10-16T20:06:05.825 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.815+0000 7f081610f640 1 --2- 172.21.15.88:0/592194043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0810151560 0x7f0810151a10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f07fc006960 tx=0x7f080800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:05.825 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.815+0000 7f08067fc640 1 -- 172.21.15.88:0/592194043 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080800e040 con 0x7f0810151560 2023-10-16T20:06:05.825 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.815+0000 7f08067fc640 1 -- 172.21.15.88:0/592194043 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f080800f040 con 0x7f0810151560 2023-10-16T20:06:05.825 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.815+0000 7f0817b72640 1 -- 172.21.15.88:0/592194043 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f081015b8e0 con 0x7f0810151560 2023-10-16T20:06:05.825 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.815+0000 7f08067fc640 1 -- 172.21.15.88:0/592194043 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0808015670 con 0x7f0810151560 2023-10-16T20:06:05.826 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.815+0000 7f0817b72640 1 -- 172.21.15.88:0/592194043 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f081015bea0 con 0x7f0810151560 2023-10-16T20:06:05.827 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.815+0000 7f0817b72640 1 -- 172.21.15.88:0/592194043 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f081004e9c0 con 0x7f0810151560 2023-10-16T20:06:05.827 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.815+0000 7f08067fc640 1 -- 172.21.15.88:0/592194043 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f08080157d0 con 0x7f0810151560 2023-10-16T20:06:05.827 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.815+0000 7f08067fc640 1 --2- 172.21.15.88:0/592194043 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f07f40664d0 0x7f07f4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:05.827 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.815+0000 7f08067fc640 1 -- 172.21.15.88:0/592194043 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f08080847b0 con 0x7f0810151560 2023-10-16T20:06:05.827 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.815+0000 7f081590e640 1 --2- 172.21.15.88:0/592194043 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f07f40664d0 0x7f07f4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:05.828 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.819+0000 7f081590e640 1 --2- 172.21.15.88:0/592194043 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f07f40664d0 0x7f07f4068990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f080c0ab000 tx=0x7f080c0a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:05.828 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:05.819+0000 7f08067fc640 1 -- 172.21.15.88:0/592194043 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f08080518d0 con 0x7f0810151560 2023-10-16T20:06:06.019 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.015+0000 7f0817b72640 1 -- 172.21.15.88:0/592194043 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0810071500 con 0x7f0810151560 2023-10-16T20:06:06.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.639+0000 7f08067fc640 1 -- 172.21.15.88:0/592194043 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0808054ef0 con 0x7f0810151560 2023-10-16T20:06:06.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.643+0000 7f0817b72640 1 -- 172.21.15.88:0/592194043 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f07f40664d0 msgr2=0x7f07f4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:06.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.643+0000 7f0817b72640 1 --2- 172.21.15.88:0/592194043 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f07f40664d0 0x7f07f4068990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f080c0ab000 tx=0x7f080c0a7070 comp rx=0 tx=0).stop 2023-10-16T20:06:06.648 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.643+0000 7f0817b72640 1 -- 172.21.15.88:0/592194043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0810151560 msgr2=0x7f0810151a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:06.648 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.643+0000 7f0817b72640 1 --2- 172.21.15.88:0/592194043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0810151560 0x7f0810151a10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f07fc006960 tx=0x7f080800f580 comp rx=0 tx=0).stop 2023-10-16T20:06:06.648 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.643+0000 7f0817b72640 1 -- 172.21.15.88:0/592194043 shutdown_connections 2023-10-16T20:06:06.648 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.643+0000 7f0817b72640 1 --2- 172.21.15.88:0/592194043 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f07f40664d0 0x7f07f4068990 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:06.648 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.643+0000 7f0817b72640 1 --2- 172.21.15.88:0/592194043 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f081015c8e0 0x7f0810148ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:06.648 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.643+0000 7f0817b72640 1 --2- 172.21.15.88:0/592194043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0810151560 0x7f0810151a10 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:06.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.643+0000 7f0817b72640 1 --2- 172.21.15.88:0/592194043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07fc005bc0 0x7f0810157890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:06.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.643+0000 7f0817b72640 1 -- 172.21.15.88:0/592194043 >> 172.21.15.88:0/592194043 conn(0x7f08100cb080 msgr2=0x7f08100adc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:06.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.643+0000 7f0817b72640 1 -- 172.21.15.88:0/592194043 shutdown_connections 2023-10-16T20:06:06.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.643+0000 7f0817b72640 1 -- 172.21.15.88:0/592194043 wait complete. 2023-10-16T20:06:06.668 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T20:06:06.810 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.803+0000 7f6459c17640 1 Processor -- start 2023-10-16T20:06:06.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.803+0000 7f6459c17640 1 -- start start 2023-10-16T20:06:06.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.807+0000 7f6459c17640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f645415cb20 0x7f64540cca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:06.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.807+0000 7f6459c17640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f64540abd60 0x7f645406d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:06.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.807+0000 7f6459c17640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f645406e010 0x7f645406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:06.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.807+0000 7f6459c17640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f645406e9e0 con 0x7f645415cb20 2023-10-16T20:06:06.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.807+0000 7f6459c17640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f645406dc50 con 0x7f64540abd60 2023-10-16T20:06:06.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.807+0000 7f6459c17640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6454069f30 con 0x7f645406e010 2023-10-16T20:06:06.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.807+0000 7f64537fe640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f645415cb20 0x7f64540cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:06.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.807+0000 7f64537fe640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f645415cb20 0x7f64540cca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40062/0 (socket says 172.21.15.88:40062) 2023-10-16T20:06:06.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.807+0000 7f64537fe640 1 -- 172.21.15.88:0/471802388 learned_addr learned my addr 172.21.15.88:0/471802388 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:06.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.807+0000 7f6453fff640 1 --2- 172.21.15.88:0/471802388 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f645406e010 0x7f645406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:06.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.807+0000 7f6452ffd640 1 --2- 172.21.15.88:0/471802388 >> v2:172.21.15.94:3300/0 conn(0x7f64540abd60 0x7f645406d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:06.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.807+0000 7f64537fe640 1 -- 172.21.15.88:0/471802388 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f645406e010 msgr2=0x7f645406e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:06.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.807+0000 7f64537fe640 1 --2- 172.21.15.88:0/471802388 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f645406e010 0x7f645406e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:06.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.807+0000 7f64537fe640 1 -- 172.21.15.88:0/471802388 >> v2:172.21.15.94:3300/0 conn(0x7f64540abd60 msgr2=0x7f645406d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:06.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.807+0000 7f64537fe640 1 --2- 172.21.15.88:0/471802388 >> v2:172.21.15.94:3300/0 conn(0x7f64540abd60 0x7f645406d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:06.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.807+0000 7f64537fe640 1 -- 172.21.15.88:0/471802388 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64540cd160 con 0x7f645415cb20 2023-10-16T20:06:06.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.807+0000 7f6453fff640 1 --2- 172.21.15.88:0/471802388 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f645406e010 0x7f645406e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:06:06.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.807+0000 7f6452ffd640 1 --2- 172.21.15.88:0/471802388 >> v2:172.21.15.94:3300/0 conn(0x7f64540abd60 0x7f645406d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:06:06.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.811+0000 7f64537fe640 1 --2- 172.21.15.88:0/471802388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f645415cb20 0x7f64540cca60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f64540cee60 tx=0x7f644401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d991d4ff04297e5f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:06.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.811+0000 7f64527fc640 1 -- 172.21.15.88:0/471802388 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64440aa620 con 0x7f645415cb20 2023-10-16T20:06:06.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.811+0000 7f64527fc640 1 -- 172.21.15.88:0/471802388 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64440aa780 con 0x7f645415cb20 2023-10-16T20:06:06.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.811+0000 7f64527fc640 1 -- 172.21.15.88:0/471802388 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64440aab10 con 0x7f645415cb20 2023-10-16T20:06:06.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.811+0000 7f6459c17640 1 -- 172.21.15.88:0/471802388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f645415cb20 msgr2=0x7f64540cca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:06.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.811+0000 7f6459c17640 1 --2- 172.21.15.88:0/471802388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f645415cb20 0x7f64540cca60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f64540cee60 tx=0x7f644401c500 comp rx=0 tx=0).stop 2023-10-16T20:06:06.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.811+0000 7f6459c17640 1 -- 172.21.15.88:0/471802388 shutdown_connections 2023-10-16T20:06:06.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.811+0000 7f6459c17640 1 --2- 172.21.15.88:0/471802388 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f645406e010 0x7f645406e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:06.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.811+0000 7f6459c17640 1 --2- 172.21.15.88:0/471802388 >> v2:172.21.15.94:3300/0 conn(0x7f64540abd60 0x7f645406d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:06.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.811+0000 7f6459c17640 1 --2- 172.21.15.88:0/471802388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f645415cb20 0x7f64540cca60 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:06.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.811+0000 7f6459c17640 1 -- 172.21.15.88:0/471802388 >> 172.21.15.88:0/471802388 conn(0x7f64540b0980 msgr2=0x7f64540b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:06.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.811+0000 7f6459c17640 1 -- 172.21.15.88:0/471802388 shutdown_connections 2023-10-16T20:06:06.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.811+0000 7f6459c17640 1 -- 172.21.15.88:0/471802388 wait complete. 2023-10-16T20:06:06.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.811+0000 7f6459c17640 1 Processor -- start 2023-10-16T20:06:06.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.811+0000 7f6459c17640 1 -- start start 2023-10-16T20:06:06.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.815+0000 7f6459c17640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f645406e010 0x7f64540ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:06.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.815+0000 7f6459c17640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f64540abd60 0x7f6454152530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:06.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.815+0000 7f6459c17640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f645415cb20 0x7f6454157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:06.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.815+0000 7f6459c17640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64540aef40 con 0x7f645406e010 2023-10-16T20:06:06.822 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.815+0000 7f6459c17640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64540caf00 con 0x7f645415cb20 2023-10-16T20:06:06.822 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.815+0000 7f6459c17640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64540cb1e0 con 0x7f64540abd60 2023-10-16T20:06:06.822 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.815+0000 7f64537fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f645406e010 0x7f64540ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:06.822 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.815+0000 7f64537fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f645406e010 0x7f64540ca590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40068/0 (socket says 172.21.15.88:40068) 2023-10-16T20:06:06.822 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.815+0000 7f64537fe640 1 -- 172.21.15.88:0/3428840293 learned_addr learned my addr 172.21.15.88:0/3428840293 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:06.822 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.815+0000 7f6452ffd640 1 --2- 172.21.15.88:0/3428840293 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f64540abd60 0x7f6454152530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:06.823 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.815+0000 7f6453fff640 1 --2- 172.21.15.88:0/3428840293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f645415cb20 0x7f6454157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:06.823 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.815+0000 7f64537fe640 1 -- 172.21.15.88:0/3428840293 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f64540abd60 msgr2=0x7f6454152530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:06.823 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.815+0000 7f64537fe640 1 --2- 172.21.15.88:0/3428840293 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f64540abd60 0x7f6454152530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:06.823 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.815+0000 7f64537fe640 1 -- 172.21.15.88:0/3428840293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f645415cb20 msgr2=0x7f6454157920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:06.823 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.815+0000 7f64537fe640 1 --2- 172.21.15.88:0/3428840293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f645415cb20 0x7f6454157920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:06.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.815+0000 7f64537fe640 1 -- 172.21.15.88:0/3428840293 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64440094d0 con 0x7f645406e010 2023-10-16T20:06:06.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.815+0000 7f64537fe640 1 --2- 172.21.15.88:0/3428840293 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f645406e010 0x7f64540ca590 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f64440ac040 tx=0x7f644401ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:06.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.815+0000 7f643bfff640 1 -- 172.21.15.88:0/3428840293 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64440bd5f0 con 0x7f645406e010 2023-10-16T20:06:06.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.815+0000 7f643bfff640 1 -- 172.21.15.88:0/3428840293 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64440bd770 con 0x7f645406e010 2023-10-16T20:06:06.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.815+0000 7f643bfff640 1 -- 172.21.15.88:0/3428840293 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64440bdb20 con 0x7f645406e010 2023-10-16T20:06:06.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.815+0000 7f6459c17640 1 -- 172.21.15.88:0/3428840293 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64540ae8f0 con 0x7f645406e010 2023-10-16T20:06:06.825 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.815+0000 7f6459c17640 1 -- 172.21.15.88:0/3428840293 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64540caad0 con 0x7f645406e010 2023-10-16T20:06:06.826 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.815+0000 7f6459c17640 1 -- 172.21.15.88:0/3428840293 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64540cacc0 con 0x7f645406e010 2023-10-16T20:06:06.826 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.815+0000 7f643bfff640 1 -- 172.21.15.88:0/3428840293 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f64440b33f0 con 0x7f645406e010 2023-10-16T20:06:06.826 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.819+0000 7f643bfff640 1 --2- 172.21.15.88:0/3428840293 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f64400664a0 0x7f6440068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:06.826 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.819+0000 7f643bfff640 1 -- 172.21.15.88:0/3428840293 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f644411ed00 con 0x7f645406e010 2023-10-16T20:06:06.826 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.819+0000 7f6452ffd640 1 --2- 172.21.15.88:0/3428840293 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f64400664a0 0x7f6440068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:06.827 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.819+0000 7f6452ffd640 1 --2- 172.21.15.88:0/3428840293 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f64400664a0 0x7f6440068960 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f64541f8680 tx=0x7f6448008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:06.827 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.823+0000 7f643bfff640 1 -- 172.21.15.88:0/3428840293 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f64540cacc0 con 0x7f645406e010 2023-10-16T20:06:06.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:06.971+0000 7f6459c17640 1 -- 172.21.15.88:0/3428840293 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f64540cb320 con 0x7f645406e010 2023-10-16T20:06:07.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.643+0000 7f643bfff640 1 -- 172.21.15.88:0/3428840293 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v31) v1 ==== 181+0+0 (secure 0 0 0) 0x7f64540cb320 con 0x7f645406e010 2023-10-16T20:06:07.647 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' removed 2023-10-16T20:06:07.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.643+0000 7f6459c17640 1 -- 172.21.15.88:0/3428840293 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f64400664a0 msgr2=0x7f6440068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:07.651 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.643+0000 7f6459c17640 1 --2- 172.21.15.88:0/3428840293 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f64400664a0 0x7f6440068960 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f64541f8680 tx=0x7f6448008040 comp rx=0 tx=0).stop 2023-10-16T20:06:07.651 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.643+0000 7f6459c17640 1 -- 172.21.15.88:0/3428840293 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f645406e010 msgr2=0x7f64540ca590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:07.651 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.643+0000 7f6459c17640 1 --2- 172.21.15.88:0/3428840293 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f645406e010 0x7f64540ca590 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f64440ac040 tx=0x7f644401ad40 comp rx=0 tx=0).stop 2023-10-16T20:06:07.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.647+0000 7f6459c17640 1 -- 172.21.15.88:0/3428840293 shutdown_connections 2023-10-16T20:06:07.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.647+0000 7f6459c17640 1 --2- 172.21.15.88:0/3428840293 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f64400664a0 0x7f6440068960 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:07.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.647+0000 7f6459c17640 1 --2- 172.21.15.88:0/3428840293 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f64540abd60 0x7f6454152530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:07.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.647+0000 7f6459c17640 1 --2- 172.21.15.88:0/3428840293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f645415cb20 0x7f6454157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:07.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.647+0000 7f6459c17640 1 --2- 172.21.15.88:0/3428840293 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f645406e010 0x7f64540ca590 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:07.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.647+0000 7f6459c17640 1 -- 172.21.15.88:0/3428840293 >> 172.21.15.88:0/3428840293 conn(0x7f64540b0980 msgr2=0x7f64540c94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:07.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.647+0000 7f6459c17640 1 -- 172.21.15.88:0/3428840293 shutdown_connections 2023-10-16T20:06:07.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.647+0000 7f6459c17640 1 -- 172.21.15.88:0/3428840293 wait complete. 2023-10-16T20:06:07.676 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T20:06:07.831 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.823+0000 7f1e1e238640 1 Processor -- start 2023-10-16T20:06:07.831 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.827+0000 7f1e1e238640 1 -- start start 2023-10-16T20:06:07.832 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.827+0000 7f1e1e238640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1e1815cac0 0x7f1e180d33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:07.832 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.827+0000 7f1e1e238640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e1815cea0 0x7f1e180cd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:07.833 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.827+0000 7f1e1e238640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1806b6f0 con 0x7f1e180cd8a0 2023-10-16T20:06:07.833 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.827+0000 7f1e1e238640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e180ce370 con 0x7f1e1815cac0 2023-10-16T20:06:07.833 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.827+0000 7f1e1e238640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e180ce650 con 0x7f1e1815cea0 2023-10-16T20:06:07.833 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.827+0000 7f1e1d236640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1e1815cac0 0x7f1e180d33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:07.833 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.827+0000 7f1e1da37640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f1e180cd8a0 0x7f1e180ceef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:40592/0 (socket says 172.21.15.88:40592) 2023-10-16T20:06:07.834 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.827+0000 7f1e1da37640 1 -- 172.21.15.88:0/94191899 learned_addr learned my addr 172.21.15.88:0/94191899 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:07.834 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.827+0000 7f1e1d236640 1 -- 172.21.15.88:0/94191899 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e1815cea0 msgr2=0x7f1e180cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:07.834 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.827+0000 7f1e1d236640 1 --2- 172.21.15.88:0/94191899 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e1815cea0 0x7f1e180cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:07.835 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.827+0000 7f1e1d236640 1 -- 172.21.15.88:0/94191899 >> v1:172.21.15.88:6789/0 conn(0x7f1e180cd8a0 legacy=0x7f1e180ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:07.835 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.827+0000 7f1e1d236640 1 -- 172.21.15.88:0/94191899 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1806b0a0 con 0x7f1e1815cac0 2023-10-16T20:06:07.835 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.827+0000 7f1e0ffff640 1 -- 172.21.15.88:0/94191899 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4192601882 0 0) 0x7f1e1806b6f0 con 0x7f1e180cd8a0 2023-10-16T20:06:07.836 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.831+0000 7f1e1d236640 1 --2- 172.21.15.88:0/94191899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e1815cac0 0x7f1e180d33e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1e140097c0 tx=0x7f1e1401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e491efecc012c15 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:07.837 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.831+0000 7f1e0ffff640 1 -- 172.21.15.88:0/94191899 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e140ac640 con 0x7f1e1815cac0 2023-10-16T20:06:07.837 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.831+0000 7f1e0ffff640 1 -- 172.21.15.88:0/94191899 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e140ac7a0 con 0x7f1e1815cac0 2023-10-16T20:06:07.837 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.831+0000 7f1e0ffff640 1 -- 172.21.15.88:0/94191899 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e140acb30 con 0x7f1e1815cac0 2023-10-16T20:06:07.837 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.831+0000 7f1e1e238640 1 -- 172.21.15.88:0/94191899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e1815cac0 msgr2=0x7f1e180d33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:07.837 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.831+0000 7f1e1e238640 1 --2- 172.21.15.88:0/94191899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e1815cac0 0x7f1e180d33e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1e140097c0 tx=0x7f1e1401c3c0 comp rx=0 tx=0).stop 2023-10-16T20:06:07.838 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.831+0000 7f1e1e238640 1 -- 172.21.15.88:0/94191899 shutdown_connections 2023-10-16T20:06:07.838 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.831+0000 7f1e1e238640 1 --2- 172.21.15.88:0/94191899 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e1815cea0 0x7f1e180cd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:07.838 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.831+0000 7f1e1e238640 1 --2- 172.21.15.88:0/94191899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e1815cac0 0x7f1e180d33e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:07.838 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.831+0000 7f1e1e238640 1 -- 172.21.15.88:0/94191899 >> 172.21.15.88:0/94191899 conn(0x7f1e180d28c0 msgr2=0x7f1e180d2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:07.838 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.831+0000 7f1e1e238640 1 -- 172.21.15.88:0/94191899 shutdown_connections 2023-10-16T20:06:07.839 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.831+0000 7f1e1e238640 1 -- 172.21.15.88:0/94191899 wait complete. 2023-10-16T20:06:07.841 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.831+0000 7f1e1e238640 1 Processor -- start 2023-10-16T20:06:07.841 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.831+0000 7f1e1e238640 1 -- start start 2023-10-16T20:06:07.841 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.835+0000 7f1e1e238640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e180cd8a0 0x7f1e180c6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:07.841 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.835+0000 7f1e1e238640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e1815cac0 0x7f1e18141fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:07.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.835+0000 7f1e1e238640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e1815cea0 0x7f1e180c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:07.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.835+0000 7f1e1e238640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e18153be0 con 0x7f1e1815cea0 2023-10-16T20:06:07.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.835+0000 7f1e1e238640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e180cc960 con 0x7f1e1815cac0 2023-10-16T20:06:07.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.835+0000 7f1e1e238640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e180c9380 con 0x7f1e180cd8a0 2023-10-16T20:06:07.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.835+0000 7f1e1da37640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e1815cea0 0x7f1e180c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:07.843 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.835+0000 7f1e1da37640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e1815cea0 0x7f1e180c7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40080/0 (socket says 172.21.15.88:40080) 2023-10-16T20:06:07.843 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.835+0000 7f1e1da37640 1 -- 172.21.15.88:0/1365807776 learned_addr learned my addr 172.21.15.88:0/1365807776 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:07.843 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.835+0000 7f1e1d236640 1 --2- 172.21.15.88:0/1365807776 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e180cd8a0 0x7f1e180c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:07.843 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.835+0000 7f1e1da37640 1 -- 172.21.15.88:0/1365807776 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e180cd8a0 msgr2=0x7f1e180c6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:07.843 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.835+0000 7f1e1da37640 1 --2- 172.21.15.88:0/1365807776 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e180cd8a0 0x7f1e180c6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:07.844 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.835+0000 7f1e1da37640 1 -- 172.21.15.88:0/1365807776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e1815cac0 msgr2=0x7f1e18141fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:07.844 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.835+0000 7f1e1da37640 1 --2- 172.21.15.88:0/1365807776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e1815cac0 0x7f1e18141fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:07.844 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.835+0000 7f1e1da37640 1 -- 172.21.15.88:0/1365807776 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e14009420 con 0x7f1e1815cea0 2023-10-16T20:06:07.844 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.835+0000 7f1e1d236640 1 --2- 172.21.15.88:0/1365807776 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e180cd8a0 0x7f1e180c6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:06:07.844 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.835+0000 7f1e1da37640 1 --2- 172.21.15.88:0/1365807776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e1815cea0 0x7f1e180c7ce0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1e10001570 tx=0x7f1e1000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:07.845 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.835+0000 7f1e0dffb640 1 -- 172.21.15.88:0/1365807776 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1000a5e0 con 0x7f1e1815cea0 2023-10-16T20:06:07.845 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.835+0000 7f1e0dffb640 1 -- 172.21.15.88:0/1365807776 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e1000a760 con 0x7f1e1815cea0 2023-10-16T20:06:07.845 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.835+0000 7f1e1e238640 1 -- 172.21.15.88:0/1365807776 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e18153420 con 0x7f1e1815cea0 2023-10-16T20:06:07.845 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.835+0000 7f1e0dffb640 1 -- 172.21.15.88:0/1365807776 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1000ab10 con 0x7f1e1815cea0 2023-10-16T20:06:07.845 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.835+0000 7f1e1e238640 1 -- 172.21.15.88:0/1365807776 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e180cc680 con 0x7f1e1815cea0 2023-10-16T20:06:07.846 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.835+0000 7f1e1e238640 1 -- 172.21.15.88:0/1365807776 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e18153d20 con 0x7f1e1815cea0 2023-10-16T20:06:07.846 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.835+0000 7f1e0dffb640 1 -- 172.21.15.88:0/1365807776 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1e1000ac70 con 0x7f1e1815cea0 2023-10-16T20:06:07.847 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.839+0000 7f1e0dffb640 1 --2- 172.21.15.88:0/1365807776 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1dfc0664a0 0x7f1dfc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:07.847 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.839+0000 7f1e0dffb640 1 -- 172.21.15.88:0/1365807776 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6311+0+0 (secure 0 0 0) 0x7f1e100836d0 con 0x7f1e1815cea0 2023-10-16T20:06:07.847 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.839+0000 7f1e1d236640 1 --2- 172.21.15.88:0/1365807776 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1dfc0664a0 0x7f1dfc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:07.847 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.839+0000 7f1e1d236640 1 --2- 172.21.15.88:0/1365807776 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1dfc0664a0 0x7f1dfc068960 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1e140ae000 tx=0x7f1e14008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:07.848 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.843+0000 7f1e0dffb640 1 -- 172.21.15.88:0/1365807776 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1e18153d20 con 0x7f1e1815cea0 2023-10-16T20:06:07.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:07.991+0000 7f1e1e238640 1 -- 172.21.15.88:0/1365807776 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1e18153d20 con 0x7f1e1815cea0 2023-10-16T20:06:08.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.643+0000 7f1e0dffb640 1 -- 172.21.15.88:0/1365807776 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v32) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1e18153d20 con 0x7f1e1815cea0 2023-10-16T20:06:08.650 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' removed 2023-10-16T20:06:08.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.647+0000 7f1e1e238640 1 -- 172.21.15.88:0/1365807776 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1dfc0664a0 msgr2=0x7f1dfc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:08.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.647+0000 7f1e1e238640 1 --2- 172.21.15.88:0/1365807776 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1dfc0664a0 0x7f1dfc068960 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1e140ae000 tx=0x7f1e14008040 comp rx=0 tx=0).stop 2023-10-16T20:06:08.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.647+0000 7f1e1e238640 1 -- 172.21.15.88:0/1365807776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e1815cea0 msgr2=0x7f1e180c7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:08.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.647+0000 7f1e1e238640 1 --2- 172.21.15.88:0/1365807776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e1815cea0 0x7f1e180c7ce0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1e10001570 tx=0x7f1e1000cca0 comp rx=0 tx=0).stop 2023-10-16T20:06:08.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.647+0000 7f1e1e238640 1 -- 172.21.15.88:0/1365807776 shutdown_connections 2023-10-16T20:06:08.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.647+0000 7f1e1e238640 1 --2- 172.21.15.88:0/1365807776 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1dfc0664a0 0x7f1dfc068960 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:08.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.647+0000 7f1e1e238640 1 --2- 172.21.15.88:0/1365807776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e1815cea0 0x7f1e180c7ce0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:08.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.647+0000 7f1e1e238640 1 --2- 172.21.15.88:0/1365807776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e1815cac0 0x7f1e18141fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:08.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.647+0000 7f1e1e238640 1 --2- 172.21.15.88:0/1365807776 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e180cd8a0 0x7f1e180c6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:08.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.647+0000 7f1e1e238640 1 -- 172.21.15.88:0/1365807776 >> 172.21.15.88:0/1365807776 conn(0x7f1e180d28c0 msgr2=0x7f1e180af770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:08.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.647+0000 7f1e1e238640 1 -- 172.21.15.88:0/1365807776 shutdown_connections 2023-10-16T20:06:08.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.647+0000 7f1e1e238640 1 -- 172.21.15.88:0/1365807776 wait complete. 2023-10-16T20:06:08.676 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-10-16T20:06:08.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.811+0000 7fb50d59d640 1 Processor -- start 2023-10-16T20:06:08.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.815+0000 7fb50d59d640 1 -- start start 2023-10-16T20:06:08.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.815+0000 7fb50d59d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb5081570e0 0x7fb5081611a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:08.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.815+0000 7fb50d59d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb5080cdef0 0x7fb50815d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:08.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.815+0000 7fb50d59d640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5081616e0 con 0x7fb50815d140 2023-10-16T20:06:08.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.815+0000 7fb50d59d640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb508160790 con 0x7fb5081570e0 2023-10-16T20:06:08.822 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.815+0000 7fb50d59d640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb50815dcb0 con 0x7fb5080cdef0 2023-10-16T20:06:08.822 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.815+0000 7fb506ffd640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb5081570e0 0x7fb5081611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:08.822 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.815+0000 7fb506ffd640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb5081570e0 0x7fb5081611a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40092/0 (socket says 172.21.15.88:40092) 2023-10-16T20:06:08.822 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.815+0000 7fb506ffd640 1 -- 172.21.15.88:0/2918071653 learned_addr learned my addr 172.21.15.88:0/2918071653 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:08.822 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.815+0000 7fb5077fe640 1 --2- 172.21.15.88:0/2918071653 >> v2:172.21.15.94:3300/0 conn(0x7fb5080cdef0 0x7fb50815d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:08.822 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.815+0000 7fb506ffd640 1 -- 172.21.15.88:0/2918071653 >> v2:172.21.15.94:3300/0 conn(0x7fb5080cdef0 msgr2=0x7fb50815d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:08.823 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.815+0000 7fb506ffd640 1 --2- 172.21.15.88:0/2918071653 >> v2:172.21.15.94:3300/0 conn(0x7fb5080cdef0 0x7fb50815d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:08.823 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.815+0000 7fb506ffd640 1 -- 172.21.15.88:0/2918071653 >> v1:172.21.15.94:6789/0 conn(0x7fb50815d140 legacy=0x7fb5080cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:08.823 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.815+0000 7fb506ffd640 1 -- 172.21.15.88:0/2918071653 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50815df90 con 0x7fb5081570e0 2023-10-16T20:06:08.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.819+0000 7fb506ffd640 1 --2- 172.21.15.88:0/2918071653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5081570e0 0x7fb5081611a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb5080b2510 tx=0x7fb4f801c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78ecbde019c413c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:08.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.819+0000 7fb5067fc640 1 -- 172.21.15.88:0/2918071653 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f80ac410 con 0x7fb5081570e0 2023-10-16T20:06:08.825 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.819+0000 7fb5067fc640 1 -- 172.21.15.88:0/2918071653 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f80ac570 con 0x7fb5081570e0 2023-10-16T20:06:08.825 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.819+0000 7fb5067fc640 1 -- 172.21.15.88:0/2918071653 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f80ac900 con 0x7fb5081570e0 2023-10-16T20:06:08.826 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.819+0000 7fb50d59d640 1 -- 172.21.15.88:0/2918071653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5081570e0 msgr2=0x7fb5081611a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:08.826 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.819+0000 7fb50d59d640 1 --2- 172.21.15.88:0/2918071653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5081570e0 0x7fb5081611a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb5080b2510 tx=0x7fb4f801c4c0 comp rx=0 tx=0).stop 2023-10-16T20:06:08.826 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.819+0000 7fb50d59d640 1 -- 172.21.15.88:0/2918071653 shutdown_connections 2023-10-16T20:06:08.826 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.819+0000 7fb50d59d640 1 --2- 172.21.15.88:0/2918071653 >> v2:172.21.15.94:3300/0 conn(0x7fb5080cdef0 0x7fb50815d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:08.827 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.819+0000 7fb50d59d640 1 --2- 172.21.15.88:0/2918071653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5081570e0 0x7fb5081611a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:08.827 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.819+0000 7fb50d59d640 1 -- 172.21.15.88:0/2918071653 >> 172.21.15.88:0/2918071653 conn(0x7fb5080715c0 msgr2=0x7fb5080719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:08.827 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.819+0000 7fb50d59d640 1 -- 172.21.15.88:0/2918071653 shutdown_connections 2023-10-16T20:06:08.827 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.819+0000 7fb50d59d640 1 -- 172.21.15.88:0/2918071653 wait complete. 2023-10-16T20:06:08.829 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.819+0000 7fb50d59d640 1 Processor -- start 2023-10-16T20:06:08.830 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.823+0000 7fb50d59d640 1 -- start start 2023-10-16T20:06:08.830 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.823+0000 7fb50d59d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5080cdef0 0x7fb5080b6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:08.830 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.823+0000 7fb50d59d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5081570e0 0x7fb5080c5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:08.830 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.823+0000 7fb50d59d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb50815d140 0x7fb5081550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:08.831 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.823+0000 7fb50d59d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5080c6640 con 0x7fb5080cdef0 2023-10-16T20:06:08.831 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.823+0000 7fb50d59d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5081513c0 con 0x7fb5081570e0 2023-10-16T20:06:08.831 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.823+0000 7fb50d59d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5081516a0 con 0x7fb50815d140 2023-10-16T20:06:08.831 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.823+0000 7fb506ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5080cdef0 0x7fb5080b6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:08.831 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.823+0000 7fb506ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5080cdef0 0x7fb5080b6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40104/0 (socket says 172.21.15.88:40104) 2023-10-16T20:06:08.832 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.823+0000 7fb506ffd640 1 -- 172.21.15.88:0/2714464305 learned_addr learned my addr 172.21.15.88:0/2714464305 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:08.832 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.823+0000 7fb4fffff640 1 --2- 172.21.15.88:0/2714464305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5081570e0 0x7fb5080c5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:08.832 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.823+0000 7fb5077fe640 1 --2- 172.21.15.88:0/2714464305 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb50815d140 0x7fb5081550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:08.832 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.823+0000 7fb506ffd640 1 -- 172.21.15.88:0/2714464305 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb50815d140 msgr2=0x7fb5081550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:08.832 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.823+0000 7fb506ffd640 1 --2- 172.21.15.88:0/2714464305 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb50815d140 0x7fb5081550e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:08.833 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.823+0000 7fb506ffd640 1 -- 172.21.15.88:0/2714464305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5081570e0 msgr2=0x7fb5080c5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:08.833 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.823+0000 7fb506ffd640 1 --2- 172.21.15.88:0/2714464305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5081570e0 0x7fb5080c5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:08.833 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.823+0000 7fb506ffd640 1 -- 172.21.15.88:0/2714464305 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f80094d0 con 0x7fb5080cdef0 2023-10-16T20:06:08.833 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.823+0000 7fb506ffd640 1 --2- 172.21.15.88:0/2714464305 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5080cdef0 0x7fb5080b6b90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb4f80af040 tx=0x7fb4f8019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:08.833 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.823+0000 7fb4ff7fe640 1 -- 172.21.15.88:0/2714464305 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f80ac410 con 0x7fb5080cdef0 2023-10-16T20:06:08.834 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.823+0000 7fb4ff7fe640 1 -- 172.21.15.88:0/2714464305 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f80b65f0 con 0x7fb5080cdef0 2023-10-16T20:06:08.834 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.823+0000 7fb4ff7fe640 1 -- 172.21.15.88:0/2714464305 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f80b69a0 con 0x7fb5080cdef0 2023-10-16T20:06:08.834 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.823+0000 7fb50d59d640 1 -- 172.21.15.88:0/2714464305 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb508151980 con 0x7fb5080cdef0 2023-10-16T20:06:08.834 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.823+0000 7fb50d59d640 1 -- 172.21.15.88:0/2714464305 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5080b70d0 con 0x7fb5080cdef0 2023-10-16T20:06:08.835 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.823+0000 7fb50d59d640 1 -- 172.21.15.88:0/2714464305 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb50804e9c0 con 0x7fb5080cdef0 2023-10-16T20:06:08.836 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.827+0000 7fb4ff7fe640 1 -- 172.21.15.88:0/2714464305 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb4f80b6b00 con 0x7fb5080cdef0 2023-10-16T20:06:08.836 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.827+0000 7fb4ff7fe640 1 --2- 172.21.15.88:0/2714464305 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb4e80664a0 0x7fb4e8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:08.836 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.827+0000 7fb4ff7fe640 1 -- 172.21.15.88:0/2714464305 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5781+0+0 (secure 0 0 0) 0x7fb4f811e7e0 con 0x7fb5080cdef0 2023-10-16T20:06:08.836 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.827+0000 7fb4fffff640 1 --2- 172.21.15.88:0/2714464305 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb4e80664a0 0x7fb4e8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:08.836 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.827+0000 7fb4fffff640 1 --2- 172.21.15.88:0/2714464305 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb4e80664a0 0x7fb4e8068960 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb508152bc0 tx=0x7fb4f0009060 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:08.837 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.831+0000 7fb4ff7fe640 1 -- 172.21.15.88:0/2714464305 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb4f80ebd40 con 0x7fb5080cdef0 2023-10-16T20:06:08.994 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:08.987+0000 7fb50d59d640 1 -- 172.21.15.88:0/2714464305 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb5080c6780 con 0x7fb5080cdef0 2023-10-16T20:06:09.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.647+0000 7fb4ff7fe640 1 -- 172.21.15.88:0/2714464305 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v33) v1 ==== 178+0+0 (secure 0 0 0) 0x7fb4f80ef360 con 0x7fb5080cdef0 2023-10-16T20:06:09.652 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data_ec' removed 2023-10-16T20:06:09.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.647+0000 7fb50d59d640 1 -- 172.21.15.88:0/2714464305 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb4e80664a0 msgr2=0x7fb4e8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:09.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.647+0000 7fb50d59d640 1 --2- 172.21.15.88:0/2714464305 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb4e80664a0 0x7fb4e8068960 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb508152bc0 tx=0x7fb4f0009060 comp rx=0 tx=0).stop 2023-10-16T20:06:09.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.647+0000 7fb50d59d640 1 -- 172.21.15.88:0/2714464305 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5080cdef0 msgr2=0x7fb5080b6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:09.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.647+0000 7fb50d59d640 1 --2- 172.21.15.88:0/2714464305 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5080cdef0 0x7fb5080b6b90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb4f80af040 tx=0x7fb4f8019260 comp rx=0 tx=0).stop 2023-10-16T20:06:09.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.647+0000 7fb50d59d640 1 -- 172.21.15.88:0/2714464305 shutdown_connections 2023-10-16T20:06:09.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.647+0000 7fb50d59d640 1 --2- 172.21.15.88:0/2714464305 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb4e80664a0 0x7fb4e8068960 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:09.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.647+0000 7fb50d59d640 1 --2- 172.21.15.88:0/2714464305 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb50815d140 0x7fb5081550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:09.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.647+0000 7fb50d59d640 1 --2- 172.21.15.88:0/2714464305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5081570e0 0x7fb5080c5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:09.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.647+0000 7fb50d59d640 1 --2- 172.21.15.88:0/2714464305 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5080cdef0 0x7fb5080b6b90 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:09.657 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.647+0000 7fb50d59d640 1 -- 172.21.15.88:0/2714464305 >> 172.21.15.88:0/2714464305 conn(0x7fb5080715c0 msgr2=0x7fb50815b110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:09.658 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.651+0000 7fb50d59d640 1 -- 172.21.15.88:0/2714464305 shutdown_connections 2023-10-16T20:06:09.658 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.651+0000 7fb50d59d640 1 -- 172.21.15.88:0/2714464305 wait complete. 2023-10-16T20:06:09.689 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T20:06:09.690 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T20:06:09.690 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T20:06:09.690 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:06:09.690 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T20:06:09.691 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T20:06:09.691 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T20:06:09.691 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:06:09.692 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T20:06:09.692 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T20:06:09.692 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T20:06:09.692 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:06:09.693 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T20:06:09.693 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T20:06:09.693 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T20:06:09.694 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:06:09.702 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T20:06:09.702 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T20:06:09.704 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T20:06:09.705 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T20:06:09.706 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T20:06:09.707 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T20:06:09.707 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T20:06:09.708 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T20:06:09.753 INFO:tasks.ceph.mds.a:Started 2023-10-16T20:06:09.754 DEBUG:teuthology.parallel:result is None 2023-10-16T20:06:09.755 INFO:tasks.ceph.mds.b:Started 2023-10-16T20:06:09.755 DEBUG:teuthology.parallel:result is None 2023-10-16T20:06:09.756 INFO:tasks.ceph.mds.c:Started 2023-10-16T20:06:09.757 DEBUG:teuthology.parallel:result is None 2023-10-16T20:06:09.757 INFO:tasks.ceph.mds.d:Started 2023-10-16T20:06:09.758 DEBUG:teuthology.parallel:result is None 2023-10-16T20:06:09.758 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T20:06:09.841 INFO:tasks.ceph.mds.a.smithi088.stdout:starting mds.a at 2023-10-16T20:06:09.843 INFO:tasks.ceph.mds.c.smithi088.stdout:starting mds.c at 2023-10-16T20:06:09.846 INFO:tasks.ceph.mds.b.smithi094.stdout:starting mds.b at 2023-10-16T20:06:09.846 INFO:tasks.ceph.mds.d.smithi094.stdout:starting mds.d at 2023-10-16T20:06:09.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.899+0000 7fd141854640 1 Processor -- start 2023-10-16T20:06:09.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.899+0000 7fd141854640 1 -- start start 2023-10-16T20:06:09.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.899+0000 7fd141854640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd13c0ad050 0x7fd13c0ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:09.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.899+0000 7fd141854640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd13c0cedd0 0x7fd13c0b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:09.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.899+0000 7fd141854640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd13c0cf210 con 0x7fd13c0b1820 2023-10-16T20:06:09.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.899+0000 7fd141854640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd13c0b1f80 con 0x7fd13c0cedd0 2023-10-16T20:06:09.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.899+0000 7fd141854640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c0b2260 con 0x7fd13c0ad050 2023-10-16T20:06:09.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.899+0000 7fd13affd640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd13c0ad050 0x7fd13c0ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:09.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.899+0000 7fd13a7fc640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd13c0cedd0 0x7fd13c0b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:09.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.899+0000 7fd13b7fe640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fd13c0b1820 0x7fd13c0ab120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:57194/0 (socket says 172.21.15.88:57194) 2023-10-16T20:06:09.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.899+0000 7fd13b7fe640 1 -- 172.21.15.88:0/1088553247 learned_addr learned my addr 172.21.15.88:0/1088553247 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:09.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.899+0000 7fd13affd640 1 -- 172.21.15.88:0/1088553247 >> v2:172.21.15.94:3300/0 conn(0x7fd13c0cedd0 msgr2=0x7fd13c0b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:09.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.899+0000 7fd139ffb640 1 -- 172.21.15.88:0/1088553247 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1961880494 0 0) 0x7fd13c0cf210 con 0x7fd13c0b1820 2023-10-16T20:06:09.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.899+0000 7fd13affd640 1 --2- 172.21.15.88:0/1088553247 >> v2:172.21.15.94:3300/0 conn(0x7fd13c0cedd0 0x7fd13c0b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:09.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.899+0000 7fd13affd640 1 -- 172.21.15.88:0/1088553247 >> v1:172.21.15.94:6789/0 conn(0x7fd13c0b1820 legacy=0x7fd13c0ab120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:09.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.899+0000 7fd13affd640 1 -- 172.21.15.88:0/1088553247 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd13c0b2540 con 0x7fd13c0ad050 2023-10-16T20:06:09.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.903+0000 7fd13affd640 1 --2- 172.21.15.88:0/1088553247 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd13c0ad050 0x7fd13c0ce890 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd12c0097c0 tx=0x7fd12c01c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e57ea5802568f47 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:09.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.903+0000 7fd139ffb640 1 -- 172.21.15.88:0/1088553247 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12c0ab600 con 0x7fd13c0ad050 2023-10-16T20:06:09.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.903+0000 7fd139ffb640 1 -- 172.21.15.88:0/1088553247 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd12c0a7050 con 0x7fd13c0ad050 2023-10-16T20:06:09.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.903+0000 7fd139ffb640 1 -- 172.21.15.88:0/1088553247 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12c0ab980 con 0x7fd13c0ad050 2023-10-16T20:06:09.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.903+0000 7fd141854640 1 -- 172.21.15.88:0/1088553247 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd13c0ad050 msgr2=0x7fd13c0ce890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:09.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.903+0000 7fd141854640 1 --2- 172.21.15.88:0/1088553247 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd13c0ad050 0x7fd13c0ce890 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd12c0097c0 tx=0x7fd12c01c480 comp rx=0 tx=0).stop 2023-10-16T20:06:09.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.903+0000 7fd141854640 1 -- 172.21.15.88:0/1088553247 shutdown_connections 2023-10-16T20:06:09.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.903+0000 7fd141854640 1 --2- 172.21.15.88:0/1088553247 >> v2:172.21.15.94:3300/0 conn(0x7fd13c0cedd0 0x7fd13c0b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:09.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.903+0000 7fd141854640 1 --2- 172.21.15.88:0/1088553247 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd13c0ad050 0x7fd13c0ce890 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:09.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.903+0000 7fd141854640 1 -- 172.21.15.88:0/1088553247 >> 172.21.15.88:0/1088553247 conn(0x7fd13c06a290 msgr2=0x7fd13c06a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:09.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.903+0000 7fd141854640 1 -- 172.21.15.88:0/1088553247 shutdown_connections 2023-10-16T20:06:09.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.903+0000 7fd141854640 1 -- 172.21.15.88:0/1088553247 wait complete. 2023-10-16T20:06:09.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.907+0000 7fd141854640 1 Processor -- start 2023-10-16T20:06:09.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.907+0000 7fd141854640 1 -- start start 2023-10-16T20:06:09.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.907+0000 7fd141854640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd13c0ad050 0x7fd13c0af830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:09.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.907+0000 7fd141854640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd13c0b1820 0x7fd13c0c4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:09.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.907+0000 7fd141854640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd13c0cedd0 0x7fd13c0c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:09.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.907+0000 7fd141854640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c15b410 con 0x7fd13c0cedd0 2023-10-16T20:06:09.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.907+0000 7fd141854640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c156dc0 con 0x7fd13c0b1820 2023-10-16T20:06:09.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.907+0000 7fd141854640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c0af3a0 con 0x7fd13c0ad050 2023-10-16T20:06:09.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.907+0000 7fd13b7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd13c0cedd0 0x7fd13c0c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:09.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.907+0000 7fd13affd640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd13c0ad050 0x7fd13c0af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:09.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.907+0000 7fd13b7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd13c0cedd0 0x7fd13c0c7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40132/0 (socket says 172.21.15.88:40132) 2023-10-16T20:06:09.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.907+0000 7fd13b7fe640 1 -- 172.21.15.88:0/3418374682 learned_addr learned my addr 172.21.15.88:0/3418374682 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:09.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.907+0000 7fd13a7fc640 1 --2- 172.21.15.88:0/3418374682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd13c0b1820 0x7fd13c0c4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:09.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.907+0000 7fd13affd640 1 -- 172.21.15.88:0/3418374682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd13c0b1820 msgr2=0x7fd13c0c4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:09.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.907+0000 7fd13affd640 1 --2- 172.21.15.88:0/3418374682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd13c0b1820 0x7fd13c0c4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:09.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.907+0000 7fd13affd640 1 -- 172.21.15.88:0/3418374682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd13c0cedd0 msgr2=0x7fd13c0c7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:09.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.907+0000 7fd13affd640 1 --2- 172.21.15.88:0/3418374682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd13c0cedd0 0x7fd13c0c7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:09.919 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.907+0000 7fd13affd640 1 -- 172.21.15.88:0/3418374682 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd12c009420 con 0x7fd13c0ad050 2023-10-16T20:06:09.919 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.907+0000 7fd13b7fe640 1 --2- 172.21.15.88:0/3418374682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd13c0cedd0 0x7fd13c0c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:06:09.919 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.907+0000 7fd13affd640 1 --2- 172.21.15.88:0/3418374682 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd13c0ad050 0x7fd13c0af830 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd13c0b23a0 tx=0x7fd12c01acc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:09.919 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.907+0000 7fd123fff640 1 -- 172.21.15.88:0/3418374682 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12c0b7650 con 0x7fd13c0ad050 2023-10-16T20:06:09.919 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.907+0000 7fd123fff640 1 -- 172.21.15.88:0/3418374682 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd12c0b77b0 con 0x7fd13c0ad050 2023-10-16T20:06:09.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.907+0000 7fd123fff640 1 -- 172.21.15.88:0/3418374682 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12c0b7b60 con 0x7fd13c0ad050 2023-10-16T20:06:09.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.907+0000 7fd141854640 1 -- 172.21.15.88:0/3418374682 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd13c15ae50 con 0x7fd13c0ad050 2023-10-16T20:06:09.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.907+0000 7fd141854640 1 -- 172.21.15.88:0/3418374682 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd13c0af030 con 0x7fd13c0ad050 2023-10-16T20:06:09.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.907+0000 7fd141854640 1 -- 172.21.15.88:0/3418374682 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd13c0674f0 con 0x7fd13c0ad050 2023-10-16T20:06:09.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.911+0000 7fd123fff640 1 -- 172.21.15.88:0/3418374682 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd12c0b7cc0 con 0x7fd13c0ad050 2023-10-16T20:06:09.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.911+0000 7fd123fff640 1 --2- 172.21.15.88:0/3418374682 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd1280664a0 0x7fd128068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:09.922 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.911+0000 7fd123fff640 1 -- 172.21.15.88:0/3418374682 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fd12c11efe0 con 0x7fd13c0ad050 2023-10-16T20:06:09.922 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.911+0000 7fd13a7fc640 1 --2- 172.21.15.88:0/3418374682 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd1280664a0 0x7fd128068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:09.922 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.911+0000 7fd13a7fc640 1 --2- 172.21.15.88:0/3418374682 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd1280664a0 0x7fd128068960 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd130009b60 tx=0x7fd130009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:09.922 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:09.915+0000 7fd123fff640 1 -- 172.21.15.88:0/3418374682 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd12c0ec830 con 0x7fd13c0ad050 2023-10-16T20:06:10.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.079+0000 7fd141854640 1 -- 172.21.15.88:0/3418374682 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd13c06a7a0 con 0x7fd13c0ad050 2023-10-16T20:06:10.675 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.667+0000 7fd123fff640 1 -- 172.21.15.88:0/3418374682 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v35) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd12c0b0080 con 0x7fd13c0ad050 2023-10-16T20:06:10.675 INFO:teuthology.orchestra.run.smithi088.stderr: removed all blocklist entries 2023-10-16T20:06:10.677 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.671+0000 7fd141854640 1 -- 172.21.15.88:0/3418374682 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd1280664a0 msgr2=0x7fd128068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:10.677 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.671+0000 7fd141854640 1 --2- 172.21.15.88:0/3418374682 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd1280664a0 0x7fd128068960 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd130009b60 tx=0x7fd130009190 comp rx=0 tx=0).stop 2023-10-16T20:06:10.678 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.671+0000 7fd141854640 1 -- 172.21.15.88:0/3418374682 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd13c0ad050 msgr2=0x7fd13c0af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:10.678 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.671+0000 7fd141854640 1 --2- 172.21.15.88:0/3418374682 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd13c0ad050 0x7fd13c0af830 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd13c0b23a0 tx=0x7fd12c01acc0 comp rx=0 tx=0).stop 2023-10-16T20:06:10.678 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.671+0000 7fd141854640 1 -- 172.21.15.88:0/3418374682 shutdown_connections 2023-10-16T20:06:10.678 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.671+0000 7fd141854640 1 --2- 172.21.15.88:0/3418374682 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd1280664a0 0x7fd128068960 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:10.678 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.671+0000 7fd141854640 1 --2- 172.21.15.88:0/3418374682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd13c0cedd0 0x7fd13c0c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:10.679 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.671+0000 7fd141854640 1 --2- 172.21.15.88:0/3418374682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd13c0b1820 0x7fd13c0c4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:10.679 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.671+0000 7fd141854640 1 --2- 172.21.15.88:0/3418374682 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd13c0ad050 0x7fd13c0af830 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:10.679 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.671+0000 7fd141854640 1 -- 172.21.15.88:0/3418374682 >> 172.21.15.88:0/3418374682 conn(0x7fd13c06a290 msgr2=0x7fd13c06cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:10.679 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.671+0000 7fd141854640 1 -- 172.21.15.88:0/3418374682 shutdown_connections 2023-10-16T20:06:10.679 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.671+0000 7fd141854640 1 -- 172.21.15.88:0/3418374682 wait complete. 2023-10-16T20:06:10.699 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T20:06:10.838 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.831+0000 7f3a264c9640 1 Processor -- start 2023-10-16T20:06:10.838 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.831+0000 7f3a264c9640 1 -- start start 2023-10-16T20:06:10.839 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.835+0000 7f3a264c9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a201599f0 0x7f3a200b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:10.839 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.835+0000 7f3a264c9640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3a20069e90 0x7f3a2006a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:10.840 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.835+0000 7f3a264c9640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a2006a830 con 0x7f3a200b05a0 2023-10-16T20:06:10.840 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.835+0000 7f3a264c9640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a200b1260 con 0x7f3a20069e90 2023-10-16T20:06:10.840 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.835+0000 7f3a264c9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a200ae960 con 0x7f3a201599f0 2023-10-16T20:06:10.841 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.835+0000 7f3a1ffff640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a201599f0 0x7f3a200b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:10.841 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.835+0000 7f3a24a66640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f3a200b05a0 0x7f3a200b0a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:57196/0 (socket says 172.21.15.88:57196) 2023-10-16T20:06:10.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.835+0000 7f3a24a66640 1 -- 172.21.15.88:0/790387242 learned_addr learned my addr 172.21.15.88:0/790387242 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:10.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.835+0000 7f3a1ffff640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a201599f0 0x7f3a200b2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:41514/0 (socket says 172.21.15.88:41514) 2023-10-16T20:06:10.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.835+0000 7f3a1f7fe640 1 --2- 172.21.15.88:0/790387242 >> v2:172.21.15.94:3300/0 conn(0x7f3a20069e90 0x7f3a2006a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:10.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.835+0000 7f3a1effd640 1 -- 172.21.15.88:0/790387242 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2882435129 0 0) 0x7f3a2006a830 con 0x7f3a200b05a0 2023-10-16T20:06:10.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.835+0000 7f3a1effd640 1 -- 172.21.15.88:0/790387242 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a08005740 con 0x7f3a200b05a0 2023-10-16T20:06:10.843 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.835+0000 7f3a1ffff640 1 -- 172.21.15.88:0/790387242 >> v2:172.21.15.94:3300/0 conn(0x7f3a20069e90 msgr2=0x7f3a2006a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:10.843 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.835+0000 7f3a1ffff640 1 --2- 172.21.15.88:0/790387242 >> v2:172.21.15.94:3300/0 conn(0x7f3a20069e90 0x7f3a2006a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:10.843 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.835+0000 7f3a1ffff640 1 -- 172.21.15.88:0/790387242 >> v1:172.21.15.94:6789/0 conn(0x7f3a200b05a0 legacy=0x7f3a200b0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:10.843 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.835+0000 7f3a1ffff640 1 -- 172.21.15.88:0/790387242 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a200aec40 con 0x7f3a201599f0 2023-10-16T20:06:10.844 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.839+0000 7f3a1ffff640 1 --2- 172.21.15.88:0/790387242 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a201599f0 0x7f3a200b2a30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3a100097c0 tx=0x7f3a1001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f852025dbeee9a06 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:10.844 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.839+0000 7f3a1effd640 1 -- 172.21.15.88:0/790387242 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a100ab650 con 0x7f3a201599f0 2023-10-16T20:06:10.845 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.839+0000 7f3a1effd640 1 -- 172.21.15.88:0/790387242 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a100ab7b0 con 0x7f3a201599f0 2023-10-16T20:06:10.845 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.839+0000 7f3a1effd640 1 -- 172.21.15.88:0/790387242 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a100abb40 con 0x7f3a201599f0 2023-10-16T20:06:10.845 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.839+0000 7f3a264c9640 1 -- 172.21.15.88:0/790387242 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a201599f0 msgr2=0x7f3a200b2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:10.845 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.839+0000 7f3a264c9640 1 --2- 172.21.15.88:0/790387242 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a201599f0 0x7f3a200b2a30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3a100097c0 tx=0x7f3a1001c450 comp rx=0 tx=0).stop 2023-10-16T20:06:10.845 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.839+0000 7f3a264c9640 1 -- 172.21.15.88:0/790387242 shutdown_connections 2023-10-16T20:06:10.846 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.839+0000 7f3a264c9640 1 --2- 172.21.15.88:0/790387242 >> v2:172.21.15.94:3300/0 conn(0x7f3a20069e90 0x7f3a2006a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:10.846 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.839+0000 7f3a264c9640 1 --2- 172.21.15.88:0/790387242 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a201599f0 0x7f3a200b2a30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:10.846 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.839+0000 7f3a264c9640 1 -- 172.21.15.88:0/790387242 >> 172.21.15.88:0/790387242 conn(0x7f3a200ce5e0 msgr2=0x7f3a200ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:10.846 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.839+0000 7f3a264c9640 1 -- 172.21.15.88:0/790387242 shutdown_connections 2023-10-16T20:06:10.846 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.839+0000 7f3a264c9640 1 -- 172.21.15.88:0/790387242 wait complete. 2023-10-16T20:06:10.848 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.839+0000 7f3a264c9640 1 Processor -- start 2023-10-16T20:06:10.849 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.839+0000 7f3a264c9640 1 -- start start 2023-10-16T20:06:10.849 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.843+0000 7f3a264c9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a20069e90 0x7f3a20148d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:10.849 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.843+0000 7f3a264c9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a200b05a0 0x7f3a200ab890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:10.849 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.843+0000 7f3a264c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a201599f0 0x7f3a2015bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:10.849 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.843+0000 7f3a264c9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2014f110 con 0x7f3a200b05a0 2023-10-16T20:06:10.850 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.843+0000 7f3a264c9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a200ac270 con 0x7f3a201599f0 2023-10-16T20:06:10.850 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.843+0000 7f3a264c9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a200ac550 con 0x7f3a20069e90 2023-10-16T20:06:10.850 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.843+0000 7f3a1f7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a200b05a0 0x7f3a200ab890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:10.850 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.843+0000 7f3a1f7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a200b05a0 0x7f3a200ab890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40148/0 (socket says 172.21.15.88:40148) 2023-10-16T20:06:10.850 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.843+0000 7f3a1f7fe640 1 -- 172.21.15.88:0/775276918 learned_addr learned my addr 172.21.15.88:0/775276918 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:10.851 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.843+0000 7f3a1ffff640 1 --2- 172.21.15.88:0/775276918 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a20069e90 0x7f3a20148d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:10.851 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.843+0000 7f3a24a66640 1 --2- 172.21.15.88:0/775276918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a201599f0 0x7f3a2015bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:10.851 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.843+0000 7f3a1f7fe640 1 -- 172.21.15.88:0/775276918 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a20069e90 msgr2=0x7f3a20148d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:10.851 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.843+0000 7f3a1f7fe640 1 --2- 172.21.15.88:0/775276918 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a20069e90 0x7f3a20148d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:10.851 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.843+0000 7f3a1f7fe640 1 -- 172.21.15.88:0/775276918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a201599f0 msgr2=0x7f3a2015bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:10.852 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.843+0000 7f3a1f7fe640 1 --2- 172.21.15.88:0/775276918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a201599f0 0x7f3a2015bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:10.852 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.843+0000 7f3a1f7fe640 1 -- 172.21.15.88:0/775276918 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a10009420 con 0x7f3a200b05a0 2023-10-16T20:06:10.852 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.843+0000 7f3a1f7fe640 1 --2- 172.21.15.88:0/775276918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a200b05a0 0x7f3a200ab890 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3a1400e930 tx=0x7f3a1400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:10.852 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.843+0000 7f3a1cff9640 1 -- 172.21.15.88:0/775276918 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a14009690 con 0x7f3a200b05a0 2023-10-16T20:06:10.852 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.843+0000 7f3a1cff9640 1 -- 172.21.15.88:0/775276918 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1400ca00 con 0x7f3a200b05a0 2023-10-16T20:06:10.852 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.843+0000 7f3a264c9640 1 -- 172.21.15.88:0/775276918 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a200ac830 con 0x7f3a200b05a0 2023-10-16T20:06:10.853 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.843+0000 7f3a1cff9640 1 -- 172.21.15.88:0/775276918 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1400cd70 con 0x7f3a200b05a0 2023-10-16T20:06:10.853 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.843+0000 7f3a264c9640 1 -- 172.21.15.88:0/775276918 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a201492c0 con 0x7f3a200b05a0 2023-10-16T20:06:10.854 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.843+0000 7f3a264c9640 1 -- 172.21.15.88:0/775276918 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a2014eb90 con 0x7f3a200b05a0 2023-10-16T20:06:10.854 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.843+0000 7f3a1cff9640 1 -- 172.21.15.88:0/775276918 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3a14013470 con 0x7f3a200b05a0 2023-10-16T20:06:10.854 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.847+0000 7f3a1cff9640 1 --2- 172.21.15.88:0/775276918 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3a0c0664a0 0x7f3a0c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:10.854 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.847+0000 7f3a1cff9640 1 -- 172.21.15.88:0/775276918 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f3a14083e20 con 0x7f3a200b05a0 2023-10-16T20:06:10.855 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.847+0000 7f3a1ffff640 1 --2- 172.21.15.88:0/775276918 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3a0c0664a0 0x7f3a0c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:10.855 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.847+0000 7f3a1ffff640 1 --2- 172.21.15.88:0/775276918 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3a0c0664a0 0x7f3a0c068960 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3a100af000 tx=0x7f3a100a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:10.855 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:10.851+0000 7f3a1cff9640 1 -- 172.21.15.88:0/775276918 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3a2014eb90 con 0x7f3a200b05a0 2023-10-16T20:06:11.058 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.051+0000 7f3a264c9640 1 -- 172.21.15.88:0/775276918 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3a2014eb90 con 0x7f3a200b05a0 2023-10-16T20:06:11.059 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.051+0000 7f3a1cff9640 1 -- 172.21.15.88:0/775276918 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f3a2014eb90 con 0x7f3a200b05a0 2023-10-16T20:06:11.059 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:06:11.059 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2023-10-16T20:06:11.060 INFO:teuthology.orchestra.run.smithi088.stdout: "auth_dump": [ 2023-10-16T20:06:11.060 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:06:11.060 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.a", 2023-10-16T20:06:11.060 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQAuly1lCHa1JhAAhNsz0rC3KpOhywCHwmjWyg==", 2023-10-16T20:06:11.060 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:06:11.061 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2023-10-16T20:06:11.061 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2023-10-16T20:06:11.061 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2023-10-16T20:06:11.061 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2023-10-16T20:06:11.061 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:06:11.062 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:11.062 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:06:11.062 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.b", 2023-10-16T20:06:11.062 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQAuly1lxz58MhAA69WwY8VLqgyX4KQ9pGS5QQ==", 2023-10-16T20:06:11.062 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:06:11.063 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2023-10-16T20:06:11.063 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2023-10-16T20:06:11.063 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2023-10-16T20:06:11.063 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2023-10-16T20:06:11.063 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:06:11.064 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:11.064 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:06:11.064 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.c", 2023-10-16T20:06:11.064 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQAuly1ltUzfLRAAJWxUKQWdDYVtMTHHYipcZQ==", 2023-10-16T20:06:11.064 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:06:11.064 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2023-10-16T20:06:11.065 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2023-10-16T20:06:11.065 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2023-10-16T20:06:11.065 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2023-10-16T20:06:11.065 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:06:11.065 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:11.066 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:06:11.066 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.d", 2023-10-16T20:06:11.066 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQAuly1l+TXWORAAJv6W5uoM9XLv1keyH1iJGg==", 2023-10-16T20:06:11.066 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:06:11.066 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2023-10-16T20:06:11.066 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2023-10-16T20:06:11.067 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2023-10-16T20:06:11.067 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2023-10-16T20:06:11.067 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:06:11.067 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:11.067 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:06:11.067 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.0", 2023-10-16T20:06:11.068 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQAyly1l1tzTGhAANOg63+nIoOxhLwHYO2iqQA==", 2023-10-16T20:06:11.068 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:06:11.068 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2023-10-16T20:06:11.068 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2023-10-16T20:06:11.068 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2023-10-16T20:06:11.068 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:06:11.069 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:11.069 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:06:11.069 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.1", 2023-10-16T20:06:11.069 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQA1ly1lgigrIRAA0Mzb88vvzSzGqEzZVsRUtQ==", 2023-10-16T20:06:11.069 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:06:11.070 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2023-10-16T20:06:11.070 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2023-10-16T20:06:11.070 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2023-10-16T20:06:11.070 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:06:11.070 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:11.071 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:06:11.071 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.2", 2023-10-16T20:06:11.071 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQA4ly1lN0lgKBAAALodK7N7sDf5GdZrccritg==", 2023-10-16T20:06:11.071 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:06:11.071 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2023-10-16T20:06:11.071 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2023-10-16T20:06:11.072 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2023-10-16T20:06:11.072 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:06:11.072 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:11.072 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:06:11.072 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.3", 2023-10-16T20:06:11.072 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQA7ly1lrVjALhAAOc9pmNFdvOZ1gT6VMXOgzA==", 2023-10-16T20:06:11.073 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:06:11.073 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2023-10-16T20:06:11.073 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2023-10-16T20:06:11.073 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2023-10-16T20:06:11.073 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:06:11.074 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:11.074 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:06:11.074 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.4", 2023-10-16T20:06:11.074 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQBAly1lVlqzABAAIg5Q4X0rytK7dTFXzhulLw==", 2023-10-16T20:06:11.074 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:06:11.075 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2023-10-16T20:06:11.075 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2023-10-16T20:06:11.075 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2023-10-16T20:06:11.075 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:06:11.075 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:11.075 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:06:11.076 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.5", 2023-10-16T20:06:11.076 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQBDly1l9gS2BxAA+VSkv60PdAkqsolXnZSLUg==", 2023-10-16T20:06:11.076 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:06:11.076 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2023-10-16T20:06:11.076 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2023-10-16T20:06:11.076 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2023-10-16T20:06:11.077 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:06:11.077 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:11.077 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:06:11.077 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.6", 2023-10-16T20:06:11.077 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQBGly1lADNDDhAA/1Wahdkwdt708rxp8XnTRg==", 2023-10-16T20:06:11.077 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:06:11.078 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2023-10-16T20:06:11.078 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2023-10-16T20:06:11.078 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2023-10-16T20:06:11.078 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:06:11.078 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:11.079 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:06:11.079 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.7", 2023-10-16T20:06:11.079 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQBJly1lwLN1FBAABUh5iYOJQHAcceNeGuxQKA==", 2023-10-16T20:06:11.079 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:06:11.079 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2023-10-16T20:06:11.080 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2023-10-16T20:06:11.080 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2023-10-16T20:06:11.080 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:06:11.080 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:11.080 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:06:11.080 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.0", 2023-10-16T20:06:11.081 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQAvly1looeRAhAAeLshHOVIhQqoTtlPehorIg==", 2023-10-16T20:06:11.081 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:06:11.081 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2023-10-16T20:06:11.081 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2023-10-16T20:06:11.081 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2023-10-16T20:06:11.082 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2023-10-16T20:06:11.082 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:06:11.082 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:11.082 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:06:11.082 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.1", 2023-10-16T20:06:11.083 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQAvly1lvy8qDBAAc789x2xSuuM7SHbL5XbRwg==", 2023-10-16T20:06:11.083 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:06:11.083 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2023-10-16T20:06:11.083 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2023-10-16T20:06:11.083 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2023-10-16T20:06:11.083 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2023-10-16T20:06:11.084 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:06:11.084 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:11.084 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:06:11.084 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.2", 2023-10-16T20:06:11.084 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQAvly1lhdfcCBAAJU/W9X3cX1ukGJOXsW3U7A==", 2023-10-16T20:06:11.085 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:06:11.085 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2023-10-16T20:06:11.085 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2023-10-16T20:06:11.085 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2023-10-16T20:06:11.085 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2023-10-16T20:06:11.085 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:06:11.086 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:11.086 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:06:11.086 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.3", 2023-10-16T20:06:11.086 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQAvly1lqPsnEhAAzgEkt/iWPI1jIPCKw0rCUg==", 2023-10-16T20:06:11.086 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:06:11.087 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2023-10-16T20:06:11.087 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2023-10-16T20:06:11.087 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2023-10-16T20:06:11.087 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2023-10-16T20:06:11.087 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:06:11.087 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:11.088 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:06:11.088 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.admin", 2023-10-16T20:06:11.088 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQAuly1l56PKCRAAdwHp3ILkhpj9GWIccpDmSg==", 2023-10-16T20:06:11.088 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:06:11.088 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2023-10-16T20:06:11.089 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2023-10-16T20:06:11.089 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2023-10-16T20:06:11.089 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2023-10-16T20:06:11.089 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:06:11.089 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:11.090 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:06:11.090 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.bootstrap-mds", 2023-10-16T20:06:11.090 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQBPly1lOPTOFhAAnr2BRlbIMs3Tzf+jU1k7nw==", 2023-10-16T20:06:11.090 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:06:11.090 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile bootstrap-mds" 2023-10-16T20:06:11.090 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:06:11.091 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:11.091 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:06:11.091 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.bootstrap-mgr", 2023-10-16T20:06:11.091 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQBPly1lhf3OFhAA3r2DZ5YloXadZgg/EgZpPw==", 2023-10-16T20:06:11.091 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:06:11.092 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-16T20:06:11.092 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:06:11.092 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:11.092 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:06:11.092 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.bootstrap-osd", 2023-10-16T20:06:11.092 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQBPly1l3gXPFhAAGwFra9hwdOvcZF8qdlK5Uw==", 2023-10-16T20:06:11.093 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:06:11.093 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile bootstrap-osd" 2023-10-16T20:06:11.093 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:06:11.093 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:11.093 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:06:11.093 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.bootstrap-rbd", 2023-10-16T20:06:11.094 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQBPly1l7g3PFhAAQnRC5p+UMWFfY1KaGsrsMA==", 2023-10-16T20:06:11.094 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:06:11.094 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-16T20:06:11.094 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:06:11.094 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:11.095 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:06:11.095 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-16T20:06:11.095 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQBPly1lRxbPFhAARvQQ3A7HMlvQ0yHJx5XQ/g==", 2023-10-16T20:06:11.095 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:06:11.095 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-16T20:06:11.095 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:06:11.096 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:11.096 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:06:11.096 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.bootstrap-rgw", 2023-10-16T20:06:11.096 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQBPly1lqB7PFhAA0EYxUtpz/wrZxVkCcbmnqw==", 2023-10-16T20:06:11.096 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:06:11.097 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-16T20:06:11.097 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:06:11.097 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:11.097 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:06:11.097 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mgr.x", 2023-10-16T20:06:11.097 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQAuly1l9hztIRAAGJWqLd1Aa6sZAzEHc+mXZQ==", 2023-10-16T20:06:11.098 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:06:11.098 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2023-10-16T20:06:11.098 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile mgr", 2023-10-16T20:06:11.098 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2023-10-16T20:06:11.098 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:06:11.099 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:11.099 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:06:11.099 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mgr.y", 2023-10-16T20:06:11.099 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQAuly1lDn1VHhAApRTbdVW1sInFHR4ftXIgLA==", 2023-10-16T20:06:11.099 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:06:11.100 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2023-10-16T20:06:11.100 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile mgr", 2023-10-16T20:06:11.100 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2023-10-16T20:06:11.100 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:06:11.100 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:06:11.101 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2023-10-16T20:06:11.101 INFO:teuthology.orchestra.run.smithi088.stdout:} 2023-10-16T20:06:11.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.055+0000 7f3a264c9640 1 -- 172.21.15.88:0/775276918 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3a0c0664a0 msgr2=0x7f3a0c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:11.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.055+0000 7f3a264c9640 1 --2- 172.21.15.88:0/775276918 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3a0c0664a0 0x7f3a0c068960 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3a100af000 tx=0x7f3a100a7000 comp rx=0 tx=0).stop 2023-10-16T20:06:11.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.055+0000 7f3a264c9640 1 -- 172.21.15.88:0/775276918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a200b05a0 msgr2=0x7f3a200ab890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:11.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.055+0000 7f3a264c9640 1 --2- 172.21.15.88:0/775276918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a200b05a0 0x7f3a200ab890 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3a1400e930 tx=0x7f3a1400edf0 comp rx=0 tx=0).stop 2023-10-16T20:06:11.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.055+0000 7f3a264c9640 1 -- 172.21.15.88:0/775276918 shutdown_connections 2023-10-16T20:06:11.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.055+0000 7f3a264c9640 1 --2- 172.21.15.88:0/775276918 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3a0c0664a0 0x7f3a0c068960 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:11.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.055+0000 7f3a264c9640 1 --2- 172.21.15.88:0/775276918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a201599f0 0x7f3a2015bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:11.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.055+0000 7f3a264c9640 1 --2- 172.21.15.88:0/775276918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a200b05a0 0x7f3a200ab890 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:11.104 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.055+0000 7f3a264c9640 1 --2- 172.21.15.88:0/775276918 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a20069e90 0x7f3a20148d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:11.104 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.055+0000 7f3a264c9640 1 -- 172.21.15.88:0/775276918 >> 172.21.15.88:0/775276918 conn(0x7f3a200ce5e0 msgr2=0x7f3a200b17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:11.104 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.055+0000 7f3a264c9640 1 -- 172.21.15.88:0/775276918 shutdown_connections 2023-10-16T20:06:11.104 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.055+0000 7f3a264c9640 1 -- 172.21.15.88:0/775276918 wait complete. 2023-10-16T20:06:11.105 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-16T20:06:11.244 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.239+0000 7f0439fc4640 1 Processor -- start 2023-10-16T20:06:11.245 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.239+0000 7f0439fc4640 1 -- start start 2023-10-16T20:06:11.246 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.239+0000 7f0439fc4640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0434153ea0 0x7f04340d1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:11.246 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.239+0000 7f0439fc4640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04340b5510 con 0x7f04340cd970 2023-10-16T20:06:11.246 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.239+0000 7f0439fc4640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f043415d9a0 con 0x7f04340d2660 2023-10-16T20:06:11.246 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.239+0000 7f0439fc4640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04340b4990 con 0x7f0434153ea0 2023-10-16T20:06:11.247 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.239+0000 7f0433fff640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f04340cd970 0x7f04340cdde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:40616/0 (socket says 172.21.15.88:40616) 2023-10-16T20:06:11.247 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.239+0000 7f0433fff640 1 -- 172.21.15.88:0/984292177 learned_addr learned my addr 172.21.15.88:0/984292177 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:11.247 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.239+0000 7f04337fe640 1 --2- 172.21.15.88:0/984292177 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0434153ea0 0x7f04340d1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:11.248 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.239+0000 7f0432ffd640 1 -- 172.21.15.88:0/984292177 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3909876742 0 0) 0x7f04340b5510 con 0x7f04340cd970 2023-10-16T20:06:11.248 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.239+0000 7f0432ffd640 1 -- 172.21.15.88:0/984292177 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0420003530 con 0x7f04340cd970 2023-10-16T20:06:11.248 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.239+0000 7f0432ffd640 1 -- 172.21.15.88:0/984292177 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 899113765 0 0) 0x7f043415d9a0 con 0x7f04340d2660 2023-10-16T20:06:11.249 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.239+0000 7f0432ffd640 1 -- 172.21.15.88:0/984292177 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04340b5510 con 0x7f04340d2660 2023-10-16T20:06:11.249 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.239+0000 7f04337fe640 1 -- 172.21.15.88:0/984292177 >> v1:172.21.15.94:6789/0 conn(0x7f04340d2660 legacy=0x7f043415d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:11.249 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.239+0000 7f04337fe640 1 -- 172.21.15.88:0/984292177 >> v1:172.21.15.88:6789/0 conn(0x7f04340cd970 legacy=0x7f04340cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:11.250 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.239+0000 7f04337fe640 1 -- 172.21.15.88:0/984292177 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04340b4c70 con 0x7f0434153ea0 2023-10-16T20:06:11.251 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.243+0000 7f04337fe640 1 --2- 172.21.15.88:0/984292177 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0434153ea0 0x7f04340d1d10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f04240097f0 tx=0x7f042401c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e192235cfc6489b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:11.251 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.243+0000 7f0432ffd640 1 -- 172.21.15.88:0/984292177 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04240aa5f0 con 0x7f0434153ea0 2023-10-16T20:06:11.251 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.243+0000 7f0432ffd640 1 -- 172.21.15.88:0/984292177 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04240aa750 con 0x7f0434153ea0 2023-10-16T20:06:11.252 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.243+0000 7f0432ffd640 1 -- 172.21.15.88:0/984292177 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04240aaae0 con 0x7f0434153ea0 2023-10-16T20:06:11.252 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.243+0000 7f0439fc4640 1 -- 172.21.15.88:0/984292177 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0434153ea0 msgr2=0x7f04340d1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:11.252 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.243+0000 7f0439fc4640 1 --2- 172.21.15.88:0/984292177 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0434153ea0 0x7f04340d1d10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f04240097f0 tx=0x7f042401c490 comp rx=0 tx=0).stop 2023-10-16T20:06:11.252 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.243+0000 7f0439fc4640 1 -- 172.21.15.88:0/984292177 shutdown_connections 2023-10-16T20:06:11.253 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.243+0000 7f0439fc4640 1 --2- 172.21.15.88:0/984292177 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0434153ea0 0x7f04340d1d10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:11.253 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.243+0000 7f0439fc4640 1 -- 172.21.15.88:0/984292177 >> 172.21.15.88:0/984292177 conn(0x7f04340715c0 msgr2=0x7f04340719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:11.253 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.247+0000 7f0439fc4640 1 -- 172.21.15.88:0/984292177 shutdown_connections 2023-10-16T20:06:11.253 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.247+0000 7f0439fc4640 1 -- 172.21.15.88:0/984292177 wait complete. 2023-10-16T20:06:11.255 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.247+0000 7f0439fc4640 1 Processor -- start 2023-10-16T20:06:11.256 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.247+0000 7f0439fc4640 1 -- start start 2023-10-16T20:06:11.256 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.247+0000 7f0439fc4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04340cd970 0x7f043414a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:11.256 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.247+0000 7f0439fc4640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f04340d2660 0x7f04340d33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:11.256 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.247+0000 7f0439fc4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0434153ea0 0x7f04340cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:11.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.247+0000 7f0439fc4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04340c8280 con 0x7f04340cd970 2023-10-16T20:06:11.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.247+0000 7f0439fc4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043414e690 con 0x7f0434153ea0 2023-10-16T20:06:11.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.247+0000 7f0439fc4640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04340d38e0 con 0x7f04340d2660 2023-10-16T20:06:11.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.247+0000 7f04337fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04340cd970 0x7f043414a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:11.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.247+0000 7f04337fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04340cd970 0x7f043414a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40160/0 (socket says 172.21.15.88:40160) 2023-10-16T20:06:11.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.247+0000 7f04337fe640 1 -- 172.21.15.88:0/3823659825 learned_addr learned my addr 172.21.15.88:0/3823659825 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:11.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.247+0000 7f0433fff640 1 --2- 172.21.15.88:0/3823659825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0434153ea0 0x7f04340cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:11.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.247+0000 7f042bfff640 1 --2- 172.21.15.88:0/3823659825 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f04340d2660 0x7f04340d33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:11.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.247+0000 7f04337fe640 1 -- 172.21.15.88:0/3823659825 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f04340d2660 msgr2=0x7f04340d33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:11.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.247+0000 7f04337fe640 1 --2- 172.21.15.88:0/3823659825 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f04340d2660 0x7f04340d33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:11.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.247+0000 7f04337fe640 1 -- 172.21.15.88:0/3823659825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0434153ea0 msgr2=0x7f04340cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:11.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.247+0000 7f04337fe640 1 --2- 172.21.15.88:0/3823659825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0434153ea0 0x7f04340cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:11.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.247+0000 7f04337fe640 1 -- 172.21.15.88:0/3823659825 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0424009420 con 0x7f04340cd970 2023-10-16T20:06:11.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.247+0000 7f04337fe640 1 --2- 172.21.15.88:0/3823659825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04340cd970 0x7f043414a800 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f042401a0c0 tx=0x7f0424019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:11.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.251+0000 7f0430ff9640 1 -- 172.21.15.88:0/3823659825 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04240aad90 con 0x7f04340cd970 2023-10-16T20:06:11.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.251+0000 7f0430ff9640 1 -- 172.21.15.88:0/3823659825 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04240be450 con 0x7f04340cd970 2023-10-16T20:06:11.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.251+0000 7f0430ff9640 1 -- 172.21.15.88:0/3823659825 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04240be800 con 0x7f04340cd970 2023-10-16T20:06:11.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.251+0000 7f0439fc4640 1 -- 172.21.15.88:0/3823659825 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f043414e0d0 con 0x7f04340cd970 2023-10-16T20:06:11.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.251+0000 7f0439fc4640 1 -- 172.21.15.88:0/3823659825 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0434149560 con 0x7f04340cd970 2023-10-16T20:06:11.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.251+0000 7f0439fc4640 1 -- 172.21.15.88:0/3823659825 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f043404e9c0 con 0x7f04340cd970 2023-10-16T20:06:11.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.251+0000 7f0430ff9640 1 -- 172.21.15.88:0/3823659825 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f04240be960 con 0x7f04340cd970 2023-10-16T20:06:11.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.251+0000 7f0430ff9640 1 --2- 172.21.15.88:0/3823659825 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f04140664a0 0x7f0414068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:11.263 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.251+0000 7f0430ff9640 1 -- 172.21.15.88:0/3823659825 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f04240a7070 con 0x7f04340cd970 2023-10-16T20:06:11.263 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.251+0000 7f042bfff640 1 --2- 172.21.15.88:0/3823659825 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f04140664a0 0x7f0414068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:11.263 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.251+0000 7f042bfff640 1 --2- 172.21.15.88:0/3823659825 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f04140664a0 0x7f0414068960 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f04340b58e0 tx=0x7f041c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:11.263 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.255+0000 7f0430ff9640 1 -- 172.21.15.88:0/3823659825 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f04240eb910 con 0x7f04340cd970 2023-10-16T20:06:11.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.459+0000 7f0439fc4640 1 -- 172.21.15.88:0/3823659825 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f0434168a30 con 0x7f04340cd970 2023-10-16T20:06:11.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.463+0000 7f0430ff9640 1 -- 172.21.15.88:0/3823659825 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f04240eef30 con 0x7f04340cd970 2023-10-16T20:06:11.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.467+0000 7f0439fc4640 1 -- 172.21.15.88:0/3823659825 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f04140664a0 msgr2=0x7f0414068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:11.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.467+0000 7f0439fc4640 1 --2- 172.21.15.88:0/3823659825 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f04140664a0 0x7f0414068960 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f04340b58e0 tx=0x7f041c009040 comp rx=0 tx=0).stop 2023-10-16T20:06:11.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.467+0000 7f0439fc4640 1 -- 172.21.15.88:0/3823659825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04340cd970 msgr2=0x7f043414a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:11.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.467+0000 7f0439fc4640 1 --2- 172.21.15.88:0/3823659825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04340cd970 0x7f043414a800 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f042401a0c0 tx=0x7f0424019230 comp rx=0 tx=0).stop 2023-10-16T20:06:11.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.467+0000 7f0439fc4640 1 -- 172.21.15.88:0/3823659825 shutdown_connections 2023-10-16T20:06:11.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.467+0000 7f0439fc4640 1 --2- 172.21.15.88:0/3823659825 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f04140664a0 0x7f0414068960 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:11.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.467+0000 7f0439fc4640 1 --2- 172.21.15.88:0/3823659825 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f04340d2660 0x7f04340d33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:11.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.467+0000 7f0439fc4640 1 --2- 172.21.15.88:0/3823659825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0434153ea0 0x7f04340cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:11.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.467+0000 7f0439fc4640 1 --2- 172.21.15.88:0/3823659825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04340cd970 0x7f043414a800 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:11.475 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.467+0000 7f0439fc4640 1 -- 172.21.15.88:0/3823659825 >> 172.21.15.88:0/3823659825 conn(0x7f04340715c0 msgr2=0x7f0434072790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:11.475 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.467+0000 7f0439fc4640 1 -- 172.21.15.88:0/3823659825 shutdown_connections 2023-10-16T20:06:11.475 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.467+0000 7f0439fc4640 1 -- 172.21.15.88:0/3823659825 wait complete. 2023-10-16T20:06:11.492 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-16T20:06:11.630 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.623+0000 7ff08430f640 1 Processor -- start 2023-10-16T20:06:11.631 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.623+0000 7ff08430f640 1 -- start start 2023-10-16T20:06:11.631 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.627+0000 7ff08430f640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff07c0adf80 0x7ff07c15a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:11.632 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.627+0000 7ff08430f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff07c15d480 0x7ff07c0afa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:11.632 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.627+0000 7ff08430f640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff07c15e400 con 0x7ff07c069e90 2023-10-16T20:06:11.632 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.627+0000 7ff08430f640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff07c159e40 con 0x7ff07c0adf80 2023-10-16T20:06:11.632 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.627+0000 7ff08430f640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c15de40 con 0x7ff07c15d480 2023-10-16T20:06:11.633 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.627+0000 7ff0828ac640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff07c15d480 0x7ff07c0afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:11.634 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.627+0000 7ff0818aa640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff07c0adf80 0x7ff07c15a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:11.634 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.627+0000 7ff0820ab640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7ff07c069e90 0x7ff07c06a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:57218/0 (socket says 172.21.15.88:57218) 2023-10-16T20:06:11.634 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.627+0000 7ff0828ac640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff07c15d480 0x7ff07c0afa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:41560/0 (socket says 172.21.15.88:41560) 2023-10-16T20:06:11.634 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.627+0000 7ff0820ab640 1 -- 172.21.15.88:0/980859882 learned_addr learned my addr 172.21.15.88:0/980859882 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:11.634 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.627+0000 7ff0810a9640 1 -- 172.21.15.88:0/980859882 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1773334589 0 0) 0x7ff07c15e400 con 0x7ff07c069e90 2023-10-16T20:06:11.635 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.627+0000 7ff0828ac640 1 -- 172.21.15.88:0/980859882 >> v2:172.21.15.94:3300/0 conn(0x7ff07c0adf80 msgr2=0x7ff07c15a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:11.635 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.627+0000 7ff0828ac640 1 --2- 172.21.15.88:0/980859882 >> v2:172.21.15.94:3300/0 conn(0x7ff07c0adf80 0x7ff07c15a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:11.635 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.627+0000 7ff0828ac640 1 -- 172.21.15.88:0/980859882 >> v1:172.21.15.94:6789/0 conn(0x7ff07c069e90 legacy=0x7ff07c06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:11.635 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.627+0000 7ff0828ac640 1 -- 172.21.15.88:0/980859882 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff07c15e120 con 0x7ff07c15d480 2023-10-16T20:06:11.636 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.627+0000 7ff0818aa640 1 --2- 172.21.15.88:0/980859882 >> v2:172.21.15.94:3300/0 conn(0x7ff07c0adf80 0x7ff07c15a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:06:11.637 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.631+0000 7ff0828ac640 1 --2- 172.21.15.88:0/980859882 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff07c15d480 0x7ff07c0afa20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff074009d90 tx=0x7ff07401c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64f3450daaa98e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:11.637 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.631+0000 7ff0810a9640 1 -- 172.21.15.88:0/980859882 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0740a59e0 con 0x7ff07c15d480 2023-10-16T20:06:11.637 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.631+0000 7ff0810a9640 1 -- 172.21.15.88:0/980859882 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0740a5b40 con 0x7ff07c15d480 2023-10-16T20:06:11.637 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.631+0000 7ff0810a9640 1 -- 172.21.15.88:0/980859882 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0740ad440 con 0x7ff07c15d480 2023-10-16T20:06:11.638 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.631+0000 7ff08430f640 1 -- 172.21.15.88:0/980859882 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff07c15d480 msgr2=0x7ff07c0afa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:11.638 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.631+0000 7ff08430f640 1 --2- 172.21.15.88:0/980859882 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff07c15d480 0x7ff07c0afa20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff074009d90 tx=0x7ff07401c920 comp rx=0 tx=0).stop 2023-10-16T20:06:11.638 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.631+0000 7ff08430f640 1 -- 172.21.15.88:0/980859882 shutdown_connections 2023-10-16T20:06:11.638 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.631+0000 7ff08430f640 1 --2- 172.21.15.88:0/980859882 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff07c15d480 0x7ff07c0afa20 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:11.638 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.631+0000 7ff08430f640 1 --2- 172.21.15.88:0/980859882 >> v2:172.21.15.94:3300/0 conn(0x7ff07c0adf80 0x7ff07c15a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:11.639 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.631+0000 7ff08430f640 1 -- 172.21.15.88:0/980859882 >> 172.21.15.88:0/980859882 conn(0x7ff07c0ce5e0 msgr2=0x7ff07c0ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:11.639 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.631+0000 7ff08430f640 1 -- 172.21.15.88:0/980859882 shutdown_connections 2023-10-16T20:06:11.639 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.631+0000 7ff08430f640 1 -- 172.21.15.88:0/980859882 wait complete. 2023-10-16T20:06:11.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.631+0000 7ff08430f640 1 Processor -- start 2023-10-16T20:06:11.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.631+0000 7ff08430f640 1 -- start start 2023-10-16T20:06:11.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.635+0000 7ff08430f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff07c069e90 0x7ff07c156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:11.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.635+0000 7ff08430f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff07c0adf80 0x7ff07c1439d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:11.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.635+0000 7ff08430f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff07c15d480 0x7ff07c0ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:11.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.635+0000 7ff08430f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c152900 con 0x7ff07c069e90 2023-10-16T20:06:11.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.635+0000 7ff08430f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c151fd0 con 0x7ff07c0adf80 2023-10-16T20:06:11.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.635+0000 7ff08430f640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c1522b0 con 0x7ff07c15d480 2023-10-16T20:06:11.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.635+0000 7ff0820ab640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff07c069e90 0x7ff07c156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:11.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.635+0000 7ff0820ab640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff07c069e90 0x7ff07c156990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40164/0 (socket says 172.21.15.88:40164) 2023-10-16T20:06:11.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.635+0000 7ff0820ab640 1 -- 172.21.15.88:0/4080168058 learned_addr learned my addr 172.21.15.88:0/4080168058 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:11.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.635+0000 7ff0828ac640 1 --2- 172.21.15.88:0/4080168058 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff07c15d480 0x7ff07c0ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:11.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.635+0000 7ff0818aa640 1 --2- 172.21.15.88:0/4080168058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff07c0adf80 0x7ff07c1439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:11.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.635+0000 7ff0820ab640 1 -- 172.21.15.88:0/4080168058 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff07c15d480 msgr2=0x7ff07c0ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:11.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.635+0000 7ff0820ab640 1 --2- 172.21.15.88:0/4080168058 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff07c15d480 0x7ff07c0ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:11.646 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.635+0000 7ff0820ab640 1 -- 172.21.15.88:0/4080168058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff07c0adf80 msgr2=0x7ff07c1439d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:11.646 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.635+0000 7ff0820ab640 1 --2- 172.21.15.88:0/4080168058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff07c0adf80 0x7ff07c1439d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:11.646 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.635+0000 7ff0820ab640 1 -- 172.21.15.88:0/4080168058 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff06c009420 con 0x7ff07c069e90 2023-10-16T20:06:11.646 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.635+0000 7ff0820ab640 1 --2- 172.21.15.88:0/4080168058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff07c069e90 0x7ff07c156990 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff0780011f0 tx=0x7ff07800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:11.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.635+0000 7ff072ffd640 1 -- 172.21.15.88:0/4080168058 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0780074e0 con 0x7ff07c069e90 2023-10-16T20:06:11.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.635+0000 7ff072ffd640 1 -- 172.21.15.88:0/4080168058 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff078007640 con 0x7ff07c069e90 2023-10-16T20:06:11.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.635+0000 7ff08430f640 1 -- 172.21.15.88:0/4080168058 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff074009a70 con 0x7ff07c069e90 2023-10-16T20:06:11.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.635+0000 7ff072ffd640 1 -- 172.21.15.88:0/4080168058 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff078004a10 con 0x7ff07c069e90 2023-10-16T20:06:11.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.635+0000 7ff08430f640 1 -- 172.21.15.88:0/4080168058 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff07c14f710 con 0x7ff07c069e90 2023-10-16T20:06:11.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.635+0000 7ff08430f640 1 -- 172.21.15.88:0/4080168058 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff07c06d180 con 0x7ff07c069e90 2023-10-16T20:06:11.648 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.639+0000 7ff072ffd640 1 -- 172.21.15.88:0/4080168058 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff078004b70 con 0x7ff07c069e90 2023-10-16T20:06:11.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.639+0000 7ff072ffd640 1 --2- 172.21.15.88:0/4080168058 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff0600664a0 0x7ff060068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:11.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.639+0000 7ff072ffd640 1 -- 172.21.15.88:0/4080168058 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7ff0780830c0 con 0x7ff07c069e90 2023-10-16T20:06:11.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.639+0000 7ff0818aa640 1 --2- 172.21.15.88:0/4080168058 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff0600664a0 0x7ff060068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:11.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.639+0000 7ff0818aa640 1 --2- 172.21.15.88:0/4080168058 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff0600664a0 0x7ff060068960 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff07c144850 tx=0x7ff06c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:11.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.643+0000 7ff072ffd640 1 -- 172.21.15.88:0/4080168058 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff078050890 con 0x7ff07c069e90 2023-10-16T20:06:11.852 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.847+0000 7ff08430f640 1 -- 172.21.15.88:0/4080168058 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7ff07c04e9c0 con 0x7ff07c069e90 2023-10-16T20:06:11.854 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.847+0000 7ff072ffd640 1 -- 172.21.15.88:0/4080168058 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7ff078053eb0 con 0x7ff07c069e90 2023-10-16T20:06:11.857 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.851+0000 7ff08430f640 1 -- 172.21.15.88:0/4080168058 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff0600664a0 msgr2=0x7ff060068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:11.857 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.851+0000 7ff08430f640 1 --2- 172.21.15.88:0/4080168058 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff0600664a0 0x7ff060068960 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff07c144850 tx=0x7ff06c009190 comp rx=0 tx=0).stop 2023-10-16T20:06:11.857 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.851+0000 7ff08430f640 1 -- 172.21.15.88:0/4080168058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff07c069e90 msgr2=0x7ff07c156990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:11.857 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.851+0000 7ff08430f640 1 --2- 172.21.15.88:0/4080168058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff07c069e90 0x7ff07c156990 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff0780011f0 tx=0x7ff07800c9b0 comp rx=0 tx=0).stop 2023-10-16T20:06:11.858 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.851+0000 7ff08430f640 1 -- 172.21.15.88:0/4080168058 shutdown_connections 2023-10-16T20:06:11.858 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.851+0000 7ff08430f640 1 --2- 172.21.15.88:0/4080168058 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff0600664a0 0x7ff060068960 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:11.858 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.851+0000 7ff08430f640 1 --2- 172.21.15.88:0/4080168058 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff07c15d480 0x7ff07c0ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:11.858 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.851+0000 7ff08430f640 1 --2- 172.21.15.88:0/4080168058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff07c0adf80 0x7ff07c1439d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:11.859 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.851+0000 7ff08430f640 1 --2- 172.21.15.88:0/4080168058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff07c069e90 0x7ff07c156990 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:11.859 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.851+0000 7ff08430f640 1 -- 172.21.15.88:0/4080168058 >> 172.21.15.88:0/4080168058 conn(0x7ff07c0ce5e0 msgr2=0x7ff07c0cc720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:11.859 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.851+0000 7ff08430f640 1 -- 172.21.15.88:0/4080168058 shutdown_connections 2023-10-16T20:06:11.859 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:11.851+0000 7ff08430f640 1 -- 172.21.15.88:0/4080168058 wait complete. 2023-10-16T20:06:11.876 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-16T20:06:12.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.011+0000 7f12dd7d9640 1 Processor -- start 2023-10-16T20:06:12.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.011+0000 7f12dd7d9640 1 -- start start 2023-10-16T20:06:12.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.011+0000 7f12dd7d9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f12d80d2660 0x7f12d80d1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:12.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.011+0000 7f12dd7d9640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f12d80cd970 0x7f12d80cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:12.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.011+0000 7f12dd7d9640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d80b5510 con 0x7f12d8153ea0 2023-10-16T20:06:12.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.011+0000 7f12dd7d9640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12d815d9a0 con 0x7f12d80cd970 2023-10-16T20:06:12.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.011+0000 7f12dd7d9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d80ce320 con 0x7f12d80d2660 2023-10-16T20:06:12.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.011+0000 7f12d6ffd640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f12d8153ea0 0x7f12d815d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:36754/0 (socket says 172.21.15.88:36754) 2023-10-16T20:06:12.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.011+0000 7f12d6ffd640 1 -- 172.21.15.88:0/3527434591 learned_addr learned my addr 172.21.15.88:0/3527434591 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:12.019 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.011+0000 7f12d77fe640 1 --2- 172.21.15.88:0/3527434591 >> v2:172.21.15.88:3300/0 conn(0x7f12d80cd970 0x7f12d80cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:12.019 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.011+0000 7f12d67fc640 1 --2- 172.21.15.88:0/3527434591 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f12d80d2660 0x7f12d80d1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:12.019 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.011+0000 7f12d5ffb640 1 -- 172.21.15.88:0/3527434591 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3563989994 0 0) 0x7f12d80b5510 con 0x7f12d8153ea0 2023-10-16T20:06:12.019 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.011+0000 7f12d5ffb640 1 -- 172.21.15.88:0/3527434591 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12c0003530 con 0x7f12d8153ea0 2023-10-16T20:06:12.019 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.011+0000 7f12d77fe640 1 -- 172.21.15.88:0/3527434591 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f12d80d2660 msgr2=0x7f12d80d1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:12.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.011+0000 7f12d77fe640 1 --2- 172.21.15.88:0/3527434591 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f12d80d2660 0x7f12d80d1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:12.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.011+0000 7f12d77fe640 1 -- 172.21.15.88:0/3527434591 >> v1:172.21.15.88:6789/0 conn(0x7f12d8153ea0 legacy=0x7f12d815d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:12.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.011+0000 7f12d77fe640 1 -- 172.21.15.88:0/3527434591 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d80b4990 con 0x7f12d80cd970 2023-10-16T20:06:12.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.015+0000 7f12d77fe640 1 --2- 172.21.15.88:0/3527434591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12d80cd970 0x7f12d80cdde0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009d90 tx=0x7f12d001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788485452682da75 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:12.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.015+0000 7f12d5ffb640 1 -- 172.21.15.88:0/3527434591 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d00a59e0 con 0x7f12d80cd970 2023-10-16T20:06:12.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.015+0000 7f12d5ffb640 1 -- 172.21.15.88:0/3527434591 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d00a5b40 con 0x7f12d80cd970 2023-10-16T20:06:12.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.015+0000 7f12d5ffb640 1 -- 172.21.15.88:0/3527434591 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d00ad440 con 0x7f12d80cd970 2023-10-16T20:06:12.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.015+0000 7f12dd7d9640 1 -- 172.21.15.88:0/3527434591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12d80cd970 msgr2=0x7f12d80cdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:12.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.015+0000 7f12dd7d9640 1 --2- 172.21.15.88:0/3527434591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12d80cd970 0x7f12d80cdde0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009d90 tx=0x7f12d001c920 comp rx=0 tx=0).stop 2023-10-16T20:06:12.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.015+0000 7f12dd7d9640 1 -- 172.21.15.88:0/3527434591 shutdown_connections 2023-10-16T20:06:12.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.015+0000 7f12dd7d9640 1 --2- 172.21.15.88:0/3527434591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12d80cd970 0x7f12d80cdde0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:12.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.015+0000 7f12dd7d9640 1 --2- 172.21.15.88:0/3527434591 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f12d80d2660 0x7f12d80d1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:12.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.015+0000 7f12dd7d9640 1 -- 172.21.15.88:0/3527434591 >> 172.21.15.88:0/3527434591 conn(0x7f12d80715c0 msgr2=0x7f12d80719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:12.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.015+0000 7f12dd7d9640 1 -- 172.21.15.88:0/3527434591 shutdown_connections 2023-10-16T20:06:12.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.015+0000 7f12dd7d9640 1 -- 172.21.15.88:0/3527434591 wait complete. 2023-10-16T20:06:12.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.019+0000 7f12dd7d9640 1 Processor -- start 2023-10-16T20:06:12.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.019+0000 7f12dd7d9640 1 -- start start 2023-10-16T20:06:12.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.019+0000 7f12dd7d9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12d80cd970 0x7f12d80cb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:12.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.019+0000 7f12dd7d9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12d80d2660 0x7f12d80cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:12.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.019+0000 7f12dd7d9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f12d8153ea0 0x7f12d81528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:12.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.019+0000 7f12dd7d9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d8150950 con 0x7f12d80d2660 2023-10-16T20:06:12.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.019+0000 7f12dd7d9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d80cbeb0 con 0x7f12d80cd970 2023-10-16T20:06:12.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.019+0000 7f12dd7d9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d80cc190 con 0x7f12d8153ea0 2023-10-16T20:06:12.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.019+0000 7f12d67fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12d80d2660 0x7f12d80cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:12.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.019+0000 7f12d67fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12d80d2660 0x7f12d80cb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45690/0 (socket says 172.21.15.88:45690) 2023-10-16T20:06:12.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.019+0000 7f12d67fc640 1 -- 172.21.15.88:0/1260994643 learned_addr learned my addr 172.21.15.88:0/1260994643 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:12.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.019+0000 7f12d67fc640 1 -- 172.21.15.88:0/1260994643 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f12d8153ea0 msgr2=0x7f12d81528a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:12.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.019+0000 7f12d67fc640 1 --2- 172.21.15.88:0/1260994643 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f12d8153ea0 0x7f12d81528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:12.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.019+0000 7f12d67fc640 1 -- 172.21.15.88:0/1260994643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12d80cd970 msgr2=0x7f12d80cb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:12.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.019+0000 7f12d67fc640 1 --2- 172.21.15.88:0/1260994643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12d80cd970 0x7f12d80cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:12.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.019+0000 7f12d67fc640 1 -- 172.21.15.88:0/1260994643 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d0009a70 con 0x7f12d80d2660 2023-10-16T20:06:12.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.019+0000 7f12d67fc640 1 --2- 172.21.15.88:0/1260994643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12d80d2660 0x7f12d80cb850 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f12cc00d800 tx=0x7f12cc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:12.029 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.019+0000 7f12bf7fe640 1 -- 172.21.15.88:0/1260994643 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc00b6c0 con 0x7f12d80d2660 2023-10-16T20:06:12.029 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.019+0000 7f12bf7fe640 1 -- 172.21.15.88:0/1260994643 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12cc00b820 con 0x7f12d80d2660 2023-10-16T20:06:12.029 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.019+0000 7f12dd7d9640 1 -- 172.21.15.88:0/1260994643 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d80cc470 con 0x7f12d80d2660 2023-10-16T20:06:12.029 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.019+0000 7f12bf7fe640 1 -- 172.21.15.88:0/1260994643 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc00bb90 con 0x7f12d80d2660 2023-10-16T20:06:12.029 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.019+0000 7f12dd7d9640 1 -- 172.21.15.88:0/1260994643 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d80d1eb0 con 0x7f12d80d2660 2023-10-16T20:06:12.030 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.019+0000 7f12dd7d9640 1 -- 172.21.15.88:0/1260994643 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d8150ac0 con 0x7f12d80d2660 2023-10-16T20:06:12.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.023+0000 7f12bf7fe640 1 -- 172.21.15.88:0/1260994643 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f12cc00bcf0 con 0x7f12d80d2660 2023-10-16T20:06:12.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.023+0000 7f12bf7fe640 1 --2- 172.21.15.88:0/1260994643 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f12c40664a0 0x7f12c4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:12.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.023+0000 7f12bf7fe640 1 -- 172.21.15.88:0/1260994643 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f12cc084210 con 0x7f12d80d2660 2023-10-16T20:06:12.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.023+0000 7f12d6ffd640 1 --2- 172.21.15.88:0/1260994643 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f12c40664a0 0x7f12c4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:12.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.023+0000 7f12d6ffd640 1 --2- 172.21.15.88:0/1260994643 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f12c40664a0 0x7f12c4068960 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f12c8007fd0 tx=0x7f12c8007940 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:12.032 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.027+0000 7f12bf7fe640 1 -- 172.21.15.88:0/1260994643 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f12d8150ac0 con 0x7f12d80d2660 2023-10-16T20:06:12.232 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.227+0000 7f12dd7d9640 1 -- 172.21.15.88:0/1260994643 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f12d8150a90 con 0x7f12d80d2660 2023-10-16T20:06:12.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.231+0000 7f12bf7fe640 1 -- 172.21.15.88:0/1260994643 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f12cc051890 con 0x7f12d80d2660 2023-10-16T20:06:12.237 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.231+0000 7f12dd7d9640 1 -- 172.21.15.88:0/1260994643 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f12c40664a0 msgr2=0x7f12c4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:12.237 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.231+0000 7f12dd7d9640 1 --2- 172.21.15.88:0/1260994643 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f12c40664a0 0x7f12c4068960 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f12c8007fd0 tx=0x7f12c8007940 comp rx=0 tx=0).stop 2023-10-16T20:06:12.237 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.231+0000 7f12dd7d9640 1 -- 172.21.15.88:0/1260994643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12d80d2660 msgr2=0x7f12d80cb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:12.238 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.231+0000 7f12dd7d9640 1 --2- 172.21.15.88:0/1260994643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12d80d2660 0x7f12d80cb850 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f12cc00d800 tx=0x7f12cc00dcc0 comp rx=0 tx=0).stop 2023-10-16T20:06:12.238 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.231+0000 7f12dd7d9640 1 -- 172.21.15.88:0/1260994643 shutdown_connections 2023-10-16T20:06:12.238 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.231+0000 7f12dd7d9640 1 --2- 172.21.15.88:0/1260994643 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f12c40664a0 0x7f12c4068960 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:12.238 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.231+0000 7f12dd7d9640 1 --2- 172.21.15.88:0/1260994643 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f12d8153ea0 0x7f12d81528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:12.238 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.231+0000 7f12dd7d9640 1 --2- 172.21.15.88:0/1260994643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12d80d2660 0x7f12d80cb850 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:12.239 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.231+0000 7f12dd7d9640 1 --2- 172.21.15.88:0/1260994643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12d80cd970 0x7f12d80cb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:12.239 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.231+0000 7f12dd7d9640 1 -- 172.21.15.88:0/1260994643 >> 172.21.15.88:0/1260994643 conn(0x7f12d80715c0 msgr2=0x7f12d80719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:12.239 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.231+0000 7f12dd7d9640 1 -- 172.21.15.88:0/1260994643 shutdown_connections 2023-10-16T20:06:12.239 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.231+0000 7f12dd7d9640 1 -- 172.21.15.88:0/1260994643 wait complete. 2023-10-16T20:06:12.256 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-16T20:06:12.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.387+0000 7f1e4df0f640 1 Processor -- start 2023-10-16T20:06:12.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.391+0000 7f1e4df0f640 1 -- start start 2023-10-16T20:06:12.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.391+0000 7f1e4df0f640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1e48161310 0x7f1e4806aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:12.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.391+0000 7f1e4df0f640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1e481616f0 0x7f1e4806b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:12.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.391+0000 7f1e4df0f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e4806c370 0x7f1e4806c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:12.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.391+0000 7f1e4df0f640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e48071de0 con 0x7f1e48161310 2023-10-16T20:06:12.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.391+0000 7f1e4df0f640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e4806e090 con 0x7f1e481616f0 2023-10-16T20:06:12.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.391+0000 7f1e4df0f640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e480cda50 con 0x7f1e4806c370 2023-10-16T20:06:12.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.391+0000 7f1e477fe640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1e48161310 0x7f1e4806aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:12.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.391+0000 7f1e477fe640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1e48161310 0x7f1e4806aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45692/0 (socket says 172.21.15.88:45692) 2023-10-16T20:06:12.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.391+0000 7f1e477fe640 1 -- 172.21.15.88:0/2343272275 learned_addr learned my addr 172.21.15.88:0/2343272275 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:12.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.391+0000 7f1e46ffd640 1 --2- 172.21.15.88:0/2343272275 >> v2:172.21.15.94:3300/0 conn(0x7f1e481616f0 0x7f1e4806b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:12.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.391+0000 7f1e47fff640 1 --2- 172.21.15.88:0/2343272275 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e4806c370 0x7f1e4806c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:12.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.391+0000 7f1e477fe640 1 -- 172.21.15.88:0/2343272275 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e4806c370 msgr2=0x7f1e4806c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:12.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.391+0000 7f1e477fe640 1 --2- 172.21.15.88:0/2343272275 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e4806c370 0x7f1e4806c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:12.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.391+0000 7f1e477fe640 1 -- 172.21.15.88:0/2343272275 >> v2:172.21.15.94:3300/0 conn(0x7f1e481616f0 msgr2=0x7f1e4806b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:12.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.391+0000 7f1e477fe640 1 --2- 172.21.15.88:0/2343272275 >> v2:172.21.15.94:3300/0 conn(0x7f1e481616f0 0x7f1e4806b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:12.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.391+0000 7f1e477fe640 1 -- 172.21.15.88:0/2343272275 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e48071760 con 0x7f1e48161310 2023-10-16T20:06:12.401 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.395+0000 7f1e477fe640 1 --2- 172.21.15.88:0/2343272275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e48161310 0x7f1e4806aea0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1e480ce930 tx=0x7f1e3801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d689b715fc829ae9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:12.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.395+0000 7f1e467fc640 1 -- 172.21.15.88:0/2343272275 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e380aa5f0 con 0x7f1e48161310 2023-10-16T20:06:12.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.395+0000 7f1e467fc640 1 -- 172.21.15.88:0/2343272275 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e380aa750 con 0x7f1e48161310 2023-10-16T20:06:12.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.395+0000 7f1e467fc640 1 -- 172.21.15.88:0/2343272275 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e380aaae0 con 0x7f1e48161310 2023-10-16T20:06:12.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.395+0000 7f1e4df0f640 1 -- 172.21.15.88:0/2343272275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e48161310 msgr2=0x7f1e4806aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:12.403 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.395+0000 7f1e4df0f640 1 --2- 172.21.15.88:0/2343272275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e48161310 0x7f1e4806aea0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1e480ce930 tx=0x7f1e3801c4d0 comp rx=0 tx=0).stop 2023-10-16T20:06:12.403 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.395+0000 7f1e4df0f640 1 -- 172.21.15.88:0/2343272275 shutdown_connections 2023-10-16T20:06:12.403 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.395+0000 7f1e4df0f640 1 --2- 172.21.15.88:0/2343272275 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e4806c370 0x7f1e4806c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:12.403 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.395+0000 7f1e4df0f640 1 --2- 172.21.15.88:0/2343272275 >> v2:172.21.15.94:3300/0 conn(0x7f1e481616f0 0x7f1e4806b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:12.404 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.395+0000 7f1e4df0f640 1 --2- 172.21.15.88:0/2343272275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e48161310 0x7f1e4806aea0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:12.404 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.395+0000 7f1e4df0f640 1 -- 172.21.15.88:0/2343272275 >> 172.21.15.88:0/2343272275 conn(0x7f1e48070aa0 msgr2=0x7f1e48070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:12.404 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.395+0000 7f1e4df0f640 1 -- 172.21.15.88:0/2343272275 shutdown_connections 2023-10-16T20:06:12.404 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.395+0000 7f1e4df0f640 1 -- 172.21.15.88:0/2343272275 wait complete. 2023-10-16T20:06:12.406 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.395+0000 7f1e4df0f640 1 Processor -- start 2023-10-16T20:06:12.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.395+0000 7f1e4df0f640 1 -- start start 2023-10-16T20:06:12.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.399+0000 7f1e4df0f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e4806c370 0x7f1e48153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:12.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.399+0000 7f1e4df0f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e48161310 0x7f1e480c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:12.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.399+0000 7f1e4df0f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e481616f0 0x7f1e480c6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:12.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.399+0000 7f1e4df0f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4815f280 con 0x7f1e48161310 2023-10-16T20:06:12.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.399+0000 7f1e4df0f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e480c8db0 con 0x7f1e481616f0 2023-10-16T20:06:12.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.399+0000 7f1e4df0f640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e480c8ad0 con 0x7f1e4806c370 2023-10-16T20:06:12.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.399+0000 7f1e46ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e48161310 0x7f1e480c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:12.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.399+0000 7f1e46ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e48161310 0x7f1e480c81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45700/0 (socket says 172.21.15.88:45700) 2023-10-16T20:06:12.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.399+0000 7f1e46ffd640 1 -- 172.21.15.88:0/3835427889 learned_addr learned my addr 172.21.15.88:0/3835427889 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:12.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.399+0000 7f1e477fe640 1 --2- 172.21.15.88:0/3835427889 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e4806c370 0x7f1e48153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:12.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.399+0000 7f1e47fff640 1 --2- 172.21.15.88:0/3835427889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e481616f0 0x7f1e480c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:12.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.399+0000 7f1e46ffd640 1 -- 172.21.15.88:0/3835427889 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e4806c370 msgr2=0x7f1e48153ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:12.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.399+0000 7f1e46ffd640 1 --2- 172.21.15.88:0/3835427889 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e4806c370 0x7f1e48153ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:12.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.399+0000 7f1e46ffd640 1 -- 172.21.15.88:0/3835427889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e481616f0 msgr2=0x7f1e480c6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:12.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.399+0000 7f1e46ffd640 1 --2- 172.21.15.88:0/3835427889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e481616f0 0x7f1e480c6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:12.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.399+0000 7f1e46ffd640 1 -- 172.21.15.88:0/3835427889 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e380094d0 con 0x7f1e48161310 2023-10-16T20:06:12.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.399+0000 7f1e47fff640 1 --2- 172.21.15.88:0/3835427889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e481616f0 0x7f1e480c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:06:12.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.399+0000 7f1e46ffd640 1 --2- 172.21.15.88:0/3835427889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e48161310 0x7f1e480c81a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00d7d0 tx=0x7f1e3c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:12.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.399+0000 7f1e2ffff640 1 -- 172.21.15.88:0/3835427889 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c00b6c0 con 0x7f1e48161310 2023-10-16T20:06:12.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.399+0000 7f1e2ffff640 1 -- 172.21.15.88:0/3835427889 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3c00b820 con 0x7f1e48161310 2023-10-16T20:06:12.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.399+0000 7f1e2ffff640 1 -- 172.21.15.88:0/3835427889 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c00bbd0 con 0x7f1e48161310 2023-10-16T20:06:12.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.399+0000 7f1e4df0f640 1 -- 172.21.15.88:0/3835427889 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e48066180 con 0x7f1e48161310 2023-10-16T20:06:12.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.399+0000 7f1e4df0f640 1 -- 172.21.15.88:0/3835427889 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e4815ef10 con 0x7f1e48161310 2023-10-16T20:06:12.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.399+0000 7f1e4df0f640 1 -- 172.21.15.88:0/3835427889 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e4815f430 con 0x7f1e48161310 2023-10-16T20:06:12.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.403+0000 7f1e2ffff640 1 -- 172.21.15.88:0/3835427889 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1e3c00bd30 con 0x7f1e48161310 2023-10-16T20:06:12.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.403+0000 7f1e2ffff640 1 --2- 172.21.15.88:0/3835427889 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1e340664d0 0x7f1e34068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:12.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.403+0000 7f1e2ffff640 1 -- 172.21.15.88:0/3835427889 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f1e3c0841d0 con 0x7f1e48161310 2023-10-16T20:06:12.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.403+0000 7f1e477fe640 1 --2- 172.21.15.88:0/3835427889 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1e340664d0 0x7f1e34068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:12.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.403+0000 7f1e477fe640 1 --2- 172.21.15.88:0/3835427889 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1e340664d0 0x7f1e34068990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1e380ae000 tx=0x7f1e380a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:12.414 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.407+0000 7f1e2ffff640 1 -- 172.21.15.88:0/3835427889 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1e4815f430 con 0x7f1e48161310 2023-10-16T20:06:12.613 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.607+0000 7f1e4df0f640 1 -- 172.21.15.88:0/3835427889 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f1e4815f0a0 con 0x7f1e48161310 2023-10-16T20:06:12.616 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.611+0000 7f1e2ffff640 1 -- 172.21.15.88:0/3835427889 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f1e3c0517d0 con 0x7f1e48161310 2023-10-16T20:06:12.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.611+0000 7f1e4df0f640 1 -- 172.21.15.88:0/3835427889 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1e340664d0 msgr2=0x7f1e34068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:12.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.611+0000 7f1e4df0f640 1 --2- 172.21.15.88:0/3835427889 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1e340664d0 0x7f1e34068990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1e380ae000 tx=0x7f1e380a7000 comp rx=0 tx=0).stop 2023-10-16T20:06:12.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.611+0000 7f1e4df0f640 1 -- 172.21.15.88:0/3835427889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e48161310 msgr2=0x7f1e480c81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:12.620 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.611+0000 7f1e4df0f640 1 --2- 172.21.15.88:0/3835427889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e48161310 0x7f1e480c81a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00d7d0 tx=0x7f1e3c00dc90 comp rx=0 tx=0).stop 2023-10-16T20:06:12.620 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.611+0000 7f1e4df0f640 1 -- 172.21.15.88:0/3835427889 shutdown_connections 2023-10-16T20:06:12.620 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.611+0000 7f1e4df0f640 1 --2- 172.21.15.88:0/3835427889 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1e340664d0 0x7f1e34068990 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:12.621 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.611+0000 7f1e4df0f640 1 --2- 172.21.15.88:0/3835427889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e481616f0 0x7f1e480c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:12.621 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.611+0000 7f1e4df0f640 1 --2- 172.21.15.88:0/3835427889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e48161310 0x7f1e480c81a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:12.621 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.611+0000 7f1e4df0f640 1 --2- 172.21.15.88:0/3835427889 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e4806c370 0x7f1e48153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:12.622 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.611+0000 7f1e4df0f640 1 -- 172.21.15.88:0/3835427889 >> 172.21.15.88:0/3835427889 conn(0x7f1e48070aa0 msgr2=0x7f1e480c9400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:12.622 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.611+0000 7f1e4df0f640 1 -- 172.21.15.88:0/3835427889 shutdown_connections 2023-10-16T20:06:12.622 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.611+0000 7f1e4df0f640 1 -- 172.21.15.88:0/3835427889 wait complete. 2023-10-16T20:06:12.637 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-16T20:06:12.776 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.771+0000 7f2861f62640 1 Processor -- start 2023-10-16T20:06:12.776 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.771+0000 7f2861f62640 1 -- start start 2023-10-16T20:06:12.777 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.771+0000 7f2861f62640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f285c160180 0x7f285c0cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:12.778 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.771+0000 7f2861f62640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f285c0d12d0 0x7f285c0cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:12.778 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.771+0000 7f2861f62640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f285c0b3df0 con 0x7f285c160560 2023-10-16T20:06:12.778 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.771+0000 7f2861f62640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f285c0b4280 con 0x7f285c0d12d0 2023-10-16T20:06:12.778 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.771+0000 7f2861f62640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f285c0d0220 con 0x7f285c160180 2023-10-16T20:06:12.778 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.771+0000 7f2861761640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f285c0d12d0 0x7f285c0cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:12.779 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.771+0000 7f2861761640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f285c0d12d0 0x7f285c0cd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45706/0 (socket says 172.21.15.88:45706) 2023-10-16T20:06:12.779 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.771+0000 7f2861761640 1 -- 172.21.15.88:0/625785446 learned_addr learned my addr 172.21.15.88:0/625785446 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:12.780 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.771+0000 7f2861761640 1 -- 172.21.15.88:0/625785446 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f285c160180 msgr2=0x7f285c0cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:12.780 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.771+0000 7f2860f60640 1 --2- 172.21.15.88:0/625785446 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f285c160180 0x7f285c0cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:12.780 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.771+0000 7f2861761640 1 --2- 172.21.15.88:0/625785446 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f285c160180 0x7f285c0cfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:12.780 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.771+0000 7f2861761640 1 -- 172.21.15.88:0/625785446 >> v1:172.21.15.94:6789/0 conn(0x7f285c160560 legacy=0x7f285c0d0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:12.781 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.771+0000 7f2861761640 1 -- 172.21.15.88:0/625785446 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f285c0b3af0 con 0x7f285c0d12d0 2023-10-16T20:06:12.781 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.771+0000 7f2860f60640 1 --2- 172.21.15.88:0/625785446 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f285c160180 0x7f285c0cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:06:12.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.775+0000 7f2861761640 1 --2- 172.21.15.88:0/625785446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f285c0d12d0 0x7f285c0cd4a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2854009e50 tx=0x7f285401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de35c8b5c167fe9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:12.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.775+0000 7f285b7fe640 1 -- 172.21.15.88:0/625785446 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28540a57b0 con 0x7f285c0d12d0 2023-10-16T20:06:12.783 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.775+0000 7f285b7fe640 1 -- 172.21.15.88:0/625785446 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28540a5910 con 0x7f285c0d12d0 2023-10-16T20:06:12.783 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.775+0000 7f285b7fe640 1 -- 172.21.15.88:0/625785446 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28540a5ca0 con 0x7f285c0d12d0 2023-10-16T20:06:12.783 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.775+0000 7f2861f62640 1 -- 172.21.15.88:0/625785446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f285c0d12d0 msgr2=0x7f285c0cd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:12.783 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.775+0000 7f2861f62640 1 --2- 172.21.15.88:0/625785446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f285c0d12d0 0x7f285c0cd4a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2854009e50 tx=0x7f285401c910 comp rx=0 tx=0).stop 2023-10-16T20:06:12.783 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.775+0000 7f2861f62640 1 -- 172.21.15.88:0/625785446 shutdown_connections 2023-10-16T20:06:12.784 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.775+0000 7f2861f62640 1 --2- 172.21.15.88:0/625785446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f285c0d12d0 0x7f285c0cd4a0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:12.784 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.775+0000 7f2861f62640 1 --2- 172.21.15.88:0/625785446 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f285c160180 0x7f285c0cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:12.784 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.775+0000 7f2861f62640 1 -- 172.21.15.88:0/625785446 >> 172.21.15.88:0/625785446 conn(0x7f285c089fb0 msgr2=0x7f285c0d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:12.784 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.775+0000 7f2861f62640 1 -- 172.21.15.88:0/625785446 shutdown_connections 2023-10-16T20:06:12.784 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.775+0000 7f2861f62640 1 -- 172.21.15.88:0/625785446 wait complete. 2023-10-16T20:06:12.788 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.779+0000 7f2861f62640 1 Processor -- start 2023-10-16T20:06:12.788 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.779+0000 7f2861f62640 1 -- start start 2023-10-16T20:06:12.788 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.779+0000 7f2861f62640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f285c0d12d0 0x7f285c157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:12.788 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.779+0000 7f2861f62640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f285c160180 0x7f285c0c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:12.789 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.779+0000 7f2861f62640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f285c160560 0x7f285c0b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:12.789 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.779+0000 7f2861f62640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285c0c7f40 con 0x7f285c160180 2023-10-16T20:06:12.789 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.779+0000 7f2861f62640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285c0c87e0 con 0x7f285c160560 2023-10-16T20:06:12.790 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.779+0000 7f2861f62640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f285c0c7310 con 0x7f285c0d12d0 2023-10-16T20:06:12.790 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.779+0000 7f285bfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f285c160180 0x7f285c0c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:12.790 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.779+0000 7f285bfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f285c160180 0x7f285c0c8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45714/0 (socket says 172.21.15.88:45714) 2023-10-16T20:06:12.790 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.779+0000 7f285bfff640 1 -- 172.21.15.88:0/3756689082 learned_addr learned my addr 172.21.15.88:0/3756689082 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:12.790 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.779+0000 7f2861761640 1 --2- 172.21.15.88:0/3756689082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f285c160560 0x7f285c0b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:12.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.779+0000 7f2860f60640 1 --2- 172.21.15.88:0/3756689082 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f285c0d12d0 0x7f285c157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:12.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.779+0000 7f285bfff640 1 -- 172.21.15.88:0/3756689082 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f285c0d12d0 msgr2=0x7f285c157e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:12.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.779+0000 7f285bfff640 1 --2- 172.21.15.88:0/3756689082 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f285c0d12d0 0x7f285c157e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:12.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.779+0000 7f285bfff640 1 -- 172.21.15.88:0/3756689082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f285c160560 msgr2=0x7f285c0b0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:12.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.779+0000 7f285bfff640 1 --2- 172.21.15.88:0/3756689082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f285c160560 0x7f285c0b0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:12.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.779+0000 7f285bfff640 1 -- 172.21.15.88:0/3756689082 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2854009b30 con 0x7f285c160180 2023-10-16T20:06:12.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.779+0000 7f285bfff640 1 --2- 172.21.15.88:0/3756689082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f285c160180 0x7f285c0c8c00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f284800b410 tx=0x7f284800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:12.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.779+0000 7f28597fa640 1 -- 172.21.15.88:0/3756689082 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2848004020 con 0x7f285c160180 2023-10-16T20:06:12.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.779+0000 7f28597fa640 1 -- 172.21.15.88:0/3756689082 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2848004180 con 0x7f285c160180 2023-10-16T20:06:12.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.779+0000 7f2861f62640 1 -- 172.21.15.88:0/3756689082 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f285c0c7560 con 0x7f285c160180 2023-10-16T20:06:12.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.779+0000 7f28597fa640 1 -- 172.21.15.88:0/3756689082 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2848011630 con 0x7f285c160180 2023-10-16T20:06:12.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.779+0000 7f2861f62640 1 -- 172.21.15.88:0/3756689082 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f285c0c8400 con 0x7f285c160180 2023-10-16T20:06:12.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.783+0000 7f2861f62640 1 -- 172.21.15.88:0/3756689082 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f285c0674f0 con 0x7f285c160180 2023-10-16T20:06:12.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.783+0000 7f28597fa640 1 -- 172.21.15.88:0/3756689082 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2848011790 con 0x7f285c160180 2023-10-16T20:06:12.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.783+0000 7f28597fa640 1 --2- 172.21.15.88:0/3756689082 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f28400664a0 0x7f2840068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:12.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.783+0000 7f28597fa640 1 -- 172.21.15.88:0/3756689082 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f2848083870 con 0x7f285c160180 2023-10-16T20:06:12.795 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.783+0000 7f2860f60640 1 --2- 172.21.15.88:0/3756689082 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f28400664a0 0x7f2840068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:12.795 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.783+0000 7f2860f60640 1 --2- 172.21.15.88:0/3756689082 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f28400664a0 0x7f2840068960 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f285c0b3840 tx=0x7f2850008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:12.795 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.787+0000 7f28597fa640 1 -- 172.21.15.88:0/3756689082 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2848050fa0 con 0x7f285c160180 2023-10-16T20:06:12.994 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.987+0000 7f2861f62640 1 -- 172.21.15.88:0/3756689082 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f285c0c66e0 con 0x7f285c160180 2023-10-16T20:06:12.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.991+0000 7f28597fa640 1 -- 172.21.15.88:0/3756689082 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f28480545c0 con 0x7f285c160180 2023-10-16T20:06:13.000 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.995+0000 7f2861f62640 1 -- 172.21.15.88:0/3756689082 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f28400664a0 msgr2=0x7f2840068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:13.000 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.995+0000 7f2861f62640 1 --2- 172.21.15.88:0/3756689082 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f28400664a0 0x7f2840068960 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f285c0b3840 tx=0x7f2850008040 comp rx=0 tx=0).stop 2023-10-16T20:06:13.000 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.995+0000 7f2861f62640 1 -- 172.21.15.88:0/3756689082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f285c160180 msgr2=0x7f285c0c8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:13.000 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.995+0000 7f2861f62640 1 --2- 172.21.15.88:0/3756689082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f285c160180 0x7f285c0c8c00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f284800b410 tx=0x7f284800b8d0 comp rx=0 tx=0).stop 2023-10-16T20:06:13.000 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.995+0000 7f2861f62640 1 -- 172.21.15.88:0/3756689082 shutdown_connections 2023-10-16T20:06:13.001 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.995+0000 7f2861f62640 1 --2- 172.21.15.88:0/3756689082 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f28400664a0 0x7f2840068960 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:13.001 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.995+0000 7f2861f62640 1 --2- 172.21.15.88:0/3756689082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f285c160560 0x7f285c0b0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:13.001 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.995+0000 7f2861f62640 1 --2- 172.21.15.88:0/3756689082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f285c160180 0x7f285c0c8c00 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:13.001 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.995+0000 7f2861f62640 1 --2- 172.21.15.88:0/3756689082 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f285c0d12d0 0x7f285c157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:13.001 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.995+0000 7f2861f62640 1 -- 172.21.15.88:0/3756689082 >> 172.21.15.88:0/3756689082 conn(0x7f285c089fb0 msgr2=0x7f285c06ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:13.001 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.995+0000 7f2861f62640 1 -- 172.21.15.88:0/3756689082 shutdown_connections 2023-10-16T20:06:13.002 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:12.995+0000 7f2861f62640 1 -- 172.21.15.88:0/3756689082 wait complete. 2023-10-16T20:06:13.019 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-16T20:06:13.158 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.151+0000 7f609e005640 1 Processor -- start 2023-10-16T20:06:13.159 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.151+0000 7f609e005640 1 -- start start 2023-10-16T20:06:13.160 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.155+0000 7f609e005640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f60980adf80 0x7f609815a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:13.160 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.155+0000 7f609e005640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f609815e400 con 0x7f609815d480 2023-10-16T20:06:13.160 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.155+0000 7f609e005640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f609815de40 con 0x7f6098069e90 2023-10-16T20:06:13.160 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.155+0000 7f609e005640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6098159e40 con 0x7f60980adf80 2023-10-16T20:06:13.160 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.155+0000 7f6096ffd640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f60980adf80 0x7f609815a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:13.161 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.155+0000 7f6096ffd640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f60980adf80 0x7f609815a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45730/0 (socket says 172.21.15.88:45730) 2023-10-16T20:06:13.161 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.155+0000 7f6096ffd640 1 -- 172.21.15.88:0/2204306986 learned_addr learned my addr 172.21.15.88:0/2204306986 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:13.161 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.155+0000 7f6096ffd640 1 -- 172.21.15.88:0/2204306986 >> v1:172.21.15.94:6789/0 conn(0x7f6098069e90 legacy=0x7f609806a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:13.162 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.155+0000 7f6096ffd640 1 -- 172.21.15.88:0/2204306986 >> v1:172.21.15.88:6789/0 conn(0x7f609815d480 legacy=0x7f60980afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:13.162 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.155+0000 7f6096ffd640 1 -- 172.21.15.88:0/2204306986 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f609815e120 con 0x7f60980adf80 2023-10-16T20:06:13.163 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.159+0000 7f6096ffd640 1 --2- 172.21.15.88:0/2204306986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60980adf80 0x7f609815a830 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f608c009870 tx=0x7f608c01e690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2390fa7b781641fb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:13.163 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.159+0000 7f60967fc640 1 -- 172.21.15.88:0/2204306986 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c0ac610 con 0x7f60980adf80 2023-10-16T20:06:13.164 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.159+0000 7f60967fc640 1 -- 172.21.15.88:0/2204306986 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f608c0a8040 con 0x7f60980adf80 2023-10-16T20:06:13.164 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.159+0000 7f60967fc640 1 -- 172.21.15.88:0/2204306986 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c0ac990 con 0x7f60980adf80 2023-10-16T20:06:13.165 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.159+0000 7f609e005640 1 -- 172.21.15.88:0/2204306986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60980adf80 msgr2=0x7f609815a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:13.165 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.159+0000 7f609e005640 1 --2- 172.21.15.88:0/2204306986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60980adf80 0x7f609815a830 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f608c009870 tx=0x7f608c01e690 comp rx=0 tx=0).stop 2023-10-16T20:06:13.165 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.159+0000 7f609e005640 1 -- 172.21.15.88:0/2204306986 shutdown_connections 2023-10-16T20:06:13.165 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.159+0000 7f609e005640 1 --2- 172.21.15.88:0/2204306986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60980adf80 0x7f609815a830 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:13.166 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.159+0000 7f609e005640 1 -- 172.21.15.88:0/2204306986 >> 172.21.15.88:0/2204306986 conn(0x7f60980ce5e0 msgr2=0x7f60980ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:13.166 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.159+0000 7f609e005640 1 -- 172.21.15.88:0/2204306986 shutdown_connections 2023-10-16T20:06:13.166 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.159+0000 7f609e005640 1 -- 172.21.15.88:0/2204306986 wait complete. 2023-10-16T20:06:13.168 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.159+0000 7f609e005640 1 Processor -- start 2023-10-16T20:06:13.168 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.159+0000 7f609e005640 1 -- start start 2023-10-16T20:06:13.169 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.163+0000 7f609e005640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6098069e90 0x7f6098157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:13.169 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.163+0000 7f609e005640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60980adf80 0x7f60981f9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:13.169 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.163+0000 7f609e005640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f609815d480 0x7f60981f99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:13.169 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.163+0000 7f609e005640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60981584c0 con 0x7f609815d480 2023-10-16T20:06:13.169 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.163+0000 7f609e005640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60981587a0 con 0x7f60980adf80 2023-10-16T20:06:13.170 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.163+0000 7f609e005640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6098158a80 con 0x7f6098069e90 2023-10-16T20:06:13.170 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.163+0000 7f6097fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f609815d480 0x7f60981f99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:13.170 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.163+0000 7f6097fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f609815d480 0x7f60981f99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45742/0 (socket says 172.21.15.88:45742) 2023-10-16T20:06:13.170 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.163+0000 7f6097fff640 1 -- 172.21.15.88:0/3998790546 learned_addr learned my addr 172.21.15.88:0/3998790546 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:13.170 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.163+0000 7f6096ffd640 1 --2- 172.21.15.88:0/3998790546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60980adf80 0x7f60981f9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:13.171 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.163+0000 7f60977fe640 1 --2- 172.21.15.88:0/3998790546 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6098069e90 0x7f6098157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:13.171 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.163+0000 7f6097fff640 1 -- 172.21.15.88:0/3998790546 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6098069e90 msgr2=0x7f6098157de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:13.171 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.163+0000 7f6097fff640 1 --2- 172.21.15.88:0/3998790546 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6098069e90 0x7f6098157de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:13.171 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.163+0000 7f6097fff640 1 -- 172.21.15.88:0/3998790546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60980adf80 msgr2=0x7f60981f9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:13.171 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.163+0000 7f6097fff640 1 --2- 172.21.15.88:0/3998790546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60980adf80 0x7f60981f9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:13.172 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.163+0000 7f6097fff640 1 -- 172.21.15.88:0/3998790546 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c0094d0 con 0x7f609815d480 2023-10-16T20:06:13.172 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.163+0000 7f6097fff640 1 --2- 172.21.15.88:0/3998790546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f609815d480 0x7f60981f99b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f609000c730 tx=0x7f609000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:13.172 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.163+0000 7f607ffff640 1 -- 172.21.15.88:0/3998790546 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6090003b80 con 0x7f609815d480 2023-10-16T20:06:13.172 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.163+0000 7f609e005640 1 -- 172.21.15.88:0/3998790546 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6098158d60 con 0x7f609815d480 2023-10-16T20:06:13.172 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.163+0000 7f607ffff640 1 -- 172.21.15.88:0/3998790546 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6090003ce0 con 0x7f609815d480 2023-10-16T20:06:13.173 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.163+0000 7f607ffff640 1 -- 172.21.15.88:0/3998790546 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609000aa30 con 0x7f609815d480 2023-10-16T20:06:13.173 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.163+0000 7f609e005640 1 -- 172.21.15.88:0/3998790546 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6098148d30 con 0x7f609815d480 2023-10-16T20:06:13.174 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.163+0000 7f609e005640 1 -- 172.21.15.88:0/3998790546 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6098148f50 con 0x7f609815d480 2023-10-16T20:06:13.174 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.163+0000 7f607ffff640 1 -- 172.21.15.88:0/3998790546 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f609000ab90 con 0x7f609815d480 2023-10-16T20:06:13.174 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.167+0000 7f607ffff640 1 --2- 172.21.15.88:0/3998790546 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f60840664a0 0x7f6084068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:13.174 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.167+0000 7f607ffff640 1 -- 172.21.15.88:0/3998790546 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f6090083ae0 con 0x7f609815d480 2023-10-16T20:06:13.175 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.167+0000 7f60977fe640 1 --2- 172.21.15.88:0/3998790546 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f60840664a0 0x7f6084068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:13.175 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.167+0000 7f60977fe640 1 --2- 172.21.15.88:0/3998790546 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f60840664a0 0x7f6084068960 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6088000f70 tx=0x7f6088009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:13.175 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.171+0000 7f607ffff640 1 -- 172.21.15.88:0/3998790546 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f60900510f0 con 0x7f609815d480 2023-10-16T20:06:13.375 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.371+0000 7f609e005640 1 -- 172.21.15.88:0/3998790546 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f609806d180 con 0x7f609815d480 2023-10-16T20:06:13.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.375+0000 7f607ffff640 1 -- 172.21.15.88:0/3998790546 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f6090054710 con 0x7f609815d480 2023-10-16T20:06:13.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.375+0000 7f609e005640 1 -- 172.21.15.88:0/3998790546 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f60840664a0 msgr2=0x7f6084068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:13.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.375+0000 7f609e005640 1 --2- 172.21.15.88:0/3998790546 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f60840664a0 0x7f6084068960 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6088000f70 tx=0x7f6088009040 comp rx=0 tx=0).stop 2023-10-16T20:06:13.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.375+0000 7f609e005640 1 -- 172.21.15.88:0/3998790546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f609815d480 msgr2=0x7f60981f99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:13.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.375+0000 7f609e005640 1 --2- 172.21.15.88:0/3998790546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f609815d480 0x7f60981f99b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f609000c730 tx=0x7f609000cbf0 comp rx=0 tx=0).stop 2023-10-16T20:06:13.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.375+0000 7f609e005640 1 -- 172.21.15.88:0/3998790546 shutdown_connections 2023-10-16T20:06:13.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.375+0000 7f609e005640 1 --2- 172.21.15.88:0/3998790546 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f60840664a0 0x7f6084068960 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:13.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.375+0000 7f609e005640 1 --2- 172.21.15.88:0/3998790546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f609815d480 0x7f60981f99b0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:13.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.375+0000 7f609e005640 1 --2- 172.21.15.88:0/3998790546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60980adf80 0x7f60981f9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:13.384 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.375+0000 7f609e005640 1 --2- 172.21.15.88:0/3998790546 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6098069e90 0x7f6098157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:13.384 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.375+0000 7f609e005640 1 -- 172.21.15.88:0/3998790546 >> 172.21.15.88:0/3998790546 conn(0x7f60980ce5e0 msgr2=0x7f6098149240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:13.384 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.375+0000 7f609e005640 1 -- 172.21.15.88:0/3998790546 shutdown_connections 2023-10-16T20:06:13.384 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.375+0000 7f609e005640 1 -- 172.21.15.88:0/3998790546 wait complete. 2023-10-16T20:06:13.401 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:06:13.539 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.535+0000 7f6a7279d640 1 Processor -- start 2023-10-16T20:06:13.540 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.535+0000 7f6a7279d640 1 -- start start 2023-10-16T20:06:13.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.535+0000 7f6a7279d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a6c0d22e0 0x7f6a6c161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:13.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.535+0000 7f6a7279d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f6a6c0d26c0 0x7f6a6c15d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:13.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.535+0000 7f6a7279d640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a6c0b87e0 con 0x7f6a6c15d820 2023-10-16T20:06:13.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.535+0000 7f6a7279d640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a6c15dc40 con 0x7f6a6c0d26c0 2023-10-16T20:06:13.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.535+0000 7f6a7279d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c15df10 con 0x7f6a6c0d22e0 2023-10-16T20:06:13.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.535+0000 7f6a6bfff640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a6c0d22e0 0x7f6a6c161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:13.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.535+0000 7f6a70d3a640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f6a6c15d820 0x7f6a6c0cf2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:33202/0 (socket says 172.21.15.88:33202) 2023-10-16T20:06:13.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.535+0000 7f6a70d3a640 1 -- 172.21.15.88:0/1613362825 learned_addr learned my addr 172.21.15.88:0/1613362825 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:13.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.535+0000 7f6a6b7fe640 1 --2- 172.21.15.88:0/1613362825 >> v2:172.21.15.94:3300/0 conn(0x7f6a6c0d26c0 0x7f6a6c15d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:13.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.535+0000 7f6a6affd640 1 -- 172.21.15.88:0/1613362825 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1529367417 0 0) 0x7f6a6c0b87e0 con 0x7f6a6c15d820 2023-10-16T20:06:13.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.535+0000 7f6a6bfff640 1 -- 172.21.15.88:0/1613362825 >> v2:172.21.15.94:3300/0 conn(0x7f6a6c0d26c0 msgr2=0x7f6a6c15d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:13.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.535+0000 7f6a6bfff640 1 --2- 172.21.15.88:0/1613362825 >> v2:172.21.15.94:3300/0 conn(0x7f6a6c0d26c0 0x7f6a6c15d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:13.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.535+0000 7f6a6bfff640 1 -- 172.21.15.88:0/1613362825 >> v1:172.21.15.94:6789/0 conn(0x7f6a6c15d820 legacy=0x7f6a6c0cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:13.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.535+0000 7f6a6bfff640 1 -- 172.21.15.88:0/1613362825 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6c15e1f0 con 0x7f6a6c0d22e0 2023-10-16T20:06:13.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.539+0000 7f6a6bfff640 1 --2- 172.21.15.88:0/1613362825 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a6c0d22e0 0x7f6a6c161450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c0097c0 tx=0x7f6a5c01c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9245fd0018b48395 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:13.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.539+0000 7f6a6affd640 1 -- 172.21.15.88:0/1613362825 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c0ab600 con 0x7f6a6c0d22e0 2023-10-16T20:06:13.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.539+0000 7f6a6affd640 1 -- 172.21.15.88:0/1613362825 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5c0a7050 con 0x7f6a6c0d22e0 2023-10-16T20:06:13.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.539+0000 7f6a6affd640 1 -- 172.21.15.88:0/1613362825 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c0ab980 con 0x7f6a6c0d22e0 2023-10-16T20:06:13.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.539+0000 7f6a7279d640 1 -- 172.21.15.88:0/1613362825 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a6c0d22e0 msgr2=0x7f6a6c161450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:13.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.539+0000 7f6a7279d640 1 --2- 172.21.15.88:0/1613362825 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a6c0d22e0 0x7f6a6c161450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c0097c0 tx=0x7f6a5c01c3c0 comp rx=0 tx=0).stop 2023-10-16T20:06:13.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.539+0000 7f6a7279d640 1 -- 172.21.15.88:0/1613362825 shutdown_connections 2023-10-16T20:06:13.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.539+0000 7f6a7279d640 1 --2- 172.21.15.88:0/1613362825 >> v2:172.21.15.94:3300/0 conn(0x7f6a6c0d26c0 0x7f6a6c15d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:13.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.539+0000 7f6a7279d640 1 --2- 172.21.15.88:0/1613362825 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a6c0d22e0 0x7f6a6c161450 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:13.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.539+0000 7f6a7279d640 1 -- 172.21.15.88:0/1613362825 >> 172.21.15.88:0/1613362825 conn(0x7f6a6c0715c0 msgr2=0x7f6a6c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:13.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.539+0000 7f6a7279d640 1 -- 172.21.15.88:0/1613362825 shutdown_connections 2023-10-16T20:06:13.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.539+0000 7f6a7279d640 1 -- 172.21.15.88:0/1613362825 wait complete. 2023-10-16T20:06:13.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.543+0000 7f6a7279d640 1 Processor -- start 2023-10-16T20:06:13.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.543+0000 7f6a7279d640 1 -- start start 2023-10-16T20:06:13.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.543+0000 7f6a7279d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a6c0d22e0 0x7f6a6c14e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:13.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.543+0000 7f6a7279d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a6c0d26c0 0x7f6a6c1515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:13.552 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.543+0000 7f6a7279d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a6c15d820 0x7f6a6c1520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:13.552 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.543+0000 7f6a7279d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c14ecd0 con 0x7f6a6c0d26c0 2023-10-16T20:06:13.553 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.543+0000 7f6a7279d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c14efb0 con 0x7f6a6c0d22e0 2023-10-16T20:06:13.553 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.543+0000 7f6a7279d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c14f290 con 0x7f6a6c15d820 2023-10-16T20:06:13.553 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.543+0000 7f6a6b7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a6c0d26c0 0x7f6a6c1515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:13.553 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.543+0000 7f6a6b7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a6c0d26c0 0x7f6a6c1515e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45750/0 (socket says 172.21.15.88:45750) 2023-10-16T20:06:13.554 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.543+0000 7f6a6b7fe640 1 -- 172.21.15.88:0/3581964232 learned_addr learned my addr 172.21.15.88:0/3581964232 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:13.554 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.543+0000 7f6a70d3a640 1 --2- 172.21.15.88:0/3581964232 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a6c15d820 0x7f6a6c1520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:13.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.543+0000 7f6a6bfff640 1 --2- 172.21.15.88:0/3581964232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a6c0d22e0 0x7f6a6c14e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:13.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.543+0000 7f6a6b7fe640 1 -- 172.21.15.88:0/3581964232 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a6c15d820 msgr2=0x7f6a6c1520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:13.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.543+0000 7f6a6b7fe640 1 --2- 172.21.15.88:0/3581964232 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a6c15d820 0x7f6a6c1520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:13.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.543+0000 7f6a6b7fe640 1 -- 172.21.15.88:0/3581964232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a6c0d22e0 msgr2=0x7f6a6c14e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:13.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.543+0000 7f6a6b7fe640 1 --2- 172.21.15.88:0/3581964232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a6c0d22e0 0x7f6a6c14e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:13.556 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.543+0000 7f6a6b7fe640 1 -- 172.21.15.88:0/3581964232 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5c009420 con 0x7f6a6c0d26c0 2023-10-16T20:06:13.556 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.543+0000 7f6a6b7fe640 1 --2- 172.21.15.88:0/3581964232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a6c0d26c0 0x7f6a6c1515e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000e930 tx=0x7f6a6000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:13.556 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.543+0000 7f6a68ff9640 1 -- 172.21.15.88:0/3581964232 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a60009690 con 0x7f6a6c0d26c0 2023-10-16T20:06:13.556 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.543+0000 7f6a68ff9640 1 -- 172.21.15.88:0/3581964232 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a6000ca00 con 0x7f6a6c0d26c0 2023-10-16T20:06:13.557 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.543+0000 7f6a68ff9640 1 -- 172.21.15.88:0/3581964232 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6000cd70 con 0x7f6a6c0d26c0 2023-10-16T20:06:13.557 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.543+0000 7f6a7279d640 1 -- 172.21.15.88:0/3581964232 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a6c14f570 con 0x7f6a6c0d26c0 2023-10-16T20:06:13.557 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.543+0000 7f6a7279d640 1 -- 172.21.15.88:0/3581964232 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a6c14e910 con 0x7f6a6c0d26c0 2023-10-16T20:06:13.558 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.547+0000 7f6a7279d640 1 -- 172.21.15.88:0/3581964232 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a6c04e9c0 con 0x7f6a6c0d26c0 2023-10-16T20:06:13.558 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.547+0000 7f6a68ff9640 1 -- 172.21.15.88:0/3581964232 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6a60013470 con 0x7f6a6c0d26c0 2023-10-16T20:06:13.558 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.547+0000 7f6a68ff9640 1 --2- 172.21.15.88:0/3581964232 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f6a580664d0 0x7f6a58068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:13.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.547+0000 7f6a68ff9640 1 -- 172.21.15.88:0/3581964232 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f6a60083e20 con 0x7f6a6c0d26c0 2023-10-16T20:06:13.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.547+0000 7f6a6bfff640 1 --2- 172.21.15.88:0/3581964232 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f6a580664d0 0x7f6a58068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:13.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.547+0000 7f6a6bfff640 1 --2- 172.21.15.88:0/3581964232 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f6a580664d0 0x7f6a58068990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c0ad000 tx=0x7f6a5c0a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:13.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.551+0000 7f6a68ff9640 1 -- 172.21.15.88:0/3581964232 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6a60051590 con 0x7f6a6c0d26c0 2023-10-16T20:06:13.708 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.703+0000 7f6a7279d640 1 -- 172.21.15.88:0/3581964232 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a6c0d0840 con 0x7f6a6c0d26c0 2023-10-16T20:06:13.708 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.703+0000 7f6a68ff9640 1 -- 172.21.15.88:0/3581964232 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+12990 (secure 0 0 0) 0x7f6a60021090 con 0x7f6a6c0d26c0 2023-10-16T20:06:13.709 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:06:13.709 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":37,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:06:10.679175+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:06:13.711 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.703+0000 7f6a7279d640 1 -- 172.21.15.88:0/3581964232 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f6a580664d0 msgr2=0x7f6a58068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:13.711 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.703+0000 7f6a7279d640 1 --2- 172.21.15.88:0/3581964232 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f6a580664d0 0x7f6a58068990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c0ad000 tx=0x7f6a5c0a6000 comp rx=0 tx=0).stop 2023-10-16T20:06:13.711 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.703+0000 7f6a7279d640 1 -- 172.21.15.88:0/3581964232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a6c0d26c0 msgr2=0x7f6a6c1515e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:13.711 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.703+0000 7f6a7279d640 1 --2- 172.21.15.88:0/3581964232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a6c0d26c0 0x7f6a6c1515e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000e930 tx=0x7f6a6000edf0 comp rx=0 tx=0).stop 2023-10-16T20:06:13.711 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.703+0000 7f6a7279d640 1 -- 172.21.15.88:0/3581964232 shutdown_connections 2023-10-16T20:06:13.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.703+0000 7f6a7279d640 1 --2- 172.21.15.88:0/3581964232 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f6a580664d0 0x7f6a58068990 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:13.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.703+0000 7f6a7279d640 1 --2- 172.21.15.88:0/3581964232 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a6c15d820 0x7f6a6c1520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:13.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.703+0000 7f6a7279d640 1 --2- 172.21.15.88:0/3581964232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a6c0d26c0 0x7f6a6c1515e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:13.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.703+0000 7f6a7279d640 1 --2- 172.21.15.88:0/3581964232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a6c0d22e0 0x7f6a6c14e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:13.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.703+0000 7f6a7279d640 1 -- 172.21.15.88:0/3581964232 >> 172.21.15.88:0/3581964232 conn(0x7f6a6c0715c0 msgr2=0x7f6a6c157020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:13.713 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.707+0000 7f6a7279d640 1 -- 172.21.15.88:0/3581964232 shutdown_connections 2023-10-16T20:06:13.713 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.707+0000 7f6a7279d640 1 -- 172.21.15.88:0/3581964232 wait complete. 2023-10-16T20:06:13.730 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T20:06:13.730 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:06:13.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.867+0000 7fafc4965640 1 Processor -- start 2023-10-16T20:06:13.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.867+0000 7fafc4965640 1 -- start start 2023-10-16T20:06:13.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.867+0000 7fafc4965640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fafc015cab0 0x7fafc006d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:13.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.867+0000 7fafc4965640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fafc00ccd20 0x7fafc00cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:13.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.867+0000 7fafc4965640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafc0161430 con 0x7fafc00ae160 2023-10-16T20:06:13.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.867+0000 7fafc4965640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc00cd6f0 con 0x7fafc015cab0 2023-10-16T20:06:13.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.867+0000 7fafc4965640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc00cda10 con 0x7fafc00ccd20 2023-10-16T20:06:13.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.867+0000 7fafbe59c640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fafc015cab0 0x7fafc006d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:13.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.867+0000 7fafb5d9b640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fafc00ae160 0x7fafc015ce90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:36772/0 (socket says 172.21.15.88:36772) 2023-10-16T20:06:13.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.867+0000 7fafb5d9b640 1 -- 172.21.15.88:0/2921861569 learned_addr learned my addr 172.21.15.88:0/2921861569 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:13.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.867+0000 7fafbe59c640 1 -- 172.21.15.88:0/2921861569 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fafc00ccd20 msgr2=0x7fafc00cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:13.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.867+0000 7fafbe59c640 1 --2- 172.21.15.88:0/2921861569 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fafc00ccd20 0x7fafc00cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:13.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.867+0000 7fafbe59c640 1 -- 172.21.15.88:0/2921861569 >> v1:172.21.15.88:6789/0 conn(0x7fafc00ae160 legacy=0x7fafc015ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:13.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.867+0000 7fafbe59c640 1 -- 172.21.15.88:0/2921861569 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc015d5a0 con 0x7fafc015cab0 2023-10-16T20:06:13.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.867+0000 7fafbdd9b640 1 -- 172.21.15.88:0/2921861569 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 470052918 0 0) 0x7fafc0161430 con 0x7fafc00ae160 2023-10-16T20:06:13.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.871+0000 7fafbe59c640 1 --2- 172.21.15.88:0/2921861569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fafc015cab0 0x7fafc006d490 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fafb00097c0 tx=0x7fafb001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212585d6c85618cd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:13.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.871+0000 7fafbdd9b640 1 -- 172.21.15.88:0/2921861569 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb00ac620 con 0x7fafc015cab0 2023-10-16T20:06:13.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.871+0000 7fafbdd9b640 1 -- 172.21.15.88:0/2921861569 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb00ac780 con 0x7fafc015cab0 2023-10-16T20:06:13.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.871+0000 7fafbdd9b640 1 -- 172.21.15.88:0/2921861569 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb00acb10 con 0x7fafc015cab0 2023-10-16T20:06:13.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.871+0000 7fafc4965640 1 -- 172.21.15.88:0/2921861569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fafc015cab0 msgr2=0x7fafc006d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:13.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.871+0000 7fafc4965640 1 --2- 172.21.15.88:0/2921861569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fafc015cab0 0x7fafc006d490 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fafb00097c0 tx=0x7fafb001c3c0 comp rx=0 tx=0).stop 2023-10-16T20:06:13.878 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.871+0000 7fafc4965640 1 -- 172.21.15.88:0/2921861569 shutdown_connections 2023-10-16T20:06:13.878 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.871+0000 7fafc4965640 1 --2- 172.21.15.88:0/2921861569 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fafc00ccd20 0x7fafc00cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:13.878 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.871+0000 7fafc4965640 1 --2- 172.21.15.88:0/2921861569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fafc015cab0 0x7fafc006d490 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:13.878 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.871+0000 7fafc4965640 1 -- 172.21.15.88:0/2921861569 >> 172.21.15.88:0/2921861569 conn(0x7fafc00cee20 msgr2=0x7fafc00cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:13.879 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.871+0000 7fafc4965640 1 -- 172.21.15.88:0/2921861569 shutdown_connections 2023-10-16T20:06:13.879 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.871+0000 7fafc4965640 1 -- 172.21.15.88:0/2921861569 wait complete. 2023-10-16T20:06:13.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.871+0000 7fafc4965640 1 Processor -- start 2023-10-16T20:06:13.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.871+0000 7fafc4965640 1 -- start start 2023-10-16T20:06:13.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.875+0000 7fafc4965640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fafc00ae160 0x7fafc00c78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:13.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.875+0000 7fafc4965640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafc00ccd20 0x7fafc00c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:13.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.875+0000 7fafc4965640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fafc015cab0 0x7fafc00acb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:13.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.875+0000 7fafc4965640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc00c7e00 con 0x7fafc015cab0 2023-10-16T20:06:13.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.875+0000 7fafc4965640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc00c6820 con 0x7fafc00ccd20 2023-10-16T20:06:13.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.875+0000 7fafc4965640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc00c6b00 con 0x7fafc00ae160 2023-10-16T20:06:13.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.875+0000 7fafbed9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fafc015cab0 0x7fafc00acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:13.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.875+0000 7fafbed9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fafc015cab0 0x7fafc00acb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45780/0 (socket says 172.21.15.88:45780) 2023-10-16T20:06:13.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.875+0000 7fafbed9d640 1 -- 172.21.15.88:0/2022654224 learned_addr learned my addr 172.21.15.88:0/2022654224 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:13.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.875+0000 7fafbe59c640 1 --2- 172.21.15.88:0/2022654224 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fafc00ae160 0x7fafc00c78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:13.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.875+0000 7fafb5d9b640 1 --2- 172.21.15.88:0/2022654224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafc00ccd20 0x7fafc00c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:13.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.875+0000 7fafbed9d640 1 -- 172.21.15.88:0/2022654224 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fafc00ae160 msgr2=0x7fafc00c78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:13.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.875+0000 7fafbed9d640 1 --2- 172.21.15.88:0/2022654224 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fafc00ae160 0x7fafc00c78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:13.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.875+0000 7fafbed9d640 1 -- 172.21.15.88:0/2022654224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafc00ccd20 msgr2=0x7fafc00c5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:13.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.875+0000 7fafbed9d640 1 --2- 172.21.15.88:0/2022654224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafc00ccd20 0x7fafc00c5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:13.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.875+0000 7fafbed9d640 1 -- 172.21.15.88:0/2022654224 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb0009420 con 0x7fafc015cab0 2023-10-16T20:06:13.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.875+0000 7fafbed9d640 1 --2- 172.21.15.88:0/2022654224 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fafc015cab0 0x7fafc00acb50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fafb800b9e0 tx=0x7fafb800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:13.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.875+0000 7fafb77fe640 1 -- 172.21.15.88:0/2022654224 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb800caa0 con 0x7fafc015cab0 2023-10-16T20:06:13.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.875+0000 7fafb77fe640 1 -- 172.21.15.88:0/2022654224 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb800cc20 con 0x7fafc015cab0 2023-10-16T20:06:13.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.875+0000 7fafc4965640 1 -- 172.21.15.88:0/2022654224 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc00c6de0 con 0x7fafc015cab0 2023-10-16T20:06:13.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.875+0000 7fafb77fe640 1 -- 172.21.15.88:0/2022654224 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb8014440 con 0x7fafc015cab0 2023-10-16T20:06:13.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.875+0000 7fafc4965640 1 -- 172.21.15.88:0/2022654224 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc00c73a0 con 0x7fafc015cab0 2023-10-16T20:06:13.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.875+0000 7fafc4965640 1 -- 172.21.15.88:0/2022654224 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafc00c75c0 con 0x7fafc015cab0 2023-10-16T20:06:13.887 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.879+0000 7fafb77fe640 1 -- 172.21.15.88:0/2022654224 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fafb801d3b0 con 0x7fafc015cab0 2023-10-16T20:06:13.887 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.879+0000 7fafb77fe640 1 --2- 172.21.15.88:0/2022654224 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fafa00664a0 0x7fafa0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:13.887 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.879+0000 7fafb77fe640 1 -- 172.21.15.88:0/2022654224 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fafb8083ab0 con 0x7fafc015cab0 2023-10-16T20:06:13.887 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.879+0000 7fafbe59c640 1 --2- 172.21.15.88:0/2022654224 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fafa00664a0 0x7fafa0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:13.887 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.879+0000 7fafbe59c640 1 --2- 172.21.15.88:0/2022654224 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fafa00664a0 0x7fafa0068960 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fafb00ae000 tx=0x7fafb0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:13.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:13.883+0000 7fafb77fe640 1 -- 172.21.15.88:0/2022654224 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fafb80511e0 con 0x7fafc015cab0 2023-10-16T20:06:14.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.031+0000 7fafc4965640 1 -- 172.21.15.88:0/2022654224 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fafc004e9c0 con 0x7fafc015cab0 2023-10-16T20:06:14.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.035+0000 7fafb77fe640 1 -- 172.21.15.88:0/2022654224 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7fafb80190a0 con 0x7fafc015cab0 2023-10-16T20:06:14.039 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:06:14.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.035+0000 7fafc4965640 1 -- 172.21.15.88:0/2022654224 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fafa00664a0 msgr2=0x7fafa0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:14.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.035+0000 7fafc4965640 1 --2- 172.21.15.88:0/2022654224 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fafa00664a0 0x7fafa0068960 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fafb00ae000 tx=0x7fafb0008040 comp rx=0 tx=0).stop 2023-10-16T20:06:14.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.035+0000 7fafc4965640 1 -- 172.21.15.88:0/2022654224 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fafc015cab0 msgr2=0x7fafc00acb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:14.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.035+0000 7fafc4965640 1 --2- 172.21.15.88:0/2022654224 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fafc015cab0 0x7fafc00acb50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fafb800b9e0 tx=0x7fafb800bea0 comp rx=0 tx=0).stop 2023-10-16T20:06:14.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.035+0000 7fafc4965640 1 -- 172.21.15.88:0/2022654224 shutdown_connections 2023-10-16T20:06:14.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.035+0000 7fafc4965640 1 --2- 172.21.15.88:0/2022654224 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fafa00664a0 0x7fafa0068960 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:14.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.035+0000 7fafc4965640 1 --2- 172.21.15.88:0/2022654224 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fafc015cab0 0x7fafc00acb50 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:14.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.035+0000 7fafc4965640 1 --2- 172.21.15.88:0/2022654224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafc00ccd20 0x7fafc00c5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:14.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.035+0000 7fafc4965640 1 --2- 172.21.15.88:0/2022654224 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fafc00ae160 0x7fafc00c78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:14.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.035+0000 7fafc4965640 1 -- 172.21.15.88:0/2022654224 >> 172.21.15.88:0/2022654224 conn(0x7fafc00cee20 msgr2=0x7fafc006d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:14.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.035+0000 7fafc4965640 1 -- 172.21.15.88:0/2022654224 shutdown_connections 2023-10-16T20:06:14.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.035+0000 7fafc4965640 1 -- 172.21.15.88:0/2022654224 wait complete. 2023-10-16T20:06:14.060 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T20:06:14.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.191+0000 7fa0b3409640 1 Processor -- start 2023-10-16T20:06:14.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.191+0000 7fa0b3409640 1 -- start start 2023-10-16T20:06:14.198 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.191+0000 7fa0b3409640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0ac0cde80 0x7fa0ac0d3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:14.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.191+0000 7fa0b3409640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0ac0d0c40 con 0x7fa0ac15c9a0 2023-10-16T20:06:14.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.191+0000 7fa0b3409640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0ac0ce2a0 con 0x7fa0ac0cf230 2023-10-16T20:06:14.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.191+0000 7fa0b3409640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac0d37b0 con 0x7fa0ac0cde80 2023-10-16T20:06:14.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.191+0000 7fa0b2407640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fa0ac15c9a0 0x7fa0ac0d10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:36786/0 (socket says 172.21.15.88:36786) 2023-10-16T20:06:14.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.191+0000 7fa0b2407640 1 -- 172.21.15.88:0/531846188 learned_addr learned my addr 172.21.15.88:0/531846188 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:14.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.195+0000 7fa0b1405640 1 -- 172.21.15.88:0/531846188 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1977989670 0 0) 0x7fa0ac0d0c40 con 0x7fa0ac15c9a0 2023-10-16T20:06:14.202 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.195+0000 7fa0b1c06640 1 --2- 172.21.15.88:0/531846188 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0ac0cde80 0x7fa0ac0d3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:14.202 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.195+0000 7fa0b1405640 1 -- 172.21.15.88:0/531846188 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa098003530 con 0x7fa0ac15c9a0 2023-10-16T20:06:14.202 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.195+0000 7fa0b1405640 1 -- 172.21.15.88:0/531846188 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1909017793 0 0) 0x7fa098003530 con 0x7fa0ac15c9a0 2023-10-16T20:06:14.202 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.195+0000 7fa0b1405640 1 -- 172.21.15.88:0/531846188 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0ac0cde80 msgr2=0x7fa0ac0d3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:14.202 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.195+0000 7fa0b1405640 1 --2- 172.21.15.88:0/531846188 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0ac0cde80 0x7fa0ac0d3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:14.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.195+0000 7fa0b1405640 1 -- 172.21.15.88:0/531846188 >> v1:172.21.15.94:6789/0 conn(0x7fa0ac0cf230 legacy=0x7fa0ac0cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:14.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.195+0000 7fa0b1405640 1 -- 172.21.15.88:0/531846188 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ac0d0630 con 0x7fa0ac15c9a0 2023-10-16T20:06:14.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.195+0000 7fa0b1405640 1 -- 172.21.15.88:0/531846188 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7fa0a8001800 con 0x7fa0ac15c9a0 2023-10-16T20:06:14.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.195+0000 7fa0b1405640 1 -- 172.21.15.88:0/531846188 >> v1:172.21.15.88:6789/0 conn(0x7fa0ac15c9a0 legacy=0x7fa0ac0d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:14.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.195+0000 7fa0b1405640 1 --2- 172.21.15.88:0/531846188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0980039b0 0x7fa098003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:14.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.195+0000 7fa0b1405640 1 --2- 172.21.15.88:0/531846188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0980047d0 0x7fa098004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:14.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.195+0000 7fa0b1405640 1 --2- 172.21.15.88:0/531846188 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0ac0cde80 0x7fa098005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:14.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.195+0000 7fa0b1405640 1 -- 172.21.15.88:0/531846188 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac0d37b0 con 0x7fa0980039b0 2023-10-16T20:06:14.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.195+0000 7fa0b1405640 1 -- 172.21.15.88:0/531846188 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09800ba70 con 0x7fa0980047d0 2023-10-16T20:06:14.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.195+0000 7fa0b1405640 1 -- 172.21.15.88:0/531846188 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa09800bc30 con 0x7fa0ac0cde80 2023-10-16T20:06:14.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.195+0000 7fa0b2407640 1 --2- 172.21.15.88:0/531846188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0980039b0 0x7fa098003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:14.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.195+0000 7fa0b1c06640 1 --2- 172.21.15.88:0/531846188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0980047d0 0x7fa098004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:14.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.195+0000 7fa0b2c08640 1 --2- 172.21.15.88:0/531846188 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0ac0cde80 0x7fa098005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:14.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.195+0000 7fa0b2407640 1 -- 172.21.15.88:0/531846188 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0ac0cde80 msgr2=0x7fa098005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:14.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.195+0000 7fa0b2407640 1 --2- 172.21.15.88:0/531846188 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0ac0cde80 0x7fa098005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:14.206 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.195+0000 7fa0b2407640 1 -- 172.21.15.88:0/531846188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0980047d0 msgr2=0x7fa098004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:14.206 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.195+0000 7fa0b2407640 1 --2- 172.21.15.88:0/531846188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0980047d0 0x7fa098004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:14.206 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.195+0000 7fa0b2407640 1 -- 172.21.15.88:0/531846188 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa098003530 con 0x7fa0980039b0 2023-10-16T20:06:14.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.199+0000 7fa0b2407640 1 --2- 172.21.15.88:0/531846188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0980039b0 0x7fa098003dd0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800c210 tx=0x7fa0a801cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57bf546845814050 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:14.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.199+0000 7fa0b1405640 1 -- 172.21.15.88:0/531846188 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8007270 con 0x7fa0980039b0 2023-10-16T20:06:14.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.199+0000 7fa0b1405640 1 -- 172.21.15.88:0/531846188 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a80073f0 con 0x7fa0980039b0 2023-10-16T20:06:14.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.199+0000 7fa0b1405640 1 -- 172.21.15.88:0/531846188 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a80057b0 con 0x7fa0980039b0 2023-10-16T20:06:14.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.199+0000 7fa0b3409640 1 -- 172.21.15.88:0/531846188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0980039b0 msgr2=0x7fa098003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:14.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.199+0000 7fa0b3409640 1 --2- 172.21.15.88:0/531846188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0980039b0 0x7fa098003dd0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800c210 tx=0x7fa0a801cbe0 comp rx=0 tx=0).stop 2023-10-16T20:06:14.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.199+0000 7fa0b2c08640 1 -- 172.21.15.88:0/531846188 reap_dead start 2023-10-16T20:06:14.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.199+0000 7fa0b3409640 1 -- 172.21.15.88:0/531846188 shutdown_connections 2023-10-16T20:06:14.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.199+0000 7fa0b3409640 1 -- 172.21.15.88:0/531846188 >> 172.21.15.88:0/531846188 conn(0x7fa0ac09bc30 msgr2=0x7fa0ac0d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:14.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.199+0000 7fa0b3409640 1 -- 172.21.15.88:0/531846188 shutdown_connections 2023-10-16T20:06:14.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.199+0000 7fa0b3409640 1 -- 172.21.15.88:0/531846188 wait complete. 2023-10-16T20:06:14.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.199+0000 7fa0b3409640 1 Processor -- start 2023-10-16T20:06:14.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.199+0000 7fa0b3409640 1 -- start start 2023-10-16T20:06:14.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.199+0000 7fa0b3409640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0980039b0 0x7fa0ac154120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:14.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.199+0000 7fa0b3409640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0ac154660 0x7fa0ac0c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:14.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.199+0000 7fa0b3409640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0ac0c7840 0x7fa0ac0b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:14.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.199+0000 7fa0b3409640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac0c9ac0 con 0x7fa0ac0c7840 2023-10-16T20:06:14.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.199+0000 7fa0b3409640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac0c8680 con 0x7fa0ac154660 2023-10-16T20:06:14.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.199+0000 7fa0b3409640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac0c7de0 con 0x7fa0980039b0 2023-10-16T20:06:14.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.203+0000 7fa0b2c08640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0ac0c7840 0x7fa0ac0b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:14.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.203+0000 7fa0b2c08640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0ac0c7840 0x7fa0ac0b04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45794/0 (socket says 172.21.15.88:45794) 2023-10-16T20:06:14.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.203+0000 7fa0b2407640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0980039b0 0x7fa0ac154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:14.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.203+0000 7fa0b2c08640 1 -- 172.21.15.88:0/3095581180 learned_addr learned my addr 172.21.15.88:0/3095581180 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:14.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.203+0000 7fa0b1c06640 1 --2- 172.21.15.88:0/3095581180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0ac154660 0x7fa0ac0c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:14.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.203+0000 7fa0b2c08640 1 -- 172.21.15.88:0/3095581180 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0980039b0 msgr2=0x7fa0ac154120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:14.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.203+0000 7fa0b2c08640 1 --2- 172.21.15.88:0/3095581180 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0980039b0 0x7fa0ac154120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:14.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.203+0000 7fa0b2c08640 1 -- 172.21.15.88:0/3095581180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0ac154660 msgr2=0x7fa0ac0c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:14.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.203+0000 7fa0b2c08640 1 --2- 172.21.15.88:0/3095581180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0ac154660 0x7fa0ac0c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:14.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.203+0000 7fa0b2c08640 1 -- 172.21.15.88:0/3095581180 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ac15cf00 con 0x7fa0ac0c7840 2023-10-16T20:06:14.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.203+0000 7fa0b2c08640 1 --2- 172.21.15.88:0/3095581180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0ac0c7840 0x7fa0ac0b04e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac08ce90 tx=0x7fa0a400fcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:14.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.203+0000 7fa0a2ffd640 1 -- 172.21.15.88:0/3095581180 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a4006d70 con 0x7fa0ac0c7840 2023-10-16T20:06:14.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.203+0000 7fa0a2ffd640 1 -- 172.21.15.88:0/3095581180 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a4003980 con 0x7fa0ac0c7840 2023-10-16T20:06:14.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.203+0000 7fa0a2ffd640 1 -- 172.21.15.88:0/3095581180 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a4003d30 con 0x7fa0ac0c7840 2023-10-16T20:06:14.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.203+0000 7fa0b3409640 1 -- 172.21.15.88:0/3095581180 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0ac0c80c0 con 0x7fa0ac0c7840 2023-10-16T20:06:14.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.203+0000 7fa0b3409640 1 -- 172.21.15.88:0/3095581180 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0ac155180 con 0x7fa0ac0c7840 2023-10-16T20:06:14.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.203+0000 7fa0b3409640 1 -- 172.21.15.88:0/3095581180 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0ac0c9c30 con 0x7fa0ac0c7840 2023-10-16T20:06:14.217 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.203+0000 7fa0a2ffd640 1 -- 172.21.15.88:0/3095581180 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa0a4005110 con 0x7fa0ac0c7840 2023-10-16T20:06:14.217 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.203+0000 7fa0a2ffd640 1 --2- 172.21.15.88:0/3095581180 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa0900664a0 0x7fa090068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:14.217 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.207+0000 7fa0a2ffd640 1 -- 172.21.15.88:0/3095581180 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fa0a40832f0 con 0x7fa0ac0c7840 2023-10-16T20:06:14.217 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.207+0000 7fa0b2407640 1 --2- 172.21.15.88:0/3095581180 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa0900664a0 0x7fa090068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:14.218 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.207+0000 7fa0b2407640 1 --2- 172.21.15.88:0/3095581180 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa0900664a0 0x7fa090068960 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80ab000 tx=0x7fa0a80a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:14.218 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.211+0000 7fa0a2ffd640 1 -- 172.21.15.88:0/3095581180 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa0a4050a20 con 0x7fa0ac0c7840 2023-10-16T20:06:14.366 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.359+0000 7fa0b3409640 1 -- 172.21.15.88:0/3095581180 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa0ac0674f0 con 0x7fa0ac0c7840 2023-10-16T20:06:14.366 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.359+0000 7fa0a2ffd640 1 -- 172.21.15.88:0/3095581180 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v37) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa0a4054040 con 0x7fa0ac0c7840 2023-10-16T20:06:14.367 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:06:14.367 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2023-10-16T20:06:14.367 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:06:14.368 INFO:teuthology.orchestra.run.smithi088.stdout: "poolnum": 1, 2023-10-16T20:06:14.368 INFO:teuthology.orchestra.run.smithi088.stdout: "poolname": ".mgr" 2023-10-16T20:06:14.368 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:06:14.368 INFO:teuthology.orchestra.run.smithi088.stdout:] 2023-10-16T20:06:14.370 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.363+0000 7fa0b3409640 1 -- 172.21.15.88:0/3095581180 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa0900664a0 msgr2=0x7fa090068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:14.370 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.363+0000 7fa0b3409640 1 --2- 172.21.15.88:0/3095581180 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa0900664a0 0x7fa090068960 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80ab000 tx=0x7fa0a80a7070 comp rx=0 tx=0).stop 2023-10-16T20:06:14.370 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.363+0000 7fa0b3409640 1 -- 172.21.15.88:0/3095581180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0ac0c7840 msgr2=0x7fa0ac0b04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:14.370 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.363+0000 7fa0b3409640 1 --2- 172.21.15.88:0/3095581180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0ac0c7840 0x7fa0ac0b04e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac08ce90 tx=0x7fa0a400fcd0 comp rx=0 tx=0).stop 2023-10-16T20:06:14.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.363+0000 7fa0b3409640 1 -- 172.21.15.88:0/3095581180 shutdown_connections 2023-10-16T20:06:14.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.363+0000 7fa0b3409640 1 --2- 172.21.15.88:0/3095581180 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa0900664a0 0x7fa090068960 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:14.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.363+0000 7fa0b3409640 1 --2- 172.21.15.88:0/3095581180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0ac0c7840 0x7fa0ac0b04e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:14.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.363+0000 7fa0b3409640 1 --2- 172.21.15.88:0/3095581180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0ac154660 0x7fa0ac0c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:14.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.363+0000 7fa0b3409640 1 --2- 172.21.15.88:0/3095581180 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0980039b0 0x7fa0ac154120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:14.372 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.363+0000 7fa0b3409640 1 -- 172.21.15.88:0/3095581180 >> 172.21.15.88:0/3095581180 conn(0x7fa0ac09bc30 msgr2=0x7fa0ac162180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:14.372 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.363+0000 7fa0b3409640 1 -- 172.21.15.88:0/3095581180 shutdown_connections 2023-10-16T20:06:14.372 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.363+0000 7fa0b3409640 1 -- 172.21.15.88:0/3095581180 wait complete. 2023-10-16T20:06:14.387 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'testcephfs1' 2023-10-16T20:06:14.388 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create testcephfs1_metadata --pg_num_min 64 2023-10-16T20:06:14.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.523+0000 7f887f46b640 1 Processor -- start 2023-10-16T20:06:14.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.523+0000 7f887f46b640 1 -- start start 2023-10-16T20:06:14.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.527+0000 7f887f46b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f88780c19e0 0x7f88780c9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:14.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.527+0000 7f887f46b640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f887806a5a0 0x7f887806c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:14.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.527+0000 7f887f46b640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88780cd4c0 con 0x7f88780c8db0 2023-10-16T20:06:14.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.527+0000 7f887f46b640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88780ce140 con 0x7f887806a5a0 2023-10-16T20:06:14.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.527+0000 7f887f46b640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88780cd7a0 con 0x7f88780c19e0 2023-10-16T20:06:14.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.527+0000 7f887da08640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f887806a5a0 0x7f887806c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:14.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.527+0000 7f887da08640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f887806a5a0 0x7f887806c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45802/0 (socket says 172.21.15.88:45802) 2023-10-16T20:06:14.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.527+0000 7f887da08640 1 -- 172.21.15.88:0/3587579410 learned_addr learned my addr 172.21.15.88:0/3587579410 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:14.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.527+0000 7f887d207640 1 --2- 172.21.15.88:0/3587579410 >> v2:172.21.15.94:3300/0 conn(0x7f88780c19e0 0x7f88780c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:14.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.527+0000 7f887da08640 1 -- 172.21.15.88:0/3587579410 >> v2:172.21.15.94:3300/0 conn(0x7f88780c19e0 msgr2=0x7f88780c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:14.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.527+0000 7f887da08640 1 --2- 172.21.15.88:0/3587579410 >> v2:172.21.15.94:3300/0 conn(0x7f88780c19e0 0x7f88780c9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:14.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.527+0000 7f887da08640 1 -- 172.21.15.88:0/3587579410 >> v1:172.21.15.94:6789/0 conn(0x7f88780c8db0 legacy=0x7f8878069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:14.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.527+0000 7f887da08640 1 -- 172.21.15.88:0/3587579410 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88780cda80 con 0x7f887806a5a0 2023-10-16T20:06:14.535 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.527+0000 7f887d207640 1 --2- 172.21.15.88:0/3587579410 >> v2:172.21.15.94:3300/0 conn(0x7f88780c19e0 0x7f88780c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:14.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.527+0000 7f887da08640 1 --2- 172.21.15.88:0/3587579410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f887806a5a0 0x7f887806c480 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f88780cb040 tx=0x7f887001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec3b9c78ca56d06 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:14.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.531+0000 7f886ffff640 1 -- 172.21.15.88:0/3587579410 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88700a5880 con 0x7f887806a5a0 2023-10-16T20:06:14.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.531+0000 7f886ffff640 1 -- 172.21.15.88:0/3587579410 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88700a59e0 con 0x7f887806a5a0 2023-10-16T20:06:14.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.531+0000 7f886ffff640 1 -- 172.21.15.88:0/3587579410 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88700a5d70 con 0x7f887806a5a0 2023-10-16T20:06:14.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.531+0000 7f887f46b640 1 -- 172.21.15.88:0/3587579410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f887806a5a0 msgr2=0x7f887806c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:14.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.531+0000 7f887f46b640 1 --2- 172.21.15.88:0/3587579410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f887806a5a0 0x7f887806c480 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f88780cb040 tx=0x7f887001c9a0 comp rx=0 tx=0).stop 2023-10-16T20:06:14.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.531+0000 7f887f46b640 1 -- 172.21.15.88:0/3587579410 shutdown_connections 2023-10-16T20:06:14.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.531+0000 7f887f46b640 1 --2- 172.21.15.88:0/3587579410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f887806a5a0 0x7f887806c480 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:14.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.531+0000 7f887f46b640 1 --2- 172.21.15.88:0/3587579410 >> v2:172.21.15.94:3300/0 conn(0x7f88780c19e0 0x7f88780c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:14.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.531+0000 7f887f46b640 1 -- 172.21.15.88:0/3587579410 >> 172.21.15.88:0/3587579410 conn(0x7f88780ce660 msgr2=0x7f88780cea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:14.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.531+0000 7f887f46b640 1 -- 172.21.15.88:0/3587579410 shutdown_connections 2023-10-16T20:06:14.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.531+0000 7f887f46b640 1 -- 172.21.15.88:0/3587579410 wait complete. 2023-10-16T20:06:14.540 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.531+0000 7f887f46b640 1 Processor -- start 2023-10-16T20:06:14.540 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.531+0000 7f887f46b640 1 -- start start 2023-10-16T20:06:14.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.531+0000 7f887f46b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f887806a5a0 0x7f8878157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:14.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.531+0000 7f887f46b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88780c19e0 0x7f887815d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:14.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.531+0000 7f887f46b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f88780c8db0 0x7f887815da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:14.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.531+0000 7f887f46b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88780c6830 con 0x7f88780c19e0 2023-10-16T20:06:14.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.531+0000 7f887f46b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887815a980 con 0x7f887806a5a0 2023-10-16T20:06:14.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.531+0000 7f887f46b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88780c5e90 con 0x7f88780c8db0 2023-10-16T20:06:14.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.531+0000 7f887ca06640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88780c19e0 0x7f887815d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:14.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.535+0000 7f887ca06640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88780c19e0 0x7f887815d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45810/0 (socket says 172.21.15.88:45810) 2023-10-16T20:06:14.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.535+0000 7f887ca06640 1 -- 172.21.15.88:0/581906690 learned_addr learned my addr 172.21.15.88:0/581906690 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:14.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.535+0000 7f887d207640 1 --2- 172.21.15.88:0/581906690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f887806a5a0 0x7f8878157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:14.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.535+0000 7f887da08640 1 --2- 172.21.15.88:0/581906690 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f88780c8db0 0x7f887815da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:14.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.535+0000 7f887ca06640 1 -- 172.21.15.88:0/581906690 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f88780c8db0 msgr2=0x7f887815da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:14.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.535+0000 7f887ca06640 1 --2- 172.21.15.88:0/581906690 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f88780c8db0 0x7f887815da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:14.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.535+0000 7f887ca06640 1 -- 172.21.15.88:0/581906690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f887806a5a0 msgr2=0x7f8878157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:14.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.535+0000 7f887ca06640 1 --2- 172.21.15.88:0/581906690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f887806a5a0 0x7f8878157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:14.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.535+0000 7f887ca06640 1 -- 172.21.15.88:0/581906690 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8870009b30 con 0x7f88780c19e0 2023-10-16T20:06:14.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.535+0000 7f887ca06640 1 --2- 172.21.15.88:0/581906690 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88780c19e0 0x7f887815d500 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f886800c500 tx=0x7f886800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:14.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.535+0000 7f886dffb640 1 -- 172.21.15.88:0/581906690 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8868005220 con 0x7f88780c19e0 2023-10-16T20:06:14.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.535+0000 7f886dffb640 1 -- 172.21.15.88:0/581906690 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8868005380 con 0x7f88780c19e0 2023-10-16T20:06:14.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.535+0000 7f886dffb640 1 -- 172.21.15.88:0/581906690 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8868007270 con 0x7f88780c19e0 2023-10-16T20:06:14.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.535+0000 7f887f46b640 1 -- 172.21.15.88:0/581906690 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88780c9d70 con 0x7f88780c19e0 2023-10-16T20:06:14.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.535+0000 7f887f46b640 1 -- 172.21.15.88:0/581906690 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88780c6450 con 0x7f88780c19e0 2023-10-16T20:06:14.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.535+0000 7f887f46b640 1 -- 172.21.15.88:0/581906690 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88780c6970 con 0x7f88780c19e0 2023-10-16T20:06:14.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.535+0000 7f886dffb640 1 -- 172.21.15.88:0/581906690 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f88680073d0 con 0x7f88780c19e0 2023-10-16T20:06:14.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.535+0000 7f886dffb640 1 --2- 172.21.15.88:0/581906690 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f885c0664d0 0x7f885c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:14.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.535+0000 7f886dffb640 1 -- 172.21.15.88:0/581906690 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f8868083490 con 0x7f88780c19e0 2023-10-16T20:06:14.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.535+0000 7f887d207640 1 --2- 172.21.15.88:0/581906690 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f885c0664d0 0x7f885c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:14.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.539+0000 7f887d207640 1 --2- 172.21.15.88:0/581906690 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f885c0664d0 0x7f885c068990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8874009950 tx=0x7f8874008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:14.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.539+0000 7f886dffb640 1 -- 172.21.15.88:0/581906690 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f88780c6970 con 0x7f88780c19e0 2023-10-16T20:06:14.700 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:14.695+0000 7f887f46b640 1 -- 172.21.15.88:0/581906690 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "testcephfs1_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f88780c6970 con 0x7f88780c19e0 2023-10-16T20:06:15.699 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.695+0000 7f886dffb640 1 -- 172.21.15.88:0/581906690 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "testcephfs1_metadata", "pg_num_min": 64}]=0 pool 'testcephfs1_metadata' created v38) v1 ==== 148+0+0 (secure 0 0 0) 0x7f88780c6970 con 0x7f88780c19e0 2023-10-16T20:06:15.699 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'testcephfs1_metadata' created 2023-10-16T20:06:15.702 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.695+0000 7f887f46b640 1 -- 172.21.15.88:0/581906690 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f885c0664d0 msgr2=0x7f885c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:15.702 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.695+0000 7f887f46b640 1 --2- 172.21.15.88:0/581906690 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f885c0664d0 0x7f885c068990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8874009950 tx=0x7f8874008040 comp rx=0 tx=0).stop 2023-10-16T20:06:15.702 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.695+0000 7f887f46b640 1 -- 172.21.15.88:0/581906690 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88780c19e0 msgr2=0x7f887815d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:15.702 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.695+0000 7f887f46b640 1 --2- 172.21.15.88:0/581906690 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88780c19e0 0x7f887815d500 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f886800c500 tx=0x7f886800c9c0 comp rx=0 tx=0).stop 2023-10-16T20:06:15.703 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.695+0000 7f887f46b640 1 -- 172.21.15.88:0/581906690 shutdown_connections 2023-10-16T20:06:15.703 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.695+0000 7f887f46b640 1 --2- 172.21.15.88:0/581906690 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f885c0664d0 0x7f885c068990 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:15.703 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.695+0000 7f887f46b640 1 --2- 172.21.15.88:0/581906690 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f88780c8db0 0x7f887815da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:15.703 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.695+0000 7f887f46b640 1 --2- 172.21.15.88:0/581906690 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88780c19e0 0x7f887815d500 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:15.703 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.695+0000 7f887f46b640 1 --2- 172.21.15.88:0/581906690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f887806a5a0 0x7f8878157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:15.704 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.695+0000 7f887f46b640 1 -- 172.21.15.88:0/581906690 >> 172.21.15.88:0/581906690 conn(0x7f88780ce660 msgr2=0x7f8878069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:15.704 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.695+0000 7f887f46b640 1 -- 172.21.15.88:0/581906690 shutdown_connections 2023-10-16T20:06:15.704 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.695+0000 7f887f46b640 1 -- 172.21.15.88:0/581906690 wait complete. 2023-10-16T20:06:15.730 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create testcephfs1_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T20:06:15.878 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.871+0000 7fb393965640 1 Processor -- start 2023-10-16T20:06:15.879 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.871+0000 7fb393965640 1 -- start start 2023-10-16T20:06:15.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.875+0000 7fb393965640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb38c15ac00 0x7fb38c0b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:15.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.875+0000 7fb393965640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb38c0b2930 0x7fb38c06c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:15.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.875+0000 7fb393965640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb38c0b2d80 con 0x7fb38c15a820 2023-10-16T20:06:15.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.875+0000 7fb393965640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb38c0ce4b0 con 0x7fb38c15ac00 2023-10-16T20:06:15.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.875+0000 7fb393965640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c0cf8f0 con 0x7fb38c0b2930 2023-10-16T20:06:15.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.875+0000 7fb390f00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb38c15ac00 0x7fb38c0b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:15.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.875+0000 7fb390f00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb38c15ac00 0x7fb38c0b23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45824/0 (socket says 172.21.15.88:45824) 2023-10-16T20:06:15.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.875+0000 7fb390f00640 1 -- 172.21.15.88:0/544144781 learned_addr learned my addr 172.21.15.88:0/544144781 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:15.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.875+0000 7fb391f02640 1 --2- 172.21.15.88:0/544144781 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb38c0b2930 0x7fb38c06c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:15.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.875+0000 7fb390f00640 1 -- 172.21.15.88:0/544144781 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb38c0b2930 msgr2=0x7fb38c06c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:15.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.875+0000 7fb390f00640 1 --2- 172.21.15.88:0/544144781 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb38c0b2930 0x7fb38c06c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:15.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.875+0000 7fb390f00640 1 -- 172.21.15.88:0/544144781 >> v1:172.21.15.94:6789/0 conn(0x7fb38c15a820 legacy=0x7fb38c0cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:15.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.875+0000 7fb390f00640 1 -- 172.21.15.88:0/544144781 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb38c15e270 con 0x7fb38c15ac00 2023-10-16T20:06:15.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.879+0000 7fb390f00640 1 --2- 172.21.15.88:0/544144781 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb38c15ac00 0x7fb38c0b23f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb37c009870 tx=0x7fb37c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80aa9ffb0131ff14 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:15.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.879+0000 7fb383fff640 1 -- 172.21.15.88:0/544144781 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c0aa670 con 0x7fb38c15ac00 2023-10-16T20:06:15.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.879+0000 7fb383fff640 1 -- 172.21.15.88:0/544144781 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37c0aa7d0 con 0x7fb38c15ac00 2023-10-16T20:06:15.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.879+0000 7fb383fff640 1 -- 172.21.15.88:0/544144781 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c0aab60 con 0x7fb38c15ac00 2023-10-16T20:06:15.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.879+0000 7fb393965640 1 -- 172.21.15.88:0/544144781 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb38c15ac00 msgr2=0x7fb38c0b23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:15.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.879+0000 7fb393965640 1 --2- 172.21.15.88:0/544144781 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb38c15ac00 0x7fb38c0b23f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb37c009870 tx=0x7fb37c01c510 comp rx=0 tx=0).stop 2023-10-16T20:06:15.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.879+0000 7fb393965640 1 -- 172.21.15.88:0/544144781 shutdown_connections 2023-10-16T20:06:15.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.879+0000 7fb393965640 1 --2- 172.21.15.88:0/544144781 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb38c0b2930 0x7fb38c06c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:15.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.879+0000 7fb393965640 1 --2- 172.21.15.88:0/544144781 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb38c15ac00 0x7fb38c0b23f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:15.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.879+0000 7fb393965640 1 -- 172.21.15.88:0/544144781 >> 172.21.15.88:0/544144781 conn(0x7fb38c0cd3f0 msgr2=0x7fb38c0cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:15.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.879+0000 7fb393965640 1 -- 172.21.15.88:0/544144781 shutdown_connections 2023-10-16T20:06:15.887 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.879+0000 7fb393965640 1 -- 172.21.15.88:0/544144781 wait complete. 2023-10-16T20:06:15.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.879+0000 7fb393965640 1 Processor -- start 2023-10-16T20:06:15.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.879+0000 7fb393965640 1 -- start start 2023-10-16T20:06:15.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.883+0000 7fb393965640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb38c0b2930 0x7fb38c0c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:15.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.883+0000 7fb393965640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb38c15a820 0x7fb38c1561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:15.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.883+0000 7fb393965640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb38c15ac00 0x7fb38c152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:15.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.883+0000 7fb393965640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c0c4410 con 0x7fb38c15a820 2023-10-16T20:06:15.891 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.883+0000 7fb393965640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c1554f0 con 0x7fb38c15ac00 2023-10-16T20:06:15.891 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.883+0000 7fb393965640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c1557d0 con 0x7fb38c0b2930 2023-10-16T20:06:15.891 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.883+0000 7fb390f00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb38c15a820 0x7fb38c1561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:15.891 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.883+0000 7fb390f00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb38c15a820 0x7fb38c1561e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45832/0 (socket says 172.21.15.88:45832) 2023-10-16T20:06:15.891 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.883+0000 7fb390f00640 1 -- 172.21.15.88:0/854585662 learned_addr learned my addr 172.21.15.88:0/854585662 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:15.892 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.883+0000 7fb391701640 1 --2- 172.21.15.88:0/854585662 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb38c0b2930 0x7fb38c0c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:15.892 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.883+0000 7fb391f02640 1 --2- 172.21.15.88:0/854585662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb38c15ac00 0x7fb38c152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:15.892 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.883+0000 7fb390f00640 1 -- 172.21.15.88:0/854585662 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb38c0b2930 msgr2=0x7fb38c0c3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:15.892 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.883+0000 7fb390f00640 1 --2- 172.21.15.88:0/854585662 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb38c0b2930 0x7fb38c0c3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:15.892 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.883+0000 7fb390f00640 1 -- 172.21.15.88:0/854585662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb38c15ac00 msgr2=0x7fb38c152640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:15.893 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.883+0000 7fb390f00640 1 --2- 172.21.15.88:0/854585662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb38c15ac00 0x7fb38c152640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:15.893 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.883+0000 7fb390f00640 1 -- 172.21.15.88:0/854585662 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb37c0094d0 con 0x7fb38c15a820 2023-10-16T20:06:15.893 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.883+0000 7fb391f02640 1 --2- 172.21.15.88:0/854585662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb38c15ac00 0x7fb38c152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:15.893 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.883+0000 7fb390f00640 1 --2- 172.21.15.88:0/854585662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb38c15a820 0x7fb38c1561e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb37c01a140 tx=0x7fb37c01ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:15.893 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.883+0000 7fb381ffb640 1 -- 172.21.15.88:0/854585662 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c0b3800 con 0x7fb38c15a820 2023-10-16T20:06:15.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.883+0000 7fb381ffb640 1 -- 172.21.15.88:0/854585662 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37c0b3980 con 0x7fb38c15a820 2023-10-16T20:06:15.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.883+0000 7fb393965640 1 -- 172.21.15.88:0/854585662 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb38c0c4660 con 0x7fb38c15a820 2023-10-16T20:06:15.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.883+0000 7fb381ffb640 1 -- 172.21.15.88:0/854585662 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c0b3d30 con 0x7fb38c15a820 2023-10-16T20:06:15.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.883+0000 7fb393965640 1 -- 172.21.15.88:0/854585662 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb38c155130 con 0x7fb38c15a820 2023-10-16T20:06:15.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.883+0000 7fb393965640 1 -- 172.21.15.88:0/854585662 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb38c04e9c0 con 0x7fb38c15a820 2023-10-16T20:06:15.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.883+0000 7fb381ffb640 1 -- 172.21.15.88:0/854585662 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb37c0c7420 con 0x7fb38c15a820 2023-10-16T20:06:15.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.887+0000 7fb381ffb640 1 --2- 172.21.15.88:0/854585662 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb3700664a0 0x7fb370068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:15.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.887+0000 7fb381ffb640 1 -- 172.21.15.88:0/854585662 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5552+0+0 (secure 0 0 0) 0x7fb37c11e8b0 con 0x7fb38c15a820 2023-10-16T20:06:15.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.887+0000 7fb391701640 1 --2- 172.21.15.88:0/854585662 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb3700664a0 0x7fb370068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:15.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.887+0000 7fb391701640 1 --2- 172.21.15.88:0/854585662 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb3700664a0 0x7fb370068960 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb388007fd0 tx=0x7fb388009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:15.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:15.891+0000 7fb381ffb640 1 -- 172.21.15.88:0/854585662 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb37c0ebf80 con 0x7fb38c15a820 2023-10-16T20:06:16.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.043+0000 7fb393965640 1 -- 172.21.15.88:0/854585662 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "testcephfs1_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb38c0cfdb0 con 0x7fb38c15a820 2023-10-16T20:06:16.703 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.699+0000 7fb381ffb640 1 -- 172.21.15.88:0/854585662 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "testcephfs1_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'testcephfs1_data' created v39) v1 ==== 180+0+0 (secure 0 0 0) 0x7fb37c0ef5a0 con 0x7fb38c15a820 2023-10-16T20:06:16.703 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'testcephfs1_data' created 2023-10-16T20:06:16.705 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.699+0000 7fb393965640 1 -- 172.21.15.88:0/854585662 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb3700664a0 msgr2=0x7fb370068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:16.706 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.699+0000 7fb393965640 1 --2- 172.21.15.88:0/854585662 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb3700664a0 0x7fb370068960 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb388007fd0 tx=0x7fb388009040 comp rx=0 tx=0).stop 2023-10-16T20:06:16.706 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.699+0000 7fb393965640 1 -- 172.21.15.88:0/854585662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb38c15a820 msgr2=0x7fb38c1561e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:16.706 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.699+0000 7fb393965640 1 --2- 172.21.15.88:0/854585662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb38c15a820 0x7fb38c1561e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb37c01a140 tx=0x7fb37c01ad50 comp rx=0 tx=0).stop 2023-10-16T20:06:16.707 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.699+0000 7fb393965640 1 -- 172.21.15.88:0/854585662 shutdown_connections 2023-10-16T20:06:16.707 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.699+0000 7fb393965640 1 --2- 172.21.15.88:0/854585662 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb3700664a0 0x7fb370068960 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:16.707 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.699+0000 7fb393965640 1 --2- 172.21.15.88:0/854585662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb38c15ac00 0x7fb38c152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:16.707 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.699+0000 7fb393965640 1 --2- 172.21.15.88:0/854585662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb38c15a820 0x7fb38c1561e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:16.708 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.699+0000 7fb393965640 1 --2- 172.21.15.88:0/854585662 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb38c0b2930 0x7fb38c0c3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:16.708 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.699+0000 7fb393965640 1 -- 172.21.15.88:0/854585662 >> 172.21.15.88:0/854585662 conn(0x7fb38c0cd3f0 msgr2=0x7fb38c15c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:16.708 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.699+0000 7fb393965640 1 -- 172.21.15.88:0/854585662 shutdown_connections 2023-10-16T20:06:16.708 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.699+0000 7fb393965640 1 -- 172.21.15.88:0/854585662 wait complete. 2023-10-16T20:06:16.745 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new testcephfs1 testcephfs1_metadata testcephfs1_data 2023-10-16T20:06:16.887 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.879+0000 7fad5f33e640 1 Processor -- start 2023-10-16T20:06:16.887 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.879+0000 7fad5f33e640 1 -- start start 2023-10-16T20:06:16.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.883+0000 7fad5f33e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad581500a0 0x7fad58159f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:16.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.883+0000 7fad5f33e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fad580711f0 0x7fad5815aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:16.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.883+0000 7fad5f33e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad5815aa10 con 0x7fad5806c2e0 2023-10-16T20:06:16.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.883+0000 7fad5f33e640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad58150480 con 0x7fad580711f0 2023-10-16T20:06:16.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.883+0000 7fad5f33e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5815a490 con 0x7fad581500a0 2023-10-16T20:06:16.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.883+0000 7fad5d0da640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad581500a0 0x7fad58159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:16.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.883+0000 7fad5d8db640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fad580711f0 0x7fad5815aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:16.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.883+0000 7fad5d0da640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad581500a0 0x7fad58159f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:35966/0 (socket says 172.21.15.88:35966) 2023-10-16T20:06:16.891 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.883+0000 7fad5d0da640 1 -- 172.21.15.88:0/3906616361 learned_addr learned my addr 172.21.15.88:0/3906616361 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:16.891 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.883+0000 7fad5c8d9640 1 --1- 172.21.15.88:0/3906616361 >> v1:172.21.15.94:6789/0 conn(0x7fad5806c2e0 0x7fad58070ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:33238/0 (socket says 172.21.15.88:33238) 2023-10-16T20:06:16.891 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.883+0000 7fad5d0da640 1 -- 172.21.15.88:0/3906616361 >> v2:172.21.15.94:3300/0 conn(0x7fad580711f0 msgr2=0x7fad5815aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:16.891 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.883+0000 7fad5d0da640 1 --2- 172.21.15.88:0/3906616361 >> v2:172.21.15.94:3300/0 conn(0x7fad580711f0 0x7fad5815aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:16.892 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.883+0000 7fad5d0da640 1 -- 172.21.15.88:0/3906616361 >> v1:172.21.15.94:6789/0 conn(0x7fad5806c2e0 legacy=0x7fad58070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:16.892 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.883+0000 7fad5d0da640 1 -- 172.21.15.88:0/3906616361 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5806ca10 con 0x7fad581500a0 2023-10-16T20:06:16.893 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.887+0000 7fad5d0da640 1 --2- 172.21.15.88:0/3906616361 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad581500a0 0x7fad58159f50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fad5400bb10 tx=0x7fad5401ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9aa4852c63b6ae88 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:16.893 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.887+0000 7fad4ffff640 1 -- 172.21.15.88:0/3906616361 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad540a7b50 con 0x7fad581500a0 2023-10-16T20:06:16.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.887+0000 7fad4ffff640 1 -- 172.21.15.88:0/3906616361 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad540a7cb0 con 0x7fad581500a0 2023-10-16T20:06:16.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.887+0000 7fad4ffff640 1 -- 172.21.15.88:0/3906616361 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad540af660 con 0x7fad581500a0 2023-10-16T20:06:16.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.887+0000 7fad5f33e640 1 -- 172.21.15.88:0/3906616361 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad581500a0 msgr2=0x7fad58159f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:16.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.887+0000 7fad5f33e640 1 --2- 172.21.15.88:0/3906616361 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad581500a0 0x7fad58159f50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fad5400bb10 tx=0x7fad5401ec10 comp rx=0 tx=0).stop 2023-10-16T20:06:16.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.887+0000 7fad5f33e640 1 -- 172.21.15.88:0/3906616361 shutdown_connections 2023-10-16T20:06:16.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.887+0000 7fad5f33e640 1 --2- 172.21.15.88:0/3906616361 >> v2:172.21.15.94:3300/0 conn(0x7fad580711f0 0x7fad5815aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:16.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.887+0000 7fad5f33e640 1 --2- 172.21.15.88:0/3906616361 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad581500a0 0x7fad58159f50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:16.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.887+0000 7fad5f33e640 1 -- 172.21.15.88:0/3906616361 >> 172.21.15.88:0/3906616361 conn(0x7fad580cb080 msgr2=0x7fad580cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:16.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.887+0000 7fad5f33e640 1 -- 172.21.15.88:0/3906616361 shutdown_connections 2023-10-16T20:06:16.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.887+0000 7fad5f33e640 1 -- 172.21.15.88:0/3906616361 wait complete. 2023-10-16T20:06:16.897 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.887+0000 7fad5f33e640 1 Processor -- start 2023-10-16T20:06:16.898 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.887+0000 7fad5f33e640 1 -- start start 2023-10-16T20:06:16.898 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.891+0000 7fad5f33e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad5806c2e0 0x7fad58154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:16.898 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.891+0000 7fad5f33e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad580711f0 0x7fad580b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:16.898 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.891+0000 7fad5f33e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad581500a0 0x7fad580b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:16.898 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.891+0000 7fad5f33e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad58153b60 con 0x7fad5806c2e0 2023-10-16T20:06:16.899 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.891+0000 7fad5f33e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad58153e40 con 0x7fad581500a0 2023-10-16T20:06:16.899 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.891+0000 7fad5f33e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad58154120 con 0x7fad580711f0 2023-10-16T20:06:16.899 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.891+0000 7fad5d0da640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad5806c2e0 0x7fad58154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:16.899 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.891+0000 7fad5d0da640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad5806c2e0 0x7fad58154c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45842/0 (socket says 172.21.15.88:45842) 2023-10-16T20:06:16.899 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.891+0000 7fad5d0da640 1 -- 172.21.15.88:0/1791290621 learned_addr learned my addr 172.21.15.88:0/1791290621 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:16.900 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.891+0000 7fad5d8db640 1 --2- 172.21.15.88:0/1791290621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad581500a0 0x7fad580b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:16.900 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.891+0000 7fad5c8d9640 1 --2- 172.21.15.88:0/1791290621 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad580711f0 0x7fad580b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:16.900 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.891+0000 7fad5d0da640 1 -- 172.21.15.88:0/1791290621 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad580711f0 msgr2=0x7fad580b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:16.900 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.891+0000 7fad5d0da640 1 --2- 172.21.15.88:0/1791290621 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad580711f0 0x7fad580b3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:16.900 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.891+0000 7fad5d0da640 1 -- 172.21.15.88:0/1791290621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad581500a0 msgr2=0x7fad580b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:16.901 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.891+0000 7fad5d0da640 1 --2- 172.21.15.88:0/1791290621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad581500a0 0x7fad580b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:16.901 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.891+0000 7fad5d0da640 1 -- 172.21.15.88:0/1791290621 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5400b770 con 0x7fad5806c2e0 2023-10-16T20:06:16.901 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.891+0000 7fad5d8db640 1 --2- 172.21.15.88:0/1791290621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad581500a0 0x7fad580b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:06:16.901 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.891+0000 7fad5d0da640 1 --2- 172.21.15.88:0/1791290621 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad5806c2e0 0x7fad58154c00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fad5400b2c0 tx=0x7fad5401d450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:16.901 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.891+0000 7fad4dffb640 1 -- 172.21.15.88:0/1791290621 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad540afe30 con 0x7fad5806c2e0 2023-10-16T20:06:16.902 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.891+0000 7fad4dffb640 1 -- 172.21.15.88:0/1791290621 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad540ba480 con 0x7fad5806c2e0 2023-10-16T20:06:16.902 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.891+0000 7fad4dffb640 1 -- 172.21.15.88:0/1791290621 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad540ba830 con 0x7fad5806c2e0 2023-10-16T20:06:16.902 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.891+0000 7fad5f33e640 1 -- 172.21.15.88:0/1791290621 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad58154400 con 0x7fad5806c2e0 2023-10-16T20:06:16.902 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.891+0000 7fad5f33e640 1 -- 172.21.15.88:0/1791290621 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad58143670 con 0x7fad5806c2e0 2023-10-16T20:06:16.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.891+0000 7fad5f33e640 1 -- 172.21.15.88:0/1791290621 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad58143860 con 0x7fad5806c2e0 2023-10-16T20:06:16.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.895+0000 7fad4dffb640 1 -- 172.21.15.88:0/1791290621 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fad540ba990 con 0x7fad5806c2e0 2023-10-16T20:06:16.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.895+0000 7fad4dffb640 1 --2- 172.21.15.88:0/1791290621 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fad3c0664a0 0x7fad3c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:16.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.895+0000 7fad4dffb640 1 -- 172.21.15.88:0/1791290621 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fad5411f430 con 0x7fad5806c2e0 2023-10-16T20:06:16.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.895+0000 7fad5c8d9640 1 --2- 172.21.15.88:0/1791290621 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fad3c0664a0 0x7fad3c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:16.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.895+0000 7fad5c8d9640 1 --2- 172.21.15.88:0/1791290621 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fad3c0664a0 0x7fad3c068960 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fad580b32d0 tx=0x7fad48007e50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:16.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:16.899+0000 7fad4dffb640 1 -- 172.21.15.88:0/1791290621 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fad540ec7d0 con 0x7fad5806c2e0 2023-10-16T20:06:17.087 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.083+0000 7fad5f33e640 1 -- 172.21.15.88:0/1791290621 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "testcephfs1", "metadata": "testcephfs1_metadata", "data": "testcephfs1_data"} v 0) v1 -- 0x7fad5804e9c0 con 0x7fad5806c2e0 2023-10-16T20:06:17.718 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.711+0000 7fad4dffb640 1 -- 172.21.15.88:0/1791290621 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "testcephfs1", "metadata": "testcephfs1_metadata", "data": "testcephfs1_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) v1 ==== 187+0+0 (secure 0 0 0) 0x7fad540b40d0 con 0x7fad5806c2e0 2023-10-16T20:06:17.718 INFO:teuthology.orchestra.run.smithi088.stderr:new fs with metadata pool 5 and data pool 6 2023-10-16T20:06:17.720 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.715+0000 7fad5f33e640 1 -- 172.21.15.88:0/1791290621 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fad3c0664a0 msgr2=0x7fad3c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:17.720 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.715+0000 7fad5f33e640 1 --2- 172.21.15.88:0/1791290621 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fad3c0664a0 0x7fad3c068960 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fad580b32d0 tx=0x7fad48007e50 comp rx=0 tx=0).stop 2023-10-16T20:06:17.721 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.715+0000 7fad5f33e640 1 -- 172.21.15.88:0/1791290621 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad5806c2e0 msgr2=0x7fad58154c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:17.721 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.715+0000 7fad5f33e640 1 --2- 172.21.15.88:0/1791290621 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad5806c2e0 0x7fad58154c00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fad5400b2c0 tx=0x7fad5401d450 comp rx=0 tx=0).stop 2023-10-16T20:06:17.721 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.715+0000 7fad5f33e640 1 -- 172.21.15.88:0/1791290621 shutdown_connections 2023-10-16T20:06:17.721 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.715+0000 7fad5f33e640 1 --2- 172.21.15.88:0/1791290621 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fad3c0664a0 0x7fad3c068960 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:17.721 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.715+0000 7fad5f33e640 1 --2- 172.21.15.88:0/1791290621 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fad580711f0 0x7fad580b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:17.722 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.715+0000 7fad5f33e640 1 --2- 172.21.15.88:0/1791290621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad581500a0 0x7fad580b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:17.722 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.715+0000 7fad5f33e640 1 --2- 172.21.15.88:0/1791290621 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad5806c2e0 0x7fad58154c00 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:17.722 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.715+0000 7fad5f33e640 1 -- 172.21.15.88:0/1791290621 >> 172.21.15.88:0/1791290621 conn(0x7fad580cb080 msgr2=0x7fad5814f020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:17.722 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.715+0000 7fad5f33e640 1 -- 172.21.15.88:0/1791290621 shutdown_connections 2023-10-16T20:06:17.722 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.715+0000 7fad5f33e640 1 -- 172.21.15.88:0/1791290621 wait complete. 2023-10-16T20:06:17.746 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:06:17.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.879+0000 7f3695794640 1 Processor -- start 2023-10-16T20:06:17.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.879+0000 7f3695794640 1 -- start start 2023-10-16T20:06:17.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.879+0000 7f3695794640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f36901570c0 0x7f36901612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:17.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.879+0000 7f3695794640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f36900cd950 0x7f36900cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:17.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.879+0000 7f3695794640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36900ce2e0 con 0x7f36901574a0 2023-10-16T20:06:17.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.879+0000 7f3695794640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36901608b0 con 0x7f36901570c0 2023-10-16T20:06:17.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.879+0000 7f3695794640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3690161800 con 0x7f36900cd950 2023-10-16T20:06:17.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.879+0000 7f36867fc640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f36901574a0 0x7f369015d2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:36790/0 (socket says 172.21.15.88:36790) 2023-10-16T20:06:17.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.879+0000 7f36867fc640 1 -- 172.21.15.88:0/3747830195 learned_addr learned my addr 172.21.15.88:0/3747830195 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:17.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.879+0000 7f368effd640 1 --2- 172.21.15.88:0/3747830195 >> v2:172.21.15.94:3300/0 conn(0x7f36901570c0 0x7f36901612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:17.891 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.879+0000 7f368f7fe640 1 --2- 172.21.15.88:0/3747830195 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f36900cd950 0x7f36900cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:17.891 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.879+0000 7f368e7fc640 1 -- 172.21.15.88:0/3747830195 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2352482188 0 0) 0x7f3678001120 con 0x7f36901574a0 2023-10-16T20:06:17.891 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.879+0000 7f368e7fc640 1 -- 172.21.15.88:0/3747830195 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f367c003530 con 0x7f36901574a0 2023-10-16T20:06:17.892 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.879+0000 7f368e7fc640 1 -- 172.21.15.88:0/3747830195 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2888360189 0 0) 0x7f367c003530 con 0x7f36901574a0 2023-10-16T20:06:17.892 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.883+0000 7f368e7fc640 1 -- 172.21.15.88:0/3747830195 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f36900cd950 msgr2=0x7f36900cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:17.892 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.883+0000 7f368e7fc640 1 --2- 172.21.15.88:0/3747830195 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f36900cd950 0x7f36900cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:17.893 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.883+0000 7f368e7fc640 1 -- 172.21.15.88:0/3747830195 >> v2:172.21.15.94:3300/0 conn(0x7f36901570c0 msgr2=0x7f36901612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:17.893 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.883+0000 7f368e7fc640 1 --2- 172.21.15.88:0/3747830195 >> v2:172.21.15.94:3300/0 conn(0x7f36901570c0 0x7f36901612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:17.893 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.883+0000 7f368e7fc640 1 -- 172.21.15.88:0/3747830195 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f369015ddd0 con 0x7f36901574a0 2023-10-16T20:06:17.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.883+0000 7f368e7fc640 1 -- 172.21.15.88:0/3747830195 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f3678001830 con 0x7f36901574a0 2023-10-16T20:06:17.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.883+0000 7f368e7fc640 1 -- 172.21.15.88:0/3747830195 >> v1:172.21.15.88:6789/0 conn(0x7f36901574a0 legacy=0x7f369015d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:17.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.883+0000 7f368e7fc640 1 --2- 172.21.15.88:0/3747830195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f367c003980 0x7f367c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:17.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.883+0000 7f368e7fc640 1 --2- 172.21.15.88:0/3747830195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f367c0047d0 0x7f367c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:17.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.883+0000 7f368e7fc640 1 --2- 172.21.15.88:0/3747830195 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f36900cd950 0x7f367c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:17.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.883+0000 7f368e7fc640 1 -- 172.21.15.88:0/3747830195 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36901608b0 con 0x7f367c003980 2023-10-16T20:06:17.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.883+0000 7f368e7fc640 1 -- 172.21.15.88:0/3747830195 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3690161800 con 0x7f367c0047d0 2023-10-16T20:06:17.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.883+0000 7f368e7fc640 1 -- 172.21.15.88:0/3747830195 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f367c00b990 con 0x7f36900cd950 2023-10-16T20:06:17.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.883+0000 7f368effd640 1 --2- 172.21.15.88:0/3747830195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f367c003980 0x7f367c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:17.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.883+0000 7f36867fc640 1 --2- 172.21.15.88:0/3747830195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f367c0047d0 0x7f367c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:17.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.883+0000 7f368f7fe640 1 --2- 172.21.15.88:0/3747830195 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f36900cd950 0x7f367c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:17.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.883+0000 7f368effd640 1 -- 172.21.15.88:0/3747830195 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f36900cd950 msgr2=0x7f367c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:17.897 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.883+0000 7f368effd640 1 --2- 172.21.15.88:0/3747830195 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f36900cd950 0x7f367c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:17.897 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.883+0000 7f368effd640 1 -- 172.21.15.88:0/3747830195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f367c0047d0 msgr2=0x7f367c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:17.897 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.883+0000 7f368effd640 1 --2- 172.21.15.88:0/3747830195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f367c0047d0 0x7f367c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:17.897 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.883+0000 7f368effd640 1 -- 172.21.15.88:0/3747830195 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f367c003530 con 0x7f367c003980 2023-10-16T20:06:17.901 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.883+0000 7f368effd640 1 --2- 172.21.15.88:0/3747830195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f367c003980 0x7f367c003da0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3680009ae0 tx=0x7f368001ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd7df48a7a8f0338 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:17.901 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.883+0000 7f368e7fc640 1 -- 172.21.15.88:0/3747830195 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36800a6470 con 0x7f367c003980 2023-10-16T20:06:17.901 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.883+0000 7f368e7fc640 1 -- 172.21.15.88:0/3747830195 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36800a65d0 con 0x7f367c003980 2023-10-16T20:06:17.902 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.883+0000 7f368e7fc640 1 -- 172.21.15.88:0/3747830195 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36800a6980 con 0x7f367c003980 2023-10-16T20:06:17.902 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.887+0000 7f3695794640 1 -- 172.21.15.88:0/3747830195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f367c003980 msgr2=0x7f367c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:17.902 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.887+0000 7f3695794640 1 --2- 172.21.15.88:0/3747830195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f367c003980 0x7f367c003da0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3680009ae0 tx=0x7f368001ca60 comp rx=0 tx=0).stop 2023-10-16T20:06:17.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.887+0000 7f368f7fe640 1 -- 172.21.15.88:0/3747830195 reap_dead start 2023-10-16T20:06:17.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.887+0000 7f3695794640 1 -- 172.21.15.88:0/3747830195 shutdown_connections 2023-10-16T20:06:17.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.887+0000 7f3695794640 1 -- 172.21.15.88:0/3747830195 >> 172.21.15.88:0/3747830195 conn(0x7f36900715c0 msgr2=0x7f36900719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:17.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.887+0000 7f3695794640 1 -- 172.21.15.88:0/3747830195 shutdown_connections 2023-10-16T20:06:17.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.887+0000 7f3695794640 1 -- 172.21.15.88:0/3747830195 wait complete. 2023-10-16T20:06:17.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.887+0000 7f3695794640 1 Processor -- start 2023-10-16T20:06:17.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.887+0000 7f3695794640 1 -- start start 2023-10-16T20:06:17.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.887+0000 7f3695794640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f367c003980 0x7f369014fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:17.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.887+0000 7f3695794640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36900caad0 0x7f369015e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:17.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.887+0000 7f3695794640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f369014ece0 0x7f369014f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:17.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.887+0000 7f3695794640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36900caf50 con 0x7f36900caad0 2023-10-16T20:06:17.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.887+0000 7f3695794640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369014f6d0 con 0x7f369014ece0 2023-10-16T20:06:17.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.887+0000 7f3695794640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36900ccaa0 con 0x7f367c003980 2023-10-16T20:06:17.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.887+0000 7f36867fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36900caad0 0x7f369015e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:17.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.887+0000 7f36867fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36900caad0 0x7f369015e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45850/0 (socket says 172.21.15.88:45850) 2023-10-16T20:06:17.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.887+0000 7f36867fc640 1 -- 172.21.15.88:0/3611551583 learned_addr learned my addr 172.21.15.88:0/3611551583 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:17.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.887+0000 7f368effd640 1 --2- 172.21.15.88:0/3611551583 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f367c003980 0x7f369014fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:17.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.887+0000 7f368f7fe640 1 --2- 172.21.15.88:0/3611551583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f369014ece0 0x7f369014f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:17.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.891+0000 7f36867fc640 1 -- 172.21.15.88:0/3611551583 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f367c003980 msgr2=0x7f369014fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:17.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.891+0000 7f36867fc640 1 --2- 172.21.15.88:0/3611551583 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f367c003980 0x7f369014fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:17.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.891+0000 7f36867fc640 1 -- 172.21.15.88:0/3611551583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f369014ece0 msgr2=0x7f369014f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:17.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.891+0000 7f36867fc640 1 --2- 172.21.15.88:0/3611551583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f369014ece0 0x7f369014f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:17.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.891+0000 7f36867fc640 1 -- 172.21.15.88:0/3611551583 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36900cc4e0 con 0x7f36900caad0 2023-10-16T20:06:17.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.891+0000 7f36867fc640 1 --2- 172.21.15.88:0/3611551583 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36900caad0 0x7f369015e670 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3678004f60 tx=0x7f3678004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:17.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.891+0000 7f3687fff640 1 -- 172.21.15.88:0/3611551583 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367800f6d0 con 0x7f36900caad0 2023-10-16T20:06:17.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.891+0000 7f3687fff640 1 -- 172.21.15.88:0/3611551583 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367800f830 con 0x7f36900caad0 2023-10-16T20:06:17.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.891+0000 7f3687fff640 1 -- 172.21.15.88:0/3611551583 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367800fbe0 con 0x7f36900caad0 2023-10-16T20:06:17.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.891+0000 7f3695794640 1 -- 172.21.15.88:0/3611551583 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36900cc7c0 con 0x7f36900caad0 2023-10-16T20:06:17.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.891+0000 7f3695794640 1 -- 172.21.15.88:0/3611551583 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36900c7a70 con 0x7f36900caad0 2023-10-16T20:06:17.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.891+0000 7f3695794640 1 -- 172.21.15.88:0/3611551583 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f369004e9c0 con 0x7f36900caad0 2023-10-16T20:06:17.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.891+0000 7f3687fff640 1 -- 172.21.15.88:0/3611551583 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f367800fd40 con 0x7f36900caad0 2023-10-16T20:06:17.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.895+0000 7f3687fff640 1 --2- 172.21.15.88:0/3611551583 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f36700664d0 0x7f3670068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:17.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.895+0000 7f3687fff640 1 -- 172.21.15.88:0/3611551583 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f36780833f0 con 0x7f36900caad0 2023-10-16T20:06:17.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.895+0000 7f368effd640 1 --2- 172.21.15.88:0/3611551583 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f36700664d0 0x7f3670068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:17.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.895+0000 7f368effd640 1 --2- 172.21.15.88:0/3611551583 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f36700664d0 0x7f3670068990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f36800ad000 tx=0x7f36800b2040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:17.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:17.899+0000 7f3687fff640 1 -- 172.21.15.88:0/3611551583 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f36780507c0 con 0x7f36900caad0 2023-10-16T20:06:18.056 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.051+0000 7f3695794640 1 -- 172.21.15.88:0/3611551583 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36900ce2e0 con 0x7f36900caad0 2023-10-16T20:06:18.057 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.051+0000 7f3687fff640 1 -- 172.21.15.88:0/3611551583 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16735 (secure 0 0 0) 0x7f36900ce2e0 con 0x7f36900caad0 2023-10-16T20:06:18.057 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:06:18.058 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":40,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:06:17.088057+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T20:06:14.743500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"testcephfs1_data","create_time":"2023-10-16T20:06:16.091837+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:06:18.059 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.051+0000 7f3695794640 1 -- 172.21.15.88:0/3611551583 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f36700664d0 msgr2=0x7f3670068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:18.059 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.051+0000 7f3695794640 1 --2- 172.21.15.88:0/3611551583 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f36700664d0 0x7f3670068990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f36800ad000 tx=0x7f36800b2040 comp rx=0 tx=0).stop 2023-10-16T20:06:18.059 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.051+0000 7f3695794640 1 -- 172.21.15.88:0/3611551583 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36900caad0 msgr2=0x7f369015e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:18.060 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.051+0000 7f3695794640 1 --2- 172.21.15.88:0/3611551583 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36900caad0 0x7f369015e670 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3678004f60 tx=0x7f3678004f90 comp rx=0 tx=0).stop 2023-10-16T20:06:18.060 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.051+0000 7f3695794640 1 -- 172.21.15.88:0/3611551583 shutdown_connections 2023-10-16T20:06:18.060 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.051+0000 7f3695794640 1 --2- 172.21.15.88:0/3611551583 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f36700664d0 0x7f3670068990 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:18.060 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.051+0000 7f3695794640 1 --2- 172.21.15.88:0/3611551583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f369014ece0 0x7f369014f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:18.060 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.051+0000 7f3695794640 1 --2- 172.21.15.88:0/3611551583 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36900caad0 0x7f369015e670 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:18.061 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.051+0000 7f3695794640 1 --2- 172.21.15.88:0/3611551583 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f367c003980 0x7f369014fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:18.061 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.055+0000 7f3695794640 1 -- 172.21.15.88:0/3611551583 >> 172.21.15.88:0/3611551583 conn(0x7f36900715c0 msgr2=0x7f36900cf3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:18.061 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.055+0000 7f3695794640 1 -- 172.21.15.88:0/3611551583 shutdown_connections 2023-10-16T20:06:18.061 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.055+0000 7f3695794640 1 -- 172.21.15.88:0/3611551583 wait complete. 2023-10-16T20:06:18.078 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:06:18.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.211+0000 7fbca629c640 1 Processor -- start 2023-10-16T20:06:18.217 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.211+0000 7fbca629c640 1 -- start start 2023-10-16T20:06:18.218 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.211+0000 7fbca629c640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fbca00d22e0 0x7fbca0161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:18.218 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.211+0000 7fbca629c640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbca00b87e0 con 0x7fbca00d26c0 2023-10-16T20:06:18.218 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.211+0000 7fbca629c640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbca015df10 con 0x7fbca015d9f0 2023-10-16T20:06:18.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.211+0000 7fbca629c640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbca015e1f0 con 0x7fbca00d22e0 2023-10-16T20:06:18.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.211+0000 7fbc9f7fe640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fbca00d26c0 0x7fbca015d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:36804/0 (socket says 172.21.15.88:36804) 2023-10-16T20:06:18.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.211+0000 7fbc9f7fe640 1 -- 172.21.15.88:0/2206117392 learned_addr learned my addr 172.21.15.88:0/2206117392 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:18.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.211+0000 7fbc9ffff640 1 --2- 172.21.15.88:0/2206117392 >> v2:172.21.15.94:3300/0 conn(0x7fbca00d22e0 0x7fbca0161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:18.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.211+0000 7fbc9effd640 1 -- 172.21.15.88:0/2206117392 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 352635323 0 0) 0x7fbca00b87e0 con 0x7fbca00d26c0 2023-10-16T20:06:18.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.211+0000 7fbc9effd640 1 -- 172.21.15.88:0/2206117392 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc88003530 con 0x7fbca00d26c0 2023-10-16T20:06:18.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.211+0000 7fbc9ffff640 1 -- 172.21.15.88:0/2206117392 >> v1:172.21.15.94:6789/0 conn(0x7fbca015d9f0 legacy=0x7fbca00cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:18.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.211+0000 7fbc9ffff640 1 -- 172.21.15.88:0/2206117392 >> v1:172.21.15.88:6789/0 conn(0x7fbca00d26c0 legacy=0x7fbca015d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:18.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.211+0000 7fbc9effd640 1 -- 172.21.15.88:0/2206117392 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 635377840 0 0) 0x7fbc88003530 con 0x7fbca00d26c0 2023-10-16T20:06:18.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.211+0000 7fbc9ffff640 1 -- 172.21.15.88:0/2206117392 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca0160970 con 0x7fbca00d22e0 2023-10-16T20:06:18.222 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.215+0000 7fbc9ffff640 1 --2- 172.21.15.88:0/2206117392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbca00d22e0 0x7fbca0161450 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbc900097c0 tx=0x7fbc9001c2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a2b4c7e38a9f99 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:18.222 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.215+0000 7fbc9effd640 1 -- 172.21.15.88:0/2206117392 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc940016a0 con 0x7fbca00d22e0 2023-10-16T20:06:18.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.215+0000 7fbc9effd640 1 -- 172.21.15.88:0/2206117392 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc900a7050 con 0x7fbca00d22e0 2023-10-16T20:06:18.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.215+0000 7fbc9effd640 1 -- 172.21.15.88:0/2206117392 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc900ab400 con 0x7fbca00d22e0 2023-10-16T20:06:18.225 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.219+0000 7fbca629c640 1 -- 172.21.15.88:0/2206117392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbca00d22e0 msgr2=0x7fbca0161450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:18.225 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.219+0000 7fbca629c640 1 --2- 172.21.15.88:0/2206117392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbca00d22e0 0x7fbca0161450 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbc900097c0 tx=0x7fbc9001c2e0 comp rx=0 tx=0).stop 2023-10-16T20:06:18.225 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.219+0000 7fbca629c640 1 -- 172.21.15.88:0/2206117392 shutdown_connections 2023-10-16T20:06:18.225 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.219+0000 7fbca629c640 1 --2- 172.21.15.88:0/2206117392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbca00d22e0 0x7fbca0161450 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:18.226 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.219+0000 7fbca629c640 1 -- 172.21.15.88:0/2206117392 >> 172.21.15.88:0/2206117392 conn(0x7fbca00715c0 msgr2=0x7fbca00719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:18.226 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.219+0000 7fbca629c640 1 -- 172.21.15.88:0/2206117392 shutdown_connections 2023-10-16T20:06:18.226 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.219+0000 7fbca629c640 1 -- 172.21.15.88:0/2206117392 wait complete. 2023-10-16T20:06:18.226 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.219+0000 7fbca629c640 1 Processor -- start 2023-10-16T20:06:18.229 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.219+0000 7fbca629c640 1 -- start start 2023-10-16T20:06:18.229 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.219+0000 7fbca629c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbca00d22e0 0x7fbca0154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:18.229 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.219+0000 7fbca629c640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbca00d26c0 0x7fbca00c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:18.229 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.219+0000 7fbca629c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbca015d9f0 0x7fbca0156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:18.230 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.219+0000 7fbca629c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca0156fe0 con 0x7fbca00d22e0 2023-10-16T20:06:18.230 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.219+0000 7fbca629c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca01552e0 con 0x7fbca015d9f0 2023-10-16T20:06:18.230 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.219+0000 7fbca629c640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca0155000 con 0x7fbca00d26c0 2023-10-16T20:06:18.230 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.219+0000 7fbc9ffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbca00d22e0 0x7fbca0154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:18.230 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.219+0000 7fbc9ffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbca00d22e0 0x7fbca0154900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45860/0 (socket says 172.21.15.88:45860) 2023-10-16T20:06:18.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.219+0000 7fbc9ffff640 1 -- 172.21.15.88:0/2585037552 learned_addr learned my addr 172.21.15.88:0/2585037552 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:18.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.219+0000 7fbc9f7fe640 1 --2- 172.21.15.88:0/2585037552 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbca00d26c0 0x7fbca00c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:18.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.219+0000 7fbca4839640 1 --2- 172.21.15.88:0/2585037552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbca015d9f0 0x7fbca0156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:18.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.219+0000 7fbc9ffff640 1 -- 172.21.15.88:0/2585037552 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbca00d26c0 msgr2=0x7fbca00c9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:18.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.223+0000 7fbc9ffff640 1 --2- 172.21.15.88:0/2585037552 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbca00d26c0 0x7fbca00c9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:18.232 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.223+0000 7fbc9ffff640 1 -- 172.21.15.88:0/2585037552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbca015d9f0 msgr2=0x7fbca0156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:18.232 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.223+0000 7fbc9ffff640 1 --2- 172.21.15.88:0/2585037552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbca015d9f0 0x7fbca0156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:18.232 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.223+0000 7fbc9ffff640 1 -- 172.21.15.88:0/2585037552 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc90009420 con 0x7fbca00d22e0 2023-10-16T20:06:18.232 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.223+0000 7fbc9f7fe640 1 --2- 172.21.15.88:0/2585037552 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbca00d26c0 0x7fbca00c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:18.232 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.223+0000 7fbc9ffff640 1 --2- 172.21.15.88:0/2585037552 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbca00d22e0 0x7fbca0154900 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fbc9001c7c0 tx=0x7fbc90019080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:18.233 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.223+0000 7fbc9cff9640 1 -- 172.21.15.88:0/2585037552 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc900a4ec0 con 0x7fbca00d22e0 2023-10-16T20:06:18.233 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.223+0000 7fbc9cff9640 1 -- 172.21.15.88:0/2585037552 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc900abe80 con 0x7fbca00d22e0 2023-10-16T20:06:18.233 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.223+0000 7fbca629c640 1 -- 172.21.15.88:0/2585037552 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca0159180 con 0x7fbca00d22e0 2023-10-16T20:06:18.233 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.223+0000 7fbc9cff9640 1 -- 172.21.15.88:0/2585037552 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc900b4600 con 0x7fbca00d22e0 2023-10-16T20:06:18.233 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.223+0000 7fbca629c640 1 -- 172.21.15.88:0/2585037552 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca0159740 con 0x7fbca00d22e0 2023-10-16T20:06:18.234 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.223+0000 7fbca629c640 1 -- 172.21.15.88:0/2585037552 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca004e9c0 con 0x7fbca00d22e0 2023-10-16T20:06:18.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.223+0000 7fbc9cff9640 1 -- 172.21.15.88:0/2585037552 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbc900b2040 con 0x7fbca00d22e0 2023-10-16T20:06:18.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.223+0000 7fbc9cff9640 1 --2- 172.21.15.88:0/2585037552 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbc8c0664a0 0x7fbc8c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:18.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.223+0000 7fbc9cff9640 1 -- 172.21.15.88:0/2585037552 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fbc9011df40 con 0x7fbca00d22e0 2023-10-16T20:06:18.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.227+0000 7fbc9f7fe640 1 --2- 172.21.15.88:0/2585037552 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbc8c0664a0 0x7fbc8c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:18.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.227+0000 7fbc9f7fe640 1 --2- 172.21.15.88:0/2585037552 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbc8c0664a0 0x7fbc8c068960 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbc94001ec0 tx=0x7fbc94001d70 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:18.237 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.227+0000 7fbc9cff9640 1 -- 172.21.15.88:0/2585037552 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbc900eb260 con 0x7fbca00d22e0 2023-10-16T20:06:18.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.379+0000 7fbca629c640 1 -- 172.21.15.88:0/2585037552 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbca0072950 con 0x7fbca00d22e0 2023-10-16T20:06:18.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.383+0000 7fbc9cff9640 1 -- 172.21.15.88:0/2585037552 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+16722 (secure 0 0 0) 0x7fbc900b0090 con 0x7fbca00d22e0 2023-10-16T20:06:18.388 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:06:18.388 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":41,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:06:18.191297+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T20:06:14.743500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"testcephfs1_data","create_time":"2023-10-16T20:06:16.091837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:06:18.390 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.383+0000 7fbca629c640 1 -- 172.21.15.88:0/2585037552 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbc8c0664a0 msgr2=0x7fbc8c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:18.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.383+0000 7fbca629c640 1 --2- 172.21.15.88:0/2585037552 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbc8c0664a0 0x7fbc8c068960 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbc94001ec0 tx=0x7fbc94001d70 comp rx=0 tx=0).stop 2023-10-16T20:06:18.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.383+0000 7fbca629c640 1 -- 172.21.15.88:0/2585037552 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbca00d22e0 msgr2=0x7fbca0154900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:18.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.383+0000 7fbca629c640 1 --2- 172.21.15.88:0/2585037552 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbca00d22e0 0x7fbca0154900 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fbc9001c7c0 tx=0x7fbc90019080 comp rx=0 tx=0).stop 2023-10-16T20:06:18.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.383+0000 7fbca629c640 1 -- 172.21.15.88:0/2585037552 shutdown_connections 2023-10-16T20:06:18.392 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.383+0000 7fbca629c640 1 --2- 172.21.15.88:0/2585037552 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbc8c0664a0 0x7fbc8c068960 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:18.392 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.383+0000 7fbca629c640 1 --2- 172.21.15.88:0/2585037552 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbca00d26c0 0x7fbca00c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:18.392 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.383+0000 7fbca629c640 1 --2- 172.21.15.88:0/2585037552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbca015d9f0 0x7fbca0156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:18.392 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.383+0000 7fbca629c640 1 --2- 172.21.15.88:0/2585037552 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbca00d22e0 0x7fbca0154900 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:18.392 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.383+0000 7fbca629c640 1 -- 172.21.15.88:0/2585037552 >> 172.21.15.88:0/2585037552 conn(0x7fbca00715c0 msgr2=0x7fbca015fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:18.393 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.383+0000 7fbca629c640 1 -- 172.21.15.88:0/2585037552 shutdown_connections 2023-10-16T20:06:18.393 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.383+0000 7fbca629c640 1 -- 172.21.15.88:0/2585037552 wait complete. 2023-10-16T20:06:18.409 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set testcephfs1 standby_count_wanted 0 2023-10-16T20:06:18.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.543+0000 7f89b6446640 1 Processor -- start 2023-10-16T20:06:18.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.547+0000 7f89b6446640 1 -- start start 2023-10-16T20:06:18.553 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.547+0000 7f89b6446640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f89b015cf90 0x7f89b00ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:18.553 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.547+0000 7f89b6446640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89b0161540 0x7f89b00cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:18.553 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.547+0000 7f89b6446640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f89b0071360 0x7f89b00717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:18.553 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.547+0000 7f89b6446640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89b0071d30 con 0x7f89b015cf90 2023-10-16T20:06:18.554 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.547+0000 7f89b6446640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89b00ca5f0 con 0x7f89b0071360 2023-10-16T20:06:18.554 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.547+0000 7f89b6446640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b00ca840 con 0x7f89b0161540 2023-10-16T20:06:18.554 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.547+0000 7f89affff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f89b015cf90 0x7f89b00ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:18.554 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.547+0000 7f89affff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f89b015cf90 0x7f89b00ca0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45874/0 (socket says 172.21.15.88:45874) 2023-10-16T20:06:18.554 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.547+0000 7f89affff640 1 -- 172.21.15.88:0/2326913104 learned_addr learned my addr 172.21.15.88:0/2326913104 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:18.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.547+0000 7f89b49e3640 1 --2- 172.21.15.88:0/2326913104 >> v2:172.21.15.94:3300/0 conn(0x7f89b0071360 0x7f89b00717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:18.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.547+0000 7f89af7fe640 1 --2- 172.21.15.88:0/2326913104 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89b0161540 0x7f89b00cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:18.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.547+0000 7f89affff640 1 -- 172.21.15.88:0/2326913104 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89b0161540 msgr2=0x7f89b00cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:18.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.547+0000 7f89affff640 1 --2- 172.21.15.88:0/2326913104 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89b0161540 0x7f89b00cb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:18.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.547+0000 7f89affff640 1 -- 172.21.15.88:0/2326913104 >> v2:172.21.15.94:3300/0 conn(0x7f89b0071360 msgr2=0x7f89b00717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:18.556 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.547+0000 7f89affff640 1 --2- 172.21.15.88:0/2326913104 >> v2:172.21.15.94:3300/0 conn(0x7f89b0071360 0x7f89b00717f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:18.556 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.547+0000 7f89affff640 1 -- 172.21.15.88:0/2326913104 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b00cab20 con 0x7f89b015cf90 2023-10-16T20:06:18.556 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.547+0000 7f89b49e3640 1 --2- 172.21.15.88:0/2326913104 >> v2:172.21.15.94:3300/0 conn(0x7f89b0071360 0x7f89b00717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:06:18.556 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.547+0000 7f89af7fe640 1 --2- 172.21.15.88:0/2326913104 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89b0161540 0x7f89b00cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:06:18.557 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.551+0000 7f89affff640 1 --2- 172.21.15.88:0/2326913104 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89b015cf90 0x7f89b00ca0b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f89b00c9e80 tx=0x7f89a001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e004ca4338cb40 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:18.557 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.551+0000 7f89aeffd640 1 -- 172.21.15.88:0/2326913104 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a00aa5f0 con 0x7f89b015cf90 2023-10-16T20:06:18.558 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.551+0000 7f89aeffd640 1 -- 172.21.15.88:0/2326913104 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a00aa750 con 0x7f89b015cf90 2023-10-16T20:06:18.558 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.551+0000 7f89aeffd640 1 -- 172.21.15.88:0/2326913104 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a00aaae0 con 0x7f89b015cf90 2023-10-16T20:06:18.558 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.551+0000 7f89b6446640 1 -- 172.21.15.88:0/2326913104 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89b015cf90 msgr2=0x7f89b00ca0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:18.558 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.551+0000 7f89b6446640 1 --2- 172.21.15.88:0/2326913104 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89b015cf90 0x7f89b00ca0b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f89b00c9e80 tx=0x7f89a001c4d0 comp rx=0 tx=0).stop 2023-10-16T20:06:18.558 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.551+0000 7f89b6446640 1 -- 172.21.15.88:0/2326913104 shutdown_connections 2023-10-16T20:06:18.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.551+0000 7f89b6446640 1 --2- 172.21.15.88:0/2326913104 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89b0161540 0x7f89b00cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:18.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.551+0000 7f89b6446640 1 --2- 172.21.15.88:0/2326913104 >> v2:172.21.15.94:3300/0 conn(0x7f89b0071360 0x7f89b00717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:18.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.551+0000 7f89b6446640 1 --2- 172.21.15.88:0/2326913104 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89b015cf90 0x7f89b00ca0b0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:18.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.551+0000 7f89b6446640 1 -- 172.21.15.88:0/2326913104 >> 172.21.15.88:0/2326913104 conn(0x7f89b00cf710 msgr2=0x7f89b00cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:18.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.551+0000 7f89b6446640 1 -- 172.21.15.88:0/2326913104 shutdown_connections 2023-10-16T20:06:18.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.551+0000 7f89b6446640 1 -- 172.21.15.88:0/2326913104 wait complete. 2023-10-16T20:06:18.561 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.551+0000 7f89b6446640 1 Processor -- start 2023-10-16T20:06:18.562 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.551+0000 7f89b6446640 1 -- start start 2023-10-16T20:06:18.562 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.555+0000 7f89b6446640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89b0071360 0x7f89b0149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:18.562 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.555+0000 7f89b6446640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89b015cf90 0x7f89b00c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:18.562 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.555+0000 7f89b6446640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89b0161540 0x7f89b0154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:18.562 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.555+0000 7f89b6446640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b00c6b50 con 0x7f89b0071360 2023-10-16T20:06:18.563 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.555+0000 7f89b6446640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b00c6e30 con 0x7f89b0161540 2023-10-16T20:06:18.563 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.555+0000 7f89b6446640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b00c7110 con 0x7f89b015cf90 2023-10-16T20:06:18.563 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.555+0000 7f89affff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89b0071360 0x7f89b0149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:18.563 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.555+0000 7f89affff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89b0071360 0x7f89b0149c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45878/0 (socket says 172.21.15.88:45878) 2023-10-16T20:06:18.563 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.555+0000 7f89affff640 1 -- 172.21.15.88:0/2435531929 learned_addr learned my addr 172.21.15.88:0/2435531929 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:18.564 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.555+0000 7f89af7fe640 1 --2- 172.21.15.88:0/2435531929 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89b015cf90 0x7f89b00c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:18.564 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.555+0000 7f89b49e3640 1 --2- 172.21.15.88:0/2435531929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89b0161540 0x7f89b0154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:18.564 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.555+0000 7f89affff640 1 -- 172.21.15.88:0/2435531929 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89b015cf90 msgr2=0x7f89b00c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:18.564 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.555+0000 7f89affff640 1 --2- 172.21.15.88:0/2435531929 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89b015cf90 0x7f89b00c88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:18.564 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.555+0000 7f89affff640 1 -- 172.21.15.88:0/2435531929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89b0161540 msgr2=0x7f89b0154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:18.564 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.555+0000 7f89affff640 1 --2- 172.21.15.88:0/2435531929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89b0161540 0x7f89b0154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:18.565 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.555+0000 7f89affff640 1 -- 172.21.15.88:0/2435531929 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a00094d0 con 0x7f89b0071360 2023-10-16T20:06:18.565 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.555+0000 7f89af7fe640 1 --2- 172.21.15.88:0/2435531929 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89b015cf90 0x7f89b00c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:18.565 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.555+0000 7f89b49e3640 1 --2- 172.21.15.88:0/2435531929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89b0161540 0x7f89b0154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:18.565 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.555+0000 7f89affff640 1 --2- 172.21.15.88:0/2435531929 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89b0071360 0x7f89b0149c40 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f89a001ab80 tx=0x7f89a001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:18.565 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.555+0000 7f89acff9640 1 -- 172.21.15.88:0/2435531929 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a00b3800 con 0x7f89b0071360 2023-10-16T20:06:18.566 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.555+0000 7f89acff9640 1 -- 172.21.15.88:0/2435531929 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a00b3980 con 0x7f89b0071360 2023-10-16T20:06:18.566 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.555+0000 7f89acff9640 1 -- 172.21.15.88:0/2435531929 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a00b3d30 con 0x7f89b0071360 2023-10-16T20:06:18.566 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.555+0000 7f89b6446640 1 -- 172.21.15.88:0/2435531929 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b00c73f0 con 0x7f89b0071360 2023-10-16T20:06:18.566 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.555+0000 7f89b6446640 1 -- 172.21.15.88:0/2435531929 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b00c7bf0 con 0x7f89b0071360 2023-10-16T20:06:18.567 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.555+0000 7f89b6446640 1 -- 172.21.15.88:0/2435531929 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89b004e9c0 con 0x7f89b0071360 2023-10-16T20:06:18.567 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.559+0000 7f89acff9640 1 -- 172.21.15.88:0/2435531929 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f89a00c6440 con 0x7f89b0071360 2023-10-16T20:06:18.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.559+0000 7f89acff9640 1 --2- 172.21.15.88:0/2435531929 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f899c0664a0 0x7f899c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:18.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.559+0000 7f89acff9640 1 -- 172.21.15.88:0/2435531929 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f89a00a7070 con 0x7f89b0071360 2023-10-16T20:06:18.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.559+0000 7f89af7fe640 1 --2- 172.21.15.88:0/2435531929 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f899c0664a0 0x7f899c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:18.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.559+0000 7f89af7fe640 1 --2- 172.21.15.88:0/2435531929 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f899c0664a0 0x7f899c068960 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f89b00c9300 tx=0x7f89a4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:18.569 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.563+0000 7f89acff9640 1 -- 172.21.15.88:0/2435531929 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f89a00ebf30 con 0x7f89b0071360 2023-10-16T20:06:18.752 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:18.747+0000 7f89b6446640 1 -- 172.21.15.88:0/2435531929 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "testcephfs1", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f89b00c6520 con 0x7f89b0071360 2023-10-16T20:06:19.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.735+0000 7f89acff9640 1 -- 172.21.15.88:0/2435531929 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "testcephfs1", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 123+0+0 (secure 0 0 0) 0x7f89a00ef550 con 0x7f89b0071360 2023-10-16T20:06:19.742 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.735+0000 7f89b6446640 1 -- 172.21.15.88:0/2435531929 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f899c0664a0 msgr2=0x7f899c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:19.742 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.735+0000 7f89b6446640 1 --2- 172.21.15.88:0/2435531929 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f899c0664a0 0x7f899c068960 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f89b00c9300 tx=0x7f89a4008040 comp rx=0 tx=0).stop 2023-10-16T20:06:19.742 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.735+0000 7f89b6446640 1 -- 172.21.15.88:0/2435531929 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89b0071360 msgr2=0x7f89b0149c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:19.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.735+0000 7f89b6446640 1 --2- 172.21.15.88:0/2435531929 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89b0071360 0x7f89b0149c40 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f89a001ab80 tx=0x7f89a001ad10 comp rx=0 tx=0).stop 2023-10-16T20:06:19.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.735+0000 7f89b6446640 1 -- 172.21.15.88:0/2435531929 shutdown_connections 2023-10-16T20:06:19.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.735+0000 7f89b6446640 1 --2- 172.21.15.88:0/2435531929 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f899c0664a0 0x7f899c068960 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:19.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.735+0000 7f89b6446640 1 --2- 172.21.15.88:0/2435531929 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f89b015cf90 0x7f89b00c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:19.744 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.735+0000 7f89b6446640 1 --2- 172.21.15.88:0/2435531929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89b0161540 0x7f89b0154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:19.744 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.735+0000 7f89b6446640 1 --2- 172.21.15.88:0/2435531929 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89b0071360 0x7f89b0149c40 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:19.744 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.735+0000 7f89b6446640 1 -- 172.21.15.88:0/2435531929 >> 172.21.15.88:0/2435531929 conn(0x7f89b00cf710 msgr2=0x7f89b00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:19.744 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.735+0000 7f89b6446640 1 -- 172.21.15.88:0/2435531929 shutdown_connections 2023-10-16T20:06:19.744 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.735+0000 7f89b6446640 1 -- 172.21.15.88:0/2435531929 wait complete. 2023-10-16T20:06:19.764 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T20:06:19.765 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set testcephfs1 allow_standby_replay false 2023-10-16T20:06:19.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.903+0000 7ff57a809640 1 Processor -- start 2023-10-16T20:06:19.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.903+0000 7ff57a809640 1 -- start start 2023-10-16T20:06:19.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.907+0000 7ff57a809640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff57415ab70 0x7ff5740ca940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:19.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.907+0000 7ff57a809640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff574065fa0 0x7ff5740cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:19.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.907+0000 7ff57a809640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff574071980 con 0x7ff5740b4ac0 2023-10-16T20:06:19.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.907+0000 7ff57a809640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5740b48f0 con 0x7ff574065fa0 2023-10-16T20:06:19.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.907+0000 7ff57a809640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff574069e90 con 0x7ff57415ab70 2023-10-16T20:06:19.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.907+0000 7ff579006640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7ff5740b4ac0 0x7ff57406c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:36814/0 (socket says 172.21.15.88:36814) 2023-10-16T20:06:19.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.907+0000 7ff579006640 1 -- 172.21.15.88:0/992789576 learned_addr learned my addr 172.21.15.88:0/992789576 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:19.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.907+0000 7ff57a008640 1 --2- 172.21.15.88:0/992789576 >> v2:172.21.15.88:3300/0 conn(0x7ff574065fa0 0x7ff5740cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:19.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.907+0000 7ff579807640 1 --2- 172.21.15.88:0/992789576 >> v2:172.21.15.94:3300/0 conn(0x7ff57415ab70 0x7ff5740ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:19.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.907+0000 7ff57a008640 1 -- 172.21.15.88:0/992789576 >> v2:172.21.15.94:3300/0 conn(0x7ff57415ab70 msgr2=0x7ff5740ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:19.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.907+0000 7ff57a008640 1 --2- 172.21.15.88:0/992789576 >> v2:172.21.15.94:3300/0 conn(0x7ff57415ab70 0x7ff5740ca940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:19.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.907+0000 7ff57a008640 1 -- 172.21.15.88:0/992789576 >> v1:172.21.15.88:6789/0 conn(0x7ff5740b4ac0 legacy=0x7ff57406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:19.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.907+0000 7ff57a008640 1 -- 172.21.15.88:0/992789576 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5740cb3c0 con 0x7ff574065fa0 2023-10-16T20:06:19.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.907+0000 7ff578805640 1 -- 172.21.15.88:0/992789576 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1338723458 0 0) 0x7ff574071980 con 0x7ff5740b4ac0 2023-10-16T20:06:19.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.911+0000 7ff57a008640 1 --2- 172.21.15.88:0/992789576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff574065fa0 0x7ff5740cae80 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff57406c340 tx=0x7ff56c01c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e6a2503dba0827a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:19.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.911+0000 7ff578805640 1 -- 172.21.15.88:0/992789576 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c0a59b0 con 0x7ff574065fa0 2023-10-16T20:06:19.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.911+0000 7ff578805640 1 -- 172.21.15.88:0/992789576 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff56c0a5b10 con 0x7ff574065fa0 2023-10-16T20:06:19.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.911+0000 7ff578805640 1 -- 172.21.15.88:0/992789576 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c0a5ea0 con 0x7ff574065fa0 2023-10-16T20:06:19.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.911+0000 7ff57a809640 1 -- 172.21.15.88:0/992789576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff574065fa0 msgr2=0x7ff5740cae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:19.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.911+0000 7ff57a809640 1 --2- 172.21.15.88:0/992789576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff574065fa0 0x7ff5740cae80 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff57406c340 tx=0x7ff56c01c8f0 comp rx=0 tx=0).stop 2023-10-16T20:06:19.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.911+0000 7ff57a809640 1 -- 172.21.15.88:0/992789576 shutdown_connections 2023-10-16T20:06:19.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.911+0000 7ff57a809640 1 --2- 172.21.15.88:0/992789576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff574065fa0 0x7ff5740cae80 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:19.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.911+0000 7ff57a809640 1 --2- 172.21.15.88:0/992789576 >> v2:172.21.15.94:3300/0 conn(0x7ff57415ab70 0x7ff5740ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:19.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.911+0000 7ff57a809640 1 -- 172.21.15.88:0/992789576 >> 172.21.15.88:0/992789576 conn(0x7ff5740b0820 msgr2=0x7ff5740b0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:19.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.911+0000 7ff57a809640 1 -- 172.21.15.88:0/992789576 shutdown_connections 2023-10-16T20:06:19.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.911+0000 7ff57a809640 1 -- 172.21.15.88:0/992789576 wait complete. 2023-10-16T20:06:19.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.911+0000 7ff57a809640 1 Processor -- start 2023-10-16T20:06:19.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.911+0000 7ff57a809640 1 -- start start 2023-10-16T20:06:19.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.915+0000 7ff57a809640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff574065fa0 0x7ff5740c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:19.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.915+0000 7ff57a809640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5740b4ac0 0x7ff574153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:19.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.915+0000 7ff57a809640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff57415ab70 0x7ff5740c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:19.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.915+0000 7ff57a809640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5740b0310 con 0x7ff5740b4ac0 2023-10-16T20:06:19.922 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.915+0000 7ff57a809640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff574163630 con 0x7ff574065fa0 2023-10-16T20:06:19.922 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.915+0000 7ff57a809640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff574163910 con 0x7ff57415ab70 2023-10-16T20:06:19.922 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.915+0000 7ff579006640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5740b4ac0 0x7ff574153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:19.922 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.915+0000 7ff579006640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5740b4ac0 0x7ff574153a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45904/0 (socket says 172.21.15.88:45904) 2023-10-16T20:06:19.922 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.915+0000 7ff579006640 1 -- 172.21.15.88:0/1861601864 learned_addr learned my addr 172.21.15.88:0/1861601864 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:19.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.915+0000 7ff57a008640 1 --2- 172.21.15.88:0/1861601864 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff57415ab70 0x7ff5740c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:19.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.915+0000 7ff579807640 1 --2- 172.21.15.88:0/1861601864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff574065fa0 0x7ff5740c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:19.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.915+0000 7ff579006640 1 -- 172.21.15.88:0/1861601864 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff57415ab70 msgr2=0x7ff5740c3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:19.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.915+0000 7ff579006640 1 --2- 172.21.15.88:0/1861601864 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff57415ab70 0x7ff5740c3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:19.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.915+0000 7ff579006640 1 -- 172.21.15.88:0/1861601864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff574065fa0 msgr2=0x7ff5740c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:19.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.915+0000 7ff579006640 1 --2- 172.21.15.88:0/1861601864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff574065fa0 0x7ff5740c8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:19.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.915+0000 7ff579006640 1 -- 172.21.15.88:0/1861601864 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff56c009a70 con 0x7ff5740b4ac0 2023-10-16T20:06:19.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.915+0000 7ff579006640 1 --2- 172.21.15.88:0/1861601864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5740b4ac0 0x7ff574153a60 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff56400c4b0 tx=0x7ff56400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:19.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.915+0000 7ff56a7fc640 1 -- 172.21.15.88:0/1861601864 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5640074e0 con 0x7ff5740b4ac0 2023-10-16T20:06:19.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.915+0000 7ff56a7fc640 1 -- 172.21.15.88:0/1861601864 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff564007660 con 0x7ff5740b4ac0 2023-10-16T20:06:19.925 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.915+0000 7ff57a809640 1 -- 172.21.15.88:0/1861601864 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff574163bf0 con 0x7ff5740b4ac0 2023-10-16T20:06:19.925 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.915+0000 7ff56a7fc640 1 -- 172.21.15.88:0/1861601864 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff564004a30 con 0x7ff5740b4ac0 2023-10-16T20:06:19.925 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.915+0000 7ff57a809640 1 -- 172.21.15.88:0/1861601864 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff574163ed0 con 0x7ff5740b4ac0 2023-10-16T20:06:19.926 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.915+0000 7ff57a809640 1 -- 172.21.15.88:0/1861601864 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5740c8630 con 0x7ff5740b4ac0 2023-10-16T20:06:19.926 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.915+0000 7ff56a7fc640 1 -- 172.21.15.88:0/1861601864 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff564004b90 con 0x7ff5740b4ac0 2023-10-16T20:06:19.926 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.919+0000 7ff56a7fc640 1 --2- 172.21.15.88:0/1861601864 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff5580664d0 0x7ff558068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:19.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.919+0000 7ff56a7fc640 1 -- 172.21.15.88:0/1861601864 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6049+0+0 (secure 0 0 0) 0x7ff564083570 con 0x7ff5740b4ac0 2023-10-16T20:06:19.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.919+0000 7ff579807640 1 --2- 172.21.15.88:0/1861601864 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff5580664d0 0x7ff558068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:19.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.919+0000 7ff579807640 1 --2- 172.21.15.88:0/1861601864 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff5580664d0 0x7ff558068990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff570009af0 tx=0x7ff570008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:19.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:19.923+0000 7ff56a7fc640 1 -- 172.21.15.88:0/1861601864 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff5640508b0 con 0x7ff5740b4ac0 2023-10-16T20:06:20.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.103+0000 7ff57a809640 1 -- 172.21.15.88:0/1861601864 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "testcephfs1", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff574072510 con 0x7ff5740b4ac0 2023-10-16T20:06:20.745 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.739+0000 7ff56a7fc640 1 -- 172.21.15.88:0/1861601864 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "testcephfs1", "var": "allow_standby_replay", "val": "false"}]=0 v18) v1 ==== 127+0+0 (secure 0 0 0) 0x7ff564053ed0 con 0x7ff5740b4ac0 2023-10-16T20:06:20.748 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.743+0000 7ff57a809640 1 -- 172.21.15.88:0/1861601864 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff5580664d0 msgr2=0x7ff558068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:20.749 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.743+0000 7ff57a809640 1 --2- 172.21.15.88:0/1861601864 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff5580664d0 0x7ff558068990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff570009af0 tx=0x7ff570008040 comp rx=0 tx=0).stop 2023-10-16T20:06:20.749 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.743+0000 7ff57a809640 1 -- 172.21.15.88:0/1861601864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5740b4ac0 msgr2=0x7ff574153a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:20.749 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.743+0000 7ff57a809640 1 --2- 172.21.15.88:0/1861601864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5740b4ac0 0x7ff574153a60 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff56400c4b0 tx=0x7ff56400c970 comp rx=0 tx=0).stop 2023-10-16T20:06:20.749 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.743+0000 7ff57a809640 1 -- 172.21.15.88:0/1861601864 shutdown_connections 2023-10-16T20:06:20.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.743+0000 7ff57a809640 1 --2- 172.21.15.88:0/1861601864 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff5580664d0 0x7ff558068990 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:20.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.743+0000 7ff57a809640 1 --2- 172.21.15.88:0/1861601864 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff57415ab70 0x7ff5740c3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:20.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.743+0000 7ff57a809640 1 --2- 172.21.15.88:0/1861601864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5740b4ac0 0x7ff574153a60 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:20.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.743+0000 7ff57a809640 1 --2- 172.21.15.88:0/1861601864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff574065fa0 0x7ff5740c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:20.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.743+0000 7ff57a809640 1 -- 172.21.15.88:0/1861601864 >> 172.21.15.88:0/1861601864 conn(0x7ff5740b0820 msgr2=0x7ff5740b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:20.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.743+0000 7ff57a809640 1 -- 172.21.15.88:0/1861601864 shutdown_connections 2023-10-16T20:06:20.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.743+0000 7ff57a809640 1 -- 172.21.15.88:0/1861601864 wait complete. 2023-10-16T20:06:20.767 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'testcephfs1' 2023-10-16T20:06:20.768 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T20:06:20.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.899+0000 7f879a91d640 1 Processor -- start 2023-10-16T20:06:20.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.899+0000 7f879a91d640 1 -- start start 2023-10-16T20:06:20.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.903+0000 7f879a91d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f87940b0c10 0x7f87940b1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:20.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.903+0000 7f879a91d640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87940b2570 con 0x7f87940ade20 2023-10-16T20:06:20.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.903+0000 7f879a91d640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87940cca50 con 0x7f87940ada40 2023-10-16T20:06:20.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.903+0000 7f879a91d640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87940b1fb0 con 0x7f87940b0c10 2023-10-16T20:06:20.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.903+0000 7f87937fe640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f87940ade20 0x7f87940b0500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:36822/0 (socket says 172.21.15.88:36822) 2023-10-16T20:06:20.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.903+0000 7f87937fe640 1 -- 172.21.15.88:0/3685181288 learned_addr learned my addr 172.21.15.88:0/3685181288 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:20.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.903+0000 7f8798eba640 1 --2- 172.21.15.88:0/3685181288 >> v2:172.21.15.88:3300/0 conn(0x7f87940b0c10 0x7f87940b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:20.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.903+0000 7f8792ffd640 1 -- 172.21.15.88:0/3685181288 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4027753195 0 0) 0x7f87940b2570 con 0x7f87940ade20 2023-10-16T20:06:20.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.903+0000 7f8798eba640 1 -- 172.21.15.88:0/3685181288 >> v1:172.21.15.94:6789/0 conn(0x7f87940ada40 legacy=0x7f879415adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:20.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.903+0000 7f8798eba640 1 -- 172.21.15.88:0/3685181288 >> v1:172.21.15.88:6789/0 conn(0x7f87940ade20 legacy=0x7f87940b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:20.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.903+0000 7f8798eba640 1 -- 172.21.15.88:0/3685181288 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87940b2290 con 0x7f87940b0c10 2023-10-16T20:06:20.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.903+0000 7f8798eba640 1 --2- 172.21.15.88:0/3685181288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87940b0c10 0x7f87940b1920 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f878c009dc0 tx=0x7f878c01c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce07bd7804349083 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:20.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.903+0000 7f8792ffd640 1 -- 172.21.15.88:0/3685181288 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c0a5950 con 0x7f87940b0c10 2023-10-16T20:06:20.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.903+0000 7f8792ffd640 1 -- 172.21.15.88:0/3685181288 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878c0a5ab0 con 0x7f87940b0c10 2023-10-16T20:06:20.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.903+0000 7f8792ffd640 1 -- 172.21.15.88:0/3685181288 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c0a5e40 con 0x7f87940b0c10 2023-10-16T20:06:20.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.907+0000 7f879a91d640 1 -- 172.21.15.88:0/3685181288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87940b0c10 msgr2=0x7f87940b1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:20.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.907+0000 7f879a91d640 1 --2- 172.21.15.88:0/3685181288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87940b0c10 0x7f87940b1920 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f878c009dc0 tx=0x7f878c01c870 comp rx=0 tx=0).stop 2023-10-16T20:06:20.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.907+0000 7f879a91d640 1 -- 172.21.15.88:0/3685181288 shutdown_connections 2023-10-16T20:06:20.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.907+0000 7f879a91d640 1 --2- 172.21.15.88:0/3685181288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87940b0c10 0x7f87940b1920 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:20.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.907+0000 7f879a91d640 1 -- 172.21.15.88:0/3685181288 >> 172.21.15.88:0/3685181288 conn(0x7f87940ce5e0 msgr2=0x7f87940ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:20.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.907+0000 7f879a91d640 1 -- 172.21.15.88:0/3685181288 shutdown_connections 2023-10-16T20:06:20.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.907+0000 7f879a91d640 1 -- 172.21.15.88:0/3685181288 wait complete. 2023-10-16T20:06:20.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.907+0000 7f879a91d640 1 Processor -- start 2023-10-16T20:06:20.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.907+0000 7f879a91d640 1 -- start start 2023-10-16T20:06:20.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.907+0000 7f879a91d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87940ada40 0x7f879413fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:20.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.907+0000 7f879a91d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87940ade20 0x7f879415d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:20.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.907+0000 7f879a91d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f87940b0c10 0x7f8794142650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:20.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.907+0000 7f879a91d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87941380c0 con 0x7f87940ade20 2023-10-16T20:06:20.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.907+0000 7f879a91d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8794142b90 con 0x7f87940ada40 2023-10-16T20:06:20.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.907+0000 7f879a91d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8794138360 con 0x7f87940b0c10 2023-10-16T20:06:20.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.907+0000 7f87937fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87940ade20 0x7f879415d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:20.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.907+0000 7f87937fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87940ade20 0x7f879415d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45920/0 (socket says 172.21.15.88:45920) 2023-10-16T20:06:20.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.907+0000 7f87937fe640 1 -- 172.21.15.88:0/666449955 learned_addr learned my addr 172.21.15.88:0/666449955 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:20.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.907+0000 7f8798eba640 1 --2- 172.21.15.88:0/666449955 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f87940b0c10 0x7f8794142650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:20.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.907+0000 7f8793fff640 1 --2- 172.21.15.88:0/666449955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87940ada40 0x7f879413fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:20.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.911+0000 7f87937fe640 1 -- 172.21.15.88:0/666449955 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f87940b0c10 msgr2=0x7f8794142650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:20.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.911+0000 7f87937fe640 1 --2- 172.21.15.88:0/666449955 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f87940b0c10 0x7f8794142650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:20.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.911+0000 7f87937fe640 1 -- 172.21.15.88:0/666449955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87940ada40 msgr2=0x7f879413fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:20.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.911+0000 7f87937fe640 1 --2- 172.21.15.88:0/666449955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87940ada40 0x7f879413fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:20.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.911+0000 7f87937fe640 1 -- 172.21.15.88:0/666449955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f878c009a70 con 0x7f87940ade20 2023-10-16T20:06:20.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.911+0000 7f87937fe640 1 --2- 172.21.15.88:0/666449955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87940ade20 0x7f879415d000 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f878800c4a0 tx=0x7f878800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:20.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.911+0000 7f8790ff9640 1 -- 172.21.15.88:0/666449955 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87880074e0 con 0x7f87940ade20 2023-10-16T20:06:20.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.911+0000 7f8790ff9640 1 -- 172.21.15.88:0/666449955 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8788007640 con 0x7f87940ade20 2023-10-16T20:06:20.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.911+0000 7f8790ff9640 1 -- 172.21.15.88:0/666449955 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8788004a30 con 0x7f87940ade20 2023-10-16T20:06:20.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.911+0000 7f879a91d640 1 -- 172.21.15.88:0/666449955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8794138640 con 0x7f87940ade20 2023-10-16T20:06:20.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.911+0000 7f879a91d640 1 -- 172.21.15.88:0/666449955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8794138c00 con 0x7f87940ade20 2023-10-16T20:06:20.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.911+0000 7f879a91d640 1 -- 172.21.15.88:0/666449955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f879404e9c0 con 0x7f87940ade20 2023-10-16T20:06:20.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.911+0000 7f8790ff9640 1 -- 172.21.15.88:0/666449955 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8788004b90 con 0x7f87940ade20 2023-10-16T20:06:20.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.911+0000 7f8790ff9640 1 --2- 172.21.15.88:0/666449955 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f87800664a0 0x7f8780068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:20.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.911+0000 7f8790ff9640 1 -- 172.21.15.88:0/666449955 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f87880831d0 con 0x7f87940ade20 2023-10-16T20:06:20.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.911+0000 7f8793fff640 1 --2- 172.21.15.88:0/666449955 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f87800664a0 0x7f8780068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:20.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.915+0000 7f8793fff640 1 --2- 172.21.15.88:0/666449955 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f87800664a0 0x7f8780068960 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f87940ac210 tx=0x7f8784009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:20.922 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:20.915+0000 7f8790ff9640 1 -- 172.21.15.88:0/666449955 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8788050590 con 0x7f87940ade20 2023-10-16T20:06:21.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.095+0000 7f879a91d640 1 -- 172.21.15.88:0/666449955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f879413bcf0 con 0x7f87940ade20 2023-10-16T20:06:21.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.099+0000 7f8790ff9640 1 -- 172.21.15.88:0/666449955 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f8788018090 con 0x7f87940ade20 2023-10-16T20:06:21.103 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:06:21.104 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/696292235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":696292235},{"type":"v1","addr":"172.21.15.94:6834","nonce":696292235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/2152672724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2152672724},{"type":"v1","addr":"172.21.15.88:6835","nonce":2152672724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3107974207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":3107974207},{"type":"v1","addr":"172.21.15.88:6837","nonce":3107974207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T20:06:17.088386+0000","modified":"2023-10-16T20:06:20.108161+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5535":{"gid":5535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6835/2190281591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":2190281591},{"type":"v1","addr":"172.21.15.94:6835","nonce":2190281591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"testcephfs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-16T20:06:21.104 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 18 2023-10-16T20:06:21.105 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.099+0000 7f879a91d640 1 -- 172.21.15.88:0/666449955 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f87800664a0 msgr2=0x7f8780068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:21.105 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.099+0000 7f879a91d640 1 --2- 172.21.15.88:0/666449955 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f87800664a0 0x7f8780068960 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f87940ac210 tx=0x7f8784009040 comp rx=0 tx=0).stop 2023-10-16T20:06:21.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.099+0000 7f879a91d640 1 -- 172.21.15.88:0/666449955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87940ade20 msgr2=0x7f879415d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:21.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.099+0000 7f879a91d640 1 --2- 172.21.15.88:0/666449955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87940ade20 0x7f879415d000 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f878800c4a0 tx=0x7f878800c960 comp rx=0 tx=0).stop 2023-10-16T20:06:21.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.099+0000 7f879a91d640 1 -- 172.21.15.88:0/666449955 shutdown_connections 2023-10-16T20:06:21.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.099+0000 7f879a91d640 1 --2- 172.21.15.88:0/666449955 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f87800664a0 0x7f8780068960 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:21.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.099+0000 7f879a91d640 1 --2- 172.21.15.88:0/666449955 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f87940b0c10 0x7f8794142650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:21.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.099+0000 7f879a91d640 1 --2- 172.21.15.88:0/666449955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87940ade20 0x7f879415d000 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:21.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.099+0000 7f879a91d640 1 --2- 172.21.15.88:0/666449955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87940ada40 0x7f879413fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:21.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.099+0000 7f879a91d640 1 -- 172.21.15.88:0/666449955 >> 172.21.15.88:0/666449955 conn(0x7f87940ce5e0 msgr2=0x7f8794142f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:21.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.099+0000 7f879a91d640 1 -- 172.21.15.88:0/666449955 shutdown_connections 2023-10-16T20:06:21.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.099+0000 7f879a91d640 1 -- 172.21.15.88:0/666449955 wait complete. 2023-10-16T20:06:21.124 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:06:21.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.255+0000 7f9986eea640 1 Processor -- start 2023-10-16T20:06:21.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.255+0000 7f9986eea640 1 -- start start 2023-10-16T20:06:21.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f9986eea640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9980069e90 0x7f998006a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:21.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f9986eea640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f99800b05a0 0x7f99800b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:21.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f9986eea640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f998006a830 con 0x7f99801599f0 2023-10-16T20:06:21.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f9986eea640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99800b1260 con 0x7f99800b05a0 2023-10-16T20:06:21.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f9986eea640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99800b3140 con 0x7f9980069e90 2023-10-16T20:06:21.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f9984c86640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f99801599f0 0x7f99800b2a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:36832/0 (socket says 172.21.15.88:36832) 2023-10-16T20:06:21.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f9984c86640 1 -- 172.21.15.88:0/3207360029 learned_addr learned my addr 172.21.15.88:0/3207360029 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:21.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f997ffff640 1 --2- 172.21.15.88:0/3207360029 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9980069e90 0x7f998006a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:21.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f9985487640 1 --2- 172.21.15.88:0/3207360029 >> v2:172.21.15.94:3300/0 conn(0x7f99800b05a0 0x7f99800b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:21.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f997f7fe640 1 -- 172.21.15.88:0/3207360029 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1093018858 0 0) 0x7f998006a830 con 0x7f99801599f0 2023-10-16T20:06:21.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f997f7fe640 1 -- 172.21.15.88:0/3207360029 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9970003530 con 0x7f99801599f0 2023-10-16T20:06:21.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f997f7fe640 1 -- 172.21.15.88:0/3207360029 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3897324621 0 0) 0x7f9970003530 con 0x7f99801599f0 2023-10-16T20:06:21.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f997f7fe640 1 -- 172.21.15.88:0/3207360029 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9980069e90 msgr2=0x7f998006a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:21.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f997f7fe640 1 --2- 172.21.15.88:0/3207360029 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9980069e90 0x7f998006a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:21.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f997f7fe640 1 -- 172.21.15.88:0/3207360029 >> v2:172.21.15.94:3300/0 conn(0x7f99800b05a0 msgr2=0x7f99800b0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:21.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f997f7fe640 1 --2- 172.21.15.88:0/3207360029 >> v2:172.21.15.94:3300/0 conn(0x7f99800b05a0 0x7f99800b0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:21.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f997f7fe640 1 -- 172.21.15.88:0/3207360029 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99800ae960 con 0x7f99801599f0 2023-10-16T20:06:21.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f997f7fe640 1 -- 172.21.15.88:0/3207360029 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f9974001680 con 0x7f99801599f0 2023-10-16T20:06:21.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f997f7fe640 1 -- 172.21.15.88:0/3207360029 >> v1:172.21.15.88:6789/0 conn(0x7f99801599f0 legacy=0x7f99800b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:21.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f997f7fe640 1 --2- 172.21.15.88:0/3207360029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9970003980 0x7f9970003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:21.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f997f7fe640 1 --2- 172.21.15.88:0/3207360029 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9980069e90 0x7f99700047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:21.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f997f7fe640 1 --2- 172.21.15.88:0/3207360029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9970004d10 0x7f9970005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:21.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f997f7fe640 1 -- 172.21.15.88:0/3207360029 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99800b1260 con 0x7f9970003980 2023-10-16T20:06:21.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f997f7fe640 1 -- 172.21.15.88:0/3207360029 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99800b3140 con 0x7f9970004d10 2023-10-16T20:06:21.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f997f7fe640 1 -- 172.21.15.88:0/3207360029 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997000b990 con 0x7f9980069e90 2023-10-16T20:06:21.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f9984c86640 1 --2- 172.21.15.88:0/3207360029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9970003980 0x7f9970003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:21.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f997ffff640 1 --2- 172.21.15.88:0/3207360029 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9980069e90 0x7f99700047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:21.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f9985487640 1 --2- 172.21.15.88:0/3207360029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9970004d10 0x7f9970005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:21.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f9984c86640 1 -- 172.21.15.88:0/3207360029 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9980069e90 msgr2=0x7f99700047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:21.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f9984c86640 1 --2- 172.21.15.88:0/3207360029 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9980069e90 0x7f99700047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:21.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f9984c86640 1 -- 172.21.15.88:0/3207360029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9970004d10 msgr2=0x7f9970005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:21.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f9984c86640 1 --2- 172.21.15.88:0/3207360029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9970004d10 0x7f9970005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:21.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.259+0000 7f9984c86640 1 -- 172.21.15.88:0/3207360029 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9970003530 con 0x7f9970003980 2023-10-16T20:06:21.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.263+0000 7f9984c86640 1 --2- 172.21.15.88:0/3207360029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9970003980 0x7f9970003da0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9974007ce0 tx=0x7f997401ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5dee592e1ec015c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:21.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.263+0000 7f997f7fe640 1 -- 172.21.15.88:0/3207360029 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99740a7a50 con 0x7f9970003980 2023-10-16T20:06:21.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.263+0000 7f997f7fe640 1 -- 172.21.15.88:0/3207360029 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99740a7bb0 con 0x7f9970003980 2023-10-16T20:06:21.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.263+0000 7f997f7fe640 1 -- 172.21.15.88:0/3207360029 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99740058e0 con 0x7f9970003980 2023-10-16T20:06:21.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.263+0000 7f9986eea640 1 -- 172.21.15.88:0/3207360029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9970003980 msgr2=0x7f9970003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:21.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.263+0000 7f9986eea640 1 --2- 172.21.15.88:0/3207360029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9970003980 0x7f9970003da0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9974007ce0 tx=0x7f997401ee60 comp rx=0 tx=0).stop 2023-10-16T20:06:21.276 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.263+0000 7f9985487640 1 -- 172.21.15.88:0/3207360029 reap_dead start 2023-10-16T20:06:21.276 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.263+0000 7f9986eea640 1 -- 172.21.15.88:0/3207360029 shutdown_connections 2023-10-16T20:06:21.276 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.263+0000 7f9986eea640 1 -- 172.21.15.88:0/3207360029 >> 172.21.15.88:0/3207360029 conn(0x7f99800ce5e0 msgr2=0x7f99800ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:21.276 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.263+0000 7f9986eea640 1 -- 172.21.15.88:0/3207360029 shutdown_connections 2023-10-16T20:06:21.276 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.263+0000 7f9986eea640 1 -- 172.21.15.88:0/3207360029 wait complete. 2023-10-16T20:06:21.277 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.263+0000 7f9986eea640 1 Processor -- start 2023-10-16T20:06:21.277 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.263+0000 7f9986eea640 1 -- start start 2023-10-16T20:06:21.277 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.267+0000 7f9986eea640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9970003980 0x7f9980146270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:21.278 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.267+0000 7f9986eea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9980149e60 0x7f998014b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:21.278 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.267+0000 7f9986eea640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f998014e700 0x7f9980146e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:21.278 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.267+0000 7f9986eea640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998014a420 con 0x7f9970003980 2023-10-16T20:06:21.279 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.267+0000 7f9986eea640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998014a700 con 0x7f9980149e60 2023-10-16T20:06:21.279 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.267+0000 7f9986eea640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f998014a9e0 con 0x7f998014e700 2023-10-16T20:06:21.279 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.267+0000 7f9984c86640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9970003980 0x7f9980146270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:21.279 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.267+0000 7f9984c86640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9970003980 0x7f9980146270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45934/0 (socket says 172.21.15.88:45934) 2023-10-16T20:06:21.280 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.267+0000 7f9984c86640 1 -- 172.21.15.88:0/3905797866 learned_addr learned my addr 172.21.15.88:0/3905797866 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:21.280 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.267+0000 7f997ffff640 1 --2- 172.21.15.88:0/3905797866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9980149e60 0x7f998014b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:21.280 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.267+0000 7f9985487640 1 --2- 172.21.15.88:0/3905797866 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f998014e700 0x7f9980146e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:21.280 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.267+0000 7f9984c86640 1 -- 172.21.15.88:0/3905797866 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f998014e700 msgr2=0x7f9980146e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:21.280 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.267+0000 7f9984c86640 1 --2- 172.21.15.88:0/3905797866 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f998014e700 0x7f9980146e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:21.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.267+0000 7f9984c86640 1 -- 172.21.15.88:0/3905797866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9980149e60 msgr2=0x7f998014b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:21.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.267+0000 7f9984c86640 1 --2- 172.21.15.88:0/3905797866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9980149e60 0x7f998014b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:21.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.267+0000 7f9984c86640 1 -- 172.21.15.88:0/3905797866 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f998014acc0 con 0x7f9970003980 2023-10-16T20:06:21.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.267+0000 7f997ffff640 1 --2- 172.21.15.88:0/3905797866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9980149e60 0x7f998014b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:21.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.267+0000 7f9984c86640 1 --2- 172.21.15.88:0/3905797866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9970003980 0x7f9980146270 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f99740ad040 tx=0x7f9970005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:21.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.267+0000 7f997d7fa640 1 -- 172.21.15.88:0/3905797866 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9974004ab0 con 0x7f9970003980 2023-10-16T20:06:21.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.267+0000 7f997d7fa640 1 -- 172.21.15.88:0/3905797866 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99740b5070 con 0x7f9970003980 2023-10-16T20:06:21.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.267+0000 7f9986eea640 1 -- 172.21.15.88:0/3905797866 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f998014afa0 con 0x7f9970003980 2023-10-16T20:06:21.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.267+0000 7f997d7fa640 1 -- 172.21.15.88:0/3905797866 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9974004e50 con 0x7f9970003980 2023-10-16T20:06:21.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.267+0000 7f9986eea640 1 -- 172.21.15.88:0/3905797866 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99801473d0 con 0x7f9970003980 2023-10-16T20:06:21.283 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.267+0000 7f9986eea640 1 -- 172.21.15.88:0/3905797866 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99800b0ef0 con 0x7f9970003980 2023-10-16T20:06:21.283 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.271+0000 7f997d7fa640 1 -- 172.21.15.88:0/3905797866 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9974007070 con 0x7f9970003980 2023-10-16T20:06:21.284 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.271+0000 7f997d7fa640 1 --2- 172.21.15.88:0/3905797866 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f99640664a0 0x7f9964068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:21.284 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.271+0000 7f997d7fa640 1 -- 172.21.15.88:0/3905797866 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f99740b7070 con 0x7f9970003980 2023-10-16T20:06:21.284 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.271+0000 7f997ffff640 1 --2- 172.21.15.88:0/3905797866 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f99640664a0 0x7f9964068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:21.284 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.271+0000 7f997ffff640 1 --2- 172.21.15.88:0/3905797866 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f99640664a0 0x7f9964068960 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f998014efd0 tx=0x7f996c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:21.284 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.275+0000 7f997d7fa640 1 -- 172.21.15.88:0/3905797866 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f99740ed6b0 con 0x7f9970003980 2023-10-16T20:06:21.432 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.427+0000 7f9986eea640 1 -- 172.21.15.88:0/3905797866 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f998004e9c0 con 0x7f9970003980 2023-10-16T20:06:21.433 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.427+0000 7f997d7fa640 1 -- 172.21.15.88:0/3905797866 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+16722 (secure 0 0 0) 0x7f99740f0cd0 con 0x7f9970003980 2023-10-16T20:06:21.433 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:06:21.434 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":41,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:06:18.191297+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T20:06:14.743500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"testcephfs1_data","create_time":"2023-10-16T20:06:16.091837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:06:21.435 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.431+0000 7f9986eea640 1 -- 172.21.15.88:0/3905797866 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f99640664a0 msgr2=0x7f9964068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:21.435 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.431+0000 7f9986eea640 1 --2- 172.21.15.88:0/3905797866 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f99640664a0 0x7f9964068960 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f998014efd0 tx=0x7f996c008040 comp rx=0 tx=0).stop 2023-10-16T20:06:21.436 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.431+0000 7f9986eea640 1 -- 172.21.15.88:0/3905797866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9970003980 msgr2=0x7f9980146270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:21.436 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.431+0000 7f9986eea640 1 --2- 172.21.15.88:0/3905797866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9970003980 0x7f9980146270 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f99740ad040 tx=0x7f9970005800 comp rx=0 tx=0).stop 2023-10-16T20:06:21.436 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.431+0000 7f9986eea640 1 -- 172.21.15.88:0/3905797866 shutdown_connections 2023-10-16T20:06:21.436 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.431+0000 7f9986eea640 1 --2- 172.21.15.88:0/3905797866 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f99640664a0 0x7f9964068960 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:21.437 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.431+0000 7f9986eea640 1 --2- 172.21.15.88:0/3905797866 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f998014e700 0x7f9980146e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:21.437 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.431+0000 7f9986eea640 1 --2- 172.21.15.88:0/3905797866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9980149e60 0x7f998014b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:21.437 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.431+0000 7f9986eea640 1 --2- 172.21.15.88:0/3905797866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9970003980 0x7f9980146270 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:21.437 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.431+0000 7f9986eea640 1 -- 172.21.15.88:0/3905797866 >> 172.21.15.88:0/3905797866 conn(0x7f99800ce5e0 msgr2=0x7f9980071a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:21.437 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.431+0000 7f9986eea640 1 -- 172.21.15.88:0/3905797866 shutdown_connections 2023-10-16T20:06:21.437 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.431+0000 7f9986eea640 1 -- 172.21.15.88:0/3905797866 wait complete. 2023-10-16T20:06:21.455 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T20:06:21.455 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T20:06:21.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.587+0000 7f431edc4640 1 Processor -- start 2023-10-16T20:06:21.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.587+0000 7f431edc4640 1 -- start start 2023-10-16T20:06:21.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.587+0000 7f431edc4640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f431806b100 0x7f431806c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:21.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.587+0000 7f431edc4640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f43180715c0 0x7f431806c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:21.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.587+0000 7f431edc4640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f431806b4e0 0x7f431806cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:21.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.591+0000 7f431edc4640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43180ae720 con 0x7f431806b100 2023-10-16T20:06:21.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.591+0000 7f431edc4640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43180c9a10 con 0x7f43180715c0 2023-10-16T20:06:21.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.591+0000 7f431edc4640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f431806d650 con 0x7f431806b4e0 2023-10-16T20:06:21.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.591+0000 7f431cb60640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f431806b100 0x7f431806c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:21.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.591+0000 7f431cb60640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f431806b100 0x7f431806c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45944/0 (socket says 172.21.15.88:45944) 2023-10-16T20:06:21.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.591+0000 7f431cb60640 1 -- 172.21.15.88:0/3064382498 learned_addr learned my addr 172.21.15.88:0/3064382498 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:21.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.591+0000 7f431cb60640 1 -- 172.21.15.88:0/3064382498 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f431806b4e0 msgr2=0x7f431806cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:21.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.591+0000 7f430ffff640 1 --2- 172.21.15.88:0/3064382498 >> v2:172.21.15.94:3300/0 conn(0x7f43180715c0 0x7f431806c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:21.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.591+0000 7f431d361640 1 --2- 172.21.15.88:0/3064382498 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f431806b4e0 0x7f431806cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:21.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.591+0000 7f431cb60640 1 --2- 172.21.15.88:0/3064382498 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f431806b4e0 0x7f431806cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:21.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.591+0000 7f431cb60640 1 -- 172.21.15.88:0/3064382498 >> v2:172.21.15.94:3300/0 conn(0x7f43180715c0 msgr2=0x7f431806c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:21.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.591+0000 7f431cb60640 1 --2- 172.21.15.88:0/3064382498 >> v2:172.21.15.94:3300/0 conn(0x7f43180715c0 0x7f431806c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:21.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.591+0000 7f431cb60640 1 -- 172.21.15.88:0/3064382498 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43180c9420 con 0x7f431806b100 2023-10-16T20:06:21.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.591+0000 7f430ffff640 1 --2- 172.21.15.88:0/3064382498 >> v2:172.21.15.94:3300/0 conn(0x7f43180715c0 0x7f431806c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:06:21.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.591+0000 7f431d361640 1 --2- 172.21.15.88:0/3064382498 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f431806b4e0 0x7f431806cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:06:21.600 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.591+0000 7f431cb60640 1 --2- 172.21.15.88:0/3064382498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f431806b100 0x7f431806c370 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f43180cce10 tx=0x7f431401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4bc8cb19a833c92 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:21.600 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.591+0000 7f430f7fe640 1 -- 172.21.15.88:0/3064382498 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43140aa620 con 0x7f431806b100 2023-10-16T20:06:21.601 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.591+0000 7f430f7fe640 1 -- 172.21.15.88:0/3064382498 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43140aa780 con 0x7f431806b100 2023-10-16T20:06:21.601 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.591+0000 7f430f7fe640 1 -- 172.21.15.88:0/3064382498 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43140aab10 con 0x7f431806b100 2023-10-16T20:06:21.601 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.595+0000 7f431edc4640 1 -- 172.21.15.88:0/3064382498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f431806b100 msgr2=0x7f431806c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:21.601 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.595+0000 7f431edc4640 1 --2- 172.21.15.88:0/3064382498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f431806b100 0x7f431806c370 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f43180cce10 tx=0x7f431401c500 comp rx=0 tx=0).stop 2023-10-16T20:06:21.601 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.595+0000 7f431edc4640 1 -- 172.21.15.88:0/3064382498 shutdown_connections 2023-10-16T20:06:21.602 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.595+0000 7f431edc4640 1 --2- 172.21.15.88:0/3064382498 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f431806b4e0 0x7f431806cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:21.602 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.595+0000 7f431edc4640 1 --2- 172.21.15.88:0/3064382498 >> v2:172.21.15.94:3300/0 conn(0x7f43180715c0 0x7f431806c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:21.602 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.595+0000 7f431edc4640 1 --2- 172.21.15.88:0/3064382498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f431806b100 0x7f431806c370 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:21.602 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.595+0000 7f431edc4640 1 -- 172.21.15.88:0/3064382498 >> 172.21.15.88:0/3064382498 conn(0x7f4318070aa0 msgr2=0x7f4318070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:21.603 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.595+0000 7f431edc4640 1 -- 172.21.15.88:0/3064382498 shutdown_connections 2023-10-16T20:06:21.603 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.595+0000 7f431edc4640 1 -- 172.21.15.88:0/3064382498 wait complete. 2023-10-16T20:06:21.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.595+0000 7f431edc4640 1 Processor -- start 2023-10-16T20:06:21.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.595+0000 7f431edc4640 1 -- start start 2023-10-16T20:06:21.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.595+0000 7f431edc4640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f431806b100 0x7f43181495b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:21.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.595+0000 7f431edc4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f431806b4e0 0x7f43180b4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:21.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.595+0000 7f431edc4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43180715c0 0x7f431815db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:21.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.595+0000 7f431edc4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4318159de0 con 0x7f431806b4e0 2023-10-16T20:06:21.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.595+0000 7f431edc4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43181602c0 con 0x7f43180715c0 2023-10-16T20:06:21.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.595+0000 7f431edc4640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43181605a0 con 0x7f431806b100 2023-10-16T20:06:21.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.595+0000 7f430ffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f431806b4e0 0x7f43180b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:21.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.595+0000 7f430ffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f431806b4e0 0x7f43180b4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45952/0 (socket says 172.21.15.88:45952) 2023-10-16T20:06:21.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.595+0000 7f430ffff640 1 -- 172.21.15.88:0/3522787184 learned_addr learned my addr 172.21.15.88:0/3522787184 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:21.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.595+0000 7f431cb60640 1 --2- 172.21.15.88:0/3522787184 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f431806b100 0x7f43181495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:21.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.595+0000 7f431d361640 1 --2- 172.21.15.88:0/3522787184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43180715c0 0x7f431815db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:21.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.599+0000 7f430ffff640 1 -- 172.21.15.88:0/3522787184 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f431806b100 msgr2=0x7f43181495b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:21.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.599+0000 7f430ffff640 1 --2- 172.21.15.88:0/3522787184 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f431806b100 0x7f43181495b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:21.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.599+0000 7f430ffff640 1 -- 172.21.15.88:0/3522787184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43180715c0 msgr2=0x7f431815db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:21.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.599+0000 7f430ffff640 1 --2- 172.21.15.88:0/3522787184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43180715c0 0x7f431815db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:21.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.599+0000 7f430ffff640 1 -- 172.21.15.88:0/3522787184 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43140094d0 con 0x7f431806b4e0 2023-10-16T20:06:21.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.599+0000 7f430ffff640 1 --2- 172.21.15.88:0/3522787184 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f431806b4e0 0x7f43180b4520 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f430800c7c0 tx=0x7f430800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:21.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.599+0000 7f430d7fa640 1 -- 172.21.15.88:0/3522787184 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4308007a30 con 0x7f431806b4e0 2023-10-16T20:06:21.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.599+0000 7f430d7fa640 1 -- 172.21.15.88:0/3522787184 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4308007b90 con 0x7f431806b4e0 2023-10-16T20:06:21.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.599+0000 7f431edc4640 1 -- 172.21.15.88:0/3522787184 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4318160880 con 0x7f431806b4e0 2023-10-16T20:06:21.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.599+0000 7f430d7fa640 1 -- 172.21.15.88:0/3522787184 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430800ce70 con 0x7f431806b4e0 2023-10-16T20:06:21.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.599+0000 7f431edc4640 1 -- 172.21.15.88:0/3522787184 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f431815f440 con 0x7f431806b4e0 2023-10-16T20:06:21.610 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.599+0000 7f431edc4640 1 -- 172.21.15.88:0/3522787184 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f431804e9c0 con 0x7f431806b4e0 2023-10-16T20:06:21.611 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.599+0000 7f430d7fa640 1 -- 172.21.15.88:0/3522787184 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f43080113f0 con 0x7f431806b4e0 2023-10-16T20:06:21.611 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.599+0000 7f430d7fa640 1 --2- 172.21.15.88:0/3522787184 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f43040664a0 0x7f4304068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:21.611 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.599+0000 7f430d7fa640 1 -- 172.21.15.88:0/3522787184 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f4308084440 con 0x7f431806b4e0 2023-10-16T20:06:21.611 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.599+0000 7f431cb60640 1 --2- 172.21.15.88:0/3522787184 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f43040664a0 0x7f4304068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:21.612 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.603+0000 7f431cb60640 1 --2- 172.21.15.88:0/3522787184 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f43040664a0 0x7f4304068960 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f43140ae000 tx=0x7f43140b5040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:21.612 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.603+0000 7f430d7fa640 1 -- 172.21.15.88:0/3522787184 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4308051800 con 0x7f431806b4e0 2023-10-16T20:06:21.678 INFO:tasks.ceph.mds.b.smithi094.stderr:2023-10-16T20:06:21.674+0000 7fa95b9d6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T20:06:21.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.751+0000 7f431edc4640 1 -- 172.21.15.88:0/3522787184 --> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f43180b1b80 con 0x7f43040664a0 2023-10-16T20:06:21.767 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.763+0000 7f430d7fa640 1 -- 172.21.15.88:0/3522787184 <== mgr.4110 v2:172.21.15.88:6832/4078483333 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287790 (secure 0 0 0) 0x7f43180b1b80 con 0x7f43040664a0 2023-10-16T20:06:21.770 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:06:21.772 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2023-10-16T20:06:21.774 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.767+0000 7f431edc4640 1 -- 172.21.15.88:0/3522787184 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f43040664a0 msgr2=0x7f4304068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:21.774 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.767+0000 7f431edc4640 1 --2- 172.21.15.88:0/3522787184 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f43040664a0 0x7f4304068960 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f43140ae000 tx=0x7f43140b5040 comp rx=0 tx=0).stop 2023-10-16T20:06:21.774 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.767+0000 7f431edc4640 1 -- 172.21.15.88:0/3522787184 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f431806b4e0 msgr2=0x7f43180b4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:21.774 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.767+0000 7f431edc4640 1 --2- 172.21.15.88:0/3522787184 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f431806b4e0 0x7f43180b4520 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f430800c7c0 tx=0x7f430800cc80 comp rx=0 tx=0).stop 2023-10-16T20:06:21.774 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.767+0000 7f431edc4640 1 -- 172.21.15.88:0/3522787184 shutdown_connections 2023-10-16T20:06:21.775 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.767+0000 7f431edc4640 1 --2- 172.21.15.88:0/3522787184 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f43040664a0 0x7f4304068960 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:21.775 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.767+0000 7f431edc4640 1 --2- 172.21.15.88:0/3522787184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43180715c0 0x7f431815db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:21.775 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.767+0000 7f431edc4640 1 --2- 172.21.15.88:0/3522787184 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f431806b4e0 0x7f43180b4520 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:21.775 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.767+0000 7f431edc4640 1 --2- 172.21.15.88:0/3522787184 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f431806b100 0x7f43181495b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:21.775 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.767+0000 7f431edc4640 1 -- 172.21.15.88:0/3522787184 >> 172.21.15.88:0/3522787184 conn(0x7f4318070aa0 msgr2=0x7f431806cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:21.776 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.767+0000 7f431edc4640 1 -- 172.21.15.88:0/3522787184 shutdown_connections 2023-10-16T20:06:21.776 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.767+0000 7f431edc4640 1 -- 172.21.15.88:0/3522787184 wait complete. 2023-10-16T20:06:21.791 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2023-10-16T20:06:20.462297+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":172,"acting":172,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":41,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233392,"kb_used_data":4700,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754741328,"statfs":{"total":773094113280,"available":772855119872,"internally_reserved":0,"allocated":4812800,"data_stored":2705780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":234016313},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.760440"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.220769+0000","last_change":"2023-10-16T20:06:17.217901+0000","last_active":"2023-10-16T20:06:17.220769+0000","last_peered":"2023-10-16T20:06:17.220769+0000","last_clean":"2023-10-16T20:06:17.220769+0000","last_became_active":"2023-10-16T20:06:17.217583+0000","last_became_peered":"2023-10-16T20:06:17.217583+0000","last_unstale":"2023-10-16T20:06:17.220769+0000","last_undegraded":"2023-10-16T20:06:17.220769+0000","last_fullsized":"2023-10-16T20:06:17.220769+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:24:29.941143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.215707+0000","last_change":"2023-10-16T20:06:17.211493+0000","last_active":"2023-10-16T20:06:17.215707+0000","last_peered":"2023-10-16T20:06:17.215707+0000","last_clean":"2023-10-16T20:06:17.215707+0000","last_became_active":"2023-10-16T20:06:17.211196+0000","last_became_peered":"2023-10-16T20:06:17.211196+0000","last_unstale":"2023-10-16T20:06:17.215707+0000","last_undegraded":"2023-10-16T20:06:17.215707+0000","last_fullsized":"2023-10-16T20:06:17.215707+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:50:32.312772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T20:06:16.698995+0000","last_change":"2023-10-16T20:06:16.698995+0000","last_active":"2023-10-16T20:06:16.698995+0000","last_peered":"2023-10-16T20:06:16.698995+0000","last_clean":"2023-10-16T20:06:16.698995+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T20:06:16.698995+0000","last_undegraded":"2023-10-16T20:06:16.698995+0000","last_fullsized":"2023-10-16T20:06:16.698995+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357788+0000","last_change":"2023-10-16T20:06:17.216483+0000","last_active":"2023-10-16T20:06:18.357788+0000","last_peered":"2023-10-16T20:06:18.357788+0000","last_clean":"2023-10-16T20:06:18.357788+0000","last_became_active":"2023-10-16T20:06:17.216202+0000","last_became_peered":"2023-10-16T20:06:17.216202+0000","last_unstale":"2023-10-16T20:06:18.357788+0000","last_undegraded":"2023-10-16T20:06:18.357788+0000","last_fullsized":"2023-10-16T20:06:18.357788+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:32:02.453729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357619+0000","last_change":"2023-10-16T20:06:17.208325+0000","last_active":"2023-10-16T20:06:18.357619+0000","last_peered":"2023-10-16T20:06:18.357619+0000","last_clean":"2023-10-16T20:06:18.357619+0000","last_became_active":"2023-10-16T20:06:17.208061+0000","last_became_peered":"2023-10-16T20:06:17.208061+0000","last_unstale":"2023-10-16T20:06:18.357619+0000","last_undegraded":"2023-10-16T20:06:18.357619+0000","last_fullsized":"2023-10-16T20:06:18.357619+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:02:28.165753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.217274+0000","last_change":"2023-10-16T20:06:17.208416+0000","last_active":"2023-10-16T20:06:17.217274+0000","last_peered":"2023-10-16T20:06:17.217274+0000","last_clean":"2023-10-16T20:06:17.217274+0000","last_became_active":"2023-10-16T20:06:17.207713+0000","last_became_peered":"2023-10-16T20:06:17.207713+0000","last_unstale":"2023-10-16T20:06:17.217274+0000","last_undegraded":"2023-10-16T20:06:17.217274+0000","last_fullsized":"2023-10-16T20:06:17.217274+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:39:06.086998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.213838+0000","last_change":"2023-10-16T20:06:17.206736+0000","last_active":"2023-10-16T20:06:17.213838+0000","last_peered":"2023-10-16T20:06:17.213838+0000","last_clean":"2023-10-16T20:06:17.213838+0000","last_became_active":"2023-10-16T20:06:17.206435+0000","last_became_peered":"2023-10-16T20:06:17.206435+0000","last_unstale":"2023-10-16T20:06:17.213838+0000","last_undegraded":"2023-10-16T20:06:17.213838+0000","last_fullsized":"2023-10-16T20:06:17.213838+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:50:32.312772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.218213+0000","last_change":"2023-10-16T20:06:17.214147+0000","last_active":"2023-10-16T20:06:17.218213+0000","last_peered":"2023-10-16T20:06:17.218213+0000","last_clean":"2023-10-16T20:06:17.218213+0000","last_became_active":"2023-10-16T20:06:17.213998+0000","last_became_peered":"2023-10-16T20:06:17.213998+0000","last_unstale":"2023-10-16T20:06:17.218213+0000","last_undegraded":"2023-10-16T20:06:17.218213+0000","last_fullsized":"2023-10-16T20:06:17.218213+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:36:27.881926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T20:06:16.698995+0000","last_change":"2023-10-16T20:06:16.698995+0000","last_active":"2023-10-16T20:06:16.698995+0000","last_peered":"2023-10-16T20:06:16.698995+0000","last_clean":"2023-10-16T20:06:16.698995+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T20:06:16.698995+0000","last_undegraded":"2023-10-16T20:06:16.698995+0000","last_fullsized":"2023-10-16T20:06:16.698995+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.222492+0000","last_change":"2023-10-16T20:06:17.218989+0000","last_active":"2023-10-16T20:06:17.222492+0000","last_peered":"2023-10-16T20:06:17.222492+0000","last_clean":"2023-10-16T20:06:17.222492+0000","last_became_active":"2023-10-16T20:06:17.218726+0000","last_became_peered":"2023-10-16T20:06:17.218726+0000","last_unstale":"2023-10-16T20:06:17.222492+0000","last_undegraded":"2023-10-16T20:06:17.222492+0000","last_fullsized":"2023-10-16T20:06:17.222492+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:27:39.178553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.221629+0000","last_change":"2023-10-16T20:06:17.217372+0000","last_active":"2023-10-16T20:06:17.221629+0000","last_peered":"2023-10-16T20:06:17.221629+0000","last_clean":"2023-10-16T20:06:17.221629+0000","last_became_active":"2023-10-16T20:06:17.216730+0000","last_became_peered":"2023-10-16T20:06:17.216730+0000","last_unstale":"2023-10-16T20:06:17.221629+0000","last_undegraded":"2023-10-16T20:06:17.221629+0000","last_fullsized":"2023-10-16T20:06:17.221629+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:18:14.480179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T20:06:16.698995+0000","last_change":"2023-10-16T20:06:16.698995+0000","last_active":"2023-10-16T20:06:16.698995+0000","last_peered":"2023-10-16T20:06:16.698995+0000","last_clean":"2023-10-16T20:06:16.698995+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T20:06:16.698995+0000","last_undegraded":"2023-10-16T20:06:16.698995+0000","last_fullsized":"2023-10-16T20:06:16.698995+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.221752+0000","last_change":"2023-10-16T20:06:17.218455+0000","last_active":"2023-10-16T20:06:17.221752+0000","last_peered":"2023-10-16T20:06:17.221752+0000","last_clean":"2023-10-16T20:06:17.221752+0000","last_became_active":"2023-10-16T20:06:17.217565+0000","last_became_peered":"2023-10-16T20:06:17.217565+0000","last_unstale":"2023-10-16T20:06:17.221752+0000","last_undegraded":"2023-10-16T20:06:17.221752+0000","last_fullsized":"2023-10-16T20:06:17.221752+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:13.874932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.222220+0000","last_change":"2023-10-16T20:06:17.214285+0000","last_active":"2023-10-16T20:06:17.222220+0000","last_peered":"2023-10-16T20:06:17.222220+0000","last_clean":"2023-10-16T20:06:17.222220+0000","last_became_active":"2023-10-16T20:06:17.213551+0000","last_became_peered":"2023-10-16T20:06:17.213551+0000","last_unstale":"2023-10-16T20:06:17.222220+0000","last_undegraded":"2023-10-16T20:06:17.222220+0000","last_fullsized":"2023-10-16T20:06:17.222220+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:10:43.145100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.213855+0000","last_change":"2023-10-16T20:06:17.208025+0000","last_active":"2023-10-16T20:06:17.213855+0000","last_peered":"2023-10-16T20:06:17.213855+0000","last_clean":"2023-10-16T20:06:17.213855+0000","last_became_active":"2023-10-16T20:06:17.207739+0000","last_became_peered":"2023-10-16T20:06:17.207739+0000","last_unstale":"2023-10-16T20:06:17.213855+0000","last_undegraded":"2023-10-16T20:06:17.213855+0000","last_fullsized":"2023-10-16T20:06:17.213855+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:36:27.881926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196502+0000","last_change":"2023-10-16T20:06:17.222232+0000","last_active":"2023-10-16T20:06:18.196502+0000","last_peered":"2023-10-16T20:06:18.196502+0000","last_clean":"2023-10-16T20:06:18.196502+0000","last_became_active":"2023-10-16T20:06:17.221892+0000","last_became_peered":"2023-10-16T20:06:17.221892+0000","last_unstale":"2023-10-16T20:06:18.196502+0000","last_undegraded":"2023-10-16T20:06:18.196502+0000","last_fullsized":"2023-10-16T20:06:18.196502+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:58:45.132214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197149+0000","last_change":"2023-10-16T20:06:17.224446+0000","last_active":"2023-10-16T20:06:18.197149+0000","last_peered":"2023-10-16T20:06:18.197149+0000","last_clean":"2023-10-16T20:06:18.197149+0000","last_became_active":"2023-10-16T20:06:17.224113+0000","last_became_peered":"2023-10-16T20:06:17.224113+0000","last_unstale":"2023-10-16T20:06:18.197149+0000","last_undegraded":"2023-10-16T20:06:18.197149+0000","last_fullsized":"2023-10-16T20:06:18.197149+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:37:43.948080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198392+0000","last_change":"2023-10-16T20:06:17.214181+0000","last_active":"2023-10-16T20:06:18.198392+0000","last_peered":"2023-10-16T20:06:18.198392+0000","last_clean":"2023-10-16T20:06:18.198392+0000","last_became_active":"2023-10-16T20:06:17.213876+0000","last_became_peered":"2023-10-16T20:06:17.213876+0000","last_unstale":"2023-10-16T20:06:18.198392+0000","last_undegraded":"2023-10-16T20:06:18.198392+0000","last_fullsized":"2023-10-16T20:06:18.198392+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:39:06.086998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357861+0000","last_change":"2023-10-16T20:06:17.205826+0000","last_active":"2023-10-16T20:06:18.357861+0000","last_peered":"2023-10-16T20:06:18.357861+0000","last_clean":"2023-10-16T20:06:18.357861+0000","last_became_active":"2023-10-16T20:06:17.204888+0000","last_became_peered":"2023-10-16T20:06:17.204888+0000","last_unstale":"2023-10-16T20:06:18.357861+0000","last_undegraded":"2023-10-16T20:06:18.357861+0000","last_fullsized":"2023-10-16T20:06:18.357861+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:29:06.675309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197777+0000","last_change":"2023-10-16T20:06:17.217103+0000","last_active":"2023-10-16T20:06:18.197777+0000","last_peered":"2023-10-16T20:06:18.197777+0000","last_clean":"2023-10-16T20:06:18.197777+0000","last_became_active":"2023-10-16T20:06:17.216717+0000","last_became_peered":"2023-10-16T20:06:17.216717+0000","last_unstale":"2023-10-16T20:06:18.197777+0000","last_undegraded":"2023-10-16T20:06:18.197777+0000","last_fullsized":"2023-10-16T20:06:18.197777+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:20:06.934366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.206185+0000","last_change":"2023-10-16T20:06:17.201875+0000","last_active":"2023-10-16T20:06:17.206185+0000","last_peered":"2023-10-16T20:06:17.206185+0000","last_clean":"2023-10-16T20:06:17.206185+0000","last_became_active":"2023-10-16T20:06:17.200537+0000","last_became_peered":"2023-10-16T20:06:17.200537+0000","last_unstale":"2023-10-16T20:06:17.206185+0000","last_undegraded":"2023-10-16T20:06:17.206185+0000","last_fullsized":"2023-10-16T20:06:17.206185+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:23:28.625212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.216225+0000","last_change":"2023-10-16T20:06:17.211432+0000","last_active":"2023-10-16T20:06:17.216225+0000","last_peered":"2023-10-16T20:06:17.216225+0000","last_clean":"2023-10-16T20:06:17.216225+0000","last_became_active":"2023-10-16T20:06:17.211100+0000","last_became_peered":"2023-10-16T20:06:17.211100+0000","last_unstale":"2023-10-16T20:06:17.216225+0000","last_undegraded":"2023-10-16T20:06:17.216225+0000","last_fullsized":"2023-10-16T20:06:17.216225+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:06:45.516915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196828+0000","last_change":"2023-10-16T20:06:17.225476+0000","last_active":"2023-10-16T20:06:18.196828+0000","last_peered":"2023-10-16T20:06:18.196828+0000","last_clean":"2023-10-16T20:06:18.196828+0000","last_became_active":"2023-10-16T20:06:17.224667+0000","last_became_peered":"2023-10-16T20:06:17.224667+0000","last_unstale":"2023-10-16T20:06:18.196828+0000","last_undegraded":"2023-10-16T20:06:18.196828+0000","last_fullsized":"2023-10-16T20:06:18.196828+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:31:31.346356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357610+0000","last_change":"2023-10-16T20:06:17.216704+0000","last_active":"2023-10-16T20:06:18.357610+0000","last_peered":"2023-10-16T20:06:18.357610+0000","last_clean":"2023-10-16T20:06:18.357610+0000","last_became_active":"2023-10-16T20:06:17.216341+0000","last_became_peered":"2023-10-16T20:06:17.216341+0000","last_unstale":"2023-10-16T20:06:18.357610+0000","last_undegraded":"2023-10-16T20:06:18.357610+0000","last_fullsized":"2023-10-16T20:06:18.357610+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:14:46.135352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198413+0000","last_change":"2023-10-16T20:06:17.216103+0000","last_active":"2023-10-16T20:06:18.198413+0000","last_peered":"2023-10-16T20:06:18.198413+0000","last_clean":"2023-10-16T20:06:18.198413+0000","last_became_active":"2023-10-16T20:06:17.215844+0000","last_became_peered":"2023-10-16T20:06:17.215844+0000","last_unstale":"2023-10-16T20:06:18.198413+0000","last_undegraded":"2023-10-16T20:06:18.198413+0000","last_fullsized":"2023-10-16T20:06:18.198413+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:01.646291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.216083+0000","last_change":"2023-10-16T20:06:17.207691+0000","last_active":"2023-10-16T20:06:17.216083+0000","last_peered":"2023-10-16T20:06:17.216083+0000","last_clean":"2023-10-16T20:06:17.216083+0000","last_became_active":"2023-10-16T20:06:17.207437+0000","last_became_peered":"2023-10-16T20:06:17.207437+0000","last_unstale":"2023-10-16T20:06:17.216083+0000","last_undegraded":"2023-10-16T20:06:17.216083+0000","last_fullsized":"2023-10-16T20:06:17.216083+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:23:28.625212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198320+0000","last_change":"2023-10-16T20:06:17.211359+0000","last_active":"2023-10-16T20:06:18.198320+0000","last_peered":"2023-10-16T20:06:18.198320+0000","last_clean":"2023-10-16T20:06:18.198320+0000","last_became_active":"2023-10-16T20:06:17.210506+0000","last_became_peered":"2023-10-16T20:06:17.210506+0000","last_unstale":"2023-10-16T20:06:18.198320+0000","last_undegraded":"2023-10-16T20:06:18.198320+0000","last_fullsized":"2023-10-16T20:06:18.198320+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:32:44.093091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.222529+0000","last_change":"2023-10-16T20:06:17.219679+0000","last_active":"2023-10-16T20:06:17.222529+0000","last_peered":"2023-10-16T20:06:17.222529+0000","last_clean":"2023-10-16T20:06:17.222529+0000","last_became_active":"2023-10-16T20:06:17.214175+0000","last_became_peered":"2023-10-16T20:06:17.214175+0000","last_unstale":"2023-10-16T20:06:17.222529+0000","last_undegraded":"2023-10-16T20:06:17.222529+0000","last_fullsized":"2023-10-16T20:06:17.222529+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:22:32.981812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197630+0000","last_change":"2023-10-16T20:06:17.210945+0000","last_active":"2023-10-16T20:06:18.197630+0000","last_peered":"2023-10-16T20:06:18.197630+0000","last_clean":"2023-10-16T20:06:18.197630+0000","last_became_active":"2023-10-16T20:06:17.210629+0000","last_became_peered":"2023-10-16T20:06:17.210629+0000","last_unstale":"2023-10-16T20:06:18.197630+0000","last_undegraded":"2023-10-16T20:06:18.197630+0000","last_fullsized":"2023-10-16T20:06:18.197630+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:27:10.715147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197643+0000","last_change":"2023-10-16T20:06:17.212963+0000","last_active":"2023-10-16T20:06:18.197643+0000","last_peered":"2023-10-16T20:06:18.197643+0000","last_clean":"2023-10-16T20:06:18.197643+0000","last_became_active":"2023-10-16T20:06:17.212693+0000","last_became_peered":"2023-10-16T20:06:17.212693+0000","last_unstale":"2023-10-16T20:06:18.197643+0000","last_undegraded":"2023-10-16T20:06:18.197643+0000","last_fullsized":"2023-10-16T20:06:18.197643+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:10:43.145100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197558+0000","last_change":"2023-10-16T20:06:17.219075+0000","last_active":"2023-10-16T20:06:18.197558+0000","last_peered":"2023-10-16T20:06:18.197558+0000","last_clean":"2023-10-16T20:06:18.197558+0000","last_became_active":"2023-10-16T20:06:17.218774+0000","last_became_peered":"2023-10-16T20:06:17.218774+0000","last_unstale":"2023-10-16T20:06:18.197558+0000","last_undegraded":"2023-10-16T20:06:18.197558+0000","last_fullsized":"2023-10-16T20:06:18.197558+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:05:25.874296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.209799+0000","last_change":"2023-10-16T20:06:17.200644+0000","last_active":"2023-10-16T20:06:17.209799+0000","last_peered":"2023-10-16T20:06:17.209799+0000","last_clean":"2023-10-16T20:06:17.209799+0000","last_became_active":"2023-10-16T20:06:17.200350+0000","last_became_peered":"2023-10-16T20:06:17.200350+0000","last_unstale":"2023-10-16T20:06:17.209799+0000","last_undegraded":"2023-10-16T20:06:17.209799+0000","last_fullsized":"2023-10-16T20:06:17.209799+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:58:21.190657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196740+0000","last_change":"2023-10-16T20:06:17.224294+0000","last_active":"2023-10-16T20:06:18.196740+0000","last_peered":"2023-10-16T20:06:18.196740+0000","last_clean":"2023-10-16T20:06:18.196740+0000","last_became_active":"2023-10-16T20:06:17.223988+0000","last_became_peered":"2023-10-16T20:06:17.223988+0000","last_unstale":"2023-10-16T20:06:18.196740+0000","last_undegraded":"2023-10-16T20:06:18.196740+0000","last_fullsized":"2023-10-16T20:06:18.196740+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:03:40.257975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"40'4","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198438+0000","last_change":"2023-10-16T20:06:16.732366+0000","last_active":"2023-10-16T20:06:18.198438+0000","last_peered":"2023-10-16T20:06:18.198438+0000","last_clean":"2023-10-16T20:06:18.198438+0000","last_became_active":"2023-10-16T20:06:16.732035+0000","last_became_peered":"2023-10-16T20:06:16.732035+0000","last_unstale":"2023-10-16T20:06:18.198438+0000","last_undegraded":"2023-10-16T20:06:18.198438+0000","last_fullsized":"2023-10-16T20:06:18.198438+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:35:43.770049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.216156+0000","last_change":"2023-10-16T20:06:17.207721+0000","last_active":"2023-10-16T20:06:17.216156+0000","last_peered":"2023-10-16T20:06:17.216156+0000","last_clean":"2023-10-16T20:06:17.216156+0000","last_became_active":"2023-10-16T20:06:17.207470+0000","last_became_peered":"2023-10-16T20:06:17.207470+0000","last_unstale":"2023-10-16T20:06:17.216156+0000","last_undegraded":"2023-10-16T20:06:17.216156+0000","last_fullsized":"2023-10-16T20:06:17.216156+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:56:27.147860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T20:06:15.688195+0000","last_change":"2023-10-16T20:06:15.688195+0000","last_active":"2023-10-16T20:06:15.688195+0000","last_peered":"2023-10-16T20:06:15.688195+0000","last_clean":"2023-10-16T20:06:15.688195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T20:06:15.688195+0000","last_undegraded":"2023-10-16T20:06:15.688195+0000","last_fullsized":"2023-10-16T20:06:15.688195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.222274+0000","last_change":"2023-10-16T20:06:17.215735+0000","last_active":"2023-10-16T20:06:17.222274+0000","last_peered":"2023-10-16T20:06:17.222274+0000","last_clean":"2023-10-16T20:06:17.222274+0000","last_became_active":"2023-10-16T20:06:17.214986+0000","last_became_peered":"2023-10-16T20:06:17.214986+0000","last_unstale":"2023-10-16T20:06:17.222274+0000","last_undegraded":"2023-10-16T20:06:17.222274+0000","last_fullsized":"2023-10-16T20:06:17.222274+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:35:44.780849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"40'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.733064+0000","last_change":"2023-10-16T20:06:16.739387+0000","last_active":"2023-10-16T20:06:17.733064+0000","last_peered":"2023-10-16T20:06:17.733064+0000","last_clean":"2023-10-16T20:06:17.733064+0000","last_became_active":"2023-10-16T20:06:16.739070+0000","last_became_peered":"2023-10-16T20:06:16.739070+0000","last_unstale":"2023-10-16T20:06:17.733064+0000","last_undegraded":"2023-10-16T20:06:17.733064+0000","last_fullsized":"2023-10-16T20:06:17.733064+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:04:52.152758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.217801+0000","last_change":"2023-10-16T20:06:17.213835+0000","last_active":"2023-10-16T20:06:17.217801+0000","last_peered":"2023-10-16T20:06:17.217801+0000","last_clean":"2023-10-16T20:06:17.217801+0000","last_became_active":"2023-10-16T20:06:17.213516+0000","last_became_peered":"2023-10-16T20:06:17.213516+0000","last_unstale":"2023-10-16T20:06:17.217801+0000","last_undegraded":"2023-10-16T20:06:17.217801+0000","last_fullsized":"2023-10-16T20:06:17.217801+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:25:13.284821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"40'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.732355+0000","last_change":"2023-10-16T20:06:16.730019+0000","last_active":"2023-10-16T20:06:17.732355+0000","last_peered":"2023-10-16T20:06:17.732355+0000","last_clean":"2023-10-16T20:06:17.732355+0000","last_became_active":"2023-10-16T20:06:16.729804+0000","last_became_peered":"2023-10-16T20:06:16.729804+0000","last_unstale":"2023-10-16T20:06:17.732355+0000","last_undegraded":"2023-10-16T20:06:17.732355+0000","last_fullsized":"2023-10-16T20:06:17.732355+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:56:26.137061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T20:06:16.698995+0000","last_change":"2023-10-16T20:06:16.698995+0000","last_active":"2023-10-16T20:06:16.698995+0000","last_peered":"2023-10-16T20:06:16.698995+0000","last_clean":"2023-10-16T20:06:16.698995+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T20:06:16.698995+0000","last_undegraded":"2023-10-16T20:06:16.698995+0000","last_fullsized":"2023-10-16T20:06:16.698995+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"40'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.732663+0000","last_change":"2023-10-16T20:06:16.731997+0000","last_active":"2023-10-16T20:06:17.732663+0000","last_peered":"2023-10-16T20:06:17.732663+0000","last_clean":"2023-10-16T20:06:17.732663+0000","last_became_active":"2023-10-16T20:06:16.731714+0000","last_became_peered":"2023-10-16T20:06:16.731714+0000","last_unstale":"2023-10-16T20:06:17.732663+0000","last_undegraded":"2023-10-16T20:06:17.732663+0000","last_fullsized":"2023-10-16T20:06:17.732663+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:25:12.274021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.220705+0000","last_change":"2023-10-16T20:06:17.214298+0000","last_active":"2023-10-16T20:06:17.220705+0000","last_peered":"2023-10-16T20:06:17.220705+0000","last_clean":"2023-10-16T20:06:17.220705+0000","last_became_active":"2023-10-16T20:06:17.214038+0000","last_became_peered":"2023-10-16T20:06:17.214038+0000","last_unstale":"2023-10-16T20:06:17.220705+0000","last_undegraded":"2023-10-16T20:06:17.220705+0000","last_fullsized":"2023-10-16T20:06:17.220705+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:38:40.789443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"40'1","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196763+0000","last_change":"2023-10-16T20:06:16.737162+0000","last_active":"2023-10-16T20:06:18.196763+0000","last_peered":"2023-10-16T20:06:18.196763+0000","last_clean":"2023-10-16T20:06:18.196763+0000","last_became_active":"2023-10-16T20:06:16.736888+0000","last_became_peered":"2023-10-16T20:06:16.736888+0000","last_unstale":"2023-10-16T20:06:18.196763+0000","last_undegraded":"2023-10-16T20:06:18.196763+0000","last_fullsized":"2023-10-16T20:06:18.196763+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:22:31.971013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.211586+0000","last_change":"2023-10-16T20:06:17.208067+0000","last_active":"2023-10-16T20:06:17.211586+0000","last_peered":"2023-10-16T20:06:17.211586+0000","last_clean":"2023-10-16T20:06:17.211586+0000","last_became_active":"2023-10-16T20:06:17.207799+0000","last_became_peered":"2023-10-16T20:06:17.207799+0000","last_unstale":"2023-10-16T20:06:17.211586+0000","last_undegraded":"2023-10-16T20:06:17.211586+0000","last_fullsized":"2023-10-16T20:06:17.211586+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:40:58.934858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.195959+0000","last_change":"2023-10-16T20:06:16.729238+0000","last_active":"2023-10-16T20:06:17.195959+0000","last_peered":"2023-10-16T20:06:17.195959+0000","last_clean":"2023-10-16T20:06:17.195959+0000","last_became_active":"2023-10-16T20:06:16.728975+0000","last_became_peered":"2023-10-16T20:06:16.728975+0000","last_unstale":"2023-10-16T20:06:17.195959+0000","last_undegraded":"2023-10-16T20:06:17.195959+0000","last_fullsized":"2023-10-16T20:06:17.195959+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:36:26.871127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.220833+0000","last_change":"2023-10-16T20:06:17.218801+0000","last_active":"2023-10-16T20:06:17.220833+0000","last_peered":"2023-10-16T20:06:17.220833+0000","last_clean":"2023-10-16T20:06:17.220833+0000","last_became_active":"2023-10-16T20:06:17.218219+0000","last_became_peered":"2023-10-16T20:06:17.218219+0000","last_unstale":"2023-10-16T20:06:17.220833+0000","last_undegraded":"2023-10-16T20:06:17.220833+0000","last_fullsized":"2023-10-16T20:06:17.220833+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:27:39.178553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.202521+0000","last_change":"2023-10-16T20:06:16.720994+0000","last_active":"2023-10-16T20:06:17.202521+0000","last_peered":"2023-10-16T20:06:17.202521+0000","last_clean":"2023-10-16T20:06:17.202521+0000","last_became_active":"2023-10-16T20:06:16.720623+0000","last_became_peered":"2023-10-16T20:06:16.720623+0000","last_unstale":"2023-10-16T20:06:17.202521+0000","last_undegraded":"2023-10-16T20:06:17.202521+0000","last_fullsized":"2023-10-16T20:06:17.202521+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:32:43.082291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357768+0000","last_change":"2023-10-16T20:06:17.216335+0000","last_active":"2023-10-16T20:06:18.357768+0000","last_peered":"2023-10-16T20:06:18.357768+0000","last_clean":"2023-10-16T20:06:18.357768+0000","last_became_active":"2023-10-16T20:06:17.216070+0000","last_became_peered":"2023-10-16T20:06:17.216070+0000","last_unstale":"2023-10-16T20:06:18.357768+0000","last_undegraded":"2023-10-16T20:06:18.357768+0000","last_fullsized":"2023-10-16T20:06:18.357768+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:08:22.444705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.206243+0000","last_change":"2023-10-16T20:06:16.736582+0000","last_active":"2023-10-16T20:06:17.206243+0000","last_peered":"2023-10-16T20:06:17.206243+0000","last_clean":"2023-10-16T20:06:17.206243+0000","last_became_active":"2023-10-16T20:06:16.736322+0000","last_became_peered":"2023-10-16T20:06:16.736322+0000","last_unstale":"2023-10-16T20:06:17.206243+0000","last_undegraded":"2023-10-16T20:06:17.206243+0000","last_fullsized":"2023-10-16T20:06:17.206243+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:38:29.780087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197108+0000","last_change":"2023-10-16T20:06:17.220312+0000","last_active":"2023-10-16T20:06:18.197108+0000","last_peered":"2023-10-16T20:06:18.197108+0000","last_clean":"2023-10-16T20:06:18.197108+0000","last_became_active":"2023-10-16T20:06:17.220033+0000","last_became_peered":"2023-10-16T20:06:17.220033+0000","last_unstale":"2023-10-16T20:06:18.197108+0000","last_undegraded":"2023-10-16T20:06:18.197108+0000","last_fullsized":"2023-10-16T20:06:18.197108+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:00:59.428210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.198151+0000","last_change":"2023-10-16T20:06:16.725615+0000","last_active":"2023-10-16T20:06:17.198151+0000","last_peered":"2023-10-16T20:06:17.198151+0000","last_clean":"2023-10-16T20:06:17.198151+0000","last_became_active":"2023-10-16T20:06:16.725257+0000","last_became_peered":"2023-10-16T20:06:16.725257+0000","last_unstale":"2023-10-16T20:06:17.198151+0000","last_undegraded":"2023-10-16T20:06:17.198151+0000","last_fullsized":"2023-10-16T20:06:17.198151+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:36:24.034397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T20:06:16.698995+0000","last_change":"2023-10-16T20:06:16.698995+0000","last_active":"2023-10-16T20:06:16.698995+0000","last_peered":"2023-10-16T20:06:16.698995+0000","last_clean":"2023-10-16T20:06:16.698995+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T20:06:16.698995+0000","last_undegraded":"2023-10-16T20:06:16.698995+0000","last_fullsized":"2023-10-16T20:06:16.698995+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"40'3","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.733519+0000","last_change":"2023-10-16T20:06:16.734102+0000","last_active":"2023-10-16T20:06:17.733519+0000","last_peered":"2023-10-16T20:06:17.733519+0000","last_clean":"2023-10-16T20:06:17.733519+0000","last_became_active":"2023-10-16T20:06:16.733829+0000","last_became_peered":"2023-10-16T20:06:16.733829+0000","last_unstale":"2023-10-16T20:06:17.733519+0000","last_undegraded":"2023-10-16T20:06:17.733519+0000","last_fullsized":"2023-10-16T20:06:17.733519+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:32:58.366700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.221860+0000","last_change":"2023-10-16T20:06:17.214205+0000","last_active":"2023-10-16T20:06:17.221860+0000","last_peered":"2023-10-16T20:06:17.221860+0000","last_clean":"2023-10-16T20:06:17.221860+0000","last_became_active":"2023-10-16T20:06:17.213261+0000","last_became_peered":"2023-10-16T20:06:17.213261+0000","last_unstale":"2023-10-16T20:06:17.221860+0000","last_undegraded":"2023-10-16T20:06:17.221860+0000","last_fullsized":"2023-10-16T20:06:17.221860+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:20:06.934366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"40'6","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.738226+0000","last_change":"2023-10-16T20:06:16.721479+0000","last_active":"2023-10-16T20:06:17.738226+0000","last_peered":"2023-10-16T20:06:17.738226+0000","last_clean":"2023-10-16T20:06:17.738226+0000","last_became_active":"2023-10-16T20:06:16.721136+0000","last_became_peered":"2023-10-16T20:06:16.721136+0000","last_unstale":"2023-10-16T20:06:17.738226+0000","last_undegraded":"2023-10-16T20:06:17.738226+0000","last_fullsized":"2023-10-16T20:06:17.738226+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:32:58.366700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T20:06:16.698995+0000","last_change":"2023-10-16T20:06:16.698995+0000","last_active":"2023-10-16T20:06:16.698995+0000","last_peered":"2023-10-16T20:06:16.698995+0000","last_clean":"2023-10-16T20:06:16.698995+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T20:06:16.698995+0000","last_undegraded":"2023-10-16T20:06:16.698995+0000","last_fullsized":"2023-10-16T20:06:16.698995+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"40'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.732488+0000","last_change":"2023-10-16T20:06:16.735005+0000","last_active":"2023-10-16T20:06:17.732488+0000","last_peered":"2023-10-16T20:06:17.732488+0000","last_clean":"2023-10-16T20:06:17.732488+0000","last_became_active":"2023-10-16T20:06:16.734752+0000","last_became_peered":"2023-10-16T20:06:16.734752+0000","last_unstale":"2023-10-16T20:06:17.732488+0000","last_undegraded":"2023-10-16T20:06:17.732488+0000","last_fullsized":"2023-10-16T20:06:17.732488+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:21:23.740779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.219739+0000","last_change":"2023-10-16T20:06:17.215993+0000","last_active":"2023-10-16T20:06:17.219739+0000","last_peered":"2023-10-16T20:06:17.219739+0000","last_clean":"2023-10-16T20:06:17.219739+0000","last_became_active":"2023-10-16T20:06:17.215736+0000","last_became_peered":"2023-10-16T20:06:17.215736+0000","last_unstale":"2023-10-16T20:06:17.219739+0000","last_undegraded":"2023-10-16T20:06:17.219739+0000","last_fullsized":"2023-10-16T20:06:17.219739+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:36:25.045197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.195984+0000","last_change":"2023-10-16T20:06:16.733426+0000","last_active":"2023-10-16T20:06:17.195984+0000","last_peered":"2023-10-16T20:06:17.195984+0000","last_clean":"2023-10-16T20:06:17.195984+0000","last_became_active":"2023-10-16T20:06:16.732600+0000","last_became_peered":"2023-10-16T20:06:16.732600+0000","last_unstale":"2023-10-16T20:06:17.195984+0000","last_undegraded":"2023-10-16T20:06:17.195984+0000","last_fullsized":"2023-10-16T20:06:17.195984+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:12.864132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196653+0000","last_change":"2023-10-16T20:06:17.219925+0000","last_active":"2023-10-16T20:06:18.196653+0000","last_peered":"2023-10-16T20:06:18.196653+0000","last_clean":"2023-10-16T20:06:18.196653+0000","last_became_active":"2023-10-16T20:06:17.219584+0000","last_became_peered":"2023-10-16T20:06:17.219584+0000","last_unstale":"2023-10-16T20:06:18.196653+0000","last_undegraded":"2023-10-16T20:06:18.196653+0000","last_fullsized":"2023-10-16T20:06:18.196653+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:44:14.727872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197391+0000","last_change":"2023-10-16T20:06:16.732499+0000","last_active":"2023-10-16T20:06:18.197391+0000","last_peered":"2023-10-16T20:06:18.197391+0000","last_clean":"2023-10-16T20:06:18.197391+0000","last_became_active":"2023-10-16T20:06:16.732244+0000","last_became_peered":"2023-10-16T20:06:16.732244+0000","last_unstale":"2023-10-16T20:06:18.197391+0000","last_undegraded":"2023-10-16T20:06:18.197391+0000","last_fullsized":"2023-10-16T20:06:18.197391+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:40:57.924059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.219863+0000","last_change":"2023-10-16T20:06:17.215267+0000","last_active":"2023-10-16T20:06:17.219863+0000","last_peered":"2023-10-16T20:06:17.219863+0000","last_clean":"2023-10-16T20:06:17.219863+0000","last_became_active":"2023-10-16T20:06:17.215017+0000","last_became_peered":"2023-10-16T20:06:17.215017+0000","last_unstale":"2023-10-16T20:06:17.219863+0000","last_undegraded":"2023-10-16T20:06:17.219863+0000","last_fullsized":"2023-10-16T20:06:17.219863+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:38:40.789443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"40'8","reported_seq":20,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196540+0000","last_change":"2023-10-16T20:06:16.730165+0000","last_active":"2023-10-16T20:06:18.196540+0000","last_peered":"2023-10-16T20:06:18.196540+0000","last_clean":"2023-10-16T20:06:18.196540+0000","last_became_active":"2023-10-16T20:06:16.729894+0000","last_became_peered":"2023-10-16T20:06:16.729894+0000","last_unstale":"2023-10-16T20:06:18.196540+0000","last_undegraded":"2023-10-16T20:06:18.196540+0000","last_fullsized":"2023-10-16T20:06:18.196540+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:36:26.871127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.217655+0000","last_change":"2023-10-16T20:06:17.214039+0000","last_active":"2023-10-16T20:06:17.217655+0000","last_peered":"2023-10-16T20:06:17.217655+0000","last_clean":"2023-10-16T20:06:17.217655+0000","last_became_active":"2023-10-16T20:06:17.213551+0000","last_became_peered":"2023-10-16T20:06:17.213551+0000","last_unstale":"2023-10-16T20:06:17.217655+0000","last_undegraded":"2023-10-16T20:06:17.217655+0000","last_fullsized":"2023-10-16T20:06:17.217655+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:22:32.981812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"40'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.735204+0000","last_change":"2023-10-16T20:06:16.726366+0000","last_active":"2023-10-16T20:06:17.735204+0000","last_peered":"2023-10-16T20:06:17.735204+0000","last_clean":"2023-10-16T20:06:17.735204+0000","last_became_active":"2023-10-16T20:06:16.726086+0000","last_became_peered":"2023-10-16T20:06:16.726086+0000","last_unstale":"2023-10-16T20:06:17.735204+0000","last_undegraded":"2023-10-16T20:06:17.735204+0000","last_fullsized":"2023-10-16T20:06:17.735204+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:21:23.740779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.216516+0000","last_change":"2023-10-16T20:06:17.207345+0000","last_active":"2023-10-16T20:06:17.216516+0000","last_peered":"2023-10-16T20:06:17.216516+0000","last_clean":"2023-10-16T20:06:17.216516+0000","last_became_active":"2023-10-16T20:06:17.207082+0000","last_became_peered":"2023-10-16T20:06:17.207082+0000","last_unstale":"2023-10-16T20:06:17.216516+0000","last_undegraded":"2023-10-16T20:06:17.216516+0000","last_fullsized":"2023-10-16T20:06:17.216516+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:04:53.163558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"40'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.733912+0000","last_change":"2023-10-16T20:06:16.727808+0000","last_active":"2023-10-16T20:06:17.733912+0000","last_peered":"2023-10-16T20:06:17.733912+0000","last_clean":"2023-10-16T20:06:17.733912+0000","last_became_active":"2023-10-16T20:06:16.727400+0000","last_became_peered":"2023-10-16T20:06:16.727400+0000","last_unstale":"2023-10-16T20:06:17.733912+0000","last_undegraded":"2023-10-16T20:06:17.733912+0000","last_fullsized":"2023-10-16T20:06:17.733912+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:00.635492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.222291+0000","last_change":"2023-10-16T20:06:17.216271+0000","last_active":"2023-10-16T20:06:17.222291+0000","last_peered":"2023-10-16T20:06:17.222291+0000","last_clean":"2023-10-16T20:06:17.222291+0000","last_became_active":"2023-10-16T20:06:17.215201+0000","last_became_peered":"2023-10-16T20:06:17.215201+0000","last_unstale":"2023-10-16T20:06:17.222291+0000","last_undegraded":"2023-10-16T20:06:17.222291+0000","last_fullsized":"2023-10-16T20:06:17.222291+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:59:17.101302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"40'1","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357862+0000","last_change":"2023-10-16T20:06:16.724780+0000","last_active":"2023-10-16T20:06:18.357862+0000","last_peered":"2023-10-16T20:06:18.357862+0000","last_clean":"2023-10-16T20:06:18.357862+0000","last_became_active":"2023-10-16T20:06:16.724476+0000","last_became_peered":"2023-10-16T20:06:16.724476+0000","last_unstale":"2023-10-16T20:06:18.357862+0000","last_undegraded":"2023-10-16T20:06:18.357862+0000","last_fullsized":"2023-10-16T20:06:18.357862+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:19:37.004795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.358046+0000","last_change":"2023-10-16T20:06:17.209520+0000","last_active":"2023-10-16T20:06:18.358046+0000","last_peered":"2023-10-16T20:06:18.358046+0000","last_clean":"2023-10-16T20:06:18.358046+0000","last_became_active":"2023-10-16T20:06:17.209236+0000","last_became_peered":"2023-10-16T20:06:17.209236+0000","last_unstale":"2023-10-16T20:06:18.358046+0000","last_undegraded":"2023-10-16T20:06:18.358046+0000","last_fullsized":"2023-10-16T20:06:18.358046+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:31:26.166083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.196031+0000","last_change":"2023-10-16T20:06:16.734625+0000","last_active":"2023-10-16T20:06:17.196031+0000","last_peered":"2023-10-16T20:06:17.196031+0000","last_clean":"2023-10-16T20:06:17.196031+0000","last_became_active":"2023-10-16T20:06:16.734353+0000","last_became_peered":"2023-10-16T20:06:16.734353+0000","last_unstale":"2023-10-16T20:06:17.196031+0000","last_undegraded":"2023-10-16T20:06:17.196031+0000","last_fullsized":"2023-10-16T20:06:17.196031+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:38:39.778643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196838+0000","last_change":"2023-10-16T20:06:17.219357+0000","last_active":"2023-10-16T20:06:18.196838+0000","last_peered":"2023-10-16T20:06:18.196838+0000","last_clean":"2023-10-16T20:06:18.196838+0000","last_became_active":"2023-10-16T20:06:17.218995+0000","last_became_peered":"2023-10-16T20:06:17.218995+0000","last_unstale":"2023-10-16T20:06:18.196838+0000","last_undegraded":"2023-10-16T20:06:18.196838+0000","last_fullsized":"2023-10-16T20:06:18.196838+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:44:43.545772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"40'2","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357665+0000","last_change":"2023-10-16T20:06:16.732782+0000","last_active":"2023-10-16T20:06:18.357665+0000","last_peered":"2023-10-16T20:06:18.357665+0000","last_clean":"2023-10-16T20:06:18.357665+0000","last_became_active":"2023-10-16T20:06:16.732498+0000","last_became_peered":"2023-10-16T20:06:16.732498+0000","last_unstale":"2023-10-16T20:06:18.357665+0000","last_undegraded":"2023-10-16T20:06:18.357665+0000","last_fullsized":"2023-10-16T20:06:18.357665+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:21:27.852426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T20:06:16.698995+0000","last_change":"2023-10-16T20:06:16.698995+0000","last_active":"2023-10-16T20:06:16.698995+0000","last_peered":"2023-10-16T20:06:16.698995+0000","last_clean":"2023-10-16T20:06:16.698995+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T20:06:16.698995+0000","last_undegraded":"2023-10-16T20:06:16.698995+0000","last_fullsized":"2023-10-16T20:06:16.698995+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357669+0000","last_change":"2023-10-16T20:06:16.732525+0000","last_active":"2023-10-16T20:06:18.357669+0000","last_peered":"2023-10-16T20:06:18.357669+0000","last_clean":"2023-10-16T20:06:18.357669+0000","last_became_active":"2023-10-16T20:06:16.732199+0000","last_became_peered":"2023-10-16T20:06:16.732199+0000","last_unstale":"2023-10-16T20:06:18.357669+0000","last_undegraded":"2023-10-16T20:06:18.357669+0000","last_fullsized":"2023-10-16T20:06:18.357669+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:12:42.667116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.216176+0000","last_change":"2023-10-16T20:06:17.209547+0000","last_active":"2023-10-16T20:06:17.216176+0000","last_peered":"2023-10-16T20:06:17.216176+0000","last_clean":"2023-10-16T20:06:17.216176+0000","last_became_active":"2023-10-16T20:06:17.209281+0000","last_became_peered":"2023-10-16T20:06:17.209281+0000","last_unstale":"2023-10-16T20:06:17.216176+0000","last_undegraded":"2023-10-16T20:06:17.216176+0000","last_fullsized":"2023-10-16T20:06:17.216176+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:58:21.190657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197648+0000","last_change":"2023-10-16T20:06:16.727775+0000","last_active":"2023-10-16T20:06:18.197648+0000","last_peered":"2023-10-16T20:06:18.197648+0000","last_clean":"2023-10-16T20:06:18.197648+0000","last_became_active":"2023-10-16T20:06:16.727492+0000","last_became_peered":"2023-10-16T20:06:16.727492+0000","last_unstale":"2023-10-16T20:06:18.197648+0000","last_undegraded":"2023-10-16T20:06:18.197648+0000","last_fullsized":"2023-10-16T20:06:18.197648+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:32:58.366700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.220851+0000","last_change":"2023-10-16T20:06:17.218801+0000","last_active":"2023-10-16T20:06:17.220851+0000","last_peered":"2023-10-16T20:06:17.220851+0000","last_clean":"2023-10-16T20:06:17.220851+0000","last_became_active":"2023-10-16T20:06:17.218567+0000","last_became_peered":"2023-10-16T20:06:17.218567+0000","last_unstale":"2023-10-16T20:06:17.220851+0000","last_undegraded":"2023-10-16T20:06:17.220851+0000","last_fullsized":"2023-10-16T20:06:17.220851+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:13.874932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357557+0000","last_change":"2023-10-16T20:06:16.727754+0000","last_active":"2023-10-16T20:06:18.357557+0000","last_peered":"2023-10-16T20:06:18.357557+0000","last_clean":"2023-10-16T20:06:18.357557+0000","last_became_active":"2023-10-16T20:06:16.727497+0000","last_became_peered":"2023-10-16T20:06:16.727497+0000","last_unstale":"2023-10-16T20:06:18.357557+0000","last_undegraded":"2023-10-16T20:06:18.357557+0000","last_fullsized":"2023-10-16T20:06:18.357557+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:08:12.647505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.209898+0000","last_change":"2023-10-16T20:06:17.201294+0000","last_active":"2023-10-16T20:06:17.209898+0000","last_peered":"2023-10-16T20:06:17.209898+0000","last_clean":"2023-10-16T20:06:17.209898+0000","last_became_active":"2023-10-16T20:06:17.200853+0000","last_became_peered":"2023-10-16T20:06:17.200853+0000","last_unstale":"2023-10-16T20:06:17.209898+0000","last_undegraded":"2023-10-16T20:06:17.209898+0000","last_fullsized":"2023-10-16T20:06:17.209898+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:01.646291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"40'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.732982+0000","last_change":"2023-10-16T20:06:16.733877+0000","last_active":"2023-10-16T20:06:17.732982+0000","last_peered":"2023-10-16T20:06:17.732982+0000","last_clean":"2023-10-16T20:06:17.732982+0000","last_became_active":"2023-10-16T20:06:16.733570+0000","last_became_peered":"2023-10-16T20:06:16.733570+0000","last_unstale":"2023-10-16T20:06:17.732982+0000","last_undegraded":"2023-10-16T20:06:17.732982+0000","last_fullsized":"2023-10-16T20:06:17.732982+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:38:29.780087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197199+0000","last_change":"2023-10-16T20:06:17.220488+0000","last_active":"2023-10-16T20:06:18.197199+0000","last_peered":"2023-10-16T20:06:18.197199+0000","last_clean":"2023-10-16T20:06:18.197199+0000","last_became_active":"2023-10-16T20:06:17.220060+0000","last_became_peered":"2023-10-16T20:06:17.220060+0000","last_unstale":"2023-10-16T20:06:18.197199+0000","last_undegraded":"2023-10-16T20:06:18.197199+0000","last_fullsized":"2023-10-16T20:06:18.197199+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:59:16.509971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357857+0000","last_change":"2023-10-16T20:06:16.726490+0000","last_active":"2023-10-16T20:06:18.357857+0000","last_peered":"2023-10-16T20:06:18.357857+0000","last_clean":"2023-10-16T20:06:18.357857+0000","last_became_active":"2023-10-16T20:06:16.726210+0000","last_became_peered":"2023-10-16T20:06:16.726210+0000","last_unstale":"2023-10-16T20:06:18.357857+0000","last_undegraded":"2023-10-16T20:06:18.357857+0000","last_fullsized":"2023-10-16T20:06:18.357857+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:13:14.492857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.217565+0000","last_change":"2023-10-16T20:06:17.213630+0000","last_active":"2023-10-16T20:06:17.217565+0000","last_peered":"2023-10-16T20:06:17.217565+0000","last_clean":"2023-10-16T20:06:17.217565+0000","last_became_active":"2023-10-16T20:06:17.213352+0000","last_became_peered":"2023-10-16T20:06:17.213352+0000","last_unstale":"2023-10-16T20:06:17.217565+0000","last_undegraded":"2023-10-16T20:06:17.217565+0000","last_fullsized":"2023-10-16T20:06:17.217565+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:18:14.480179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T20:06:15.688195+0000","last_change":"2023-10-16T20:06:15.688195+0000","last_active":"2023-10-16T20:06:15.688195+0000","last_peered":"2023-10-16T20:06:15.688195+0000","last_clean":"2023-10-16T20:06:15.688195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T20:06:15.688195+0000","last_undegraded":"2023-10-16T20:06:15.688195+0000","last_fullsized":"2023-10-16T20:06:15.688195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T20:06:16.698995+0000","last_change":"2023-10-16T20:06:16.698995+0000","last_active":"2023-10-16T20:06:16.698995+0000","last_peered":"2023-10-16T20:06:16.698995+0000","last_clean":"2023-10-16T20:06:16.698995+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T20:06:16.698995+0000","last_undegraded":"2023-10-16T20:06:16.698995+0000","last_fullsized":"2023-10-16T20:06:16.698995+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T20:06:15.688195+0000","last_change":"2023-10-16T20:06:15.688195+0000","last_active":"2023-10-16T20:06:15.688195+0000","last_peered":"2023-10-16T20:06:15.688195+0000","last_clean":"2023-10-16T20:06:15.688195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T20:06:15.688195+0000","last_undegraded":"2023-10-16T20:06:15.688195+0000","last_fullsized":"2023-10-16T20:06:15.688195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.216410+0000","last_change":"2023-10-16T20:06:17.208914+0000","last_active":"2023-10-16T20:06:17.216410+0000","last_peered":"2023-10-16T20:06:17.216410+0000","last_clean":"2023-10-16T20:06:17.216410+0000","last_became_active":"2023-10-16T20:06:17.208609+0000","last_became_peered":"2023-10-16T20:06:17.208609+0000","last_unstale":"2023-10-16T20:06:17.216410+0000","last_undegraded":"2023-10-16T20:06:17.216410+0000","last_fullsized":"2023-10-16T20:06:17.216410+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:32:44.093091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197750+0000","last_change":"2023-10-16T20:06:16.733015+0000","last_active":"2023-10-16T20:06:18.197750+0000","last_peered":"2023-10-16T20:06:18.197750+0000","last_clean":"2023-10-16T20:06:18.197750+0000","last_became_active":"2023-10-16T20:06:16.731413+0000","last_became_peered":"2023-10-16T20:06:16.731413+0000","last_unstale":"2023-10-16T20:06:18.197750+0000","last_undegraded":"2023-10-16T20:06:18.197750+0000","last_fullsized":"2023-10-16T20:06:18.197750+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:59:16.090503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.222240+0000","last_change":"2023-10-16T20:06:17.214304+0000","last_active":"2023-10-16T20:06:17.222240+0000","last_peered":"2023-10-16T20:06:17.222240+0000","last_clean":"2023-10-16T20:06:17.222240+0000","last_became_active":"2023-10-16T20:06:17.213521+0000","last_became_peered":"2023-10-16T20:06:17.213521+0000","last_unstale":"2023-10-16T20:06:17.222240+0000","last_undegraded":"2023-10-16T20:06:17.222240+0000","last_fullsized":"2023-10-16T20:06:17.222240+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:27:10.715147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.202689+0000","last_change":"2023-10-16T20:06:16.730112+0000","last_active":"2023-10-16T20:06:17.202689+0000","last_peered":"2023-10-16T20:06:17.202689+0000","last_clean":"2023-10-16T20:06:17.202689+0000","last_became_active":"2023-10-16T20:06:16.729781+0000","last_became_peered":"2023-10-16T20:06:16.729781+0000","last_unstale":"2023-10-16T20:06:17.202689+0000","last_undegraded":"2023-10-16T20:06:17.202689+0000","last_fullsized":"2023-10-16T20:06:17.202689+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:39:05.076198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197008+0000","last_change":"2023-10-16T20:06:17.221997+0000","last_active":"2023-10-16T20:06:18.197008+0000","last_peered":"2023-10-16T20:06:18.197008+0000","last_clean":"2023-10-16T20:06:18.197008+0000","last_became_active":"2023-10-16T20:06:17.221704+0000","last_became_peered":"2023-10-16T20:06:17.221704+0000","last_unstale":"2023-10-16T20:06:18.197008+0000","last_undegraded":"2023-10-16T20:06:18.197008+0000","last_fullsized":"2023-10-16T20:06:18.197008+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:38:22.493819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.201889+0000","last_change":"2023-10-16T20:06:16.733171+0000","last_active":"2023-10-16T20:06:17.201889+0000","last_peered":"2023-10-16T20:06:17.201889+0000","last_clean":"2023-10-16T20:06:17.201889+0000","last_became_active":"2023-10-16T20:06:16.732869+0000","last_became_peered":"2023-10-16T20:06:16.732869+0000","last_unstale":"2023-10-16T20:06:17.201889+0000","last_undegraded":"2023-10-16T20:06:17.201889+0000","last_fullsized":"2023-10-16T20:06:17.201889+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:04:52.152758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.221418+0000","last_change":"2023-10-16T20:06:17.215196+0000","last_active":"2023-10-16T20:06:17.221418+0000","last_peered":"2023-10-16T20:06:17.221418+0000","last_clean":"2023-10-16T20:06:17.221418+0000","last_became_active":"2023-10-16T20:06:17.214556+0000","last_became_peered":"2023-10-16T20:06:17.214556+0000","last_unstale":"2023-10-16T20:06:17.221418+0000","last_undegraded":"2023-10-16T20:06:17.221418+0000","last_fullsized":"2023-10-16T20:06:17.221418+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:24:29.941143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196810+0000","last_change":"2023-10-16T20:06:16.728925+0000","last_active":"2023-10-16T20:06:18.196810+0000","last_peered":"2023-10-16T20:06:18.196810+0000","last_clean":"2023-10-16T20:06:18.196810+0000","last_became_active":"2023-10-16T20:06:16.728663+0000","last_became_peered":"2023-10-16T20:06:16.728663+0000","last_unstale":"2023-10-16T20:06:18.196810+0000","last_undegraded":"2023-10-16T20:06:18.196810+0000","last_fullsized":"2023-10-16T20:06:18.196810+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:50:31.301972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":182,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.208038+0000","last_change":"2023-10-16T20:04:46.335822+0000","last_active":"2023-10-16T20:06:17.208038+0000","last_peered":"2023-10-16T20:06:17.208038+0000","last_clean":"2023-10-16T20:06:17.208038+0000","last_became_active":"2023-10-16T20:04:46.334617+0000","last_became_peered":"2023-10-16T20:04:46.334617+0000","last_unstale":"2023-10-16T20:06:17.208038+0000","last_undegraded":"2023-10-16T20:06:17.208038+0000","last_fullsized":"2023-10-16T20:06:17.208038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:45.306240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:45.306240+0000","last_clean_scrub_stamp":"2023-10-16T20:04:45.306240+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:42:11.153721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":112,"acting":112,"num_store_stats":7},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":58,"acting":58,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509462,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1048576,"data_stored":783264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.86199999999999999}]},{"osd":1,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.4900000000000002,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":0.752}]},{"osd":2,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.0550000000000002,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.80300000000000005}]},{"osd":3,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.629,"5min":2.629,"15min":2.629},"last":0.67400000000000004}]},{"osd":4,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":0.63200000000000001}]},{"osd":5,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.8809999999999998,"5min":2.8809999999999998,"15min":2.8809999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.65400000000000003}]},{"osd":6,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.56999999999999995}]}]},{"osd":6,"up_from":15,"seq":64424509463,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":192896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":4.5880000000000001,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":4.6139999999999999,"5min":4.6139999999999999,"15min":4.6139999999999999},"last":0.629}]},{"osd":1,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":4.6639999999999997,"5min":4.6639999999999997,"15min":4.6639999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":4.8140000000000001,"5min":4.8140000000000001,"15min":4.8140000000000001},"last":0.60899999999999999}]},{"osd":2,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":5.3769999999999998,"5min":5.3769999999999998,"15min":5.3769999999999998},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.79000000000000004}]},{"osd":3,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":4.5460000000000003,"5min":4.5460000000000003,"15min":4.5460000000000003},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.66100000000000003}]},{"osd":4,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.51900000000000002}]},{"osd":5,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.76400000000000001}]},{"osd":7,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.88500000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509461,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":462848,"data_stored":193008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.70999999999999996}]},{"osd":1,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.623},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.76200000000000001}]},{"osd":2,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.747},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.67800000000000005}]},{"osd":3,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.73199999999999998}]},{"osd":4,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.50700000000000001}]},{"osd":6,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.56000000000000005}]},{"osd":7,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.53800000000000003}]}]},{"osd":4,"up_from":15,"seq":64424509461,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31632,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340208,"statfs":{"total":96636764160,"available":96604372992,"internally_reserved":0,"allocated":454656,"data_stored":192896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":3.835,"5min":3.835,"15min":3.835},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":3.7610000000000001,"5min":3.7610000000000001,"15min":3.7610000000000001},"last":0.75700000000000001}]},{"osd":1,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.57599999999999996}]},{"osd":2,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.628},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.60799999999999998}]},{"osd":3,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.72399999999999998}]},{"osd":5,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.5}]},{"osd":6,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.55100000000000005}]},{"osd":7,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.8919999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.74099999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509462,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":194542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.47599999999999998}]},{"osd":1,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.497}]},{"osd":2,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.54200000000000004}]},{"osd":4,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.88200000000000001}]},{"osd":5,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.876}]},{"osd":6,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.95499999999999996}]},{"osd":7,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.75800000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509461,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":192896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.53400000000000003}]},{"osd":1,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.436}]},{"osd":3,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.48399999999999999}]},{"osd":4,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.59699999999999998}]},{"osd":5,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.63}]},{"osd":6,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.80100000000000005}]},{"osd":7,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.70499999999999996}]}]},{"osd":1,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":192896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.867,"5min":3.867,"15min":3.867},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.52700000000000002}]},{"osd":2,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":5.6040000000000001,"5min":5.6040000000000001,"15min":5.6040000000000001},"last":0.434},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":7.8380000000000001,"5min":7.8380000000000001,"15min":7.8380000000000001},"last":0.45100000000000001}]},{"osd":3,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":5.2039999999999997,"5min":5.2039999999999997,"15min":5.2039999999999997},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":3.9470000000000001,"5min":3.9470000000000001,"15min":3.9470000000000001},"last":0.47399999999999998}]},{"osd":4,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":4.1500000000000004,"5min":4.1500000000000004,"15min":4.1500000000000004},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":4.2619999999999996,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":0.79400000000000004}]},{"osd":5,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":4.0910000000000002,"5min":4.0910000000000002,"15min":4.0910000000000002},"last":0.86099999999999999}]},{"osd":6,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":4.21,"5min":4.21,"15min":4.21},"last":0.88},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.83399999999999996}]},{"osd":7,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":4.1849999999999996,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":0.82099999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1011712,"data_stored":763382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":3.9289999999999998,"5min":3.9289999999999998,"15min":3.9289999999999998},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":4.0910000000000002,"5min":4.0910000000000002,"15min":4.0910000000000002},"last":0.32900000000000001}]},{"osd":2,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":4.1520000000000001,"5min":4.1520000000000001,"15min":4.1520000000000001},"last":0.434},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":4.1879999999999997,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.45100000000000001}]},{"osd":3,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":4.3440000000000003,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.49099999999999999}]},{"osd":4,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":4.5250000000000004,"5min":4.5250000000000004,"15min":4.5250000000000004},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.69799999999999995}]},{"osd":5,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":3.9990000000000001,"5min":3.9990000000000001,"15min":3.9990000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":4.4779999999999998,"5min":4.4779999999999998,"15min":4.4779999999999998},"last":0.81999999999999995}]},{"osd":6,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":3.956,"5min":3.956,"15min":3.956},"last":0.71199999999999997}]},{"osd":7,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":4.0289999999999999,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":4.5640000000000001,"5min":4.5640000000000001,"15min":4.5640000000000001},"last":0.72199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T20:06:21.798 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T20:06:21.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.963+0000 7f44f1d3e640 1 Processor -- start 2023-10-16T20:06:21.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.963+0000 7f44f1d3e640 1 -- start start 2023-10-16T20:06:21.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.967+0000 7f44f1d3e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f44ec153ea0 0x7f44ec0d1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:21.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.967+0000 7f44f1d3e640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f44ec0cd970 0x7f44ec0cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:21.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.967+0000 7f44f1d3e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44ec0b5510 con 0x7f44ec0d2660 2023-10-16T20:06:21.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.967+0000 7f44f1d3e640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44ec15d9a0 con 0x7f44ec0cd970 2023-10-16T20:06:21.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.967+0000 7f44f1d3e640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44ec0b4990 con 0x7f44ec153ea0 2023-10-16T20:06:21.974 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.967+0000 7f44ebfff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f44ec0cd970 0x7f44ec0cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:21.974 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.967+0000 7f44ebfff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f44ec0cd970 0x7f44ec0cdd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45968/0 (socket says 172.21.15.88:45968) 2023-10-16T20:06:21.974 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.967+0000 7f44ebfff640 1 -- 172.21.15.88:0/3843591728 learned_addr learned my addr 172.21.15.88:0/3843591728 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:21.974 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.967+0000 7f44eb7fe640 1 --2- 172.21.15.88:0/3843591728 >> v2:172.21.15.94:3300/0 conn(0x7f44ec153ea0 0x7f44ec0d1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:21.974 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.967+0000 7f44ebfff640 1 -- 172.21.15.88:0/3843591728 >> v2:172.21.15.94:3300/0 conn(0x7f44ec153ea0 msgr2=0x7f44ec0d1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:21.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.967+0000 7f44ebfff640 1 --2- 172.21.15.88:0/3843591728 >> v2:172.21.15.94:3300/0 conn(0x7f44ec153ea0 0x7f44ec0d1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:21.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.967+0000 7f44ebfff640 1 -- 172.21.15.88:0/3843591728 >> v1:172.21.15.94:6789/0 conn(0x7f44ec0d2660 legacy=0x7f44ec15d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:21.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.967+0000 7f44ebfff640 1 -- 172.21.15.88:0/3843591728 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ec0b4c70 con 0x7f44ec0cd970 2023-10-16T20:06:21.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.967+0000 7f44eb7fe640 1 --2- 172.21.15.88:0/3843591728 >> v2:172.21.15.94:3300/0 conn(0x7f44ec153ea0 0x7f44ec0d1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:06:21.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.971+0000 7f44ebfff640 1 --2- 172.21.15.88:0/3843591728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44ec0cd970 0x7f44ec0cdd80 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f44ec06c610 tx=0x7f44e401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61712ccc94a544cf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:21.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.971+0000 7f44eaffd640 1 -- 172.21.15.88:0/3843591728 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e40a5820 con 0x7f44ec0cd970 2023-10-16T20:06:21.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.971+0000 7f44eaffd640 1 -- 172.21.15.88:0/3843591728 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e40a5980 con 0x7f44ec0cd970 2023-10-16T20:06:21.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.971+0000 7f44eaffd640 1 -- 172.21.15.88:0/3843591728 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e40a5d10 con 0x7f44ec0cd970 2023-10-16T20:06:21.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.971+0000 7f44f1d3e640 1 -- 172.21.15.88:0/3843591728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44ec0cd970 msgr2=0x7f44ec0cdd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:21.978 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.971+0000 7f44f1d3e640 1 --2- 172.21.15.88:0/3843591728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44ec0cd970 0x7f44ec0cdd80 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f44ec06c610 tx=0x7f44e401c960 comp rx=0 tx=0).stop 2023-10-16T20:06:21.978 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.971+0000 7f44f1d3e640 1 -- 172.21.15.88:0/3843591728 shutdown_connections 2023-10-16T20:06:21.978 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.971+0000 7f44f1d3e640 1 --2- 172.21.15.88:0/3843591728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44ec0cd970 0x7f44ec0cdd80 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:21.978 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.971+0000 7f44f1d3e640 1 --2- 172.21.15.88:0/3843591728 >> v2:172.21.15.94:3300/0 conn(0x7f44ec153ea0 0x7f44ec0d1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:21.978 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.971+0000 7f44f1d3e640 1 -- 172.21.15.88:0/3843591728 >> 172.21.15.88:0/3843591728 conn(0x7f44ec0715c0 msgr2=0x7f44ec0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:21.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.971+0000 7f44f1d3e640 1 -- 172.21.15.88:0/3843591728 shutdown_connections 2023-10-16T20:06:21.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.971+0000 7f44f1d3e640 1 -- 172.21.15.88:0/3843591728 wait complete. 2023-10-16T20:06:21.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.971+0000 7f44f1d3e640 1 Processor -- start 2023-10-16T20:06:21.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.971+0000 7f44f1d3e640 1 -- start start 2023-10-16T20:06:21.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.975+0000 7f44f1d3e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44ec0cd970 0x7f44ec0c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:21.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.975+0000 7f44f1d3e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44ec0d2660 0x7f44ec14c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:21.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.975+0000 7f44f1d3e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44ec153ea0 0x7f44ec14c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:21.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.975+0000 7f44f1d3e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec156b90 con 0x7f44ec0d2660 2023-10-16T20:06:21.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.975+0000 7f44f1d3e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec155f80 con 0x7f44ec153ea0 2023-10-16T20:06:21.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.975+0000 7f44f1d3e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec156260 con 0x7f44ec0cd970 2023-10-16T20:06:21.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.975+0000 7f44e3fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44ec0d2660 0x7f44ec14c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:21.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.975+0000 7f44e3fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44ec0d2660 0x7f44ec14c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45980/0 (socket says 172.21.15.88:45980) 2023-10-16T20:06:21.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.975+0000 7f44e3fff640 1 -- 172.21.15.88:0/3672081017 learned_addr learned my addr 172.21.15.88:0/3672081017 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:21.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.975+0000 7f44eb7fe640 1 --2- 172.21.15.88:0/3672081017 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44ec0cd970 0x7f44ec0c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:21.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.975+0000 7f44ebfff640 1 --2- 172.21.15.88:0/3672081017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44ec153ea0 0x7f44ec14c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:21.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.975+0000 7f44e3fff640 1 -- 172.21.15.88:0/3672081017 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44ec0cd970 msgr2=0x7f44ec0c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:21.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.975+0000 7f44e3fff640 1 --2- 172.21.15.88:0/3672081017 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44ec0cd970 0x7f44ec0c92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:21.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.975+0000 7f44e3fff640 1 -- 172.21.15.88:0/3672081017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44ec153ea0 msgr2=0x7f44ec14c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:21.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.975+0000 7f44e3fff640 1 --2- 172.21.15.88:0/3672081017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44ec153ea0 0x7f44ec14c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:21.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.975+0000 7f44e3fff640 1 -- 172.21.15.88:0/3672081017 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e4009b30 con 0x7f44ec0d2660 2023-10-16T20:06:21.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.975+0000 7f44ebfff640 1 --2- 172.21.15.88:0/3672081017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44ec153ea0 0x7f44ec14c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:06:21.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.975+0000 7f44e3fff640 1 --2- 172.21.15.88:0/3672081017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44ec0d2660 0x7f44ec14c2f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f44d400c450 tx=0x7f44d400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:21.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.975+0000 7f44e8ff9640 1 -- 172.21.15.88:0/3672081017 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d4012830 con 0x7f44ec0d2660 2023-10-16T20:06:21.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.975+0000 7f44e8ff9640 1 -- 172.21.15.88:0/3672081017 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d40129b0 con 0x7f44ec0d2660 2023-10-16T20:06:21.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.975+0000 7f44e8ff9640 1 -- 172.21.15.88:0/3672081017 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d4012d60 con 0x7f44ec0d2660 2023-10-16T20:06:21.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.975+0000 7f44f1d3e640 1 -- 172.21.15.88:0/3672081017 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44ec156540 con 0x7f44ec0d2660 2023-10-16T20:06:21.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.975+0000 7f44f1d3e640 1 -- 172.21.15.88:0/3672081017 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44ec0c9800 con 0x7f44ec0d2660 2023-10-16T20:06:21.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.975+0000 7f44f1d3e640 1 -- 172.21.15.88:0/3672081017 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44ec04e9c0 con 0x7f44ec0d2660 2023-10-16T20:06:21.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.975+0000 7f44e8ff9640 1 -- 172.21.15.88:0/3672081017 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f44d401c460 con 0x7f44ec0d2660 2023-10-16T20:06:21.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.979+0000 7f44e8ff9640 1 --2- 172.21.15.88:0/3672081017 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f44cc0664a0 0x7f44cc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:21.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.979+0000 7f44e8ff9640 1 -- 172.21.15.88:0/3672081017 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f44d4083ee0 con 0x7f44ec0d2660 2023-10-16T20:06:21.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.979+0000 7f44eb7fe640 1 --2- 172.21.15.88:0/3672081017 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f44cc0664a0 0x7f44cc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:21.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.979+0000 7f44eb7fe640 1 --2- 172.21.15.88:0/3672081017 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f44cc0664a0 0x7f44cc068960 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f44dc006fd0 tx=0x7f44dc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:21.988 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:21.983+0000 7f44e8ff9640 1 -- 172.21.15.88:0/3672081017 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f44d4051160 con 0x7f44ec0d2660 2023-10-16T20:06:22.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.127+0000 7f44f1d3e640 1 -- 172.21.15.88:0/3672081017 --> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f44ec0b4620 con 0x7f44cc0664a0 2023-10-16T20:06:22.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.135+0000 7f44e8ff9640 1 -- 172.21.15.88:0/3672081017 <== mgr.4110 v2:172.21.15.88:6832/4078483333 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287790 (secure 0 0 0) 0x7f44dc00c050 con 0x7f44cc0664a0 2023-10-16T20:06:22.143 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:06:22.144 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2023-10-16T20:06:22.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.139+0000 7f44f1d3e640 1 -- 172.21.15.88:0/3672081017 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f44cc0664a0 msgr2=0x7f44cc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:22.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.139+0000 7f44f1d3e640 1 --2- 172.21.15.88:0/3672081017 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f44cc0664a0 0x7f44cc068960 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f44dc006fd0 tx=0x7f44dc008040 comp rx=0 tx=0).stop 2023-10-16T20:06:22.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.139+0000 7f44f1d3e640 1 -- 172.21.15.88:0/3672081017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44ec0d2660 msgr2=0x7f44ec14c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:22.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.139+0000 7f44f1d3e640 1 --2- 172.21.15.88:0/3672081017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44ec0d2660 0x7f44ec14c2f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f44d400c450 tx=0x7f44d400c910 comp rx=0 tx=0).stop 2023-10-16T20:06:22.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.139+0000 7f44f1d3e640 1 -- 172.21.15.88:0/3672081017 shutdown_connections 2023-10-16T20:06:22.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.139+0000 7f44f1d3e640 1 --2- 172.21.15.88:0/3672081017 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f44cc0664a0 0x7f44cc068960 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:22.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.139+0000 7f44f1d3e640 1 --2- 172.21.15.88:0/3672081017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44ec153ea0 0x7f44ec14c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:22.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.139+0000 7f44f1d3e640 1 --2- 172.21.15.88:0/3672081017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44ec0d2660 0x7f44ec14c2f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:22.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.139+0000 7f44f1d3e640 1 --2- 172.21.15.88:0/3672081017 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44ec0cd970 0x7f44ec0c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:22.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.139+0000 7f44f1d3e640 1 -- 172.21.15.88:0/3672081017 >> 172.21.15.88:0/3672081017 conn(0x7f44ec0715c0 msgr2=0x7f44ec15ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:22.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.139+0000 7f44f1d3e640 1 -- 172.21.15.88:0/3672081017 shutdown_connections 2023-10-16T20:06:22.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.139+0000 7f44f1d3e640 1 -- 172.21.15.88:0/3672081017 wait complete. 2023-10-16T20:06:22.163 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2023-10-16T20:06:20.462297+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":172,"acting":172,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":41,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233392,"kb_used_data":4700,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754741328,"statfs":{"total":773094113280,"available":772855119872,"internally_reserved":0,"allocated":4812800,"data_stored":2705780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":234016313},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.760440"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.220769+0000","last_change":"2023-10-16T20:06:17.217901+0000","last_active":"2023-10-16T20:06:17.220769+0000","last_peered":"2023-10-16T20:06:17.220769+0000","last_clean":"2023-10-16T20:06:17.220769+0000","last_became_active":"2023-10-16T20:06:17.217583+0000","last_became_peered":"2023-10-16T20:06:17.217583+0000","last_unstale":"2023-10-16T20:06:17.220769+0000","last_undegraded":"2023-10-16T20:06:17.220769+0000","last_fullsized":"2023-10-16T20:06:17.220769+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:24:29.941143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.215707+0000","last_change":"2023-10-16T20:06:17.211493+0000","last_active":"2023-10-16T20:06:17.215707+0000","last_peered":"2023-10-16T20:06:17.215707+0000","last_clean":"2023-10-16T20:06:17.215707+0000","last_became_active":"2023-10-16T20:06:17.211196+0000","last_became_peered":"2023-10-16T20:06:17.211196+0000","last_unstale":"2023-10-16T20:06:17.215707+0000","last_undegraded":"2023-10-16T20:06:17.215707+0000","last_fullsized":"2023-10-16T20:06:17.215707+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:50:32.312772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T20:06:16.698995+0000","last_change":"2023-10-16T20:06:16.698995+0000","last_active":"2023-10-16T20:06:16.698995+0000","last_peered":"2023-10-16T20:06:16.698995+0000","last_clean":"2023-10-16T20:06:16.698995+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T20:06:16.698995+0000","last_undegraded":"2023-10-16T20:06:16.698995+0000","last_fullsized":"2023-10-16T20:06:16.698995+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357788+0000","last_change":"2023-10-16T20:06:17.216483+0000","last_active":"2023-10-16T20:06:18.357788+0000","last_peered":"2023-10-16T20:06:18.357788+0000","last_clean":"2023-10-16T20:06:18.357788+0000","last_became_active":"2023-10-16T20:06:17.216202+0000","last_became_peered":"2023-10-16T20:06:17.216202+0000","last_unstale":"2023-10-16T20:06:18.357788+0000","last_undegraded":"2023-10-16T20:06:18.357788+0000","last_fullsized":"2023-10-16T20:06:18.357788+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:32:02.453729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357619+0000","last_change":"2023-10-16T20:06:17.208325+0000","last_active":"2023-10-16T20:06:18.357619+0000","last_peered":"2023-10-16T20:06:18.357619+0000","last_clean":"2023-10-16T20:06:18.357619+0000","last_became_active":"2023-10-16T20:06:17.208061+0000","last_became_peered":"2023-10-16T20:06:17.208061+0000","last_unstale":"2023-10-16T20:06:18.357619+0000","last_undegraded":"2023-10-16T20:06:18.357619+0000","last_fullsized":"2023-10-16T20:06:18.357619+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:02:28.165753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.217274+0000","last_change":"2023-10-16T20:06:17.208416+0000","last_active":"2023-10-16T20:06:17.217274+0000","last_peered":"2023-10-16T20:06:17.217274+0000","last_clean":"2023-10-16T20:06:17.217274+0000","last_became_active":"2023-10-16T20:06:17.207713+0000","last_became_peered":"2023-10-16T20:06:17.207713+0000","last_unstale":"2023-10-16T20:06:17.217274+0000","last_undegraded":"2023-10-16T20:06:17.217274+0000","last_fullsized":"2023-10-16T20:06:17.217274+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:39:06.086998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.213838+0000","last_change":"2023-10-16T20:06:17.206736+0000","last_active":"2023-10-16T20:06:17.213838+0000","last_peered":"2023-10-16T20:06:17.213838+0000","last_clean":"2023-10-16T20:06:17.213838+0000","last_became_active":"2023-10-16T20:06:17.206435+0000","last_became_peered":"2023-10-16T20:06:17.206435+0000","last_unstale":"2023-10-16T20:06:17.213838+0000","last_undegraded":"2023-10-16T20:06:17.213838+0000","last_fullsized":"2023-10-16T20:06:17.213838+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:50:32.312772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.218213+0000","last_change":"2023-10-16T20:06:17.214147+0000","last_active":"2023-10-16T20:06:17.218213+0000","last_peered":"2023-10-16T20:06:17.218213+0000","last_clean":"2023-10-16T20:06:17.218213+0000","last_became_active":"2023-10-16T20:06:17.213998+0000","last_became_peered":"2023-10-16T20:06:17.213998+0000","last_unstale":"2023-10-16T20:06:17.218213+0000","last_undegraded":"2023-10-16T20:06:17.218213+0000","last_fullsized":"2023-10-16T20:06:17.218213+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:36:27.881926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T20:06:16.698995+0000","last_change":"2023-10-16T20:06:16.698995+0000","last_active":"2023-10-16T20:06:16.698995+0000","last_peered":"2023-10-16T20:06:16.698995+0000","last_clean":"2023-10-16T20:06:16.698995+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T20:06:16.698995+0000","last_undegraded":"2023-10-16T20:06:16.698995+0000","last_fullsized":"2023-10-16T20:06:16.698995+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.222492+0000","last_change":"2023-10-16T20:06:17.218989+0000","last_active":"2023-10-16T20:06:17.222492+0000","last_peered":"2023-10-16T20:06:17.222492+0000","last_clean":"2023-10-16T20:06:17.222492+0000","last_became_active":"2023-10-16T20:06:17.218726+0000","last_became_peered":"2023-10-16T20:06:17.218726+0000","last_unstale":"2023-10-16T20:06:17.222492+0000","last_undegraded":"2023-10-16T20:06:17.222492+0000","last_fullsized":"2023-10-16T20:06:17.222492+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:27:39.178553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.221629+0000","last_change":"2023-10-16T20:06:17.217372+0000","last_active":"2023-10-16T20:06:17.221629+0000","last_peered":"2023-10-16T20:06:17.221629+0000","last_clean":"2023-10-16T20:06:17.221629+0000","last_became_active":"2023-10-16T20:06:17.216730+0000","last_became_peered":"2023-10-16T20:06:17.216730+0000","last_unstale":"2023-10-16T20:06:17.221629+0000","last_undegraded":"2023-10-16T20:06:17.221629+0000","last_fullsized":"2023-10-16T20:06:17.221629+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:18:14.480179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T20:06:16.698995+0000","last_change":"2023-10-16T20:06:16.698995+0000","last_active":"2023-10-16T20:06:16.698995+0000","last_peered":"2023-10-16T20:06:16.698995+0000","last_clean":"2023-10-16T20:06:16.698995+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T20:06:16.698995+0000","last_undegraded":"2023-10-16T20:06:16.698995+0000","last_fullsized":"2023-10-16T20:06:16.698995+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.221752+0000","last_change":"2023-10-16T20:06:17.218455+0000","last_active":"2023-10-16T20:06:17.221752+0000","last_peered":"2023-10-16T20:06:17.221752+0000","last_clean":"2023-10-16T20:06:17.221752+0000","last_became_active":"2023-10-16T20:06:17.217565+0000","last_became_peered":"2023-10-16T20:06:17.217565+0000","last_unstale":"2023-10-16T20:06:17.221752+0000","last_undegraded":"2023-10-16T20:06:17.221752+0000","last_fullsized":"2023-10-16T20:06:17.221752+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:13.874932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.222220+0000","last_change":"2023-10-16T20:06:17.214285+0000","last_active":"2023-10-16T20:06:17.222220+0000","last_peered":"2023-10-16T20:06:17.222220+0000","last_clean":"2023-10-16T20:06:17.222220+0000","last_became_active":"2023-10-16T20:06:17.213551+0000","last_became_peered":"2023-10-16T20:06:17.213551+0000","last_unstale":"2023-10-16T20:06:17.222220+0000","last_undegraded":"2023-10-16T20:06:17.222220+0000","last_fullsized":"2023-10-16T20:06:17.222220+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:10:43.145100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.213855+0000","last_change":"2023-10-16T20:06:17.208025+0000","last_active":"2023-10-16T20:06:17.213855+0000","last_peered":"2023-10-16T20:06:17.213855+0000","last_clean":"2023-10-16T20:06:17.213855+0000","last_became_active":"2023-10-16T20:06:17.207739+0000","last_became_peered":"2023-10-16T20:06:17.207739+0000","last_unstale":"2023-10-16T20:06:17.213855+0000","last_undegraded":"2023-10-16T20:06:17.213855+0000","last_fullsized":"2023-10-16T20:06:17.213855+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:36:27.881926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196502+0000","last_change":"2023-10-16T20:06:17.222232+0000","last_active":"2023-10-16T20:06:18.196502+0000","last_peered":"2023-10-16T20:06:18.196502+0000","last_clean":"2023-10-16T20:06:18.196502+0000","last_became_active":"2023-10-16T20:06:17.221892+0000","last_became_peered":"2023-10-16T20:06:17.221892+0000","last_unstale":"2023-10-16T20:06:18.196502+0000","last_undegraded":"2023-10-16T20:06:18.196502+0000","last_fullsized":"2023-10-16T20:06:18.196502+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:58:45.132214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197149+0000","last_change":"2023-10-16T20:06:17.224446+0000","last_active":"2023-10-16T20:06:18.197149+0000","last_peered":"2023-10-16T20:06:18.197149+0000","last_clean":"2023-10-16T20:06:18.197149+0000","last_became_active":"2023-10-16T20:06:17.224113+0000","last_became_peered":"2023-10-16T20:06:17.224113+0000","last_unstale":"2023-10-16T20:06:18.197149+0000","last_undegraded":"2023-10-16T20:06:18.197149+0000","last_fullsized":"2023-10-16T20:06:18.197149+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:37:43.948080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198392+0000","last_change":"2023-10-16T20:06:17.214181+0000","last_active":"2023-10-16T20:06:18.198392+0000","last_peered":"2023-10-16T20:06:18.198392+0000","last_clean":"2023-10-16T20:06:18.198392+0000","last_became_active":"2023-10-16T20:06:17.213876+0000","last_became_peered":"2023-10-16T20:06:17.213876+0000","last_unstale":"2023-10-16T20:06:18.198392+0000","last_undegraded":"2023-10-16T20:06:18.198392+0000","last_fullsized":"2023-10-16T20:06:18.198392+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:39:06.086998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357861+0000","last_change":"2023-10-16T20:06:17.205826+0000","last_active":"2023-10-16T20:06:18.357861+0000","last_peered":"2023-10-16T20:06:18.357861+0000","last_clean":"2023-10-16T20:06:18.357861+0000","last_became_active":"2023-10-16T20:06:17.204888+0000","last_became_peered":"2023-10-16T20:06:17.204888+0000","last_unstale":"2023-10-16T20:06:18.357861+0000","last_undegraded":"2023-10-16T20:06:18.357861+0000","last_fullsized":"2023-10-16T20:06:18.357861+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:29:06.675309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197777+0000","last_change":"2023-10-16T20:06:17.217103+0000","last_active":"2023-10-16T20:06:18.197777+0000","last_peered":"2023-10-16T20:06:18.197777+0000","last_clean":"2023-10-16T20:06:18.197777+0000","last_became_active":"2023-10-16T20:06:17.216717+0000","last_became_peered":"2023-10-16T20:06:17.216717+0000","last_unstale":"2023-10-16T20:06:18.197777+0000","last_undegraded":"2023-10-16T20:06:18.197777+0000","last_fullsized":"2023-10-16T20:06:18.197777+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:20:06.934366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.206185+0000","last_change":"2023-10-16T20:06:17.201875+0000","last_active":"2023-10-16T20:06:17.206185+0000","last_peered":"2023-10-16T20:06:17.206185+0000","last_clean":"2023-10-16T20:06:17.206185+0000","last_became_active":"2023-10-16T20:06:17.200537+0000","last_became_peered":"2023-10-16T20:06:17.200537+0000","last_unstale":"2023-10-16T20:06:17.206185+0000","last_undegraded":"2023-10-16T20:06:17.206185+0000","last_fullsized":"2023-10-16T20:06:17.206185+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:23:28.625212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.216225+0000","last_change":"2023-10-16T20:06:17.211432+0000","last_active":"2023-10-16T20:06:17.216225+0000","last_peered":"2023-10-16T20:06:17.216225+0000","last_clean":"2023-10-16T20:06:17.216225+0000","last_became_active":"2023-10-16T20:06:17.211100+0000","last_became_peered":"2023-10-16T20:06:17.211100+0000","last_unstale":"2023-10-16T20:06:17.216225+0000","last_undegraded":"2023-10-16T20:06:17.216225+0000","last_fullsized":"2023-10-16T20:06:17.216225+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:06:45.516915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196828+0000","last_change":"2023-10-16T20:06:17.225476+0000","last_active":"2023-10-16T20:06:18.196828+0000","last_peered":"2023-10-16T20:06:18.196828+0000","last_clean":"2023-10-16T20:06:18.196828+0000","last_became_active":"2023-10-16T20:06:17.224667+0000","last_became_peered":"2023-10-16T20:06:17.224667+0000","last_unstale":"2023-10-16T20:06:18.196828+0000","last_undegraded":"2023-10-16T20:06:18.196828+0000","last_fullsized":"2023-10-16T20:06:18.196828+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:31:31.346356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357610+0000","last_change":"2023-10-16T20:06:17.216704+0000","last_active":"2023-10-16T20:06:18.357610+0000","last_peered":"2023-10-16T20:06:18.357610+0000","last_clean":"2023-10-16T20:06:18.357610+0000","last_became_active":"2023-10-16T20:06:17.216341+0000","last_became_peered":"2023-10-16T20:06:17.216341+0000","last_unstale":"2023-10-16T20:06:18.357610+0000","last_undegraded":"2023-10-16T20:06:18.357610+0000","last_fullsized":"2023-10-16T20:06:18.357610+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:14:46.135352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198413+0000","last_change":"2023-10-16T20:06:17.216103+0000","last_active":"2023-10-16T20:06:18.198413+0000","last_peered":"2023-10-16T20:06:18.198413+0000","last_clean":"2023-10-16T20:06:18.198413+0000","last_became_active":"2023-10-16T20:06:17.215844+0000","last_became_peered":"2023-10-16T20:06:17.215844+0000","last_unstale":"2023-10-16T20:06:18.198413+0000","last_undegraded":"2023-10-16T20:06:18.198413+0000","last_fullsized":"2023-10-16T20:06:18.198413+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:01.646291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.216083+0000","last_change":"2023-10-16T20:06:17.207691+0000","last_active":"2023-10-16T20:06:17.216083+0000","last_peered":"2023-10-16T20:06:17.216083+0000","last_clean":"2023-10-16T20:06:17.216083+0000","last_became_active":"2023-10-16T20:06:17.207437+0000","last_became_peered":"2023-10-16T20:06:17.207437+0000","last_unstale":"2023-10-16T20:06:17.216083+0000","last_undegraded":"2023-10-16T20:06:17.216083+0000","last_fullsized":"2023-10-16T20:06:17.216083+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:23:28.625212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198320+0000","last_change":"2023-10-16T20:06:17.211359+0000","last_active":"2023-10-16T20:06:18.198320+0000","last_peered":"2023-10-16T20:06:18.198320+0000","last_clean":"2023-10-16T20:06:18.198320+0000","last_became_active":"2023-10-16T20:06:17.210506+0000","last_became_peered":"2023-10-16T20:06:17.210506+0000","last_unstale":"2023-10-16T20:06:18.198320+0000","last_undegraded":"2023-10-16T20:06:18.198320+0000","last_fullsized":"2023-10-16T20:06:18.198320+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:32:44.093091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.222529+0000","last_change":"2023-10-16T20:06:17.219679+0000","last_active":"2023-10-16T20:06:17.222529+0000","last_peered":"2023-10-16T20:06:17.222529+0000","last_clean":"2023-10-16T20:06:17.222529+0000","last_became_active":"2023-10-16T20:06:17.214175+0000","last_became_peered":"2023-10-16T20:06:17.214175+0000","last_unstale":"2023-10-16T20:06:17.222529+0000","last_undegraded":"2023-10-16T20:06:17.222529+0000","last_fullsized":"2023-10-16T20:06:17.222529+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:22:32.981812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197630+0000","last_change":"2023-10-16T20:06:17.210945+0000","last_active":"2023-10-16T20:06:18.197630+0000","last_peered":"2023-10-16T20:06:18.197630+0000","last_clean":"2023-10-16T20:06:18.197630+0000","last_became_active":"2023-10-16T20:06:17.210629+0000","last_became_peered":"2023-10-16T20:06:17.210629+0000","last_unstale":"2023-10-16T20:06:18.197630+0000","last_undegraded":"2023-10-16T20:06:18.197630+0000","last_fullsized":"2023-10-16T20:06:18.197630+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:27:10.715147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197643+0000","last_change":"2023-10-16T20:06:17.212963+0000","last_active":"2023-10-16T20:06:18.197643+0000","last_peered":"2023-10-16T20:06:18.197643+0000","last_clean":"2023-10-16T20:06:18.197643+0000","last_became_active":"2023-10-16T20:06:17.212693+0000","last_became_peered":"2023-10-16T20:06:17.212693+0000","last_unstale":"2023-10-16T20:06:18.197643+0000","last_undegraded":"2023-10-16T20:06:18.197643+0000","last_fullsized":"2023-10-16T20:06:18.197643+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:10:43.145100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197558+0000","last_change":"2023-10-16T20:06:17.219075+0000","last_active":"2023-10-16T20:06:18.197558+0000","last_peered":"2023-10-16T20:06:18.197558+0000","last_clean":"2023-10-16T20:06:18.197558+0000","last_became_active":"2023-10-16T20:06:17.218774+0000","last_became_peered":"2023-10-16T20:06:17.218774+0000","last_unstale":"2023-10-16T20:06:18.197558+0000","last_undegraded":"2023-10-16T20:06:18.197558+0000","last_fullsized":"2023-10-16T20:06:18.197558+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:05:25.874296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.209799+0000","last_change":"2023-10-16T20:06:17.200644+0000","last_active":"2023-10-16T20:06:17.209799+0000","last_peered":"2023-10-16T20:06:17.209799+0000","last_clean":"2023-10-16T20:06:17.209799+0000","last_became_active":"2023-10-16T20:06:17.200350+0000","last_became_peered":"2023-10-16T20:06:17.200350+0000","last_unstale":"2023-10-16T20:06:17.209799+0000","last_undegraded":"2023-10-16T20:06:17.209799+0000","last_fullsized":"2023-10-16T20:06:17.209799+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:58:21.190657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196740+0000","last_change":"2023-10-16T20:06:17.224294+0000","last_active":"2023-10-16T20:06:18.196740+0000","last_peered":"2023-10-16T20:06:18.196740+0000","last_clean":"2023-10-16T20:06:18.196740+0000","last_became_active":"2023-10-16T20:06:17.223988+0000","last_became_peered":"2023-10-16T20:06:17.223988+0000","last_unstale":"2023-10-16T20:06:18.196740+0000","last_undegraded":"2023-10-16T20:06:18.196740+0000","last_fullsized":"2023-10-16T20:06:18.196740+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:03:40.257975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"40'4","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198438+0000","last_change":"2023-10-16T20:06:16.732366+0000","last_active":"2023-10-16T20:06:18.198438+0000","last_peered":"2023-10-16T20:06:18.198438+0000","last_clean":"2023-10-16T20:06:18.198438+0000","last_became_active":"2023-10-16T20:06:16.732035+0000","last_became_peered":"2023-10-16T20:06:16.732035+0000","last_unstale":"2023-10-16T20:06:18.198438+0000","last_undegraded":"2023-10-16T20:06:18.198438+0000","last_fullsized":"2023-10-16T20:06:18.198438+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:35:43.770049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.216156+0000","last_change":"2023-10-16T20:06:17.207721+0000","last_active":"2023-10-16T20:06:17.216156+0000","last_peered":"2023-10-16T20:06:17.216156+0000","last_clean":"2023-10-16T20:06:17.216156+0000","last_became_active":"2023-10-16T20:06:17.207470+0000","last_became_peered":"2023-10-16T20:06:17.207470+0000","last_unstale":"2023-10-16T20:06:17.216156+0000","last_undegraded":"2023-10-16T20:06:17.216156+0000","last_fullsized":"2023-10-16T20:06:17.216156+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:56:27.147860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T20:06:15.688195+0000","last_change":"2023-10-16T20:06:15.688195+0000","last_active":"2023-10-16T20:06:15.688195+0000","last_peered":"2023-10-16T20:06:15.688195+0000","last_clean":"2023-10-16T20:06:15.688195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T20:06:15.688195+0000","last_undegraded":"2023-10-16T20:06:15.688195+0000","last_fullsized":"2023-10-16T20:06:15.688195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.222274+0000","last_change":"2023-10-16T20:06:17.215735+0000","last_active":"2023-10-16T20:06:17.222274+0000","last_peered":"2023-10-16T20:06:17.222274+0000","last_clean":"2023-10-16T20:06:17.222274+0000","last_became_active":"2023-10-16T20:06:17.214986+0000","last_became_peered":"2023-10-16T20:06:17.214986+0000","last_unstale":"2023-10-16T20:06:17.222274+0000","last_undegraded":"2023-10-16T20:06:17.222274+0000","last_fullsized":"2023-10-16T20:06:17.222274+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:35:44.780849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"40'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.733064+0000","last_change":"2023-10-16T20:06:16.739387+0000","last_active":"2023-10-16T20:06:17.733064+0000","last_peered":"2023-10-16T20:06:17.733064+0000","last_clean":"2023-10-16T20:06:17.733064+0000","last_became_active":"2023-10-16T20:06:16.739070+0000","last_became_peered":"2023-10-16T20:06:16.739070+0000","last_unstale":"2023-10-16T20:06:17.733064+0000","last_undegraded":"2023-10-16T20:06:17.733064+0000","last_fullsized":"2023-10-16T20:06:17.733064+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:04:52.152758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.217801+0000","last_change":"2023-10-16T20:06:17.213835+0000","last_active":"2023-10-16T20:06:17.217801+0000","last_peered":"2023-10-16T20:06:17.217801+0000","last_clean":"2023-10-16T20:06:17.217801+0000","last_became_active":"2023-10-16T20:06:17.213516+0000","last_became_peered":"2023-10-16T20:06:17.213516+0000","last_unstale":"2023-10-16T20:06:17.217801+0000","last_undegraded":"2023-10-16T20:06:17.217801+0000","last_fullsized":"2023-10-16T20:06:17.217801+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:25:13.284821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"40'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.732355+0000","last_change":"2023-10-16T20:06:16.730019+0000","last_active":"2023-10-16T20:06:17.732355+0000","last_peered":"2023-10-16T20:06:17.732355+0000","last_clean":"2023-10-16T20:06:17.732355+0000","last_became_active":"2023-10-16T20:06:16.729804+0000","last_became_peered":"2023-10-16T20:06:16.729804+0000","last_unstale":"2023-10-16T20:06:17.732355+0000","last_undegraded":"2023-10-16T20:06:17.732355+0000","last_fullsized":"2023-10-16T20:06:17.732355+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:56:26.137061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T20:06:16.698995+0000","last_change":"2023-10-16T20:06:16.698995+0000","last_active":"2023-10-16T20:06:16.698995+0000","last_peered":"2023-10-16T20:06:16.698995+0000","last_clean":"2023-10-16T20:06:16.698995+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T20:06:16.698995+0000","last_undegraded":"2023-10-16T20:06:16.698995+0000","last_fullsized":"2023-10-16T20:06:16.698995+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"40'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.732663+0000","last_change":"2023-10-16T20:06:16.731997+0000","last_active":"2023-10-16T20:06:17.732663+0000","last_peered":"2023-10-16T20:06:17.732663+0000","last_clean":"2023-10-16T20:06:17.732663+0000","last_became_active":"2023-10-16T20:06:16.731714+0000","last_became_peered":"2023-10-16T20:06:16.731714+0000","last_unstale":"2023-10-16T20:06:17.732663+0000","last_undegraded":"2023-10-16T20:06:17.732663+0000","last_fullsized":"2023-10-16T20:06:17.732663+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:25:12.274021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.220705+0000","last_change":"2023-10-16T20:06:17.214298+0000","last_active":"2023-10-16T20:06:17.220705+0000","last_peered":"2023-10-16T20:06:17.220705+0000","last_clean":"2023-10-16T20:06:17.220705+0000","last_became_active":"2023-10-16T20:06:17.214038+0000","last_became_peered":"2023-10-16T20:06:17.214038+0000","last_unstale":"2023-10-16T20:06:17.220705+0000","last_undegraded":"2023-10-16T20:06:17.220705+0000","last_fullsized":"2023-10-16T20:06:17.220705+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:38:40.789443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"40'1","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196763+0000","last_change":"2023-10-16T20:06:16.737162+0000","last_active":"2023-10-16T20:06:18.196763+0000","last_peered":"2023-10-16T20:06:18.196763+0000","last_clean":"2023-10-16T20:06:18.196763+0000","last_became_active":"2023-10-16T20:06:16.736888+0000","last_became_peered":"2023-10-16T20:06:16.736888+0000","last_unstale":"2023-10-16T20:06:18.196763+0000","last_undegraded":"2023-10-16T20:06:18.196763+0000","last_fullsized":"2023-10-16T20:06:18.196763+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:22:31.971013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.211586+0000","last_change":"2023-10-16T20:06:17.208067+0000","last_active":"2023-10-16T20:06:17.211586+0000","last_peered":"2023-10-16T20:06:17.211586+0000","last_clean":"2023-10-16T20:06:17.211586+0000","last_became_active":"2023-10-16T20:06:17.207799+0000","last_became_peered":"2023-10-16T20:06:17.207799+0000","last_unstale":"2023-10-16T20:06:17.211586+0000","last_undegraded":"2023-10-16T20:06:17.211586+0000","last_fullsized":"2023-10-16T20:06:17.211586+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:40:58.934858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.195959+0000","last_change":"2023-10-16T20:06:16.729238+0000","last_active":"2023-10-16T20:06:17.195959+0000","last_peered":"2023-10-16T20:06:17.195959+0000","last_clean":"2023-10-16T20:06:17.195959+0000","last_became_active":"2023-10-16T20:06:16.728975+0000","last_became_peered":"2023-10-16T20:06:16.728975+0000","last_unstale":"2023-10-16T20:06:17.195959+0000","last_undegraded":"2023-10-16T20:06:17.195959+0000","last_fullsized":"2023-10-16T20:06:17.195959+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:36:26.871127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.220833+0000","last_change":"2023-10-16T20:06:17.218801+0000","last_active":"2023-10-16T20:06:17.220833+0000","last_peered":"2023-10-16T20:06:17.220833+0000","last_clean":"2023-10-16T20:06:17.220833+0000","last_became_active":"2023-10-16T20:06:17.218219+0000","last_became_peered":"2023-10-16T20:06:17.218219+0000","last_unstale":"2023-10-16T20:06:17.220833+0000","last_undegraded":"2023-10-16T20:06:17.220833+0000","last_fullsized":"2023-10-16T20:06:17.220833+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:27:39.178553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.202521+0000","last_change":"2023-10-16T20:06:16.720994+0000","last_active":"2023-10-16T20:06:17.202521+0000","last_peered":"2023-10-16T20:06:17.202521+0000","last_clean":"2023-10-16T20:06:17.202521+0000","last_became_active":"2023-10-16T20:06:16.720623+0000","last_became_peered":"2023-10-16T20:06:16.720623+0000","last_unstale":"2023-10-16T20:06:17.202521+0000","last_undegraded":"2023-10-16T20:06:17.202521+0000","last_fullsized":"2023-10-16T20:06:17.202521+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:32:43.082291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357768+0000","last_change":"2023-10-16T20:06:17.216335+0000","last_active":"2023-10-16T20:06:18.357768+0000","last_peered":"2023-10-16T20:06:18.357768+0000","last_clean":"2023-10-16T20:06:18.357768+0000","last_became_active":"2023-10-16T20:06:17.216070+0000","last_became_peered":"2023-10-16T20:06:17.216070+0000","last_unstale":"2023-10-16T20:06:18.357768+0000","last_undegraded":"2023-10-16T20:06:18.357768+0000","last_fullsized":"2023-10-16T20:06:18.357768+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:08:22.444705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.206243+0000","last_change":"2023-10-16T20:06:16.736582+0000","last_active":"2023-10-16T20:06:17.206243+0000","last_peered":"2023-10-16T20:06:17.206243+0000","last_clean":"2023-10-16T20:06:17.206243+0000","last_became_active":"2023-10-16T20:06:16.736322+0000","last_became_peered":"2023-10-16T20:06:16.736322+0000","last_unstale":"2023-10-16T20:06:17.206243+0000","last_undegraded":"2023-10-16T20:06:17.206243+0000","last_fullsized":"2023-10-16T20:06:17.206243+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:38:29.780087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197108+0000","last_change":"2023-10-16T20:06:17.220312+0000","last_active":"2023-10-16T20:06:18.197108+0000","last_peered":"2023-10-16T20:06:18.197108+0000","last_clean":"2023-10-16T20:06:18.197108+0000","last_became_active":"2023-10-16T20:06:17.220033+0000","last_became_peered":"2023-10-16T20:06:17.220033+0000","last_unstale":"2023-10-16T20:06:18.197108+0000","last_undegraded":"2023-10-16T20:06:18.197108+0000","last_fullsized":"2023-10-16T20:06:18.197108+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:00:59.428210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.198151+0000","last_change":"2023-10-16T20:06:16.725615+0000","last_active":"2023-10-16T20:06:17.198151+0000","last_peered":"2023-10-16T20:06:17.198151+0000","last_clean":"2023-10-16T20:06:17.198151+0000","last_became_active":"2023-10-16T20:06:16.725257+0000","last_became_peered":"2023-10-16T20:06:16.725257+0000","last_unstale":"2023-10-16T20:06:17.198151+0000","last_undegraded":"2023-10-16T20:06:17.198151+0000","last_fullsized":"2023-10-16T20:06:17.198151+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:36:24.034397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T20:06:16.698995+0000","last_change":"2023-10-16T20:06:16.698995+0000","last_active":"2023-10-16T20:06:16.698995+0000","last_peered":"2023-10-16T20:06:16.698995+0000","last_clean":"2023-10-16T20:06:16.698995+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T20:06:16.698995+0000","last_undegraded":"2023-10-16T20:06:16.698995+0000","last_fullsized":"2023-10-16T20:06:16.698995+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"40'3","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.733519+0000","last_change":"2023-10-16T20:06:16.734102+0000","last_active":"2023-10-16T20:06:17.733519+0000","last_peered":"2023-10-16T20:06:17.733519+0000","last_clean":"2023-10-16T20:06:17.733519+0000","last_became_active":"2023-10-16T20:06:16.733829+0000","last_became_peered":"2023-10-16T20:06:16.733829+0000","last_unstale":"2023-10-16T20:06:17.733519+0000","last_undegraded":"2023-10-16T20:06:17.733519+0000","last_fullsized":"2023-10-16T20:06:17.733519+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:32:58.366700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.221860+0000","last_change":"2023-10-16T20:06:17.214205+0000","last_active":"2023-10-16T20:06:17.221860+0000","last_peered":"2023-10-16T20:06:17.221860+0000","last_clean":"2023-10-16T20:06:17.221860+0000","last_became_active":"2023-10-16T20:06:17.213261+0000","last_became_peered":"2023-10-16T20:06:17.213261+0000","last_unstale":"2023-10-16T20:06:17.221860+0000","last_undegraded":"2023-10-16T20:06:17.221860+0000","last_fullsized":"2023-10-16T20:06:17.221860+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:20:06.934366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"40'6","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.738226+0000","last_change":"2023-10-16T20:06:16.721479+0000","last_active":"2023-10-16T20:06:17.738226+0000","last_peered":"2023-10-16T20:06:17.738226+0000","last_clean":"2023-10-16T20:06:17.738226+0000","last_became_active":"2023-10-16T20:06:16.721136+0000","last_became_peered":"2023-10-16T20:06:16.721136+0000","last_unstale":"2023-10-16T20:06:17.738226+0000","last_undegraded":"2023-10-16T20:06:17.738226+0000","last_fullsized":"2023-10-16T20:06:17.738226+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:32:58.366700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T20:06:16.698995+0000","last_change":"2023-10-16T20:06:16.698995+0000","last_active":"2023-10-16T20:06:16.698995+0000","last_peered":"2023-10-16T20:06:16.698995+0000","last_clean":"2023-10-16T20:06:16.698995+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T20:06:16.698995+0000","last_undegraded":"2023-10-16T20:06:16.698995+0000","last_fullsized":"2023-10-16T20:06:16.698995+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"40'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.732488+0000","last_change":"2023-10-16T20:06:16.735005+0000","last_active":"2023-10-16T20:06:17.732488+0000","last_peered":"2023-10-16T20:06:17.732488+0000","last_clean":"2023-10-16T20:06:17.732488+0000","last_became_active":"2023-10-16T20:06:16.734752+0000","last_became_peered":"2023-10-16T20:06:16.734752+0000","last_unstale":"2023-10-16T20:06:17.732488+0000","last_undegraded":"2023-10-16T20:06:17.732488+0000","last_fullsized":"2023-10-16T20:06:17.732488+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:21:23.740779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.219739+0000","last_change":"2023-10-16T20:06:17.215993+0000","last_active":"2023-10-16T20:06:17.219739+0000","last_peered":"2023-10-16T20:06:17.219739+0000","last_clean":"2023-10-16T20:06:17.219739+0000","last_became_active":"2023-10-16T20:06:17.215736+0000","last_became_peered":"2023-10-16T20:06:17.215736+0000","last_unstale":"2023-10-16T20:06:17.219739+0000","last_undegraded":"2023-10-16T20:06:17.219739+0000","last_fullsized":"2023-10-16T20:06:17.219739+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:36:25.045197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.195984+0000","last_change":"2023-10-16T20:06:16.733426+0000","last_active":"2023-10-16T20:06:17.195984+0000","last_peered":"2023-10-16T20:06:17.195984+0000","last_clean":"2023-10-16T20:06:17.195984+0000","last_became_active":"2023-10-16T20:06:16.732600+0000","last_became_peered":"2023-10-16T20:06:16.732600+0000","last_unstale":"2023-10-16T20:06:17.195984+0000","last_undegraded":"2023-10-16T20:06:17.195984+0000","last_fullsized":"2023-10-16T20:06:17.195984+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:12.864132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196653+0000","last_change":"2023-10-16T20:06:17.219925+0000","last_active":"2023-10-16T20:06:18.196653+0000","last_peered":"2023-10-16T20:06:18.196653+0000","last_clean":"2023-10-16T20:06:18.196653+0000","last_became_active":"2023-10-16T20:06:17.219584+0000","last_became_peered":"2023-10-16T20:06:17.219584+0000","last_unstale":"2023-10-16T20:06:18.196653+0000","last_undegraded":"2023-10-16T20:06:18.196653+0000","last_fullsized":"2023-10-16T20:06:18.196653+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:44:14.727872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197391+0000","last_change":"2023-10-16T20:06:16.732499+0000","last_active":"2023-10-16T20:06:18.197391+0000","last_peered":"2023-10-16T20:06:18.197391+0000","last_clean":"2023-10-16T20:06:18.197391+0000","last_became_active":"2023-10-16T20:06:16.732244+0000","last_became_peered":"2023-10-16T20:06:16.732244+0000","last_unstale":"2023-10-16T20:06:18.197391+0000","last_undegraded":"2023-10-16T20:06:18.197391+0000","last_fullsized":"2023-10-16T20:06:18.197391+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:40:57.924059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.219863+0000","last_change":"2023-10-16T20:06:17.215267+0000","last_active":"2023-10-16T20:06:17.219863+0000","last_peered":"2023-10-16T20:06:17.219863+0000","last_clean":"2023-10-16T20:06:17.219863+0000","last_became_active":"2023-10-16T20:06:17.215017+0000","last_became_peered":"2023-10-16T20:06:17.215017+0000","last_unstale":"2023-10-16T20:06:17.219863+0000","last_undegraded":"2023-10-16T20:06:17.219863+0000","last_fullsized":"2023-10-16T20:06:17.219863+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:38:40.789443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"40'8","reported_seq":20,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196540+0000","last_change":"2023-10-16T20:06:16.730165+0000","last_active":"2023-10-16T20:06:18.196540+0000","last_peered":"2023-10-16T20:06:18.196540+0000","last_clean":"2023-10-16T20:06:18.196540+0000","last_became_active":"2023-10-16T20:06:16.729894+0000","last_became_peered":"2023-10-16T20:06:16.729894+0000","last_unstale":"2023-10-16T20:06:18.196540+0000","last_undegraded":"2023-10-16T20:06:18.196540+0000","last_fullsized":"2023-10-16T20:06:18.196540+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:36:26.871127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.217655+0000","last_change":"2023-10-16T20:06:17.214039+0000","last_active":"2023-10-16T20:06:17.217655+0000","last_peered":"2023-10-16T20:06:17.217655+0000","last_clean":"2023-10-16T20:06:17.217655+0000","last_became_active":"2023-10-16T20:06:17.213551+0000","last_became_peered":"2023-10-16T20:06:17.213551+0000","last_unstale":"2023-10-16T20:06:17.217655+0000","last_undegraded":"2023-10-16T20:06:17.217655+0000","last_fullsized":"2023-10-16T20:06:17.217655+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:22:32.981812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"40'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.735204+0000","last_change":"2023-10-16T20:06:16.726366+0000","last_active":"2023-10-16T20:06:17.735204+0000","last_peered":"2023-10-16T20:06:17.735204+0000","last_clean":"2023-10-16T20:06:17.735204+0000","last_became_active":"2023-10-16T20:06:16.726086+0000","last_became_peered":"2023-10-16T20:06:16.726086+0000","last_unstale":"2023-10-16T20:06:17.735204+0000","last_undegraded":"2023-10-16T20:06:17.735204+0000","last_fullsized":"2023-10-16T20:06:17.735204+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:21:23.740779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.216516+0000","last_change":"2023-10-16T20:06:17.207345+0000","last_active":"2023-10-16T20:06:17.216516+0000","last_peered":"2023-10-16T20:06:17.216516+0000","last_clean":"2023-10-16T20:06:17.216516+0000","last_became_active":"2023-10-16T20:06:17.207082+0000","last_became_peered":"2023-10-16T20:06:17.207082+0000","last_unstale":"2023-10-16T20:06:17.216516+0000","last_undegraded":"2023-10-16T20:06:17.216516+0000","last_fullsized":"2023-10-16T20:06:17.216516+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:04:53.163558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"40'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.733912+0000","last_change":"2023-10-16T20:06:16.727808+0000","last_active":"2023-10-16T20:06:17.733912+0000","last_peered":"2023-10-16T20:06:17.733912+0000","last_clean":"2023-10-16T20:06:17.733912+0000","last_became_active":"2023-10-16T20:06:16.727400+0000","last_became_peered":"2023-10-16T20:06:16.727400+0000","last_unstale":"2023-10-16T20:06:17.733912+0000","last_undegraded":"2023-10-16T20:06:17.733912+0000","last_fullsized":"2023-10-16T20:06:17.733912+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:00.635492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.222291+0000","last_change":"2023-10-16T20:06:17.216271+0000","last_active":"2023-10-16T20:06:17.222291+0000","last_peered":"2023-10-16T20:06:17.222291+0000","last_clean":"2023-10-16T20:06:17.222291+0000","last_became_active":"2023-10-16T20:06:17.215201+0000","last_became_peered":"2023-10-16T20:06:17.215201+0000","last_unstale":"2023-10-16T20:06:17.222291+0000","last_undegraded":"2023-10-16T20:06:17.222291+0000","last_fullsized":"2023-10-16T20:06:17.222291+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:59:17.101302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"40'1","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357862+0000","last_change":"2023-10-16T20:06:16.724780+0000","last_active":"2023-10-16T20:06:18.357862+0000","last_peered":"2023-10-16T20:06:18.357862+0000","last_clean":"2023-10-16T20:06:18.357862+0000","last_became_active":"2023-10-16T20:06:16.724476+0000","last_became_peered":"2023-10-16T20:06:16.724476+0000","last_unstale":"2023-10-16T20:06:18.357862+0000","last_undegraded":"2023-10-16T20:06:18.357862+0000","last_fullsized":"2023-10-16T20:06:18.357862+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:19:37.004795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.358046+0000","last_change":"2023-10-16T20:06:17.209520+0000","last_active":"2023-10-16T20:06:18.358046+0000","last_peered":"2023-10-16T20:06:18.358046+0000","last_clean":"2023-10-16T20:06:18.358046+0000","last_became_active":"2023-10-16T20:06:17.209236+0000","last_became_peered":"2023-10-16T20:06:17.209236+0000","last_unstale":"2023-10-16T20:06:18.358046+0000","last_undegraded":"2023-10-16T20:06:18.358046+0000","last_fullsized":"2023-10-16T20:06:18.358046+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:31:26.166083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.196031+0000","last_change":"2023-10-16T20:06:16.734625+0000","last_active":"2023-10-16T20:06:17.196031+0000","last_peered":"2023-10-16T20:06:17.196031+0000","last_clean":"2023-10-16T20:06:17.196031+0000","last_became_active":"2023-10-16T20:06:16.734353+0000","last_became_peered":"2023-10-16T20:06:16.734353+0000","last_unstale":"2023-10-16T20:06:17.196031+0000","last_undegraded":"2023-10-16T20:06:17.196031+0000","last_fullsized":"2023-10-16T20:06:17.196031+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:38:39.778643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196838+0000","last_change":"2023-10-16T20:06:17.219357+0000","last_active":"2023-10-16T20:06:18.196838+0000","last_peered":"2023-10-16T20:06:18.196838+0000","last_clean":"2023-10-16T20:06:18.196838+0000","last_became_active":"2023-10-16T20:06:17.218995+0000","last_became_peered":"2023-10-16T20:06:17.218995+0000","last_unstale":"2023-10-16T20:06:18.196838+0000","last_undegraded":"2023-10-16T20:06:18.196838+0000","last_fullsized":"2023-10-16T20:06:18.196838+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:44:43.545772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"40'2","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357665+0000","last_change":"2023-10-16T20:06:16.732782+0000","last_active":"2023-10-16T20:06:18.357665+0000","last_peered":"2023-10-16T20:06:18.357665+0000","last_clean":"2023-10-16T20:06:18.357665+0000","last_became_active":"2023-10-16T20:06:16.732498+0000","last_became_peered":"2023-10-16T20:06:16.732498+0000","last_unstale":"2023-10-16T20:06:18.357665+0000","last_undegraded":"2023-10-16T20:06:18.357665+0000","last_fullsized":"2023-10-16T20:06:18.357665+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:21:27.852426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T20:06:16.698995+0000","last_change":"2023-10-16T20:06:16.698995+0000","last_active":"2023-10-16T20:06:16.698995+0000","last_peered":"2023-10-16T20:06:16.698995+0000","last_clean":"2023-10-16T20:06:16.698995+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T20:06:16.698995+0000","last_undegraded":"2023-10-16T20:06:16.698995+0000","last_fullsized":"2023-10-16T20:06:16.698995+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357669+0000","last_change":"2023-10-16T20:06:16.732525+0000","last_active":"2023-10-16T20:06:18.357669+0000","last_peered":"2023-10-16T20:06:18.357669+0000","last_clean":"2023-10-16T20:06:18.357669+0000","last_became_active":"2023-10-16T20:06:16.732199+0000","last_became_peered":"2023-10-16T20:06:16.732199+0000","last_unstale":"2023-10-16T20:06:18.357669+0000","last_undegraded":"2023-10-16T20:06:18.357669+0000","last_fullsized":"2023-10-16T20:06:18.357669+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:12:42.667116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.216176+0000","last_change":"2023-10-16T20:06:17.209547+0000","last_active":"2023-10-16T20:06:17.216176+0000","last_peered":"2023-10-16T20:06:17.216176+0000","last_clean":"2023-10-16T20:06:17.216176+0000","last_became_active":"2023-10-16T20:06:17.209281+0000","last_became_peered":"2023-10-16T20:06:17.209281+0000","last_unstale":"2023-10-16T20:06:17.216176+0000","last_undegraded":"2023-10-16T20:06:17.216176+0000","last_fullsized":"2023-10-16T20:06:17.216176+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:58:21.190657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197648+0000","last_change":"2023-10-16T20:06:16.727775+0000","last_active":"2023-10-16T20:06:18.197648+0000","last_peered":"2023-10-16T20:06:18.197648+0000","last_clean":"2023-10-16T20:06:18.197648+0000","last_became_active":"2023-10-16T20:06:16.727492+0000","last_became_peered":"2023-10-16T20:06:16.727492+0000","last_unstale":"2023-10-16T20:06:18.197648+0000","last_undegraded":"2023-10-16T20:06:18.197648+0000","last_fullsized":"2023-10-16T20:06:18.197648+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:32:58.366700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.220851+0000","last_change":"2023-10-16T20:06:17.218801+0000","last_active":"2023-10-16T20:06:17.220851+0000","last_peered":"2023-10-16T20:06:17.220851+0000","last_clean":"2023-10-16T20:06:17.220851+0000","last_became_active":"2023-10-16T20:06:17.218567+0000","last_became_peered":"2023-10-16T20:06:17.218567+0000","last_unstale":"2023-10-16T20:06:17.220851+0000","last_undegraded":"2023-10-16T20:06:17.220851+0000","last_fullsized":"2023-10-16T20:06:17.220851+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:13.874932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357557+0000","last_change":"2023-10-16T20:06:16.727754+0000","last_active":"2023-10-16T20:06:18.357557+0000","last_peered":"2023-10-16T20:06:18.357557+0000","last_clean":"2023-10-16T20:06:18.357557+0000","last_became_active":"2023-10-16T20:06:16.727497+0000","last_became_peered":"2023-10-16T20:06:16.727497+0000","last_unstale":"2023-10-16T20:06:18.357557+0000","last_undegraded":"2023-10-16T20:06:18.357557+0000","last_fullsized":"2023-10-16T20:06:18.357557+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:08:12.647505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.209898+0000","last_change":"2023-10-16T20:06:17.201294+0000","last_active":"2023-10-16T20:06:17.209898+0000","last_peered":"2023-10-16T20:06:17.209898+0000","last_clean":"2023-10-16T20:06:17.209898+0000","last_became_active":"2023-10-16T20:06:17.200853+0000","last_became_peered":"2023-10-16T20:06:17.200853+0000","last_unstale":"2023-10-16T20:06:17.209898+0000","last_undegraded":"2023-10-16T20:06:17.209898+0000","last_fullsized":"2023-10-16T20:06:17.209898+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:01.646291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"40'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.732982+0000","last_change":"2023-10-16T20:06:16.733877+0000","last_active":"2023-10-16T20:06:17.732982+0000","last_peered":"2023-10-16T20:06:17.732982+0000","last_clean":"2023-10-16T20:06:17.732982+0000","last_became_active":"2023-10-16T20:06:16.733570+0000","last_became_peered":"2023-10-16T20:06:16.733570+0000","last_unstale":"2023-10-16T20:06:17.732982+0000","last_undegraded":"2023-10-16T20:06:17.732982+0000","last_fullsized":"2023-10-16T20:06:17.732982+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:38:29.780087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197199+0000","last_change":"2023-10-16T20:06:17.220488+0000","last_active":"2023-10-16T20:06:18.197199+0000","last_peered":"2023-10-16T20:06:18.197199+0000","last_clean":"2023-10-16T20:06:18.197199+0000","last_became_active":"2023-10-16T20:06:17.220060+0000","last_became_peered":"2023-10-16T20:06:17.220060+0000","last_unstale":"2023-10-16T20:06:18.197199+0000","last_undegraded":"2023-10-16T20:06:18.197199+0000","last_fullsized":"2023-10-16T20:06:18.197199+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:59:16.509971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357857+0000","last_change":"2023-10-16T20:06:16.726490+0000","last_active":"2023-10-16T20:06:18.357857+0000","last_peered":"2023-10-16T20:06:18.357857+0000","last_clean":"2023-10-16T20:06:18.357857+0000","last_became_active":"2023-10-16T20:06:16.726210+0000","last_became_peered":"2023-10-16T20:06:16.726210+0000","last_unstale":"2023-10-16T20:06:18.357857+0000","last_undegraded":"2023-10-16T20:06:18.357857+0000","last_fullsized":"2023-10-16T20:06:18.357857+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:13:14.492857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.217565+0000","last_change":"2023-10-16T20:06:17.213630+0000","last_active":"2023-10-16T20:06:17.217565+0000","last_peered":"2023-10-16T20:06:17.217565+0000","last_clean":"2023-10-16T20:06:17.217565+0000","last_became_active":"2023-10-16T20:06:17.213352+0000","last_became_peered":"2023-10-16T20:06:17.213352+0000","last_unstale":"2023-10-16T20:06:17.217565+0000","last_undegraded":"2023-10-16T20:06:17.217565+0000","last_fullsized":"2023-10-16T20:06:17.217565+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:18:14.480179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T20:06:15.688195+0000","last_change":"2023-10-16T20:06:15.688195+0000","last_active":"2023-10-16T20:06:15.688195+0000","last_peered":"2023-10-16T20:06:15.688195+0000","last_clean":"2023-10-16T20:06:15.688195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T20:06:15.688195+0000","last_undegraded":"2023-10-16T20:06:15.688195+0000","last_fullsized":"2023-10-16T20:06:15.688195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T20:06:16.698995+0000","last_change":"2023-10-16T20:06:16.698995+0000","last_active":"2023-10-16T20:06:16.698995+0000","last_peered":"2023-10-16T20:06:16.698995+0000","last_clean":"2023-10-16T20:06:16.698995+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T20:06:16.698995+0000","last_undegraded":"2023-10-16T20:06:16.698995+0000","last_fullsized":"2023-10-16T20:06:16.698995+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T20:06:15.688195+0000","last_change":"2023-10-16T20:06:15.688195+0000","last_active":"2023-10-16T20:06:15.688195+0000","last_peered":"2023-10-16T20:06:15.688195+0000","last_clean":"2023-10-16T20:06:15.688195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T20:06:15.688195+0000","last_undegraded":"2023-10-16T20:06:15.688195+0000","last_fullsized":"2023-10-16T20:06:15.688195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.216410+0000","last_change":"2023-10-16T20:06:17.208914+0000","last_active":"2023-10-16T20:06:17.216410+0000","last_peered":"2023-10-16T20:06:17.216410+0000","last_clean":"2023-10-16T20:06:17.216410+0000","last_became_active":"2023-10-16T20:06:17.208609+0000","last_became_peered":"2023-10-16T20:06:17.208609+0000","last_unstale":"2023-10-16T20:06:17.216410+0000","last_undegraded":"2023-10-16T20:06:17.216410+0000","last_fullsized":"2023-10-16T20:06:17.216410+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:32:44.093091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197750+0000","last_change":"2023-10-16T20:06:16.733015+0000","last_active":"2023-10-16T20:06:18.197750+0000","last_peered":"2023-10-16T20:06:18.197750+0000","last_clean":"2023-10-16T20:06:18.197750+0000","last_became_active":"2023-10-16T20:06:16.731413+0000","last_became_peered":"2023-10-16T20:06:16.731413+0000","last_unstale":"2023-10-16T20:06:18.197750+0000","last_undegraded":"2023-10-16T20:06:18.197750+0000","last_fullsized":"2023-10-16T20:06:18.197750+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:59:16.090503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.222240+0000","last_change":"2023-10-16T20:06:17.214304+0000","last_active":"2023-10-16T20:06:17.222240+0000","last_peered":"2023-10-16T20:06:17.222240+0000","last_clean":"2023-10-16T20:06:17.222240+0000","last_became_active":"2023-10-16T20:06:17.213521+0000","last_became_peered":"2023-10-16T20:06:17.213521+0000","last_unstale":"2023-10-16T20:06:17.222240+0000","last_undegraded":"2023-10-16T20:06:17.222240+0000","last_fullsized":"2023-10-16T20:06:17.222240+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:27:10.715147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.202689+0000","last_change":"2023-10-16T20:06:16.730112+0000","last_active":"2023-10-16T20:06:17.202689+0000","last_peered":"2023-10-16T20:06:17.202689+0000","last_clean":"2023-10-16T20:06:17.202689+0000","last_became_active":"2023-10-16T20:06:16.729781+0000","last_became_peered":"2023-10-16T20:06:16.729781+0000","last_unstale":"2023-10-16T20:06:17.202689+0000","last_undegraded":"2023-10-16T20:06:17.202689+0000","last_fullsized":"2023-10-16T20:06:17.202689+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:39:05.076198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197008+0000","last_change":"2023-10-16T20:06:17.221997+0000","last_active":"2023-10-16T20:06:18.197008+0000","last_peered":"2023-10-16T20:06:18.197008+0000","last_clean":"2023-10-16T20:06:18.197008+0000","last_became_active":"2023-10-16T20:06:17.221704+0000","last_became_peered":"2023-10-16T20:06:17.221704+0000","last_unstale":"2023-10-16T20:06:18.197008+0000","last_undegraded":"2023-10-16T20:06:18.197008+0000","last_fullsized":"2023-10-16T20:06:18.197008+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:38:22.493819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.201889+0000","last_change":"2023-10-16T20:06:16.733171+0000","last_active":"2023-10-16T20:06:17.201889+0000","last_peered":"2023-10-16T20:06:17.201889+0000","last_clean":"2023-10-16T20:06:17.201889+0000","last_became_active":"2023-10-16T20:06:16.732869+0000","last_became_peered":"2023-10-16T20:06:16.732869+0000","last_unstale":"2023-10-16T20:06:17.201889+0000","last_undegraded":"2023-10-16T20:06:17.201889+0000","last_fullsized":"2023-10-16T20:06:17.201889+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:04:52.152758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.221418+0000","last_change":"2023-10-16T20:06:17.215196+0000","last_active":"2023-10-16T20:06:17.221418+0000","last_peered":"2023-10-16T20:06:17.221418+0000","last_clean":"2023-10-16T20:06:17.221418+0000","last_became_active":"2023-10-16T20:06:17.214556+0000","last_became_peered":"2023-10-16T20:06:17.214556+0000","last_unstale":"2023-10-16T20:06:17.221418+0000","last_undegraded":"2023-10-16T20:06:17.221418+0000","last_fullsized":"2023-10-16T20:06:17.221418+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:24:29.941143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196810+0000","last_change":"2023-10-16T20:06:16.728925+0000","last_active":"2023-10-16T20:06:18.196810+0000","last_peered":"2023-10-16T20:06:18.196810+0000","last_clean":"2023-10-16T20:06:18.196810+0000","last_became_active":"2023-10-16T20:06:16.728663+0000","last_became_peered":"2023-10-16T20:06:16.728663+0000","last_unstale":"2023-10-16T20:06:18.196810+0000","last_undegraded":"2023-10-16T20:06:18.196810+0000","last_fullsized":"2023-10-16T20:06:18.196810+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:50:31.301972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":182,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.208038+0000","last_change":"2023-10-16T20:04:46.335822+0000","last_active":"2023-10-16T20:06:17.208038+0000","last_peered":"2023-10-16T20:06:17.208038+0000","last_clean":"2023-10-16T20:06:17.208038+0000","last_became_active":"2023-10-16T20:04:46.334617+0000","last_became_peered":"2023-10-16T20:04:46.334617+0000","last_unstale":"2023-10-16T20:06:17.208038+0000","last_undegraded":"2023-10-16T20:06:17.208038+0000","last_fullsized":"2023-10-16T20:06:17.208038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:45.306240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:45.306240+0000","last_clean_scrub_stamp":"2023-10-16T20:04:45.306240+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:42:11.153721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":112,"acting":112,"num_store_stats":7},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":58,"acting":58,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509462,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1048576,"data_stored":783264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.86199999999999999}]},{"osd":1,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.4900000000000002,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":0.752}]},{"osd":2,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.0550000000000002,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.80300000000000005}]},{"osd":3,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.629,"5min":2.629,"15min":2.629},"last":0.67400000000000004}]},{"osd":4,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":0.63200000000000001}]},{"osd":5,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.8809999999999998,"5min":2.8809999999999998,"15min":2.8809999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.65400000000000003}]},{"osd":6,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.56999999999999995}]}]},{"osd":6,"up_from":15,"seq":64424509463,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":192896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":4.5880000000000001,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":4.6139999999999999,"5min":4.6139999999999999,"15min":4.6139999999999999},"last":0.629}]},{"osd":1,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":4.6639999999999997,"5min":4.6639999999999997,"15min":4.6639999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":4.8140000000000001,"5min":4.8140000000000001,"15min":4.8140000000000001},"last":0.60899999999999999}]},{"osd":2,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":5.3769999999999998,"5min":5.3769999999999998,"15min":5.3769999999999998},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.79000000000000004}]},{"osd":3,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":4.5460000000000003,"5min":4.5460000000000003,"15min":4.5460000000000003},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.66100000000000003}]},{"osd":4,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.51900000000000002}]},{"osd":5,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.76400000000000001}]},{"osd":7,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.88500000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509461,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":462848,"data_stored":193008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.70999999999999996}]},{"osd":1,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.623},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.76200000000000001}]},{"osd":2,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.747},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.67800000000000005}]},{"osd":3,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.73199999999999998}]},{"osd":4,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.50700000000000001}]},{"osd":6,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.56000000000000005}]},{"osd":7,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.53800000000000003}]}]},{"osd":4,"up_from":15,"seq":64424509461,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31632,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340208,"statfs":{"total":96636764160,"available":96604372992,"internally_reserved":0,"allocated":454656,"data_stored":192896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":3.835,"5min":3.835,"15min":3.835},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":3.7610000000000001,"5min":3.7610000000000001,"15min":3.7610000000000001},"last":0.75700000000000001}]},{"osd":1,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.57599999999999996}]},{"osd":2,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.628},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.60799999999999998}]},{"osd":3,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.72399999999999998}]},{"osd":5,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.5}]},{"osd":6,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.55100000000000005}]},{"osd":7,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.8919999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.74099999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509462,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":194542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.47599999999999998}]},{"osd":1,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.497}]},{"osd":2,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.54200000000000004}]},{"osd":4,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.88200000000000001}]},{"osd":5,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.876}]},{"osd":6,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.95499999999999996}]},{"osd":7,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.75800000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509461,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":192896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.53400000000000003}]},{"osd":1,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.436}]},{"osd":3,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.48399999999999999}]},{"osd":4,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.59699999999999998}]},{"osd":5,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.63}]},{"osd":6,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.80100000000000005}]},{"osd":7,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.70499999999999996}]}]},{"osd":1,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":192896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.867,"5min":3.867,"15min":3.867},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.52700000000000002}]},{"osd":2,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":5.6040000000000001,"5min":5.6040000000000001,"15min":5.6040000000000001},"last":0.434},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":7.8380000000000001,"5min":7.8380000000000001,"15min":7.8380000000000001},"last":0.45100000000000001}]},{"osd":3,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":5.2039999999999997,"5min":5.2039999999999997,"15min":5.2039999999999997},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":3.9470000000000001,"5min":3.9470000000000001,"15min":3.9470000000000001},"last":0.47399999999999998}]},{"osd":4,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":4.1500000000000004,"5min":4.1500000000000004,"15min":4.1500000000000004},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":4.2619999999999996,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":0.79400000000000004}]},{"osd":5,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":4.0910000000000002,"5min":4.0910000000000002,"15min":4.0910000000000002},"last":0.86099999999999999}]},{"osd":6,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":4.21,"5min":4.21,"15min":4.21},"last":0.88},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.83399999999999996}]},{"osd":7,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":4.1849999999999996,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":0.82099999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1011712,"data_stored":763382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":3.9289999999999998,"5min":3.9289999999999998,"15min":3.9289999999999998},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":4.0910000000000002,"5min":4.0910000000000002,"15min":4.0910000000000002},"last":0.32900000000000001}]},{"osd":2,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":4.1520000000000001,"5min":4.1520000000000001,"15min":4.1520000000000001},"last":0.434},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":4.1879999999999997,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.45100000000000001}]},{"osd":3,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":4.3440000000000003,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.49099999999999999}]},{"osd":4,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":4.5250000000000004,"5min":4.5250000000000004,"15min":4.5250000000000004},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.69799999999999995}]},{"osd":5,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":3.9990000000000001,"5min":3.9990000000000001,"15min":3.9990000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":4.4779999999999998,"5min":4.4779999999999998,"15min":4.4779999999999998},"last":0.81999999999999995}]},{"osd":6,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":3.956,"5min":3.956,"15min":3.956},"last":0.71199999999999997}]},{"osd":7,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":4.0289999999999999,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":4.5640000000000001,"5min":4.5640000000000001,"15min":4.5640000000000001},"last":0.72199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T20:06:22.169 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T20:06:22.343 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.335+0000 7f5d18f35640 1 Processor -- start 2023-10-16T20:06:22.343 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.335+0000 7f5d18f35640 1 -- start start 2023-10-16T20:06:22.345 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.339+0000 7f5d18f35640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5d1413f730 0x7f5d140cc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:22.345 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.339+0000 7f5d18f35640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d140abaa0 con 0x7f5d1413fb10 2023-10-16T20:06:22.345 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.339+0000 7f5d18f35640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d140abd80 con 0x7f5d1406a5a0 2023-10-16T20:06:22.345 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.339+0000 7f5d18f35640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d140ab890 con 0x7f5d1413f730 2023-10-16T20:06:22.345 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.339+0000 7f5d1259c640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5d1413f730 0x7f5d140cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:22.345 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.339+0000 7f5d1259c640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5d1413f730 0x7f5d140cc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57526/0 (socket says 172.21.15.88:57526) 2023-10-16T20:06:22.346 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.339+0000 7f5d1259c640 1 -- 172.21.15.88:0/4051280878 learned_addr learned my addr 172.21.15.88:0/4051280878 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:22.346 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.339+0000 7f5d11d9b640 1 --1- 172.21.15.88:0/4051280878 >> v1:172.21.15.88:6789/0 conn(0x7f5d1413fb10 0x7f5d14069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:51130/0 (socket says 172.21.15.88:51130) 2023-10-16T20:06:22.346 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.339+0000 7f5d1259c640 1 -- 172.21.15.88:0/4051280878 >> v1:172.21.15.94:6789/0 conn(0x7f5d1406a5a0 legacy=0x7f5d1415d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:22.346 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.339+0000 7f5d1259c640 1 -- 172.21.15.88:0/4051280878 >> v1:172.21.15.88:6789/0 conn(0x7f5d1413fb10 legacy=0x7f5d14069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:22.346 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.339+0000 7f5d1259c640 1 -- 172.21.15.88:0/4051280878 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d140ac060 con 0x7f5d1413f730 2023-10-16T20:06:22.347 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.339+0000 7f5d1159a640 1 -- 172.21.15.88:0/4051280878 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 225782167 0 0) 0x7f5d140abaa0 con 0x7f5d1413fb10 2023-10-16T20:06:22.348 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.343+0000 7f5d1259c640 1 --2- 172.21.15.88:0/4051280878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d1413f730 0x7f5d140cc120 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5d04009870 tx=0x7f5d0401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b8a1983010c6ec server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:22.348 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.343+0000 7f5d1159a640 1 -- 172.21.15.88:0/4051280878 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d040ab620 con 0x7f5d1413f730 2023-10-16T20:06:22.348 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.343+0000 7f5d1159a640 1 -- 172.21.15.88:0/4051280878 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d040ab780 con 0x7f5d1413f730 2023-10-16T20:06:22.348 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.343+0000 7f5d1159a640 1 -- 172.21.15.88:0/4051280878 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d040abb10 con 0x7f5d1413f730 2023-10-16T20:06:22.349 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.343+0000 7f5d18f35640 1 -- 172.21.15.88:0/4051280878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d1413f730 msgr2=0x7f5d140cc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:22.350 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.343+0000 7f5d18f35640 1 --2- 172.21.15.88:0/4051280878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d1413f730 0x7f5d140cc120 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5d04009870 tx=0x7f5d0401c510 comp rx=0 tx=0).stop 2023-10-16T20:06:22.350 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.343+0000 7f5d18f35640 1 -- 172.21.15.88:0/4051280878 shutdown_connections 2023-10-16T20:06:22.350 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.343+0000 7f5d18f35640 1 --2- 172.21.15.88:0/4051280878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d1413f730 0x7f5d140cc120 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:22.350 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.343+0000 7f5d18f35640 1 -- 172.21.15.88:0/4051280878 >> 172.21.15.88:0/4051280878 conn(0x7f5d140ce5e0 msgr2=0x7f5d140ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:22.351 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.343+0000 7f5d18f35640 1 -- 172.21.15.88:0/4051280878 shutdown_connections 2023-10-16T20:06:22.351 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.343+0000 7f5d18f35640 1 -- 172.21.15.88:0/4051280878 wait complete. 2023-10-16T20:06:22.352 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.347+0000 7f5d18f35640 1 Processor -- start 2023-10-16T20:06:22.353 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.347+0000 7f5d18f35640 1 -- start start 2023-10-16T20:06:22.353 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.347+0000 7f5d18f35640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d1406a5a0 0x7f5d140afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:22.353 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.347+0000 7f5d18f35640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5d1413f730 0x7f5d1413ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:22.353 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.347+0000 7f5d18f35640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d1413fb10 0x7f5d1413c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:22.353 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.347+0000 7f5d18f35640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d14141740 con 0x7f5d1406a5a0 2023-10-16T20:06:22.354 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.347+0000 7f5d18f35640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d14140ac0 con 0x7f5d1413fb10 2023-10-16T20:06:22.354 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.347+0000 7f5d18f35640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d14140da0 con 0x7f5d1413f730 2023-10-16T20:06:22.354 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.347+0000 7f5d1259c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d1406a5a0 0x7f5d140afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:22.354 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.347+0000 7f5d1259c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d1406a5a0 0x7f5d140afed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57540/0 (socket says 172.21.15.88:57540) 2023-10-16T20:06:22.354 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.347+0000 7f5d1259c640 1 -- 172.21.15.88:0/1743989853 learned_addr learned my addr 172.21.15.88:0/1743989853 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:22.355 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.347+0000 7f5d11d9b640 1 --2- 172.21.15.88:0/1743989853 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5d1413f730 0x7f5d1413ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:22.355 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.347+0000 7f5d12d9d640 1 --2- 172.21.15.88:0/1743989853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d1413fb10 0x7f5d1413c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:22.355 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.347+0000 7f5d1259c640 1 -- 172.21.15.88:0/1743989853 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5d1413f730 msgr2=0x7f5d1413ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:22.355 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.347+0000 7f5d1259c640 1 --2- 172.21.15.88:0/1743989853 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5d1413f730 0x7f5d1413ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:22.355 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.347+0000 7f5d1259c640 1 -- 172.21.15.88:0/1743989853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d1413fb10 msgr2=0x7f5d1413c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:22.356 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.347+0000 7f5d1259c640 1 --2- 172.21.15.88:0/1743989853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d1413fb10 0x7f5d1413c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:22.356 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.347+0000 7f5d1259c640 1 -- 172.21.15.88:0/1743989853 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d040094d0 con 0x7f5d1406a5a0 2023-10-16T20:06:22.356 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.347+0000 7f5d1259c640 1 --2- 172.21.15.88:0/1743989853 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d1406a5a0 0x7f5d140afed0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5d040ad040 tx=0x7f5d0401ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:22.357 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.347+0000 7f5d02ffd640 1 -- 172.21.15.88:0/1743989853 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d040be5f0 con 0x7f5d1406a5a0 2023-10-16T20:06:22.357 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.347+0000 7f5d02ffd640 1 -- 172.21.15.88:0/1743989853 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d040be770 con 0x7f5d1406a5a0 2023-10-16T20:06:22.358 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.347+0000 7f5d02ffd640 1 -- 172.21.15.88:0/1743989853 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d040beb20 con 0x7f5d1406a5a0 2023-10-16T20:06:22.358 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.347+0000 7f5d18f35640 1 -- 172.21.15.88:0/1743989853 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d14141080 con 0x7f5d1406a5a0 2023-10-16T20:06:22.358 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.347+0000 7f5d18f35640 1 -- 172.21.15.88:0/1743989853 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d14137de0 con 0x7f5d1406a5a0 2023-10-16T20:06:22.358 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.347+0000 7f5d18f35640 1 -- 172.21.15.88:0/1743989853 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d1404e9c0 con 0x7f5d1406a5a0 2023-10-16T20:06:22.358 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.351+0000 7f5d02ffd640 1 -- 172.21.15.88:0/1743989853 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5d040b43f0 con 0x7f5d1406a5a0 2023-10-16T20:06:22.359 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.351+0000 7f5d02ffd640 1 --2- 172.21.15.88:0/1743989853 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5cf40664a0 0x7f5cf4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:22.359 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.351+0000 7f5d02ffd640 1 -- 172.21.15.88:0/1743989853 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f5d0411ec50 con 0x7f5d1406a5a0 2023-10-16T20:06:22.359 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.351+0000 7f5d11d9b640 1 --2- 172.21.15.88:0/1743989853 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5cf40664a0 0x7f5cf4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:22.359 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.351+0000 7f5d11d9b640 1 --2- 172.21.15.88:0/1743989853 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5cf40664a0 0x7f5cf4068960 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5d1413b700 tx=0x7f5d08007bc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:22.360 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.355+0000 7f5d02ffd640 1 -- 172.21.15.88:0/1743989853 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5d040ebf80 con 0x7f5d1406a5a0 2023-10-16T20:06:22.511 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.507+0000 7f5d18f35640 1 -- 172.21.15.88:0/1743989853 --> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5d14069e90 con 0x7f5cf40664a0 2023-10-16T20:06:22.518 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.511+0000 7f5d02ffd640 1 -- 172.21.15.88:0/1743989853 <== mgr.4110 v2:172.21.15.88:6832/4078483333 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289511 (secure 0 0 0) 0x7f5d14069e90 con 0x7f5cf40664a0 2023-10-16T20:06:22.521 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:06:22.523 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2023-10-16T20:06:22.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.515+0000 7f5d18f35640 1 -- 172.21.15.88:0/1743989853 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5cf40664a0 msgr2=0x7f5cf4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:22.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.515+0000 7f5d18f35640 1 --2- 172.21.15.88:0/1743989853 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5cf40664a0 0x7f5cf4068960 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5d1413b700 tx=0x7f5d08007bc0 comp rx=0 tx=0).stop 2023-10-16T20:06:22.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.515+0000 7f5d18f35640 1 -- 172.21.15.88:0/1743989853 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d1406a5a0 msgr2=0x7f5d140afed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:22.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.515+0000 7f5d18f35640 1 --2- 172.21.15.88:0/1743989853 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d1406a5a0 0x7f5d140afed0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5d040ad040 tx=0x7f5d0401ad50 comp rx=0 tx=0).stop 2023-10-16T20:06:22.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.515+0000 7f5d18f35640 1 -- 172.21.15.88:0/1743989853 shutdown_connections 2023-10-16T20:06:22.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.515+0000 7f5d18f35640 1 --2- 172.21.15.88:0/1743989853 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5cf40664a0 0x7f5cf4068960 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:22.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.515+0000 7f5d18f35640 1 --2- 172.21.15.88:0/1743989853 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5d1413f730 0x7f5d1413ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:22.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.515+0000 7f5d18f35640 1 --2- 172.21.15.88:0/1743989853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d1413fb10 0x7f5d1413c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:22.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.515+0000 7f5d18f35640 1 --2- 172.21.15.88:0/1743989853 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d1406a5a0 0x7f5d140afed0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:22.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.515+0000 7f5d18f35640 1 -- 172.21.15.88:0/1743989853 >> 172.21.15.88:0/1743989853 conn(0x7f5d140ce5e0 msgr2=0x7f5d1415e550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:22.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.515+0000 7f5d18f35640 1 -- 172.21.15.88:0/1743989853 shutdown_connections 2023-10-16T20:06:22.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.515+0000 7f5d18f35640 1 -- 172.21.15.88:0/1743989853 wait complete. 2023-10-16T20:06:22.541 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2023-10-16T20:06:22.462831+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233448,"kb_used_data":4752,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754741272,"statfs":{"total":773094113280,"available":772855062528,"internally_reserved":0,"allocated":4866048,"data_stored":2732341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":234016313},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001270"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.220769+0000","last_change":"2023-10-16T20:06:17.217901+0000","last_active":"2023-10-16T20:06:17.220769+0000","last_peered":"2023-10-16T20:06:17.220769+0000","last_clean":"2023-10-16T20:06:17.220769+0000","last_became_active":"2023-10-16T20:06:17.217583+0000","last_became_peered":"2023-10-16T20:06:17.217583+0000","last_unstale":"2023-10-16T20:06:17.220769+0000","last_undegraded":"2023-10-16T20:06:17.220769+0000","last_fullsized":"2023-10-16T20:06:17.220769+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:24:29.941143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.215707+0000","last_change":"2023-10-16T20:06:17.211493+0000","last_active":"2023-10-16T20:06:17.215707+0000","last_peered":"2023-10-16T20:06:17.215707+0000","last_clean":"2023-10-16T20:06:17.215707+0000","last_became_active":"2023-10-16T20:06:17.211196+0000","last_became_peered":"2023-10-16T20:06:17.211196+0000","last_unstale":"2023-10-16T20:06:17.215707+0000","last_undegraded":"2023-10-16T20:06:17.215707+0000","last_fullsized":"2023-10-16T20:06:17.215707+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:50:32.312772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198456+0000","last_change":"2023-10-16T20:06:17.211314+0000","last_active":"2023-10-16T20:06:18.198456+0000","last_peered":"2023-10-16T20:06:18.198456+0000","last_clean":"2023-10-16T20:06:18.198456+0000","last_became_active":"2023-10-16T20:06:17.210991+0000","last_became_peered":"2023-10-16T20:06:17.210991+0000","last_unstale":"2023-10-16T20:06:18.198456+0000","last_undegraded":"2023-10-16T20:06:18.198456+0000","last_fullsized":"2023-10-16T20:06:18.198456+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:07:43.835803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357788+0000","last_change":"2023-10-16T20:06:17.216483+0000","last_active":"2023-10-16T20:06:18.357788+0000","last_peered":"2023-10-16T20:06:18.357788+0000","last_clean":"2023-10-16T20:06:18.357788+0000","last_became_active":"2023-10-16T20:06:17.216202+0000","last_became_peered":"2023-10-16T20:06:17.216202+0000","last_unstale":"2023-10-16T20:06:18.357788+0000","last_undegraded":"2023-10-16T20:06:18.357788+0000","last_fullsized":"2023-10-16T20:06:18.357788+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:32:02.453729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357619+0000","last_change":"2023-10-16T20:06:17.208325+0000","last_active":"2023-10-16T20:06:18.357619+0000","last_peered":"2023-10-16T20:06:18.357619+0000","last_clean":"2023-10-16T20:06:18.357619+0000","last_became_active":"2023-10-16T20:06:17.208061+0000","last_became_peered":"2023-10-16T20:06:17.208061+0000","last_unstale":"2023-10-16T20:06:18.357619+0000","last_undegraded":"2023-10-16T20:06:18.357619+0000","last_fullsized":"2023-10-16T20:06:18.357619+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:02:28.165753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.217274+0000","last_change":"2023-10-16T20:06:17.208416+0000","last_active":"2023-10-16T20:06:17.217274+0000","last_peered":"2023-10-16T20:06:17.217274+0000","last_clean":"2023-10-16T20:06:17.217274+0000","last_became_active":"2023-10-16T20:06:17.207713+0000","last_became_peered":"2023-10-16T20:06:17.207713+0000","last_unstale":"2023-10-16T20:06:17.217274+0000","last_undegraded":"2023-10-16T20:06:17.217274+0000","last_fullsized":"2023-10-16T20:06:17.217274+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:39:06.086998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.213838+0000","last_change":"2023-10-16T20:06:17.206736+0000","last_active":"2023-10-16T20:06:17.213838+0000","last_peered":"2023-10-16T20:06:17.213838+0000","last_clean":"2023-10-16T20:06:17.213838+0000","last_became_active":"2023-10-16T20:06:17.206435+0000","last_became_peered":"2023-10-16T20:06:17.206435+0000","last_unstale":"2023-10-16T20:06:17.213838+0000","last_undegraded":"2023-10-16T20:06:17.213838+0000","last_fullsized":"2023-10-16T20:06:17.213838+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:50:32.312772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.218213+0000","last_change":"2023-10-16T20:06:17.214147+0000","last_active":"2023-10-16T20:06:17.218213+0000","last_peered":"2023-10-16T20:06:17.218213+0000","last_clean":"2023-10-16T20:06:17.218213+0000","last_became_active":"2023-10-16T20:06:17.213998+0000","last_became_peered":"2023-10-16T20:06:17.213998+0000","last_unstale":"2023-10-16T20:06:17.218213+0000","last_undegraded":"2023-10-16T20:06:17.218213+0000","last_fullsized":"2023-10-16T20:06:17.218213+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:36:27.881926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198636+0000","last_change":"2023-10-16T20:06:17.213204+0000","last_active":"2023-10-16T20:06:18.198636+0000","last_peered":"2023-10-16T20:06:18.198636+0000","last_clean":"2023-10-16T20:06:18.198636+0000","last_became_active":"2023-10-16T20:06:17.212563+0000","last_became_peered":"2023-10-16T20:06:17.212563+0000","last_unstale":"2023-10-16T20:06:18.198636+0000","last_undegraded":"2023-10-16T20:06:18.198636+0000","last_fullsized":"2023-10-16T20:06:18.198636+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:25:04.009896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.222492+0000","last_change":"2023-10-16T20:06:17.218989+0000","last_active":"2023-10-16T20:06:17.222492+0000","last_peered":"2023-10-16T20:06:17.222492+0000","last_clean":"2023-10-16T20:06:17.222492+0000","last_became_active":"2023-10-16T20:06:17.218726+0000","last_became_peered":"2023-10-16T20:06:17.218726+0000","last_unstale":"2023-10-16T20:06:17.222492+0000","last_undegraded":"2023-10-16T20:06:17.222492+0000","last_fullsized":"2023-10-16T20:06:17.222492+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:27:39.178553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.221629+0000","last_change":"2023-10-16T20:06:17.217372+0000","last_active":"2023-10-16T20:06:17.221629+0000","last_peered":"2023-10-16T20:06:17.221629+0000","last_clean":"2023-10-16T20:06:17.221629+0000","last_became_active":"2023-10-16T20:06:17.216730+0000","last_became_peered":"2023-10-16T20:06:17.216730+0000","last_unstale":"2023-10-16T20:06:17.221629+0000","last_undegraded":"2023-10-16T20:06:17.221629+0000","last_fullsized":"2023-10-16T20:06:17.221629+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:18:14.480179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198265+0000","last_change":"2023-10-16T20:06:17.214475+0000","last_active":"2023-10-16T20:06:18.198265+0000","last_peered":"2023-10-16T20:06:18.198265+0000","last_clean":"2023-10-16T20:06:18.198265+0000","last_became_active":"2023-10-16T20:06:17.214211+0000","last_became_peered":"2023-10-16T20:06:17.214211+0000","last_unstale":"2023-10-16T20:06:18.198265+0000","last_undegraded":"2023-10-16T20:06:18.198265+0000","last_fullsized":"2023-10-16T20:06:18.198265+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:02:29.473591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.221752+0000","last_change":"2023-10-16T20:06:17.218455+0000","last_active":"2023-10-16T20:06:17.221752+0000","last_peered":"2023-10-16T20:06:17.221752+0000","last_clean":"2023-10-16T20:06:17.221752+0000","last_became_active":"2023-10-16T20:06:17.217565+0000","last_became_peered":"2023-10-16T20:06:17.217565+0000","last_unstale":"2023-10-16T20:06:17.221752+0000","last_undegraded":"2023-10-16T20:06:17.221752+0000","last_fullsized":"2023-10-16T20:06:17.221752+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:13.874932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.222220+0000","last_change":"2023-10-16T20:06:17.214285+0000","last_active":"2023-10-16T20:06:17.222220+0000","last_peered":"2023-10-16T20:06:17.222220+0000","last_clean":"2023-10-16T20:06:17.222220+0000","last_became_active":"2023-10-16T20:06:17.213551+0000","last_became_peered":"2023-10-16T20:06:17.213551+0000","last_unstale":"2023-10-16T20:06:17.222220+0000","last_undegraded":"2023-10-16T20:06:17.222220+0000","last_fullsized":"2023-10-16T20:06:17.222220+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:10:43.145100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.213855+0000","last_change":"2023-10-16T20:06:17.208025+0000","last_active":"2023-10-16T20:06:17.213855+0000","last_peered":"2023-10-16T20:06:17.213855+0000","last_clean":"2023-10-16T20:06:17.213855+0000","last_became_active":"2023-10-16T20:06:17.207739+0000","last_became_peered":"2023-10-16T20:06:17.207739+0000","last_unstale":"2023-10-16T20:06:17.213855+0000","last_undegraded":"2023-10-16T20:06:17.213855+0000","last_fullsized":"2023-10-16T20:06:17.213855+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:36:27.881926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196502+0000","last_change":"2023-10-16T20:06:17.222232+0000","last_active":"2023-10-16T20:06:18.196502+0000","last_peered":"2023-10-16T20:06:18.196502+0000","last_clean":"2023-10-16T20:06:18.196502+0000","last_became_active":"2023-10-16T20:06:17.221892+0000","last_became_peered":"2023-10-16T20:06:17.221892+0000","last_unstale":"2023-10-16T20:06:18.196502+0000","last_undegraded":"2023-10-16T20:06:18.196502+0000","last_fullsized":"2023-10-16T20:06:18.196502+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:58:45.132214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197149+0000","last_change":"2023-10-16T20:06:17.224446+0000","last_active":"2023-10-16T20:06:18.197149+0000","last_peered":"2023-10-16T20:06:18.197149+0000","last_clean":"2023-10-16T20:06:18.197149+0000","last_became_active":"2023-10-16T20:06:17.224113+0000","last_became_peered":"2023-10-16T20:06:17.224113+0000","last_unstale":"2023-10-16T20:06:18.197149+0000","last_undegraded":"2023-10-16T20:06:18.197149+0000","last_fullsized":"2023-10-16T20:06:18.197149+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:37:43.948080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198392+0000","last_change":"2023-10-16T20:06:17.214181+0000","last_active":"2023-10-16T20:06:18.198392+0000","last_peered":"2023-10-16T20:06:18.198392+0000","last_clean":"2023-10-16T20:06:18.198392+0000","last_became_active":"2023-10-16T20:06:17.213876+0000","last_became_peered":"2023-10-16T20:06:17.213876+0000","last_unstale":"2023-10-16T20:06:18.198392+0000","last_undegraded":"2023-10-16T20:06:18.198392+0000","last_fullsized":"2023-10-16T20:06:18.198392+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:39:06.086998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357861+0000","last_change":"2023-10-16T20:06:17.205826+0000","last_active":"2023-10-16T20:06:18.357861+0000","last_peered":"2023-10-16T20:06:18.357861+0000","last_clean":"2023-10-16T20:06:18.357861+0000","last_became_active":"2023-10-16T20:06:17.204888+0000","last_became_peered":"2023-10-16T20:06:17.204888+0000","last_unstale":"2023-10-16T20:06:18.357861+0000","last_undegraded":"2023-10-16T20:06:18.357861+0000","last_fullsized":"2023-10-16T20:06:18.357861+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:29:06.675309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197777+0000","last_change":"2023-10-16T20:06:17.217103+0000","last_active":"2023-10-16T20:06:18.197777+0000","last_peered":"2023-10-16T20:06:18.197777+0000","last_clean":"2023-10-16T20:06:18.197777+0000","last_became_active":"2023-10-16T20:06:17.216717+0000","last_became_peered":"2023-10-16T20:06:17.216717+0000","last_unstale":"2023-10-16T20:06:18.197777+0000","last_undegraded":"2023-10-16T20:06:18.197777+0000","last_fullsized":"2023-10-16T20:06:18.197777+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:20:06.934366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.206185+0000","last_change":"2023-10-16T20:06:17.201875+0000","last_active":"2023-10-16T20:06:17.206185+0000","last_peered":"2023-10-16T20:06:17.206185+0000","last_clean":"2023-10-16T20:06:17.206185+0000","last_became_active":"2023-10-16T20:06:17.200537+0000","last_became_peered":"2023-10-16T20:06:17.200537+0000","last_unstale":"2023-10-16T20:06:17.206185+0000","last_undegraded":"2023-10-16T20:06:17.206185+0000","last_fullsized":"2023-10-16T20:06:17.206185+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:23:28.625212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.216225+0000","last_change":"2023-10-16T20:06:17.211432+0000","last_active":"2023-10-16T20:06:17.216225+0000","last_peered":"2023-10-16T20:06:17.216225+0000","last_clean":"2023-10-16T20:06:17.216225+0000","last_became_active":"2023-10-16T20:06:17.211100+0000","last_became_peered":"2023-10-16T20:06:17.211100+0000","last_unstale":"2023-10-16T20:06:17.216225+0000","last_undegraded":"2023-10-16T20:06:17.216225+0000","last_fullsized":"2023-10-16T20:06:17.216225+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:06:45.516915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196828+0000","last_change":"2023-10-16T20:06:17.225476+0000","last_active":"2023-10-16T20:06:18.196828+0000","last_peered":"2023-10-16T20:06:18.196828+0000","last_clean":"2023-10-16T20:06:18.196828+0000","last_became_active":"2023-10-16T20:06:17.224667+0000","last_became_peered":"2023-10-16T20:06:17.224667+0000","last_unstale":"2023-10-16T20:06:18.196828+0000","last_undegraded":"2023-10-16T20:06:18.196828+0000","last_fullsized":"2023-10-16T20:06:18.196828+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:31:31.346356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357610+0000","last_change":"2023-10-16T20:06:17.216704+0000","last_active":"2023-10-16T20:06:18.357610+0000","last_peered":"2023-10-16T20:06:18.357610+0000","last_clean":"2023-10-16T20:06:18.357610+0000","last_became_active":"2023-10-16T20:06:17.216341+0000","last_became_peered":"2023-10-16T20:06:17.216341+0000","last_unstale":"2023-10-16T20:06:18.357610+0000","last_undegraded":"2023-10-16T20:06:18.357610+0000","last_fullsized":"2023-10-16T20:06:18.357610+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:14:46.135352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198413+0000","last_change":"2023-10-16T20:06:17.216103+0000","last_active":"2023-10-16T20:06:18.198413+0000","last_peered":"2023-10-16T20:06:18.198413+0000","last_clean":"2023-10-16T20:06:18.198413+0000","last_became_active":"2023-10-16T20:06:17.215844+0000","last_became_peered":"2023-10-16T20:06:17.215844+0000","last_unstale":"2023-10-16T20:06:18.198413+0000","last_undegraded":"2023-10-16T20:06:18.198413+0000","last_fullsized":"2023-10-16T20:06:18.198413+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:01.646291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.216083+0000","last_change":"2023-10-16T20:06:17.207691+0000","last_active":"2023-10-16T20:06:17.216083+0000","last_peered":"2023-10-16T20:06:17.216083+0000","last_clean":"2023-10-16T20:06:17.216083+0000","last_became_active":"2023-10-16T20:06:17.207437+0000","last_became_peered":"2023-10-16T20:06:17.207437+0000","last_unstale":"2023-10-16T20:06:17.216083+0000","last_undegraded":"2023-10-16T20:06:17.216083+0000","last_fullsized":"2023-10-16T20:06:17.216083+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:23:28.625212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198320+0000","last_change":"2023-10-16T20:06:17.211359+0000","last_active":"2023-10-16T20:06:18.198320+0000","last_peered":"2023-10-16T20:06:18.198320+0000","last_clean":"2023-10-16T20:06:18.198320+0000","last_became_active":"2023-10-16T20:06:17.210506+0000","last_became_peered":"2023-10-16T20:06:17.210506+0000","last_unstale":"2023-10-16T20:06:18.198320+0000","last_undegraded":"2023-10-16T20:06:18.198320+0000","last_fullsized":"2023-10-16T20:06:18.198320+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:32:44.093091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.222529+0000","last_change":"2023-10-16T20:06:17.219679+0000","last_active":"2023-10-16T20:06:17.222529+0000","last_peered":"2023-10-16T20:06:17.222529+0000","last_clean":"2023-10-16T20:06:17.222529+0000","last_became_active":"2023-10-16T20:06:17.214175+0000","last_became_peered":"2023-10-16T20:06:17.214175+0000","last_unstale":"2023-10-16T20:06:17.222529+0000","last_undegraded":"2023-10-16T20:06:17.222529+0000","last_fullsized":"2023-10-16T20:06:17.222529+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:22:32.981812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197630+0000","last_change":"2023-10-16T20:06:17.210945+0000","last_active":"2023-10-16T20:06:18.197630+0000","last_peered":"2023-10-16T20:06:18.197630+0000","last_clean":"2023-10-16T20:06:18.197630+0000","last_became_active":"2023-10-16T20:06:17.210629+0000","last_became_peered":"2023-10-16T20:06:17.210629+0000","last_unstale":"2023-10-16T20:06:18.197630+0000","last_undegraded":"2023-10-16T20:06:18.197630+0000","last_fullsized":"2023-10-16T20:06:18.197630+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:27:10.715147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197643+0000","last_change":"2023-10-16T20:06:17.212963+0000","last_active":"2023-10-16T20:06:18.197643+0000","last_peered":"2023-10-16T20:06:18.197643+0000","last_clean":"2023-10-16T20:06:18.197643+0000","last_became_active":"2023-10-16T20:06:17.212693+0000","last_became_peered":"2023-10-16T20:06:17.212693+0000","last_unstale":"2023-10-16T20:06:18.197643+0000","last_undegraded":"2023-10-16T20:06:18.197643+0000","last_fullsized":"2023-10-16T20:06:18.197643+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:10:43.145100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197558+0000","last_change":"2023-10-16T20:06:17.219075+0000","last_active":"2023-10-16T20:06:18.197558+0000","last_peered":"2023-10-16T20:06:18.197558+0000","last_clean":"2023-10-16T20:06:18.197558+0000","last_became_active":"2023-10-16T20:06:17.218774+0000","last_became_peered":"2023-10-16T20:06:17.218774+0000","last_unstale":"2023-10-16T20:06:18.197558+0000","last_undegraded":"2023-10-16T20:06:18.197558+0000","last_fullsized":"2023-10-16T20:06:18.197558+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:05:25.874296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.209799+0000","last_change":"2023-10-16T20:06:17.200644+0000","last_active":"2023-10-16T20:06:17.209799+0000","last_peered":"2023-10-16T20:06:17.209799+0000","last_clean":"2023-10-16T20:06:17.209799+0000","last_became_active":"2023-10-16T20:06:17.200350+0000","last_became_peered":"2023-10-16T20:06:17.200350+0000","last_unstale":"2023-10-16T20:06:17.209799+0000","last_undegraded":"2023-10-16T20:06:17.209799+0000","last_fullsized":"2023-10-16T20:06:17.209799+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:58:21.190657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196740+0000","last_change":"2023-10-16T20:06:17.224294+0000","last_active":"2023-10-16T20:06:18.196740+0000","last_peered":"2023-10-16T20:06:18.196740+0000","last_clean":"2023-10-16T20:06:18.196740+0000","last_became_active":"2023-10-16T20:06:17.223988+0000","last_became_peered":"2023-10-16T20:06:17.223988+0000","last_unstale":"2023-10-16T20:06:18.196740+0000","last_undegraded":"2023-10-16T20:06:18.196740+0000","last_fullsized":"2023-10-16T20:06:18.196740+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:03:40.257975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"40'4","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198438+0000","last_change":"2023-10-16T20:06:16.732366+0000","last_active":"2023-10-16T20:06:18.198438+0000","last_peered":"2023-10-16T20:06:18.198438+0000","last_clean":"2023-10-16T20:06:18.198438+0000","last_became_active":"2023-10-16T20:06:16.732035+0000","last_became_peered":"2023-10-16T20:06:16.732035+0000","last_unstale":"2023-10-16T20:06:18.198438+0000","last_undegraded":"2023-10-16T20:06:18.198438+0000","last_fullsized":"2023-10-16T20:06:18.198438+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:35:43.770049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.216156+0000","last_change":"2023-10-16T20:06:17.207721+0000","last_active":"2023-10-16T20:06:17.216156+0000","last_peered":"2023-10-16T20:06:17.216156+0000","last_clean":"2023-10-16T20:06:17.216156+0000","last_became_active":"2023-10-16T20:06:17.207470+0000","last_became_peered":"2023-10-16T20:06:17.207470+0000","last_unstale":"2023-10-16T20:06:17.216156+0000","last_undegraded":"2023-10-16T20:06:17.216156+0000","last_fullsized":"2023-10-16T20:06:17.216156+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:56:27.147860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"40'1","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198456+0000","last_change":"2023-10-16T20:06:16.730576+0000","last_active":"2023-10-16T20:06:18.198456+0000","last_peered":"2023-10-16T20:06:18.198456+0000","last_clean":"2023-10-16T20:06:18.198456+0000","last_became_active":"2023-10-16T20:06:16.729963+0000","last_became_peered":"2023-10-16T20:06:16.729963+0000","last_unstale":"2023-10-16T20:06:18.198456+0000","last_undegraded":"2023-10-16T20:06:18.198456+0000","last_fullsized":"2023-10-16T20:06:18.198456+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:51:10.650711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.222274+0000","last_change":"2023-10-16T20:06:17.215735+0000","last_active":"2023-10-16T20:06:17.222274+0000","last_peered":"2023-10-16T20:06:17.222274+0000","last_clean":"2023-10-16T20:06:17.222274+0000","last_became_active":"2023-10-16T20:06:17.214986+0000","last_became_peered":"2023-10-16T20:06:17.214986+0000","last_unstale":"2023-10-16T20:06:17.222274+0000","last_undegraded":"2023-10-16T20:06:17.222274+0000","last_fullsized":"2023-10-16T20:06:17.222274+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:35:44.780849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"40'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.733064+0000","last_change":"2023-10-16T20:06:16.739387+0000","last_active":"2023-10-16T20:06:17.733064+0000","last_peered":"2023-10-16T20:06:17.733064+0000","last_clean":"2023-10-16T20:06:17.733064+0000","last_became_active":"2023-10-16T20:06:16.739070+0000","last_became_peered":"2023-10-16T20:06:16.739070+0000","last_unstale":"2023-10-16T20:06:17.733064+0000","last_undegraded":"2023-10-16T20:06:17.733064+0000","last_fullsized":"2023-10-16T20:06:17.733064+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:04:52.152758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.217801+0000","last_change":"2023-10-16T20:06:17.213835+0000","last_active":"2023-10-16T20:06:17.217801+0000","last_peered":"2023-10-16T20:06:17.217801+0000","last_clean":"2023-10-16T20:06:17.217801+0000","last_became_active":"2023-10-16T20:06:17.213516+0000","last_became_peered":"2023-10-16T20:06:17.213516+0000","last_unstale":"2023-10-16T20:06:17.217801+0000","last_undegraded":"2023-10-16T20:06:17.217801+0000","last_fullsized":"2023-10-16T20:06:17.217801+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:25:13.284821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"40'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.732355+0000","last_change":"2023-10-16T20:06:16.730019+0000","last_active":"2023-10-16T20:06:17.732355+0000","last_peered":"2023-10-16T20:06:17.732355+0000","last_clean":"2023-10-16T20:06:17.732355+0000","last_became_active":"2023-10-16T20:06:16.729804+0000","last_became_peered":"2023-10-16T20:06:16.729804+0000","last_unstale":"2023-10-16T20:06:17.732355+0000","last_undegraded":"2023-10-16T20:06:17.732355+0000","last_fullsized":"2023-10-16T20:06:17.732355+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:56:26.137061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198084+0000","last_change":"2023-10-16T20:06:17.214073+0000","last_active":"2023-10-16T20:06:18.198084+0000","last_peered":"2023-10-16T20:06:18.198084+0000","last_clean":"2023-10-16T20:06:18.198084+0000","last_became_active":"2023-10-16T20:06:17.213793+0000","last_became_peered":"2023-10-16T20:06:17.213793+0000","last_unstale":"2023-10-16T20:06:18.198084+0000","last_undegraded":"2023-10-16T20:06:18.198084+0000","last_fullsized":"2023-10-16T20:06:18.198084+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:17:45.690804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"40'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.732663+0000","last_change":"2023-10-16T20:06:16.731997+0000","last_active":"2023-10-16T20:06:17.732663+0000","last_peered":"2023-10-16T20:06:17.732663+0000","last_clean":"2023-10-16T20:06:17.732663+0000","last_became_active":"2023-10-16T20:06:16.731714+0000","last_became_peered":"2023-10-16T20:06:16.731714+0000","last_unstale":"2023-10-16T20:06:17.732663+0000","last_undegraded":"2023-10-16T20:06:17.732663+0000","last_fullsized":"2023-10-16T20:06:17.732663+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:25:12.274021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.220705+0000","last_change":"2023-10-16T20:06:17.214298+0000","last_active":"2023-10-16T20:06:17.220705+0000","last_peered":"2023-10-16T20:06:17.220705+0000","last_clean":"2023-10-16T20:06:17.220705+0000","last_became_active":"2023-10-16T20:06:17.214038+0000","last_became_peered":"2023-10-16T20:06:17.214038+0000","last_unstale":"2023-10-16T20:06:17.220705+0000","last_undegraded":"2023-10-16T20:06:17.220705+0000","last_fullsized":"2023-10-16T20:06:17.220705+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:38:40.789443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"40'1","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196763+0000","last_change":"2023-10-16T20:06:16.737162+0000","last_active":"2023-10-16T20:06:18.196763+0000","last_peered":"2023-10-16T20:06:18.196763+0000","last_clean":"2023-10-16T20:06:18.196763+0000","last_became_active":"2023-10-16T20:06:16.736888+0000","last_became_peered":"2023-10-16T20:06:16.736888+0000","last_unstale":"2023-10-16T20:06:18.196763+0000","last_undegraded":"2023-10-16T20:06:18.196763+0000","last_fullsized":"2023-10-16T20:06:18.196763+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:22:31.971013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.211586+0000","last_change":"2023-10-16T20:06:17.208067+0000","last_active":"2023-10-16T20:06:17.211586+0000","last_peered":"2023-10-16T20:06:17.211586+0000","last_clean":"2023-10-16T20:06:17.211586+0000","last_became_active":"2023-10-16T20:06:17.207799+0000","last_became_peered":"2023-10-16T20:06:17.207799+0000","last_unstale":"2023-10-16T20:06:17.211586+0000","last_undegraded":"2023-10-16T20:06:17.211586+0000","last_fullsized":"2023-10-16T20:06:17.211586+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:40:58.934858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.195959+0000","last_change":"2023-10-16T20:06:16.729238+0000","last_active":"2023-10-16T20:06:17.195959+0000","last_peered":"2023-10-16T20:06:17.195959+0000","last_clean":"2023-10-16T20:06:17.195959+0000","last_became_active":"2023-10-16T20:06:16.728975+0000","last_became_peered":"2023-10-16T20:06:16.728975+0000","last_unstale":"2023-10-16T20:06:17.195959+0000","last_undegraded":"2023-10-16T20:06:17.195959+0000","last_fullsized":"2023-10-16T20:06:17.195959+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:36:26.871127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.220833+0000","last_change":"2023-10-16T20:06:17.218801+0000","last_active":"2023-10-16T20:06:17.220833+0000","last_peered":"2023-10-16T20:06:17.220833+0000","last_clean":"2023-10-16T20:06:17.220833+0000","last_became_active":"2023-10-16T20:06:17.218219+0000","last_became_peered":"2023-10-16T20:06:17.218219+0000","last_unstale":"2023-10-16T20:06:17.220833+0000","last_undegraded":"2023-10-16T20:06:17.220833+0000","last_fullsized":"2023-10-16T20:06:17.220833+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:27:39.178553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.202521+0000","last_change":"2023-10-16T20:06:16.720994+0000","last_active":"2023-10-16T20:06:17.202521+0000","last_peered":"2023-10-16T20:06:17.202521+0000","last_clean":"2023-10-16T20:06:17.202521+0000","last_became_active":"2023-10-16T20:06:16.720623+0000","last_became_peered":"2023-10-16T20:06:16.720623+0000","last_unstale":"2023-10-16T20:06:17.202521+0000","last_undegraded":"2023-10-16T20:06:17.202521+0000","last_fullsized":"2023-10-16T20:06:17.202521+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:32:43.082291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357768+0000","last_change":"2023-10-16T20:06:17.216335+0000","last_active":"2023-10-16T20:06:18.357768+0000","last_peered":"2023-10-16T20:06:18.357768+0000","last_clean":"2023-10-16T20:06:18.357768+0000","last_became_active":"2023-10-16T20:06:17.216070+0000","last_became_peered":"2023-10-16T20:06:17.216070+0000","last_unstale":"2023-10-16T20:06:18.357768+0000","last_undegraded":"2023-10-16T20:06:18.357768+0000","last_fullsized":"2023-10-16T20:06:18.357768+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:08:22.444705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.206243+0000","last_change":"2023-10-16T20:06:16.736582+0000","last_active":"2023-10-16T20:06:17.206243+0000","last_peered":"2023-10-16T20:06:17.206243+0000","last_clean":"2023-10-16T20:06:17.206243+0000","last_became_active":"2023-10-16T20:06:16.736322+0000","last_became_peered":"2023-10-16T20:06:16.736322+0000","last_unstale":"2023-10-16T20:06:17.206243+0000","last_undegraded":"2023-10-16T20:06:17.206243+0000","last_fullsized":"2023-10-16T20:06:17.206243+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:38:29.780087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197108+0000","last_change":"2023-10-16T20:06:17.220312+0000","last_active":"2023-10-16T20:06:18.197108+0000","last_peered":"2023-10-16T20:06:18.197108+0000","last_clean":"2023-10-16T20:06:18.197108+0000","last_became_active":"2023-10-16T20:06:17.220033+0000","last_became_peered":"2023-10-16T20:06:17.220033+0000","last_unstale":"2023-10-16T20:06:18.197108+0000","last_undegraded":"2023-10-16T20:06:18.197108+0000","last_fullsized":"2023-10-16T20:06:18.197108+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:00:59.428210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.198151+0000","last_change":"2023-10-16T20:06:16.725615+0000","last_active":"2023-10-16T20:06:17.198151+0000","last_peered":"2023-10-16T20:06:17.198151+0000","last_clean":"2023-10-16T20:06:17.198151+0000","last_became_active":"2023-10-16T20:06:16.725257+0000","last_became_peered":"2023-10-16T20:06:16.725257+0000","last_unstale":"2023-10-16T20:06:17.198151+0000","last_undegraded":"2023-10-16T20:06:17.198151+0000","last_fullsized":"2023-10-16T20:06:17.198151+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:36:24.034397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198481+0000","last_change":"2023-10-16T20:06:17.215733+0000","last_active":"2023-10-16T20:06:18.198481+0000","last_peered":"2023-10-16T20:06:18.198481+0000","last_clean":"2023-10-16T20:06:18.198481+0000","last_became_active":"2023-10-16T20:06:17.215495+0000","last_became_peered":"2023-10-16T20:06:17.215495+0000","last_unstale":"2023-10-16T20:06:18.198481+0000","last_undegraded":"2023-10-16T20:06:18.198481+0000","last_fullsized":"2023-10-16T20:06:18.198481+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:26:53.217625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"40'3","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.733519+0000","last_change":"2023-10-16T20:06:16.734102+0000","last_active":"2023-10-16T20:06:17.733519+0000","last_peered":"2023-10-16T20:06:17.733519+0000","last_clean":"2023-10-16T20:06:17.733519+0000","last_became_active":"2023-10-16T20:06:16.733829+0000","last_became_peered":"2023-10-16T20:06:16.733829+0000","last_unstale":"2023-10-16T20:06:17.733519+0000","last_undegraded":"2023-10-16T20:06:17.733519+0000","last_fullsized":"2023-10-16T20:06:17.733519+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:32:58.366700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.221860+0000","last_change":"2023-10-16T20:06:17.214205+0000","last_active":"2023-10-16T20:06:17.221860+0000","last_peered":"2023-10-16T20:06:17.221860+0000","last_clean":"2023-10-16T20:06:17.221860+0000","last_became_active":"2023-10-16T20:06:17.213261+0000","last_became_peered":"2023-10-16T20:06:17.213261+0000","last_unstale":"2023-10-16T20:06:17.221860+0000","last_undegraded":"2023-10-16T20:06:17.221860+0000","last_fullsized":"2023-10-16T20:06:17.221860+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:20:06.934366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"40'6","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.738226+0000","last_change":"2023-10-16T20:06:16.721479+0000","last_active":"2023-10-16T20:06:17.738226+0000","last_peered":"2023-10-16T20:06:17.738226+0000","last_clean":"2023-10-16T20:06:17.738226+0000","last_became_active":"2023-10-16T20:06:16.721136+0000","last_became_peered":"2023-10-16T20:06:16.721136+0000","last_unstale":"2023-10-16T20:06:17.738226+0000","last_undegraded":"2023-10-16T20:06:17.738226+0000","last_fullsized":"2023-10-16T20:06:17.738226+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:32:58.366700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198082+0000","last_change":"2023-10-16T20:06:17.211029+0000","last_active":"2023-10-16T20:06:18.198082+0000","last_peered":"2023-10-16T20:06:18.198082+0000","last_clean":"2023-10-16T20:06:18.198082+0000","last_became_active":"2023-10-16T20:06:17.210758+0000","last_became_peered":"2023-10-16T20:06:17.210758+0000","last_unstale":"2023-10-16T20:06:18.198082+0000","last_undegraded":"2023-10-16T20:06:18.198082+0000","last_fullsized":"2023-10-16T20:06:18.198082+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:26:41.239758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"40'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.732488+0000","last_change":"2023-10-16T20:06:16.735005+0000","last_active":"2023-10-16T20:06:17.732488+0000","last_peered":"2023-10-16T20:06:17.732488+0000","last_clean":"2023-10-16T20:06:17.732488+0000","last_became_active":"2023-10-16T20:06:16.734752+0000","last_became_peered":"2023-10-16T20:06:16.734752+0000","last_unstale":"2023-10-16T20:06:17.732488+0000","last_undegraded":"2023-10-16T20:06:17.732488+0000","last_fullsized":"2023-10-16T20:06:17.732488+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:21:23.740779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.219739+0000","last_change":"2023-10-16T20:06:17.215993+0000","last_active":"2023-10-16T20:06:17.219739+0000","last_peered":"2023-10-16T20:06:17.219739+0000","last_clean":"2023-10-16T20:06:17.219739+0000","last_became_active":"2023-10-16T20:06:17.215736+0000","last_became_peered":"2023-10-16T20:06:17.215736+0000","last_unstale":"2023-10-16T20:06:17.219739+0000","last_undegraded":"2023-10-16T20:06:17.219739+0000","last_fullsized":"2023-10-16T20:06:17.219739+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:36:25.045197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.195984+0000","last_change":"2023-10-16T20:06:16.733426+0000","last_active":"2023-10-16T20:06:17.195984+0000","last_peered":"2023-10-16T20:06:17.195984+0000","last_clean":"2023-10-16T20:06:17.195984+0000","last_became_active":"2023-10-16T20:06:16.732600+0000","last_became_peered":"2023-10-16T20:06:16.732600+0000","last_unstale":"2023-10-16T20:06:17.195984+0000","last_undegraded":"2023-10-16T20:06:17.195984+0000","last_fullsized":"2023-10-16T20:06:17.195984+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:12.864132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196653+0000","last_change":"2023-10-16T20:06:17.219925+0000","last_active":"2023-10-16T20:06:18.196653+0000","last_peered":"2023-10-16T20:06:18.196653+0000","last_clean":"2023-10-16T20:06:18.196653+0000","last_became_active":"2023-10-16T20:06:17.219584+0000","last_became_peered":"2023-10-16T20:06:17.219584+0000","last_unstale":"2023-10-16T20:06:18.196653+0000","last_undegraded":"2023-10-16T20:06:18.196653+0000","last_fullsized":"2023-10-16T20:06:18.196653+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:44:14.727872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197391+0000","last_change":"2023-10-16T20:06:16.732499+0000","last_active":"2023-10-16T20:06:18.197391+0000","last_peered":"2023-10-16T20:06:18.197391+0000","last_clean":"2023-10-16T20:06:18.197391+0000","last_became_active":"2023-10-16T20:06:16.732244+0000","last_became_peered":"2023-10-16T20:06:16.732244+0000","last_unstale":"2023-10-16T20:06:18.197391+0000","last_undegraded":"2023-10-16T20:06:18.197391+0000","last_fullsized":"2023-10-16T20:06:18.197391+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:40:57.924059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.219863+0000","last_change":"2023-10-16T20:06:17.215267+0000","last_active":"2023-10-16T20:06:17.219863+0000","last_peered":"2023-10-16T20:06:17.219863+0000","last_clean":"2023-10-16T20:06:17.219863+0000","last_became_active":"2023-10-16T20:06:17.215017+0000","last_became_peered":"2023-10-16T20:06:17.215017+0000","last_unstale":"2023-10-16T20:06:17.219863+0000","last_undegraded":"2023-10-16T20:06:17.219863+0000","last_fullsized":"2023-10-16T20:06:17.219863+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:38:40.789443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"40'8","reported_seq":20,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196540+0000","last_change":"2023-10-16T20:06:16.730165+0000","last_active":"2023-10-16T20:06:18.196540+0000","last_peered":"2023-10-16T20:06:18.196540+0000","last_clean":"2023-10-16T20:06:18.196540+0000","last_became_active":"2023-10-16T20:06:16.729894+0000","last_became_peered":"2023-10-16T20:06:16.729894+0000","last_unstale":"2023-10-16T20:06:18.196540+0000","last_undegraded":"2023-10-16T20:06:18.196540+0000","last_fullsized":"2023-10-16T20:06:18.196540+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:36:26.871127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.217655+0000","last_change":"2023-10-16T20:06:17.214039+0000","last_active":"2023-10-16T20:06:17.217655+0000","last_peered":"2023-10-16T20:06:17.217655+0000","last_clean":"2023-10-16T20:06:17.217655+0000","last_became_active":"2023-10-16T20:06:17.213551+0000","last_became_peered":"2023-10-16T20:06:17.213551+0000","last_unstale":"2023-10-16T20:06:17.217655+0000","last_undegraded":"2023-10-16T20:06:17.217655+0000","last_fullsized":"2023-10-16T20:06:17.217655+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:22:32.981812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"40'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.735204+0000","last_change":"2023-10-16T20:06:16.726366+0000","last_active":"2023-10-16T20:06:17.735204+0000","last_peered":"2023-10-16T20:06:17.735204+0000","last_clean":"2023-10-16T20:06:17.735204+0000","last_became_active":"2023-10-16T20:06:16.726086+0000","last_became_peered":"2023-10-16T20:06:16.726086+0000","last_unstale":"2023-10-16T20:06:17.735204+0000","last_undegraded":"2023-10-16T20:06:17.735204+0000","last_fullsized":"2023-10-16T20:06:17.735204+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:21:23.740779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.216516+0000","last_change":"2023-10-16T20:06:17.207345+0000","last_active":"2023-10-16T20:06:17.216516+0000","last_peered":"2023-10-16T20:06:17.216516+0000","last_clean":"2023-10-16T20:06:17.216516+0000","last_became_active":"2023-10-16T20:06:17.207082+0000","last_became_peered":"2023-10-16T20:06:17.207082+0000","last_unstale":"2023-10-16T20:06:17.216516+0000","last_undegraded":"2023-10-16T20:06:17.216516+0000","last_fullsized":"2023-10-16T20:06:17.216516+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:04:53.163558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"40'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.733912+0000","last_change":"2023-10-16T20:06:16.727808+0000","last_active":"2023-10-16T20:06:17.733912+0000","last_peered":"2023-10-16T20:06:17.733912+0000","last_clean":"2023-10-16T20:06:17.733912+0000","last_became_active":"2023-10-16T20:06:16.727400+0000","last_became_peered":"2023-10-16T20:06:16.727400+0000","last_unstale":"2023-10-16T20:06:17.733912+0000","last_undegraded":"2023-10-16T20:06:17.733912+0000","last_fullsized":"2023-10-16T20:06:17.733912+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:00.635492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.222291+0000","last_change":"2023-10-16T20:06:17.216271+0000","last_active":"2023-10-16T20:06:17.222291+0000","last_peered":"2023-10-16T20:06:17.222291+0000","last_clean":"2023-10-16T20:06:17.222291+0000","last_became_active":"2023-10-16T20:06:17.215201+0000","last_became_peered":"2023-10-16T20:06:17.215201+0000","last_unstale":"2023-10-16T20:06:17.222291+0000","last_undegraded":"2023-10-16T20:06:17.222291+0000","last_fullsized":"2023-10-16T20:06:17.222291+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:59:17.101302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"40'1","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357862+0000","last_change":"2023-10-16T20:06:16.724780+0000","last_active":"2023-10-16T20:06:18.357862+0000","last_peered":"2023-10-16T20:06:18.357862+0000","last_clean":"2023-10-16T20:06:18.357862+0000","last_became_active":"2023-10-16T20:06:16.724476+0000","last_became_peered":"2023-10-16T20:06:16.724476+0000","last_unstale":"2023-10-16T20:06:18.357862+0000","last_undegraded":"2023-10-16T20:06:18.357862+0000","last_fullsized":"2023-10-16T20:06:18.357862+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:19:37.004795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.358046+0000","last_change":"2023-10-16T20:06:17.209520+0000","last_active":"2023-10-16T20:06:18.358046+0000","last_peered":"2023-10-16T20:06:18.358046+0000","last_clean":"2023-10-16T20:06:18.358046+0000","last_became_active":"2023-10-16T20:06:17.209236+0000","last_became_peered":"2023-10-16T20:06:17.209236+0000","last_unstale":"2023-10-16T20:06:18.358046+0000","last_undegraded":"2023-10-16T20:06:18.358046+0000","last_fullsized":"2023-10-16T20:06:18.358046+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:31:26.166083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.196031+0000","last_change":"2023-10-16T20:06:16.734625+0000","last_active":"2023-10-16T20:06:17.196031+0000","last_peered":"2023-10-16T20:06:17.196031+0000","last_clean":"2023-10-16T20:06:17.196031+0000","last_became_active":"2023-10-16T20:06:16.734353+0000","last_became_peered":"2023-10-16T20:06:16.734353+0000","last_unstale":"2023-10-16T20:06:17.196031+0000","last_undegraded":"2023-10-16T20:06:17.196031+0000","last_fullsized":"2023-10-16T20:06:17.196031+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:38:39.778643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196838+0000","last_change":"2023-10-16T20:06:17.219357+0000","last_active":"2023-10-16T20:06:18.196838+0000","last_peered":"2023-10-16T20:06:18.196838+0000","last_clean":"2023-10-16T20:06:18.196838+0000","last_became_active":"2023-10-16T20:06:17.218995+0000","last_became_peered":"2023-10-16T20:06:17.218995+0000","last_unstale":"2023-10-16T20:06:18.196838+0000","last_undegraded":"2023-10-16T20:06:18.196838+0000","last_fullsized":"2023-10-16T20:06:18.196838+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:44:43.545772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"40'2","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357665+0000","last_change":"2023-10-16T20:06:16.732782+0000","last_active":"2023-10-16T20:06:18.357665+0000","last_peered":"2023-10-16T20:06:18.357665+0000","last_clean":"2023-10-16T20:06:18.357665+0000","last_became_active":"2023-10-16T20:06:16.732498+0000","last_became_peered":"2023-10-16T20:06:16.732498+0000","last_unstale":"2023-10-16T20:06:18.357665+0000","last_undegraded":"2023-10-16T20:06:18.357665+0000","last_fullsized":"2023-10-16T20:06:18.357665+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:21:27.852426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198137+0000","last_change":"2023-10-16T20:06:17.221923+0000","last_active":"2023-10-16T20:06:18.198137+0000","last_peered":"2023-10-16T20:06:18.198137+0000","last_clean":"2023-10-16T20:06:18.198137+0000","last_became_active":"2023-10-16T20:06:17.221536+0000","last_became_peered":"2023-10-16T20:06:17.221536+0000","last_unstale":"2023-10-16T20:06:18.198137+0000","last_undegraded":"2023-10-16T20:06:18.198137+0000","last_fullsized":"2023-10-16T20:06:18.198137+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:00:56.525352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357669+0000","last_change":"2023-10-16T20:06:16.732525+0000","last_active":"2023-10-16T20:06:18.357669+0000","last_peered":"2023-10-16T20:06:18.357669+0000","last_clean":"2023-10-16T20:06:18.357669+0000","last_became_active":"2023-10-16T20:06:16.732199+0000","last_became_peered":"2023-10-16T20:06:16.732199+0000","last_unstale":"2023-10-16T20:06:18.357669+0000","last_undegraded":"2023-10-16T20:06:18.357669+0000","last_fullsized":"2023-10-16T20:06:18.357669+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:12:42.667116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.216176+0000","last_change":"2023-10-16T20:06:17.209547+0000","last_active":"2023-10-16T20:06:17.216176+0000","last_peered":"2023-10-16T20:06:17.216176+0000","last_clean":"2023-10-16T20:06:17.216176+0000","last_became_active":"2023-10-16T20:06:17.209281+0000","last_became_peered":"2023-10-16T20:06:17.209281+0000","last_unstale":"2023-10-16T20:06:17.216176+0000","last_undegraded":"2023-10-16T20:06:17.216176+0000","last_fullsized":"2023-10-16T20:06:17.216176+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:58:21.190657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197648+0000","last_change":"2023-10-16T20:06:16.727775+0000","last_active":"2023-10-16T20:06:18.197648+0000","last_peered":"2023-10-16T20:06:18.197648+0000","last_clean":"2023-10-16T20:06:18.197648+0000","last_became_active":"2023-10-16T20:06:16.727492+0000","last_became_peered":"2023-10-16T20:06:16.727492+0000","last_unstale":"2023-10-16T20:06:18.197648+0000","last_undegraded":"2023-10-16T20:06:18.197648+0000","last_fullsized":"2023-10-16T20:06:18.197648+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:32:58.366700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.220851+0000","last_change":"2023-10-16T20:06:17.218801+0000","last_active":"2023-10-16T20:06:17.220851+0000","last_peered":"2023-10-16T20:06:17.220851+0000","last_clean":"2023-10-16T20:06:17.220851+0000","last_became_active":"2023-10-16T20:06:17.218567+0000","last_became_peered":"2023-10-16T20:06:17.218567+0000","last_unstale":"2023-10-16T20:06:17.220851+0000","last_undegraded":"2023-10-16T20:06:17.220851+0000","last_fullsized":"2023-10-16T20:06:17.220851+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:13.874932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357557+0000","last_change":"2023-10-16T20:06:16.727754+0000","last_active":"2023-10-16T20:06:18.357557+0000","last_peered":"2023-10-16T20:06:18.357557+0000","last_clean":"2023-10-16T20:06:18.357557+0000","last_became_active":"2023-10-16T20:06:16.727497+0000","last_became_peered":"2023-10-16T20:06:16.727497+0000","last_unstale":"2023-10-16T20:06:18.357557+0000","last_undegraded":"2023-10-16T20:06:18.357557+0000","last_fullsized":"2023-10-16T20:06:18.357557+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:08:12.647505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.209898+0000","last_change":"2023-10-16T20:06:17.201294+0000","last_active":"2023-10-16T20:06:17.209898+0000","last_peered":"2023-10-16T20:06:17.209898+0000","last_clean":"2023-10-16T20:06:17.209898+0000","last_became_active":"2023-10-16T20:06:17.200853+0000","last_became_peered":"2023-10-16T20:06:17.200853+0000","last_unstale":"2023-10-16T20:06:17.209898+0000","last_undegraded":"2023-10-16T20:06:17.209898+0000","last_fullsized":"2023-10-16T20:06:17.209898+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:01.646291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"40'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.732982+0000","last_change":"2023-10-16T20:06:16.733877+0000","last_active":"2023-10-16T20:06:17.732982+0000","last_peered":"2023-10-16T20:06:17.732982+0000","last_clean":"2023-10-16T20:06:17.732982+0000","last_became_active":"2023-10-16T20:06:16.733570+0000","last_became_peered":"2023-10-16T20:06:16.733570+0000","last_unstale":"2023-10-16T20:06:17.732982+0000","last_undegraded":"2023-10-16T20:06:17.732982+0000","last_fullsized":"2023-10-16T20:06:17.732982+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:38:29.780087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197199+0000","last_change":"2023-10-16T20:06:17.220488+0000","last_active":"2023-10-16T20:06:18.197199+0000","last_peered":"2023-10-16T20:06:18.197199+0000","last_clean":"2023-10-16T20:06:18.197199+0000","last_became_active":"2023-10-16T20:06:17.220060+0000","last_became_peered":"2023-10-16T20:06:17.220060+0000","last_unstale":"2023-10-16T20:06:18.197199+0000","last_undegraded":"2023-10-16T20:06:18.197199+0000","last_fullsized":"2023-10-16T20:06:18.197199+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:59:16.509971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357857+0000","last_change":"2023-10-16T20:06:16.726490+0000","last_active":"2023-10-16T20:06:18.357857+0000","last_peered":"2023-10-16T20:06:18.357857+0000","last_clean":"2023-10-16T20:06:18.357857+0000","last_became_active":"2023-10-16T20:06:16.726210+0000","last_became_peered":"2023-10-16T20:06:16.726210+0000","last_unstale":"2023-10-16T20:06:18.357857+0000","last_undegraded":"2023-10-16T20:06:18.357857+0000","last_fullsized":"2023-10-16T20:06:18.357857+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:13:14.492857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.217565+0000","last_change":"2023-10-16T20:06:17.213630+0000","last_active":"2023-10-16T20:06:17.217565+0000","last_peered":"2023-10-16T20:06:17.217565+0000","last_clean":"2023-10-16T20:06:17.217565+0000","last_became_active":"2023-10-16T20:06:17.213352+0000","last_became_peered":"2023-10-16T20:06:17.213352+0000","last_unstale":"2023-10-16T20:06:17.217565+0000","last_undegraded":"2023-10-16T20:06:17.217565+0000","last_fullsized":"2023-10-16T20:06:17.217565+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:18:14.480179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198443+0000","last_change":"2023-10-16T20:06:16.739495+0000","last_active":"2023-10-16T20:06:18.198443+0000","last_peered":"2023-10-16T20:06:18.198443+0000","last_clean":"2023-10-16T20:06:18.198443+0000","last_became_active":"2023-10-16T20:06:16.739215+0000","last_became_peered":"2023-10-16T20:06:16.739215+0000","last_unstale":"2023-10-16T20:06:18.198443+0000","last_undegraded":"2023-10-16T20:06:18.198443+0000","last_fullsized":"2023-10-16T20:06:18.198443+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:35:40.182424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198226+0000","last_change":"2023-10-16T20:06:17.212113+0000","last_active":"2023-10-16T20:06:18.198226+0000","last_peered":"2023-10-16T20:06:18.198226+0000","last_clean":"2023-10-16T20:06:18.198226+0000","last_became_active":"2023-10-16T20:06:17.211842+0000","last_became_peered":"2023-10-16T20:06:17.211842+0000","last_unstale":"2023-10-16T20:06:18.198226+0000","last_undegraded":"2023-10-16T20:06:18.198226+0000","last_fullsized":"2023-10-16T20:06:18.198226+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:34:14.717626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198360+0000","last_change":"2023-10-16T20:06:16.730618+0000","last_active":"2023-10-16T20:06:18.198360+0000","last_peered":"2023-10-16T20:06:18.198360+0000","last_clean":"2023-10-16T20:06:18.198360+0000","last_became_active":"2023-10-16T20:06:16.729429+0000","last_became_peered":"2023-10-16T20:06:16.729429+0000","last_unstale":"2023-10-16T20:06:18.198360+0000","last_undegraded":"2023-10-16T20:06:18.198360+0000","last_fullsized":"2023-10-16T20:06:18.198360+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:18:42.478975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.216410+0000","last_change":"2023-10-16T20:06:17.208914+0000","last_active":"2023-10-16T20:06:17.216410+0000","last_peered":"2023-10-16T20:06:17.216410+0000","last_clean":"2023-10-16T20:06:17.216410+0000","last_became_active":"2023-10-16T20:06:17.208609+0000","last_became_peered":"2023-10-16T20:06:17.208609+0000","last_unstale":"2023-10-16T20:06:17.216410+0000","last_undegraded":"2023-10-16T20:06:17.216410+0000","last_fullsized":"2023-10-16T20:06:17.216410+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:32:44.093091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197750+0000","last_change":"2023-10-16T20:06:16.733015+0000","last_active":"2023-10-16T20:06:18.197750+0000","last_peered":"2023-10-16T20:06:18.197750+0000","last_clean":"2023-10-16T20:06:18.197750+0000","last_became_active":"2023-10-16T20:06:16.731413+0000","last_became_peered":"2023-10-16T20:06:16.731413+0000","last_unstale":"2023-10-16T20:06:18.197750+0000","last_undegraded":"2023-10-16T20:06:18.197750+0000","last_fullsized":"2023-10-16T20:06:18.197750+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:59:16.090503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.222240+0000","last_change":"2023-10-16T20:06:17.214304+0000","last_active":"2023-10-16T20:06:17.222240+0000","last_peered":"2023-10-16T20:06:17.222240+0000","last_clean":"2023-10-16T20:06:17.222240+0000","last_became_active":"2023-10-16T20:06:17.213521+0000","last_became_peered":"2023-10-16T20:06:17.213521+0000","last_unstale":"2023-10-16T20:06:17.222240+0000","last_undegraded":"2023-10-16T20:06:17.222240+0000","last_fullsized":"2023-10-16T20:06:17.222240+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:27:10.715147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.202689+0000","last_change":"2023-10-16T20:06:16.730112+0000","last_active":"2023-10-16T20:06:17.202689+0000","last_peered":"2023-10-16T20:06:17.202689+0000","last_clean":"2023-10-16T20:06:17.202689+0000","last_became_active":"2023-10-16T20:06:16.729781+0000","last_became_peered":"2023-10-16T20:06:16.729781+0000","last_unstale":"2023-10-16T20:06:17.202689+0000","last_undegraded":"2023-10-16T20:06:17.202689+0000","last_fullsized":"2023-10-16T20:06:17.202689+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:39:05.076198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197008+0000","last_change":"2023-10-16T20:06:17.221997+0000","last_active":"2023-10-16T20:06:18.197008+0000","last_peered":"2023-10-16T20:06:18.197008+0000","last_clean":"2023-10-16T20:06:18.197008+0000","last_became_active":"2023-10-16T20:06:17.221704+0000","last_became_peered":"2023-10-16T20:06:17.221704+0000","last_unstale":"2023-10-16T20:06:18.197008+0000","last_undegraded":"2023-10-16T20:06:18.197008+0000","last_fullsized":"2023-10-16T20:06:18.197008+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:38:22.493819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.201889+0000","last_change":"2023-10-16T20:06:16.733171+0000","last_active":"2023-10-16T20:06:17.201889+0000","last_peered":"2023-10-16T20:06:17.201889+0000","last_clean":"2023-10-16T20:06:17.201889+0000","last_became_active":"2023-10-16T20:06:16.732869+0000","last_became_peered":"2023-10-16T20:06:16.732869+0000","last_unstale":"2023-10-16T20:06:17.201889+0000","last_undegraded":"2023-10-16T20:06:17.201889+0000","last_fullsized":"2023-10-16T20:06:17.201889+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:04:52.152758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.221418+0000","last_change":"2023-10-16T20:06:17.215196+0000","last_active":"2023-10-16T20:06:17.221418+0000","last_peered":"2023-10-16T20:06:17.221418+0000","last_clean":"2023-10-16T20:06:17.221418+0000","last_became_active":"2023-10-16T20:06:17.214556+0000","last_became_peered":"2023-10-16T20:06:17.214556+0000","last_unstale":"2023-10-16T20:06:17.221418+0000","last_undegraded":"2023-10-16T20:06:17.221418+0000","last_fullsized":"2023-10-16T20:06:17.221418+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:24:29.941143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196810+0000","last_change":"2023-10-16T20:06:16.728925+0000","last_active":"2023-10-16T20:06:18.196810+0000","last_peered":"2023-10-16T20:06:18.196810+0000","last_clean":"2023-10-16T20:06:18.196810+0000","last_became_active":"2023-10-16T20:06:16.728663+0000","last_became_peered":"2023-10-16T20:06:16.728663+0000","last_unstale":"2023-10-16T20:06:18.196810+0000","last_undegraded":"2023-10-16T20:06:18.196810+0000","last_fullsized":"2023-10-16T20:06:18.196810+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:50:31.301972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":182,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.208038+0000","last_change":"2023-10-16T20:04:46.335822+0000","last_active":"2023-10-16T20:06:17.208038+0000","last_peered":"2023-10-16T20:06:17.208038+0000","last_clean":"2023-10-16T20:06:17.208038+0000","last_became_active":"2023-10-16T20:04:46.334617+0000","last_became_peered":"2023-10-16T20:04:46.334617+0000","last_unstale":"2023-10-16T20:06:17.208038+0000","last_undegraded":"2023-10-16T20:06:17.208038+0000","last_fullsized":"2023-10-16T20:06:17.208038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:45.306240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:45.306240+0000","last_clean_scrub_stamp":"2023-10-16T20:04:45.306240+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:42:11.153721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509462,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1048576,"data_stored":783264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.86199999999999999}]},{"osd":1,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.4900000000000002,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":0.752}]},{"osd":2,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.0550000000000002,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.80300000000000005}]},{"osd":3,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.629,"5min":2.629,"15min":2.629},"last":0.67400000000000004}]},{"osd":4,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":0.63200000000000001}]},{"osd":5,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.8809999999999998,"5min":2.8809999999999998,"15min":2.8809999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.65400000000000003}]},{"osd":6,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.56999999999999995}]}]},{"osd":6,"up_from":15,"seq":64424509463,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":192896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":4.5880000000000001,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":4.6139999999999999,"5min":4.6139999999999999,"15min":4.6139999999999999},"last":0.629}]},{"osd":1,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":4.6639999999999997,"5min":4.6639999999999997,"15min":4.6639999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":4.8140000000000001,"5min":4.8140000000000001,"15min":4.8140000000000001},"last":0.60899999999999999}]},{"osd":2,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":5.3769999999999998,"5min":5.3769999999999998,"15min":5.3769999999999998},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.79000000000000004}]},{"osd":3,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":4.5460000000000003,"5min":4.5460000000000003,"15min":4.5460000000000003},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.66100000000000003}]},{"osd":4,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.51900000000000002}]},{"osd":5,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.76400000000000001}]},{"osd":7,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.88500000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509461,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":462848,"data_stored":193008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.70999999999999996}]},{"osd":1,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.623},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.76200000000000001}]},{"osd":2,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.747},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.67800000000000005}]},{"osd":3,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.73199999999999998}]},{"osd":4,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.50700000000000001}]},{"osd":6,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.56000000000000005}]},{"osd":7,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.53800000000000003}]}]},{"osd":4,"up_from":15,"seq":64424509461,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31632,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340208,"statfs":{"total":96636764160,"available":96604372992,"internally_reserved":0,"allocated":454656,"data_stored":192896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":3.835,"5min":3.835,"15min":3.835},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":3.7610000000000001,"5min":3.7610000000000001,"15min":3.7610000000000001},"last":0.75700000000000001}]},{"osd":1,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.57599999999999996}]},{"osd":2,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.628},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.60799999999999998}]},{"osd":3,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.72399999999999998}]},{"osd":5,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.5}]},{"osd":6,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.55100000000000005}]},{"osd":7,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.8919999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.74099999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509462,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":194542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.47599999999999998}]},{"osd":1,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.497}]},{"osd":2,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.54200000000000004}]},{"osd":4,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.88200000000000001}]},{"osd":5,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.876}]},{"osd":6,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.95499999999999996}]},{"osd":7,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.75800000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509461,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":192896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.53400000000000003}]},{"osd":1,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.436}]},{"osd":3,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.48399999999999999}]},{"osd":4,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.59699999999999998}]},{"osd":5,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.63}]},{"osd":6,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.80100000000000005}]},{"osd":7,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.70499999999999996}]}]},{"osd":1,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":192896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.867,"5min":3.867,"15min":3.867},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.52700000000000002}]},{"osd":2,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":5.6040000000000001,"5min":5.6040000000000001,"15min":5.6040000000000001},"last":0.434},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":7.8380000000000001,"5min":7.8380000000000001,"15min":7.8380000000000001},"last":0.45100000000000001}]},{"osd":3,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":5.2039999999999997,"5min":5.2039999999999997,"15min":5.2039999999999997},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":3.9470000000000001,"5min":3.9470000000000001,"15min":3.9470000000000001},"last":0.47399999999999998}]},{"osd":4,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":4.1500000000000004,"5min":4.1500000000000004,"15min":4.1500000000000004},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":4.2619999999999996,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":0.79400000000000004}]},{"osd":5,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":4.0910000000000002,"5min":4.0910000000000002,"15min":4.0910000000000002},"last":0.86099999999999999}]},{"osd":6,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":4.21,"5min":4.21,"15min":4.21},"last":0.88},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.83399999999999996}]},{"osd":7,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":4.1849999999999996,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":0.82099999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28072,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343768,"statfs":{"total":96636764160,"available":96608018432,"internally_reserved":0,"allocated":1064960,"data_stored":789943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":3.9289999999999998,"5min":3.9289999999999998,"15min":3.9289999999999998},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":4.0910000000000002,"5min":4.0910000000000002,"15min":4.0910000000000002},"last":0.36899999999999999}]},{"osd":2,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":4.1520000000000001,"5min":4.1520000000000001,"15min":4.1520000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":4.1879999999999997,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.45600000000000002}]},{"osd":3,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":4.3440000000000003,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.438}]},{"osd":4,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":4.5250000000000004,"5min":4.5250000000000004,"15min":4.5250000000000004},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.68300000000000005}]},{"osd":5,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":3.9990000000000001,"5min":3.9990000000000001,"15min":3.9990000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":4.4779999999999998,"5min":4.4779999999999998,"15min":4.4779999999999998},"last":0.70799999999999996}]},{"osd":6,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":3.956,"5min":3.956,"15min":3.956},"last":0.52100000000000002}]},{"osd":7,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":4.0289999999999999,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":4.5640000000000001,"5min":4.5640000000000001,"15min":4.5640000000000001},"last":0.66600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T20:06:22.547 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-16T20:06:22.722 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.715+0000 7f5243965640 1 Processor -- start 2023-10-16T20:06:22.722 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.715+0000 7f5243965640 1 -- start start 2023-10-16T20:06:22.724 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.719+0000 7f5243965640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f523c06a0f0 0x7f523c0c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:22.724 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.719+0000 7f5243965640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f523c1611e0 0x7f523c0ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:22.724 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.719+0000 7f5243965640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f523c165ae0 con 0x7f523c06a4d0 2023-10-16T20:06:22.724 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.719+0000 7f5243965640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f523c0c9c80 con 0x7f523c06a0f0 2023-10-16T20:06:22.725 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.719+0000 7f5243965640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f523c1655f0 con 0x7f523c1611e0 2023-10-16T20:06:22.725 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.719+0000 7f5241701640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f523c06a0f0 0x7f523c0c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:22.725 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.719+0000 7f5240f00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f523c06a4d0 0x7f523c160ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:51138/0 (socket says 172.21.15.88:51138) 2023-10-16T20:06:22.725 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.719+0000 7f5240f00640 1 -- 172.21.15.88:0/2463030588 learned_addr learned my addr 172.21.15.88:0/2463030588 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:22.725 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.719+0000 7f5241701640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f523c06a0f0 0x7f523c0c8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57542/0 (socket says 172.21.15.88:57542) 2023-10-16T20:06:22.726 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.719+0000 7f5241f02640 1 --2- 172.21.15.88:0/2463030588 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f523c1611e0 0x7f523c0ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:22.726 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.719+0000 7f5241701640 1 -- 172.21.15.88:0/2463030588 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f523c1611e0 msgr2=0x7f523c0ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:22.726 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.719+0000 7f5241701640 1 --2- 172.21.15.88:0/2463030588 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f523c1611e0 0x7f523c0ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:22.726 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.719+0000 7f5241701640 1 -- 172.21.15.88:0/2463030588 >> v1:172.21.15.88:6789/0 conn(0x7f523c06a4d0 legacy=0x7f523c160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:22.726 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.719+0000 7f5233fff640 1 -- 172.21.15.88:0/2463030588 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 13573868 0 0) 0x7f523c165ae0 con 0x7f523c06a4d0 2023-10-16T20:06:22.727 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.719+0000 7f5241701640 1 -- 172.21.15.88:0/2463030588 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f523c0c9700 con 0x7f523c06a0f0 2023-10-16T20:06:22.728 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.719+0000 7f5241701640 1 --2- 172.21.15.88:0/2463030588 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f523c06a0f0 0x7f523c0c8c60 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f52380097c0 tx=0x7f523801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747736422b3b34c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:22.728 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.723+0000 7f5233fff640 1 -- 172.21.15.88:0/2463030588 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52380ad660 con 0x7f523c06a0f0 2023-10-16T20:06:22.728 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.723+0000 7f5233fff640 1 -- 172.21.15.88:0/2463030588 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52380ad7c0 con 0x7f523c06a0f0 2023-10-16T20:06:22.729 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.723+0000 7f5233fff640 1 -- 172.21.15.88:0/2463030588 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52380adb50 con 0x7f523c06a0f0 2023-10-16T20:06:22.729 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.723+0000 7f5243965640 1 -- 172.21.15.88:0/2463030588 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f523c06a0f0 msgr2=0x7f523c0c8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:22.729 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.723+0000 7f5243965640 1 --2- 172.21.15.88:0/2463030588 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f523c06a0f0 0x7f523c0c8c60 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f52380097c0 tx=0x7f523801c450 comp rx=0 tx=0).stop 2023-10-16T20:06:22.729 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.723+0000 7f5243965640 1 -- 172.21.15.88:0/2463030588 shutdown_connections 2023-10-16T20:06:22.729 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.723+0000 7f5243965640 1 --2- 172.21.15.88:0/2463030588 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f523c1611e0 0x7f523c0ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:22.730 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.723+0000 7f5243965640 1 --2- 172.21.15.88:0/2463030588 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f523c06a0f0 0x7f523c0c8c60 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:22.730 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.723+0000 7f5243965640 1 -- 172.21.15.88:0/2463030588 >> 172.21.15.88:0/2463030588 conn(0x7f523c0ce610 msgr2=0x7f523c0cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:22.730 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.723+0000 7f5243965640 1 -- 172.21.15.88:0/2463030588 shutdown_connections 2023-10-16T20:06:22.730 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.723+0000 7f5243965640 1 -- 172.21.15.88:0/2463030588 wait complete. 2023-10-16T20:06:22.733 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.723+0000 7f5243965640 1 Processor -- start 2023-10-16T20:06:22.733 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.723+0000 7f5243965640 1 -- start start 2023-10-16T20:06:22.733 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.723+0000 7f5243965640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f523c06a0f0 0x7f523c1568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:22.733 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.723+0000 7f5243965640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f523c06a4d0 0x7f523c0af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:22.734 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.723+0000 7f5243965640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f523c1611e0 0x7f523c0af730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:22.734 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.723+0000 7f5243965640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523c0b3b20 con 0x7f523c06a4d0 2023-10-16T20:06:22.734 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.723+0000 7f5243965640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523c1fcba0 con 0x7f523c1611e0 2023-10-16T20:06:22.734 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.723+0000 7f5243965640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f523c1fd6e0 con 0x7f523c06a0f0 2023-10-16T20:06:22.734 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.723+0000 7f5240f00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f523c06a4d0 0x7f523c0af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:22.735 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.723+0000 7f5240f00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f523c06a4d0 0x7f523c0af1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57554/0 (socket says 172.21.15.88:57554) 2023-10-16T20:06:22.735 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.723+0000 7f5240f00640 1 -- 172.21.15.88:0/1000531798 learned_addr learned my addr 172.21.15.88:0/1000531798 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:22.735 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.723+0000 7f5241f02640 1 --2- 172.21.15.88:0/1000531798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f523c1611e0 0x7f523c0af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:22.735 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.723+0000 7f5241701640 1 --2- 172.21.15.88:0/1000531798 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f523c06a0f0 0x7f523c1568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:22.735 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.727+0000 7f5240f00640 1 -- 172.21.15.88:0/1000531798 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f523c06a0f0 msgr2=0x7f523c1568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:22.736 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.727+0000 7f5240f00640 1 --2- 172.21.15.88:0/1000531798 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f523c06a0f0 0x7f523c1568f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:22.736 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.727+0000 7f5240f00640 1 -- 172.21.15.88:0/1000531798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f523c1611e0 msgr2=0x7f523c0af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:22.736 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.727+0000 7f5240f00640 1 --2- 172.21.15.88:0/1000531798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f523c1611e0 0x7f523c0af730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:22.736 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.727+0000 7f5240f00640 1 -- 172.21.15.88:0/1000531798 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5238009420 con 0x7f523c06a4d0 2023-10-16T20:06:22.736 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.727+0000 7f5240f00640 1 --2- 172.21.15.88:0/1000531798 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f523c06a4d0 0x7f523c0af1f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f522c0011f0 tx=0x7f522c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:22.737 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.727+0000 7f5231ffb640 1 -- 172.21.15.88:0/1000531798 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c0074e0 con 0x7f523c06a4d0 2023-10-16T20:06:22.737 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.727+0000 7f5231ffb640 1 -- 172.21.15.88:0/1000531798 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522c007640 con 0x7f523c06a4d0 2023-10-16T20:06:22.737 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.727+0000 7f5231ffb640 1 -- 172.21.15.88:0/1000531798 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c004a10 con 0x7f523c06a4d0 2023-10-16T20:06:22.737 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.727+0000 7f5243965640 1 -- 172.21.15.88:0/1000531798 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f523c156e30 con 0x7f523c06a4d0 2023-10-16T20:06:22.737 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.727+0000 7f5243965640 1 -- 172.21.15.88:0/1000531798 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f523c1fd090 con 0x7f523c06a4d0 2023-10-16T20:06:22.738 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.727+0000 7f5243965640 1 -- 172.21.15.88:0/1000531798 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f523c04e9c0 con 0x7f523c06a4d0 2023-10-16T20:06:22.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.727+0000 7f5231ffb640 1 -- 172.21.15.88:0/1000531798 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f522c004b70 con 0x7f523c06a4d0 2023-10-16T20:06:22.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.727+0000 7f5231ffb640 1 --2- 172.21.15.88:0/1000531798 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f52200664f0 0x7f52200689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:22.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.727+0000 7f5231ffb640 1 -- 172.21.15.88:0/1000531798 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f522c0835e0 con 0x7f523c06a4d0 2023-10-16T20:06:22.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.727+0000 7f5241701640 1 --2- 172.21.15.88:0/1000531798 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f52200664f0 0x7f52200689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:22.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.731+0000 7f5241701640 1 --2- 172.21.15.88:0/1000531798 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f52200664f0 0x7f52200689b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f52380af000 tx=0x7f52380a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:22.740 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.731+0000 7f5231ffb640 1 -- 172.21.15.88:0/1000531798 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f522c050990 con 0x7f523c06a4d0 2023-10-16T20:06:22.942 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.935+0000 7f5243965640 1 -- 172.21.15.88:0/1000531798 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f523c1fd820 con 0x7f523c06a4d0 2023-10-16T20:06:22.942 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.935+0000 7f5231ffb640 1 -- 172.21.15.88:0/1000531798 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1012 (secure 0 0 0) 0x7f522c053fb0 con 0x7f523c06a4d0 2023-10-16T20:06:22.943 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:06:22.943 INFO:teuthology.orchestra.run.smithi088.stdout:{"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":111,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":41,"num_osds":8,"num_up_osds":8,"osd_up_since":1697486683,"num_in_osds":8,"osd_in_since":1697486673,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592778,"bytes_used":239050752,"bytes_avail":772855062528,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":18,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"b","status":"up:active","gid":5535}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2023-10-16T20:06:02.459165+0000","services":{}},"progress_events":{}} 2023-10-16T20:06:22.945 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.939+0000 7f5243965640 1 -- 172.21.15.88:0/1000531798 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f52200664f0 msgr2=0x7f52200689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:22.945 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.939+0000 7f5243965640 1 --2- 172.21.15.88:0/1000531798 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f52200664f0 0x7f52200689b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f52380af000 tx=0x7f52380a7000 comp rx=0 tx=0).stop 2023-10-16T20:06:22.945 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.939+0000 7f5243965640 1 -- 172.21.15.88:0/1000531798 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f523c06a4d0 msgr2=0x7f523c0af1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:22.945 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.939+0000 7f5243965640 1 --2- 172.21.15.88:0/1000531798 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f523c06a4d0 0x7f523c0af1f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f522c0011f0 tx=0x7f522c00c9b0 comp rx=0 tx=0).stop 2023-10-16T20:06:22.946 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.939+0000 7f5243965640 1 -- 172.21.15.88:0/1000531798 shutdown_connections 2023-10-16T20:06:22.946 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.939+0000 7f5243965640 1 --2- 172.21.15.88:0/1000531798 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f52200664f0 0x7f52200689b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:22.947 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.939+0000 7f5243965640 1 --2- 172.21.15.88:0/1000531798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f523c1611e0 0x7f523c0af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:22.947 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.939+0000 7f5243965640 1 --2- 172.21.15.88:0/1000531798 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f523c06a4d0 0x7f523c0af1f0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:22.947 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.939+0000 7f5243965640 1 --2- 172.21.15.88:0/1000531798 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f523c06a0f0 0x7f523c1568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:22.947 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.939+0000 7f5243965640 1 -- 172.21.15.88:0/1000531798 >> 172.21.15.88:0/1000531798 conn(0x7f523c0ce610 msgr2=0x7f523c164920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:22.947 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.939+0000 7f5243965640 1 -- 172.21.15.88:0/1000531798 shutdown_connections 2023-10-16T20:06:22.948 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:22.939+0000 7f5243965640 1 -- 172.21.15.88:0/1000531798 wait complete. 2023-10-16T20:06:22.963 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-16T20:06:22.964 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T20:06:23.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.095+0000 7f5bcbe91640 1 Processor -- start 2023-10-16T20:06:23.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.095+0000 7f5bcbe91640 1 -- start start 2023-10-16T20:06:23.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.099+0000 7f5bcbe91640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5bc4069e90 0x7f5bc406a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:23.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.099+0000 7f5bcbe91640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bc406a830 con 0x7f5bc41599f0 2023-10-16T20:06:23.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.099+0000 7f5bcbe91640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bc40b1260 con 0x7f5bc40b05a0 2023-10-16T20:06:23.104 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.099+0000 7f5bcbe91640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc40b3140 con 0x7f5bc4069e90 2023-10-16T20:06:23.104 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.099+0000 7f5bc9c2d640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f5bc41599f0 0x7f5bc40b2a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:51144/0 (socket says 172.21.15.88:51144) 2023-10-16T20:06:23.104 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.099+0000 7f5bc9c2d640 1 -- 172.21.15.88:0/1930488521 learned_addr learned my addr 172.21.15.88:0/1930488521 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:23.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.099+0000 7f5bc942c640 1 --2- 172.21.15.88:0/1930488521 >> v2:172.21.15.94:3300/0 conn(0x7f5bc4069e90 0x7f5bc406a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:23.109 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.099+0000 7f5bc8c2b640 1 -- 172.21.15.88:0/1930488521 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 246915280 0 0) 0x7f5bc406a830 con 0x7f5bc41599f0 2023-10-16T20:06:23.109 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.099+0000 7f5bc8c2b640 1 -- 172.21.15.88:0/1930488521 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bb0005740 con 0x7f5bc41599f0 2023-10-16T20:06:23.109 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.099+0000 7f5bc8c2b640 1 -- 172.21.15.88:0/1930488521 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 707961871 0 0) 0x7f5bb0005740 con 0x7f5bc41599f0 2023-10-16T20:06:23.110 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.099+0000 7f5bc8c2b640 1 -- 172.21.15.88:0/1930488521 >> v2:172.21.15.94:3300/0 conn(0x7f5bc4069e90 msgr2=0x7f5bc406a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:23.110 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.099+0000 7f5bc8c2b640 1 --2- 172.21.15.88:0/1930488521 >> v2:172.21.15.94:3300/0 conn(0x7f5bc4069e90 0x7f5bc406a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:23.110 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.099+0000 7f5bc8c2b640 1 -- 172.21.15.88:0/1930488521 >> v1:172.21.15.94:6789/0 conn(0x7f5bc40b05a0 legacy=0x7f5bc40b0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:23.110 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.099+0000 7f5bc8c2b640 1 -- 172.21.15.88:0/1930488521 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc40ae960 con 0x7f5bc41599f0 2023-10-16T20:06:23.110 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.099+0000 7f5bc8c2b640 1 -- 172.21.15.88:0/1930488521 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f5bc0001680 con 0x7f5bc41599f0 2023-10-16T20:06:23.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.099+0000 7f5bc8c2b640 1 -- 172.21.15.88:0/1930488521 >> v1:172.21.15.88:6789/0 conn(0x7f5bc41599f0 legacy=0x7f5bc40b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:23.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.099+0000 7f5bc8c2b640 1 --2- 172.21.15.88:0/1930488521 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bb0005b90 0x7f5bb0005f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:23.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.099+0000 7f5bc8c2b640 1 --2- 172.21.15.88:0/1930488521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bb0006590 0x7f5bb0006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:23.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.099+0000 7f5bc8c2b640 1 --2- 172.21.15.88:0/1930488521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bb0007260 0x7f5bb000dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:23.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.099+0000 7f5bc8c2b640 1 -- 172.21.15.88:0/1930488521 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc40b3140 con 0x7f5bb0007260 2023-10-16T20:06:23.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.099+0000 7f5bc8c2b640 1 -- 172.21.15.88:0/1930488521 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb000e1c0 con 0x7f5bb0006590 2023-10-16T20:06:23.112 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.099+0000 7f5bc8c2b640 1 -- 172.21.15.88:0/1930488521 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb000e380 con 0x7f5bb0005b90 2023-10-16T20:06:23.112 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.099+0000 7f5bca42e640 1 --2- 172.21.15.88:0/1930488521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bb0007260 0x7f5bb000dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:23.112 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.099+0000 7f5bc9c2d640 1 --2- 172.21.15.88:0/1930488521 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bb0005b90 0x7f5bb0005f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:23.112 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.099+0000 7f5bc942c640 1 --2- 172.21.15.88:0/1930488521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bb0006590 0x7f5bb0006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:23.113 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.099+0000 7f5bca42e640 1 -- 172.21.15.88:0/1930488521 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bb0005b90 msgr2=0x7f5bb0005f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:23.113 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.099+0000 7f5bca42e640 1 --2- 172.21.15.88:0/1930488521 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bb0005b90 0x7f5bb0005f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:23.113 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.099+0000 7f5bca42e640 1 -- 172.21.15.88:0/1930488521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bb0006590 msgr2=0x7f5bb0006a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:23.113 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.099+0000 7f5bca42e640 1 --2- 172.21.15.88:0/1930488521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bb0006590 0x7f5bb0006a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:23.113 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.099+0000 7f5bca42e640 1 -- 172.21.15.88:0/1930488521 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb0005740 con 0x7f5bb0007260 2023-10-16T20:06:23.113 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.099+0000 7f5bc9c2d640 1 --2- 172.21.15.88:0/1930488521 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bb0005b90 0x7f5bb0005f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:23.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.099+0000 7f5bc942c640 1 --2- 172.21.15.88:0/1930488521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bb0006590 0x7f5bb0006a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:23.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.103+0000 7f5bca42e640 1 -- 172.21.15.88:0/1930488521 reap_dead start 2023-10-16T20:06:23.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.103+0000 7f5bca42e640 1 --2- 172.21.15.88:0/1930488521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bb0007260 0x7f5bb000dba0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc002000 tx=0x7f5bbc01c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41a85b9221b1868 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:23.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.103+0000 7f5bc8c2b640 1 -- 172.21.15.88:0/1930488521 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc003b80 con 0x7f5bb0007260 2023-10-16T20:06:23.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.103+0000 7f5bc8c2b640 1 -- 172.21.15.88:0/1930488521 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bbc003d00 con 0x7f5bb0007260 2023-10-16T20:06:23.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.103+0000 7f5bc8c2b640 1 -- 172.21.15.88:0/1930488521 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc005310 con 0x7f5bb0007260 2023-10-16T20:06:23.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.103+0000 7f5bcbe91640 1 -- 172.21.15.88:0/1930488521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bb0007260 msgr2=0x7f5bb000dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:23.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.103+0000 7f5bcbe91640 1 --2- 172.21.15.88:0/1930488521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bb0007260 0x7f5bb000dba0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc002000 tx=0x7f5bbc01c8c0 comp rx=0 tx=0).stop 2023-10-16T20:06:23.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.103+0000 7f5bcbe91640 1 -- 172.21.15.88:0/1930488521 shutdown_connections 2023-10-16T20:06:23.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.103+0000 7f5bcbe91640 1 --2- 172.21.15.88:0/1930488521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bb0007260 0x7f5bb000dba0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:23.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.103+0000 7f5bcbe91640 1 -- 172.21.15.88:0/1930488521 >> 172.21.15.88:0/1930488521 conn(0x7f5bc40ce5e0 msgr2=0x7f5bc40ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:23.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.103+0000 7f5bcbe91640 1 -- 172.21.15.88:0/1930488521 shutdown_connections 2023-10-16T20:06:23.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.103+0000 7f5bcbe91640 1 -- 172.21.15.88:0/1930488521 wait complete. 2023-10-16T20:06:23.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.103+0000 7f5bcbe91640 1 Processor -- start 2023-10-16T20:06:23.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.103+0000 7f5bcbe91640 1 -- start start 2023-10-16T20:06:23.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.107+0000 7f5bcbe91640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bb0007260 0x7f5bc414e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:23.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.107+0000 7f5bcbe91640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bc415ad20 0x7f5bc415b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:23.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.107+0000 7f5bcbe91640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bc415bc40 0x7f5bc415c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:23.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.107+0000 7f5bcbe91640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc415c600 con 0x7f5bc415ad20 2023-10-16T20:06:23.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.107+0000 7f5bcbe91640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc414e950 con 0x7f5bc415bc40 2023-10-16T20:06:23.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.107+0000 7f5bcbe91640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc415b840 con 0x7f5bb0007260 2023-10-16T20:06:23.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.107+0000 7f5bc942c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bc415ad20 0x7f5bc415b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:23.122 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.107+0000 7f5bc942c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bc415ad20 0x7f5bc415b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57568/0 (socket says 172.21.15.88:57568) 2023-10-16T20:06:23.122 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.107+0000 7f5bc942c640 1 -- 172.21.15.88:0/1223933264 learned_addr learned my addr 172.21.15.88:0/1223933264 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:23.122 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.107+0000 7f5bc9c2d640 1 --2- 172.21.15.88:0/1223933264 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bb0007260 0x7f5bc414e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:23.122 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.107+0000 7f5bc942c640 1 -- 172.21.15.88:0/1223933264 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bb0007260 msgr2=0x7f5bc414e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:23.122 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.107+0000 7f5bc942c640 1 --2- 172.21.15.88:0/1223933264 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bb0007260 0x7f5bc414e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:23.123 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.107+0000 7f5bca42e640 1 --2- 172.21.15.88:0/1223933264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bc415bc40 0x7f5bc415c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:23.123 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.107+0000 7f5bc942c640 1 -- 172.21.15.88:0/1223933264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bc415bc40 msgr2=0x7f5bc415c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:23.123 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.107+0000 7f5bc942c640 1 --2- 172.21.15.88:0/1223933264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bc415bc40 0x7f5bc415c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:23.123 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.107+0000 7f5bc942c640 1 -- 172.21.15.88:0/1223933264 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc40a7590 con 0x7f5bc415ad20 2023-10-16T20:06:23.123 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.107+0000 7f5bca42e640 1 --2- 172.21.15.88:0/1223933264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bc415bc40 0x7f5bc415c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:06:23.124 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.107+0000 7f5bc942c640 1 --2- 172.21.15.88:0/1223933264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bc415ad20 0x7f5bc415b1a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4011590 tx=0x7f5bb4011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:23.124 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.107+0000 7f5bba7fc640 1 -- 172.21.15.88:0/1223933264 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4004220 con 0x7f5bc415ad20 2023-10-16T20:06:23.124 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.107+0000 7f5bba7fc640 1 -- 172.21.15.88:0/1223933264 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb4004380 con 0x7f5bc415ad20 2023-10-16T20:06:23.124 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.107+0000 7f5bba7fc640 1 -- 172.21.15.88:0/1223933264 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4015870 con 0x7f5bc415ad20 2023-10-16T20:06:23.124 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.107+0000 7f5bcbe91640 1 -- 172.21.15.88:0/1223933264 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc40a7870 con 0x7f5bc415ad20 2023-10-16T20:06:23.125 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.107+0000 7f5bcbe91640 1 -- 172.21.15.88:0/1223933264 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc40a7e30 con 0x7f5bc415ad20 2023-10-16T20:06:23.125 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.107+0000 7f5bcbe91640 1 -- 172.21.15.88:0/1223933264 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bc404e9c0 con 0x7f5bc415ad20 2023-10-16T20:06:23.125 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.107+0000 7f5bba7fc640 1 -- 172.21.15.88:0/1223933264 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5bb40159d0 con 0x7f5bc415ad20 2023-10-16T20:06:23.125 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.111+0000 7f5bba7fc640 1 --2- 172.21.15.88:0/1223933264 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5ba80664d0 0x7f5ba8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:23.125 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.111+0000 7f5bba7fc640 1 -- 172.21.15.88:0/1223933264 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f5bb40844a0 con 0x7f5bc415ad20 2023-10-16T20:06:23.126 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.111+0000 7f5bc9c2d640 1 --2- 172.21.15.88:0/1223933264 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5ba80664d0 0x7f5ba8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:23.126 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.111+0000 7f5bc9c2d640 1 --2- 172.21.15.88:0/1223933264 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5ba80664d0 0x7f5ba8068990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0000c10 tx=0x7f5bc0001ad0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:23.126 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.115+0000 7f5bba7fc640 1 -- 172.21.15.88:0/1223933264 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5bb4051730 con 0x7f5bc415ad20 2023-10-16T20:06:23.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.259+0000 7f5bcbe91640 1 -- 172.21.15.88:0/1223933264 --> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5bc40aef20 con 0x7f5ba80664d0 2023-10-16T20:06:23.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.267+0000 7f5bba7fc640 1 -- 172.21.15.88:0/1223933264 <== mgr.4110 v2:172.21.15.88:6832/4078483333 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289511 (secure 0 0 0) 0x7f5bc0012830 con 0x7f5ba80664d0 2023-10-16T20:06:23.275 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:06:23.276 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2023-10-16T20:06:23.278 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.271+0000 7f5bcbe91640 1 -- 172.21.15.88:0/1223933264 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5ba80664d0 msgr2=0x7f5ba8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:23.278 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.271+0000 7f5bcbe91640 1 --2- 172.21.15.88:0/1223933264 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5ba80664d0 0x7f5ba8068990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0000c10 tx=0x7f5bc0001ad0 comp rx=0 tx=0).stop 2023-10-16T20:06:23.278 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.271+0000 7f5bcbe91640 1 -- 172.21.15.88:0/1223933264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bc415ad20 msgr2=0x7f5bc415b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:23.278 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.271+0000 7f5bcbe91640 1 --2- 172.21.15.88:0/1223933264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bc415ad20 0x7f5bc415b1a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4011590 tx=0x7f5bb4011a50 comp rx=0 tx=0).stop 2023-10-16T20:06:23.279 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.271+0000 7f5bcbe91640 1 -- 172.21.15.88:0/1223933264 shutdown_connections 2023-10-16T20:06:23.279 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.271+0000 7f5bcbe91640 1 --2- 172.21.15.88:0/1223933264 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5ba80664d0 0x7f5ba8068990 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:23.279 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.271+0000 7f5bcbe91640 1 --2- 172.21.15.88:0/1223933264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bc415bc40 0x7f5bc415c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:23.279 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.271+0000 7f5bcbe91640 1 --2- 172.21.15.88:0/1223933264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bc415ad20 0x7f5bc415b1a0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:23.279 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.271+0000 7f5bcbe91640 1 --2- 172.21.15.88:0/1223933264 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bb0007260 0x7f5bc414e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:23.280 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.271+0000 7f5bcbe91640 1 -- 172.21.15.88:0/1223933264 >> 172.21.15.88:0/1223933264 conn(0x7f5bc40ce5e0 msgr2=0x7f5bc4072790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:23.280 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.271+0000 7f5bcbe91640 1 -- 172.21.15.88:0/1223933264 shutdown_connections 2023-10-16T20:06:23.280 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:23.271+0000 7f5bcbe91640 1 -- 172.21.15.88:0/1223933264 wait complete. 2023-10-16T20:06:23.295 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2023-10-16T20:06:22.462831+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233448,"kb_used_data":4752,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754741272,"statfs":{"total":773094113280,"available":772855062528,"internally_reserved":0,"allocated":4866048,"data_stored":2732341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":234016313},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001270"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.220769+0000","last_change":"2023-10-16T20:06:17.217901+0000","last_active":"2023-10-16T20:06:17.220769+0000","last_peered":"2023-10-16T20:06:17.220769+0000","last_clean":"2023-10-16T20:06:17.220769+0000","last_became_active":"2023-10-16T20:06:17.217583+0000","last_became_peered":"2023-10-16T20:06:17.217583+0000","last_unstale":"2023-10-16T20:06:17.220769+0000","last_undegraded":"2023-10-16T20:06:17.220769+0000","last_fullsized":"2023-10-16T20:06:17.220769+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:24:29.941143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.215707+0000","last_change":"2023-10-16T20:06:17.211493+0000","last_active":"2023-10-16T20:06:17.215707+0000","last_peered":"2023-10-16T20:06:17.215707+0000","last_clean":"2023-10-16T20:06:17.215707+0000","last_became_active":"2023-10-16T20:06:17.211196+0000","last_became_peered":"2023-10-16T20:06:17.211196+0000","last_unstale":"2023-10-16T20:06:17.215707+0000","last_undegraded":"2023-10-16T20:06:17.215707+0000","last_fullsized":"2023-10-16T20:06:17.215707+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:50:32.312772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198456+0000","last_change":"2023-10-16T20:06:17.211314+0000","last_active":"2023-10-16T20:06:18.198456+0000","last_peered":"2023-10-16T20:06:18.198456+0000","last_clean":"2023-10-16T20:06:18.198456+0000","last_became_active":"2023-10-16T20:06:17.210991+0000","last_became_peered":"2023-10-16T20:06:17.210991+0000","last_unstale":"2023-10-16T20:06:18.198456+0000","last_undegraded":"2023-10-16T20:06:18.198456+0000","last_fullsized":"2023-10-16T20:06:18.198456+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:07:43.835803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357788+0000","last_change":"2023-10-16T20:06:17.216483+0000","last_active":"2023-10-16T20:06:18.357788+0000","last_peered":"2023-10-16T20:06:18.357788+0000","last_clean":"2023-10-16T20:06:18.357788+0000","last_became_active":"2023-10-16T20:06:17.216202+0000","last_became_peered":"2023-10-16T20:06:17.216202+0000","last_unstale":"2023-10-16T20:06:18.357788+0000","last_undegraded":"2023-10-16T20:06:18.357788+0000","last_fullsized":"2023-10-16T20:06:18.357788+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:32:02.453729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357619+0000","last_change":"2023-10-16T20:06:17.208325+0000","last_active":"2023-10-16T20:06:18.357619+0000","last_peered":"2023-10-16T20:06:18.357619+0000","last_clean":"2023-10-16T20:06:18.357619+0000","last_became_active":"2023-10-16T20:06:17.208061+0000","last_became_peered":"2023-10-16T20:06:17.208061+0000","last_unstale":"2023-10-16T20:06:18.357619+0000","last_undegraded":"2023-10-16T20:06:18.357619+0000","last_fullsized":"2023-10-16T20:06:18.357619+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:02:28.165753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.217274+0000","last_change":"2023-10-16T20:06:17.208416+0000","last_active":"2023-10-16T20:06:17.217274+0000","last_peered":"2023-10-16T20:06:17.217274+0000","last_clean":"2023-10-16T20:06:17.217274+0000","last_became_active":"2023-10-16T20:06:17.207713+0000","last_became_peered":"2023-10-16T20:06:17.207713+0000","last_unstale":"2023-10-16T20:06:17.217274+0000","last_undegraded":"2023-10-16T20:06:17.217274+0000","last_fullsized":"2023-10-16T20:06:17.217274+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:39:06.086998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.213838+0000","last_change":"2023-10-16T20:06:17.206736+0000","last_active":"2023-10-16T20:06:17.213838+0000","last_peered":"2023-10-16T20:06:17.213838+0000","last_clean":"2023-10-16T20:06:17.213838+0000","last_became_active":"2023-10-16T20:06:17.206435+0000","last_became_peered":"2023-10-16T20:06:17.206435+0000","last_unstale":"2023-10-16T20:06:17.213838+0000","last_undegraded":"2023-10-16T20:06:17.213838+0000","last_fullsized":"2023-10-16T20:06:17.213838+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:50:32.312772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.218213+0000","last_change":"2023-10-16T20:06:17.214147+0000","last_active":"2023-10-16T20:06:17.218213+0000","last_peered":"2023-10-16T20:06:17.218213+0000","last_clean":"2023-10-16T20:06:17.218213+0000","last_became_active":"2023-10-16T20:06:17.213998+0000","last_became_peered":"2023-10-16T20:06:17.213998+0000","last_unstale":"2023-10-16T20:06:17.218213+0000","last_undegraded":"2023-10-16T20:06:17.218213+0000","last_fullsized":"2023-10-16T20:06:17.218213+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:36:27.881926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198636+0000","last_change":"2023-10-16T20:06:17.213204+0000","last_active":"2023-10-16T20:06:18.198636+0000","last_peered":"2023-10-16T20:06:18.198636+0000","last_clean":"2023-10-16T20:06:18.198636+0000","last_became_active":"2023-10-16T20:06:17.212563+0000","last_became_peered":"2023-10-16T20:06:17.212563+0000","last_unstale":"2023-10-16T20:06:18.198636+0000","last_undegraded":"2023-10-16T20:06:18.198636+0000","last_fullsized":"2023-10-16T20:06:18.198636+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:25:04.009896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.222492+0000","last_change":"2023-10-16T20:06:17.218989+0000","last_active":"2023-10-16T20:06:17.222492+0000","last_peered":"2023-10-16T20:06:17.222492+0000","last_clean":"2023-10-16T20:06:17.222492+0000","last_became_active":"2023-10-16T20:06:17.218726+0000","last_became_peered":"2023-10-16T20:06:17.218726+0000","last_unstale":"2023-10-16T20:06:17.222492+0000","last_undegraded":"2023-10-16T20:06:17.222492+0000","last_fullsized":"2023-10-16T20:06:17.222492+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:27:39.178553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.221629+0000","last_change":"2023-10-16T20:06:17.217372+0000","last_active":"2023-10-16T20:06:17.221629+0000","last_peered":"2023-10-16T20:06:17.221629+0000","last_clean":"2023-10-16T20:06:17.221629+0000","last_became_active":"2023-10-16T20:06:17.216730+0000","last_became_peered":"2023-10-16T20:06:17.216730+0000","last_unstale":"2023-10-16T20:06:17.221629+0000","last_undegraded":"2023-10-16T20:06:17.221629+0000","last_fullsized":"2023-10-16T20:06:17.221629+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:18:14.480179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198265+0000","last_change":"2023-10-16T20:06:17.214475+0000","last_active":"2023-10-16T20:06:18.198265+0000","last_peered":"2023-10-16T20:06:18.198265+0000","last_clean":"2023-10-16T20:06:18.198265+0000","last_became_active":"2023-10-16T20:06:17.214211+0000","last_became_peered":"2023-10-16T20:06:17.214211+0000","last_unstale":"2023-10-16T20:06:18.198265+0000","last_undegraded":"2023-10-16T20:06:18.198265+0000","last_fullsized":"2023-10-16T20:06:18.198265+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:02:29.473591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.221752+0000","last_change":"2023-10-16T20:06:17.218455+0000","last_active":"2023-10-16T20:06:17.221752+0000","last_peered":"2023-10-16T20:06:17.221752+0000","last_clean":"2023-10-16T20:06:17.221752+0000","last_became_active":"2023-10-16T20:06:17.217565+0000","last_became_peered":"2023-10-16T20:06:17.217565+0000","last_unstale":"2023-10-16T20:06:17.221752+0000","last_undegraded":"2023-10-16T20:06:17.221752+0000","last_fullsized":"2023-10-16T20:06:17.221752+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:13.874932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.222220+0000","last_change":"2023-10-16T20:06:17.214285+0000","last_active":"2023-10-16T20:06:17.222220+0000","last_peered":"2023-10-16T20:06:17.222220+0000","last_clean":"2023-10-16T20:06:17.222220+0000","last_became_active":"2023-10-16T20:06:17.213551+0000","last_became_peered":"2023-10-16T20:06:17.213551+0000","last_unstale":"2023-10-16T20:06:17.222220+0000","last_undegraded":"2023-10-16T20:06:17.222220+0000","last_fullsized":"2023-10-16T20:06:17.222220+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:10:43.145100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.213855+0000","last_change":"2023-10-16T20:06:17.208025+0000","last_active":"2023-10-16T20:06:17.213855+0000","last_peered":"2023-10-16T20:06:17.213855+0000","last_clean":"2023-10-16T20:06:17.213855+0000","last_became_active":"2023-10-16T20:06:17.207739+0000","last_became_peered":"2023-10-16T20:06:17.207739+0000","last_unstale":"2023-10-16T20:06:17.213855+0000","last_undegraded":"2023-10-16T20:06:17.213855+0000","last_fullsized":"2023-10-16T20:06:17.213855+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:36:27.881926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196502+0000","last_change":"2023-10-16T20:06:17.222232+0000","last_active":"2023-10-16T20:06:18.196502+0000","last_peered":"2023-10-16T20:06:18.196502+0000","last_clean":"2023-10-16T20:06:18.196502+0000","last_became_active":"2023-10-16T20:06:17.221892+0000","last_became_peered":"2023-10-16T20:06:17.221892+0000","last_unstale":"2023-10-16T20:06:18.196502+0000","last_undegraded":"2023-10-16T20:06:18.196502+0000","last_fullsized":"2023-10-16T20:06:18.196502+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:58:45.132214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197149+0000","last_change":"2023-10-16T20:06:17.224446+0000","last_active":"2023-10-16T20:06:18.197149+0000","last_peered":"2023-10-16T20:06:18.197149+0000","last_clean":"2023-10-16T20:06:18.197149+0000","last_became_active":"2023-10-16T20:06:17.224113+0000","last_became_peered":"2023-10-16T20:06:17.224113+0000","last_unstale":"2023-10-16T20:06:18.197149+0000","last_undegraded":"2023-10-16T20:06:18.197149+0000","last_fullsized":"2023-10-16T20:06:18.197149+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:37:43.948080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198392+0000","last_change":"2023-10-16T20:06:17.214181+0000","last_active":"2023-10-16T20:06:18.198392+0000","last_peered":"2023-10-16T20:06:18.198392+0000","last_clean":"2023-10-16T20:06:18.198392+0000","last_became_active":"2023-10-16T20:06:17.213876+0000","last_became_peered":"2023-10-16T20:06:17.213876+0000","last_unstale":"2023-10-16T20:06:18.198392+0000","last_undegraded":"2023-10-16T20:06:18.198392+0000","last_fullsized":"2023-10-16T20:06:18.198392+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:39:06.086998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357861+0000","last_change":"2023-10-16T20:06:17.205826+0000","last_active":"2023-10-16T20:06:18.357861+0000","last_peered":"2023-10-16T20:06:18.357861+0000","last_clean":"2023-10-16T20:06:18.357861+0000","last_became_active":"2023-10-16T20:06:17.204888+0000","last_became_peered":"2023-10-16T20:06:17.204888+0000","last_unstale":"2023-10-16T20:06:18.357861+0000","last_undegraded":"2023-10-16T20:06:18.357861+0000","last_fullsized":"2023-10-16T20:06:18.357861+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:29:06.675309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197777+0000","last_change":"2023-10-16T20:06:17.217103+0000","last_active":"2023-10-16T20:06:18.197777+0000","last_peered":"2023-10-16T20:06:18.197777+0000","last_clean":"2023-10-16T20:06:18.197777+0000","last_became_active":"2023-10-16T20:06:17.216717+0000","last_became_peered":"2023-10-16T20:06:17.216717+0000","last_unstale":"2023-10-16T20:06:18.197777+0000","last_undegraded":"2023-10-16T20:06:18.197777+0000","last_fullsized":"2023-10-16T20:06:18.197777+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:20:06.934366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.206185+0000","last_change":"2023-10-16T20:06:17.201875+0000","last_active":"2023-10-16T20:06:17.206185+0000","last_peered":"2023-10-16T20:06:17.206185+0000","last_clean":"2023-10-16T20:06:17.206185+0000","last_became_active":"2023-10-16T20:06:17.200537+0000","last_became_peered":"2023-10-16T20:06:17.200537+0000","last_unstale":"2023-10-16T20:06:17.206185+0000","last_undegraded":"2023-10-16T20:06:17.206185+0000","last_fullsized":"2023-10-16T20:06:17.206185+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:23:28.625212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.216225+0000","last_change":"2023-10-16T20:06:17.211432+0000","last_active":"2023-10-16T20:06:17.216225+0000","last_peered":"2023-10-16T20:06:17.216225+0000","last_clean":"2023-10-16T20:06:17.216225+0000","last_became_active":"2023-10-16T20:06:17.211100+0000","last_became_peered":"2023-10-16T20:06:17.211100+0000","last_unstale":"2023-10-16T20:06:17.216225+0000","last_undegraded":"2023-10-16T20:06:17.216225+0000","last_fullsized":"2023-10-16T20:06:17.216225+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:06:45.516915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196828+0000","last_change":"2023-10-16T20:06:17.225476+0000","last_active":"2023-10-16T20:06:18.196828+0000","last_peered":"2023-10-16T20:06:18.196828+0000","last_clean":"2023-10-16T20:06:18.196828+0000","last_became_active":"2023-10-16T20:06:17.224667+0000","last_became_peered":"2023-10-16T20:06:17.224667+0000","last_unstale":"2023-10-16T20:06:18.196828+0000","last_undegraded":"2023-10-16T20:06:18.196828+0000","last_fullsized":"2023-10-16T20:06:18.196828+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:31:31.346356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357610+0000","last_change":"2023-10-16T20:06:17.216704+0000","last_active":"2023-10-16T20:06:18.357610+0000","last_peered":"2023-10-16T20:06:18.357610+0000","last_clean":"2023-10-16T20:06:18.357610+0000","last_became_active":"2023-10-16T20:06:17.216341+0000","last_became_peered":"2023-10-16T20:06:17.216341+0000","last_unstale":"2023-10-16T20:06:18.357610+0000","last_undegraded":"2023-10-16T20:06:18.357610+0000","last_fullsized":"2023-10-16T20:06:18.357610+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:14:46.135352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198413+0000","last_change":"2023-10-16T20:06:17.216103+0000","last_active":"2023-10-16T20:06:18.198413+0000","last_peered":"2023-10-16T20:06:18.198413+0000","last_clean":"2023-10-16T20:06:18.198413+0000","last_became_active":"2023-10-16T20:06:17.215844+0000","last_became_peered":"2023-10-16T20:06:17.215844+0000","last_unstale":"2023-10-16T20:06:18.198413+0000","last_undegraded":"2023-10-16T20:06:18.198413+0000","last_fullsized":"2023-10-16T20:06:18.198413+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:01.646291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.216083+0000","last_change":"2023-10-16T20:06:17.207691+0000","last_active":"2023-10-16T20:06:17.216083+0000","last_peered":"2023-10-16T20:06:17.216083+0000","last_clean":"2023-10-16T20:06:17.216083+0000","last_became_active":"2023-10-16T20:06:17.207437+0000","last_became_peered":"2023-10-16T20:06:17.207437+0000","last_unstale":"2023-10-16T20:06:17.216083+0000","last_undegraded":"2023-10-16T20:06:17.216083+0000","last_fullsized":"2023-10-16T20:06:17.216083+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:23:28.625212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198320+0000","last_change":"2023-10-16T20:06:17.211359+0000","last_active":"2023-10-16T20:06:18.198320+0000","last_peered":"2023-10-16T20:06:18.198320+0000","last_clean":"2023-10-16T20:06:18.198320+0000","last_became_active":"2023-10-16T20:06:17.210506+0000","last_became_peered":"2023-10-16T20:06:17.210506+0000","last_unstale":"2023-10-16T20:06:18.198320+0000","last_undegraded":"2023-10-16T20:06:18.198320+0000","last_fullsized":"2023-10-16T20:06:18.198320+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:32:44.093091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.222529+0000","last_change":"2023-10-16T20:06:17.219679+0000","last_active":"2023-10-16T20:06:17.222529+0000","last_peered":"2023-10-16T20:06:17.222529+0000","last_clean":"2023-10-16T20:06:17.222529+0000","last_became_active":"2023-10-16T20:06:17.214175+0000","last_became_peered":"2023-10-16T20:06:17.214175+0000","last_unstale":"2023-10-16T20:06:17.222529+0000","last_undegraded":"2023-10-16T20:06:17.222529+0000","last_fullsized":"2023-10-16T20:06:17.222529+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:22:32.981812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197630+0000","last_change":"2023-10-16T20:06:17.210945+0000","last_active":"2023-10-16T20:06:18.197630+0000","last_peered":"2023-10-16T20:06:18.197630+0000","last_clean":"2023-10-16T20:06:18.197630+0000","last_became_active":"2023-10-16T20:06:17.210629+0000","last_became_peered":"2023-10-16T20:06:17.210629+0000","last_unstale":"2023-10-16T20:06:18.197630+0000","last_undegraded":"2023-10-16T20:06:18.197630+0000","last_fullsized":"2023-10-16T20:06:18.197630+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:27:10.715147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197643+0000","last_change":"2023-10-16T20:06:17.212963+0000","last_active":"2023-10-16T20:06:18.197643+0000","last_peered":"2023-10-16T20:06:18.197643+0000","last_clean":"2023-10-16T20:06:18.197643+0000","last_became_active":"2023-10-16T20:06:17.212693+0000","last_became_peered":"2023-10-16T20:06:17.212693+0000","last_unstale":"2023-10-16T20:06:18.197643+0000","last_undegraded":"2023-10-16T20:06:18.197643+0000","last_fullsized":"2023-10-16T20:06:18.197643+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:10:43.145100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197558+0000","last_change":"2023-10-16T20:06:17.219075+0000","last_active":"2023-10-16T20:06:18.197558+0000","last_peered":"2023-10-16T20:06:18.197558+0000","last_clean":"2023-10-16T20:06:18.197558+0000","last_became_active":"2023-10-16T20:06:17.218774+0000","last_became_peered":"2023-10-16T20:06:17.218774+0000","last_unstale":"2023-10-16T20:06:18.197558+0000","last_undegraded":"2023-10-16T20:06:18.197558+0000","last_fullsized":"2023-10-16T20:06:18.197558+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:05:25.874296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.209799+0000","last_change":"2023-10-16T20:06:17.200644+0000","last_active":"2023-10-16T20:06:17.209799+0000","last_peered":"2023-10-16T20:06:17.209799+0000","last_clean":"2023-10-16T20:06:17.209799+0000","last_became_active":"2023-10-16T20:06:17.200350+0000","last_became_peered":"2023-10-16T20:06:17.200350+0000","last_unstale":"2023-10-16T20:06:17.209799+0000","last_undegraded":"2023-10-16T20:06:17.209799+0000","last_fullsized":"2023-10-16T20:06:17.209799+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:58:21.190657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196740+0000","last_change":"2023-10-16T20:06:17.224294+0000","last_active":"2023-10-16T20:06:18.196740+0000","last_peered":"2023-10-16T20:06:18.196740+0000","last_clean":"2023-10-16T20:06:18.196740+0000","last_became_active":"2023-10-16T20:06:17.223988+0000","last_became_peered":"2023-10-16T20:06:17.223988+0000","last_unstale":"2023-10-16T20:06:18.196740+0000","last_undegraded":"2023-10-16T20:06:18.196740+0000","last_fullsized":"2023-10-16T20:06:18.196740+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:03:40.257975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"40'4","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198438+0000","last_change":"2023-10-16T20:06:16.732366+0000","last_active":"2023-10-16T20:06:18.198438+0000","last_peered":"2023-10-16T20:06:18.198438+0000","last_clean":"2023-10-16T20:06:18.198438+0000","last_became_active":"2023-10-16T20:06:16.732035+0000","last_became_peered":"2023-10-16T20:06:16.732035+0000","last_unstale":"2023-10-16T20:06:18.198438+0000","last_undegraded":"2023-10-16T20:06:18.198438+0000","last_fullsized":"2023-10-16T20:06:18.198438+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:35:43.770049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.216156+0000","last_change":"2023-10-16T20:06:17.207721+0000","last_active":"2023-10-16T20:06:17.216156+0000","last_peered":"2023-10-16T20:06:17.216156+0000","last_clean":"2023-10-16T20:06:17.216156+0000","last_became_active":"2023-10-16T20:06:17.207470+0000","last_became_peered":"2023-10-16T20:06:17.207470+0000","last_unstale":"2023-10-16T20:06:17.216156+0000","last_undegraded":"2023-10-16T20:06:17.216156+0000","last_fullsized":"2023-10-16T20:06:17.216156+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:56:27.147860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"40'1","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198456+0000","last_change":"2023-10-16T20:06:16.730576+0000","last_active":"2023-10-16T20:06:18.198456+0000","last_peered":"2023-10-16T20:06:18.198456+0000","last_clean":"2023-10-16T20:06:18.198456+0000","last_became_active":"2023-10-16T20:06:16.729963+0000","last_became_peered":"2023-10-16T20:06:16.729963+0000","last_unstale":"2023-10-16T20:06:18.198456+0000","last_undegraded":"2023-10-16T20:06:18.198456+0000","last_fullsized":"2023-10-16T20:06:18.198456+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:51:10.650711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.222274+0000","last_change":"2023-10-16T20:06:17.215735+0000","last_active":"2023-10-16T20:06:17.222274+0000","last_peered":"2023-10-16T20:06:17.222274+0000","last_clean":"2023-10-16T20:06:17.222274+0000","last_became_active":"2023-10-16T20:06:17.214986+0000","last_became_peered":"2023-10-16T20:06:17.214986+0000","last_unstale":"2023-10-16T20:06:17.222274+0000","last_undegraded":"2023-10-16T20:06:17.222274+0000","last_fullsized":"2023-10-16T20:06:17.222274+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:35:44.780849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"40'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.733064+0000","last_change":"2023-10-16T20:06:16.739387+0000","last_active":"2023-10-16T20:06:17.733064+0000","last_peered":"2023-10-16T20:06:17.733064+0000","last_clean":"2023-10-16T20:06:17.733064+0000","last_became_active":"2023-10-16T20:06:16.739070+0000","last_became_peered":"2023-10-16T20:06:16.739070+0000","last_unstale":"2023-10-16T20:06:17.733064+0000","last_undegraded":"2023-10-16T20:06:17.733064+0000","last_fullsized":"2023-10-16T20:06:17.733064+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:04:52.152758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.217801+0000","last_change":"2023-10-16T20:06:17.213835+0000","last_active":"2023-10-16T20:06:17.217801+0000","last_peered":"2023-10-16T20:06:17.217801+0000","last_clean":"2023-10-16T20:06:17.217801+0000","last_became_active":"2023-10-16T20:06:17.213516+0000","last_became_peered":"2023-10-16T20:06:17.213516+0000","last_unstale":"2023-10-16T20:06:17.217801+0000","last_undegraded":"2023-10-16T20:06:17.217801+0000","last_fullsized":"2023-10-16T20:06:17.217801+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:25:13.284821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"40'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.732355+0000","last_change":"2023-10-16T20:06:16.730019+0000","last_active":"2023-10-16T20:06:17.732355+0000","last_peered":"2023-10-16T20:06:17.732355+0000","last_clean":"2023-10-16T20:06:17.732355+0000","last_became_active":"2023-10-16T20:06:16.729804+0000","last_became_peered":"2023-10-16T20:06:16.729804+0000","last_unstale":"2023-10-16T20:06:17.732355+0000","last_undegraded":"2023-10-16T20:06:17.732355+0000","last_fullsized":"2023-10-16T20:06:17.732355+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:56:26.137061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198084+0000","last_change":"2023-10-16T20:06:17.214073+0000","last_active":"2023-10-16T20:06:18.198084+0000","last_peered":"2023-10-16T20:06:18.198084+0000","last_clean":"2023-10-16T20:06:18.198084+0000","last_became_active":"2023-10-16T20:06:17.213793+0000","last_became_peered":"2023-10-16T20:06:17.213793+0000","last_unstale":"2023-10-16T20:06:18.198084+0000","last_undegraded":"2023-10-16T20:06:18.198084+0000","last_fullsized":"2023-10-16T20:06:18.198084+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:17:45.690804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"40'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.732663+0000","last_change":"2023-10-16T20:06:16.731997+0000","last_active":"2023-10-16T20:06:17.732663+0000","last_peered":"2023-10-16T20:06:17.732663+0000","last_clean":"2023-10-16T20:06:17.732663+0000","last_became_active":"2023-10-16T20:06:16.731714+0000","last_became_peered":"2023-10-16T20:06:16.731714+0000","last_unstale":"2023-10-16T20:06:17.732663+0000","last_undegraded":"2023-10-16T20:06:17.732663+0000","last_fullsized":"2023-10-16T20:06:17.732663+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:25:12.274021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.220705+0000","last_change":"2023-10-16T20:06:17.214298+0000","last_active":"2023-10-16T20:06:17.220705+0000","last_peered":"2023-10-16T20:06:17.220705+0000","last_clean":"2023-10-16T20:06:17.220705+0000","last_became_active":"2023-10-16T20:06:17.214038+0000","last_became_peered":"2023-10-16T20:06:17.214038+0000","last_unstale":"2023-10-16T20:06:17.220705+0000","last_undegraded":"2023-10-16T20:06:17.220705+0000","last_fullsized":"2023-10-16T20:06:17.220705+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:38:40.789443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"40'1","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196763+0000","last_change":"2023-10-16T20:06:16.737162+0000","last_active":"2023-10-16T20:06:18.196763+0000","last_peered":"2023-10-16T20:06:18.196763+0000","last_clean":"2023-10-16T20:06:18.196763+0000","last_became_active":"2023-10-16T20:06:16.736888+0000","last_became_peered":"2023-10-16T20:06:16.736888+0000","last_unstale":"2023-10-16T20:06:18.196763+0000","last_undegraded":"2023-10-16T20:06:18.196763+0000","last_fullsized":"2023-10-16T20:06:18.196763+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:22:31.971013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.211586+0000","last_change":"2023-10-16T20:06:17.208067+0000","last_active":"2023-10-16T20:06:17.211586+0000","last_peered":"2023-10-16T20:06:17.211586+0000","last_clean":"2023-10-16T20:06:17.211586+0000","last_became_active":"2023-10-16T20:06:17.207799+0000","last_became_peered":"2023-10-16T20:06:17.207799+0000","last_unstale":"2023-10-16T20:06:17.211586+0000","last_undegraded":"2023-10-16T20:06:17.211586+0000","last_fullsized":"2023-10-16T20:06:17.211586+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:40:58.934858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.195959+0000","last_change":"2023-10-16T20:06:16.729238+0000","last_active":"2023-10-16T20:06:17.195959+0000","last_peered":"2023-10-16T20:06:17.195959+0000","last_clean":"2023-10-16T20:06:17.195959+0000","last_became_active":"2023-10-16T20:06:16.728975+0000","last_became_peered":"2023-10-16T20:06:16.728975+0000","last_unstale":"2023-10-16T20:06:17.195959+0000","last_undegraded":"2023-10-16T20:06:17.195959+0000","last_fullsized":"2023-10-16T20:06:17.195959+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:36:26.871127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.220833+0000","last_change":"2023-10-16T20:06:17.218801+0000","last_active":"2023-10-16T20:06:17.220833+0000","last_peered":"2023-10-16T20:06:17.220833+0000","last_clean":"2023-10-16T20:06:17.220833+0000","last_became_active":"2023-10-16T20:06:17.218219+0000","last_became_peered":"2023-10-16T20:06:17.218219+0000","last_unstale":"2023-10-16T20:06:17.220833+0000","last_undegraded":"2023-10-16T20:06:17.220833+0000","last_fullsized":"2023-10-16T20:06:17.220833+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:27:39.178553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.202521+0000","last_change":"2023-10-16T20:06:16.720994+0000","last_active":"2023-10-16T20:06:17.202521+0000","last_peered":"2023-10-16T20:06:17.202521+0000","last_clean":"2023-10-16T20:06:17.202521+0000","last_became_active":"2023-10-16T20:06:16.720623+0000","last_became_peered":"2023-10-16T20:06:16.720623+0000","last_unstale":"2023-10-16T20:06:17.202521+0000","last_undegraded":"2023-10-16T20:06:17.202521+0000","last_fullsized":"2023-10-16T20:06:17.202521+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:32:43.082291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357768+0000","last_change":"2023-10-16T20:06:17.216335+0000","last_active":"2023-10-16T20:06:18.357768+0000","last_peered":"2023-10-16T20:06:18.357768+0000","last_clean":"2023-10-16T20:06:18.357768+0000","last_became_active":"2023-10-16T20:06:17.216070+0000","last_became_peered":"2023-10-16T20:06:17.216070+0000","last_unstale":"2023-10-16T20:06:18.357768+0000","last_undegraded":"2023-10-16T20:06:18.357768+0000","last_fullsized":"2023-10-16T20:06:18.357768+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:08:22.444705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.206243+0000","last_change":"2023-10-16T20:06:16.736582+0000","last_active":"2023-10-16T20:06:17.206243+0000","last_peered":"2023-10-16T20:06:17.206243+0000","last_clean":"2023-10-16T20:06:17.206243+0000","last_became_active":"2023-10-16T20:06:16.736322+0000","last_became_peered":"2023-10-16T20:06:16.736322+0000","last_unstale":"2023-10-16T20:06:17.206243+0000","last_undegraded":"2023-10-16T20:06:17.206243+0000","last_fullsized":"2023-10-16T20:06:17.206243+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:38:29.780087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197108+0000","last_change":"2023-10-16T20:06:17.220312+0000","last_active":"2023-10-16T20:06:18.197108+0000","last_peered":"2023-10-16T20:06:18.197108+0000","last_clean":"2023-10-16T20:06:18.197108+0000","last_became_active":"2023-10-16T20:06:17.220033+0000","last_became_peered":"2023-10-16T20:06:17.220033+0000","last_unstale":"2023-10-16T20:06:18.197108+0000","last_undegraded":"2023-10-16T20:06:18.197108+0000","last_fullsized":"2023-10-16T20:06:18.197108+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:00:59.428210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.198151+0000","last_change":"2023-10-16T20:06:16.725615+0000","last_active":"2023-10-16T20:06:17.198151+0000","last_peered":"2023-10-16T20:06:17.198151+0000","last_clean":"2023-10-16T20:06:17.198151+0000","last_became_active":"2023-10-16T20:06:16.725257+0000","last_became_peered":"2023-10-16T20:06:16.725257+0000","last_unstale":"2023-10-16T20:06:17.198151+0000","last_undegraded":"2023-10-16T20:06:17.198151+0000","last_fullsized":"2023-10-16T20:06:17.198151+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:36:24.034397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198481+0000","last_change":"2023-10-16T20:06:17.215733+0000","last_active":"2023-10-16T20:06:18.198481+0000","last_peered":"2023-10-16T20:06:18.198481+0000","last_clean":"2023-10-16T20:06:18.198481+0000","last_became_active":"2023-10-16T20:06:17.215495+0000","last_became_peered":"2023-10-16T20:06:17.215495+0000","last_unstale":"2023-10-16T20:06:18.198481+0000","last_undegraded":"2023-10-16T20:06:18.198481+0000","last_fullsized":"2023-10-16T20:06:18.198481+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:26:53.217625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"40'3","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.733519+0000","last_change":"2023-10-16T20:06:16.734102+0000","last_active":"2023-10-16T20:06:17.733519+0000","last_peered":"2023-10-16T20:06:17.733519+0000","last_clean":"2023-10-16T20:06:17.733519+0000","last_became_active":"2023-10-16T20:06:16.733829+0000","last_became_peered":"2023-10-16T20:06:16.733829+0000","last_unstale":"2023-10-16T20:06:17.733519+0000","last_undegraded":"2023-10-16T20:06:17.733519+0000","last_fullsized":"2023-10-16T20:06:17.733519+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:32:58.366700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.221860+0000","last_change":"2023-10-16T20:06:17.214205+0000","last_active":"2023-10-16T20:06:17.221860+0000","last_peered":"2023-10-16T20:06:17.221860+0000","last_clean":"2023-10-16T20:06:17.221860+0000","last_became_active":"2023-10-16T20:06:17.213261+0000","last_became_peered":"2023-10-16T20:06:17.213261+0000","last_unstale":"2023-10-16T20:06:17.221860+0000","last_undegraded":"2023-10-16T20:06:17.221860+0000","last_fullsized":"2023-10-16T20:06:17.221860+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:20:06.934366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"40'6","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.738226+0000","last_change":"2023-10-16T20:06:16.721479+0000","last_active":"2023-10-16T20:06:17.738226+0000","last_peered":"2023-10-16T20:06:17.738226+0000","last_clean":"2023-10-16T20:06:17.738226+0000","last_became_active":"2023-10-16T20:06:16.721136+0000","last_became_peered":"2023-10-16T20:06:16.721136+0000","last_unstale":"2023-10-16T20:06:17.738226+0000","last_undegraded":"2023-10-16T20:06:17.738226+0000","last_fullsized":"2023-10-16T20:06:17.738226+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:32:58.366700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198082+0000","last_change":"2023-10-16T20:06:17.211029+0000","last_active":"2023-10-16T20:06:18.198082+0000","last_peered":"2023-10-16T20:06:18.198082+0000","last_clean":"2023-10-16T20:06:18.198082+0000","last_became_active":"2023-10-16T20:06:17.210758+0000","last_became_peered":"2023-10-16T20:06:17.210758+0000","last_unstale":"2023-10-16T20:06:18.198082+0000","last_undegraded":"2023-10-16T20:06:18.198082+0000","last_fullsized":"2023-10-16T20:06:18.198082+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:26:41.239758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"40'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.732488+0000","last_change":"2023-10-16T20:06:16.735005+0000","last_active":"2023-10-16T20:06:17.732488+0000","last_peered":"2023-10-16T20:06:17.732488+0000","last_clean":"2023-10-16T20:06:17.732488+0000","last_became_active":"2023-10-16T20:06:16.734752+0000","last_became_peered":"2023-10-16T20:06:16.734752+0000","last_unstale":"2023-10-16T20:06:17.732488+0000","last_undegraded":"2023-10-16T20:06:17.732488+0000","last_fullsized":"2023-10-16T20:06:17.732488+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:21:23.740779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.219739+0000","last_change":"2023-10-16T20:06:17.215993+0000","last_active":"2023-10-16T20:06:17.219739+0000","last_peered":"2023-10-16T20:06:17.219739+0000","last_clean":"2023-10-16T20:06:17.219739+0000","last_became_active":"2023-10-16T20:06:17.215736+0000","last_became_peered":"2023-10-16T20:06:17.215736+0000","last_unstale":"2023-10-16T20:06:17.219739+0000","last_undegraded":"2023-10-16T20:06:17.219739+0000","last_fullsized":"2023-10-16T20:06:17.219739+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:36:25.045197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.195984+0000","last_change":"2023-10-16T20:06:16.733426+0000","last_active":"2023-10-16T20:06:17.195984+0000","last_peered":"2023-10-16T20:06:17.195984+0000","last_clean":"2023-10-16T20:06:17.195984+0000","last_became_active":"2023-10-16T20:06:16.732600+0000","last_became_peered":"2023-10-16T20:06:16.732600+0000","last_unstale":"2023-10-16T20:06:17.195984+0000","last_undegraded":"2023-10-16T20:06:17.195984+0000","last_fullsized":"2023-10-16T20:06:17.195984+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:12.864132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196653+0000","last_change":"2023-10-16T20:06:17.219925+0000","last_active":"2023-10-16T20:06:18.196653+0000","last_peered":"2023-10-16T20:06:18.196653+0000","last_clean":"2023-10-16T20:06:18.196653+0000","last_became_active":"2023-10-16T20:06:17.219584+0000","last_became_peered":"2023-10-16T20:06:17.219584+0000","last_unstale":"2023-10-16T20:06:18.196653+0000","last_undegraded":"2023-10-16T20:06:18.196653+0000","last_fullsized":"2023-10-16T20:06:18.196653+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:44:14.727872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197391+0000","last_change":"2023-10-16T20:06:16.732499+0000","last_active":"2023-10-16T20:06:18.197391+0000","last_peered":"2023-10-16T20:06:18.197391+0000","last_clean":"2023-10-16T20:06:18.197391+0000","last_became_active":"2023-10-16T20:06:16.732244+0000","last_became_peered":"2023-10-16T20:06:16.732244+0000","last_unstale":"2023-10-16T20:06:18.197391+0000","last_undegraded":"2023-10-16T20:06:18.197391+0000","last_fullsized":"2023-10-16T20:06:18.197391+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:40:57.924059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.219863+0000","last_change":"2023-10-16T20:06:17.215267+0000","last_active":"2023-10-16T20:06:17.219863+0000","last_peered":"2023-10-16T20:06:17.219863+0000","last_clean":"2023-10-16T20:06:17.219863+0000","last_became_active":"2023-10-16T20:06:17.215017+0000","last_became_peered":"2023-10-16T20:06:17.215017+0000","last_unstale":"2023-10-16T20:06:17.219863+0000","last_undegraded":"2023-10-16T20:06:17.219863+0000","last_fullsized":"2023-10-16T20:06:17.219863+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:38:40.789443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"40'8","reported_seq":20,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196540+0000","last_change":"2023-10-16T20:06:16.730165+0000","last_active":"2023-10-16T20:06:18.196540+0000","last_peered":"2023-10-16T20:06:18.196540+0000","last_clean":"2023-10-16T20:06:18.196540+0000","last_became_active":"2023-10-16T20:06:16.729894+0000","last_became_peered":"2023-10-16T20:06:16.729894+0000","last_unstale":"2023-10-16T20:06:18.196540+0000","last_undegraded":"2023-10-16T20:06:18.196540+0000","last_fullsized":"2023-10-16T20:06:18.196540+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:36:26.871127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.217655+0000","last_change":"2023-10-16T20:06:17.214039+0000","last_active":"2023-10-16T20:06:17.217655+0000","last_peered":"2023-10-16T20:06:17.217655+0000","last_clean":"2023-10-16T20:06:17.217655+0000","last_became_active":"2023-10-16T20:06:17.213551+0000","last_became_peered":"2023-10-16T20:06:17.213551+0000","last_unstale":"2023-10-16T20:06:17.217655+0000","last_undegraded":"2023-10-16T20:06:17.217655+0000","last_fullsized":"2023-10-16T20:06:17.217655+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:22:32.981812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"40'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.735204+0000","last_change":"2023-10-16T20:06:16.726366+0000","last_active":"2023-10-16T20:06:17.735204+0000","last_peered":"2023-10-16T20:06:17.735204+0000","last_clean":"2023-10-16T20:06:17.735204+0000","last_became_active":"2023-10-16T20:06:16.726086+0000","last_became_peered":"2023-10-16T20:06:16.726086+0000","last_unstale":"2023-10-16T20:06:17.735204+0000","last_undegraded":"2023-10-16T20:06:17.735204+0000","last_fullsized":"2023-10-16T20:06:17.735204+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:21:23.740779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.216516+0000","last_change":"2023-10-16T20:06:17.207345+0000","last_active":"2023-10-16T20:06:17.216516+0000","last_peered":"2023-10-16T20:06:17.216516+0000","last_clean":"2023-10-16T20:06:17.216516+0000","last_became_active":"2023-10-16T20:06:17.207082+0000","last_became_peered":"2023-10-16T20:06:17.207082+0000","last_unstale":"2023-10-16T20:06:17.216516+0000","last_undegraded":"2023-10-16T20:06:17.216516+0000","last_fullsized":"2023-10-16T20:06:17.216516+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:04:53.163558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"40'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.733912+0000","last_change":"2023-10-16T20:06:16.727808+0000","last_active":"2023-10-16T20:06:17.733912+0000","last_peered":"2023-10-16T20:06:17.733912+0000","last_clean":"2023-10-16T20:06:17.733912+0000","last_became_active":"2023-10-16T20:06:16.727400+0000","last_became_peered":"2023-10-16T20:06:16.727400+0000","last_unstale":"2023-10-16T20:06:17.733912+0000","last_undegraded":"2023-10-16T20:06:17.733912+0000","last_fullsized":"2023-10-16T20:06:17.733912+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:00.635492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.222291+0000","last_change":"2023-10-16T20:06:17.216271+0000","last_active":"2023-10-16T20:06:17.222291+0000","last_peered":"2023-10-16T20:06:17.222291+0000","last_clean":"2023-10-16T20:06:17.222291+0000","last_became_active":"2023-10-16T20:06:17.215201+0000","last_became_peered":"2023-10-16T20:06:17.215201+0000","last_unstale":"2023-10-16T20:06:17.222291+0000","last_undegraded":"2023-10-16T20:06:17.222291+0000","last_fullsized":"2023-10-16T20:06:17.222291+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:59:17.101302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"40'1","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357862+0000","last_change":"2023-10-16T20:06:16.724780+0000","last_active":"2023-10-16T20:06:18.357862+0000","last_peered":"2023-10-16T20:06:18.357862+0000","last_clean":"2023-10-16T20:06:18.357862+0000","last_became_active":"2023-10-16T20:06:16.724476+0000","last_became_peered":"2023-10-16T20:06:16.724476+0000","last_unstale":"2023-10-16T20:06:18.357862+0000","last_undegraded":"2023-10-16T20:06:18.357862+0000","last_fullsized":"2023-10-16T20:06:18.357862+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:19:37.004795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.358046+0000","last_change":"2023-10-16T20:06:17.209520+0000","last_active":"2023-10-16T20:06:18.358046+0000","last_peered":"2023-10-16T20:06:18.358046+0000","last_clean":"2023-10-16T20:06:18.358046+0000","last_became_active":"2023-10-16T20:06:17.209236+0000","last_became_peered":"2023-10-16T20:06:17.209236+0000","last_unstale":"2023-10-16T20:06:18.358046+0000","last_undegraded":"2023-10-16T20:06:18.358046+0000","last_fullsized":"2023-10-16T20:06:18.358046+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:31:26.166083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.196031+0000","last_change":"2023-10-16T20:06:16.734625+0000","last_active":"2023-10-16T20:06:17.196031+0000","last_peered":"2023-10-16T20:06:17.196031+0000","last_clean":"2023-10-16T20:06:17.196031+0000","last_became_active":"2023-10-16T20:06:16.734353+0000","last_became_peered":"2023-10-16T20:06:16.734353+0000","last_unstale":"2023-10-16T20:06:17.196031+0000","last_undegraded":"2023-10-16T20:06:17.196031+0000","last_fullsized":"2023-10-16T20:06:17.196031+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:38:39.778643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196838+0000","last_change":"2023-10-16T20:06:17.219357+0000","last_active":"2023-10-16T20:06:18.196838+0000","last_peered":"2023-10-16T20:06:18.196838+0000","last_clean":"2023-10-16T20:06:18.196838+0000","last_became_active":"2023-10-16T20:06:17.218995+0000","last_became_peered":"2023-10-16T20:06:17.218995+0000","last_unstale":"2023-10-16T20:06:18.196838+0000","last_undegraded":"2023-10-16T20:06:18.196838+0000","last_fullsized":"2023-10-16T20:06:18.196838+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:44:43.545772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"40'2","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357665+0000","last_change":"2023-10-16T20:06:16.732782+0000","last_active":"2023-10-16T20:06:18.357665+0000","last_peered":"2023-10-16T20:06:18.357665+0000","last_clean":"2023-10-16T20:06:18.357665+0000","last_became_active":"2023-10-16T20:06:16.732498+0000","last_became_peered":"2023-10-16T20:06:16.732498+0000","last_unstale":"2023-10-16T20:06:18.357665+0000","last_undegraded":"2023-10-16T20:06:18.357665+0000","last_fullsized":"2023-10-16T20:06:18.357665+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:21:27.852426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198137+0000","last_change":"2023-10-16T20:06:17.221923+0000","last_active":"2023-10-16T20:06:18.198137+0000","last_peered":"2023-10-16T20:06:18.198137+0000","last_clean":"2023-10-16T20:06:18.198137+0000","last_became_active":"2023-10-16T20:06:17.221536+0000","last_became_peered":"2023-10-16T20:06:17.221536+0000","last_unstale":"2023-10-16T20:06:18.198137+0000","last_undegraded":"2023-10-16T20:06:18.198137+0000","last_fullsized":"2023-10-16T20:06:18.198137+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:00:56.525352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357669+0000","last_change":"2023-10-16T20:06:16.732525+0000","last_active":"2023-10-16T20:06:18.357669+0000","last_peered":"2023-10-16T20:06:18.357669+0000","last_clean":"2023-10-16T20:06:18.357669+0000","last_became_active":"2023-10-16T20:06:16.732199+0000","last_became_peered":"2023-10-16T20:06:16.732199+0000","last_unstale":"2023-10-16T20:06:18.357669+0000","last_undegraded":"2023-10-16T20:06:18.357669+0000","last_fullsized":"2023-10-16T20:06:18.357669+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:12:42.667116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.216176+0000","last_change":"2023-10-16T20:06:17.209547+0000","last_active":"2023-10-16T20:06:17.216176+0000","last_peered":"2023-10-16T20:06:17.216176+0000","last_clean":"2023-10-16T20:06:17.216176+0000","last_became_active":"2023-10-16T20:06:17.209281+0000","last_became_peered":"2023-10-16T20:06:17.209281+0000","last_unstale":"2023-10-16T20:06:17.216176+0000","last_undegraded":"2023-10-16T20:06:17.216176+0000","last_fullsized":"2023-10-16T20:06:17.216176+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:58:21.190657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197648+0000","last_change":"2023-10-16T20:06:16.727775+0000","last_active":"2023-10-16T20:06:18.197648+0000","last_peered":"2023-10-16T20:06:18.197648+0000","last_clean":"2023-10-16T20:06:18.197648+0000","last_became_active":"2023-10-16T20:06:16.727492+0000","last_became_peered":"2023-10-16T20:06:16.727492+0000","last_unstale":"2023-10-16T20:06:18.197648+0000","last_undegraded":"2023-10-16T20:06:18.197648+0000","last_fullsized":"2023-10-16T20:06:18.197648+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:32:58.366700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.220851+0000","last_change":"2023-10-16T20:06:17.218801+0000","last_active":"2023-10-16T20:06:17.220851+0000","last_peered":"2023-10-16T20:06:17.220851+0000","last_clean":"2023-10-16T20:06:17.220851+0000","last_became_active":"2023-10-16T20:06:17.218567+0000","last_became_peered":"2023-10-16T20:06:17.218567+0000","last_unstale":"2023-10-16T20:06:17.220851+0000","last_undegraded":"2023-10-16T20:06:17.220851+0000","last_fullsized":"2023-10-16T20:06:17.220851+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:13.874932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357557+0000","last_change":"2023-10-16T20:06:16.727754+0000","last_active":"2023-10-16T20:06:18.357557+0000","last_peered":"2023-10-16T20:06:18.357557+0000","last_clean":"2023-10-16T20:06:18.357557+0000","last_became_active":"2023-10-16T20:06:16.727497+0000","last_became_peered":"2023-10-16T20:06:16.727497+0000","last_unstale":"2023-10-16T20:06:18.357557+0000","last_undegraded":"2023-10-16T20:06:18.357557+0000","last_fullsized":"2023-10-16T20:06:18.357557+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:08:12.647505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.209898+0000","last_change":"2023-10-16T20:06:17.201294+0000","last_active":"2023-10-16T20:06:17.209898+0000","last_peered":"2023-10-16T20:06:17.209898+0000","last_clean":"2023-10-16T20:06:17.209898+0000","last_became_active":"2023-10-16T20:06:17.200853+0000","last_became_peered":"2023-10-16T20:06:17.200853+0000","last_unstale":"2023-10-16T20:06:17.209898+0000","last_undegraded":"2023-10-16T20:06:17.209898+0000","last_fullsized":"2023-10-16T20:06:17.209898+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:01.646291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"40'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.732982+0000","last_change":"2023-10-16T20:06:16.733877+0000","last_active":"2023-10-16T20:06:17.732982+0000","last_peered":"2023-10-16T20:06:17.732982+0000","last_clean":"2023-10-16T20:06:17.732982+0000","last_became_active":"2023-10-16T20:06:16.733570+0000","last_became_peered":"2023-10-16T20:06:16.733570+0000","last_unstale":"2023-10-16T20:06:17.732982+0000","last_undegraded":"2023-10-16T20:06:17.732982+0000","last_fullsized":"2023-10-16T20:06:17.732982+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:38:29.780087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197199+0000","last_change":"2023-10-16T20:06:17.220488+0000","last_active":"2023-10-16T20:06:18.197199+0000","last_peered":"2023-10-16T20:06:18.197199+0000","last_clean":"2023-10-16T20:06:18.197199+0000","last_became_active":"2023-10-16T20:06:17.220060+0000","last_became_peered":"2023-10-16T20:06:17.220060+0000","last_unstale":"2023-10-16T20:06:18.197199+0000","last_undegraded":"2023-10-16T20:06:18.197199+0000","last_fullsized":"2023-10-16T20:06:18.197199+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:59:16.509971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357857+0000","last_change":"2023-10-16T20:06:16.726490+0000","last_active":"2023-10-16T20:06:18.357857+0000","last_peered":"2023-10-16T20:06:18.357857+0000","last_clean":"2023-10-16T20:06:18.357857+0000","last_became_active":"2023-10-16T20:06:16.726210+0000","last_became_peered":"2023-10-16T20:06:16.726210+0000","last_unstale":"2023-10-16T20:06:18.357857+0000","last_undegraded":"2023-10-16T20:06:18.357857+0000","last_fullsized":"2023-10-16T20:06:18.357857+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:13:14.492857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.217565+0000","last_change":"2023-10-16T20:06:17.213630+0000","last_active":"2023-10-16T20:06:17.217565+0000","last_peered":"2023-10-16T20:06:17.217565+0000","last_clean":"2023-10-16T20:06:17.217565+0000","last_became_active":"2023-10-16T20:06:17.213352+0000","last_became_peered":"2023-10-16T20:06:17.213352+0000","last_unstale":"2023-10-16T20:06:17.217565+0000","last_undegraded":"2023-10-16T20:06:17.217565+0000","last_fullsized":"2023-10-16T20:06:17.217565+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:18:14.480179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198443+0000","last_change":"2023-10-16T20:06:16.739495+0000","last_active":"2023-10-16T20:06:18.198443+0000","last_peered":"2023-10-16T20:06:18.198443+0000","last_clean":"2023-10-16T20:06:18.198443+0000","last_became_active":"2023-10-16T20:06:16.739215+0000","last_became_peered":"2023-10-16T20:06:16.739215+0000","last_unstale":"2023-10-16T20:06:18.198443+0000","last_undegraded":"2023-10-16T20:06:18.198443+0000","last_fullsized":"2023-10-16T20:06:18.198443+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:35:40.182424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198226+0000","last_change":"2023-10-16T20:06:17.212113+0000","last_active":"2023-10-16T20:06:18.198226+0000","last_peered":"2023-10-16T20:06:18.198226+0000","last_clean":"2023-10-16T20:06:18.198226+0000","last_became_active":"2023-10-16T20:06:17.211842+0000","last_became_peered":"2023-10-16T20:06:17.211842+0000","last_unstale":"2023-10-16T20:06:18.198226+0000","last_undegraded":"2023-10-16T20:06:18.198226+0000","last_fullsized":"2023-10-16T20:06:18.198226+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:34:14.717626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198360+0000","last_change":"2023-10-16T20:06:16.730618+0000","last_active":"2023-10-16T20:06:18.198360+0000","last_peered":"2023-10-16T20:06:18.198360+0000","last_clean":"2023-10-16T20:06:18.198360+0000","last_became_active":"2023-10-16T20:06:16.729429+0000","last_became_peered":"2023-10-16T20:06:16.729429+0000","last_unstale":"2023-10-16T20:06:18.198360+0000","last_undegraded":"2023-10-16T20:06:18.198360+0000","last_fullsized":"2023-10-16T20:06:18.198360+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:18:42.478975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.216410+0000","last_change":"2023-10-16T20:06:17.208914+0000","last_active":"2023-10-16T20:06:17.216410+0000","last_peered":"2023-10-16T20:06:17.216410+0000","last_clean":"2023-10-16T20:06:17.216410+0000","last_became_active":"2023-10-16T20:06:17.208609+0000","last_became_peered":"2023-10-16T20:06:17.208609+0000","last_unstale":"2023-10-16T20:06:17.216410+0000","last_undegraded":"2023-10-16T20:06:17.216410+0000","last_fullsized":"2023-10-16T20:06:17.216410+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:32:44.093091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197750+0000","last_change":"2023-10-16T20:06:16.733015+0000","last_active":"2023-10-16T20:06:18.197750+0000","last_peered":"2023-10-16T20:06:18.197750+0000","last_clean":"2023-10-16T20:06:18.197750+0000","last_became_active":"2023-10-16T20:06:16.731413+0000","last_became_peered":"2023-10-16T20:06:16.731413+0000","last_unstale":"2023-10-16T20:06:18.197750+0000","last_undegraded":"2023-10-16T20:06:18.197750+0000","last_fullsized":"2023-10-16T20:06:18.197750+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:59:16.090503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.222240+0000","last_change":"2023-10-16T20:06:17.214304+0000","last_active":"2023-10-16T20:06:17.222240+0000","last_peered":"2023-10-16T20:06:17.222240+0000","last_clean":"2023-10-16T20:06:17.222240+0000","last_became_active":"2023-10-16T20:06:17.213521+0000","last_became_peered":"2023-10-16T20:06:17.213521+0000","last_unstale":"2023-10-16T20:06:17.222240+0000","last_undegraded":"2023-10-16T20:06:17.222240+0000","last_fullsized":"2023-10-16T20:06:17.222240+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:27:10.715147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.202689+0000","last_change":"2023-10-16T20:06:16.730112+0000","last_active":"2023-10-16T20:06:17.202689+0000","last_peered":"2023-10-16T20:06:17.202689+0000","last_clean":"2023-10-16T20:06:17.202689+0000","last_became_active":"2023-10-16T20:06:16.729781+0000","last_became_peered":"2023-10-16T20:06:16.729781+0000","last_unstale":"2023-10-16T20:06:17.202689+0000","last_undegraded":"2023-10-16T20:06:17.202689+0000","last_fullsized":"2023-10-16T20:06:17.202689+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:39:05.076198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197008+0000","last_change":"2023-10-16T20:06:17.221997+0000","last_active":"2023-10-16T20:06:18.197008+0000","last_peered":"2023-10-16T20:06:18.197008+0000","last_clean":"2023-10-16T20:06:18.197008+0000","last_became_active":"2023-10-16T20:06:17.221704+0000","last_became_peered":"2023-10-16T20:06:17.221704+0000","last_unstale":"2023-10-16T20:06:18.197008+0000","last_undegraded":"2023-10-16T20:06:18.197008+0000","last_fullsized":"2023-10-16T20:06:18.197008+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:38:22.493819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.201889+0000","last_change":"2023-10-16T20:06:16.733171+0000","last_active":"2023-10-16T20:06:17.201889+0000","last_peered":"2023-10-16T20:06:17.201889+0000","last_clean":"2023-10-16T20:06:17.201889+0000","last_became_active":"2023-10-16T20:06:16.732869+0000","last_became_peered":"2023-10-16T20:06:16.732869+0000","last_unstale":"2023-10-16T20:06:17.201889+0000","last_undegraded":"2023-10-16T20:06:17.201889+0000","last_fullsized":"2023-10-16T20:06:17.201889+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:04:52.152758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.221418+0000","last_change":"2023-10-16T20:06:17.215196+0000","last_active":"2023-10-16T20:06:17.221418+0000","last_peered":"2023-10-16T20:06:17.221418+0000","last_clean":"2023-10-16T20:06:17.221418+0000","last_became_active":"2023-10-16T20:06:17.214556+0000","last_became_peered":"2023-10-16T20:06:17.214556+0000","last_unstale":"2023-10-16T20:06:17.221418+0000","last_undegraded":"2023-10-16T20:06:17.221418+0000","last_fullsized":"2023-10-16T20:06:17.221418+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:24:29.941143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196810+0000","last_change":"2023-10-16T20:06:16.728925+0000","last_active":"2023-10-16T20:06:18.196810+0000","last_peered":"2023-10-16T20:06:18.196810+0000","last_clean":"2023-10-16T20:06:18.196810+0000","last_became_active":"2023-10-16T20:06:16.728663+0000","last_became_peered":"2023-10-16T20:06:16.728663+0000","last_unstale":"2023-10-16T20:06:18.196810+0000","last_undegraded":"2023-10-16T20:06:18.196810+0000","last_fullsized":"2023-10-16T20:06:18.196810+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:50:31.301972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":182,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.208038+0000","last_change":"2023-10-16T20:04:46.335822+0000","last_active":"2023-10-16T20:06:17.208038+0000","last_peered":"2023-10-16T20:06:17.208038+0000","last_clean":"2023-10-16T20:06:17.208038+0000","last_became_active":"2023-10-16T20:04:46.334617+0000","last_became_peered":"2023-10-16T20:04:46.334617+0000","last_unstale":"2023-10-16T20:06:17.208038+0000","last_undegraded":"2023-10-16T20:06:17.208038+0000","last_fullsized":"2023-10-16T20:06:17.208038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:45.306240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:45.306240+0000","last_clean_scrub_stamp":"2023-10-16T20:04:45.306240+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:42:11.153721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509462,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1048576,"data_stored":783264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.86199999999999999}]},{"osd":1,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.4900000000000002,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":0.752}]},{"osd":2,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.0550000000000002,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.80300000000000005}]},{"osd":3,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.629,"5min":2.629,"15min":2.629},"last":0.67400000000000004}]},{"osd":4,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":0.63200000000000001}]},{"osd":5,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.8809999999999998,"5min":2.8809999999999998,"15min":2.8809999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.65400000000000003}]},{"osd":6,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.56999999999999995}]}]},{"osd":6,"up_from":15,"seq":64424509463,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":192896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":4.5880000000000001,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":4.6139999999999999,"5min":4.6139999999999999,"15min":4.6139999999999999},"last":0.629}]},{"osd":1,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":4.6639999999999997,"5min":4.6639999999999997,"15min":4.6639999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":4.8140000000000001,"5min":4.8140000000000001,"15min":4.8140000000000001},"last":0.60899999999999999}]},{"osd":2,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":5.3769999999999998,"5min":5.3769999999999998,"15min":5.3769999999999998},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.79000000000000004}]},{"osd":3,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":4.5460000000000003,"5min":4.5460000000000003,"15min":4.5460000000000003},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.66100000000000003}]},{"osd":4,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.51900000000000002}]},{"osd":5,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.76400000000000001}]},{"osd":7,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.88500000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509461,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":462848,"data_stored":193008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.70999999999999996}]},{"osd":1,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.623},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.76200000000000001}]},{"osd":2,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.747},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.67800000000000005}]},{"osd":3,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.73199999999999998}]},{"osd":4,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.50700000000000001}]},{"osd":6,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.56000000000000005}]},{"osd":7,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.53800000000000003}]}]},{"osd":4,"up_from":15,"seq":64424509461,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31632,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340208,"statfs":{"total":96636764160,"available":96604372992,"internally_reserved":0,"allocated":454656,"data_stored":192896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":3.835,"5min":3.835,"15min":3.835},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":3.7610000000000001,"5min":3.7610000000000001,"15min":3.7610000000000001},"last":0.75700000000000001}]},{"osd":1,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.57599999999999996}]},{"osd":2,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.628},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.60799999999999998}]},{"osd":3,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.72399999999999998}]},{"osd":5,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.5}]},{"osd":6,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.55100000000000005}]},{"osd":7,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.8919999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.74099999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509462,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":194542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.47599999999999998}]},{"osd":1,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.497}]},{"osd":2,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.54200000000000004}]},{"osd":4,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.88200000000000001}]},{"osd":5,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.876}]},{"osd":6,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.95499999999999996}]},{"osd":7,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.75800000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509461,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":192896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.53400000000000003}]},{"osd":1,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.436}]},{"osd":3,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.48399999999999999}]},{"osd":4,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.59699999999999998}]},{"osd":5,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.63}]},{"osd":6,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.80100000000000005}]},{"osd":7,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.70499999999999996}]}]},{"osd":1,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":192896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.867,"5min":3.867,"15min":3.867},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.52700000000000002}]},{"osd":2,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":5.6040000000000001,"5min":5.6040000000000001,"15min":5.6040000000000001},"last":0.434},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":7.8380000000000001,"5min":7.8380000000000001,"15min":7.8380000000000001},"last":0.45100000000000001}]},{"osd":3,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":5.2039999999999997,"5min":5.2039999999999997,"15min":5.2039999999999997},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":3.9470000000000001,"5min":3.9470000000000001,"15min":3.9470000000000001},"last":0.47399999999999998}]},{"osd":4,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":4.1500000000000004,"5min":4.1500000000000004,"15min":4.1500000000000004},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":4.2619999999999996,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":0.79400000000000004}]},{"osd":5,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":4.0910000000000002,"5min":4.0910000000000002,"15min":4.0910000000000002},"last":0.86099999999999999}]},{"osd":6,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":4.21,"5min":4.21,"15min":4.21},"last":0.88},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.83399999999999996}]},{"osd":7,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":4.1849999999999996,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":0.82099999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28072,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343768,"statfs":{"total":96636764160,"available":96608018432,"internally_reserved":0,"allocated":1064960,"data_stored":789943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":3.9289999999999998,"5min":3.9289999999999998,"15min":3.9289999999999998},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":4.0910000000000002,"5min":4.0910000000000002,"15min":4.0910000000000002},"last":0.36899999999999999}]},{"osd":2,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":4.1520000000000001,"5min":4.1520000000000001,"15min":4.1520000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":4.1879999999999997,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.45600000000000002}]},{"osd":3,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":4.3440000000000003,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.438}]},{"osd":4,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":4.5250000000000004,"5min":4.5250000000000004,"15min":4.5250000000000004},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.68300000000000005}]},{"osd":5,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":3.9990000000000001,"5min":3.9990000000000001,"15min":3.9990000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":4.4779999999999998,"5min":4.4779999999999998,"15min":4.4779999999999998},"last":0.70799999999999996}]},{"osd":6,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":3.956,"5min":3.956,"15min":3.956},"last":0.52100000000000002}]},{"osd":7,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":4.0289999999999999,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":4.5640000000000001,"5min":4.5640000000000001,"15min":4.5640000000000001},"last":0.66600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T20:06:26.302 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T20:06:26.329 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T20:06:26.354 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T20:06:26.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.451+0000 7fd0d6d3e640 1 Processor -- start 2023-10-16T20:06:26.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.451+0000 7fd0d6d3e640 1 -- start start 2023-10-16T20:06:26.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.451+0000 7fd0d6d3e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd0d0069e90 0x7fd0d006a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:26.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.451+0000 7fd0d6d3e640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0d006a830 con 0x7fd0d01599f0 2023-10-16T20:06:26.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.451+0000 7fd0d6d3e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0d00b1260 con 0x7fd0d00b05a0 2023-10-16T20:06:26.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.451+0000 7fd0d6d3e640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0d00b3140 con 0x7fd0d0069e90 2023-10-16T20:06:26.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.451+0000 7fd0d4ada640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fd0d01599f0 0x7fd0d00b2a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:51146/0 (socket says 172.21.15.88:51146) 2023-10-16T20:06:26.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.451+0000 7fd0d4ada640 1 -- 172.21.15.88:0/2541740477 learned_addr learned my addr 172.21.15.88:0/2541740477 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:26.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.451+0000 7fd0c7fff640 1 --2- 172.21.15.88:0/2541740477 >> v2:172.21.15.94:3300/0 conn(0x7fd0d0069e90 0x7fd0d006a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:26.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.451+0000 7fd0c77fe640 1 -- 172.21.15.88:0/2541740477 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1411545145 0 0) 0x7fd0d006a830 con 0x7fd0d01599f0 2023-10-16T20:06:26.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.451+0000 7fd0c77fe640 1 -- 172.21.15.88:0/2541740477 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0b8003530 con 0x7fd0d01599f0 2023-10-16T20:06:26.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.455+0000 7fd0c77fe640 1 -- 172.21.15.88:0/2541740477 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3602369591 0 0) 0x7fd0b8003530 con 0x7fd0d01599f0 2023-10-16T20:06:26.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.455+0000 7fd0c7fff640 1 -- 172.21.15.88:0/2541740477 >> v1:172.21.15.94:6789/0 conn(0x7fd0d00b05a0 legacy=0x7fd0d00b0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:26.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.455+0000 7fd0c7fff640 1 -- 172.21.15.88:0/2541740477 >> v1:172.21.15.88:6789/0 conn(0x7fd0d01599f0 legacy=0x7fd0d00b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:26.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.455+0000 7fd0c7fff640 1 -- 172.21.15.88:0/2541740477 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d00ae960 con 0x7fd0d0069e90 2023-10-16T20:06:26.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.455+0000 7fd0c7fff640 1 --2- 172.21.15.88:0/2541740477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0d0069e90 0x7fd0d006a2f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00097c0 tx=0x7fd0c001c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2d91f88cde8fd3f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:26.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.455+0000 7fd0c77fe640 1 -- 172.21.15.88:0/2541740477 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc001680 con 0x7fd0d0069e90 2023-10-16T20:06:26.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.455+0000 7fd0c77fe640 1 -- 172.21.15.88:0/2541740477 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c00a7050 con 0x7fd0d0069e90 2023-10-16T20:06:26.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.455+0000 7fd0c77fe640 1 -- 172.21.15.88:0/2541740477 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c00ab620 con 0x7fd0d0069e90 2023-10-16T20:06:26.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.455+0000 7fd0d6d3e640 1 -- 172.21.15.88:0/2541740477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0d0069e90 msgr2=0x7fd0d006a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:26.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.455+0000 7fd0d6d3e640 1 --2- 172.21.15.88:0/2541740477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0d0069e90 0x7fd0d006a2f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00097c0 tx=0x7fd0c001c370 comp rx=0 tx=0).stop 2023-10-16T20:06:26.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.459+0000 7fd0d6d3e640 1 -- 172.21.15.88:0/2541740477 shutdown_connections 2023-10-16T20:06:26.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.459+0000 7fd0d6d3e640 1 --2- 172.21.15.88:0/2541740477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0d0069e90 0x7fd0d006a2f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:26.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.459+0000 7fd0d6d3e640 1 -- 172.21.15.88:0/2541740477 >> 172.21.15.88:0/2541740477 conn(0x7fd0d00ce5e0 msgr2=0x7fd0d00ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:26.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.459+0000 7fd0d6d3e640 1 -- 172.21.15.88:0/2541740477 shutdown_connections 2023-10-16T20:06:26.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.459+0000 7fd0d6d3e640 1 -- 172.21.15.88:0/2541740477 wait complete. 2023-10-16T20:06:26.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.459+0000 7fd0d6d3e640 1 Processor -- start 2023-10-16T20:06:26.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.459+0000 7fd0d6d3e640 1 -- start start 2023-10-16T20:06:26.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.459+0000 7fd0d6d3e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0d0069e90 0x7fd0d0144a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:26.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.459+0000 7fd0d6d3e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0d00b05a0 0x7fd0d01579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:26.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.459+0000 7fd0d6d3e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0d01599f0 0x7fd0d0157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:26.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.459+0000 7fd0d6d3e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d0144fc0 con 0x7fd0d01599f0 2023-10-16T20:06:26.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.459+0000 7fd0d6d3e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d0146600 con 0x7fd0d0069e90 2023-10-16T20:06:26.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.459+0000 7fd0d6d3e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d01468e0 con 0x7fd0d00b05a0 2023-10-16T20:06:26.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.459+0000 7fd0d52db640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0d01599f0 0x7fd0d0157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:26.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.459+0000 7fd0d4ada640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0d0069e90 0x7fd0d0144a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:26.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.459+0000 7fd0d52db640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0d01599f0 0x7fd0d0157f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57574/0 (socket says 172.21.15.88:57574) 2023-10-16T20:06:26.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.459+0000 7fd0d52db640 1 -- 172.21.15.88:0/1081664365 learned_addr learned my addr 172.21.15.88:0/1081664365 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:26.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.459+0000 7fd0c7fff640 1 --2- 172.21.15.88:0/1081664365 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0d00b05a0 0x7fd0d01579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:26.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.459+0000 7fd0d52db640 1 -- 172.21.15.88:0/1081664365 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0d00b05a0 msgr2=0x7fd0d01579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:26.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.459+0000 7fd0d52db640 1 --2- 172.21.15.88:0/1081664365 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0d00b05a0 0x7fd0d01579e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:26.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.459+0000 7fd0d52db640 1 -- 172.21.15.88:0/1081664365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0d0069e90 msgr2=0x7fd0d0144a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:26.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.459+0000 7fd0d52db640 1 --2- 172.21.15.88:0/1081664365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0d0069e90 0x7fd0d0144a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:26.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.459+0000 7fd0d52db640 1 -- 172.21.15.88:0/1081664365 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c0009420 con 0x7fd0d01599f0 2023-10-16T20:06:26.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.463+0000 7fd0d52db640 1 --2- 172.21.15.88:0/1081664365 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0d01599f0 0x7fd0d0157f20 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800c730 tx=0x7fd0c800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:26.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.463+0000 7fd0c57fa640 1 -- 172.21.15.88:0/1081664365 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8003b80 con 0x7fd0d01599f0 2023-10-16T20:06:26.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.463+0000 7fd0c57fa640 1 -- 172.21.15.88:0/1081664365 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c8003ce0 con 0x7fd0d01599f0 2023-10-16T20:06:26.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.463+0000 7fd0c57fa640 1 -- 172.21.15.88:0/1081664365 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c800aa30 con 0x7fd0d01599f0 2023-10-16T20:06:26.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.463+0000 7fd0d6d3e640 1 -- 172.21.15.88:0/1081664365 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0d0146bc0 con 0x7fd0d01599f0 2023-10-16T20:06:26.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.463+0000 7fd0d6d3e640 1 -- 172.21.15.88:0/1081664365 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0d01473c0 con 0x7fd0d01599f0 2023-10-16T20:06:26.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.463+0000 7fd0d6d3e640 1 -- 172.21.15.88:0/1081664365 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0d00706d0 con 0x7fd0d01599f0 2023-10-16T20:06:26.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.463+0000 7fd0c57fa640 1 -- 172.21.15.88:0/1081664365 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd0c800ab90 con 0x7fd0d01599f0 2023-10-16T20:06:26.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.463+0000 7fd0c57fa640 1 --2- 172.21.15.88:0/1081664365 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd0bc0664a0 0x7fd0bc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:26.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.463+0000 7fd0c57fa640 1 -- 172.21.15.88:0/1081664365 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fd0c8083a90 con 0x7fd0d01599f0 2023-10-16T20:06:26.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.463+0000 7fd0d4ada640 1 --2- 172.21.15.88:0/1081664365 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd0bc0664a0 0x7fd0bc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:26.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.467+0000 7fd0d4ada640 1 --2- 172.21.15.88:0/1081664365 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd0bc0664a0 0x7fd0bc068960 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc000f70 tx=0x7fd0cc001e10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:26.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.467+0000 7fd0c57fa640 1 -- 172.21.15.88:0/1081664365 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd0c8050e50 con 0x7fd0d01599f0 2023-10-16T20:06:26.618 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.615+0000 7fd0d6d3e640 1 -- 172.21.15.88:0/1081664365 --> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd0d00af200 con 0x7fd0bc0664a0 2023-10-16T20:06:26.625 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.623+0000 7fd0c57fa640 1 -- 172.21.15.88:0/1081664365 <== mgr.4110 v2:172.21.15.88:6832/4078483333 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289587 (secure 0 0 0) 0x7fd0cc00f9a0 con 0x7fd0bc0664a0 2023-10-16T20:06:26.628 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:06:26.630 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2023-10-16T20:06:26.631 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.627+0000 7fd0d6d3e640 1 -- 172.21.15.88:0/1081664365 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd0bc0664a0 msgr2=0x7fd0bc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:26.631 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.627+0000 7fd0d6d3e640 1 --2- 172.21.15.88:0/1081664365 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd0bc0664a0 0x7fd0bc068960 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc000f70 tx=0x7fd0cc001e10 comp rx=0 tx=0).stop 2023-10-16T20:06:26.632 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.627+0000 7fd0d6d3e640 1 -- 172.21.15.88:0/1081664365 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0d01599f0 msgr2=0x7fd0d0157f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:26.632 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.627+0000 7fd0d6d3e640 1 --2- 172.21.15.88:0/1081664365 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0d01599f0 0x7fd0d0157f20 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800c730 tx=0x7fd0c800cbf0 comp rx=0 tx=0).stop 2023-10-16T20:06:26.632 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.627+0000 7fd0d6d3e640 1 -- 172.21.15.88:0/1081664365 shutdown_connections 2023-10-16T20:06:26.632 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.627+0000 7fd0d6d3e640 1 --2- 172.21.15.88:0/1081664365 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd0bc0664a0 0x7fd0bc068960 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:26.632 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.627+0000 7fd0d6d3e640 1 --2- 172.21.15.88:0/1081664365 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd0d00b05a0 0x7fd0d01579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:26.633 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.627+0000 7fd0d6d3e640 1 --2- 172.21.15.88:0/1081664365 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0d01599f0 0x7fd0d0157f20 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:26.633 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.627+0000 7fd0d6d3e640 1 --2- 172.21.15.88:0/1081664365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0d0069e90 0x7fd0d0144a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:26.633 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.627+0000 7fd0d6d3e640 1 -- 172.21.15.88:0/1081664365 >> 172.21.15.88:0/1081664365 conn(0x7fd0d00ce5e0 msgr2=0x7fd0d014be30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:26.633 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.627+0000 7fd0d6d3e640 1 -- 172.21.15.88:0/1081664365 shutdown_connections 2023-10-16T20:06:26.633 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.627+0000 7fd0d6d3e640 1 -- 172.21.15.88:0/1081664365 wait complete. 2023-10-16T20:06:26.648 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2023-10-16T20:06:26.463875+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233536,"kb_used_data":4872,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754741184,"statfs":{"total":773094113280,"available":772854972416,"internally_reserved":0,"allocated":4988928,"data_stored":2781884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":234016313},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.269788"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.858534+0000","last_change":"2023-10-16T20:06:17.217901+0000","last_active":"2023-10-16T20:06:18.858534+0000","last_peered":"2023-10-16T20:06:18.858534+0000","last_clean":"2023-10-16T20:06:18.858534+0000","last_became_active":"2023-10-16T20:06:17.217583+0000","last_became_peered":"2023-10-16T20:06:17.217583+0000","last_unstale":"2023-10-16T20:06:18.858534+0000","last_undegraded":"2023-10-16T20:06:18.858534+0000","last_fullsized":"2023-10-16T20:06:18.858534+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:05:25.874296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.858458+0000","last_change":"2023-10-16T20:06:17.211493+0000","last_active":"2023-10-16T20:06:18.858458+0000","last_peered":"2023-10-16T20:06:18.858458+0000","last_clean":"2023-10-16T20:06:18.858458+0000","last_became_active":"2023-10-16T20:06:17.211196+0000","last_became_peered":"2023-10-16T20:06:17.211196+0000","last_unstale":"2023-10-16T20:06:18.858458+0000","last_undegraded":"2023-10-16T20:06:18.858458+0000","last_fullsized":"2023-10-16T20:06:18.858458+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:59:16.509971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198456+0000","last_change":"2023-10-16T20:06:17.211314+0000","last_active":"2023-10-16T20:06:18.198456+0000","last_peered":"2023-10-16T20:06:18.198456+0000","last_clean":"2023-10-16T20:06:18.198456+0000","last_became_active":"2023-10-16T20:06:17.210991+0000","last_became_peered":"2023-10-16T20:06:17.210991+0000","last_unstale":"2023-10-16T20:06:18.198456+0000","last_undegraded":"2023-10-16T20:06:18.198456+0000","last_fullsized":"2023-10-16T20:06:18.198456+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:07:43.835803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357788+0000","last_change":"2023-10-16T20:06:17.216483+0000","last_active":"2023-10-16T20:06:18.357788+0000","last_peered":"2023-10-16T20:06:18.357788+0000","last_clean":"2023-10-16T20:06:18.357788+0000","last_became_active":"2023-10-16T20:06:17.216202+0000","last_became_peered":"2023-10-16T20:06:17.216202+0000","last_unstale":"2023-10-16T20:06:18.357788+0000","last_undegraded":"2023-10-16T20:06:18.357788+0000","last_fullsized":"2023-10-16T20:06:18.357788+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:32:02.453729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357619+0000","last_change":"2023-10-16T20:06:17.208325+0000","last_active":"2023-10-16T20:06:18.357619+0000","last_peered":"2023-10-16T20:06:18.357619+0000","last_clean":"2023-10-16T20:06:18.357619+0000","last_became_active":"2023-10-16T20:06:17.208061+0000","last_became_peered":"2023-10-16T20:06:17.208061+0000","last_unstale":"2023-10-16T20:06:18.357619+0000","last_undegraded":"2023-10-16T20:06:18.357619+0000","last_fullsized":"2023-10-16T20:06:18.357619+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:02:28.165753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.857741+0000","last_change":"2023-10-16T20:06:17.208416+0000","last_active":"2023-10-16T20:06:18.857741+0000","last_peered":"2023-10-16T20:06:18.857741+0000","last_clean":"2023-10-16T20:06:18.857741+0000","last_became_active":"2023-10-16T20:06:17.207713+0000","last_became_peered":"2023-10-16T20:06:17.207713+0000","last_unstale":"2023-10-16T20:06:18.857741+0000","last_undegraded":"2023-10-16T20:06:18.857741+0000","last_fullsized":"2023-10-16T20:06:18.857741+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:22:32.981812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.698758+0000","last_change":"2023-10-16T20:06:17.206736+0000","last_active":"2023-10-16T20:06:18.698758+0000","last_peered":"2023-10-16T20:06:18.698758+0000","last_clean":"2023-10-16T20:06:18.698758+0000","last_became_active":"2023-10-16T20:06:17.206435+0000","last_became_peered":"2023-10-16T20:06:17.206435+0000","last_unstale":"2023-10-16T20:06:18.698758+0000","last_undegraded":"2023-10-16T20:06:18.698758+0000","last_fullsized":"2023-10-16T20:06:18.698758+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:44:14.727872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.857988+0000","last_change":"2023-10-16T20:06:17.214147+0000","last_active":"2023-10-16T20:06:18.857988+0000","last_peered":"2023-10-16T20:06:18.857988+0000","last_clean":"2023-10-16T20:06:18.857988+0000","last_became_active":"2023-10-16T20:06:17.213998+0000","last_became_peered":"2023-10-16T20:06:17.213998+0000","last_unstale":"2023-10-16T20:06:18.857988+0000","last_undegraded":"2023-10-16T20:06:18.857988+0000","last_fullsized":"2023-10-16T20:06:18.857988+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:44:14.727872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198636+0000","last_change":"2023-10-16T20:06:17.213204+0000","last_active":"2023-10-16T20:06:18.198636+0000","last_peered":"2023-10-16T20:06:18.198636+0000","last_clean":"2023-10-16T20:06:18.198636+0000","last_became_active":"2023-10-16T20:06:17.212563+0000","last_became_peered":"2023-10-16T20:06:17.212563+0000","last_unstale":"2023-10-16T20:06:18.198636+0000","last_undegraded":"2023-10-16T20:06:18.198636+0000","last_fullsized":"2023-10-16T20:06:18.198636+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:25:04.009896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.699265+0000","last_change":"2023-10-16T20:06:17.218989+0000","last_active":"2023-10-16T20:06:18.699265+0000","last_peered":"2023-10-16T20:06:18.699265+0000","last_clean":"2023-10-16T20:06:18.699265+0000","last_became_active":"2023-10-16T20:06:17.218726+0000","last_became_peered":"2023-10-16T20:06:17.218726+0000","last_unstale":"2023-10-16T20:06:18.699265+0000","last_undegraded":"2023-10-16T20:06:18.699265+0000","last_fullsized":"2023-10-16T20:06:18.699265+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:51:34.183431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.699116+0000","last_change":"2023-10-16T20:06:17.217372+0000","last_active":"2023-10-16T20:06:18.699116+0000","last_peered":"2023-10-16T20:06:18.699116+0000","last_clean":"2023-10-16T20:06:18.699116+0000","last_became_active":"2023-10-16T20:06:17.216730+0000","last_became_peered":"2023-10-16T20:06:17.216730+0000","last_unstale":"2023-10-16T20:06:18.699116+0000","last_undegraded":"2023-10-16T20:06:18.699116+0000","last_fullsized":"2023-10-16T20:06:18.699116+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:59:16.509971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198265+0000","last_change":"2023-10-16T20:06:17.214475+0000","last_active":"2023-10-16T20:06:18.198265+0000","last_peered":"2023-10-16T20:06:18.198265+0000","last_clean":"2023-10-16T20:06:18.198265+0000","last_became_active":"2023-10-16T20:06:17.214211+0000","last_became_peered":"2023-10-16T20:06:17.214211+0000","last_unstale":"2023-10-16T20:06:18.198265+0000","last_undegraded":"2023-10-16T20:06:18.198265+0000","last_fullsized":"2023-10-16T20:06:18.198265+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:02:29.473591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.698800+0000","last_change":"2023-10-16T20:06:17.218455+0000","last_active":"2023-10-16T20:06:18.698800+0000","last_peered":"2023-10-16T20:06:18.698800+0000","last_clean":"2023-10-16T20:06:18.698800+0000","last_became_active":"2023-10-16T20:06:17.217565+0000","last_became_peered":"2023-10-16T20:06:17.217565+0000","last_unstale":"2023-10-16T20:06:18.698800+0000","last_undegraded":"2023-10-16T20:06:18.698800+0000","last_fullsized":"2023-10-16T20:06:18.698800+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:44:43.545772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.857880+0000","last_change":"2023-10-16T20:06:17.214285+0000","last_active":"2023-10-16T20:06:18.857880+0000","last_peered":"2023-10-16T20:06:18.857880+0000","last_clean":"2023-10-16T20:06:18.857880+0000","last_became_active":"2023-10-16T20:06:17.213551+0000","last_became_peered":"2023-10-16T20:06:17.213551+0000","last_unstale":"2023-10-16T20:06:18.857880+0000","last_undegraded":"2023-10-16T20:06:18.857880+0000","last_fullsized":"2023-10-16T20:06:18.857880+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:23:28.625212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.698773+0000","last_change":"2023-10-16T20:06:17.208025+0000","last_active":"2023-10-16T20:06:18.698773+0000","last_peered":"2023-10-16T20:06:18.698773+0000","last_clean":"2023-10-16T20:06:18.698773+0000","last_became_active":"2023-10-16T20:06:17.207739+0000","last_became_peered":"2023-10-16T20:06:17.207739+0000","last_unstale":"2023-10-16T20:06:18.698773+0000","last_undegraded":"2023-10-16T20:06:18.698773+0000","last_fullsized":"2023-10-16T20:06:18.698773+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:03:40.257975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196502+0000","last_change":"2023-10-16T20:06:17.222232+0000","last_active":"2023-10-16T20:06:18.196502+0000","last_peered":"2023-10-16T20:06:18.196502+0000","last_clean":"2023-10-16T20:06:18.196502+0000","last_became_active":"2023-10-16T20:06:17.221892+0000","last_became_peered":"2023-10-16T20:06:17.221892+0000","last_unstale":"2023-10-16T20:06:18.196502+0000","last_undegraded":"2023-10-16T20:06:18.196502+0000","last_fullsized":"2023-10-16T20:06:18.196502+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:58:45.132214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197149+0000","last_change":"2023-10-16T20:06:17.224446+0000","last_active":"2023-10-16T20:06:18.197149+0000","last_peered":"2023-10-16T20:06:18.197149+0000","last_clean":"2023-10-16T20:06:18.197149+0000","last_became_active":"2023-10-16T20:06:17.224113+0000","last_became_peered":"2023-10-16T20:06:17.224113+0000","last_unstale":"2023-10-16T20:06:18.197149+0000","last_undegraded":"2023-10-16T20:06:18.197149+0000","last_fullsized":"2023-10-16T20:06:18.197149+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:37:43.948080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198392+0000","last_change":"2023-10-16T20:06:17.214181+0000","last_active":"2023-10-16T20:06:18.198392+0000","last_peered":"2023-10-16T20:06:18.198392+0000","last_clean":"2023-10-16T20:06:18.198392+0000","last_became_active":"2023-10-16T20:06:17.213876+0000","last_became_peered":"2023-10-16T20:06:17.213876+0000","last_unstale":"2023-10-16T20:06:18.198392+0000","last_undegraded":"2023-10-16T20:06:18.198392+0000","last_fullsized":"2023-10-16T20:06:18.198392+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:39:06.086998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357861+0000","last_change":"2023-10-16T20:06:17.205826+0000","last_active":"2023-10-16T20:06:18.357861+0000","last_peered":"2023-10-16T20:06:18.357861+0000","last_clean":"2023-10-16T20:06:18.357861+0000","last_became_active":"2023-10-16T20:06:17.204888+0000","last_became_peered":"2023-10-16T20:06:17.204888+0000","last_unstale":"2023-10-16T20:06:18.357861+0000","last_undegraded":"2023-10-16T20:06:18.357861+0000","last_fullsized":"2023-10-16T20:06:18.357861+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:29:06.675309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197777+0000","last_change":"2023-10-16T20:06:17.217103+0000","last_active":"2023-10-16T20:06:18.197777+0000","last_peered":"2023-10-16T20:06:18.197777+0000","last_clean":"2023-10-16T20:06:18.197777+0000","last_became_active":"2023-10-16T20:06:17.216717+0000","last_became_peered":"2023-10-16T20:06:17.216717+0000","last_unstale":"2023-10-16T20:06:18.197777+0000","last_undegraded":"2023-10-16T20:06:18.197777+0000","last_fullsized":"2023-10-16T20:06:18.197777+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:20:06.934366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.698773+0000","last_change":"2023-10-16T20:06:17.201875+0000","last_active":"2023-10-16T20:06:18.698773+0000","last_peered":"2023-10-16T20:06:18.698773+0000","last_clean":"2023-10-16T20:06:18.698773+0000","last_became_active":"2023-10-16T20:06:17.200537+0000","last_became_peered":"2023-10-16T20:06:17.200537+0000","last_unstale":"2023-10-16T20:06:18.698773+0000","last_undegraded":"2023-10-16T20:06:18.698773+0000","last_fullsized":"2023-10-16T20:06:18.698773+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:31:31.346356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.858282+0000","last_change":"2023-10-16T20:06:17.211432+0000","last_active":"2023-10-16T20:06:18.858282+0000","last_peered":"2023-10-16T20:06:18.858282+0000","last_clean":"2023-10-16T20:06:18.858282+0000","last_became_active":"2023-10-16T20:06:17.211100+0000","last_became_peered":"2023-10-16T20:06:17.211100+0000","last_unstale":"2023-10-16T20:06:18.858282+0000","last_undegraded":"2023-10-16T20:06:18.858282+0000","last_fullsized":"2023-10-16T20:06:18.858282+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:27:39.178553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196828+0000","last_change":"2023-10-16T20:06:17.225476+0000","last_active":"2023-10-16T20:06:18.196828+0000","last_peered":"2023-10-16T20:06:18.196828+0000","last_clean":"2023-10-16T20:06:18.196828+0000","last_became_active":"2023-10-16T20:06:17.224667+0000","last_became_peered":"2023-10-16T20:06:17.224667+0000","last_unstale":"2023-10-16T20:06:18.196828+0000","last_undegraded":"2023-10-16T20:06:18.196828+0000","last_fullsized":"2023-10-16T20:06:18.196828+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:31:31.346356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357610+0000","last_change":"2023-10-16T20:06:17.216704+0000","last_active":"2023-10-16T20:06:18.357610+0000","last_peered":"2023-10-16T20:06:18.357610+0000","last_clean":"2023-10-16T20:06:18.357610+0000","last_became_active":"2023-10-16T20:06:17.216341+0000","last_became_peered":"2023-10-16T20:06:17.216341+0000","last_unstale":"2023-10-16T20:06:18.357610+0000","last_undegraded":"2023-10-16T20:06:18.357610+0000","last_fullsized":"2023-10-16T20:06:18.357610+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:14:46.135352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198413+0000","last_change":"2023-10-16T20:06:17.216103+0000","last_active":"2023-10-16T20:06:18.198413+0000","last_peered":"2023-10-16T20:06:18.198413+0000","last_clean":"2023-10-16T20:06:18.198413+0000","last_became_active":"2023-10-16T20:06:17.215844+0000","last_became_peered":"2023-10-16T20:06:17.215844+0000","last_unstale":"2023-10-16T20:06:18.198413+0000","last_undegraded":"2023-10-16T20:06:18.198413+0000","last_fullsized":"2023-10-16T20:06:18.198413+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:01.646291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.858591+0000","last_change":"2023-10-16T20:06:17.207691+0000","last_active":"2023-10-16T20:06:18.858591+0000","last_peered":"2023-10-16T20:06:18.858591+0000","last_clean":"2023-10-16T20:06:18.858591+0000","last_became_active":"2023-10-16T20:06:17.207437+0000","last_became_peered":"2023-10-16T20:06:17.207437+0000","last_unstale":"2023-10-16T20:06:18.858591+0000","last_undegraded":"2023-10-16T20:06:18.858591+0000","last_fullsized":"2023-10-16T20:06:18.858591+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:25:13.284821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198320+0000","last_change":"2023-10-16T20:06:17.211359+0000","last_active":"2023-10-16T20:06:18.198320+0000","last_peered":"2023-10-16T20:06:18.198320+0000","last_clean":"2023-10-16T20:06:18.198320+0000","last_became_active":"2023-10-16T20:06:17.210506+0000","last_became_peered":"2023-10-16T20:06:17.210506+0000","last_unstale":"2023-10-16T20:06:18.198320+0000","last_undegraded":"2023-10-16T20:06:18.198320+0000","last_fullsized":"2023-10-16T20:06:18.198320+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:32:44.093091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.699126+0000","last_change":"2023-10-16T20:06:17.219679+0000","last_active":"2023-10-16T20:06:18.699126+0000","last_peered":"2023-10-16T20:06:18.699126+0000","last_clean":"2023-10-16T20:06:18.699126+0000","last_became_active":"2023-10-16T20:06:17.214175+0000","last_became_peered":"2023-10-16T20:06:17.214175+0000","last_unstale":"2023-10-16T20:06:18.699126+0000","last_undegraded":"2023-10-16T20:06:18.699126+0000","last_fullsized":"2023-10-16T20:06:18.699126+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:37:43.948080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197630+0000","last_change":"2023-10-16T20:06:17.210945+0000","last_active":"2023-10-16T20:06:18.197630+0000","last_peered":"2023-10-16T20:06:18.197630+0000","last_clean":"2023-10-16T20:06:18.197630+0000","last_became_active":"2023-10-16T20:06:17.210629+0000","last_became_peered":"2023-10-16T20:06:17.210629+0000","last_unstale":"2023-10-16T20:06:18.197630+0000","last_undegraded":"2023-10-16T20:06:18.197630+0000","last_fullsized":"2023-10-16T20:06:18.197630+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:27:10.715147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197643+0000","last_change":"2023-10-16T20:06:17.212963+0000","last_active":"2023-10-16T20:06:18.197643+0000","last_peered":"2023-10-16T20:06:18.197643+0000","last_clean":"2023-10-16T20:06:18.197643+0000","last_became_active":"2023-10-16T20:06:17.212693+0000","last_became_peered":"2023-10-16T20:06:17.212693+0000","last_unstale":"2023-10-16T20:06:18.197643+0000","last_undegraded":"2023-10-16T20:06:18.197643+0000","last_fullsized":"2023-10-16T20:06:18.197643+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:10:43.145100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197558+0000","last_change":"2023-10-16T20:06:17.219075+0000","last_active":"2023-10-16T20:06:18.197558+0000","last_peered":"2023-10-16T20:06:18.197558+0000","last_clean":"2023-10-16T20:06:18.197558+0000","last_became_active":"2023-10-16T20:06:17.218774+0000","last_became_peered":"2023-10-16T20:06:17.218774+0000","last_unstale":"2023-10-16T20:06:18.197558+0000","last_undegraded":"2023-10-16T20:06:18.197558+0000","last_fullsized":"2023-10-16T20:06:18.197558+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:05:25.874296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.857727+0000","last_change":"2023-10-16T20:06:17.200644+0000","last_active":"2023-10-16T20:06:18.857727+0000","last_peered":"2023-10-16T20:06:18.857727+0000","last_clean":"2023-10-16T20:06:18.857727+0000","last_became_active":"2023-10-16T20:06:17.200350+0000","last_became_peered":"2023-10-16T20:06:17.200350+0000","last_unstale":"2023-10-16T20:06:18.857727+0000","last_undegraded":"2023-10-16T20:06:18.857727+0000","last_fullsized":"2023-10-16T20:06:18.857727+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:27:39.178553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196740+0000","last_change":"2023-10-16T20:06:17.224294+0000","last_active":"2023-10-16T20:06:18.196740+0000","last_peered":"2023-10-16T20:06:18.196740+0000","last_clean":"2023-10-16T20:06:18.196740+0000","last_became_active":"2023-10-16T20:06:17.223988+0000","last_became_peered":"2023-10-16T20:06:17.223988+0000","last_unstale":"2023-10-16T20:06:18.196740+0000","last_undegraded":"2023-10-16T20:06:18.196740+0000","last_fullsized":"2023-10-16T20:06:18.196740+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:03:40.257975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"40'4","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198438+0000","last_change":"2023-10-16T20:06:16.732366+0000","last_active":"2023-10-16T20:06:18.198438+0000","last_peered":"2023-10-16T20:06:18.198438+0000","last_clean":"2023-10-16T20:06:18.198438+0000","last_became_active":"2023-10-16T20:06:16.732035+0000","last_became_peered":"2023-10-16T20:06:16.732035+0000","last_unstale":"2023-10-16T20:06:18.198438+0000","last_undegraded":"2023-10-16T20:06:18.198438+0000","last_fullsized":"2023-10-16T20:06:18.198438+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:35:43.770049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.858386+0000","last_change":"2023-10-16T20:06:17.207721+0000","last_active":"2023-10-16T20:06:18.858386+0000","last_peered":"2023-10-16T20:06:18.858386+0000","last_clean":"2023-10-16T20:06:18.858386+0000","last_became_active":"2023-10-16T20:06:17.207470+0000","last_became_peered":"2023-10-16T20:06:17.207470+0000","last_unstale":"2023-10-16T20:06:18.858386+0000","last_undegraded":"2023-10-16T20:06:18.858386+0000","last_fullsized":"2023-10-16T20:06:18.858386+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:24:29.941143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"40'1","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198456+0000","last_change":"2023-10-16T20:06:16.730576+0000","last_active":"2023-10-16T20:06:18.198456+0000","last_peered":"2023-10-16T20:06:18.198456+0000","last_clean":"2023-10-16T20:06:18.198456+0000","last_became_active":"2023-10-16T20:06:16.729963+0000","last_became_peered":"2023-10-16T20:06:16.729963+0000","last_unstale":"2023-10-16T20:06:18.198456+0000","last_undegraded":"2023-10-16T20:06:18.198456+0000","last_fullsized":"2023-10-16T20:06:18.198456+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:51:10.650711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.858166+0000","last_change":"2023-10-16T20:06:17.215735+0000","last_active":"2023-10-16T20:06:18.858166+0000","last_peered":"2023-10-16T20:06:18.858166+0000","last_clean":"2023-10-16T20:06:18.858166+0000","last_became_active":"2023-10-16T20:06:17.214986+0000","last_became_peered":"2023-10-16T20:06:17.214986+0000","last_unstale":"2023-10-16T20:06:18.858166+0000","last_undegraded":"2023-10-16T20:06:18.858166+0000","last_fullsized":"2023-10-16T20:06:18.858166+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:38:40.789443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"40'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.467575+0000","last_change":"2023-10-16T20:06:16.739387+0000","last_active":"2023-10-16T20:06:18.467575+0000","last_peered":"2023-10-16T20:06:18.467575+0000","last_clean":"2023-10-16T20:06:18.467575+0000","last_became_active":"2023-10-16T20:06:16.739070+0000","last_became_peered":"2023-10-16T20:06:16.739070+0000","last_unstale":"2023-10-16T20:06:18.467575+0000","last_undegraded":"2023-10-16T20:06:18.467575+0000","last_fullsized":"2023-10-16T20:06:18.467575+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:04:52.152758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.858386+0000","last_change":"2023-10-16T20:06:17.213835+0000","last_active":"2023-10-16T20:06:18.858386+0000","last_peered":"2023-10-16T20:06:18.858386+0000","last_clean":"2023-10-16T20:06:18.858386+0000","last_became_active":"2023-10-16T20:06:17.213516+0000","last_became_peered":"2023-10-16T20:06:17.213516+0000","last_unstale":"2023-10-16T20:06:18.858386+0000","last_undegraded":"2023-10-16T20:06:18.858386+0000","last_fullsized":"2023-10-16T20:06:18.858386+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:37:43.948080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"40'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.464991+0000","last_change":"2023-10-16T20:06:16.730019+0000","last_active":"2023-10-16T20:06:18.464991+0000","last_peered":"2023-10-16T20:06:18.464991+0000","last_clean":"2023-10-16T20:06:18.464991+0000","last_became_active":"2023-10-16T20:06:16.729804+0000","last_became_peered":"2023-10-16T20:06:16.729804+0000","last_unstale":"2023-10-16T20:06:18.464991+0000","last_undegraded":"2023-10-16T20:06:18.464991+0000","last_fullsized":"2023-10-16T20:06:18.464991+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:56:26.137061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198084+0000","last_change":"2023-10-16T20:06:17.214073+0000","last_active":"2023-10-16T20:06:18.198084+0000","last_peered":"2023-10-16T20:06:18.198084+0000","last_clean":"2023-10-16T20:06:18.198084+0000","last_became_active":"2023-10-16T20:06:17.213793+0000","last_became_peered":"2023-10-16T20:06:17.213793+0000","last_unstale":"2023-10-16T20:06:18.198084+0000","last_undegraded":"2023-10-16T20:06:18.198084+0000","last_fullsized":"2023-10-16T20:06:18.198084+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:17:45.690804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"40'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.463762+0000","last_change":"2023-10-16T20:06:16.731997+0000","last_active":"2023-10-16T20:06:18.463762+0000","last_peered":"2023-10-16T20:06:18.463762+0000","last_clean":"2023-10-16T20:06:18.463762+0000","last_became_active":"2023-10-16T20:06:16.731714+0000","last_became_peered":"2023-10-16T20:06:16.731714+0000","last_unstale":"2023-10-16T20:06:18.463762+0000","last_undegraded":"2023-10-16T20:06:18.463762+0000","last_fullsized":"2023-10-16T20:06:18.463762+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:25:12.274021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.698980+0000","last_change":"2023-10-16T20:06:17.214298+0000","last_active":"2023-10-16T20:06:18.698980+0000","last_peered":"2023-10-16T20:06:18.698980+0000","last_clean":"2023-10-16T20:06:18.698980+0000","last_became_active":"2023-10-16T20:06:17.214038+0000","last_became_peered":"2023-10-16T20:06:17.214038+0000","last_unstale":"2023-10-16T20:06:18.698980+0000","last_undegraded":"2023-10-16T20:06:18.698980+0000","last_fullsized":"2023-10-16T20:06:18.698980+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:38:22.493819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"40'1","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196763+0000","last_change":"2023-10-16T20:06:16.737162+0000","last_active":"2023-10-16T20:06:18.196763+0000","last_peered":"2023-10-16T20:06:18.196763+0000","last_clean":"2023-10-16T20:06:18.196763+0000","last_became_active":"2023-10-16T20:06:16.736888+0000","last_became_peered":"2023-10-16T20:06:16.736888+0000","last_unstale":"2023-10-16T20:06:18.196763+0000","last_undegraded":"2023-10-16T20:06:18.196763+0000","last_fullsized":"2023-10-16T20:06:18.196763+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:22:31.971013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.858075+0000","last_change":"2023-10-16T20:06:17.208067+0000","last_active":"2023-10-16T20:06:18.858075+0000","last_peered":"2023-10-16T20:06:18.858075+0000","last_clean":"2023-10-16T20:06:18.858075+0000","last_became_active":"2023-10-16T20:06:17.207799+0000","last_became_peered":"2023-10-16T20:06:17.207799+0000","last_unstale":"2023-10-16T20:06:18.858075+0000","last_undegraded":"2023-10-16T20:06:18.858075+0000","last_fullsized":"2023-10-16T20:06:18.858075+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:36:27.881926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.467467+0000","last_change":"2023-10-16T20:06:16.729238+0000","last_active":"2023-10-16T20:06:18.467467+0000","last_peered":"2023-10-16T20:06:18.467467+0000","last_clean":"2023-10-16T20:06:18.467467+0000","last_became_active":"2023-10-16T20:06:16.728975+0000","last_became_peered":"2023-10-16T20:06:16.728975+0000","last_unstale":"2023-10-16T20:06:18.467467+0000","last_undegraded":"2023-10-16T20:06:18.467467+0000","last_fullsized":"2023-10-16T20:06:18.467467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:36:26.871127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.857978+0000","last_change":"2023-10-16T20:06:17.218801+0000","last_active":"2023-10-16T20:06:18.857978+0000","last_peered":"2023-10-16T20:06:18.857978+0000","last_clean":"2023-10-16T20:06:18.857978+0000","last_became_active":"2023-10-16T20:06:17.218219+0000","last_became_peered":"2023-10-16T20:06:17.218219+0000","last_unstale":"2023-10-16T20:06:18.857978+0000","last_undegraded":"2023-10-16T20:06:18.857978+0000","last_fullsized":"2023-10-16T20:06:18.857978+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:58:45.132214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.464912+0000","last_change":"2023-10-16T20:06:16.720994+0000","last_active":"2023-10-16T20:06:18.464912+0000","last_peered":"2023-10-16T20:06:18.464912+0000","last_clean":"2023-10-16T20:06:18.464912+0000","last_became_active":"2023-10-16T20:06:16.720623+0000","last_became_peered":"2023-10-16T20:06:16.720623+0000","last_unstale":"2023-10-16T20:06:18.464912+0000","last_undegraded":"2023-10-16T20:06:18.464912+0000","last_fullsized":"2023-10-16T20:06:18.464912+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:32:43.082291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357768+0000","last_change":"2023-10-16T20:06:17.216335+0000","last_active":"2023-10-16T20:06:18.357768+0000","last_peered":"2023-10-16T20:06:18.357768+0000","last_clean":"2023-10-16T20:06:18.357768+0000","last_became_active":"2023-10-16T20:06:17.216070+0000","last_became_peered":"2023-10-16T20:06:17.216070+0000","last_unstale":"2023-10-16T20:06:18.357768+0000","last_undegraded":"2023-10-16T20:06:18.357768+0000","last_fullsized":"2023-10-16T20:06:18.357768+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:08:22.444705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.468333+0000","last_change":"2023-10-16T20:06:16.736582+0000","last_active":"2023-10-16T20:06:18.468333+0000","last_peered":"2023-10-16T20:06:18.468333+0000","last_clean":"2023-10-16T20:06:18.468333+0000","last_became_active":"2023-10-16T20:06:16.736322+0000","last_became_peered":"2023-10-16T20:06:16.736322+0000","last_unstale":"2023-10-16T20:06:18.468333+0000","last_undegraded":"2023-10-16T20:06:18.468333+0000","last_fullsized":"2023-10-16T20:06:18.468333+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:38:29.780087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197108+0000","last_change":"2023-10-16T20:06:17.220312+0000","last_active":"2023-10-16T20:06:18.197108+0000","last_peered":"2023-10-16T20:06:18.197108+0000","last_clean":"2023-10-16T20:06:18.197108+0000","last_became_active":"2023-10-16T20:06:17.220033+0000","last_became_peered":"2023-10-16T20:06:17.220033+0000","last_unstale":"2023-10-16T20:06:18.197108+0000","last_undegraded":"2023-10-16T20:06:18.197108+0000","last_fullsized":"2023-10-16T20:06:18.197108+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:00:59.428210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.463888+0000","last_change":"2023-10-16T20:06:16.725615+0000","last_active":"2023-10-16T20:06:18.463888+0000","last_peered":"2023-10-16T20:06:18.463888+0000","last_clean":"2023-10-16T20:06:18.463888+0000","last_became_active":"2023-10-16T20:06:16.725257+0000","last_became_peered":"2023-10-16T20:06:16.725257+0000","last_unstale":"2023-10-16T20:06:18.463888+0000","last_undegraded":"2023-10-16T20:06:18.463888+0000","last_fullsized":"2023-10-16T20:06:18.463888+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:36:24.034397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198481+0000","last_change":"2023-10-16T20:06:17.215733+0000","last_active":"2023-10-16T20:06:18.198481+0000","last_peered":"2023-10-16T20:06:18.198481+0000","last_clean":"2023-10-16T20:06:18.198481+0000","last_became_active":"2023-10-16T20:06:17.215495+0000","last_became_peered":"2023-10-16T20:06:17.215495+0000","last_unstale":"2023-10-16T20:06:18.198481+0000","last_undegraded":"2023-10-16T20:06:18.198481+0000","last_fullsized":"2023-10-16T20:06:18.198481+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:26:53.217625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"40'3","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.463853+0000","last_change":"2023-10-16T20:06:16.734102+0000","last_active":"2023-10-16T20:06:18.463853+0000","last_peered":"2023-10-16T20:06:18.463853+0000","last_clean":"2023-10-16T20:06:18.463853+0000","last_became_active":"2023-10-16T20:06:16.733829+0000","last_became_peered":"2023-10-16T20:06:16.733829+0000","last_unstale":"2023-10-16T20:06:18.463853+0000","last_undegraded":"2023-10-16T20:06:18.463853+0000","last_fullsized":"2023-10-16T20:06:18.463853+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:32:58.366700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.858176+0000","last_change":"2023-10-16T20:06:17.214205+0000","last_active":"2023-10-16T20:06:18.858176+0000","last_peered":"2023-10-16T20:06:18.858176+0000","last_clean":"2023-10-16T20:06:18.858176+0000","last_became_active":"2023-10-16T20:06:17.213261+0000","last_became_peered":"2023-10-16T20:06:17.213261+0000","last_unstale":"2023-10-16T20:06:18.858176+0000","last_undegraded":"2023-10-16T20:06:18.858176+0000","last_fullsized":"2023-10-16T20:06:18.858176+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:04:53.163558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"40'6","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.468288+0000","last_change":"2023-10-16T20:06:16.721479+0000","last_active":"2023-10-16T20:06:18.468288+0000","last_peered":"2023-10-16T20:06:18.468288+0000","last_clean":"2023-10-16T20:06:18.468288+0000","last_became_active":"2023-10-16T20:06:16.721136+0000","last_became_peered":"2023-10-16T20:06:16.721136+0000","last_unstale":"2023-10-16T20:06:18.468288+0000","last_undegraded":"2023-10-16T20:06:18.468288+0000","last_fullsized":"2023-10-16T20:06:18.468288+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:32:58.366700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198082+0000","last_change":"2023-10-16T20:06:17.211029+0000","last_active":"2023-10-16T20:06:18.198082+0000","last_peered":"2023-10-16T20:06:18.198082+0000","last_clean":"2023-10-16T20:06:18.198082+0000","last_became_active":"2023-10-16T20:06:17.210758+0000","last_became_peered":"2023-10-16T20:06:17.210758+0000","last_unstale":"2023-10-16T20:06:18.198082+0000","last_undegraded":"2023-10-16T20:06:18.198082+0000","last_fullsized":"2023-10-16T20:06:18.198082+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:26:41.239758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"40'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.468258+0000","last_change":"2023-10-16T20:06:16.735005+0000","last_active":"2023-10-16T20:06:18.468258+0000","last_peered":"2023-10-16T20:06:18.468258+0000","last_clean":"2023-10-16T20:06:18.468258+0000","last_became_active":"2023-10-16T20:06:16.734752+0000","last_became_peered":"2023-10-16T20:06:16.734752+0000","last_unstale":"2023-10-16T20:06:18.468258+0000","last_undegraded":"2023-10-16T20:06:18.468258+0000","last_fullsized":"2023-10-16T20:06:18.468258+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:21:23.740779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.857997+0000","last_change":"2023-10-16T20:06:17.215993+0000","last_active":"2023-10-16T20:06:18.857997+0000","last_peered":"2023-10-16T20:06:18.857997+0000","last_clean":"2023-10-16T20:06:18.857997+0000","last_became_active":"2023-10-16T20:06:17.215736+0000","last_became_peered":"2023-10-16T20:06:17.215736+0000","last_unstale":"2023-10-16T20:06:18.857997+0000","last_undegraded":"2023-10-16T20:06:18.857997+0000","last_fullsized":"2023-10-16T20:06:18.857997+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:03:40.257975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.467499+0000","last_change":"2023-10-16T20:06:16.733426+0000","last_active":"2023-10-16T20:06:18.467499+0000","last_peered":"2023-10-16T20:06:18.467499+0000","last_clean":"2023-10-16T20:06:18.467499+0000","last_became_active":"2023-10-16T20:06:16.732600+0000","last_became_peered":"2023-10-16T20:06:16.732600+0000","last_unstale":"2023-10-16T20:06:18.467499+0000","last_undegraded":"2023-10-16T20:06:18.467499+0000","last_fullsized":"2023-10-16T20:06:18.467499+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:12.864132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196653+0000","last_change":"2023-10-16T20:06:17.219925+0000","last_active":"2023-10-16T20:06:18.196653+0000","last_peered":"2023-10-16T20:06:18.196653+0000","last_clean":"2023-10-16T20:06:18.196653+0000","last_became_active":"2023-10-16T20:06:17.219584+0000","last_became_peered":"2023-10-16T20:06:17.219584+0000","last_unstale":"2023-10-16T20:06:18.196653+0000","last_undegraded":"2023-10-16T20:06:18.196653+0000","last_fullsized":"2023-10-16T20:06:18.196653+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:44:14.727872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197391+0000","last_change":"2023-10-16T20:06:16.732499+0000","last_active":"2023-10-16T20:06:18.197391+0000","last_peered":"2023-10-16T20:06:18.197391+0000","last_clean":"2023-10-16T20:06:18.197391+0000","last_became_active":"2023-10-16T20:06:16.732244+0000","last_became_peered":"2023-10-16T20:06:16.732244+0000","last_unstale":"2023-10-16T20:06:18.197391+0000","last_undegraded":"2023-10-16T20:06:18.197391+0000","last_fullsized":"2023-10-16T20:06:18.197391+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:40:57.924059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.858062+0000","last_change":"2023-10-16T20:06:17.215267+0000","last_active":"2023-10-16T20:06:18.858062+0000","last_peered":"2023-10-16T20:06:18.858062+0000","last_clean":"2023-10-16T20:06:18.858062+0000","last_became_active":"2023-10-16T20:06:17.215017+0000","last_became_peered":"2023-10-16T20:06:17.215017+0000","last_unstale":"2023-10-16T20:06:18.858062+0000","last_undegraded":"2023-10-16T20:06:18.858062+0000","last_fullsized":"2023-10-16T20:06:18.858062+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:31:31.346356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"40'8","reported_seq":20,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196540+0000","last_change":"2023-10-16T20:06:16.730165+0000","last_active":"2023-10-16T20:06:18.196540+0000","last_peered":"2023-10-16T20:06:18.196540+0000","last_clean":"2023-10-16T20:06:18.196540+0000","last_became_active":"2023-10-16T20:06:16.729894+0000","last_became_peered":"2023-10-16T20:06:16.729894+0000","last_unstale":"2023-10-16T20:06:18.196540+0000","last_undegraded":"2023-10-16T20:06:18.196540+0000","last_fullsized":"2023-10-16T20:06:18.196540+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:36:26.871127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.858108+0000","last_change":"2023-10-16T20:06:17.214039+0000","last_active":"2023-10-16T20:06:18.858108+0000","last_peered":"2023-10-16T20:06:18.858108+0000","last_clean":"2023-10-16T20:06:18.858108+0000","last_became_active":"2023-10-16T20:06:17.213551+0000","last_became_peered":"2023-10-16T20:06:17.213551+0000","last_unstale":"2023-10-16T20:06:18.858108+0000","last_undegraded":"2023-10-16T20:06:18.858108+0000","last_fullsized":"2023-10-16T20:06:18.858108+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:44:43.545772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"40'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.463923+0000","last_change":"2023-10-16T20:06:16.726366+0000","last_active":"2023-10-16T20:06:18.463923+0000","last_peered":"2023-10-16T20:06:18.463923+0000","last_clean":"2023-10-16T20:06:18.463923+0000","last_became_active":"2023-10-16T20:06:16.726086+0000","last_became_peered":"2023-10-16T20:06:16.726086+0000","last_unstale":"2023-10-16T20:06:18.463923+0000","last_undegraded":"2023-10-16T20:06:18.463923+0000","last_fullsized":"2023-10-16T20:06:18.463923+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:21:23.740779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.699199+0000","last_change":"2023-10-16T20:06:17.207345+0000","last_active":"2023-10-16T20:06:18.699199+0000","last_peered":"2023-10-16T20:06:18.699199+0000","last_clean":"2023-10-16T20:06:18.699199+0000","last_became_active":"2023-10-16T20:06:17.207082+0000","last_became_peered":"2023-10-16T20:06:17.207082+0000","last_unstale":"2023-10-16T20:06:18.699199+0000","last_undegraded":"2023-10-16T20:06:18.699199+0000","last_fullsized":"2023-10-16T20:06:18.699199+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:05:25.874296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"40'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.465029+0000","last_change":"2023-10-16T20:06:16.727808+0000","last_active":"2023-10-16T20:06:18.465029+0000","last_peered":"2023-10-16T20:06:18.465029+0000","last_clean":"2023-10-16T20:06:18.465029+0000","last_became_active":"2023-10-16T20:06:16.727400+0000","last_became_peered":"2023-10-16T20:06:16.727400+0000","last_unstale":"2023-10-16T20:06:18.465029+0000","last_undegraded":"2023-10-16T20:06:18.465029+0000","last_fullsized":"2023-10-16T20:06:18.465029+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:00.635492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.858083+0000","last_change":"2023-10-16T20:06:17.216271+0000","last_active":"2023-10-16T20:06:18.858083+0000","last_peered":"2023-10-16T20:06:18.858083+0000","last_clean":"2023-10-16T20:06:18.858083+0000","last_became_active":"2023-10-16T20:06:17.215201+0000","last_became_peered":"2023-10-16T20:06:17.215201+0000","last_unstale":"2023-10-16T20:06:18.858083+0000","last_undegraded":"2023-10-16T20:06:18.858083+0000","last_fullsized":"2023-10-16T20:06:18.858083+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:13.874932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"40'1","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357862+0000","last_change":"2023-10-16T20:06:16.724780+0000","last_active":"2023-10-16T20:06:18.357862+0000","last_peered":"2023-10-16T20:06:18.357862+0000","last_clean":"2023-10-16T20:06:18.357862+0000","last_became_active":"2023-10-16T20:06:16.724476+0000","last_became_peered":"2023-10-16T20:06:16.724476+0000","last_unstale":"2023-10-16T20:06:18.357862+0000","last_undegraded":"2023-10-16T20:06:18.357862+0000","last_fullsized":"2023-10-16T20:06:18.357862+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:19:37.004795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.358046+0000","last_change":"2023-10-16T20:06:17.209520+0000","last_active":"2023-10-16T20:06:18.358046+0000","last_peered":"2023-10-16T20:06:18.358046+0000","last_clean":"2023-10-16T20:06:18.358046+0000","last_became_active":"2023-10-16T20:06:17.209236+0000","last_became_peered":"2023-10-16T20:06:17.209236+0000","last_unstale":"2023-10-16T20:06:18.358046+0000","last_undegraded":"2023-10-16T20:06:18.358046+0000","last_fullsized":"2023-10-16T20:06:18.358046+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:31:26.166083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.467537+0000","last_change":"2023-10-16T20:06:16.734625+0000","last_active":"2023-10-16T20:06:18.467537+0000","last_peered":"2023-10-16T20:06:18.467537+0000","last_clean":"2023-10-16T20:06:18.467537+0000","last_became_active":"2023-10-16T20:06:16.734353+0000","last_became_peered":"2023-10-16T20:06:16.734353+0000","last_unstale":"2023-10-16T20:06:18.467537+0000","last_undegraded":"2023-10-16T20:06:18.467537+0000","last_fullsized":"2023-10-16T20:06:18.467537+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:38:39.778643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196838+0000","last_change":"2023-10-16T20:06:17.219357+0000","last_active":"2023-10-16T20:06:18.196838+0000","last_peered":"2023-10-16T20:06:18.196838+0000","last_clean":"2023-10-16T20:06:18.196838+0000","last_became_active":"2023-10-16T20:06:17.218995+0000","last_became_peered":"2023-10-16T20:06:17.218995+0000","last_unstale":"2023-10-16T20:06:18.196838+0000","last_undegraded":"2023-10-16T20:06:18.196838+0000","last_fullsized":"2023-10-16T20:06:18.196838+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:44:43.545772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"40'2","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357665+0000","last_change":"2023-10-16T20:06:16.732782+0000","last_active":"2023-10-16T20:06:18.357665+0000","last_peered":"2023-10-16T20:06:18.357665+0000","last_clean":"2023-10-16T20:06:18.357665+0000","last_became_active":"2023-10-16T20:06:16.732498+0000","last_became_peered":"2023-10-16T20:06:16.732498+0000","last_unstale":"2023-10-16T20:06:18.357665+0000","last_undegraded":"2023-10-16T20:06:18.357665+0000","last_fullsized":"2023-10-16T20:06:18.357665+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:21:27.852426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198137+0000","last_change":"2023-10-16T20:06:17.221923+0000","last_active":"2023-10-16T20:06:18.198137+0000","last_peered":"2023-10-16T20:06:18.198137+0000","last_clean":"2023-10-16T20:06:18.198137+0000","last_became_active":"2023-10-16T20:06:17.221536+0000","last_became_peered":"2023-10-16T20:06:17.221536+0000","last_unstale":"2023-10-16T20:06:18.198137+0000","last_undegraded":"2023-10-16T20:06:18.198137+0000","last_fullsized":"2023-10-16T20:06:18.198137+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:00:56.525352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357669+0000","last_change":"2023-10-16T20:06:16.732525+0000","last_active":"2023-10-16T20:06:18.357669+0000","last_peered":"2023-10-16T20:06:18.357669+0000","last_clean":"2023-10-16T20:06:18.357669+0000","last_became_active":"2023-10-16T20:06:16.732199+0000","last_became_peered":"2023-10-16T20:06:16.732199+0000","last_unstale":"2023-10-16T20:06:18.357669+0000","last_undegraded":"2023-10-16T20:06:18.357669+0000","last_fullsized":"2023-10-16T20:06:18.357669+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:12:42.667116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.699020+0000","last_change":"2023-10-16T20:06:17.209547+0000","last_active":"2023-10-16T20:06:18.699020+0000","last_peered":"2023-10-16T20:06:18.699020+0000","last_clean":"2023-10-16T20:06:18.699020+0000","last_became_active":"2023-10-16T20:06:17.209281+0000","last_became_peered":"2023-10-16T20:06:17.209281+0000","last_unstale":"2023-10-16T20:06:18.699020+0000","last_undegraded":"2023-10-16T20:06:18.699020+0000","last_fullsized":"2023-10-16T20:06:18.699020+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:00:59.428210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197648+0000","last_change":"2023-10-16T20:06:16.727775+0000","last_active":"2023-10-16T20:06:18.197648+0000","last_peered":"2023-10-16T20:06:18.197648+0000","last_clean":"2023-10-16T20:06:18.197648+0000","last_became_active":"2023-10-16T20:06:16.727492+0000","last_became_peered":"2023-10-16T20:06:16.727492+0000","last_unstale":"2023-10-16T20:06:18.197648+0000","last_undegraded":"2023-10-16T20:06:18.197648+0000","last_fullsized":"2023-10-16T20:06:18.197648+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:32:58.366700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.858142+0000","last_change":"2023-10-16T20:06:17.218801+0000","last_active":"2023-10-16T20:06:18.858142+0000","last_peered":"2023-10-16T20:06:18.858142+0000","last_clean":"2023-10-16T20:06:18.858142+0000","last_became_active":"2023-10-16T20:06:17.218567+0000","last_became_peered":"2023-10-16T20:06:17.218567+0000","last_unstale":"2023-10-16T20:06:18.858142+0000","last_undegraded":"2023-10-16T20:06:18.858142+0000","last_fullsized":"2023-10-16T20:06:18.858142+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:38:22.493819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.357557+0000","last_change":"2023-10-16T20:06:16.727754+0000","last_active":"2023-10-16T20:06:18.357557+0000","last_peered":"2023-10-16T20:06:18.357557+0000","last_clean":"2023-10-16T20:06:18.357557+0000","last_became_active":"2023-10-16T20:06:16.727497+0000","last_became_peered":"2023-10-16T20:06:16.727497+0000","last_unstale":"2023-10-16T20:06:18.357557+0000","last_undegraded":"2023-10-16T20:06:18.357557+0000","last_fullsized":"2023-10-16T20:06:18.357557+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:08:12.647505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.857963+0000","last_change":"2023-10-16T20:06:17.201294+0000","last_active":"2023-10-16T20:06:18.857963+0000","last_peered":"2023-10-16T20:06:18.857963+0000","last_clean":"2023-10-16T20:06:18.857963+0000","last_became_active":"2023-10-16T20:06:17.200853+0000","last_became_peered":"2023-10-16T20:06:17.200853+0000","last_unstale":"2023-10-16T20:06:18.857963+0000","last_undegraded":"2023-10-16T20:06:18.857963+0000","last_fullsized":"2023-10-16T20:06:18.857963+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:18:14.480179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"40'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.463944+0000","last_change":"2023-10-16T20:06:16.733877+0000","last_active":"2023-10-16T20:06:18.463944+0000","last_peered":"2023-10-16T20:06:18.463944+0000","last_clean":"2023-10-16T20:06:18.463944+0000","last_became_active":"2023-10-16T20:06:16.733570+0000","last_became_peered":"2023-10-16T20:06:16.733570+0000","last_unstale":"2023-10-16T20:06:18.463944+0000","last_undegraded":"2023-10-16T20:06:18.463944+0000","last_fullsized":"2023-10-16T20:06:18.463944+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:38:29.780087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197199+0000","last_change":"2023-10-16T20:06:17.220488+0000","last_active":"2023-10-16T20:06:18.197199+0000","last_peered":"2023-10-16T20:06:18.197199+0000","last_clean":"2023-10-16T20:06:18.197199+0000","last_became_active":"2023-10-16T20:06:17.220060+0000","last_became_peered":"2023-10-16T20:06:17.220060+0000","last_unstale":"2023-10-16T20:06:18.197199+0000","last_undegraded":"2023-10-16T20:06:18.197199+0000","last_fullsized":"2023-10-16T20:06:18.197199+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:59:16.509971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"41'1","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.467875+0000","last_change":"2023-10-16T20:06:16.726490+0000","last_active":"2023-10-16T20:06:18.467875+0000","last_peered":"2023-10-16T20:06:18.467875+0000","last_clean":"2023-10-16T20:06:18.467875+0000","last_became_active":"2023-10-16T20:06:16.726210+0000","last_became_peered":"2023-10-16T20:06:16.726210+0000","last_unstale":"2023-10-16T20:06:18.467875+0000","last_undegraded":"2023-10-16T20:06:18.467875+0000","last_fullsized":"2023-10-16T20:06:18.467875+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:13:14.492857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.858301+0000","last_change":"2023-10-16T20:06:17.213630+0000","last_active":"2023-10-16T20:06:18.858301+0000","last_peered":"2023-10-16T20:06:18.858301+0000","last_clean":"2023-10-16T20:06:18.858301+0000","last_became_active":"2023-10-16T20:06:17.213352+0000","last_became_peered":"2023-10-16T20:06:17.213352+0000","last_unstale":"2023-10-16T20:06:18.858301+0000","last_undegraded":"2023-10-16T20:06:18.858301+0000","last_fullsized":"2023-10-16T20:06:18.858301+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:00:59.428210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198443+0000","last_change":"2023-10-16T20:06:16.739495+0000","last_active":"2023-10-16T20:06:18.198443+0000","last_peered":"2023-10-16T20:06:18.198443+0000","last_clean":"2023-10-16T20:06:18.198443+0000","last_became_active":"2023-10-16T20:06:16.739215+0000","last_became_peered":"2023-10-16T20:06:16.739215+0000","last_unstale":"2023-10-16T20:06:18.198443+0000","last_undegraded":"2023-10-16T20:06:18.198443+0000","last_fullsized":"2023-10-16T20:06:18.198443+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:35:40.182424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198226+0000","last_change":"2023-10-16T20:06:17.212113+0000","last_active":"2023-10-16T20:06:18.198226+0000","last_peered":"2023-10-16T20:06:18.198226+0000","last_clean":"2023-10-16T20:06:18.198226+0000","last_became_active":"2023-10-16T20:06:17.211842+0000","last_became_peered":"2023-10-16T20:06:17.211842+0000","last_unstale":"2023-10-16T20:06:18.198226+0000","last_undegraded":"2023-10-16T20:06:18.198226+0000","last_fullsized":"2023-10-16T20:06:18.198226+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:34:14.717626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.198360+0000","last_change":"2023-10-16T20:06:16.730618+0000","last_active":"2023-10-16T20:06:18.198360+0000","last_peered":"2023-10-16T20:06:18.198360+0000","last_clean":"2023-10-16T20:06:18.198360+0000","last_became_active":"2023-10-16T20:06:16.729429+0000","last_became_peered":"2023-10-16T20:06:16.729429+0000","last_unstale":"2023-10-16T20:06:18.198360+0000","last_undegraded":"2023-10-16T20:06:18.198360+0000","last_fullsized":"2023-10-16T20:06:18.198360+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:18:42.478975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.858340+0000","last_change":"2023-10-16T20:06:17.208914+0000","last_active":"2023-10-16T20:06:18.858340+0000","last_peered":"2023-10-16T20:06:18.858340+0000","last_clean":"2023-10-16T20:06:18.858340+0000","last_became_active":"2023-10-16T20:06:17.208609+0000","last_became_peered":"2023-10-16T20:06:17.208609+0000","last_unstale":"2023-10-16T20:06:18.858340+0000","last_undegraded":"2023-10-16T20:06:18.858340+0000","last_fullsized":"2023-10-16T20:06:18.858340+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:50:32.312772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197750+0000","last_change":"2023-10-16T20:06:16.733015+0000","last_active":"2023-10-16T20:06:18.197750+0000","last_peered":"2023-10-16T20:06:18.197750+0000","last_clean":"2023-10-16T20:06:18.197750+0000","last_became_active":"2023-10-16T20:06:16.731413+0000","last_became_peered":"2023-10-16T20:06:16.731413+0000","last_unstale":"2023-10-16T20:06:18.197750+0000","last_undegraded":"2023-10-16T20:06:18.197750+0000","last_fullsized":"2023-10-16T20:06:18.197750+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:59:16.090503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.857827+0000","last_change":"2023-10-16T20:06:17.214304+0000","last_active":"2023-10-16T20:06:18.857827+0000","last_peered":"2023-10-16T20:06:18.857827+0000","last_clean":"2023-10-16T20:06:18.857827+0000","last_became_active":"2023-10-16T20:06:17.213521+0000","last_became_peered":"2023-10-16T20:06:17.213521+0000","last_unstale":"2023-10-16T20:06:18.857827+0000","last_undegraded":"2023-10-16T20:06:18.857827+0000","last_fullsized":"2023-10-16T20:06:18.857827+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:58:21.190657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.464929+0000","last_change":"2023-10-16T20:06:16.730112+0000","last_active":"2023-10-16T20:06:18.464929+0000","last_peered":"2023-10-16T20:06:18.464929+0000","last_clean":"2023-10-16T20:06:18.464929+0000","last_became_active":"2023-10-16T20:06:16.729781+0000","last_became_peered":"2023-10-16T20:06:16.729781+0000","last_unstale":"2023-10-16T20:06:18.464929+0000","last_undegraded":"2023-10-16T20:06:18.464929+0000","last_fullsized":"2023-10-16T20:06:18.464929+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:39:05.076198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.197008+0000","last_change":"2023-10-16T20:06:17.221997+0000","last_active":"2023-10-16T20:06:18.197008+0000","last_peered":"2023-10-16T20:06:18.197008+0000","last_clean":"2023-10-16T20:06:18.197008+0000","last_became_active":"2023-10-16T20:06:17.221704+0000","last_became_peered":"2023-10-16T20:06:17.221704+0000","last_unstale":"2023-10-16T20:06:18.197008+0000","last_undegraded":"2023-10-16T20:06:18.197008+0000","last_fullsized":"2023-10-16T20:06:18.197008+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:38:22.493819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.468201+0000","last_change":"2023-10-16T20:06:16.733171+0000","last_active":"2023-10-16T20:06:18.468201+0000","last_peered":"2023-10-16T20:06:18.468201+0000","last_clean":"2023-10-16T20:06:18.468201+0000","last_became_active":"2023-10-16T20:06:16.732869+0000","last_became_peered":"2023-10-16T20:06:16.732869+0000","last_unstale":"2023-10-16T20:06:18.468201+0000","last_undegraded":"2023-10-16T20:06:18.468201+0000","last_fullsized":"2023-10-16T20:06:18.468201+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:04:52.152758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.699340+0000","last_change":"2023-10-16T20:06:17.215196+0000","last_active":"2023-10-16T20:06:18.699340+0000","last_peered":"2023-10-16T20:06:18.699340+0000","last_clean":"2023-10-16T20:06:18.699340+0000","last_became_active":"2023-10-16T20:06:17.214556+0000","last_became_peered":"2023-10-16T20:06:17.214556+0000","last_unstale":"2023-10-16T20:06:18.699340+0000","last_undegraded":"2023-10-16T20:06:18.699340+0000","last_fullsized":"2023-10-16T20:06:18.699340+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:16.698995+0000","last_clean_scrub_stamp":"2023-10-16T20:06:16.698995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:25:43.904047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-16T20:06:18.196810+0000","last_change":"2023-10-16T20:06:16.728925+0000","last_active":"2023-10-16T20:06:18.196810+0000","last_peered":"2023-10-16T20:06:18.196810+0000","last_clean":"2023-10-16T20:06:18.196810+0000","last_became_active":"2023-10-16T20:06:16.728663+0000","last_became_peered":"2023-10-16T20:06:16.728663+0000","last_unstale":"2023-10-16T20:06:18.196810+0000","last_undegraded":"2023-10-16T20:06:18.196810+0000","last_fullsized":"2023-10-16T20:06:18.196810+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:06:15.688195+0000","last_clean_scrub_stamp":"2023-10-16T20:06:15.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:50:31.301972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":182,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-16T20:06:17.208038+0000","last_change":"2023-10-16T20:04:46.335822+0000","last_active":"2023-10-16T20:06:17.208038+0000","last_peered":"2023-10-16T20:06:17.208038+0000","last_clean":"2023-10-16T20:06:17.208038+0000","last_became_active":"2023-10-16T20:04:46.334617+0000","last_became_peered":"2023-10-16T20:04:46.334617+0000","last_unstale":"2023-10-16T20:06:17.208038+0000","last_undegraded":"2023-10-16T20:06:17.208038+0000","last_fullsized":"2023-10-16T20:06:17.208038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:45.306240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:45.306240+0000","last_clean_scrub_stamp":"2023-10-16T20:04:45.306240+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:42:11.153721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1077248,"data_stored":791591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.873}]},{"osd":1,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.4900000000000002,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":0.72699999999999998}]},{"osd":2,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.0550000000000002,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.84599999999999997}]},{"osd":3,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.629,"5min":2.629,"15min":2.629},"last":0.89900000000000002}]},{"osd":4,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":0.79700000000000004}]},{"osd":5,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.8809999999999998,"5min":2.8809999999999998,"15min":2.8809999999999998},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.81899999999999995}]},{"osd":6,"last update":"Mon Oct 16 20:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.68700000000000006}]}]},{"osd":6,"up_from":15,"seq":64424509464,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":471040,"data_stored":200123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":4.5880000000000001,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":4.6139999999999999,"5min":4.6139999999999999,"15min":4.6139999999999999},"last":0.52300000000000002}]},{"osd":1,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":4.6639999999999997,"5min":4.6639999999999997,"15min":4.6639999999999997},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":4.8140000000000001,"5min":4.8140000000000001,"15min":4.8140000000000001},"last":0.65800000000000003}]},{"osd":2,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":5.3769999999999998,"5min":5.3769999999999998,"15min":5.3769999999999998},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.64000000000000001}]},{"osd":3,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":4.5460000000000003,"5min":4.5460000000000003,"15min":4.5460000000000003},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.81200000000000006}]},{"osd":4,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.87}]},{"osd":5,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.85499999999999998}]},{"osd":7,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.83899999999999997}]}]},{"osd":5,"up_from":15,"seq":64424509462,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31644,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340196,"statfs":{"total":96636764160,"available":96604360704,"internally_reserved":0,"allocated":475136,"data_stored":199653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.71899999999999997}]},{"osd":1,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.74099999999999999}]},{"osd":2,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.624},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.83899999999999997}]},{"osd":3,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.81899999999999995}]},{"osd":4,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.79500000000000004}]},{"osd":6,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.86699999999999999}]},{"osd":7,"last update":"Mon Oct 16 20:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.59999999999999998}]}]},{"osd":4,"up_from":15,"seq":64424509462,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31644,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340196,"statfs":{"total":96636764160,"available":96604360704,"internally_reserved":0,"allocated":475136,"data_stored":200145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":3.835,"5min":3.835,"15min":3.835},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":3.7610000000000001,"5min":3.7610000000000001,"15min":3.7610000000000001},"last":0.67500000000000004}]},{"osd":1,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.78100000000000003}]},{"osd":2,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.89600000000000002}]},{"osd":3,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.72399999999999998}]},{"osd":5,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.57699999999999996}]},{"osd":6,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.86299999999999999}]},{"osd":7,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.8919999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.93799999999999994}]}]},{"osd":3,"up_from":15,"seq":64424509463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":483328,"data_stored":201187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.42599999999999999}]},{"osd":1,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.34200000000000003}]},{"osd":2,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.41299999999999998}]},{"osd":4,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.498},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.63700000000000001}]},{"osd":5,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.65400000000000003}]},{"osd":6,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.66900000000000004}]},{"osd":7,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.53800000000000003}]}]},{"osd":2,"up_from":15,"seq":64424509462,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":471040,"data_stored":199611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.33000000000000002}]},{"osd":1,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.39100000000000001}]},{"osd":3,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.50800000000000001}]},{"osd":4,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.81899999999999995}]},{"osd":5,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.77500000000000002}]},{"osd":6,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.88500000000000001}]},{"osd":7,"last update":"Mon Oct 16 20:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.79700000000000004}]}]},{"osd":1,"up_from":15,"seq":64424509463,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":471040,"data_stored":199631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.867,"5min":3.867,"15min":3.867},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.48999999999999999}]},{"osd":2,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":5.6040000000000001,"5min":5.6040000000000001,"15min":5.6040000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":7.8380000000000001,"5min":7.8380000000000001,"15min":7.8380000000000001},"last":0.42799999999999999}]},{"osd":3,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":5.2039999999999997,"5min":5.2039999999999997,"15min":5.2039999999999997},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":3.9470000000000001,"5min":3.9470000000000001,"15min":3.9470000000000001},"last":0.52200000000000002}]},{"osd":4,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":4.1500000000000004,"5min":4.1500000000000004,"15min":4.1500000000000004},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":4.2619999999999996,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":0.94099999999999995}]},{"osd":5,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":4.0910000000000002,"5min":4.0910000000000002,"15min":4.0910000000000002},"last":0.80800000000000005}]},{"osd":6,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":4.21,"5min":4.21,"15min":4.21},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.88300000000000001}]},{"osd":7,"last update":"Mon Oct 16 20:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":4.1849999999999996,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":0.90500000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28072,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343768,"statfs":{"total":96636764160,"available":96608018432,"internally_reserved":0,"allocated":1064960,"data_stored":789943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":3.9289999999999998,"5min":3.9289999999999998,"15min":3.9289999999999998},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":4.0910000000000002,"5min":4.0910000000000002,"15min":4.0910000000000002},"last":0.55700000000000005}]},{"osd":2,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":4.1520000000000001,"5min":4.1520000000000001,"15min":4.1520000000000001},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":4.1879999999999997,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.502}]},{"osd":3,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":4.3440000000000003,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.52100000000000002}]},{"osd":4,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":4.5250000000000004,"5min":4.5250000000000004,"15min":4.5250000000000004},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.46999999999999997}]},{"osd":5,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":3.9990000000000001,"5min":3.9990000000000001,"15min":3.9990000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":4.4779999999999998,"5min":4.4779999999999998,"15min":4.4779999999999998},"last":0.626}]},{"osd":6,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":3.956,"5min":3.956,"15min":3.956},"last":0.59799999999999998}]},{"osd":7,"last update":"Mon Oct 16 20:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":4.0289999999999999,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.435},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":4.5640000000000001,"5min":4.5640000000000001,"15min":4.5640000000000001},"last":0.65900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T20:06:26.654 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T20:06:26.655 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=testcephfs1' mds allow 2023-10-16T20:06:26.834 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.831+0000 7f32fd454640 1 Processor -- start 2023-10-16T20:06:26.834 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.831+0000 7f32fd454640 1 -- start start 2023-10-16T20:06:26.835 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.835+0000 7f32fd454640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f32f80ca120 0x7f32f80b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:26.836 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.835+0000 7f32fd454640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32f815a6e0 con 0x7f32f80ce940 2023-10-16T20:06:26.836 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.835+0000 7f32fd454640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32f80ca9a0 con 0x7f32f815d190 2023-10-16T20:06:26.836 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.835+0000 7f32fd454640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32f8159be0 con 0x7f32f80ca120 2023-10-16T20:06:26.836 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.835+0000 7f32f67fc640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f32f80ca120 0x7f32f80b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:26.836 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.835+0000 7f32f6ffd640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f32f80ce940 0x7f32f80ced20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:51152/0 (socket says 172.21.15.88:51152) 2023-10-16T20:06:26.837 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.835+0000 7f32f6ffd640 1 -- 172.21.15.88:0/1696208548 learned_addr learned my addr 172.21.15.88:0/1696208548 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:26.837 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.835+0000 7f32f67fc640 1 -- 172.21.15.88:0/1696208548 >> v1:172.21.15.94:6789/0 conn(0x7f32f815d190 legacy=0x7f32f815d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:26.837 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.835+0000 7f32f67fc640 1 -- 172.21.15.88:0/1696208548 >> v1:172.21.15.88:6789/0 conn(0x7f32f80ce940 legacy=0x7f32f80ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:26.838 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.835+0000 7f32f67fc640 1 -- 172.21.15.88:0/1696208548 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f80b12e0 con 0x7f32f80ca120 2023-10-16T20:06:26.838 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.835+0000 7f32f5ffb640 1 -- 172.21.15.88:0/1696208548 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3826420164 0 0) 0x7f32f815a6e0 con 0x7f32f80ce940 2023-10-16T20:06:26.838 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.835+0000 7f32f67fc640 1 --2- 172.21.15.88:0/1696208548 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32f80ca120 0x7f32f80b2d10 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f32ec0097c0 tx=0x7f32ec01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec852e78856274c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:26.840 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.839+0000 7f32f5ffb640 1 -- 172.21.15.88:0/1696208548 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec004230 con 0x7f32f80ca120 2023-10-16T20:06:26.840 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.839+0000 7f32f5ffb640 1 -- 172.21.15.88:0/1696208548 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ec0a7050 con 0x7f32f80ca120 2023-10-16T20:06:26.840 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.839+0000 7f32f5ffb640 1 -- 172.21.15.88:0/1696208548 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec003620 con 0x7f32f80ca120 2023-10-16T20:06:26.841 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.839+0000 7f32fd454640 1 -- 172.21.15.88:0/1696208548 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32f80ca120 msgr2=0x7f32f80b2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:26.841 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.839+0000 7f32fd454640 1 --2- 172.21.15.88:0/1696208548 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32f80ca120 0x7f32f80b2d10 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f32ec0097c0 tx=0x7f32ec01c440 comp rx=0 tx=0).stop 2023-10-16T20:06:26.841 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.839+0000 7f32fd454640 1 -- 172.21.15.88:0/1696208548 shutdown_connections 2023-10-16T20:06:26.841 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.839+0000 7f32fd454640 1 --2- 172.21.15.88:0/1696208548 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32f80ca120 0x7f32f80b2d10 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:26.841 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.839+0000 7f32fd454640 1 -- 172.21.15.88:0/1696208548 >> 172.21.15.88:0/1696208548 conn(0x7f32f806a290 msgr2=0x7f32f806a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:26.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.839+0000 7f32fd454640 1 -- 172.21.15.88:0/1696208548 shutdown_connections 2023-10-16T20:06:26.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.839+0000 7f32fd454640 1 -- 172.21.15.88:0/1696208548 wait complete. 2023-10-16T20:06:26.844 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.839+0000 7f32fd454640 1 Processor -- start 2023-10-16T20:06:26.844 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.839+0000 7f32fd454640 1 -- start start 2023-10-16T20:06:26.844 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.839+0000 7f32fd454640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32f80ca120 0x7f32f80ad260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:26.845 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.839+0000 7f32fd454640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32f80ce940 0x7f32f80ad7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:26.845 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.839+0000 7f32fd454640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32f815d190 0x7f32f81561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:26.845 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.839+0000 7f32fd454640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f815c160 con 0x7f32f815d190 2023-10-16T20:06:26.845 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.839+0000 7f32fd454640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f815b4e0 con 0x7f32f80ca120 2023-10-16T20:06:26.845 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.839+0000 7f32fd454640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f815b7c0 con 0x7f32f80ce940 2023-10-16T20:06:26.845 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.839+0000 7f32f77fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32f815d190 0x7f32f81561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:26.846 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.839+0000 7f32f77fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32f815d190 0x7f32f81561f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57592/0 (socket says 172.21.15.88:57592) 2023-10-16T20:06:26.846 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.839+0000 7f32f77fe640 1 -- 172.21.15.88:0/2139459377 learned_addr learned my addr 172.21.15.88:0/2139459377 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:26.846 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.843+0000 7f32f6ffd640 1 --2- 172.21.15.88:0/2139459377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32f80ca120 0x7f32f80ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:26.846 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.843+0000 7f32f67fc640 1 --2- 172.21.15.88:0/2139459377 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32f80ce940 0x7f32f80ad7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:26.846 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.843+0000 7f32f77fe640 1 -- 172.21.15.88:0/2139459377 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32f80ce940 msgr2=0x7f32f80ad7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:26.847 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.843+0000 7f32f77fe640 1 --2- 172.21.15.88:0/2139459377 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32f80ce940 0x7f32f80ad7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:26.847 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.843+0000 7f32f77fe640 1 -- 172.21.15.88:0/2139459377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32f80ca120 msgr2=0x7f32f80ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:26.847 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.843+0000 7f32f77fe640 1 --2- 172.21.15.88:0/2139459377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32f80ca120 0x7f32f80ad260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:26.847 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.843+0000 7f32f77fe640 1 -- 172.21.15.88:0/2139459377 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ec009420 con 0x7f32f815d190 2023-10-16T20:06:26.848 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.843+0000 7f32f6ffd640 1 --2- 172.21.15.88:0/2139459377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32f80ca120 0x7f32f80ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:26.848 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.843+0000 7f32f67fc640 1 --2- 172.21.15.88:0/2139459377 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32f80ce940 0x7f32f80ad7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:26.848 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.843+0000 7f32f77fe640 1 --2- 172.21.15.88:0/2139459377 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32f815d190 0x7f32f81561f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f32f000b970 tx=0x7f32f000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:26.848 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.843+0000 7f32df7fe640 1 -- 172.21.15.88:0/2139459377 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f000cb40 con 0x7f32f815d190 2023-10-16T20:06:26.848 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.843+0000 7f32df7fe640 1 -- 172.21.15.88:0/2139459377 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f000cca0 con 0x7f32f815d190 2023-10-16T20:06:26.849 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.843+0000 7f32df7fe640 1 -- 172.21.15.88:0/2139459377 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f0014680 con 0x7f32f815d190 2023-10-16T20:06:26.849 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.843+0000 7f32fd454640 1 -- 172.21.15.88:0/2139459377 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f815baa0 con 0x7f32f815d190 2023-10-16T20:06:26.849 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.843+0000 7f32fd454640 1 -- 172.21.15.88:0/2139459377 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f815b120 con 0x7f32f815d190 2023-10-16T20:06:26.850 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.843+0000 7f32fd454640 1 -- 172.21.15.88:0/2139459377 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f815c2a0 con 0x7f32f815d190 2023-10-16T20:06:26.850 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.843+0000 7f32df7fe640 1 -- 172.21.15.88:0/2139459377 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f32f001d3b0 con 0x7f32f815d190 2023-10-16T20:06:26.851 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.847+0000 7f32df7fe640 1 --2- 172.21.15.88:0/2139459377 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f32e40664a0 0x7f32e4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:26.851 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.847+0000 7f32df7fe640 1 -- 172.21.15.88:0/2139459377 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f32f0083dc0 con 0x7f32f815d190 2023-10-16T20:06:26.851 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.847+0000 7f32f6ffd640 1 --2- 172.21.15.88:0/2139459377 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f32e40664a0 0x7f32e4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:26.851 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.847+0000 7f32f6ffd640 1 --2- 172.21.15.88:0/2139459377 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f32e40664a0 0x7f32e4068960 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f32e8007fd0 tx=0x7f32e8007c20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:26.852 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:26.851+0000 7f32df7fe640 1 -- 172.21.15.88:0/2139459377 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f32f815c2a0 con 0x7f32f815d190 2023-10-16T20:06:27.053 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.051+0000 7f32fd454640 1 -- 172.21.15.88:0/2139459377 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=testcephfs1", "mds", "allow"]} v 0) v1 -- 0x7f32f815c2a0 con 0x7f32f815d190 2023-10-16T20:06:27.056 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.055+0000 7f32df7fe640 1 -- 172.21.15.88:0/2139459377 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=testcephfs1", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 195+0+152 (secure 0 0 0) 0x7f32f815c2a0 con 0x7f32f815d190 2023-10-16T20:06:27.057 INFO:teuthology.orchestra.run.smithi088.stdout:[client.0] 2023-10-16T20:06:27.057 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQAvly1looeRAhAAeLshHOVIhQqoTtlPehorIg== 2023-10-16T20:06:27.057 INFO:teuthology.orchestra.run.smithi088.stdout: caps mds = "allow" 2023-10-16T20:06:27.058 INFO:teuthology.orchestra.run.smithi088.stdout: caps mon = "allow r" 2023-10-16T20:06:27.058 INFO:teuthology.orchestra.run.smithi088.stdout: caps osd = "allow rw tag cephfs data=testcephfs1" 2023-10-16T20:06:27.058 INFO:teuthology.orchestra.run.smithi088.stderr:updated caps for client.0 2023-10-16T20:06:27.059 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.055+0000 7f32fd454640 1 -- 172.21.15.88:0/2139459377 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f32e40664a0 msgr2=0x7f32e4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:27.059 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.055+0000 7f32fd454640 1 --2- 172.21.15.88:0/2139459377 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f32e40664a0 0x7f32e4068960 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f32e8007fd0 tx=0x7f32e8007c20 comp rx=0 tx=0).stop 2023-10-16T20:06:27.060 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.055+0000 7f32fd454640 1 -- 172.21.15.88:0/2139459377 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32f815d190 msgr2=0x7f32f81561f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:27.060 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.055+0000 7f32fd454640 1 --2- 172.21.15.88:0/2139459377 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32f815d190 0x7f32f81561f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f32f000b970 tx=0x7f32f000be30 comp rx=0 tx=0).stop 2023-10-16T20:06:27.060 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.059+0000 7f32fd454640 1 -- 172.21.15.88:0/2139459377 shutdown_connections 2023-10-16T20:06:27.060 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.059+0000 7f32fd454640 1 --2- 172.21.15.88:0/2139459377 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f32e40664a0 0x7f32e4068960 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:27.060 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.059+0000 7f32fd454640 1 --2- 172.21.15.88:0/2139459377 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32f80ce940 0x7f32f80ad7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:27.061 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.059+0000 7f32fd454640 1 --2- 172.21.15.88:0/2139459377 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32f815d190 0x7f32f81561f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:27.061 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.059+0000 7f32fd454640 1 --2- 172.21.15.88:0/2139459377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32f80ca120 0x7f32f80ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:27.061 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.059+0000 7f32fd454640 1 -- 172.21.15.88:0/2139459377 >> 172.21.15.88:0/2139459377 conn(0x7f32f806a290 msgr2=0x7f32f80b0640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:27.061 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.059+0000 7f32fd454640 1 -- 172.21.15.88:0/2139459377 shutdown_connections 2023-10-16T20:06:27.061 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.059+0000 7f32fd454640 1 -- 172.21.15.88:0/2139459377 wait complete. 2023-10-16T20:06:27.078 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T20:06:27.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.215+0000 7f49da054640 1 Processor -- start 2023-10-16T20:06:27.217 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.215+0000 7f49da054640 1 -- start start 2023-10-16T20:06:27.217 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.215+0000 7f49da054640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f49d415d9f0 0x7f49d4161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:27.218 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.215+0000 7f49da054640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f49d40d26c0 0x7f49d40cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:27.218 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.215+0000 7f49da054640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d40b87e0 con 0x7f49d40d22e0 2023-10-16T20:06:27.218 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.215+0000 7f49da054640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d415df10 con 0x7f49d40d26c0 2023-10-16T20:06:27.218 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.215+0000 7f49da054640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d415e1f0 con 0x7f49d415d9f0 2023-10-16T20:06:27.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.215+0000 7f49d37fe640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f49d40d22e0 0x7f49d415d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:51160/0 (socket says 172.21.15.88:51160) 2023-10-16T20:06:27.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.215+0000 7f49d37fe640 1 -- 172.21.15.88:0/3508084514 learned_addr learned my addr 172.21.15.88:0/3508084514 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:27.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.215+0000 7f49d3fff640 1 --2- 172.21.15.88:0/3508084514 >> v2:172.21.15.88:3300/0 conn(0x7f49d40d26c0 0x7f49d40cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:27.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.215+0000 7f49d2ffd640 1 --2- 172.21.15.88:0/3508084514 >> v2:172.21.15.94:3300/0 conn(0x7f49d415d9f0 0x7f49d4161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:27.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.215+0000 7f49d27fc640 1 -- 172.21.15.88:0/3508084514 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1262595789 0 0) 0x7f49d40b87e0 con 0x7f49d40d22e0 2023-10-16T20:06:27.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.215+0000 7f49d27fc640 1 -- 172.21.15.88:0/3508084514 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49bc003530 con 0x7f49d40d22e0 2023-10-16T20:06:27.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.215+0000 7f49d3fff640 1 -- 172.21.15.88:0/3508084514 >> v2:172.21.15.94:3300/0 conn(0x7f49d415d9f0 msgr2=0x7f49d4161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:27.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.215+0000 7f49d3fff640 1 --2- 172.21.15.88:0/3508084514 >> v2:172.21.15.94:3300/0 conn(0x7f49d415d9f0 0x7f49d4161450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:27.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.215+0000 7f49d3fff640 1 -- 172.21.15.88:0/3508084514 >> v1:172.21.15.88:6789/0 conn(0x7f49d40d22e0 legacy=0x7f49d415d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:27.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.215+0000 7f49d3fff640 1 -- 172.21.15.88:0/3508084514 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d4160970 con 0x7f49d40d26c0 2023-10-16T20:06:27.222 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.219+0000 7f49d3fff640 1 --2- 172.21.15.88:0/3508084514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49d40d26c0 0x7f49d40cf2e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f49d4072520 tx=0x7f49cc01c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b17379b3668e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:27.222 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.219+0000 7f49d27fc640 1 -- 172.21.15.88:0/3508084514 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc0a5910 con 0x7f49d40d26c0 2023-10-16T20:06:27.222 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.219+0000 7f49d27fc640 1 -- 172.21.15.88:0/3508084514 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49cc0a5a70 con 0x7f49d40d26c0 2023-10-16T20:06:27.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.219+0000 7f49d27fc640 1 -- 172.21.15.88:0/3508084514 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc0a5e00 con 0x7f49d40d26c0 2023-10-16T20:06:27.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.219+0000 7f49da054640 1 -- 172.21.15.88:0/3508084514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49d40d26c0 msgr2=0x7f49d40cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:27.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.219+0000 7f49da054640 1 --2- 172.21.15.88:0/3508084514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49d40d26c0 0x7f49d40cf2e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f49d4072520 tx=0x7f49cc01c850 comp rx=0 tx=0).stop 2023-10-16T20:06:27.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.219+0000 7f49da054640 1 -- 172.21.15.88:0/3508084514 shutdown_connections 2023-10-16T20:06:27.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.219+0000 7f49da054640 1 --2- 172.21.15.88:0/3508084514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49d40d26c0 0x7f49d40cf2e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:27.224 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.219+0000 7f49da054640 1 --2- 172.21.15.88:0/3508084514 >> v2:172.21.15.94:3300/0 conn(0x7f49d415d9f0 0x7f49d4161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:27.224 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.219+0000 7f49da054640 1 -- 172.21.15.88:0/3508084514 >> 172.21.15.88:0/3508084514 conn(0x7f49d40715c0 msgr2=0x7f49d40719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:27.224 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.219+0000 7f49da054640 1 -- 172.21.15.88:0/3508084514 shutdown_connections 2023-10-16T20:06:27.224 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.219+0000 7f49da054640 1 -- 172.21.15.88:0/3508084514 wait complete. 2023-10-16T20:06:27.226 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.223+0000 7f49da054640 1 Processor -- start 2023-10-16T20:06:27.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.223+0000 7f49da054640 1 -- start start 2023-10-16T20:06:27.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.223+0000 7f49da054640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49d40d22e0 0x7f49d40c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:27.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.223+0000 7f49da054640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49d40d26c0 0x7f49d40c9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:27.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.223+0000 7f49da054640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49d415d9f0 0x7f49d414a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:27.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.223+0000 7f49da054640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d40c7fe0 con 0x7f49d40d22e0 2023-10-16T20:06:27.228 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.223+0000 7f49da054640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d40c82c0 con 0x7f49d40d26c0 2023-10-16T20:06:27.228 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.223+0000 7f49da054640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d40c85a0 con 0x7f49d415d9f0 2023-10-16T20:06:27.228 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.223+0000 7f49d37fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49d40d22e0 0x7f49d40c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:27.228 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.223+0000 7f49d37fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49d40d22e0 0x7f49d40c7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57618/0 (socket says 172.21.15.88:57618) 2023-10-16T20:06:27.228 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.223+0000 7f49d37fe640 1 -- 172.21.15.88:0/2623281396 learned_addr learned my addr 172.21.15.88:0/2623281396 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:27.228 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.223+0000 7f49d3fff640 1 --2- 172.21.15.88:0/2623281396 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49d415d9f0 0x7f49d414a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:27.229 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.223+0000 7f49d2ffd640 1 --2- 172.21.15.88:0/2623281396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49d40d26c0 0x7f49d40c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:27.229 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.223+0000 7f49d37fe640 1 -- 172.21.15.88:0/2623281396 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49d415d9f0 msgr2=0x7f49d414a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:27.229 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.223+0000 7f49d37fe640 1 --2- 172.21.15.88:0/2623281396 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49d415d9f0 0x7f49d414a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:27.229 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.223+0000 7f49d37fe640 1 -- 172.21.15.88:0/2623281396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49d40d26c0 msgr2=0x7f49d40c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:27.229 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.223+0000 7f49d37fe640 1 --2- 172.21.15.88:0/2623281396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49d40d26c0 0x7f49d40c9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:27.230 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.223+0000 7f49d37fe640 1 -- 172.21.15.88:0/2623281396 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc009a70 con 0x7f49d40d22e0 2023-10-16T20:06:27.230 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.223+0000 7f49d2ffd640 1 --2- 172.21.15.88:0/2623281396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49d40d26c0 0x7f49d40c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:27.230 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.223+0000 7f49d37fe640 1 --2- 172.21.15.88:0/2623281396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49d40d22e0 0x7f49d40c7a00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f49c400c4d0 tx=0x7f49c400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:27.230 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.223+0000 7f49bbfff640 1 -- 172.21.15.88:0/2623281396 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c4005cf0 con 0x7f49d40d22e0 2023-10-16T20:06:27.230 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.223+0000 7f49bbfff640 1 -- 172.21.15.88:0/2623281396 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c4005e50 con 0x7f49d40d22e0 2023-10-16T20:06:27.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.223+0000 7f49bbfff640 1 -- 172.21.15.88:0/2623281396 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c4007270 con 0x7f49d40d22e0 2023-10-16T20:06:27.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.223+0000 7f49da054640 1 -- 172.21.15.88:0/2623281396 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d40c8880 con 0x7f49d40d22e0 2023-10-16T20:06:27.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.223+0000 7f49da054640 1 -- 172.21.15.88:0/2623281396 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d41570d0 con 0x7f49d40d22e0 2023-10-16T20:06:27.233 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.227+0000 7f49da054640 1 -- 172.21.15.88:0/2623281396 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49d404e9c0 con 0x7f49d40d22e0 2023-10-16T20:06:27.233 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.227+0000 7f49bbfff640 1 -- 172.21.15.88:0/2623281396 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f49c40073d0 con 0x7f49d40d22e0 2023-10-16T20:06:27.233 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.227+0000 7f49bbfff640 1 --2- 172.21.15.88:0/2623281396 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f49c00664a0 0x7f49c0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:27.234 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.227+0000 7f49bbfff640 1 -- 172.21.15.88:0/2623281396 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f49c40835d0 con 0x7f49d40d22e0 2023-10-16T20:06:27.234 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.227+0000 7f49d2ffd640 1 --2- 172.21.15.88:0/2623281396 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f49c00664a0 0x7f49c0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:27.234 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.227+0000 7f49d2ffd640 1 --2- 172.21.15.88:0/2623281396 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f49c00664a0 0x7f49c0068960 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f49d40cd010 tx=0x7f49c8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:27.234 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.231+0000 7f49bbfff640 1 -- 172.21.15.88:0/2623281396 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f49c4050900 con 0x7f49d40d22e0 2023-10-16T20:06:27.414 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.411+0000 7f49da054640 1 -- 172.21.15.88:0/2623281396 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49d40d0840 con 0x7f49d40d22e0 2023-10-16T20:06:27.415 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.411+0000 7f49bbfff640 1 -- 172.21.15.88:0/2623281396 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f49c4053f20 con 0x7f49d40d22e0 2023-10-16T20:06:27.415 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:06:27.415 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/696292235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":696292235},{"type":"v1","addr":"172.21.15.94:6834","nonce":696292235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/2152672724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2152672724},{"type":"v1","addr":"172.21.15.88:6835","nonce":2152672724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3107974207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":3107974207},{"type":"v1","addr":"172.21.15.88:6837","nonce":3107974207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T20:06:17.088386+0000","modified":"2023-10-16T20:06:20.108161+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5535":{"gid":5535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6835/2190281591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":2190281591},{"type":"v1","addr":"172.21.15.94:6835","nonce":2190281591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"testcephfs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-16T20:06:27.416 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 18 2023-10-16T20:06:27.417 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.415+0000 7f49da054640 1 -- 172.21.15.88:0/2623281396 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f49c00664a0 msgr2=0x7f49c0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:27.417 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.415+0000 7f49da054640 1 --2- 172.21.15.88:0/2623281396 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f49c00664a0 0x7f49c0068960 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f49d40cd010 tx=0x7f49c8008040 comp rx=0 tx=0).stop 2023-10-16T20:06:27.418 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.415+0000 7f49da054640 1 -- 172.21.15.88:0/2623281396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49d40d22e0 msgr2=0x7f49d40c7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:27.418 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.415+0000 7f49da054640 1 --2- 172.21.15.88:0/2623281396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49d40d22e0 0x7f49d40c7a00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f49c400c4d0 tx=0x7f49c400c990 comp rx=0 tx=0).stop 2023-10-16T20:06:27.418 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.415+0000 7f49da054640 1 -- 172.21.15.88:0/2623281396 shutdown_connections 2023-10-16T20:06:27.418 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.415+0000 7f49da054640 1 --2- 172.21.15.88:0/2623281396 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f49c00664a0 0x7f49c0068960 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:27.418 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.415+0000 7f49da054640 1 --2- 172.21.15.88:0/2623281396 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49d415d9f0 0x7f49d414a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:27.419 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.415+0000 7f49da054640 1 --2- 172.21.15.88:0/2623281396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49d40d26c0 0x7f49d40c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:27.419 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.415+0000 7f49da054640 1 --2- 172.21.15.88:0/2623281396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49d40d22e0 0x7f49d40c7a00 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:27.419 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.415+0000 7f49da054640 1 -- 172.21.15.88:0/2623281396 >> 172.21.15.88:0/2623281396 conn(0x7f49d40715c0 msgr2=0x7f49d40719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:27.419 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.415+0000 7f49da054640 1 -- 172.21.15.88:0/2623281396 shutdown_connections 2023-10-16T20:06:27.419 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.415+0000 7f49da054640 1 -- 172.21.15.88:0/2623281396 wait complete. 2023-10-16T20:06:27.839 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T20:06:17.088386+0000', 'modified': '2023-10-16T20:06:20.108161+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5535}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5535': {'gid': 5535, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.94:6835/2190281591', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6833', 'nonce': 2190281591}, {'type': 'v1', 'addr': '172.21.15.94:6835', 'nonce': 2190281591}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'testcephfs1', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T20:06:27.839 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T20:06:27.840 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-16T20:06:27.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.979+0000 7f3b82246640 1 Processor -- start 2023-10-16T20:06:27.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.979+0000 7f3b82246640 1 -- start start 2023-10-16T20:06:27.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.979+0000 7f3b82246640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3b7c069e90 0x7f3b7c06a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:27.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.979+0000 7f3b82246640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3b7c0adf80 0x7f3b7c15a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:27.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.979+0000 7f3b82246640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b7c15e400 con 0x7f3b7c15d480 2023-10-16T20:06:27.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.979+0000 7f3b82246640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b7c159e40 con 0x7f3b7c069e90 2023-10-16T20:06:27.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.979+0000 7f3b82246640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c06a7b0 con 0x7f3b7c0adf80 2023-10-16T20:06:27.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.979+0000 7f3b7bfff640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f3b7c15d480 0x7f3b7c0afa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:51168/0 (socket says 172.21.15.88:51168) 2023-10-16T20:06:27.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.979+0000 7f3b7bfff640 1 -- 172.21.15.88:0/694238808 learned_addr learned my addr 172.21.15.88:0/694238808 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:27.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.983+0000 7f3b7b7fe640 1 --2- 172.21.15.88:0/694238808 >> v2:172.21.15.94:3300/0 conn(0x7f3b7c069e90 0x7f3b7c06a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:27.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.983+0000 7f3b73fff640 1 --2- 172.21.15.88:0/694238808 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3b7c0adf80 0x7f3b7c15a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:27.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.983+0000 7f3b7affd640 1 -- 172.21.15.88:0/694238808 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2190013296 0 0) 0x7f3b7c15e400 con 0x7f3b7c15d480 2023-10-16T20:06:27.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.983+0000 7f3b7affd640 1 -- 172.21.15.88:0/694238808 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b68003530 con 0x7f3b7c15d480 2023-10-16T20:06:27.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.983+0000 7f3b7b7fe640 1 -- 172.21.15.88:0/694238808 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3b7c0adf80 msgr2=0x7f3b7c15a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:27.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.983+0000 7f3b7b7fe640 1 --2- 172.21.15.88:0/694238808 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3b7c0adf80 0x7f3b7c15a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:27.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.983+0000 7f3b7b7fe640 1 -- 172.21.15.88:0/694238808 >> v1:172.21.15.88:6789/0 conn(0x7f3b7c15d480 legacy=0x7f3b7c0afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:27.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.983+0000 7f3b7b7fe640 1 -- 172.21.15.88:0/694238808 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b7c15de40 con 0x7f3b7c069e90 2023-10-16T20:06:27.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.983+0000 7f3b7b7fe640 1 --2- 172.21.15.88:0/694238808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b7c069e90 0x7f3b7c06a270 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c0097c0 tx=0x7f3b6c01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e915d7758d9529b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:27.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.987+0000 7f3b7affd640 1 -- 172.21.15.88:0/694238808 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c004230 con 0x7f3b7c069e90 2023-10-16T20:06:27.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.987+0000 7f3b7affd640 1 -- 172.21.15.88:0/694238808 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b6c0a7050 con 0x7f3b7c069e90 2023-10-16T20:06:27.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.987+0000 7f3b7affd640 1 -- 172.21.15.88:0/694238808 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c003620 con 0x7f3b7c069e90 2023-10-16T20:06:27.988 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.987+0000 7f3b82246640 1 -- 172.21.15.88:0/694238808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b7c069e90 msgr2=0x7f3b7c06a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:27.988 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.987+0000 7f3b82246640 1 --2- 172.21.15.88:0/694238808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b7c069e90 0x7f3b7c06a270 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c0097c0 tx=0x7f3b6c01c450 comp rx=0 tx=0).stop 2023-10-16T20:06:27.989 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.987+0000 7f3b82246640 1 -- 172.21.15.88:0/694238808 shutdown_connections 2023-10-16T20:06:27.989 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.987+0000 7f3b82246640 1 --2- 172.21.15.88:0/694238808 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3b7c0adf80 0x7f3b7c15a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:27.989 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.987+0000 7f3b82246640 1 --2- 172.21.15.88:0/694238808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b7c069e90 0x7f3b7c06a270 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:27.989 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.987+0000 7f3b82246640 1 -- 172.21.15.88:0/694238808 >> 172.21.15.88:0/694238808 conn(0x7f3b7c0ce5e0 msgr2=0x7f3b7c0ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:27.990 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.987+0000 7f3b82246640 1 -- 172.21.15.88:0/694238808 shutdown_connections 2023-10-16T20:06:27.990 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.987+0000 7f3b82246640 1 -- 172.21.15.88:0/694238808 wait complete. 2023-10-16T20:06:27.994 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.987+0000 7f3b82246640 1 Processor -- start 2023-10-16T20:06:27.994 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.987+0000 7f3b82246640 1 -- start start 2023-10-16T20:06:27.994 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.987+0000 7f3b82246640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3b7c069e90 0x7f3b7c070130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:27.995 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.987+0000 7f3b82246640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b7c0adf80 0x7f3b7c070670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:27.995 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.987+0000 7f3b82246640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b7c15d480 0x7f3b7c070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:27.995 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.987+0000 7f3b82246640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c0718c0 con 0x7f3b7c0adf80 2023-10-16T20:06:27.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.987+0000 7f3b82246640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c156920 con 0x7f3b7c15d480 2023-10-16T20:06:27.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.987+0000 7f3b82246640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c156e50 con 0x7f3b7c069e90 2023-10-16T20:06:27.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.987+0000 7f3b73fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b7c0adf80 0x7f3b7c070670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:27.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.987+0000 7f3b73fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b7c0adf80 0x7f3b7c070670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57628/0 (socket says 172.21.15.88:57628) 2023-10-16T20:06:27.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.987+0000 7f3b73fff640 1 -- 172.21.15.88:0/952474478 learned_addr learned my addr 172.21.15.88:0/952474478 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:27.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.987+0000 7f3b7bfff640 1 --2- 172.21.15.88:0/952474478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b7c15d480 0x7f3b7c070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:27.998 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.991+0000 7f3b7b7fe640 1 --2- 172.21.15.88:0/952474478 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3b7c069e90 0x7f3b7c070130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:27.998 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.991+0000 7f3b73fff640 1 -- 172.21.15.88:0/952474478 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3b7c069e90 msgr2=0x7f3b7c070130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:27.999 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.991+0000 7f3b73fff640 1 --2- 172.21.15.88:0/952474478 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3b7c069e90 0x7f3b7c070130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:27.999 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.991+0000 7f3b73fff640 1 -- 172.21.15.88:0/952474478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b7c15d480 msgr2=0x7f3b7c070bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:27.999 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.991+0000 7f3b73fff640 1 --2- 172.21.15.88:0/952474478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b7c15d480 0x7f3b7c070bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:27.999 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.991+0000 7f3b73fff640 1 -- 172.21.15.88:0/952474478 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b6c009420 con 0x7f3b7c0adf80 2023-10-16T20:06:28.000 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.991+0000 7f3b73fff640 1 --2- 172.21.15.88:0/952474478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b7c0adf80 0x7f3b7c070670 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400be50 tx=0x7f3b64013490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:28.000 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.991+0000 7f3b78ff9640 1 -- 172.21.15.88:0/952474478 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b64015040 con 0x7f3b7c0adf80 2023-10-16T20:06:28.000 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.991+0000 7f3b78ff9640 1 -- 172.21.15.88:0/952474478 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b64019400 con 0x7f3b7c0adf80 2023-10-16T20:06:28.000 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.991+0000 7f3b78ff9640 1 -- 172.21.15.88:0/952474478 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b64010a80 con 0x7f3b7c0adf80 2023-10-16T20:06:28.000 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.991+0000 7f3b82246640 1 -- 172.21.15.88:0/952474478 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b7c156b70 con 0x7f3b7c0adf80 2023-10-16T20:06:28.001 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.991+0000 7f3b82246640 1 -- 172.21.15.88:0/952474478 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b7c071550 con 0x7f3b7c0adf80 2023-10-16T20:06:28.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.991+0000 7f3b78ff9640 1 -- 172.21.15.88:0/952474478 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3b64019770 con 0x7f3b7c0adf80 2023-10-16T20:06:28.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.995+0000 7f3b78ff9640 1 --2- 172.21.15.88:0/952474478 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3b5c0664a0 0x7f3b5c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:28.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.995+0000 7f3b78ff9640 1 -- 172.21.15.88:0/952474478 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f3b640888c0 con 0x7f3b7c0adf80 2023-10-16T20:06:28.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.995+0000 7f3b855d3000 1 Processor -- start 2023-10-16T20:06:28.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.995+0000 7f3b7b7fe640 1 --2- 172.21.15.88:0/952474478 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3b5c0664a0 0x7f3b5c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:28.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.995+0000 7f3b855d3000 1 -- start start 2023-10-16T20:06:28.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.995+0000 7f3b7b7fe640 1 --2- 172.21.15.88:0/952474478 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3b5c0664a0 0x7f3b5c068960 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c0ad000 tx=0x7f3b6c0a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:28.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.995+0000 7f3b855d3000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a164a7a8a0 0x55a164a7ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:28.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.995+0000 7f3b855d3000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a164a7b1c0 0x55a164a30710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:28.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.995+0000 7f3b855d3000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a164a30c50 0x55a164a31030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:28.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.995+0000 7f3b855d3000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a164825cc0 con 0x55a164a7b1c0 2023-10-16T20:06:28.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.995+0000 7f3b855d3000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a164888920 con 0x55a164a30c50 2023-10-16T20:06:28.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.995+0000 7f3b855d3000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a164885e50 con 0x55a164a7a8a0 2023-10-16T20:06:28.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.995+0000 7f3b73fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a164a7b1c0 0x55a164a30710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:28.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.995+0000 7f3b7bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a164a30c50 0x55a164a31030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:28.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.995+0000 7f3b7b7fe640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a164a7a8a0 0x55a164a7ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:28.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.995+0000 7f3b73fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a164a7b1c0 0x55a164a30710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57640/0 (socket says 172.21.15.88:57640) 2023-10-16T20:06:28.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.995+0000 7f3b7bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a164a30c50 0x55a164a31030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:39770/0 (socket says 172.21.15.88:39770) 2023-10-16T20:06:28.010 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.995+0000 7f3b73fff640 1 -- 172.21.15.88:0/1200948620 learned_addr learned my addr 172.21.15.88:0/1200948620 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:28.010 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.995+0000 7f3b7b7fe640 1 --2- 172.21.15.88:0/1200948620 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a164a7a8a0 0x55a164a7ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:49306/0 (socket says 172.21.15.88:49306) 2023-10-16T20:06:28.010 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.995+0000 7f3b73fff640 1 -- 172.21.15.88:0/1200948620 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a164a7a8a0 msgr2=0x55a164a7ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:28.010 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.995+0000 7f3b73fff640 1 --2- 172.21.15.88:0/1200948620 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a164a7a8a0 0x55a164a7ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.995+0000 7f3b73fff640 1 -- 172.21.15.88:0/1200948620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a164a30c50 msgr2=0x55a164a31030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:28.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.995+0000 7f3b73fff640 1 --2- 172.21.15.88:0/1200948620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a164a30c50 0x55a164a31030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.995+0000 7f3b73fff640 1 -- 172.21.15.88:0/1200948620 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a16457a890 con 0x55a164a7b1c0 2023-10-16T20:06:28.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.995+0000 7f3b73fff640 1 --2- 172.21.15.88:0/1200948620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a164a7b1c0 0x55a164a30710 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3b64059790 tx=0x7f3b640598c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b472c772e540c9e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:28.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.995+0000 7f3b717fa640 1 -- 172.21.15.88:0/1200948620 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b64057ba0 con 0x55a164a7b1c0 2023-10-16T20:06:28.013 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.995+0000 7f3b717fa640 1 -- 172.21.15.88:0/1200948620 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b64055f10 con 0x55a164a7b1c0 2023-10-16T20:06:28.013 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.995+0000 7f3b717fa640 1 -- 172.21.15.88:0/1200948620 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b640618b0 con 0x55a164a7b1c0 2023-10-16T20:06:28.014 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b855d3000 1 -- 172.21.15.88:0/1200948620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a164a7b1c0 msgr2=0x55a164a30710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:28.014 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b855d3000 1 --2- 172.21.15.88:0/1200948620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a164a7b1c0 0x55a164a30710 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3b64059790 tx=0x7f3b640598c0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.014 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b855d3000 1 -- 172.21.15.88:0/1200948620 shutdown_connections 2023-10-16T20:06:28.014 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b855d3000 1 --2- 172.21.15.88:0/1200948620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a164a30c50 0x55a164a31030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.014 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b855d3000 1 --2- 172.21.15.88:0/1200948620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a164a7b1c0 0x55a164a30710 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b855d3000 1 --2- 172.21.15.88:0/1200948620 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a164a7a8a0 0x55a164a7ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b855d3000 1 -- 172.21.15.88:0/1200948620 >> 172.21.15.88:0/1200948620 conn(0x55a164a27fc0 msgr2=0x55a164a2eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:28.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b855d3000 1 -- 172.21.15.88:0/1200948620 shutdown_connections 2023-10-16T20:06:28.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b855d3000 1 -- 172.21.15.88:0/1200948620 wait complete. 2023-10-16T20:06:28.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b855d3000 1 -- start start 2023-10-16T20:06:28.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b855d3000 1 Processor -- start 2023-10-16T20:06:28.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b855d3000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a164a30c50 0x55a164a35fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:28.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b855d3000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a164a7a8a0 0x55a164a36500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:28.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b855d3000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a164a7b1c0 0x55a164a68d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:28.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b855d3000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a164881740 con 0x55a164a7b1c0 2023-10-16T20:06:28.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b855d3000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a164840d10 con 0x55a164a7a8a0 2023-10-16T20:06:28.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b855d3000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1648409b0 con 0x55a164a30c50 2023-10-16T20:06:28.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b855d3000 10 client.0 _ll_register_callbacks cb 0x55a164a1c390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T20:06:28.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b7bfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a164a7b1c0 0x55a164a68d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:28.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b7bfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a164a7b1c0 0x55a164a68d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57648/0 (socket says 172.21.15.88:57648) 2023-10-16T20:06:28.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b7bfff640 1 -- 172.21.15.88:0/3216533633 learned_addr learned my addr 172.21.15.88:0/3216533633 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:28.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b73fff640 1 --2- 172.21.15.88:0/3216533633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a164a7a8a0 0x55a164a36500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:28.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b7b7fe640 1 --2- 172.21.15.88:0/3216533633 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a164a30c50 0x55a164a35fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:28.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b7bfff640 1 -- 172.21.15.88:0/3216533633 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a164a30c50 msgr2=0x55a164a35fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:28.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b7bfff640 1 --2- 172.21.15.88:0/3216533633 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a164a30c50 0x55a164a35fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.019 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b7bfff640 1 -- 172.21.15.88:0/3216533633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a164a7a8a0 msgr2=0x55a164a36500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:28.019 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b7bfff640 1 --2- 172.21.15.88:0/3216533633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a164a7a8a0 0x55a164a36500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.019 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b7bfff640 1 -- 172.21.15.88:0/3216533633 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b7c071550 con 0x55a164a7b1c0 2023-10-16T20:06:28.019 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b7bfff640 1 --2- 172.21.15.88:0/3216533633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a164a7b1c0 0x55a164a68d80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f3b74007b00 tx=0x7f3b74012690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f47faaac0491b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:28.019 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b5a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2023-10-16T20:06:28.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b5a7fc640 1 -- 172.21.15.88:0/3216533633 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b74015440 con 0x55a164a7b1c0 2023-10-16T20:06:28.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b855d3000 1 -- 172.21.15.88:0/3216533633 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a164835370 con 0x55a164a7b1c0 2023-10-16T20:06:28.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b5a7fc640 1 -- 172.21.15.88:0/3216533633 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f3b740155a0 con 0x55a164a7b1c0 2023-10-16T20:06:28.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b5a7fc640 1 -- 172.21.15.88:0/3216533633 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b74015760 con 0x55a164a7b1c0 2023-10-16T20:06:28.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b855d3000 10 client.5727 fetch_fsmap learned FSMap version 18 2023-10-16T20:06:28.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b855d3000 1 -- 172.21.15.88:0/3216533633 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55a164843460 con 0x55a164a7b1c0 2023-10-16T20:06:28.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b5a7fc640 1 -- 172.21.15.88:0/3216533633 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a164835370 con 0x55a164a7b1c0 2023-10-16T20:06:28.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b5a7fc640 1 -- 172.21.15.88:0/3216533633 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 18) v1 ==== 3299+0+0 (secure 0 0 0) 0x7f3b740158c0 con 0x55a164a7b1c0 2023-10-16T20:06:28.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b855d3000 10 client.5727 fetch_fsmap finished waiting for FSMap version 18 2023-10-16T20:06:28.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:27.999+0000 7f3b855d3000 10 client.5727 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-16T20:06:28.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.003+0000 7f3b855d3000 20 client.5727 populate_metadata read hostname 'smithi088' 2023-10-16T20:06:28.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.003+0000 7f3b855d3000 1 --2- 172.21.15.88:0/3216533633 >> [v2:172.21.15.94:6833/2190281591,v1:172.21.15.94:6835/2190281591] conn(0x55a164a8a940 0x55a164a8cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:28.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.003+0000 7f3b855d3000 4 client.5727 mds_command: new command op to 5535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T20:06:28.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.003+0000 7f3b855d3000 1 -- 172.21.15.88:0/3216533633 --> [v2:172.21.15.94:6833/2190281591,v1:172.21.15.94:6835/2190281591] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a16481b2d0 con 0x55a164a8a940 2023-10-16T20:06:28.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.003+0000 7f3b7b7fe640 1 --2- 172.21.15.88:0/3216533633 >> [v2:172.21.15.94:6833/2190281591,v1:172.21.15.94:6835/2190281591] conn(0x55a164a8a940 0x55a164a8cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:28.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.003+0000 7f3b7b7fe640 1 --2- 172.21.15.88:0/3216533633 >> [v2:172.21.15.94:6833/2190281591,v1:172.21.15.94:6835/2190281591] conn(0x55a164a8a940 0x55a164a8cd30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5de7b250b53cf5e server_cookie=4917d77b54298c20 in_seq=0 out_seq=0 2023-10-16T20:06:28.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.003+0000 7f3b5a7fc640 10 client.5727 ms_handle_connect on v2:172.21.15.94:6833/2190281591 2023-10-16T20:06:28.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.003+0000 7f3b5a7fc640 1 -- 172.21.15.88:0/3216533633 <== mds.0 v2:172.21.15.94:6833/2190281591 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a16481b2d0 con 0x55a164a8a940 2023-10-16T20:06:28.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.003+0000 7f3b5a7fc640 10 client.5727 handle_command_reply: tid=0 2023-10-16T20:06:28.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.003+0000 7f3b855d3000 1 client.5727 shutdown 2023-10-16T20:06:28.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.003+0000 7f3b855d3000 1 -- 172.21.15.88:0/3216533633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a164a7b1c0 msgr2=0x55a164a68d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:28.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.003+0000 7f3b855d3000 1 --2- 172.21.15.88:0/3216533633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a164a7b1c0 0x55a164a68d80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f3b74007b00 tx=0x7f3b74012690 comp rx=0 tx=0).stop 2023-10-16T20:06:28.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.003+0000 7f3b855d3000 1 -- 172.21.15.88:0/3216533633 shutdown_connections 2023-10-16T20:06:28.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.003+0000 7f3b855d3000 1 --2- 172.21.15.88:0/3216533633 >> [v2:172.21.15.94:6833/2190281591,v1:172.21.15.94:6835/2190281591] conn(0x55a164a8a940 0x55a164a8cd30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.003+0000 7f3b855d3000 1 --2- 172.21.15.88:0/3216533633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a164a7b1c0 0x55a164a68d80 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.003+0000 7f3b855d3000 1 --2- 172.21.15.88:0/3216533633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a164a7a8a0 0x55a164a36500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.003+0000 7f3b855d3000 1 --2- 172.21.15.88:0/3216533633 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a164a30c50 0x55a164a35fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.003+0000 7f3b5a7fc640 0 client.5727 ms_handle_reset on v2:172.21.15.94:6833/2190281591 2023-10-16T20:06:28.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.003+0000 7f3b855d3000 1 -- 172.21.15.88:0/3216533633 >> 172.21.15.88:0/3216533633 conn(0x55a164a27fc0 msgr2=0x55a164a2f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:28.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.003+0000 7f3b855d3000 1 -- 172.21.15.88:0/3216533633 shutdown_connections 2023-10-16T20:06:28.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.003+0000 7f3b855d3000 1 -- 172.21.15.88:0/3216533633 wait complete. 2023-10-16T20:06:28.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.003+0000 7f3b855d3000 20 client.5727 trim_cache size 0 max 16384 2023-10-16T20:06:28.030 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.019+0000 7f3b855d3000 1 Processor -- start 2023-10-16T20:06:28.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.019+0000 7f3b855d3000 1 -- start start 2023-10-16T20:06:28.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.019+0000 7f3b855d3000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a164a30c50 0x55a164a8e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:28.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.019+0000 7f3b855d3000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a164a7a8a0 0x55a164a8e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:28.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.019+0000 7f3b855d3000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a164a7b1c0 0x55a164a639a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:28.032 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.019+0000 7f3b855d3000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a16453b940 con 0x55a164a7b1c0 2023-10-16T20:06:28.032 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.019+0000 7f3b855d3000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a16487ba40 con 0x55a164a30c50 2023-10-16T20:06:28.032 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.019+0000 7f3b855d3000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1648404d0 con 0x55a164a7a8a0 2023-10-16T20:06:28.032 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.019+0000 7f3b7bfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a164a7b1c0 0x55a164a639a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:28.032 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.019+0000 7f3b7bfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a164a7b1c0 0x55a164a639a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57656/0 (socket says 172.21.15.88:57656) 2023-10-16T20:06:28.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.019+0000 7f3b7bfff640 1 -- 172.21.15.88:0/2344417503 learned_addr learned my addr 172.21.15.88:0/2344417503 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:28.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.019+0000 7f3b7b7fe640 1 --2- 172.21.15.88:0/2344417503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a164a30c50 0x55a164a8e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:28.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.019+0000 7f3b73fff640 1 --2- 172.21.15.88:0/2344417503 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a164a7a8a0 0x55a164a8e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:28.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.019+0000 7f3b7bfff640 1 -- 172.21.15.88:0/2344417503 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a164a7a8a0 msgr2=0x55a164a8e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:28.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.019+0000 7f3b7bfff640 1 --2- 172.21.15.88:0/2344417503 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a164a7a8a0 0x55a164a8e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.019+0000 7f3b7bfff640 1 -- 172.21.15.88:0/2344417503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a164a30c50 msgr2=0x55a164a8e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:28.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.019+0000 7f3b7bfff640 1 --2- 172.21.15.88:0/2344417503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a164a30c50 0x55a164a8e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.019+0000 7f3b7bfff640 1 -- 172.21.15.88:0/2344417503 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a164829760 con 0x55a164a7b1c0 2023-10-16T20:06:28.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.019+0000 7f3b73fff640 1 --2- 172.21.15.88:0/2344417503 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a164a7a8a0 0x55a164a8e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:06:28.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.019+0000 7f3b7bfff640 1 --2- 172.21.15.88:0/2344417503 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a164a7b1c0 0x55a164a639a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3b740126d0 tx=0x7f3b74016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27d17769b2e369f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:28.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.019+0000 7f3b71ffb640 1 -- 172.21.15.88:0/2344417503 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b74015d30 con 0x55a164a7b1c0 2023-10-16T20:06:28.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.019+0000 7f3b71ffb640 1 -- 172.21.15.88:0/2344417503 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b7400b040 con 0x55a164a7b1c0 2023-10-16T20:06:28.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.019+0000 7f3b71ffb640 1 -- 172.21.15.88:0/2344417503 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b74015680 con 0x55a164a7b1c0 2023-10-16T20:06:28.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b855d3000 1 -- 172.21.15.88:0/2344417503 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a164a7b1c0 msgr2=0x55a164a639a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:28.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b855d3000 1 --2- 172.21.15.88:0/2344417503 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a164a7b1c0 0x55a164a639a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3b740126d0 tx=0x7f3b74016000 comp rx=0 tx=0).stop 2023-10-16T20:06:28.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b855d3000 1 -- 172.21.15.88:0/2344417503 shutdown_connections 2023-10-16T20:06:28.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b855d3000 1 --2- 172.21.15.88:0/2344417503 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a164a7a8a0 0x55a164a8e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b855d3000 1 --2- 172.21.15.88:0/2344417503 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a164a7b1c0 0x55a164a639a0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b855d3000 1 --2- 172.21.15.88:0/2344417503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a164a30c50 0x55a164a8e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b855d3000 1 -- 172.21.15.88:0/2344417503 >> 172.21.15.88:0/2344417503 conn(0x55a164a27fc0 msgr2=0x55a164a8cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:28.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b855d3000 1 -- 172.21.15.88:0/2344417503 shutdown_connections 2023-10-16T20:06:28.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b855d3000 1 -- 172.21.15.88:0/2344417503 wait complete. 2023-10-16T20:06:28.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b855d3000 1 -- start start 2023-10-16T20:06:28.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b855d3000 1 Processor -- start 2023-10-16T20:06:28.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b855d3000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a164a30c50 0x55a164a67050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:28.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b855d3000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a164a7a8a0 0x55a164a960f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:28.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b855d3000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a164a7b1c0 0x55a164a96630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:28.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b855d3000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a164857a10 con 0x55a164a7b1c0 2023-10-16T20:06:28.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b855d3000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a16488b200 con 0x55a164a7a8a0 2023-10-16T20:06:28.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b855d3000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a164885740 con 0x55a164a30c50 2023-10-16T20:06:28.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b855d3000 10 client.0 _ll_register_callbacks cb 0x7f3b640595d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T20:06:28.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b7bfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a164a7b1c0 0x55a164a96630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:28.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b7bfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a164a7b1c0 0x55a164a96630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57670/0 (socket says 172.21.15.88:57670) 2023-10-16T20:06:28.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b7bfff640 1 -- 172.21.15.88:0/435465956 learned_addr learned my addr 172.21.15.88:0/435465956 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:28.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b7b7fe640 1 --2- 172.21.15.88:0/435465956 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a164a30c50 0x55a164a67050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:28.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b73fff640 1 --2- 172.21.15.88:0/435465956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a164a7a8a0 0x55a164a960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:28.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b7bfff640 1 -- 172.21.15.88:0/435465956 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a164a30c50 msgr2=0x55a164a67050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:28.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b7bfff640 1 --2- 172.21.15.88:0/435465956 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a164a30c50 0x55a164a67050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b7bfff640 1 -- 172.21.15.88:0/435465956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a164a7a8a0 msgr2=0x55a164a960f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:28.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b7bfff640 1 --2- 172.21.15.88:0/435465956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a164a7a8a0 0x55a164a960f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b7bfff640 1 -- 172.21.15.88:0/435465956 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a164843460 con 0x55a164a7b1c0 2023-10-16T20:06:28.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b7bfff640 1 --2- 172.21.15.88:0/435465956 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a164a7b1c0 0x55a164a96630 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3b7402b040 tx=0x7f3b740014e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7a9b85d908e260 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:28.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b5a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2023-10-16T20:06:28.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b5a7fc640 1 -- 172.21.15.88:0/435465956 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7401e640 con 0x55a164a7b1c0 2023-10-16T20:06:28.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b5a7fc640 1 -- 172.21.15.88:0/435465956 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f3b7403bc10 con 0x55a164a7b1c0 2023-10-16T20:06:28.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b855d3000 1 -- 172.21.15.88:0/435465956 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a164a333b0 con 0x55a164a7b1c0 2023-10-16T20:06:28.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b5a7fc640 1 -- 172.21.15.88:0/435465956 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b74015a00 con 0x55a164a7b1c0 2023-10-16T20:06:28.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b855d3000 10 client.5733 fetch_fsmap learned FSMap version 18 2023-10-16T20:06:28.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b855d3000 1 -- 172.21.15.88:0/435465956 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55a16457acc0 con 0x55a164a7b1c0 2023-10-16T20:06:28.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.023+0000 7f3b5a7fc640 1 -- 172.21.15.88:0/435465956 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a164a333b0 con 0x55a164a7b1c0 2023-10-16T20:06:28.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b5a7fc640 1 -- 172.21.15.88:0/435465956 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 18) v1 ==== 3299+0+0 (secure 0 0 0) 0x7f3b74038030 con 0x55a164a7b1c0 2023-10-16T20:06:28.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b855d3000 10 client.5733 fetch_fsmap finished waiting for FSMap version 18 2023-10-16T20:06:28.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b855d3000 10 client.5733 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-16T20:06:28.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b855d3000 20 client.5733 populate_metadata read hostname 'smithi088' 2023-10-16T20:06:28.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b855d3000 1 --2- 172.21.15.88:0/435465956 >> [v2:172.21.15.94:6833/2190281591,v1:172.21.15.94:6835/2190281591] conn(0x55a164ad9690 0x55a164ad9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:28.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b855d3000 4 client.5733 mds_command: new command op to 5535 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T20:06:28.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b855d3000 1 -- 172.21.15.88:0/435465956 --> [v2:172.21.15.94:6833/2190281591,v1:172.21.15.94:6835/2190281591] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a1648196f0 con 0x55a164ad9690 2023-10-16T20:06:28.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b7b7fe640 1 --2- 172.21.15.88:0/435465956 >> [v2:172.21.15.94:6833/2190281591,v1:172.21.15.94:6835/2190281591] conn(0x55a164ad9690 0x55a164ad9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:28.044 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b7b7fe640 1 --2- 172.21.15.88:0/435465956 >> [v2:172.21.15.94:6833/2190281591,v1:172.21.15.94:6835/2190281591] conn(0x55a164ad9690 0x55a164ad9a70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95d9fbffdcf93d41 server_cookie=db93695667872839 in_seq=0 out_seq=0 2023-10-16T20:06:28.044 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b5a7fc640 10 client.5733 ms_handle_connect on v2:172.21.15.94:6833/2190281591 2023-10-16T20:06:28.044 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b5a7fc640 1 -- 172.21.15.88:0/435465956 <== mds.0 v2:172.21.15.94:6833/2190281591 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+336 (crc 0 0 0) 0x55a1648196f0 con 0x55a164ad9690 2023-10-16T20:06:28.044 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b5a7fc640 10 client.5733 handle_command_reply: tid=0 2023-10-16T20:06:28.044 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b855d3000 1 client.5733 shutdown 2023-10-16T20:06:28.044 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b855d3000 1 -- 172.21.15.88:0/435465956 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a164a7b1c0 msgr2=0x55a164a96630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:28.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b855d3000 1 --2- 172.21.15.88:0/435465956 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a164a7b1c0 0x55a164a96630 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3b7402b040 tx=0x7f3b740014e0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b855d3000 1 -- 172.21.15.88:0/435465956 shutdown_connections 2023-10-16T20:06:28.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b855d3000 1 --2- 172.21.15.88:0/435465956 >> [v2:172.21.15.94:6833/2190281591,v1:172.21.15.94:6835/2190281591] conn(0x55a164ad9690 0x55a164ad9a70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b855d3000 1 --2- 172.21.15.88:0/435465956 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a164a7b1c0 0x55a164a96630 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b5a7fc640 0 client.5733 ms_handle_reset on v2:172.21.15.94:6833/2190281591 2023-10-16T20:06:28.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b855d3000 1 --2- 172.21.15.88:0/435465956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a164a7a8a0 0x55a164a960f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b855d3000 1 --2- 172.21.15.88:0/435465956 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55a164a30c50 0x55a164a67050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b855d3000 1 -- 172.21.15.88:0/435465956 >> 172.21.15.88:0/435465956 conn(0x55a164a27fc0 msgr2=0x55a164a8cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:28.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b855d3000 1 -- 172.21.15.88:0/435465956 shutdown_connections 2023-10-16T20:06:28.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b855d3000 1 -- 172.21.15.88:0/435465956 wait complete. 2023-10-16T20:06:28.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b855d3000 20 client.5733 trim_cache size 0 max 16384 2023-10-16T20:06:28.047 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2023-10-16T20:06:28.047 INFO:teuthology.orchestra.run.smithi088.stdout: "cluster_fsid": "d11187c0-3dd8-4c92-8d13-6ec536230748", 2023-10-16T20:06:28.048 INFO:teuthology.orchestra.run.smithi088.stdout: "whoami": 0, 2023-10-16T20:06:28.048 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 5535, 2023-10-16T20:06:28.048 INFO:teuthology.orchestra.run.smithi088.stdout: "want_state": "up:active", 2023-10-16T20:06:28.048 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "up:active", 2023-10-16T20:06:28.048 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "testcephfs1", 2023-10-16T20:06:28.049 INFO:teuthology.orchestra.run.smithi088.stdout: "rank_uptime": 10.310091010000001, 2023-10-16T20:06:28.049 INFO:teuthology.orchestra.run.smithi088.stdout: "mdsmap_epoch": 18, 2023-10-16T20:06:28.049 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch": 41, 2023-10-16T20:06:28.049 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch_barrier": 41, 2023-10-16T20:06:28.049 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 18.179305629000002 2023-10-16T20:06:28.050 INFO:teuthology.orchestra.run.smithi088.stdout:} 2023-10-16T20:06:28.051 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b82246640 1 -- 172.21.15.88:0/952474478 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3b5c0664a0 msgr2=0x7f3b5c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:28.051 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b82246640 1 --2- 172.21.15.88:0/952474478 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3b5c0664a0 0x7f3b5c068960 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c0ad000 tx=0x7f3b6c0a6000 comp rx=0 tx=0).stop 2023-10-16T20:06:28.051 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b82246640 1 -- 172.21.15.88:0/952474478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b7c0adf80 msgr2=0x7f3b7c070670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:28.051 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b82246640 1 --2- 172.21.15.88:0/952474478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b7c0adf80 0x7f3b7c070670 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400be50 tx=0x7f3b64013490 comp rx=0 tx=0).stop 2023-10-16T20:06:28.052 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b82246640 1 -- 172.21.15.88:0/952474478 shutdown_connections 2023-10-16T20:06:28.052 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b82246640 1 --2- 172.21.15.88:0/952474478 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3b5c0664a0 0x7f3b5c068960 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.052 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b82246640 1 --2- 172.21.15.88:0/952474478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b7c15d480 0x7f3b7c070bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.052 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b82246640 1 --2- 172.21.15.88:0/952474478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b7c0adf80 0x7f3b7c070670 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.052 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b82246640 1 --2- 172.21.15.88:0/952474478 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3b7c069e90 0x7f3b7c070130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.053 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b82246640 1 -- 172.21.15.88:0/952474478 >> 172.21.15.88:0/952474478 conn(0x7f3b7c0ce5e0 msgr2=0x7f3b7c0b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:28.053 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b82246640 1 -- 172.21.15.88:0/952474478 shutdown_connections 2023-10-16T20:06:28.053 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.027+0000 7f3b82246640 1 -- 172.21.15.88:0/952474478 wait complete. 2023-10-16T20:06:28.053 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T20:06:28.192 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.191+0000 7fcbc2f65640 1 Processor -- start 2023-10-16T20:06:28.193 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.191+0000 7fcbc2f65640 1 -- start start 2023-10-16T20:06:28.194 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.191+0000 7fcbc2f65640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fcbbc0d22e0 0x7fcbbc161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:28.194 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.191+0000 7fcbc2f65640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcbbc0d26c0 0x7fcbbc15d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:28.195 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.191+0000 7fcbc2f65640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fcbbc15d820 0x7fcbbc0cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:28.195 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.191+0000 7fcbc2f65640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbbc0b87e0 con 0x7fcbbc15d820 2023-10-16T20:06:28.195 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.191+0000 7fcbc2f65640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbbc15dc70 con 0x7fcbbc0d22e0 2023-10-16T20:06:28.195 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.191+0000 7fcbc2f65640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc15df10 con 0x7fcbbc0d26c0 2023-10-16T20:06:28.195 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.191+0000 7fcbc1502640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fcbbc15d820 0x7fcbbc0cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:28.196 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.191+0000 7fcbc1502640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fcbbc15d820 0x7fcbbc0cf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57686/0 (socket says 172.21.15.88:57686) 2023-10-16T20:06:28.196 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.191+0000 7fcbc1502640 1 -- 172.21.15.88:0/3210659848 learned_addr learned my addr 172.21.15.88:0/3210659848 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:28.196 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.191+0000 7fcbc0d01640 1 --2- 172.21.15.88:0/3210659848 >> v2:172.21.15.94:3300/0 conn(0x7fcbbc0d22e0 0x7fcbbc161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:28.196 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.191+0000 7fcbbbfff640 1 --2- 172.21.15.88:0/3210659848 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcbbc0d26c0 0x7fcbbc15d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:28.196 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.191+0000 7fcbc1502640 1 -- 172.21.15.88:0/3210659848 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcbbc0d26c0 msgr2=0x7fcbbc15d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:28.196 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.191+0000 7fcbc1502640 1 --2- 172.21.15.88:0/3210659848 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcbbc0d26c0 0x7fcbbc15d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.191+0000 7fcbc1502640 1 -- 172.21.15.88:0/3210659848 >> v2:172.21.15.94:3300/0 conn(0x7fcbbc0d22e0 msgr2=0x7fcbbc161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:28.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.191+0000 7fcbc1502640 1 --2- 172.21.15.88:0/3210659848 >> v2:172.21.15.94:3300/0 conn(0x7fcbbc0d22e0 0x7fcbbc161450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.191+0000 7fcbc1502640 1 -- 172.21.15.88:0/3210659848 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbbc15e1f0 con 0x7fcbbc15d820 2023-10-16T20:06:28.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.191+0000 7fcbc0d01640 1 --2- 172.21.15.88:0/3210659848 >> v2:172.21.15.94:3300/0 conn(0x7fcbbc0d22e0 0x7fcbbc161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:28.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.195+0000 7fcbc1502640 1 --2- 172.21.15.88:0/3210659848 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbbc15d820 0x7fcbbc0cf2e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc08bb40 tx=0x7fcbb401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d424a48412ead40 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:28.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.195+0000 7fcbbb7fe640 1 -- 172.21.15.88:0/3210659848 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb40a5790 con 0x7fcbbc15d820 2023-10-16T20:06:28.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.195+0000 7fcbbb7fe640 1 -- 172.21.15.88:0/3210659848 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb40a58f0 con 0x7fcbbc15d820 2023-10-16T20:06:28.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.195+0000 7fcbbb7fe640 1 -- 172.21.15.88:0/3210659848 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb40a5c80 con 0x7fcbbc15d820 2023-10-16T20:06:28.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.195+0000 7fcbc2f65640 1 -- 172.21.15.88:0/3210659848 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbbc15d820 msgr2=0x7fcbbc0cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:28.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.195+0000 7fcbc2f65640 1 --2- 172.21.15.88:0/3210659848 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbbc15d820 0x7fcbbc0cf2e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc08bb40 tx=0x7fcbb401c910 comp rx=0 tx=0).stop 2023-10-16T20:06:28.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.195+0000 7fcbc2f65640 1 -- 172.21.15.88:0/3210659848 shutdown_connections 2023-10-16T20:06:28.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.195+0000 7fcbc2f65640 1 --2- 172.21.15.88:0/3210659848 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcbbc0d26c0 0x7fcbbc15d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.195+0000 7fcbc2f65640 1 --2- 172.21.15.88:0/3210659848 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbbc15d820 0x7fcbbc0cf2e0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.195+0000 7fcbc2f65640 1 --2- 172.21.15.88:0/3210659848 >> v2:172.21.15.94:3300/0 conn(0x7fcbbc0d22e0 0x7fcbbc161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.195+0000 7fcbc2f65640 1 -- 172.21.15.88:0/3210659848 >> 172.21.15.88:0/3210659848 conn(0x7fcbbc0715c0 msgr2=0x7fcbbc0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:28.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.195+0000 7fcbc2f65640 1 -- 172.21.15.88:0/3210659848 shutdown_connections 2023-10-16T20:06:28.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.195+0000 7fcbc2f65640 1 -- 172.21.15.88:0/3210659848 wait complete. 2023-10-16T20:06:28.202 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.199+0000 7fcbc2f65640 1 Processor -- start 2023-10-16T20:06:28.202 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.199+0000 7fcbc2f65640 1 -- start start 2023-10-16T20:06:28.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.199+0000 7fcbc2f65640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcbbc0d22e0 0x7fcbbc154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:28.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.199+0000 7fcbc2f65640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbbc0d26c0 0x7fcbbc1567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:28.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.199+0000 7fcbc2f65640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcbbc15d820 0x7fcbbc14a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:28.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.199+0000 7fcbc2f65640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc156ce0 con 0x7fcbbc0d26c0 2023-10-16T20:06:28.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.199+0000 7fcbc2f65640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc14ae60 con 0x7fcbbc15d820 2023-10-16T20:06:28.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.199+0000 7fcbc2f65640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc155020 con 0x7fcbbc0d22e0 2023-10-16T20:06:28.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.199+0000 7fcbbbfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbbc0d26c0 0x7fcbbc1567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:28.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.199+0000 7fcbbbfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbbc0d26c0 0x7fcbbc1567a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57694/0 (socket says 172.21.15.88:57694) 2023-10-16T20:06:28.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.203+0000 7fcbbbfff640 1 -- 172.21.15.88:0/4046450015 learned_addr learned my addr 172.21.15.88:0/4046450015 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:28.206 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.203+0000 7fcbc0d01640 1 --2- 172.21.15.88:0/4046450015 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcbbc0d22e0 0x7fcbbc154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:28.206 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.203+0000 7fcbc1502640 1 --2- 172.21.15.88:0/4046450015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcbbc15d820 0x7fcbbc14a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:28.206 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.203+0000 7fcbbbfff640 1 -- 172.21.15.88:0/4046450015 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcbbc0d22e0 msgr2=0x7fcbbc154990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:28.206 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.203+0000 7fcbbbfff640 1 --2- 172.21.15.88:0/4046450015 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcbbc0d22e0 0x7fcbbc154990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.206 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.203+0000 7fcbbbfff640 1 -- 172.21.15.88:0/4046450015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcbbc15d820 msgr2=0x7fcbbc14a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:28.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.203+0000 7fcbbbfff640 1 --2- 172.21.15.88:0/4046450015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcbbc15d820 0x7fcbbc14a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.203+0000 7fcbbbfff640 1 -- 172.21.15.88:0/4046450015 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb4009b30 con 0x7fcbbc0d26c0 2023-10-16T20:06:28.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.203+0000 7fcbbbfff640 1 --2- 172.21.15.88:0/4046450015 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbbc0d26c0 0x7fcbbc1567a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fcba800d800 tx=0x7fcba800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:28.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.203+0000 7fcbb97fa640 1 -- 172.21.15.88:0/4046450015 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba800b6c0 con 0x7fcbbc0d26c0 2023-10-16T20:06:28.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.203+0000 7fcbb97fa640 1 -- 172.21.15.88:0/4046450015 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba800b820 con 0x7fcbbc0d26c0 2023-10-16T20:06:28.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.203+0000 7fcbc2f65640 1 -- 172.21.15.88:0/4046450015 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbbc155300 con 0x7fcbbc0d26c0 2023-10-16T20:06:28.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.203+0000 7fcbb97fa640 1 -- 172.21.15.88:0/4046450015 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba800bb90 con 0x7fcbbc0d26c0 2023-10-16T20:06:28.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.203+0000 7fcbc2f65640 1 -- 172.21.15.88:0/4046450015 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbbc1558c0 con 0x7fcbbc0d26c0 2023-10-16T20:06:28.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.203+0000 7fcbc2f65640 1 -- 172.21.15.88:0/4046450015 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbbc04e9c0 con 0x7fcbbc0d26c0 2023-10-16T20:06:28.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.203+0000 7fcbb97fa640 1 -- 172.21.15.88:0/4046450015 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcba800bcf0 con 0x7fcbbc0d26c0 2023-10-16T20:06:28.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.207+0000 7fcbb97fa640 1 --2- 172.21.15.88:0/4046450015 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcba00664d0 0x7fcba0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:28.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.207+0000 7fcbb97fa640 1 -- 172.21.15.88:0/4046450015 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fcba80844e0 con 0x7fcbbc0d26c0 2023-10-16T20:06:28.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.207+0000 7fcbc0d01640 1 --2- 172.21.15.88:0/4046450015 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcba00664d0 0x7fcba0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:28.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.207+0000 7fcbc0d01640 1 --2- 172.21.15.88:0/4046450015 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcba00664d0 0x7fcba0068990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcbb00098d0 tx=0x7fcbb0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:28.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.211+0000 7fcbb97fa640 1 -- 172.21.15.88:0/4046450015 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcba8051780 con 0x7fcbbc0d26c0 2023-10-16T20:06:28.396 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.395+0000 7fcbc2f65640 1 -- 172.21.15.88:0/4046450015 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcbbc0cb330 con 0x7fcbbc0d26c0 2023-10-16T20:06:28.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.395+0000 7fcbb97fa640 1 -- 172.21.15.88:0/4046450015 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+88 (secure 0 0 0) 0x7fcba8054da0 con 0x7fcbbc0d26c0 2023-10-16T20:06:28.397 INFO:teuthology.orchestra.run.smithi088.stdout:name: testcephfs1, metadata pool: testcephfs1_metadata, data pools: [testcephfs1_data ] 2023-10-16T20:06:28.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.395+0000 7fcbc2f65640 1 -- 172.21.15.88:0/4046450015 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcba00664d0 msgr2=0x7fcba0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:28.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.395+0000 7fcbc2f65640 1 --2- 172.21.15.88:0/4046450015 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcba00664d0 0x7fcba0068990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcbb00098d0 tx=0x7fcbb0008040 comp rx=0 tx=0).stop 2023-10-16T20:06:28.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.395+0000 7fcbc2f65640 1 -- 172.21.15.88:0/4046450015 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbbc0d26c0 msgr2=0x7fcbbc1567a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:28.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.395+0000 7fcbc2f65640 1 --2- 172.21.15.88:0/4046450015 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbbc0d26c0 0x7fcbbc1567a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fcba800d800 tx=0x7fcba800dcc0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.395+0000 7fcbc2f65640 1 -- 172.21.15.88:0/4046450015 shutdown_connections 2023-10-16T20:06:28.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.395+0000 7fcbc2f65640 1 --2- 172.21.15.88:0/4046450015 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcba00664d0 0x7fcba0068990 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.395+0000 7fcbc2f65640 1 --2- 172.21.15.88:0/4046450015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcbbc15d820 0x7fcbbc14a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.395+0000 7fcbc2f65640 1 --2- 172.21.15.88:0/4046450015 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbbc0d26c0 0x7fcbbc1567a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.395+0000 7fcbc2f65640 1 --2- 172.21.15.88:0/4046450015 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcbbc0d22e0 0x7fcbbc154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:28.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.395+0000 7fcbc2f65640 1 -- 172.21.15.88:0/4046450015 >> 172.21.15.88:0/4046450015 conn(0x7fcbbc0715c0 msgr2=0x7fcbbc0cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:28.401 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.395+0000 7fcbc2f65640 1 -- 172.21.15.88:0/4046450015 shutdown_connections 2023-10-16T20:06:28.401 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:28.395+0000 7fcbc2f65640 1 -- 172.21.15.88:0/4046450015 wait complete. 2023-10-16T20:06:28.418 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T20:06:28.418 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T20:06:28.418 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi088.front.sepia.ceph.com 2023-10-16T20:06:28.419 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T20:06:28.419 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T20:06:28.419 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T20:06:28.420 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T20:06:28.422 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T20:06:28.422 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T20:06:28.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:28.423 DEBUG:teuthology.orchestra.run.smithi088:> ip addr 2023-10-16T20:06:28.433 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T20:06:28.433 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T20:06:28.433 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T20:06:28.434 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:06:28.434 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2023-10-16T20:06:28.434 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:06:28.434 INFO:teuthology.orchestra.run.smithi088.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:06:28.434 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f0 brd ff:ff:ff:ff:ff:ff 2023-10-16T20:06:28.435 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f0 2023-10-16T20:06:28.435 INFO:teuthology.orchestra.run.smithi088.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:06:28.435 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f1 brd ff:ff:ff:ff:ff:ff 2023-10-16T20:06:28.435 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f1 2023-10-16T20:06:28.435 INFO:teuthology.orchestra.run.smithi088.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:06:28.436 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2c brd ff:ff:ff:ff:ff:ff 2023-10-16T20:06:28.436 INFO:teuthology.orchestra.run.smithi088.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T20:06:28.436 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2d brd ff:ff:ff:ff:ff:ff 2023-10-16T20:06:28.436 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.21.15.88/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-16T20:06:28.436 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft 42343sec preferred_lft 42343sec 2023-10-16T20:06:28.437 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link 2023-10-16T20:06:28.437 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:06:28.437 INFO:teuthology.orchestra.run.smithi088.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-16T20:06:28.437 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 02:42:d9:3b:fb:39 brd ff:ff:ff:ff:ff:ff 2023-10-16T20:06:28.437 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-16T20:06:28.438 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:06:28.438 INFO:teuthology.orchestra.run.smithi088.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T20:06:28.438 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-16T20:06:28.438 INFO:teuthology.orchestra.run.smithi088.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T20:06:28.438 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:06:28.439 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-16T20:06:28.439 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:06:28.439 INFO:teuthology.orchestra.run.smithi088.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T20:06:28.439 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:06:28.439 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-16T20:06:28.440 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:06:28.440 INFO:teuthology.orchestra.run.smithi088.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T20:06:28.440 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:06:28.440 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-16T20:06:28.440 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:06:28.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:28.441 DEBUG:teuthology.orchestra.run.smithi088:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T20:06:28.497 INFO:teuthology.orchestra.run.smithi088.stdout:1 2023-10-16T20:06:28.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:28.500 DEBUG:teuthology.orchestra.run.smithi088:> route 2023-10-16T20:06:28.553 INFO:teuthology.orchestra.run.smithi088.stdout:Kernel IP routing table 2023-10-16T20:06:28.554 INFO:teuthology.orchestra.run.smithi088.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T20:06:28.554 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-16T20:06:28.554 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T20:06:28.554 INFO:teuthology.orchestra.run.smithi088.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-16T20:06:28.554 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T20:06:28.555 INFO:teuthology.orchestra.run.smithi088.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:06:28.555 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:06:28.555 INFO:teuthology.orchestra.run.smithi088.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:06:28.555 INFO:teuthology.orchestra.run.smithi088.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T20:06:28.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:28.556 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec bash -c ' 2023-10-16T20:06:28.556 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2023-10-16T20:06:28.557 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T20:06:28.557 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T20:06:28.557 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T20:06:28.558 DEBUG:teuthology.orchestra.run.smithi088:> ') 2023-10-16T20:06:28.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:28.646 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2023-10-16T20:06:28.655 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T20:06:28.655 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T20:06:28.656 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T20:06:28.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:28.657 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set brx.0 up 2023-10-16T20:06:28.719 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-16T20:06:28.720 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T20:06:28.720 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T20:06:28.771 INFO:teuthology.orchestra.run.smithi088.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T20:06:28.771 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T20:06:28.772 DEBUG:teuthology.orchestra.run.smithi088:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T20:06:28.822 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2023-10-16T20:06:28.883 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T20:06:28.936 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2023-10-16T20:06:28.937 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2023-10-16T20:06:28.937 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2023-10-16T20:06:28.937 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2023-10-16T20:06:28.937 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2023-10-16T20:06:28.938 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2023-10-16T20:06:28.938 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2023-10-16T20:06:28.938 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2023-10-16T20:06:28.938 INFO:teuthology.orchestra.run.smithi088.stdout:/run/lock 2023-10-16T20:06:28.938 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2023-10-16T20:06:28.939 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2023-10-16T20:06:28.939 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2023-10-16T20:06:28.939 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T20:06:28.939 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2023-10-16T20:06:28.940 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2023-10-16T20:06:28.940 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2023-10-16T20:06:28.940 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2023-10-16T20:06:28.940 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/fs/nfsd 2023-10-16T20:06:28.940 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2023-10-16T20:06:28.941 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2023-10-16T20:06:28.941 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T20:06:28.941 INFO:teuthology.orchestra.run.smithi088.stdout:/run/rpc_pipefs 2023-10-16T20:06:28.941 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2023-10-16T20:06:28.941 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2023-10-16T20:06:28.942 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T20:06:28.942 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T20:06:28.942 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T20:06:28.942 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T20:06:28.942 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2023-10-16T20:06:28.943 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:06:28.943 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:06:28.943 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:06:28.943 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:06:28.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:28.945 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2023-10-16T20:06:28.995 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-16T20:06:28.995 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-16T20:06:29.042 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2023-10-16T20:06:29.064 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T20:06:29.117 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2023-10-16T20:06:29.117 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2023-10-16T20:06:29.117 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2023-10-16T20:06:29.117 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2023-10-16T20:06:29.118 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2023-10-16T20:06:29.118 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2023-10-16T20:06:29.118 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2023-10-16T20:06:29.118 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2023-10-16T20:06:29.118 INFO:teuthology.orchestra.run.smithi088.stdout:/run/lock 2023-10-16T20:06:29.119 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2023-10-16T20:06:29.119 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2023-10-16T20:06:29.119 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2023-10-16T20:06:29.119 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T20:06:29.119 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2023-10-16T20:06:29.120 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2023-10-16T20:06:29.120 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2023-10-16T20:06:29.120 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2023-10-16T20:06:29.120 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/fs/nfsd 2023-10-16T20:06:29.120 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2023-10-16T20:06:29.120 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2023-10-16T20:06:29.121 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T20:06:29.121 INFO:teuthology.orchestra.run.smithi088.stdout:/run/rpc_pipefs 2023-10-16T20:06:29.121 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2023-10-16T20:06:29.121 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2023-10-16T20:06:29.121 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T20:06:29.122 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T20:06:29.122 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T20:06:29.122 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T20:06:29.122 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2023-10-16T20:06:29.122 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:06:29.123 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:06:29.123 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:06:29.123 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:06:29.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:29.124 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2023-10-16T20:06:29.137 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:2023-10-16T20:06:29.135+0000 7fd982c45580 -1 init, newargv = 0x55e272b1be60 newargc=15 2023-10-16T20:06:29.137 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[49414]: starting ceph client 2023-10-16T20:06:29.145 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[49414]: starting fuse 2023-10-16T20:06:30.248 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2023-10-16T20:06:30.273 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T20:06:30.325 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2023-10-16T20:06:30.325 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2023-10-16T20:06:30.326 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2023-10-16T20:06:30.326 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2023-10-16T20:06:30.326 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2023-10-16T20:06:30.326 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2023-10-16T20:06:30.326 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2023-10-16T20:06:30.327 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2023-10-16T20:06:30.327 INFO:teuthology.orchestra.run.smithi088.stdout:/run/lock 2023-10-16T20:06:30.327 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2023-10-16T20:06:30.327 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2023-10-16T20:06:30.327 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2023-10-16T20:06:30.328 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T20:06:30.328 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2023-10-16T20:06:30.328 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2023-10-16T20:06:30.328 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2023-10-16T20:06:30.328 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2023-10-16T20:06:30.329 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/fs/nfsd 2023-10-16T20:06:30.329 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2023-10-16T20:06:30.329 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2023-10-16T20:06:30.329 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T20:06:30.329 INFO:teuthology.orchestra.run.smithi088.stdout:/run/rpc_pipefs 2023-10-16T20:06:30.329 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2023-10-16T20:06:30.330 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2023-10-16T20:06:30.330 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T20:06:30.330 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T20:06:30.330 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T20:06:30.330 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T20:06:30.331 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2023-10-16T20:06:30.331 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:06:30.331 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:06:30.331 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:06:30.331 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:06:30.331 INFO:teuthology.orchestra.run.smithi088.stdout:/home/ubuntu/cephtest/mnt.0 2023-10-16T20:06:30.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:30.333 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2023-10-16T20:06:30.383 INFO:teuthology.orchestra.run.smithi088.stdout:43 2023-10-16T20:06:30.384 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-10-16T20:06:30.384 DEBUG:teuthology.orchestra.run.smithi088:> sudo stdin-killer -- python3 -c ' 2023-10-16T20:06:30.385 DEBUG:teuthology.orchestra.run.smithi088:> import glob 2023-10-16T20:06:30.385 DEBUG:teuthology.orchestra.run.smithi088:> import re 2023-10-16T20:06:30.385 DEBUG:teuthology.orchestra.run.smithi088:> import os 2023-10-16T20:06:30.386 DEBUG:teuthology.orchestra.run.smithi088:> import subprocess 2023-10-16T20:06:30.386 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:06:30.386 DEBUG:teuthology.orchestra.run.smithi088:> def _find_admin_socket(client_name): 2023-10-16T20:06:30.387 DEBUG:teuthology.orchestra.run.smithi088:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-16T20:06:30.387 DEBUG:teuthology.orchestra.run.smithi088:> files = glob.glob(asok_path) 2023-10-16T20:06:30.387 DEBUG:teuthology.orchestra.run.smithi088:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-16T20:06:30.388 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:06:30.388 DEBUG:teuthology.orchestra.run.smithi088:> # Given a non-glob path, it better be there 2023-10-16T20:06:30.388 DEBUG:teuthology.orchestra.run.smithi088:> if "*" not in asok_path: 2023-10-16T20:06:30.389 DEBUG:teuthology.orchestra.run.smithi088:> assert(len(files) == 1) 2023-10-16T20:06:30.389 DEBUG:teuthology.orchestra.run.smithi088:> return files[0] 2023-10-16T20:06:30.390 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:06:30.390 DEBUG:teuthology.orchestra.run.smithi088:> for f in files: 2023-10-16T20:06:30.390 DEBUG:teuthology.orchestra.run.smithi088:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-16T20:06:30.391 DEBUG:teuthology.orchestra.run.smithi088:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-16T20:06:30.391 DEBUG:teuthology.orchestra.run.smithi088:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-16T20:06:30.392 DEBUG:teuthology.orchestra.run.smithi088:> contents = proc_f.read() 2023-10-16T20:06:30.392 DEBUG:teuthology.orchestra.run.smithi088:> if mountpoint in contents: 2023-10-16T20:06:30.393 DEBUG:teuthology.orchestra.run.smithi088:> return f 2023-10-16T20:06:30.393 DEBUG:teuthology.orchestra.run.smithi088:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-16T20:06:30.393 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:06:30.394 DEBUG:teuthology.orchestra.run.smithi088:> print(_find_admin_socket("client.0")) 2023-10-16T20:06:30.394 DEBUG:teuthology.orchestra.run.smithi088:> ' 2023-10-16T20:06:30.514 INFO:teuthology.orchestra.run.smithi088.stdout:/var/run/ceph/ceph-client.0.49414.asok 2023-10-16T20:06:30.518 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-16T20:06:30.526 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.49414.asok 2023-10-16T20:06:30.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:30.527 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.49414.asok status 2023-10-16T20:06:30.696 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2023-10-16T20:06:30.696 INFO:teuthology.orchestra.run.smithi088.stdout: "metadata": { 2023-10-16T20:06:30.696 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_sha1": "4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0", 2023-10-16T20:06:30.697 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_version": "ceph version 18.0.0-6730-g4ba8a190 (4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0) reef (dev)", 2023-10-16T20:06:30.697 INFO:teuthology.orchestra.run.smithi088.stdout: "entity_id": "0", 2023-10-16T20:06:30.697 INFO:teuthology.orchestra.run.smithi088.stdout: "hostname": "smithi088", 2023-10-16T20:06:30.697 INFO:teuthology.orchestra.run.smithi088.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-16T20:06:30.697 INFO:teuthology.orchestra.run.smithi088.stdout: "pid": "49414", 2023-10-16T20:06:30.697 INFO:teuthology.orchestra.run.smithi088.stdout: "root": "/" 2023-10-16T20:06:30.698 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:30.698 INFO:teuthology.orchestra.run.smithi088.stdout: "dentry_count": 0, 2023-10-16T20:06:30.698 INFO:teuthology.orchestra.run.smithi088.stdout: "dentry_pinned_count": 0, 2023-10-16T20:06:30.698 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 5198, 2023-10-16T20:06:30.698 INFO:teuthology.orchestra.run.smithi088.stdout: "inst": { 2023-10-16T20:06:30.699 INFO:teuthology.orchestra.run.smithi088.stdout: "name": { 2023-10-16T20:06:30.699 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "client", 2023-10-16T20:06:30.699 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 5198 2023-10-16T20:06:30.699 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:30.699 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": { 2023-10-16T20:06:30.699 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2023-10-16T20:06:30.700 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:0", 2023-10-16T20:06:30.700 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 2892711171 2023-10-16T20:06:30.700 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:06:30.700 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:30.700 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": { 2023-10-16T20:06:30.701 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2023-10-16T20:06:30.701 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:0", 2023-10-16T20:06:30.701 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 2892711171 2023-10-16T20:06:30.701 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:30.701 INFO:teuthology.orchestra.run.smithi088.stdout: "inst_str": "client.5198 172.21.15.88:0/2892711171", 2023-10-16T20:06:30.702 INFO:teuthology.orchestra.run.smithi088.stdout: "addr_str": "172.21.15.88:0/2892711171", 2023-10-16T20:06:30.702 INFO:teuthology.orchestra.run.smithi088.stdout: "inode_count": 1, 2023-10-16T20:06:30.702 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_epoch": 18, 2023-10-16T20:06:30.702 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_epoch": 41, 2023-10-16T20:06:30.702 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_epoch_barrier": 0, 2023-10-16T20:06:30.702 INFO:teuthology.orchestra.run.smithi088.stdout: "blocklisted": false, 2023-10-16T20:06:30.703 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "testcephfs1" 2023-10-16T20:06:30.703 INFO:teuthology.orchestra.run.smithi088.stdout:} 2023-10-16T20:06:30.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:30.712 DEBUG:teuthology.orchestra.run.smithi088:> stat --file-system '--printf=%T 2023-10-16T20:06:30.712 DEBUG:teuthology.orchestra.run.smithi088:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-16T20:06:30.723 INFO:teuthology.orchestra.run.smithi088.stdout:fuseblk 2023-10-16T20:06:30.723 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-16T20:06:30.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:30.724 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T20:06:30.792 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create data_new --pg_num_min 64 2023-10-16T20:06:30.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.971+0000 7f584ecd6640 1 Processor -- start 2023-10-16T20:06:30.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.971+0000 7f584ecd6640 1 -- start start 2023-10-16T20:06:30.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.975+0000 7f584ecd6640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f58480c8cb0 0x7f5848069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:30.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.975+0000 7f584ecd6640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5848159b10 0x7f584806a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:30.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.975+0000 7f584ecd6640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f584815d660 con 0x7f5848159730 2023-10-16T20:06:30.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.975+0000 7f584ecd6640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f584815e2e0 con 0x7f58480c8cb0 2023-10-16T20:06:30.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.975+0000 7f584ecd6640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f584815d940 con 0x7f5848159b10 2023-10-16T20:06:30.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.975+0000 7f584ca72640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f5848159730 0x7f58480ad700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:51176/0 (socket says 172.21.15.88:51176) 2023-10-16T20:06:30.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.975+0000 7f584ca72640 1 -- 172.21.15.88:0/2808199682 learned_addr learned my addr 172.21.15.88:0/2808199682 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:30.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.975+0000 7f583ffff640 1 --2- 172.21.15.88:0/2808199682 >> v2:172.21.15.88:3300/0 conn(0x7f58480c8cb0 0x7f5848069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:30.978 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.975+0000 7f584d273640 1 --2- 172.21.15.88:0/2808199682 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5848159b10 0x7f584806a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:30.978 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.975+0000 7f583ffff640 1 -- 172.21.15.88:0/2808199682 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5848159b10 msgr2=0x7f584806a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:30.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.975+0000 7f583f7fe640 1 -- 172.21.15.88:0/2808199682 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1871083551 0 0) 0x7f584815d660 con 0x7f5848159730 2023-10-16T20:06:30.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.975+0000 7f583ffff640 1 --2- 172.21.15.88:0/2808199682 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5848159b10 0x7f584806a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:30.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.975+0000 7f583ffff640 1 -- 172.21.15.88:0/2808199682 >> v1:172.21.15.88:6789/0 conn(0x7f5848159730 legacy=0x7f58480ad700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:30.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.975+0000 7f583ffff640 1 -- 172.21.15.88:0/2808199682 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f584815dc20 con 0x7f58480c8cb0 2023-10-16T20:06:30.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.975+0000 7f584d273640 1 --2- 172.21.15.88:0/2808199682 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5848159b10 0x7f584806a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:06:30.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.979+0000 7f583ffff640 1 --2- 172.21.15.88:0/2808199682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58480c8cb0 0x7f5848069e90 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f58380097c0 tx=0x7f583801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e677211814629779 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:30.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.979+0000 7f583f7fe640 1 -- 172.21.15.88:0/2808199682 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58380ab600 con 0x7f58480c8cb0 2023-10-16T20:06:30.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.979+0000 7f583f7fe640 1 -- 172.21.15.88:0/2808199682 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58380a7050 con 0x7f58480c8cb0 2023-10-16T20:06:30.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.979+0000 7f583f7fe640 1 -- 172.21.15.88:0/2808199682 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58380ab980 con 0x7f58480c8cb0 2023-10-16T20:06:30.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.979+0000 7f584ecd6640 1 -- 172.21.15.88:0/2808199682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58480c8cb0 msgr2=0x7f5848069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:30.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.979+0000 7f584ecd6640 1 --2- 172.21.15.88:0/2808199682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58480c8cb0 0x7f5848069e90 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f58380097c0 tx=0x7f583801c450 comp rx=0 tx=0).stop 2023-10-16T20:06:30.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.979+0000 7f584ecd6640 1 -- 172.21.15.88:0/2808199682 shutdown_connections 2023-10-16T20:06:30.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.979+0000 7f584ecd6640 1 --2- 172.21.15.88:0/2808199682 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5848159b10 0x7f584806a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:30.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.979+0000 7f584ecd6640 1 --2- 172.21.15.88:0/2808199682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58480c8cb0 0x7f5848069e90 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:30.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.979+0000 7f584ecd6640 1 -- 172.21.15.88:0/2808199682 >> 172.21.15.88:0/2808199682 conn(0x7f58480ce660 msgr2=0x7f58480cea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:30.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.979+0000 7f584ecd6640 1 -- 172.21.15.88:0/2808199682 shutdown_connections 2023-10-16T20:06:30.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.979+0000 7f584ecd6640 1 -- 172.21.15.88:0/2808199682 wait complete. 2023-10-16T20:06:30.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.979+0000 7f584ecd6640 1 Processor -- start 2023-10-16T20:06:30.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.979+0000 7f584ecd6640 1 -- start start 2023-10-16T20:06:30.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.983+0000 7f584ecd6640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f58480c8cb0 0x7f58481453e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:30.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.983+0000 7f584ecd6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5848159730 0x7f5848158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:30.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.983+0000 7f584ecd6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5848159b10 0x7f5848158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:30.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.983+0000 7f584ecd6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5848145920 con 0x7f5848159730 2023-10-16T20:06:30.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.983+0000 7f584ecd6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584815ce60 con 0x7f5848159b10 2023-10-16T20:06:30.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.983+0000 7f584ecd6640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5848159fe0 con 0x7f58480c8cb0 2023-10-16T20:06:30.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.983+0000 7f583ffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5848159730 0x7f5848158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:30.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.983+0000 7f583ffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5848159730 0x7f5848158650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57714/0 (socket says 172.21.15.88:57714) 2023-10-16T20:06:30.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.983+0000 7f583ffff640 1 -- 172.21.15.88:0/2660268136 learned_addr learned my addr 172.21.15.88:0/2660268136 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:30.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.983+0000 7f584ca72640 1 --2- 172.21.15.88:0/2660268136 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f58480c8cb0 0x7f58481453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:30.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.983+0000 7f584d273640 1 --2- 172.21.15.88:0/2660268136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5848159b10 0x7f5848158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:30.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.983+0000 7f583ffff640 1 -- 172.21.15.88:0/2660268136 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f58480c8cb0 msgr2=0x7f58481453e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:30.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.983+0000 7f583ffff640 1 --2- 172.21.15.88:0/2660268136 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f58480c8cb0 0x7f58481453e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:30.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.983+0000 7f583ffff640 1 -- 172.21.15.88:0/2660268136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5848159b10 msgr2=0x7f5848158b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:30.988 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.983+0000 7f583ffff640 1 --2- 172.21.15.88:0/2660268136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5848159b10 0x7f5848158b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:30.988 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.983+0000 7f583ffff640 1 -- 172.21.15.88:0/2660268136 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5838009420 con 0x7f5848159730 2023-10-16T20:06:30.988 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.983+0000 7f583ffff640 1 --2- 172.21.15.88:0/2660268136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5848159730 0x7f5848158650 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f58380ae040 tx=0x7f583801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:30.988 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.983+0000 7f583d7fa640 1 -- 172.21.15.88:0/2660268136 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58380b7650 con 0x7f5848159730 2023-10-16T20:06:30.988 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.983+0000 7f583d7fa640 1 -- 172.21.15.88:0/2660268136 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58380abe90 con 0x7f5848159730 2023-10-16T20:06:30.989 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.983+0000 7f584ecd6640 1 -- 172.21.15.88:0/2660268136 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5848156a70 con 0x7f5848159730 2023-10-16T20:06:30.989 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.983+0000 7f583d7fa640 1 -- 172.21.15.88:0/2660268136 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58380b7a10 con 0x7f5848159730 2023-10-16T20:06:30.989 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.983+0000 7f584ecd6640 1 -- 172.21.15.88:0/2660268136 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f584815a510 con 0x7f5848159730 2023-10-16T20:06:30.990 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.983+0000 7f584ecd6640 1 -- 172.21.15.88:0/2660268136 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5848156c00 con 0x7f5848159730 2023-10-16T20:06:30.990 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.983+0000 7f583d7fa640 1 -- 172.21.15.88:0/2660268136 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f58380b7b70 con 0x7f5848159730 2023-10-16T20:06:30.990 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.987+0000 7f583d7fa640 1 --2- 172.21.15.88:0/2660268136 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f58340664d0 0x7f5834068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:30.990 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.987+0000 7f583d7fa640 1 -- 172.21.15.88:0/2660268136 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f583811e990 con 0x7f5848159730 2023-10-16T20:06:30.991 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.987+0000 7f584ca72640 1 --2- 172.21.15.88:0/2660268136 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f58340664d0 0x7f5834068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:30.991 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.987+0000 7f584ca72640 1 --2- 172.21.15.88:0/2660268136 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f58340664d0 0x7f5834068990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5844000f70 tx=0x7f5844009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:30.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:30.991+0000 7f583d7fa640 1 -- 172.21.15.88:0/2660268136 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f58380ebcc0 con 0x7f5848159730 2023-10-16T20:06:31.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.139+0000 7f584ecd6640 1 -- 172.21.15.88:0/2660268136 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64} v 0) v1 -- 0x7f584804e9c0 con 0x7f5848159730 2023-10-16T20:06:31.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.211+0000 7f583d7fa640 1 -- 172.21.15.88:0/2660268136 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64}]=0 pool 'data_new' created v42) v1 ==== 124+0+0 (secure 0 0 0) 0x7f58380ef2e0 con 0x7f5848159730 2023-10-16T20:06:31.212 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'data_new' created 2023-10-16T20:06:31.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.211+0000 7f584ecd6640 1 -- 172.21.15.88:0/2660268136 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f58340664d0 msgr2=0x7f5834068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:31.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.211+0000 7f584ecd6640 1 --2- 172.21.15.88:0/2660268136 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f58340664d0 0x7f5834068990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5844000f70 tx=0x7f5844009040 comp rx=0 tx=0).stop 2023-10-16T20:06:31.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.211+0000 7f584ecd6640 1 -- 172.21.15.88:0/2660268136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5848159730 msgr2=0x7f5848158650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:31.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.211+0000 7f584ecd6640 1 --2- 172.21.15.88:0/2660268136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5848159730 0x7f5848158650 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f58380ae040 tx=0x7f583801ac90 comp rx=0 tx=0).stop 2023-10-16T20:06:31.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.211+0000 7f584ecd6640 1 -- 172.21.15.88:0/2660268136 shutdown_connections 2023-10-16T20:06:31.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.211+0000 7f584ecd6640 1 --2- 172.21.15.88:0/2660268136 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f58340664d0 0x7f5834068990 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:31.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.211+0000 7f584ecd6640 1 --2- 172.21.15.88:0/2660268136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5848159b10 0x7f5848158b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:31.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.211+0000 7f584ecd6640 1 --2- 172.21.15.88:0/2660268136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5848159730 0x7f5848158650 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:31.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.211+0000 7f584ecd6640 1 --2- 172.21.15.88:0/2660268136 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f58480c8cb0 0x7f58481453e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:31.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.211+0000 7f584ecd6640 1 -- 172.21.15.88:0/2660268136 >> 172.21.15.88:0/2660268136 conn(0x7f58480ce660 msgr2=0x7f58480cea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:31.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.211+0000 7f584ecd6640 1 -- 172.21.15.88:0/2660268136 shutdown_connections 2023-10-16T20:06:31.217 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.211+0000 7f584ecd6640 1 -- 172.21.15.88:0/2660268136 wait complete. 2023-10-16T20:06:31.240 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool testcephfs1 data_new 2023-10-16T20:06:31.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.375+0000 7f953bc79640 1 Processor -- start 2023-10-16T20:06:31.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.375+0000 7f953bc79640 1 -- start start 2023-10-16T20:06:31.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.379+0000 7f953bc79640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f95340b4260 0x7f953415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:31.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.379+0000 7f953bc79640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f953406d3f0 0x7f953406d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:31.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.379+0000 7f953bc79640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f953415ad30 con 0x7f95340b0da0 2023-10-16T20:06:31.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.379+0000 7f953bc79640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f953406ddc0 con 0x7f953406d3f0 2023-10-16T20:06:31.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.379+0000 7f953bc79640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95340b4640 con 0x7f95340b4260 2023-10-16T20:06:31.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.379+0000 7f953a216640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f953406d3f0 0x7f953406d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:31.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.379+0000 7f953a216640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f953406d3f0 0x7f953406d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57718/0 (socket says 172.21.15.88:57718) 2023-10-16T20:06:31.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.379+0000 7f953a216640 1 -- 172.21.15.88:0/1682111418 learned_addr learned my addr 172.21.15.88:0/1682111418 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:31.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.379+0000 7f9539a15640 1 --2- 172.21.15.88:0/1682111418 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f95340b4260 0x7f953415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:31.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.379+0000 7f953a216640 1 -- 172.21.15.88:0/1682111418 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f95340b4260 msgr2=0x7f953415dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:31.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.379+0000 7f953a216640 1 --2- 172.21.15.88:0/1682111418 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f95340b4260 0x7f953415dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:31.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.379+0000 7f953a216640 1 -- 172.21.15.88:0/1682111418 >> v1:172.21.15.88:6789/0 conn(0x7f95340b0da0 legacy=0x7f9534159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:31.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.379+0000 7f953a216640 1 -- 172.21.15.88:0/1682111418 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953415a770 con 0x7f953406d3f0 2023-10-16T20:06:31.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.379+0000 7f9539a15640 1 --2- 172.21.15.88:0/1682111418 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f95340b4260 0x7f953415dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:06:31.384 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.383+0000 7f953a216640 1 --2- 172.21.15.88:0/1682111418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f953406d3f0 0x7f953406d880 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f952c009e50 tx=0x7f952c01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05c99715d4c3167 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:31.384 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.383+0000 7f9538a13640 1 -- 172.21.15.88:0/1682111418 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952c0a5810 con 0x7f953406d3f0 2023-10-16T20:06:31.384 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.383+0000 7f9538a13640 1 -- 172.21.15.88:0/1682111418 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f952c0a5970 con 0x7f953406d3f0 2023-10-16T20:06:31.384 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.383+0000 7f9538a13640 1 -- 172.21.15.88:0/1682111418 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952c0a5d00 con 0x7f953406d3f0 2023-10-16T20:06:31.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.383+0000 7f953bc79640 1 -- 172.21.15.88:0/1682111418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f953406d3f0 msgr2=0x7f953406d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:31.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.383+0000 7f953bc79640 1 --2- 172.21.15.88:0/1682111418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f953406d3f0 0x7f953406d880 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f952c009e50 tx=0x7f952c01c950 comp rx=0 tx=0).stop 2023-10-16T20:06:31.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.383+0000 7f953bc79640 1 -- 172.21.15.88:0/1682111418 shutdown_connections 2023-10-16T20:06:31.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.383+0000 7f953bc79640 1 --2- 172.21.15.88:0/1682111418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f953406d3f0 0x7f953406d880 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:31.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.383+0000 7f953bc79640 1 --2- 172.21.15.88:0/1682111418 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f95340b4260 0x7f953415dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:31.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.383+0000 7f953bc79640 1 -- 172.21.15.88:0/1682111418 >> 172.21.15.88:0/1682111418 conn(0x7f95340cb080 msgr2=0x7f95340cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:31.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.383+0000 7f953bc79640 1 -- 172.21.15.88:0/1682111418 shutdown_connections 2023-10-16T20:06:31.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.383+0000 7f953bc79640 1 -- 172.21.15.88:0/1682111418 wait complete. 2023-10-16T20:06:31.389 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.383+0000 7f953bc79640 1 Processor -- start 2023-10-16T20:06:31.389 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.383+0000 7f953bc79640 1 -- start start 2023-10-16T20:06:31.389 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.387+0000 7f953bc79640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f953406d3f0 0x7f953415aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:31.389 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.387+0000 7f953bc79640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95340b0da0 0x7f95341537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:31.390 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.387+0000 7f953bc79640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f95340b4260 0x7f95341522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:31.390 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.387+0000 7f953bc79640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953415c180 con 0x7f95340b0da0 2023-10-16T20:06:31.390 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.387+0000 7f953bc79640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953415b570 con 0x7f953406d3f0 2023-10-16T20:06:31.390 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.387+0000 7f953bc79640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f953415b850 con 0x7f95340b4260 2023-10-16T20:06:31.390 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.387+0000 7f9539214640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95340b0da0 0x7f95341537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:31.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.387+0000 7f9539214640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95340b0da0 0x7f95341537f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57734/0 (socket says 172.21.15.88:57734) 2023-10-16T20:06:31.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.387+0000 7f9539214640 1 -- 172.21.15.88:0/2138304923 learned_addr learned my addr 172.21.15.88:0/2138304923 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:31.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.387+0000 7f953a216640 1 --2- 172.21.15.88:0/2138304923 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f95340b4260 0x7f95341522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:31.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.387+0000 7f9539a15640 1 --2- 172.21.15.88:0/2138304923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f953406d3f0 0x7f953415aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:31.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.387+0000 7f9539214640 1 -- 172.21.15.88:0/2138304923 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f95340b4260 msgr2=0x7f95341522e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:31.392 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.387+0000 7f9539214640 1 --2- 172.21.15.88:0/2138304923 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f95340b4260 0x7f95341522e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:31.392 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.387+0000 7f9539214640 1 -- 172.21.15.88:0/2138304923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f953406d3f0 msgr2=0x7f953415aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:31.392 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.387+0000 7f9539214640 1 --2- 172.21.15.88:0/2138304923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f953406d3f0 0x7f953415aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:31.392 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.387+0000 7f9539214640 1 -- 172.21.15.88:0/2138304923 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952c009b30 con 0x7f95340b0da0 2023-10-16T20:06:31.392 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.387+0000 7f9539a15640 1 --2- 172.21.15.88:0/2138304923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f953406d3f0 0x7f953415aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:31.393 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.387+0000 7f9539214640 1 --2- 172.21.15.88:0/2138304923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95340b0da0 0x7f95341537f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f952400c500 tx=0x7f952400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:31.393 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.387+0000 7f952a7fc640 1 -- 172.21.15.88:0/2138304923 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9524005220 con 0x7f95340b0da0 2023-10-16T20:06:31.393 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.387+0000 7f952a7fc640 1 -- 172.21.15.88:0/2138304923 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9524005380 con 0x7f95340b0da0 2023-10-16T20:06:31.393 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.387+0000 7f952a7fc640 1 -- 172.21.15.88:0/2138304923 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9524007270 con 0x7f95340b0da0 2023-10-16T20:06:31.393 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.387+0000 7f953bc79640 1 -- 172.21.15.88:0/2138304923 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f953415bb30 con 0x7f95340b0da0 2023-10-16T20:06:31.393 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.387+0000 7f953bc79640 1 -- 172.21.15.88:0/2138304923 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9534153d30 con 0x7f95340b0da0 2023-10-16T20:06:31.394 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.387+0000 7f953bc79640 1 -- 172.21.15.88:0/2138304923 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f953404e9c0 con 0x7f95340b0da0 2023-10-16T20:06:31.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.391+0000 7f952a7fc640 1 -- 172.21.15.88:0/2138304923 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f95240073d0 con 0x7f95340b0da0 2023-10-16T20:06:31.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.391+0000 7f952a7fc640 1 --2- 172.21.15.88:0/2138304923 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f95180664a0 0x7f9518068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:31.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.391+0000 7f952a7fc640 1 -- 172.21.15.88:0/2138304923 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6412+0+0 (secure 0 0 0) 0x7f95240847d0 con 0x7f95340b0da0 2023-10-16T20:06:31.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.391+0000 7f9539a15640 1 --2- 172.21.15.88:0/2138304923 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f95180664a0 0x7f9518068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:31.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.391+0000 7f9539a15640 1 --2- 172.21.15.88:0/2138304923 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f95180664a0 0x7f9518068960 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9530006fd0 tx=0x7f9530008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:31.396 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.395+0000 7f952a7fc640 1 -- 172.21.15.88:0/2138304923 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9524051980 con 0x7f95340b0da0 2023-10-16T20:06:31.579 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:31.575+0000 7f953bc79640 1 -- 172.21.15.88:0/2138304923 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "testcephfs1", "pool": "data_new"} v 0) v1 -- 0x7f9534071500 con 0x7f95340b0da0 2023-10-16T20:06:32.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.215+0000 7f952a7fc640 1 -- 172.21.15.88:0/2138304923 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "testcephfs1", "pool": "data_new"}]=0 added data pool 7 to fsmap v19) v1 ==== 136+0+0 (secure 0 0 0) 0x7f9524054fa0 con 0x7f95340b0da0 2023-10-16T20:06:32.215 INFO:teuthology.orchestra.run.smithi088.stderr:added data pool 7 to fsmap 2023-10-16T20:06:32.218 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.215+0000 7f953bc79640 1 -- 172.21.15.88:0/2138304923 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f95180664a0 msgr2=0x7f9518068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:32.218 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.215+0000 7f953bc79640 1 --2- 172.21.15.88:0/2138304923 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f95180664a0 0x7f9518068960 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9530006fd0 tx=0x7f9530008040 comp rx=0 tx=0).stop 2023-10-16T20:06:32.218 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.215+0000 7f953bc79640 1 -- 172.21.15.88:0/2138304923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95340b0da0 msgr2=0x7f95341537f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:32.218 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.215+0000 7f953bc79640 1 --2- 172.21.15.88:0/2138304923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95340b0da0 0x7f95341537f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f952400c500 tx=0x7f952400c9c0 comp rx=0 tx=0).stop 2023-10-16T20:06:32.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.215+0000 7f953bc79640 1 -- 172.21.15.88:0/2138304923 shutdown_connections 2023-10-16T20:06:32.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.215+0000 7f953bc79640 1 --2- 172.21.15.88:0/2138304923 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f95180664a0 0x7f9518068960 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:32.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.215+0000 7f953bc79640 1 --2- 172.21.15.88:0/2138304923 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f95340b4260 0x7f95341522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:32.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.215+0000 7f953bc79640 1 --2- 172.21.15.88:0/2138304923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95340b0da0 0x7f95341537f0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:32.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.215+0000 7f953bc79640 1 --2- 172.21.15.88:0/2138304923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f953406d3f0 0x7f953415aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:32.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.215+0000 7f953bc79640 1 -- 172.21.15.88:0/2138304923 >> 172.21.15.88:0/2138304923 conn(0x7f95340cb080 msgr2=0x7f9534070630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:32.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.219+0000 7f953bc79640 1 -- 172.21.15.88:0/2138304923 shutdown_connections 2023-10-16T20:06:32.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.219+0000 7f953bc79640 1 -- 172.21.15.88:0/2138304923 wait complete. 2023-10-16T20:06:32.257 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T20:06:32.394 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.391+0000 7f5c73609640 1 Processor -- start 2023-10-16T20:06:32.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.391+0000 7f5c73609640 1 -- start start 2023-10-16T20:06:32.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.395+0000 7f5c73609640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c6c1599f0 0x7f5c6c0b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:32.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.395+0000 7f5c73609640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5c6c0b0cb0 0x7f5c6c06a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:32.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.395+0000 7f5c73609640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c6c06a830 con 0x7f5c6c069e90 2023-10-16T20:06:32.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.395+0000 7f5c73609640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c6c0b1260 con 0x7f5c6c0b0cb0 2023-10-16T20:06:32.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.395+0000 7f5c73609640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c0ae960 con 0x7f5c6c1599f0 2023-10-16T20:06:32.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.395+0000 7f5c71ba6640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5c6c0b0cb0 0x7f5c6c06a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:32.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.395+0000 7f5c713a5640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c6c1599f0 0x7f5c6c0b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:32.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.395+0000 7f5c71ba6640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5c6c0b0cb0 0x7f5c6c06a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43068/0 (socket says 172.21.15.88:43068) 2023-10-16T20:06:32.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.395+0000 7f5c71ba6640 1 -- 172.21.15.88:0/1624943347 learned_addr learned my addr 172.21.15.88:0/1624943347 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:32.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.395+0000 7f5c70ba4640 1 --1- 172.21.15.88:0/1624943347 >> v1:172.21.15.94:6789/0 conn(0x7f5c6c069e90 0x7f5c6c0b05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:39512/0 (socket says 172.21.15.88:39512) 2023-10-16T20:06:32.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.395+0000 7f5c71ba6640 1 -- 172.21.15.88:0/1624943347 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c6c1599f0 msgr2=0x7f5c6c0b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:32.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.395+0000 7f5c71ba6640 1 --2- 172.21.15.88:0/1624943347 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c6c1599f0 0x7f5c6c0b2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:32.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.395+0000 7f5c71ba6640 1 -- 172.21.15.88:0/1624943347 >> v1:172.21.15.94:6789/0 conn(0x7f5c6c069e90 legacy=0x7f5c6c0b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:32.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.395+0000 7f5c71ba6640 1 -- 172.21.15.88:0/1624943347 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c6c0aec40 con 0x7f5c6c0b0cb0 2023-10-16T20:06:32.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.399+0000 7f5c71ba6640 1 --2- 172.21.15.88:0/1624943347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c6c0b0cb0 0x7f5c6c06a2f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009e50 tx=0x7f5c6401c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b157191ad509594 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:32.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.399+0000 7f5c63fff640 1 -- 172.21.15.88:0/1624943347 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c640a58a0 con 0x7f5c6c0b0cb0 2023-10-16T20:06:32.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.399+0000 7f5c63fff640 1 -- 172.21.15.88:0/1624943347 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c640a5a00 con 0x7f5c6c0b0cb0 2023-10-16T20:06:32.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.399+0000 7f5c63fff640 1 -- 172.21.15.88:0/1624943347 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c640a5d90 con 0x7f5c6c0b0cb0 2023-10-16T20:06:32.401 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.399+0000 7f5c73609640 1 -- 172.21.15.88:0/1624943347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c6c0b0cb0 msgr2=0x7f5c6c06a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:32.401 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.399+0000 7f5c73609640 1 --2- 172.21.15.88:0/1624943347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c6c0b0cb0 0x7f5c6c06a2f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009e50 tx=0x7f5c6401c9e0 comp rx=0 tx=0).stop 2023-10-16T20:06:32.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.399+0000 7f5c73609640 1 -- 172.21.15.88:0/1624943347 shutdown_connections 2023-10-16T20:06:32.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.399+0000 7f5c73609640 1 --2- 172.21.15.88:0/1624943347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c6c0b0cb0 0x7f5c6c06a2f0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:32.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.399+0000 7f5c73609640 1 --2- 172.21.15.88:0/1624943347 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c6c1599f0 0x7f5c6c0b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:32.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.399+0000 7f5c73609640 1 -- 172.21.15.88:0/1624943347 >> 172.21.15.88:0/1624943347 conn(0x7f5c6c0ce5e0 msgr2=0x7f5c6c0ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:32.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.399+0000 7f5c73609640 1 -- 172.21.15.88:0/1624943347 shutdown_connections 2023-10-16T20:06:32.403 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.399+0000 7f5c73609640 1 -- 172.21.15.88:0/1624943347 wait complete. 2023-10-16T20:06:32.404 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.399+0000 7f5c73609640 1 Processor -- start 2023-10-16T20:06:32.404 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.399+0000 7f5c73609640 1 -- start start 2023-10-16T20:06:32.404 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.403+0000 7f5c73609640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c6c069e90 0x7f5c6c1494f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:32.404 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.403+0000 7f5c73609640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c6c0b0cb0 0x7f5c6c158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:32.405 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.403+0000 7f5c73609640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c6c1599f0 0x7f5c6c14a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:32.405 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.403+0000 7f5c73609640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c155ef0 con 0x7f5c6c0b0cb0 2023-10-16T20:06:32.405 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.403+0000 7f5c73609640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c155650 con 0x7f5c6c1599f0 2023-10-16T20:06:32.405 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.403+0000 7f5c73609640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c155930 con 0x7f5c6c069e90 2023-10-16T20:06:32.405 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.403+0000 7f5c713a5640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c6c069e90 0x7f5c6c1494f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:32.406 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.403+0000 7f5c70ba4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c6c0b0cb0 0x7f5c6c158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:32.406 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.403+0000 7f5c713a5640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c6c069e90 0x7f5c6c1494f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:57302/0 (socket says 172.21.15.88:57302) 2023-10-16T20:06:32.406 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.403+0000 7f5c713a5640 1 -- 172.21.15.88:0/1556880010 learned_addr learned my addr 172.21.15.88:0/1556880010 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:32.406 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.403+0000 7f5c71ba6640 1 --2- 172.21.15.88:0/1556880010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c6c1599f0 0x7f5c6c14a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:32.406 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.403+0000 7f5c713a5640 1 -- 172.21.15.88:0/1556880010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c6c1599f0 msgr2=0x7f5c6c14a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:32.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.403+0000 7f5c713a5640 1 --2- 172.21.15.88:0/1556880010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c6c1599f0 0x7f5c6c14a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:32.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.403+0000 7f5c713a5640 1 -- 172.21.15.88:0/1556880010 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c6c0b0cb0 msgr2=0x7f5c6c158940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:32.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.403+0000 7f5c713a5640 1 --2- 172.21.15.88:0/1556880010 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c6c0b0cb0 0x7f5c6c158940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:32.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.403+0000 7f5c713a5640 1 -- 172.21.15.88:0/1556880010 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c64009b30 con 0x7f5c6c069e90 2023-10-16T20:06:32.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.403+0000 7f5c70ba4640 1 --2- 172.21.15.88:0/1556880010 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c6c0b0cb0 0x7f5c6c158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:06:32.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.403+0000 7f5c713a5640 1 --2- 172.21.15.88:0/1556880010 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c6c069e90 0x7f5c6c1494f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800e870 tx=0x7f5c6800ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:32.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.403+0000 7f5c61ffb640 1 -- 172.21.15.88:0/1556880010 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c68009510 con 0x7f5c6c069e90 2023-10-16T20:06:32.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.403+0000 7f5c61ffb640 1 -- 172.21.15.88:0/1556880010 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6800eec0 con 0x7f5c6c069e90 2023-10-16T20:06:32.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.403+0000 7f5c61ffb640 1 -- 172.21.15.88:0/1556880010 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6800cb40 con 0x7f5c6c069e90 2023-10-16T20:06:32.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.403+0000 7f5c73609640 1 -- 172.21.15.88:0/1556880010 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c6c155c10 con 0x7f5c6c069e90 2023-10-16T20:06:32.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.403+0000 7f5c73609640 1 -- 172.21.15.88:0/1556880010 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c6c158e80 con 0x7f5c6c069e90 2023-10-16T20:06:32.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.403+0000 7f5c73609640 1 -- 172.21.15.88:0/1556880010 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c6c06a660 con 0x7f5c6c069e90 2023-10-16T20:06:32.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.407+0000 7f5c61ffb640 1 -- 172.21.15.88:0/1556880010 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5c6800cca0 con 0x7f5c6c069e90 2023-10-16T20:06:32.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.407+0000 7f5c61ffb640 1 --2- 172.21.15.88:0/1556880010 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5c500664a0 0x7f5c50068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:32.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.407+0000 7f5c61ffb640 1 -- 172.21.15.88:0/1556880010 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f5c68084290 con 0x7f5c6c069e90 2023-10-16T20:06:32.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.407+0000 7f5c70ba4640 1 --2- 172.21.15.88:0/1556880010 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5c500664a0 0x7f5c50068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:32.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.407+0000 7f5c70ba4640 1 --2- 172.21.15.88:0/1556880010 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5c500664a0 0x7f5c50068960 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c1f8af0 tx=0x7f5c5c007e90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:32.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.411+0000 7f5c61ffb640 1 -- 172.21.15.88:0/1556880010 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5c68051430 con 0x7f5c6c069e90 2023-10-16T20:06:32.593 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.591+0000 7f5c73609640 1 -- 172.21.15.88:0/1556880010 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c6c155a70 con 0x7f5c6c069e90 2023-10-16T20:06:32.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.591+0000 7f5c61ffb640 1 -- 172.21.15.88:0/1556880010 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4917 (secure 0 0 0) 0x7f5c6c155a70 con 0x7f5c6c069e90 2023-10-16T20:06:32.595 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:06:32.595 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/696292235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":696292235},{"type":"v1","addr":"172.21.15.94:6834","nonce":696292235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/2152672724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2152672724},{"type":"v1","addr":"172.21.15.88:6835","nonce":2152672724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3107974207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":3107974207},{"type":"v1","addr":"172.21.15.88:6837","nonce":3107974207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T20:06:17.088386+0000","modified":"2023-10-16T20:06:31.579567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5535":{"gid":5535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6835/2190281591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":2190281591},{"type":"v1","addr":"172.21.15.94:6835","nonce":2190281591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"testcephfs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-16T20:06:32.595 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 19 2023-10-16T20:06:32.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.595+0000 7f5c73609640 1 -- 172.21.15.88:0/1556880010 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5c500664a0 msgr2=0x7f5c50068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:32.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.595+0000 7f5c73609640 1 --2- 172.21.15.88:0/1556880010 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5c500664a0 0x7f5c50068960 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c1f8af0 tx=0x7f5c5c007e90 comp rx=0 tx=0).stop 2023-10-16T20:06:32.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.595+0000 7f5c73609640 1 -- 172.21.15.88:0/1556880010 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c6c069e90 msgr2=0x7f5c6c1494f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:32.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.595+0000 7f5c73609640 1 --2- 172.21.15.88:0/1556880010 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c6c069e90 0x7f5c6c1494f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800e870 tx=0x7f5c6800ec70 comp rx=0 tx=0).stop 2023-10-16T20:06:32.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.595+0000 7f5c73609640 1 -- 172.21.15.88:0/1556880010 shutdown_connections 2023-10-16T20:06:32.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.595+0000 7f5c73609640 1 --2- 172.21.15.88:0/1556880010 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5c500664a0 0x7f5c50068960 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:32.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.595+0000 7f5c73609640 1 --2- 172.21.15.88:0/1556880010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c6c1599f0 0x7f5c6c14a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:32.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.595+0000 7f5c73609640 1 --2- 172.21.15.88:0/1556880010 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c6c0b0cb0 0x7f5c6c158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:32.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.595+0000 7f5c73609640 1 --2- 172.21.15.88:0/1556880010 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c6c069e90 0x7f5c6c1494f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:32.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.595+0000 7f5c73609640 1 -- 172.21.15.88:0/1556880010 >> 172.21.15.88:0/1556880010 conn(0x7f5c6c0ce5e0 msgr2=0x7f5c6c071830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:32.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.595+0000 7f5c73609640 1 -- 172.21.15.88:0/1556880010 shutdown_connections 2023-10-16T20:06:32.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.595+0000 7f5c73609640 1 -- 172.21.15.88:0/1556880010 wait complete. 2023-10-16T20:06:32.615 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:06:32.754 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.751+0000 7f51d70c9640 1 Processor -- start 2023-10-16T20:06:32.755 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.751+0000 7f51d70c9640 1 -- start start 2023-10-16T20:06:32.756 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.755+0000 7f51d70c9640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f51d0160420 0x7f51d0069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:32.756 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.755+0000 7f51d70c9640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f51d006a3d0 0x7f51d00c3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:32.756 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.755+0000 7f51d70c9640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51d006a820 con 0x7f51d0160040 2023-10-16T20:06:32.757 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.755+0000 7f51d70c9640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51d00cd7d0 con 0x7f51d0160420 2023-10-16T20:06:32.757 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.755+0000 7f51d70c9640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51d00cdab0 con 0x7f51d006a3d0 2023-10-16T20:06:32.757 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.755+0000 7f51c7fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f51d0160420 0x7f51d0069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:32.757 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.755+0000 7f51c7fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f51d0160420 0x7f51d0069e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43082/0 (socket says 172.21.15.88:43082) 2023-10-16T20:06:32.757 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.755+0000 7f51c7fff640 1 -- 172.21.15.88:0/3311020421 learned_addr learned my addr 172.21.15.88:0/3311020421 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:32.758 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.755+0000 7f51d4e65640 1 --1- 172.21.15.88:0/3311020421 >> v1:172.21.15.88:6789/0 conn(0x7f51d0160040 0x7f51d0071f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:41422/0 (socket says 172.21.15.88:41422) 2023-10-16T20:06:32.758 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.755+0000 7f51d5666640 1 --2- 172.21.15.88:0/3311020421 >> v2:172.21.15.94:3300/0 conn(0x7f51d006a3d0 0x7f51d00c3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:32.758 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.755+0000 7f51c7fff640 1 -- 172.21.15.88:0/3311020421 >> v2:172.21.15.94:3300/0 conn(0x7f51d006a3d0 msgr2=0x7f51d00c3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:32.758 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.755+0000 7f51c7fff640 1 --2- 172.21.15.88:0/3311020421 >> v2:172.21.15.94:3300/0 conn(0x7f51d006a3d0 0x7f51d00c3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:32.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.755+0000 7f51c7fff640 1 -- 172.21.15.88:0/3311020421 >> v1:172.21.15.88:6789/0 conn(0x7f51d0160040 legacy=0x7f51d0071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:32.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.755+0000 7f51c7fff640 1 -- 172.21.15.88:0/3311020421 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d00cdd90 con 0x7f51d0160420 2023-10-16T20:06:32.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.755+0000 7f51c77fe640 1 -- 172.21.15.88:0/3311020421 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1434654372 0 0) 0x7f51cc001120 con 0x7f51d0160040 2023-10-16T20:06:32.760 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.759+0000 7f51c7fff640 1 --2- 172.21.15.88:0/3311020421 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51d0160420 0x7f51d0069e90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f51d006af20 tx=0x7f51c001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2398410233477863 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:32.760 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.759+0000 7f51c77fe640 1 -- 172.21.15.88:0/3311020421 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c00aa5f0 con 0x7f51d0160420 2023-10-16T20:06:32.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.759+0000 7f51c77fe640 1 -- 172.21.15.88:0/3311020421 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c00aa750 con 0x7f51d0160420 2023-10-16T20:06:32.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.759+0000 7f51c77fe640 1 -- 172.21.15.88:0/3311020421 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c00aaae0 con 0x7f51d0160420 2023-10-16T20:06:32.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.759+0000 7f51d70c9640 1 -- 172.21.15.88:0/3311020421 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51d0160420 msgr2=0x7f51d0069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:32.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.759+0000 7f51d70c9640 1 --2- 172.21.15.88:0/3311020421 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51d0160420 0x7f51d0069e90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f51d006af20 tx=0x7f51c001c4d0 comp rx=0 tx=0).stop 2023-10-16T20:06:32.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.759+0000 7f51d70c9640 1 -- 172.21.15.88:0/3311020421 shutdown_connections 2023-10-16T20:06:32.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.759+0000 7f51d70c9640 1 --2- 172.21.15.88:0/3311020421 >> v2:172.21.15.94:3300/0 conn(0x7f51d006a3d0 0x7f51d00c3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:32.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.759+0000 7f51d70c9640 1 --2- 172.21.15.88:0/3311020421 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51d0160420 0x7f51d0069e90 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:32.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.759+0000 7f51d70c9640 1 -- 172.21.15.88:0/3311020421 >> 172.21.15.88:0/3311020421 conn(0x7f51d006d340 msgr2=0x7f51d006d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:32.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.759+0000 7f51d70c9640 1 -- 172.21.15.88:0/3311020421 shutdown_connections 2023-10-16T20:06:32.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.759+0000 7f51d70c9640 1 -- 172.21.15.88:0/3311020421 wait complete. 2023-10-16T20:06:32.764 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.759+0000 7f51d70c9640 1 Processor -- start 2023-10-16T20:06:32.764 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.759+0000 7f51d70c9640 1 -- start start 2023-10-16T20:06:32.765 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.763+0000 7f51d70c9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51d006a3d0 0x7f51d00b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:32.765 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.763+0000 7f51d70c9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f51d0160040 0x7f51d0155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:32.765 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.763+0000 7f51d70c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51d0160420 0x7f51d0155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:32.765 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.763+0000 7f51d70c9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d01564f0 con 0x7f51d006a3d0 2023-10-16T20:06:32.765 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.763+0000 7f51d70c9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d00c7770 con 0x7f51d0160420 2023-10-16T20:06:32.766 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.763+0000 7f51d70c9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d00cc0f0 con 0x7f51d0160040 2023-10-16T20:06:32.766 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.763+0000 7f51d4e65640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51d006a3d0 0x7f51d00b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:32.766 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.763+0000 7f51d4e65640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51d006a3d0 0x7f51d00b2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43092/0 (socket says 172.21.15.88:43092) 2023-10-16T20:06:32.766 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.763+0000 7f51d4e65640 1 -- 172.21.15.88:0/1486876923 learned_addr learned my addr 172.21.15.88:0/1486876923 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:32.766 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.763+0000 7f51c7fff640 1 --2- 172.21.15.88:0/1486876923 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f51d0160040 0x7f51d0155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:32.767 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.763+0000 7f51d5666640 1 --2- 172.21.15.88:0/1486876923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51d0160420 0x7f51d0155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:32.767 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.763+0000 7f51d4e65640 1 -- 172.21.15.88:0/1486876923 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f51d0160040 msgr2=0x7f51d0155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:32.767 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.763+0000 7f51d4e65640 1 --2- 172.21.15.88:0/1486876923 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f51d0160040 0x7f51d0155a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:32.767 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.763+0000 7f51d4e65640 1 -- 172.21.15.88:0/1486876923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51d0160420 msgr2=0x7f51d0155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:32.767 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.763+0000 7f51d4e65640 1 --2- 172.21.15.88:0/1486876923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51d0160420 0x7f51d0155fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:32.767 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.763+0000 7f51d4e65640 1 -- 172.21.15.88:0/1486876923 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c00094d0 con 0x7f51d006a3d0 2023-10-16T20:06:32.768 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.763+0000 7f51d5666640 1 --2- 172.21.15.88:0/1486876923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51d0160420 0x7f51d0155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:32.768 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.763+0000 7f51d4e65640 1 --2- 172.21.15.88:0/1486876923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51d006a3d0 0x7f51d00b2610 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00c7b0 tx=0x7f51cc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:32.768 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.763+0000 7f51c57fa640 1 -- 172.21.15.88:0/1486876923 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc007270 con 0x7f51d006a3d0 2023-10-16T20:06:32.768 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.763+0000 7f51c57fa640 1 -- 172.21.15.88:0/1486876923 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51cc0073d0 con 0x7f51d006a3d0 2023-10-16T20:06:32.768 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.763+0000 7f51c57fa640 1 -- 172.21.15.88:0/1486876923 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc0047f0 con 0x7f51d006a3d0 2023-10-16T20:06:32.769 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.763+0000 7f51d70c9640 1 -- 172.21.15.88:0/1486876923 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d00c6e40 con 0x7f51d006a3d0 2023-10-16T20:06:32.769 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.763+0000 7f51d70c9640 1 -- 172.21.15.88:0/1486876923 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d00c7400 con 0x7f51d006a3d0 2023-10-16T20:06:32.770 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.763+0000 7f51d70c9640 1 -- 172.21.15.88:0/1486876923 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d004e9c0 con 0x7f51d006a3d0 2023-10-16T20:06:32.770 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.763+0000 7f51c57fa640 1 -- 172.21.15.88:0/1486876923 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f51cc004950 con 0x7f51d006a3d0 2023-10-16T20:06:32.770 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.767+0000 7f51c57fa640 1 --2- 172.21.15.88:0/1486876923 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f51bc0664a0 0x7f51bc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:32.770 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.767+0000 7f51c57fa640 1 -- 172.21.15.88:0/1486876923 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f51cc082ff0 con 0x7f51d006a3d0 2023-10-16T20:06:32.770 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.767+0000 7f51c7fff640 1 --2- 172.21.15.88:0/1486876923 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f51bc0664a0 0x7f51bc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:32.771 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.767+0000 7f51c7fff640 1 --2- 172.21.15.88:0/1486876923 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f51bc0664a0 0x7f51bc068960 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f51c00ae000 tx=0x7f51c00a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:32.771 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.771+0000 7f51c57fa640 1 -- 172.21.15.88:0/1486876923 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f51cc0b4090 con 0x7f51d006a3d0 2023-10-16T20:06:32.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.919+0000 7f51d70c9640 1 -- 172.21.15.88:0/1486876923 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51d006a820 con 0x7f51d006a3d0 2023-10-16T20:06:32.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.923+0000 7f51c57fa640 1 -- 172.21.15.88:0/1486876923 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+18519 (secure 0 0 0) 0x7f51d006a820 con 0x7f51d006a3d0 2023-10-16T20:06:32.924 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:06:32.925 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":43,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:06:31.579359+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T20:06:14.743500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"testcephfs1_data","create_time":"2023-10-16T20:06:16.091837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-10-16T20:06:31.184462+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:06:32.926 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.923+0000 7f51d70c9640 1 -- 172.21.15.88:0/1486876923 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f51bc0664a0 msgr2=0x7f51bc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:32.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.923+0000 7f51d70c9640 1 --2- 172.21.15.88:0/1486876923 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f51bc0664a0 0x7f51bc068960 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f51c00ae000 tx=0x7f51c00a7000 comp rx=0 tx=0).stop 2023-10-16T20:06:32.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.923+0000 7f51d70c9640 1 -- 172.21.15.88:0/1486876923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51d006a3d0 msgr2=0x7f51d00b2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:32.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.923+0000 7f51d70c9640 1 --2- 172.21.15.88:0/1486876923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51d006a3d0 0x7f51d00b2610 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00c7b0 tx=0x7f51cc00cc70 comp rx=0 tx=0).stop 2023-10-16T20:06:32.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.923+0000 7f51d70c9640 1 -- 172.21.15.88:0/1486876923 shutdown_connections 2023-10-16T20:06:32.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.923+0000 7f51d70c9640 1 --2- 172.21.15.88:0/1486876923 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f51bc0664a0 0x7f51bc068960 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:32.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.923+0000 7f51d70c9640 1 --2- 172.21.15.88:0/1486876923 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f51d0160040 0x7f51d0155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:32.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.923+0000 7f51d70c9640 1 --2- 172.21.15.88:0/1486876923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51d0160420 0x7f51d0155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:32.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.923+0000 7f51d70c9640 1 --2- 172.21.15.88:0/1486876923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51d006a3d0 0x7f51d00b2610 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:32.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.923+0000 7f51d70c9640 1 -- 172.21.15.88:0/1486876923 >> 172.21.15.88:0/1486876923 conn(0x7f51d006d340 msgr2=0x7f51d0156b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:32.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.923+0000 7f51d70c9640 1 -- 172.21.15.88:0/1486876923 shutdown_connections 2023-10-16T20:06:32.929 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:32.923+0000 7f51d70c9640 1 -- 172.21.15.88:0/1486876923 wait complete. 2023-10-16T20:06:32.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:32.946 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch layoutfile) 2023-10-16T20:06:32.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:32.966 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir layoutdir) 2023-10-16T20:06:33.024 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds 'allow rw' mon 'allow r' osd 'allow rw pool=testcephfs1_data,allow rw pool=data_new' 2023-10-16T20:06:33.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.207+0000 7fa4073d9640 1 Processor -- start 2023-10-16T20:06:33.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.207+0000 7fa4073d9640 1 -- start start 2023-10-16T20:06:33.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.207+0000 7fa4073d9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa400157130 0x7fa40015de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:33.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.207+0000 7fa4073d9640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa400161470 con 0x7fa400157510 2023-10-16T20:06:33.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.211+0000 7fa4073d9640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa400161750 con 0x7fa4000ce0b0 2023-10-16T20:06:33.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.211+0000 7fa4073d9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4000b88a0 con 0x7fa400157130 2023-10-16T20:06:33.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.211+0000 7fa404974640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fa400157510 0x7fa4000cd9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:41434/0 (socket says 172.21.15.88:41434) 2023-10-16T20:06:33.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.211+0000 7fa404974640 1 -- 172.21.15.88:0/141846964 learned_addr learned my addr 172.21.15.88:0/141846964 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:33.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.211+0000 7fa405175640 1 --2- 172.21.15.88:0/141846964 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa400157130 0x7fa40015de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:33.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.211+0000 7fa405175640 1 -- 172.21.15.88:0/141846964 >> v1:172.21.15.94:6789/0 conn(0x7fa4000ce0b0 legacy=0x7fa400160860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:33.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.211+0000 7fa3f7fff640 1 -- 172.21.15.88:0/141846964 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1544939059 0 0) 0x7fa400161470 con 0x7fa400157510 2023-10-16T20:06:33.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.211+0000 7fa405175640 1 -- 172.21.15.88:0/141846964 >> v1:172.21.15.88:6789/0 conn(0x7fa400157510 legacy=0x7fa4000cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:33.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.211+0000 7fa405175640 1 -- 172.21.15.88:0/141846964 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40015d380 con 0x7fa400157130 2023-10-16T20:06:33.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.215+0000 7fa405175640 1 --2- 172.21.15.88:0/141846964 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa400157130 0x7fa40015de60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc0097f0 tx=0x7fa3fc01c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7d0626f249f0101 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:33.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.215+0000 7fa3f7fff640 1 -- 172.21.15.88:0/141846964 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc0aa5f0 con 0x7fa400157130 2023-10-16T20:06:33.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.215+0000 7fa3f7fff640 1 -- 172.21.15.88:0/141846964 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3fc0aa750 con 0x7fa400157130 2023-10-16T20:06:33.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.215+0000 7fa3f7fff640 1 -- 172.21.15.88:0/141846964 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc0aaae0 con 0x7fa400157130 2023-10-16T20:06:33.217 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.215+0000 7fa4073d9640 1 -- 172.21.15.88:0/141846964 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa400157130 msgr2=0x7fa40015de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:33.218 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.215+0000 7fa4073d9640 1 --2- 172.21.15.88:0/141846964 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa400157130 0x7fa40015de60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc0097f0 tx=0x7fa3fc01c490 comp rx=0 tx=0).stop 2023-10-16T20:06:33.218 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.215+0000 7fa4073d9640 1 -- 172.21.15.88:0/141846964 shutdown_connections 2023-10-16T20:06:33.218 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.215+0000 7fa4073d9640 1 --2- 172.21.15.88:0/141846964 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa400157130 0x7fa40015de60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:33.218 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.215+0000 7fa4073d9640 1 -- 172.21.15.88:0/141846964 >> 172.21.15.88:0/141846964 conn(0x7fa4000715c0 msgr2=0x7fa4000719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:33.218 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.215+0000 7fa4073d9640 1 -- 172.21.15.88:0/141846964 shutdown_connections 2023-10-16T20:06:33.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.215+0000 7fa4073d9640 1 -- 172.21.15.88:0/141846964 wait complete. 2023-10-16T20:06:33.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.215+0000 7fa4073d9640 1 Processor -- start 2023-10-16T20:06:33.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.215+0000 7fa4073d9640 1 -- start start 2023-10-16T20:06:33.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.219+0000 7fa4073d9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4000ce0b0 0x7fa4000cb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:33.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.219+0000 7fa4073d9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa400157130 0x7fa40014d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:33.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.219+0000 7fa4073d9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa400157510 0x7fa40014da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:33.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.219+0000 7fa4073d9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4000cb670 con 0x7fa4000ce0b0 2023-10-16T20:06:33.222 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.219+0000 7fa4073d9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4000ca720 con 0x7fa400157130 2023-10-16T20:06:33.222 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.219+0000 7fa4073d9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40014dfd0 con 0x7fa400157510 2023-10-16T20:06:33.222 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.219+0000 7fa405175640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4000ce0b0 0x7fa4000cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:33.222 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.219+0000 7fa405175640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4000ce0b0 0x7fa4000cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43106/0 (socket says 172.21.15.88:43106) 2023-10-16T20:06:33.222 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.219+0000 7fa405175640 1 -- 172.21.15.88:0/2485885570 learned_addr learned my addr 172.21.15.88:0/2485885570 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:33.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.219+0000 7fa404974640 1 --2- 172.21.15.88:0/2485885570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa400157130 0x7fa40014d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:33.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.219+0000 7fa405976640 1 --2- 172.21.15.88:0/2485885570 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa400157510 0x7fa40014da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:33.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.219+0000 7fa405175640 1 -- 172.21.15.88:0/2485885570 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa400157510 msgr2=0x7fa40014da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:33.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.219+0000 7fa405175640 1 --2- 172.21.15.88:0/2485885570 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa400157510 0x7fa40014da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:33.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.219+0000 7fa405175640 1 -- 172.21.15.88:0/2485885570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa400157130 msgr2=0x7fa40014d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:33.224 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.219+0000 7fa405175640 1 --2- 172.21.15.88:0/2485885570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa400157130 0x7fa40014d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:33.224 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.219+0000 7fa405175640 1 -- 172.21.15.88:0/2485885570 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3fc009420 con 0x7fa4000ce0b0 2023-10-16T20:06:33.225 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.219+0000 7fa405175640 1 --2- 172.21.15.88:0/2485885570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4000ce0b0 0x7fa4000cb130 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc0097f0 tx=0x7fa3fc01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:33.225 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.219+0000 7fa3f5ffb640 1 -- 172.21.15.88:0/2485885570 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc004220 con 0x7fa4000ce0b0 2023-10-16T20:06:33.226 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.219+0000 7fa3f5ffb640 1 -- 172.21.15.88:0/2485885570 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3fc0b8070 con 0x7fa4000ce0b0 2023-10-16T20:06:33.226 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.219+0000 7fa3f5ffb640 1 -- 172.21.15.88:0/2485885570 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc003620 con 0x7fa4000ce0b0 2023-10-16T20:06:33.226 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.219+0000 7fa4073d9640 1 -- 172.21.15.88:0/2485885570 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4000b50e0 con 0x7fa4000ce0b0 2023-10-16T20:06:33.226 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.219+0000 7fa4073d9640 1 -- 172.21.15.88:0/2485885570 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4000ca930 con 0x7fa4000ce0b0 2023-10-16T20:06:33.226 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.219+0000 7fa4073d9640 1 -- 172.21.15.88:0/2485885570 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa40004e9c0 con 0x7fa4000ce0b0 2023-10-16T20:06:33.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.223+0000 7fa3f5ffb640 1 -- 172.21.15.88:0/2485885570 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa3fc003780 con 0x7fa4000ce0b0 2023-10-16T20:06:33.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.223+0000 7fa3f5ffb640 1 --2- 172.21.15.88:0/2485885570 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa3e40664a0 0x7fa3e4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:33.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.223+0000 7fa3f5ffb640 1 -- 172.21.15.88:0/2485885570 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6449+0+0 (secure 0 0 0) 0x7fa3fc0ba070 con 0x7fa4000ce0b0 2023-10-16T20:06:33.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.223+0000 7fa404974640 1 --2- 172.21.15.88:0/2485885570 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa3e40664a0 0x7fa3e4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:33.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.223+0000 7fa404974640 1 --2- 172.21.15.88:0/2485885570 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa3e40664a0 0x7fa3e4068960 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa4000ce750 tx=0x7fa3f0007240 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:33.228 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.227+0000 7fa3f5ffb640 1 -- 172.21.15.88:0/2485885570 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa3fc0ebd50 con 0x7fa4000ce0b0 2023-10-16T20:06:33.431 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.431+0000 7fa4073d9640 1 -- 172.21.15.88:0/2485885570 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rw", "mon", "allow r", "osd", "allow rw pool=testcephfs1_data,allow rw pool=data_new"]} v 0) v1 -- 0x7fa4000b5590 con 0x7fa4000ce0b0 2023-10-16T20:06:33.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.431+0000 7fa3f5ffb640 1 -- 172.21.15.88:0/2485885570 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rw", "mon", "allow r", "osd", "allow rw pool=testcephfs1_data,allow rw pool=data_new"]}]=0 updated caps for client.0 v14) v1 ==== 215+0+172 (secure 0 0 0) 0x7fa3fc0c3070 con 0x7fa4000ce0b0 2023-10-16T20:06:33.435 INFO:teuthology.orchestra.run.smithi088.stdout:[client.0] 2023-10-16T20:06:33.435 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQAvly1looeRAhAAeLshHOVIhQqoTtlPehorIg== 2023-10-16T20:06:33.435 INFO:teuthology.orchestra.run.smithi088.stdout: caps mds = "allow rw" 2023-10-16T20:06:33.435 INFO:teuthology.orchestra.run.smithi088.stdout: caps mon = "allow r" 2023-10-16T20:06:33.436 INFO:teuthology.orchestra.run.smithi088.stdout: caps osd = "allow rw pool=testcephfs1_data,allow rw pool=data_new" 2023-10-16T20:06:33.436 INFO:teuthology.orchestra.run.smithi088.stderr:updated caps for client.0 2023-10-16T20:06:33.437 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.435+0000 7fa4073d9640 1 -- 172.21.15.88:0/2485885570 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa3e40664a0 msgr2=0x7fa3e4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:33.437 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.435+0000 7fa4073d9640 1 --2- 172.21.15.88:0/2485885570 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa3e40664a0 0x7fa3e4068960 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa4000ce750 tx=0x7fa3f0007240 comp rx=0 tx=0).stop 2023-10-16T20:06:33.437 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.435+0000 7fa4073d9640 1 -- 172.21.15.88:0/2485885570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4000ce0b0 msgr2=0x7fa4000cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:33.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.435+0000 7fa4073d9640 1 --2- 172.21.15.88:0/2485885570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4000ce0b0 0x7fa4000cb130 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc0097f0 tx=0x7fa3fc01acd0 comp rx=0 tx=0).stop 2023-10-16T20:06:33.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.435+0000 7fa4073d9640 1 -- 172.21.15.88:0/2485885570 shutdown_connections 2023-10-16T20:06:33.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.435+0000 7fa4073d9640 1 --2- 172.21.15.88:0/2485885570 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa3e40664a0 0x7fa3e4068960 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:33.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.435+0000 7fa4073d9640 1 --2- 172.21.15.88:0/2485885570 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa400157510 0x7fa40014da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:33.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.435+0000 7fa4073d9640 1 --2- 172.21.15.88:0/2485885570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa400157130 0x7fa40014d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:33.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.435+0000 7fa4073d9640 1 --2- 172.21.15.88:0/2485885570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4000ce0b0 0x7fa4000cb130 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:33.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.435+0000 7fa4073d9640 1 -- 172.21.15.88:0/2485885570 >> 172.21.15.88:0/2485885570 conn(0x7fa4000715c0 msgr2=0x7fa40015bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:33.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.435+0000 7fa4073d9640 1 -- 172.21.15.88:0/2485885570 shutdown_connections 2023-10-16T20:06:33.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.435+0000 7fa4073d9640 1 -- 172.21.15.88:0/2485885570 wait complete. 2023-10-16T20:06:33.455 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:06:33.455 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:06:33.465 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:06:33.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.647+0000 7f3bad548640 1 Processor -- start 2023-10-16T20:06:33.648 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.647+0000 7f3bad548640 1 -- start start 2023-10-16T20:06:33.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.647+0000 7f3bad548640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3ba8159820 0x7f3ba80ca820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:33.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.647+0000 7f3bad548640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba8071ba0 con 0x7f3ba80b1360 2023-10-16T20:06:33.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.647+0000 7f3bad548640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba8071e80 con 0x7f3ba80b17c0 2023-10-16T20:06:33.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.647+0000 7f3bad548640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba8072440 con 0x7f3ba8159820 2023-10-16T20:06:33.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.647+0000 7f3ba77fe640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f3ba80b1360 0x7f3ba80711a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:41444/0 (socket says 172.21.15.88:41444) 2023-10-16T20:06:33.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.647+0000 7f3ba77fe640 1 -- 172.21.15.88:0/682842704 learned_addr learned my addr 172.21.15.88:0/682842704 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:33.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.647+0000 7f3ba7fff640 1 --2- 172.21.15.88:0/682842704 >> v2:172.21.15.94:3300/0 conn(0x7f3ba8159820 0x7f3ba80ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:33.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.647+0000 7f3ba6ffd640 1 -- 172.21.15.88:0/682842704 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 188304683 0 0) 0x7f3ba8071ba0 con 0x7f3ba80b1360 2023-10-16T20:06:33.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.647+0000 7f3ba6ffd640 1 -- 172.21.15.88:0/682842704 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b90003530 con 0x7f3ba80b1360 2023-10-16T20:06:33.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.647+0000 7f3ba6ffd640 1 -- 172.21.15.88:0/682842704 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1388558622 0 0) 0x7f3b90003530 con 0x7f3ba80b1360 2023-10-16T20:06:33.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.647+0000 7f3ba6ffd640 1 -- 172.21.15.88:0/682842704 >> v2:172.21.15.94:3300/0 conn(0x7f3ba8159820 msgr2=0x7f3ba80ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:33.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.647+0000 7f3ba6ffd640 1 --2- 172.21.15.88:0/682842704 >> v2:172.21.15.94:3300/0 conn(0x7f3ba8159820 0x7f3ba80ca820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:33.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.647+0000 7f3ba6ffd640 1 -- 172.21.15.88:0/682842704 >> v1:172.21.15.94:6789/0 conn(0x7f3ba80b17c0 legacy=0x7f3ba80aff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:33.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.647+0000 7f3ba6ffd640 1 -- 172.21.15.88:0/682842704 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba8072160 con 0x7f3ba80b1360 2023-10-16T20:06:33.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.647+0000 7f3ba6ffd640 1 -- 172.21.15.88:0/682842704 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f3b9c001680 con 0x7f3ba80b1360 2023-10-16T20:06:33.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.647+0000 7f3ba6ffd640 1 -- 172.21.15.88:0/682842704 >> v1:172.21.15.88:6789/0 conn(0x7f3ba80b1360 legacy=0x7f3ba80711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:33.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.647+0000 7f3ba6ffd640 1 --2- 172.21.15.88:0/682842704 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3b90003980 0x7f3b90003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:33.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.647+0000 7f3ba6ffd640 1 --2- 172.21.15.88:0/682842704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b90004380 0x7f3b90004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:33.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.647+0000 7f3ba6ffd640 1 --2- 172.21.15.88:0/682842704 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b90005050 0x7f3b9000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:33.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.647+0000 7f3ba6ffd640 1 -- 172.21.15.88:0/682842704 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba8072440 con 0x7f3b90005050 2023-10-16T20:06:33.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.647+0000 7f3ba6ffd640 1 -- 172.21.15.88:0/682842704 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9000bfb0 con 0x7f3b90004380 2023-10-16T20:06:33.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.647+0000 7f3ba6ffd640 1 -- 172.21.15.88:0/682842704 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9000c170 con 0x7f3b90003980 2023-10-16T20:06:33.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.647+0000 7f3bacd47640 1 --2- 172.21.15.88:0/682842704 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b90005050 0x7f3b9000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:33.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.647+0000 7f3ba7fff640 1 --2- 172.21.15.88:0/682842704 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3b90003980 0x7f3b90003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:33.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.647+0000 7f3ba77fe640 1 --2- 172.21.15.88:0/682842704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b90004380 0x7f3b90004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:33.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.647+0000 7f3bacd47640 1 -- 172.21.15.88:0/682842704 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3b90003980 msgr2=0x7f3b90003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:33.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.647+0000 7f3bacd47640 1 --2- 172.21.15.88:0/682842704 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3b90003980 0x7f3b90003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:33.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.647+0000 7f3bacd47640 1 -- 172.21.15.88:0/682842704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b90004380 msgr2=0x7f3b90004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:33.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.647+0000 7f3bacd47640 1 --2- 172.21.15.88:0/682842704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b90004380 0x7f3b90004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:33.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.647+0000 7f3bacd47640 1 -- 172.21.15.88:0/682842704 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b90003530 con 0x7f3b90005050 2023-10-16T20:06:33.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.651+0000 7f3bacd47640 1 -- 172.21.15.88:0/682842704 reap_dead start 2023-10-16T20:06:33.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.651+0000 7f3bacd47640 1 --2- 172.21.15.88:0/682842704 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b90005050 0x7f3b9000b990 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3ba80b1c30 tx=0x7f3ba001c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ac9b7ec57752e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:33.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.651+0000 7f3ba6ffd640 1 -- 172.21.15.88:0/682842704 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0003b80 con 0x7f3b90005050 2023-10-16T20:06:33.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.651+0000 7f3ba6ffd640 1 -- 172.21.15.88:0/682842704 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba0003ce0 con 0x7f3b90005050 2023-10-16T20:06:33.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.651+0000 7f3ba6ffd640 1 -- 172.21.15.88:0/682842704 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba00052f0 con 0x7f3b90005050 2023-10-16T20:06:33.661 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.651+0000 7f3bad548640 1 -- 172.21.15.88:0/682842704 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b90005050 msgr2=0x7f3b9000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:33.661 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.651+0000 7f3bad548640 1 --2- 172.21.15.88:0/682842704 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b90005050 0x7f3b9000b990 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3ba80b1c30 tx=0x7f3ba001c820 comp rx=0 tx=0).stop 2023-10-16T20:06:33.661 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.651+0000 7f3bad548640 1 -- 172.21.15.88:0/682842704 shutdown_connections 2023-10-16T20:06:33.661 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.651+0000 7f3bad548640 1 --2- 172.21.15.88:0/682842704 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b90005050 0x7f3b9000b990 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:33.661 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.651+0000 7f3bad548640 1 -- 172.21.15.88:0/682842704 >> 172.21.15.88:0/682842704 conn(0x7f3ba806a4c0 msgr2=0x7f3ba806c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:33.662 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.651+0000 7f3bad548640 1 -- 172.21.15.88:0/682842704 shutdown_connections 2023-10-16T20:06:33.662 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.651+0000 7f3bad548640 1 -- 172.21.15.88:0/682842704 wait complete. 2023-10-16T20:06:33.662 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.655+0000 7f3bad548640 1 Processor -- start 2023-10-16T20:06:33.662 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.655+0000 7f3bad548640 1 -- start start 2023-10-16T20:06:33.662 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.655+0000 7f3bad548640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b90005050 0x7f3ba814d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:33.663 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.655+0000 7f3bad548640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ba81dfa80 0x7f3ba80af770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:33.663 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.655+0000 7f3bad548640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ba81dfee0 0x7f3ba8155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:33.663 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.655+0000 7f3bad548640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba814dba0 con 0x7f3ba81dfee0 2023-10-16T20:06:33.663 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.655+0000 7f3bad548640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba80ae480 con 0x7f3b90005050 2023-10-16T20:06:33.663 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.655+0000 7f3bad548640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba80ae6d0 con 0x7f3ba81dfa80 2023-10-16T20:06:33.664 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.655+0000 7f3bacd47640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ba81dfee0 0x7f3ba8155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:33.664 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.655+0000 7f3bacd47640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ba81dfee0 0x7f3ba8155010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43126/0 (socket says 172.21.15.88:43126) 2023-10-16T20:06:33.664 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.655+0000 7f3bacd47640 1 -- 172.21.15.88:0/1832312644 learned_addr learned my addr 172.21.15.88:0/1832312644 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:33.664 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.655+0000 7f3ba7fff640 1 --2- 172.21.15.88:0/1832312644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b90005050 0x7f3ba814d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:33.664 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.655+0000 7f3ba77fe640 1 --2- 172.21.15.88:0/1832312644 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ba81dfa80 0x7f3ba80af770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:33.665 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.655+0000 7f3bacd47640 1 -- 172.21.15.88:0/1832312644 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ba81dfa80 msgr2=0x7f3ba80af770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:33.665 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.655+0000 7f3bacd47640 1 --2- 172.21.15.88:0/1832312644 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ba81dfa80 0x7f3ba80af770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:33.665 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.655+0000 7f3bacd47640 1 -- 172.21.15.88:0/1832312644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b90005050 msgr2=0x7f3ba814d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:33.665 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.655+0000 7f3bacd47640 1 --2- 172.21.15.88:0/1832312644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b90005050 0x7f3ba814d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:33.665 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.655+0000 7f3bacd47640 1 -- 172.21.15.88:0/1832312644 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba80ae9b0 con 0x7f3ba81dfee0 2023-10-16T20:06:33.666 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.655+0000 7f3ba7fff640 1 --2- 172.21.15.88:0/1832312644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b90005050 0x7f3ba814d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:33.666 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.655+0000 7f3bacd47640 1 --2- 172.21.15.88:0/1832312644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ba81dfee0 0x7f3ba8155010 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3ba001c650 tx=0x7f3b90005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:33.666 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.655+0000 7f3ba4ff9640 1 -- 172.21.15.88:0/1832312644 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0005850 con 0x7f3ba81dfee0 2023-10-16T20:06:33.666 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.655+0000 7f3ba4ff9640 1 -- 172.21.15.88:0/1832312644 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba00b4430 con 0x7f3ba81dfee0 2023-10-16T20:06:33.666 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.655+0000 7f3ba4ff9640 1 -- 172.21.15.88:0/1832312644 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba00b47c0 con 0x7f3ba81dfee0 2023-10-16T20:06:33.667 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.655+0000 7f3bad548640 1 -- 172.21.15.88:0/1832312644 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba80aec90 con 0x7f3ba81dfee0 2023-10-16T20:06:33.667 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.655+0000 7f3bad548640 1 -- 172.21.15.88:0/1832312644 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba80af250 con 0x7f3ba81dfee0 2023-10-16T20:06:33.667 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.659+0000 7f3bad548640 1 -- 172.21.15.88:0/1832312644 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba80af440 con 0x7f3ba81dfee0 2023-10-16T20:06:33.668 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.659+0000 7f3ba4ff9640 1 -- 172.21.15.88:0/1832312644 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3ba00a6410 con 0x7f3ba81dfee0 2023-10-16T20:06:33.668 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.659+0000 7f3ba4ff9640 1 --2- 172.21.15.88:0/1832312644 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3b940664a0 0x7f3b94068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:33.668 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.659+0000 7f3ba4ff9640 1 -- 172.21.15.88:0/1832312644 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f3ba00b0070 con 0x7f3ba81dfee0 2023-10-16T20:06:33.669 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.659+0000 7f3ba7fff640 1 --2- 172.21.15.88:0/1832312644 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3b940664a0 0x7f3b94068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:33.669 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.659+0000 7f3ba7fff640 1 --2- 172.21.15.88:0/1832312644 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3b940664a0 0x7f3b94068960 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3b98009b60 tx=0x7f3b98008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:33.669 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.663+0000 7f3ba4ff9640 1 -- 172.21.15.88:0/1832312644 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3ba00ea3b0 con 0x7f3ba81dfee0 2023-10-16T20:06:33.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.815+0000 7f3bad548640 1 -- 172.21.15.88:0/1832312644 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ba80674f0 con 0x7f3ba81dfee0 2023-10-16T20:06:33.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.815+0000 7f3ba4ff9640 1 -- 172.21.15.88:0/1832312644 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+18506 (secure 0 0 0) 0x7f3ba00ed9d0 con 0x7f3ba81dfee0 2023-10-16T20:06:33.818 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:06:33.818 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":44,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:06:33.228336+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T20:06:14.743500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"testcephfs1_data","create_time":"2023-10-16T20:06:16.091837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-10-16T20:06:31.184462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:06:33.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.819+0000 7f3bad548640 1 -- 172.21.15.88:0/1832312644 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3b940664a0 msgr2=0x7f3b94068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:33.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.819+0000 7f3bad548640 1 --2- 172.21.15.88:0/1832312644 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3b940664a0 0x7f3b94068960 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3b98009b60 tx=0x7f3b98008040 comp rx=0 tx=0).stop 2023-10-16T20:06:33.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.819+0000 7f3bad548640 1 -- 172.21.15.88:0/1832312644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ba81dfee0 msgr2=0x7f3ba8155010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:33.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.819+0000 7f3bad548640 1 --2- 172.21.15.88:0/1832312644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ba81dfee0 0x7f3ba8155010 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3ba001c650 tx=0x7f3b90005800 comp rx=0 tx=0).stop 2023-10-16T20:06:33.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.819+0000 7f3bad548640 1 -- 172.21.15.88:0/1832312644 shutdown_connections 2023-10-16T20:06:33.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.819+0000 7f3bad548640 1 --2- 172.21.15.88:0/1832312644 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3b940664a0 0x7f3b94068960 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:33.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.819+0000 7f3bad548640 1 --2- 172.21.15.88:0/1832312644 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3ba81dfa80 0x7f3ba80af770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:33.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.819+0000 7f3bad548640 1 --2- 172.21.15.88:0/1832312644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ba81dfee0 0x7f3ba8155010 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:33.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.819+0000 7f3bad548640 1 --2- 172.21.15.88:0/1832312644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b90005050 0x7f3ba814d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:33.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.819+0000 7f3bad548640 1 -- 172.21.15.88:0/1832312644 >> 172.21.15.88:0/1832312644 conn(0x7f3ba806a4c0 msgr2=0x7f3ba80b0180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:33.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.819+0000 7f3bad548640 1 -- 172.21.15.88:0/1832312644 shutdown_connections 2023-10-16T20:06:33.822 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.819+0000 7f3bad548640 1 -- 172.21.15.88:0/1832312644 wait complete. 2023-10-16T20:06:33.839 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'testcephfs1_metadata', 'create_time': '2023-10-16T20:06:14.743500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'testcephfs1_data', 'create_time': '2023-10-16T20:06:16.091837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2023-10-16T20:06:31.184462+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T20:06:33.839 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:06:33.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.979+0000 7f6f79bd9640 1 Processor -- start 2023-10-16T20:06:33.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.979+0000 7f6f79bd9640 1 -- start start 2023-10-16T20:06:33.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.979+0000 7f6f79bd9640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f6f740ce400 0x7f6f740ce870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:33.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.979+0000 7f6f79bd9640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f74158820 con 0x7f6f740cb980 2023-10-16T20:06:33.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.979+0000 7f6f79bd9640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f740cbd60 con 0x7f6f7415ae00 2023-10-16T20:06:33.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.979+0000 7f6f79bd9640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f7415bdd0 con 0x7f6f740ce400 2023-10-16T20:06:33.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.979+0000 7f6f737fe640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f6f740cb980 0x7f6f740b6250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:41454/0 (socket says 172.21.15.88:41454) 2023-10-16T20:06:33.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.979+0000 7f6f737fe640 1 -- 172.21.15.88:0/3606346706 learned_addr learned my addr 172.21.15.88:0/3606346706 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:33.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.979+0000 7f6f73fff640 1 --2- 172.21.15.88:0/3606346706 >> v2:172.21.15.88:3300/0 conn(0x7f6f740ce400 0x7f6f740ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:33.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.979+0000 7f6f727fc640 1 -- 172.21.15.88:0/3606346706 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1654759783 0 0) 0x7f6f74158820 con 0x7f6f740cb980 2023-10-16T20:06:33.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.979+0000 7f6f727fc640 1 -- 172.21.15.88:0/3606346706 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f5c003530 con 0x7f6f740cb980 2023-10-16T20:06:33.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.979+0000 7f6f73fff640 1 -- 172.21.15.88:0/3606346706 >> v1:172.21.15.94:6789/0 conn(0x7f6f7415ae00 legacy=0x7f6f7415eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:33.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.979+0000 7f6f73fff640 1 -- 172.21.15.88:0/3606346706 >> v1:172.21.15.88:6789/0 conn(0x7f6f740cb980 legacy=0x7f6f740b6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:33.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.979+0000 7f6f73fff640 1 -- 172.21.15.88:0/3606346706 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7415b270 con 0x7f6f740ce400 2023-10-16T20:06:33.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.983+0000 7f6f73fff640 1 --2- 172.21.15.88:0/3606346706 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f740ce400 0x7f6f740ce870 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c009d90 tx=0x7f6f6c01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dcaa186ad3bdac5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:33.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.983+0000 7f6f727fc640 1 -- 172.21.15.88:0/3606346706 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c0a5900 con 0x7f6f740ce400 2023-10-16T20:06:33.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.983+0000 7f6f727fc640 1 -- 172.21.15.88:0/3606346706 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6c0a5a60 con 0x7f6f740ce400 2023-10-16T20:06:33.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.983+0000 7f6f727fc640 1 -- 172.21.15.88:0/3606346706 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c0a5df0 con 0x7f6f740ce400 2023-10-16T20:06:33.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.983+0000 7f6f79bd9640 1 -- 172.21.15.88:0/3606346706 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f740ce400 msgr2=0x7f6f740ce870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:33.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.983+0000 7f6f79bd9640 1 --2- 172.21.15.88:0/3606346706 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f740ce400 0x7f6f740ce870 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c009d90 tx=0x7f6f6c01c950 comp rx=0 tx=0).stop 2023-10-16T20:06:33.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.983+0000 7f6f79bd9640 1 -- 172.21.15.88:0/3606346706 shutdown_connections 2023-10-16T20:06:33.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.983+0000 7f6f79bd9640 1 --2- 172.21.15.88:0/3606346706 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f740ce400 0x7f6f740ce870 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:33.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.983+0000 7f6f79bd9640 1 -- 172.21.15.88:0/3606346706 >> 172.21.15.88:0/3606346706 conn(0x7f6f740cd2b0 msgr2=0x7f6f740cd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:33.988 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.983+0000 7f6f79bd9640 1 -- 172.21.15.88:0/3606346706 shutdown_connections 2023-10-16T20:06:33.988 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.983+0000 7f6f79bd9640 1 -- 172.21.15.88:0/3606346706 wait complete. 2023-10-16T20:06:33.988 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.987+0000 7f6f79bd9640 1 Processor -- start 2023-10-16T20:06:33.989 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.987+0000 7f6f79bd9640 1 -- start start 2023-10-16T20:06:33.990 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.987+0000 7f6f79bd9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6f740cb980 0x7f6f740c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:33.990 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.987+0000 7f6f79bd9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f740ce400 0x7f6f74150d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:33.990 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.987+0000 7f6f79bd9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f7415ae00 0x7f6f740c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:33.991 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.987+0000 7f6f79bd9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f74151270 con 0x7f6f7415ae00 2023-10-16T20:06:33.991 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.987+0000 7f6f79bd9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f740c5d50 con 0x7f6f740ce400 2023-10-16T20:06:33.991 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.987+0000 7f6f79bd9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f740c6030 con 0x7f6f740cb980 2023-10-16T20:06:33.991 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.987+0000 7f6f737fe640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6f740cb980 0x7f6f740c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:33.991 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.987+0000 7f6f72ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f740ce400 0x7f6f74150d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:33.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.987+0000 7f6f737fe640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6f740cb980 0x7f6f740c6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:57378/0 (socket says 172.21.15.88:57378) 2023-10-16T20:06:33.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.987+0000 7f6f737fe640 1 -- 172.21.15.88:0/1480226530 learned_addr learned my addr 172.21.15.88:0/1480226530 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:33.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.987+0000 7f6f72ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f740ce400 0x7f6f74150d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:47648/0 (socket says 172.21.15.88:47648) 2023-10-16T20:06:33.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.987+0000 7f6f73fff640 1 --2- 172.21.15.88:0/1480226530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f7415ae00 0x7f6f740c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:33.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.987+0000 7f6f737fe640 1 -- 172.21.15.88:0/1480226530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f740ce400 msgr2=0x7f6f74150d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:33.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.987+0000 7f6f737fe640 1 --2- 172.21.15.88:0/1480226530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f740ce400 0x7f6f74150d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:33.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.987+0000 7f6f737fe640 1 -- 172.21.15.88:0/1480226530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f7415ae00 msgr2=0x7f6f740c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:33.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.987+0000 7f6f737fe640 1 --2- 172.21.15.88:0/1480226530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f7415ae00 0x7f6f740c8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:33.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.987+0000 7f6f737fe640 1 -- 172.21.15.88:0/1480226530 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f6c009a70 con 0x7f6f740cb980 2023-10-16T20:06:33.994 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.987+0000 7f6f737fe640 1 --2- 172.21.15.88:0/1480226530 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6f740cb980 0x7f6f740c6df0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6f6400c4d0 tx=0x7f6f6400c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:33.995 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.987+0000 7f6f5bfff640 1 -- 172.21.15.88:0/1480226530 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f64005cf0 con 0x7f6f740cb980 2023-10-16T20:06:33.995 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.987+0000 7f6f5bfff640 1 -- 172.21.15.88:0/1480226530 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f64005e50 con 0x7f6f740cb980 2023-10-16T20:06:33.995 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.987+0000 7f6f79bd9640 1 -- 172.21.15.88:0/1480226530 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f740c6310 con 0x7f6f740cb980 2023-10-16T20:06:33.995 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.987+0000 7f6f5bfff640 1 -- 172.21.15.88:0/1480226530 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f64007270 con 0x7f6f740cb980 2023-10-16T20:06:33.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.991+0000 7f6f79bd9640 1 -- 172.21.15.88:0/1480226530 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f740c68d0 con 0x7f6f740cb980 2023-10-16T20:06:33.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.991+0000 7f6f79bd9640 1 -- 172.21.15.88:0/1480226530 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f7415f1f0 con 0x7f6f740cb980 2023-10-16T20:06:33.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.991+0000 7f6f5bfff640 1 -- 172.21.15.88:0/1480226530 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6f640073d0 con 0x7f6f740cb980 2023-10-16T20:06:33.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.991+0000 7f6f5bfff640 1 --2- 172.21.15.88:0/1480226530 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f6f600664a0 0x7f6f60068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:33.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.991+0000 7f6f5bfff640 1 -- 172.21.15.88:0/1480226530 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f6f64083a00 con 0x7f6f740cb980 2023-10-16T20:06:33.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.991+0000 7f6f72ffd640 1 --2- 172.21.15.88:0/1480226530 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f6f600664a0 0x7f6f60068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:33.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.991+0000 7f6f72ffd640 1 --2- 172.21.15.88:0/1480226530 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f6f600664a0 0x7f6f60068960 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6f74154830 tx=0x7f6f68009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:33.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:33.995+0000 7f6f5bfff640 1 -- 172.21.15.88:0/1480226530 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6f64050d70 con 0x7f6f740cb980 2023-10-16T20:06:34.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.147+0000 7f6f79bd9640 1 -- 172.21.15.88:0/1480226530 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6f7415f1f0 con 0x7f6f740cb980 2023-10-16T20:06:34.149 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.147+0000 7f6f5bfff640 1 -- 172.21.15.88:0/1480226530 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6f7415f1f0 con 0x7f6f740cb980 2023-10-16T20:06:34.149 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:06:34.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.151+0000 7f6f79bd9640 1 -- 172.21.15.88:0/1480226530 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f6f600664a0 msgr2=0x7f6f60068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:34.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.151+0000 7f6f79bd9640 1 --2- 172.21.15.88:0/1480226530 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f6f600664a0 0x7f6f60068960 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6f74154830 tx=0x7f6f68009040 comp rx=0 tx=0).stop 2023-10-16T20:06:34.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.151+0000 7f6f79bd9640 1 -- 172.21.15.88:0/1480226530 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6f740cb980 msgr2=0x7f6f740c6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:34.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.151+0000 7f6f79bd9640 1 --2- 172.21.15.88:0/1480226530 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6f740cb980 0x7f6f740c6df0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6f6400c4d0 tx=0x7f6f6400c990 comp rx=0 tx=0).stop 2023-10-16T20:06:34.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.151+0000 7f6f79bd9640 1 -- 172.21.15.88:0/1480226530 shutdown_connections 2023-10-16T20:06:34.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.151+0000 7f6f79bd9640 1 --2- 172.21.15.88:0/1480226530 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f6f600664a0 0x7f6f60068960 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:34.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.151+0000 7f6f79bd9640 1 --2- 172.21.15.88:0/1480226530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f7415ae00 0x7f6f740c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:34.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.151+0000 7f6f79bd9640 1 --2- 172.21.15.88:0/1480226530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f740ce400 0x7f6f74150d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:34.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.151+0000 7f6f79bd9640 1 --2- 172.21.15.88:0/1480226530 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6f740cb980 0x7f6f740c6df0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:34.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.151+0000 7f6f79bd9640 1 -- 172.21.15.88:0/1480226530 >> 172.21.15.88:0/1480226530 conn(0x7f6f740cd2b0 msgr2=0x7f6f741538d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:34.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.151+0000 7f6f79bd9640 1 -- 172.21.15.88:0/1480226530 shutdown_connections 2023-10-16T20:06:34.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.151+0000 7f6f79bd9640 1 -- 172.21.15.88:0/1480226530 wait complete. 2023-10-16T20:06:34.171 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_metadata pg_num 2023-10-16T20:06:34.311 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.307+0000 7f90c7e6b640 1 Processor -- start 2023-10-16T20:06:34.311 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.311+0000 7f90c7e6b640 1 -- start start 2023-10-16T20:06:34.313 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.311+0000 7f90c7e6b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f90c00b4260 0x7f90c015dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:34.313 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.311+0000 7f90c7e6b640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f90c006d3f0 0x7f90c006d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:34.313 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.311+0000 7f90c7e6b640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90c015ad30 con 0x7f90c00b0da0 2023-10-16T20:06:34.313 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.311+0000 7f90c7e6b640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90c006ddc0 con 0x7f90c006d3f0 2023-10-16T20:06:34.314 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.311+0000 7f90c7e6b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c00b4640 con 0x7f90c00b4260 2023-10-16T20:06:34.314 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.311+0000 7f90c6408640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f90c006d3f0 0x7f90c006d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:34.314 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.311+0000 7f90c6408640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f90c006d3f0 0x7f90c006d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43150/0 (socket says 172.21.15.88:43150) 2023-10-16T20:06:34.314 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.311+0000 7f90c6408640 1 -- 172.21.15.88:0/2820406667 learned_addr learned my addr 172.21.15.88:0/2820406667 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:34.314 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.311+0000 7f90c5c07640 1 --2- 172.21.15.88:0/2820406667 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f90c00b4260 0x7f90c015dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:34.315 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.311+0000 7f90c6408640 1 -- 172.21.15.88:0/2820406667 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f90c00b4260 msgr2=0x7f90c015dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:34.315 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.311+0000 7f90c6408640 1 --2- 172.21.15.88:0/2820406667 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f90c00b4260 0x7f90c015dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:34.315 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.311+0000 7f90c6408640 1 -- 172.21.15.88:0/2820406667 >> v1:172.21.15.94:6789/0 conn(0x7f90c00b0da0 legacy=0x7f90c0159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:34.315 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.311+0000 7f90c6408640 1 -- 172.21.15.88:0/2820406667 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c015a770 con 0x7f90c006d3f0 2023-10-16T20:06:34.316 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.315+0000 7f90c6408640 1 --2- 172.21.15.88:0/2820406667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90c006d3f0 0x7f90c006d880 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009e50 tx=0x7f90b801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c244eb716fccfe server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:34.316 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.315+0000 7f90c4c05640 1 -- 172.21.15.88:0/2820406667 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b80a5810 con 0x7f90c006d3f0 2023-10-16T20:06:34.316 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.315+0000 7f90c4c05640 1 -- 172.21.15.88:0/2820406667 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b80a5970 con 0x7f90c006d3f0 2023-10-16T20:06:34.317 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.315+0000 7f90c4c05640 1 -- 172.21.15.88:0/2820406667 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b80a5d00 con 0x7f90c006d3f0 2023-10-16T20:06:34.317 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.315+0000 7f90c7e6b640 1 -- 172.21.15.88:0/2820406667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90c006d3f0 msgr2=0x7f90c006d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:34.318 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.315+0000 7f90c7e6b640 1 --2- 172.21.15.88:0/2820406667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90c006d3f0 0x7f90c006d880 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009e50 tx=0x7f90b801c950 comp rx=0 tx=0).stop 2023-10-16T20:06:34.318 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.315+0000 7f90c7e6b640 1 -- 172.21.15.88:0/2820406667 shutdown_connections 2023-10-16T20:06:34.318 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.315+0000 7f90c7e6b640 1 --2- 172.21.15.88:0/2820406667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90c006d3f0 0x7f90c006d880 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:34.318 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.315+0000 7f90c7e6b640 1 --2- 172.21.15.88:0/2820406667 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f90c00b4260 0x7f90c015dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:34.318 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.315+0000 7f90c7e6b640 1 -- 172.21.15.88:0/2820406667 >> 172.21.15.88:0/2820406667 conn(0x7f90c00cb080 msgr2=0x7f90c00cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:34.319 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.315+0000 7f90c7e6b640 1 -- 172.21.15.88:0/2820406667 shutdown_connections 2023-10-16T20:06:34.319 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.315+0000 7f90c7e6b640 1 -- 172.21.15.88:0/2820406667 wait complete. 2023-10-16T20:06:34.320 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.315+0000 7f90c7e6b640 1 Processor -- start 2023-10-16T20:06:34.320 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.315+0000 7f90c7e6b640 1 -- start start 2023-10-16T20:06:34.321 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.319+0000 7f90c7e6b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90c006d3f0 0x7f90c013eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:34.321 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.319+0000 7f90c7e6b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f90c00b0da0 0x7f90c01516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:34.321 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.319+0000 7f90c7e6b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90c00b4260 0x7f90c0151be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:34.321 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.319+0000 7f90c7e6b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c00ac7b0 con 0x7f90c006d3f0 2023-10-16T20:06:34.321 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.319+0000 7f90c7e6b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c014f8b0 con 0x7f90c00b4260 2023-10-16T20:06:34.322 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.319+0000 7f90c7e6b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c014f2f0 con 0x7f90c00b0da0 2023-10-16T20:06:34.322 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.319+0000 7f90c5c07640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90c006d3f0 0x7f90c013eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:34.322 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.319+0000 7f90c5c07640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90c006d3f0 0x7f90c013eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43164/0 (socket says 172.21.15.88:43164) 2023-10-16T20:06:34.322 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.319+0000 7f90c5c07640 1 -- 172.21.15.88:0/3781891109 learned_addr learned my addr 172.21.15.88:0/3781891109 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:34.322 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.319+0000 7f90c5406640 1 --2- 172.21.15.88:0/3781891109 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f90c00b0da0 0x7f90c01516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:34.323 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.319+0000 7f90c6408640 1 --2- 172.21.15.88:0/3781891109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90c00b4260 0x7f90c0151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:34.323 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.319+0000 7f90c5c07640 1 -- 172.21.15.88:0/3781891109 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f90c00b0da0 msgr2=0x7f90c01516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:34.323 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.319+0000 7f90c5c07640 1 --2- 172.21.15.88:0/3781891109 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f90c00b0da0 0x7f90c01516a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:34.323 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.319+0000 7f90c5c07640 1 -- 172.21.15.88:0/3781891109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90c00b4260 msgr2=0x7f90c0151be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:34.323 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.319+0000 7f90c5c07640 1 --2- 172.21.15.88:0/3781891109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90c00b4260 0x7f90c0151be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:34.324 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.319+0000 7f90c5c07640 1 -- 172.21.15.88:0/3781891109 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b8009b30 con 0x7f90c006d3f0 2023-10-16T20:06:34.325 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.319+0000 7f90c5c07640 1 --2- 172.21.15.88:0/3781891109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90c006d3f0 0x7f90c013eed0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00d800 tx=0x7f90bc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:34.325 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.319+0000 7f90b67fc640 1 -- 172.21.15.88:0/3781891109 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc00b6c0 con 0x7f90c006d3f0 2023-10-16T20:06:34.325 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.319+0000 7f90b67fc640 1 -- 172.21.15.88:0/3781891109 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90bc00b860 con 0x7f90c006d3f0 2023-10-16T20:06:34.326 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.319+0000 7f90c7e6b640 1 -- 172.21.15.88:0/3781891109 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c014f5d0 con 0x7f90c006d3f0 2023-10-16T20:06:34.326 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.319+0000 7f90b67fc640 1 -- 172.21.15.88:0/3781891109 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc00bc10 con 0x7f90c006d3f0 2023-10-16T20:06:34.326 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.319+0000 7f90c7e6b640 1 -- 172.21.15.88:0/3781891109 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90c013f410 con 0x7f90c006d3f0 2023-10-16T20:06:34.326 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.319+0000 7f90c7e6b640 1 -- 172.21.15.88:0/3781891109 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90c00ac8f0 con 0x7f90c006d3f0 2023-10-16T20:06:34.327 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.319+0000 7f90b67fc640 1 -- 172.21.15.88:0/3781891109 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f90bc01e460 con 0x7f90c006d3f0 2023-10-16T20:06:34.327 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.323+0000 7f90b67fc640 1 --2- 172.21.15.88:0/3781891109 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f90a40664a0 0x7f90a4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:34.327 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.323+0000 7f90b67fc640 1 -- 172.21.15.88:0/3781891109 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f90bc0845e0 con 0x7f90c006d3f0 2023-10-16T20:06:34.327 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.323+0000 7f90c5406640 1 --2- 172.21.15.88:0/3781891109 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f90a40664a0 0x7f90a4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:34.327 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.323+0000 7f90c5406640 1 --2- 172.21.15.88:0/3781891109 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f90a40664a0 0x7f90a4068960 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f90c00b23f0 tx=0x7f90b0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:34.328 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.327+0000 7f90b67fc640 1 -- 172.21.15.88:0/3781891109 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f90c00ac8f0 con 0x7f90c006d3f0 2023-10-16T20:06:34.476 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.475+0000 7f90c7e6b640 1 -- 172.21.15.88:0/3781891109 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"} v 0) v1 -- 0x7f90c00ac8f0 con 0x7f90c006d3f0 2023-10-16T20:06:34.476 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.475+0000 7f90b67fc640 1 -- 172.21.15.88:0/3781891109 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"}]=0 v44) v1 ==== 109+0+11 (secure 0 0 0) 0x7f90c00ac8f0 con 0x7f90c006d3f0 2023-10-16T20:06:34.477 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:06:34.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.475+0000 7f90c7e6b640 1 -- 172.21.15.88:0/3781891109 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f90a40664a0 msgr2=0x7f90a4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:34.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.475+0000 7f90c7e6b640 1 --2- 172.21.15.88:0/3781891109 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f90a40664a0 0x7f90a4068960 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f90c00b23f0 tx=0x7f90b0009040 comp rx=0 tx=0).stop 2023-10-16T20:06:34.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.475+0000 7f90c7e6b640 1 -- 172.21.15.88:0/3781891109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90c006d3f0 msgr2=0x7f90c013eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:34.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.475+0000 7f90c7e6b640 1 --2- 172.21.15.88:0/3781891109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90c006d3f0 0x7f90c013eed0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00d800 tx=0x7f90bc00dcc0 comp rx=0 tx=0).stop 2023-10-16T20:06:34.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.475+0000 7f90c7e6b640 1 -- 172.21.15.88:0/3781891109 shutdown_connections 2023-10-16T20:06:34.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.475+0000 7f90c7e6b640 1 --2- 172.21.15.88:0/3781891109 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f90a40664a0 0x7f90a4068960 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:34.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.475+0000 7f90c7e6b640 1 --2- 172.21.15.88:0/3781891109 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f90c00b0da0 0x7f90c01516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:34.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.475+0000 7f90c7e6b640 1 --2- 172.21.15.88:0/3781891109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90c00b4260 0x7f90c0151be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:34.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.475+0000 7f90c7e6b640 1 --2- 172.21.15.88:0/3781891109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90c006d3f0 0x7f90c013eed0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:34.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.475+0000 7f90c7e6b640 1 -- 172.21.15.88:0/3781891109 >> 172.21.15.88:0/3781891109 conn(0x7f90c00cb080 msgr2=0x7f90c00b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:34.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.475+0000 7f90c7e6b640 1 -- 172.21.15.88:0/3781891109 shutdown_connections 2023-10-16T20:06:34.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.475+0000 7f90c7e6b640 1 -- 172.21.15.88:0/3781891109 wait complete. 2023-10-16T20:06:34.498 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_data pg_num 2023-10-16T20:06:34.638 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.635+0000 7f59576c9640 1 Processor -- start 2023-10-16T20:06:34.638 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.635+0000 7f59576c9640 1 -- start start 2023-10-16T20:06:34.639 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f59576c9640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5950071a70 0x7f5950161540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:34.639 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f59576c9640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5950161a80 con 0x7f59500ca0b0 2023-10-16T20:06:34.639 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f59576c9640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59500ca840 con 0x7f595015cf90 2023-10-16T20:06:34.640 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f59576c9640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59500cab20 con 0x7f5950071a70 2023-10-16T20:06:34.640 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f5954c64640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f59500ca0b0 0x7f5950071360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:41464/0 (socket says 172.21.15.88:41464) 2023-10-16T20:06:34.640 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f5954c64640 1 -- 172.21.15.88:0/2133031799 learned_addr learned my addr 172.21.15.88:0/2133031799 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:34.642 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f5955c66640 1 --2- 172.21.15.88:0/2133031799 >> v2:172.21.15.94:3300/0 conn(0x7f5950071a70 0x7f5950161540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:34.642 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f5947fff640 1 -- 172.21.15.88:0/2133031799 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2219797039 0 0) 0x7f5950161a80 con 0x7f59500ca0b0 2023-10-16T20:06:34.642 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f5947fff640 1 -- 172.21.15.88:0/2133031799 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f593c003530 con 0x7f59500ca0b0 2023-10-16T20:06:34.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f5947fff640 1 -- 172.21.15.88:0/2133031799 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1302821522 0 0) 0x7f593c003530 con 0x7f59500ca0b0 2023-10-16T20:06:34.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f5947fff640 1 -- 172.21.15.88:0/2133031799 >> v2:172.21.15.94:3300/0 conn(0x7f5950071a70 msgr2=0x7f5950161540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:34.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f5947fff640 1 --2- 172.21.15.88:0/2133031799 >> v2:172.21.15.94:3300/0 conn(0x7f5950071a70 0x7f5950161540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:34.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f5947fff640 1 -- 172.21.15.88:0/2133031799 >> v1:172.21.15.94:6789/0 conn(0x7f595015cf90 legacy=0x7f59500cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:34.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f5947fff640 1 -- 172.21.15.88:0/2133031799 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59500cae00 con 0x7f59500ca0b0 2023-10-16T20:06:34.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f5947fff640 1 -- 172.21.15.88:0/2133031799 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f5940001680 con 0x7f59500ca0b0 2023-10-16T20:06:34.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f5947fff640 1 -- 172.21.15.88:0/2133031799 >> v1:172.21.15.88:6789/0 conn(0x7f59500ca0b0 legacy=0x7f5950071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:34.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f5947fff640 1 --2- 172.21.15.88:0/2133031799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f593c003980 0x7f593c003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:34.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f5947fff640 1 --2- 172.21.15.88:0/2133031799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f593c004380 0x7f593c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:34.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f5947fff640 1 --2- 172.21.15.88:0/2133031799 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f593c005050 0x7f593c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:34.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f5947fff640 1 -- 172.21.15.88:0/2133031799 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59500cab20 con 0x7f593c004380 2023-10-16T20:06:34.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f5947fff640 1 -- 172.21.15.88:0/2133031799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593c00bfb0 con 0x7f593c003980 2023-10-16T20:06:34.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f5947fff640 1 -- 172.21.15.88:0/2133031799 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f593c00c170 con 0x7f593c005050 2023-10-16T20:06:34.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f5954c64640 1 --2- 172.21.15.88:0/2133031799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f593c004380 0x7f593c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:34.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f5955c66640 1 --2- 172.21.15.88:0/2133031799 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f593c005050 0x7f593c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:34.646 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f5955465640 1 --2- 172.21.15.88:0/2133031799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f593c003980 0x7f593c003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:34.646 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f5954c64640 1 -- 172.21.15.88:0/2133031799 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f593c005050 msgr2=0x7f593c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:34.646 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f5954c64640 1 --2- 172.21.15.88:0/2133031799 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f593c005050 0x7f593c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:34.646 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f5954c64640 1 -- 172.21.15.88:0/2133031799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f593c003980 msgr2=0x7f593c003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:34.646 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f5954c64640 1 --2- 172.21.15.88:0/2133031799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f593c003980 0x7f593c003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:34.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f5954c64640 1 -- 172.21.15.88:0/2133031799 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f593c003530 con 0x7f593c004380 2023-10-16T20:06:34.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f5955c66640 1 --2- 172.21.15.88:0/2133031799 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f593c005050 0x7f593c00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:34.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f5955465640 1 --2- 172.21.15.88:0/2133031799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f593c003980 0x7f593c003d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:34.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.639+0000 7f5955c66640 1 -- 172.21.15.88:0/2133031799 reap_dead start 2023-10-16T20:06:34.651 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.643+0000 7f5954c64640 1 --2- 172.21.15.88:0/2133031799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f593c004380 0x7f593c004850 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f594000c190 tx=0x7f594001cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d8d9bcb81a991e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:34.651 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.643+0000 7f5947fff640 1 -- 172.21.15.88:0/2133031799 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5940005ae0 con 0x7f593c004380 2023-10-16T20:06:34.651 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.643+0000 7f5947fff640 1 -- 172.21.15.88:0/2133031799 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5940005c60 con 0x7f593c004380 2023-10-16T20:06:34.651 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.643+0000 7f5947fff640 1 -- 172.21.15.88:0/2133031799 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59400072b0 con 0x7f593c004380 2023-10-16T20:06:34.651 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.643+0000 7f59576c9640 1 -- 172.21.15.88:0/2133031799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f593c004380 msgr2=0x7f593c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:34.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.643+0000 7f59576c9640 1 --2- 172.21.15.88:0/2133031799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f593c004380 0x7f593c004850 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f594000c190 tx=0x7f594001cb40 comp rx=0 tx=0).stop 2023-10-16T20:06:34.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.643+0000 7f59576c9640 1 -- 172.21.15.88:0/2133031799 shutdown_connections 2023-10-16T20:06:34.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.643+0000 7f59576c9640 1 --2- 172.21.15.88:0/2133031799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f593c004380 0x7f593c004850 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:34.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.643+0000 7f59576c9640 1 -- 172.21.15.88:0/2133031799 >> 172.21.15.88:0/2133031799 conn(0x7f59500cf710 msgr2=0x7f59500cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:34.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.643+0000 7f59576c9640 1 -- 172.21.15.88:0/2133031799 shutdown_connections 2023-10-16T20:06:34.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.643+0000 7f59576c9640 1 -- 172.21.15.88:0/2133031799 wait complete. 2023-10-16T20:06:34.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.643+0000 7f59576c9640 1 Processor -- start 2023-10-16T20:06:34.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.643+0000 7f59576c9640 1 -- start start 2023-10-16T20:06:34.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.647+0000 7f59576c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f593c004380 0x7f5950155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:34.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.647+0000 7f59576c9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59501574d0 0x7f59501416d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:34.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.647+0000 7f59576c9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5950141c10 0x7f59500c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:34.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.647+0000 7f59576c9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5950142090 con 0x7f59501574d0 2023-10-16T20:06:34.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.647+0000 7f59576c9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5950157040 con 0x7f593c004380 2023-10-16T20:06:34.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.647+0000 7f59576c9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5950142560 con 0x7f5950141c10 2023-10-16T20:06:34.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.647+0000 7f5954c64640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59501574d0 0x7f59501416d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:34.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.647+0000 7f5954c64640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59501574d0 0x7f59501416d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43182/0 (socket says 172.21.15.88:43182) 2023-10-16T20:06:34.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.647+0000 7f5954c64640 1 -- 172.21.15.88:0/958656244 learned_addr learned my addr 172.21.15.88:0/958656244 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:34.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.647+0000 7f5955465640 1 --2- 172.21.15.88:0/958656244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f593c004380 0x7f5950155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:34.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.647+0000 7f5955c66640 1 --2- 172.21.15.88:0/958656244 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5950141c10 0x7f59500c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:34.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.647+0000 7f5954c64640 1 -- 172.21.15.88:0/958656244 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5950141c10 msgr2=0x7f59500c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:34.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.647+0000 7f5954c64640 1 --2- 172.21.15.88:0/958656244 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5950141c10 0x7f59500c8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:34.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.647+0000 7f5954c64640 1 -- 172.21.15.88:0/958656244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f593c004380 msgr2=0x7f5950155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:34.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.647+0000 7f5954c64640 1 --2- 172.21.15.88:0/958656244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f593c004380 0x7f5950155e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:34.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.647+0000 7f5954c64640 1 -- 172.21.15.88:0/958656244 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5950156430 con 0x7f59501574d0 2023-10-16T20:06:34.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.647+0000 7f5955c66640 1 --2- 172.21.15.88:0/958656244 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5950141c10 0x7f59500c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:34.657 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.647+0000 7f5955465640 1 --2- 172.21.15.88:0/958656244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f593c004380 0x7f5950155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:34.657 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.647+0000 7f5954c64640 1 --2- 172.21.15.88:0/958656244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59501574d0 0x7f59501416d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f594000c190 tx=0x7f593c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:34.657 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.647+0000 7f5945ffb640 1 -- 172.21.15.88:0/958656244 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59400b4070 con 0x7f59501574d0 2023-10-16T20:06:34.657 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.647+0000 7f5945ffb640 1 -- 172.21.15.88:0/958656244 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5940007070 con 0x7f59501574d0 2023-10-16T20:06:34.657 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.647+0000 7f59576c9640 1 -- 172.21.15.88:0/958656244 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5950156710 con 0x7f59501574d0 2023-10-16T20:06:34.658 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.647+0000 7f5945ffb640 1 -- 172.21.15.88:0/958656244 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59400073e0 con 0x7f59501574d0 2023-10-16T20:06:34.658 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.647+0000 7f59576c9640 1 -- 172.21.15.88:0/958656244 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5950156cd0 con 0x7f59501574d0 2023-10-16T20:06:34.658 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.647+0000 7f59576c9640 1 -- 172.21.15.88:0/958656244 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f595004e9c0 con 0x7f59501574d0 2023-10-16T20:06:34.658 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.647+0000 7f5945ffb640 1 -- 172.21.15.88:0/958656244 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5940007540 con 0x7f59501574d0 2023-10-16T20:06:34.659 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.651+0000 7f5945ffb640 1 --2- 172.21.15.88:0/958656244 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f59340664d0 0x7f5934068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:34.659 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.651+0000 7f5945ffb640 1 -- 172.21.15.88:0/958656244 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f594011de60 con 0x7f59501574d0 2023-10-16T20:06:34.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.651+0000 7f5955465640 1 --2- 172.21.15.88:0/958656244 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f59340664d0 0x7f5934068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:34.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.651+0000 7f5955465640 1 --2- 172.21.15.88:0/958656244 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f59340664d0 0x7f5934068990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f594c005630 tx=0x7f594c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:34.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.655+0000 7f5945ffb640 1 -- 172.21.15.88:0/958656244 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f59400eafe0 con 0x7f59501574d0 2023-10-16T20:06:34.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.803+0000 7f59576c9640 1 -- 172.21.15.88:0/958656244 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"} v 0) v1 -- 0x7f5950157180 con 0x7f59501574d0 2023-10-16T20:06:34.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.803+0000 7f5945ffb640 1 -- 172.21.15.88:0/958656244 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"}]=0 v44) v1 ==== 105+0+11 (secure 0 0 0) 0x7f59400ee600 con 0x7f59501574d0 2023-10-16T20:06:34.806 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2023-10-16T20:06:34.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.807+0000 7f59576c9640 1 -- 172.21.15.88:0/958656244 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f59340664d0 msgr2=0x7f5934068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:34.808 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.807+0000 7f59576c9640 1 --2- 172.21.15.88:0/958656244 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f59340664d0 0x7f5934068990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f594c005630 tx=0x7f594c009040 comp rx=0 tx=0).stop 2023-10-16T20:06:34.808 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.807+0000 7f59576c9640 1 -- 172.21.15.88:0/958656244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59501574d0 msgr2=0x7f59501416d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:34.808 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.807+0000 7f59576c9640 1 --2- 172.21.15.88:0/958656244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59501574d0 0x7f59501416d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f594000c190 tx=0x7f593c005800 comp rx=0 tx=0).stop 2023-10-16T20:06:34.808 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.807+0000 7f59576c9640 1 -- 172.21.15.88:0/958656244 shutdown_connections 2023-10-16T20:06:34.809 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.807+0000 7f59576c9640 1 --2- 172.21.15.88:0/958656244 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f59340664d0 0x7f5934068990 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:34.809 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.807+0000 7f59576c9640 1 --2- 172.21.15.88:0/958656244 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5950141c10 0x7f59500c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:34.809 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.807+0000 7f59576c9640 1 --2- 172.21.15.88:0/958656244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59501574d0 0x7f59501416d0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:34.809 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.807+0000 7f59576c9640 1 --2- 172.21.15.88:0/958656244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f593c004380 0x7f5950155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:34.809 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.807+0000 7f59576c9640 1 -- 172.21.15.88:0/958656244 >> 172.21.15.88:0/958656244 conn(0x7f59500cf710 msgr2=0x7f595006b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:34.810 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.807+0000 7f59576c9640 1 -- 172.21.15.88:0/958656244 shutdown_connections 2023-10-16T20:06:34.810 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.807+0000 7f59576c9640 1 -- 172.21.15.88:0/958656244 wait complete. 2023-10-16T20:06:34.826 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-10-16T20:06:34.969 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.967+0000 7f53e1b1d640 1 Processor -- start 2023-10-16T20:06:34.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.967+0000 7f53e1b1d640 1 -- start start 2023-10-16T20:06:34.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.967+0000 7f53e1b1d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f53dc071a70 0x7f53dc161540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:34.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.967+0000 7f53e1b1d640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53dc161a80 con 0x7f53dc15cf90 2023-10-16T20:06:34.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.967+0000 7f53e1b1d640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53dc0ca840 con 0x7f53dc0ca0b0 2023-10-16T20:06:34.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.967+0000 7f53e1b1d640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53dc0cab20 con 0x7f53dc071a70 2023-10-16T20:06:34.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.967+0000 7f53db7fe640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f53dc15cf90 0x7f53dc0cb8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:41466/0 (socket says 172.21.15.88:41466) 2023-10-16T20:06:34.972 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.967+0000 7f53db7fe640 1 -- 172.21.15.88:0/1622227280 learned_addr learned my addr 172.21.15.88:0/1622227280 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:34.972 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.967+0000 7f53dbfff640 1 --2- 172.21.15.88:0/1622227280 >> v2:172.21.15.88:3300/0 conn(0x7f53dc071a70 0x7f53dc161540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:34.972 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.971+0000 7f53da7fc640 1 -- 172.21.15.88:0/1622227280 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2291348255 0 0) 0x7f53dc161a80 con 0x7f53dc15cf90 2023-10-16T20:06:34.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.971+0000 7f53dbfff640 1 -- 172.21.15.88:0/1622227280 >> v1:172.21.15.94:6789/0 conn(0x7f53dc0ca0b0 legacy=0x7f53dc071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:34.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.971+0000 7f53dbfff640 1 -- 172.21.15.88:0/1622227280 >> v1:172.21.15.88:6789/0 conn(0x7f53dc15cf90 legacy=0x7f53dc0cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:34.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.971+0000 7f53dbfff640 1 -- 172.21.15.88:0/1622227280 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53dc0cae00 con 0x7f53dc071a70 2023-10-16T20:06:34.974 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.971+0000 7f53dbfff640 1 --2- 172.21.15.88:0/1622227280 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53dc071a70 0x7f53dc161540 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f53d4009d90 tx=0x7f53d401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60aaa202423b85fa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:34.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.971+0000 7f53da7fc640 1 -- 172.21.15.88:0/1622227280 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d40a5900 con 0x7f53dc071a70 2023-10-16T20:06:34.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.971+0000 7f53da7fc640 1 -- 172.21.15.88:0/1622227280 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d40a5a60 con 0x7f53dc071a70 2023-10-16T20:06:34.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.971+0000 7f53da7fc640 1 -- 172.21.15.88:0/1622227280 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d40a5df0 con 0x7f53dc071a70 2023-10-16T20:06:34.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.971+0000 7f53e1b1d640 1 -- 172.21.15.88:0/1622227280 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53dc071a70 msgr2=0x7f53dc161540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:34.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.971+0000 7f53e1b1d640 1 --2- 172.21.15.88:0/1622227280 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53dc071a70 0x7f53dc161540 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f53d4009d90 tx=0x7f53d401c950 comp rx=0 tx=0).stop 2023-10-16T20:06:34.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.975+0000 7f53e1b1d640 1 -- 172.21.15.88:0/1622227280 shutdown_connections 2023-10-16T20:06:34.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.975+0000 7f53e1b1d640 1 --2- 172.21.15.88:0/1622227280 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53dc071a70 0x7f53dc161540 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:34.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.975+0000 7f53e1b1d640 1 -- 172.21.15.88:0/1622227280 >> 172.21.15.88:0/1622227280 conn(0x7f53dc0cf710 msgr2=0x7f53dc0cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:34.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.975+0000 7f53e1b1d640 1 -- 172.21.15.88:0/1622227280 shutdown_connections 2023-10-16T20:06:34.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.975+0000 7f53e1b1d640 1 -- 172.21.15.88:0/1622227280 wait complete. 2023-10-16T20:06:34.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.975+0000 7f53e1b1d640 1 Processor -- start 2023-10-16T20:06:34.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.975+0000 7f53e1b1d640 1 -- start start 2023-10-16T20:06:34.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.975+0000 7f53e1b1d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53dc071a70 0x7f53dc14a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:34.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.975+0000 7f53e1b1d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53dc0ca0b0 0x7f53dc1520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:34.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.975+0000 7f53e1b1d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53dc15cf90 0x7f53dc159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:34.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.975+0000 7f53e1b1d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53dc14ac00 con 0x7f53dc0ca0b0 2023-10-16T20:06:34.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.975+0000 7f53e1b1d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53dc152eb0 con 0x7f53dc071a70 2023-10-16T20:06:34.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.975+0000 7f53e1b1d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53dc0b39f0 con 0x7f53dc15cf90 2023-10-16T20:06:34.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.975+0000 7f53daffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53dc0ca0b0 0x7f53dc1520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:34.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.975+0000 7f53daffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53dc0ca0b0 0x7f53dc1520a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43194/0 (socket says 172.21.15.88:43194) 2023-10-16T20:06:34.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.975+0000 7f53daffd640 1 -- 172.21.15.88:0/3349683444 learned_addr learned my addr 172.21.15.88:0/3349683444 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:34.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.975+0000 7f53db7fe640 1 --2- 172.21.15.88:0/3349683444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53dc071a70 0x7f53dc14a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:34.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.975+0000 7f53dbfff640 1 --2- 172.21.15.88:0/3349683444 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53dc15cf90 0x7f53dc159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:34.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.975+0000 7f53daffd640 1 -- 172.21.15.88:0/3349683444 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53dc15cf90 msgr2=0x7f53dc159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:34.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.975+0000 7f53daffd640 1 --2- 172.21.15.88:0/3349683444 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53dc15cf90 0x7f53dc159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:34.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.975+0000 7f53daffd640 1 -- 172.21.15.88:0/3349683444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53dc071a70 msgr2=0x7f53dc14a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:34.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.975+0000 7f53daffd640 1 --2- 172.21.15.88:0/3349683444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53dc071a70 0x7f53dc14a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:34.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.975+0000 7f53daffd640 1 -- 172.21.15.88:0/3349683444 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d4009a70 con 0x7f53dc0ca0b0 2023-10-16T20:06:34.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.975+0000 7f53daffd640 1 --2- 172.21.15.88:0/3349683444 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53dc0ca0b0 0x7f53dc1520a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f53d000c530 tx=0x7f53d000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:34.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.979+0000 7f53c3fff640 1 -- 172.21.15.88:0/3349683444 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d0005220 con 0x7f53dc0ca0b0 2023-10-16T20:06:34.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.979+0000 7f53c3fff640 1 -- 172.21.15.88:0/3349683444 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d0005380 con 0x7f53dc0ca0b0 2023-10-16T20:06:34.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.979+0000 7f53c3fff640 1 -- 172.21.15.88:0/3349683444 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d0007270 con 0x7f53dc0ca0b0 2023-10-16T20:06:34.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.979+0000 7f53e1b1d640 1 -- 172.21.15.88:0/3349683444 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53dc1528f0 con 0x7f53dc0ca0b0 2023-10-16T20:06:34.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.979+0000 7f53e1b1d640 1 -- 172.21.15.88:0/3349683444 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53dc0b3680 con 0x7f53dc0ca0b0 2023-10-16T20:06:34.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.979+0000 7f53e1b1d640 1 -- 172.21.15.88:0/3349683444 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53dc152640 con 0x7f53dc0ca0b0 2023-10-16T20:06:34.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.979+0000 7f53c3fff640 1 -- 172.21.15.88:0/3349683444 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f53d00073d0 con 0x7f53dc0ca0b0 2023-10-16T20:06:34.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.979+0000 7f53c3fff640 1 --2- 172.21.15.88:0/3349683444 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f53c80664a0 0x7f53c8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:34.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.979+0000 7f53c3fff640 1 -- 172.21.15.88:0/3349683444 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f53d0083840 con 0x7f53dc0ca0b0 2023-10-16T20:06:34.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.979+0000 7f53db7fe640 1 --2- 172.21.15.88:0/3349683444 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f53c80664a0 0x7f53c8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:34.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.979+0000 7f53db7fe640 1 --2- 172.21.15.88:0/3349683444 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f53c80664a0 0x7f53c8068960 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f53cc007bc0 tx=0x7f53cc007b10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:34.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:34.983+0000 7f53c3fff640 1 -- 172.21.15.88:0/3349683444 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f53d0050b10 con 0x7f53dc0ca0b0 2023-10-16T20:06:35.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.131+0000 7f53e1b1d640 1 -- 172.21.15.88:0/3349683444 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f53dc04e9c0 con 0x7f53dc0ca0b0 2023-10-16T20:06:35.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.135+0000 7f53c3fff640 1 -- 172.21.15.88:0/3349683444 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v44) v1 ==== 97+0+11 (secure 0 0 0) 0x7f53d0054130 con 0x7f53dc0ca0b0 2023-10-16T20:06:35.136 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:06:35.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.135+0000 7f53e1b1d640 1 -- 172.21.15.88:0/3349683444 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f53c80664a0 msgr2=0x7f53c8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:35.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.135+0000 7f53e1b1d640 1 --2- 172.21.15.88:0/3349683444 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f53c80664a0 0x7f53c8068960 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f53cc007bc0 tx=0x7f53cc007b10 comp rx=0 tx=0).stop 2023-10-16T20:06:35.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.135+0000 7f53e1b1d640 1 -- 172.21.15.88:0/3349683444 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53dc0ca0b0 msgr2=0x7f53dc1520a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:35.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.135+0000 7f53e1b1d640 1 --2- 172.21.15.88:0/3349683444 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53dc0ca0b0 0x7f53dc1520a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f53d000c530 tx=0x7f53d000c9f0 comp rx=0 tx=0).stop 2023-10-16T20:06:35.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.135+0000 7f53e1b1d640 1 -- 172.21.15.88:0/3349683444 shutdown_connections 2023-10-16T20:06:35.138 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.135+0000 7f53e1b1d640 1 --2- 172.21.15.88:0/3349683444 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f53c80664a0 0x7f53c8068960 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:35.139 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.135+0000 7f53e1b1d640 1 --2- 172.21.15.88:0/3349683444 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53dc15cf90 0x7f53dc159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:35.139 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.135+0000 7f53e1b1d640 1 --2- 172.21.15.88:0/3349683444 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53dc0ca0b0 0x7f53dc1520a0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:35.139 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.135+0000 7f53e1b1d640 1 --2- 172.21.15.88:0/3349683444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53dc071a70 0x7f53dc14a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:35.139 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.135+0000 7f53e1b1d640 1 -- 172.21.15.88:0/3349683444 >> 172.21.15.88:0/3349683444 conn(0x7f53dc0cf710 msgr2=0x7f53dc0cb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:35.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.135+0000 7f53e1b1d640 1 -- 172.21.15.88:0/3349683444 shutdown_connections 2023-10-16T20:06:35.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.135+0000 7f53e1b1d640 1 -- 172.21.15.88:0/3349683444 wait complete. 2023-10-16T20:06:35.157 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T20:06:35.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.295+0000 7fdc4359e640 1 Processor -- start 2023-10-16T20:06:35.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.295+0000 7fdc4359e640 1 -- start start 2023-10-16T20:06:35.298 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.295+0000 7fdc4359e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdc44160180 0x7fdc440cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:35.298 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.295+0000 7fdc4359e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fdc440d12d0 0x7fdc440cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:35.298 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.295+0000 7fdc4359e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc440b3df0 con 0x7fdc44160560 2023-10-16T20:06:35.299 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.295+0000 7fdc4359e640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc440b4280 con 0x7fdc440d12d0 2023-10-16T20:06:35.299 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.295+0000 7fdc4359e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc440d0220 con 0x7fdc44160180 2023-10-16T20:06:35.299 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.295+0000 7fdc4259c640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdc44160180 0x7fdc440cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:35.299 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.295+0000 7fdc4259c640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdc44160180 0x7fdc440cfce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:57434/0 (socket says 172.21.15.88:57434) 2023-10-16T20:06:35.299 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.295+0000 7fdc4259c640 1 -- 172.21.15.88:0/4288400469 learned_addr learned my addr 172.21.15.88:0/4288400469 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:35.300 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.295+0000 7fdc42d9d640 1 --2- 172.21.15.88:0/4288400469 >> v2:172.21.15.94:3300/0 conn(0x7fdc440d12d0 0x7fdc440cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:35.300 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.295+0000 7fdc4259c640 1 -- 172.21.15.88:0/4288400469 >> v2:172.21.15.94:3300/0 conn(0x7fdc440d12d0 msgr2=0x7fdc440cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:35.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.295+0000 7fdc4259c640 1 --2- 172.21.15.88:0/4288400469 >> v2:172.21.15.94:3300/0 conn(0x7fdc440d12d0 0x7fdc440cd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:35.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.295+0000 7fdc4259c640 1 -- 172.21.15.88:0/4288400469 >> v1:172.21.15.94:6789/0 conn(0x7fdc44160560 legacy=0x7fdc440d0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:35.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.295+0000 7fdc4259c640 1 -- 172.21.15.88:0/4288400469 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc440b3af0 con 0x7fdc44160180 2023-10-16T20:06:35.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.299+0000 7fdc4259c640 1 --2- 172.21.15.88:0/4288400469 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdc44160180 0x7fdc440cfce0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdc34009870 tx=0x7fdc3401c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a59b96977d0bc55b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:35.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.299+0000 7fdc4159a640 1 -- 172.21.15.88:0/4288400469 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc340a6040 con 0x7fdc44160180 2023-10-16T20:06:35.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.299+0000 7fdc4159a640 1 -- 172.21.15.88:0/4288400469 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc340aa600 con 0x7fdc44160180 2023-10-16T20:06:35.304 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.299+0000 7fdc4159a640 1 -- 172.21.15.88:0/4288400469 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc340aa990 con 0x7fdc44160180 2023-10-16T20:06:35.304 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.299+0000 7fdc4359e640 1 -- 172.21.15.88:0/4288400469 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdc44160180 msgr2=0x7fdc440cfce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:35.304 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.299+0000 7fdc4359e640 1 --2- 172.21.15.88:0/4288400469 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdc44160180 0x7fdc440cfce0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdc34009870 tx=0x7fdc3401c480 comp rx=0 tx=0).stop 2023-10-16T20:06:35.304 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.299+0000 7fdc4359e640 1 -- 172.21.15.88:0/4288400469 shutdown_connections 2023-10-16T20:06:35.304 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.299+0000 7fdc4359e640 1 --2- 172.21.15.88:0/4288400469 >> v2:172.21.15.94:3300/0 conn(0x7fdc440d12d0 0x7fdc440cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:35.305 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.299+0000 7fdc4359e640 1 --2- 172.21.15.88:0/4288400469 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdc44160180 0x7fdc440cfce0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:35.305 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.299+0000 7fdc4359e640 1 -- 172.21.15.88:0/4288400469 >> 172.21.15.88:0/4288400469 conn(0x7fdc44089fb0 msgr2=0x7fdc440d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:35.305 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.299+0000 7fdc4359e640 1 -- 172.21.15.88:0/4288400469 shutdown_connections 2023-10-16T20:06:35.305 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.299+0000 7fdc4359e640 1 -- 172.21.15.88:0/4288400469 wait complete. 2023-10-16T20:06:35.308 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.303+0000 7fdc4359e640 1 Processor -- start 2023-10-16T20:06:35.308 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.303+0000 7fdc4359e640 1 -- start start 2023-10-16T20:06:35.308 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.303+0000 7fdc4359e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdc440d12d0 0x7fdc44152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:35.308 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.303+0000 7fdc4359e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc44160180 0x7fdc44154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:35.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.303+0000 7fdc4359e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc44160560 0x7fdc4415ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:35.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.303+0000 7fdc4359e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc44156380 con 0x7fdc44160560 2023-10-16T20:06:35.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.303+0000 7fdc4359e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc44156660 con 0x7fdc44160180 2023-10-16T20:06:35.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.303+0000 7fdc4359e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc44156940 con 0x7fdc440d12d0 2023-10-16T20:06:35.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.303+0000 7fdc42d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc44160560 0x7fdc4415ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:35.310 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.303+0000 7fdc42d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc44160560 0x7fdc4415ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43196/0 (socket says 172.21.15.88:43196) 2023-10-16T20:06:35.310 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.303+0000 7fdc42d9d640 1 -- 172.21.15.88:0/3749499515 learned_addr learned my addr 172.21.15.88:0/3749499515 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:35.310 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.303+0000 7fdc4259c640 1 --2- 172.21.15.88:0/3749499515 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdc440d12d0 0x7fdc44152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:35.310 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.303+0000 7fdc41d9b640 1 --2- 172.21.15.88:0/3749499515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc44160180 0x7fdc44154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:35.310 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.303+0000 7fdc42d9d640 1 -- 172.21.15.88:0/3749499515 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdc440d12d0 msgr2=0x7fdc44152d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:35.311 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.303+0000 7fdc42d9d640 1 --2- 172.21.15.88:0/3749499515 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdc440d12d0 0x7fdc44152d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:35.311 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.303+0000 7fdc42d9d640 1 -- 172.21.15.88:0/3749499515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc44160180 msgr2=0x7fdc44154c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:35.311 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.303+0000 7fdc42d9d640 1 --2- 172.21.15.88:0/3749499515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc44160180 0x7fdc44154c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:35.311 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.303+0000 7fdc42d9d640 1 -- 172.21.15.88:0/3749499515 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc340094d0 con 0x7fdc44160560 2023-10-16T20:06:35.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.303+0000 7fdc42d9d640 1 --2- 172.21.15.88:0/3749499515 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc44160560 0x7fdc4415ec40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c00ef90 tx=0x7fdc3c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:35.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.303+0000 7fdc32ffd640 1 -- 172.21.15.88:0/3749499515 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c009ca0 con 0x7fdc44160560 2023-10-16T20:06:35.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.303+0000 7fdc32ffd640 1 -- 172.21.15.88:0/3749499515 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc3c00f040 con 0x7fdc44160560 2023-10-16T20:06:35.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.303+0000 7fdc4359e640 1 -- 172.21.15.88:0/3749499515 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc44156c20 con 0x7fdc44160560 2023-10-16T20:06:35.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.303+0000 7fdc32ffd640 1 -- 172.21.15.88:0/3749499515 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c0146c0 con 0x7fdc44160560 2023-10-16T20:06:35.313 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.303+0000 7fdc4359e640 1 -- 172.21.15.88:0/3749499515 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc4415f180 con 0x7fdc44160560 2023-10-16T20:06:35.313 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.307+0000 7fdc4359e640 1 -- 172.21.15.88:0/3749499515 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc440674f0 con 0x7fdc44160560 2023-10-16T20:06:35.314 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.307+0000 7fdc32ffd640 1 -- 172.21.15.88:0/3749499515 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdc3c014820 con 0x7fdc44160560 2023-10-16T20:06:35.314 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.307+0000 7fdc32ffd640 1 --2- 172.21.15.88:0/3749499515 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fdc240664a0 0x7fdc24068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:35.314 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.307+0000 7fdc32ffd640 1 -- 172.21.15.88:0/3749499515 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6449+0+0 (secure 0 0 0) 0x7fdc3c084fe0 con 0x7fdc44160560 2023-10-16T20:06:35.314 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.307+0000 7fdc4259c640 1 --2- 172.21.15.88:0/3749499515 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fdc240664a0 0x7fdc24068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:35.315 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.307+0000 7fdc4259c640 1 --2- 172.21.15.88:0/3749499515 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fdc240664a0 0x7fdc24068960 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdc340ae000 tx=0x7fdc340b5040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:35.315 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.311+0000 7fdc32ffd640 1 -- 172.21.15.88:0/3749499515 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdc3c052180 con 0x7fdc44160560 2023-10-16T20:06:35.475 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.471+0000 7fdc4359e640 1 -- 172.21.15.88:0/3749499515 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdc441570d0 con 0x7fdc44160560 2023-10-16T20:06:35.475 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.475+0000 7fdc32ffd640 1 -- 172.21.15.88:0/3749499515 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v44) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdc3c018080 con 0x7fdc44160560 2023-10-16T20:06:35.476 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2023-10-16T20:06:35.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.475+0000 7fdc4359e640 1 -- 172.21.15.88:0/3749499515 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fdc240664a0 msgr2=0x7fdc24068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:35.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.475+0000 7fdc4359e640 1 --2- 172.21.15.88:0/3749499515 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fdc240664a0 0x7fdc24068960 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdc340ae000 tx=0x7fdc340b5040 comp rx=0 tx=0).stop 2023-10-16T20:06:35.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.475+0000 7fdc4359e640 1 -- 172.21.15.88:0/3749499515 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc44160560 msgr2=0x7fdc4415ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:35.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.475+0000 7fdc4359e640 1 --2- 172.21.15.88:0/3749499515 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc44160560 0x7fdc4415ec40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c00ef90 tx=0x7fdc3c00c490 comp rx=0 tx=0).stop 2023-10-16T20:06:35.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.475+0000 7fdc4359e640 1 -- 172.21.15.88:0/3749499515 shutdown_connections 2023-10-16T20:06:35.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.475+0000 7fdc4359e640 1 --2- 172.21.15.88:0/3749499515 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fdc240664a0 0x7fdc24068960 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:35.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.475+0000 7fdc4359e640 1 --2- 172.21.15.88:0/3749499515 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc44160560 0x7fdc4415ec40 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:35.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.475+0000 7fdc4359e640 1 --2- 172.21.15.88:0/3749499515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc44160180 0x7fdc44154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:35.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.475+0000 7fdc4359e640 1 --2- 172.21.15.88:0/3749499515 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdc440d12d0 0x7fdc44152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:35.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.475+0000 7fdc4359e640 1 -- 172.21.15.88:0/3749499515 >> 172.21.15.88:0/3749499515 conn(0x7fdc44089fb0 msgr2=0x7fdc440b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:35.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.475+0000 7fdc4359e640 1 -- 172.21.15.88:0/3749499515 shutdown_connections 2023-10-16T20:06:35.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.475+0000 7fdc4359e640 1 -- 172.21.15.88:0/3749499515 wait complete. 2023-10-16T20:06:35.496 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:06:35.496 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:06:35.507 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:06:35.687 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.683+0000 7f063586b640 1 Processor -- start 2023-10-16T20:06:35.687 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.687+0000 7f063586b640 1 -- start start 2023-10-16T20:06:35.688 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.687+0000 7f063586b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f063006b100 0x7f063006c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:35.688 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.687+0000 7f063586b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f063006b4e0 0x7f06300c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:35.689 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.687+0000 7f063586b640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06300ae720 con 0x7f06300715c0 2023-10-16T20:06:35.689 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.687+0000 7f063586b640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06300c9a10 con 0x7f063006b4e0 2023-10-16T20:06:35.689 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.687+0000 7f063586b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f063006d650 con 0x7f063006b100 2023-10-16T20:06:35.689 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.687+0000 7f06267fc640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f06300715c0 0x7f063006c8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:41472/0 (socket says 172.21.15.88:41472) 2023-10-16T20:06:35.689 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.687+0000 7f06267fc640 1 -- 172.21.15.88:0/285628904 learned_addr learned my addr 172.21.15.88:0/285628904 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:35.690 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.687+0000 7f062f7fe640 1 --2- 172.21.15.88:0/285628904 >> v2:172.21.15.94:3300/0 conn(0x7f063006b4e0 0x7f06300c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:35.690 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.687+0000 7f062effd640 1 --2- 172.21.15.88:0/285628904 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f063006b100 0x7f063006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:35.691 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.687+0000 7f062e7fc640 1 -- 172.21.15.88:0/285628904 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1866218137 0 0) 0x7f06180014a0 con 0x7f06300715c0 2023-10-16T20:06:35.691 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.687+0000 7f062e7fc640 1 -- 172.21.15.88:0/285628904 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f061c003530 con 0x7f06300715c0 2023-10-16T20:06:35.691 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.687+0000 7f062f7fe640 1 -- 172.21.15.88:0/285628904 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f063006b100 msgr2=0x7f063006c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:35.692 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.687+0000 7f062f7fe640 1 --2- 172.21.15.88:0/285628904 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f063006b100 0x7f063006c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:35.692 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.687+0000 7f062f7fe640 1 -- 172.21.15.88:0/285628904 >> v1:172.21.15.88:6789/0 conn(0x7f06300715c0 legacy=0x7f063006c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:35.692 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.687+0000 7f062f7fe640 1 -- 172.21.15.88:0/285628904 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06300c9420 con 0x7f063006b4e0 2023-10-16T20:06:35.693 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.691+0000 7f062f7fe640 1 --2- 172.21.15.88:0/285628904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f063006b4e0 0x7f06300c9ea0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f06280097c0 tx=0x7f062801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d69f68f69ccde764 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:35.693 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.691+0000 7f062e7fc640 1 -- 172.21.15.88:0/285628904 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06280ab680 con 0x7f063006b4e0 2023-10-16T20:06:35.694 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.691+0000 7f062e7fc640 1 -- 172.21.15.88:0/285628904 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06280ab7e0 con 0x7f063006b4e0 2023-10-16T20:06:35.694 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.691+0000 7f062e7fc640 1 -- 172.21.15.88:0/285628904 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06280abb70 con 0x7f063006b4e0 2023-10-16T20:06:35.694 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.691+0000 7f063586b640 1 -- 172.21.15.88:0/285628904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f063006b4e0 msgr2=0x7f06300c9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:35.694 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.691+0000 7f063586b640 1 --2- 172.21.15.88:0/285628904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f063006b4e0 0x7f06300c9ea0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f06280097c0 tx=0x7f062801c480 comp rx=0 tx=0).stop 2023-10-16T20:06:35.694 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.691+0000 7f063586b640 1 -- 172.21.15.88:0/285628904 shutdown_connections 2023-10-16T20:06:35.695 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.691+0000 7f063586b640 1 --2- 172.21.15.88:0/285628904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f063006b4e0 0x7f06300c9ea0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:35.695 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.691+0000 7f063586b640 1 --2- 172.21.15.88:0/285628904 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f063006b100 0x7f063006c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:35.695 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.691+0000 7f063586b640 1 -- 172.21.15.88:0/285628904 >> 172.21.15.88:0/285628904 conn(0x7f0630070aa0 msgr2=0x7f0630070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:35.695 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.691+0000 7f063586b640 1 -- 172.21.15.88:0/285628904 shutdown_connections 2023-10-16T20:06:35.695 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.691+0000 7f063586b640 1 -- 172.21.15.88:0/285628904 wait complete. 2023-10-16T20:06:35.697 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.691+0000 7f063586b640 1 Processor -- start 2023-10-16T20:06:35.697 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.695+0000 7f063586b640 1 -- start start 2023-10-16T20:06:35.698 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.695+0000 7f063586b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f063006b100 0x7f0630152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:35.698 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.695+0000 7f063586b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f063006b4e0 0x7f06301549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:35.698 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.695+0000 7f063586b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06300715c0 0x7f0630153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:35.698 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.695+0000 7f063586b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06300b2ae0 con 0x7f063006b100 2023-10-16T20:06:35.699 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.695+0000 7f063586b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06300b1ed0 con 0x7f06300715c0 2023-10-16T20:06:35.699 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.695+0000 7f063586b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06300b21b0 con 0x7f063006b4e0 2023-10-16T20:06:35.699 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.695+0000 7f062effd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f063006b100 0x7f0630152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:35.699 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.695+0000 7f062effd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f063006b100 0x7f0630152420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43200/0 (socket says 172.21.15.88:43200) 2023-10-16T20:06:35.699 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.695+0000 7f062effd640 1 -- 172.21.15.88:0/3825684817 learned_addr learned my addr 172.21.15.88:0/3825684817 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:35.700 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.695+0000 7f062f7fe640 1 --2- 172.21.15.88:0/3825684817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06300715c0 0x7f0630153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:35.700 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.695+0000 7f06267fc640 1 --2- 172.21.15.88:0/3825684817 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f063006b4e0 0x7f06301549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:35.700 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.695+0000 7f062effd640 1 -- 172.21.15.88:0/3825684817 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f063006b4e0 msgr2=0x7f06301549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:35.700 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.695+0000 7f062effd640 1 --2- 172.21.15.88:0/3825684817 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f063006b4e0 0x7f06301549f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:35.700 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.695+0000 7f062effd640 1 -- 172.21.15.88:0/3825684817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06300715c0 msgr2=0x7f0630153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:35.701 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.695+0000 7f062effd640 1 --2- 172.21.15.88:0/3825684817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06300715c0 0x7f0630153c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:35.701 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.695+0000 7f062effd640 1 -- 172.21.15.88:0/3825684817 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0628009420 con 0x7f063006b100 2023-10-16T20:06:35.701 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.695+0000 7f062f7fe640 1 --2- 172.21.15.88:0/3825684817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06300715c0 0x7f0630153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:06:35.701 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.695+0000 7f06267fc640 1 --2- 172.21.15.88:0/3825684817 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f063006b4e0 0x7f06301549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:06:35.701 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.695+0000 7f062effd640 1 --2- 172.21.15.88:0/3825684817 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f063006b100 0x7f0630152420 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f062000e930 tx=0x7f062000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:35.702 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.695+0000 7f062cff9640 1 -- 172.21.15.88:0/3825684817 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0620009690 con 0x7f063006b100 2023-10-16T20:06:35.702 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.695+0000 7f062cff9640 1 -- 172.21.15.88:0/3825684817 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062000ca00 con 0x7f063006b100 2023-10-16T20:06:35.702 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.695+0000 7f062cff9640 1 -- 172.21.15.88:0/3825684817 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062000cd70 con 0x7f063006b100 2023-10-16T20:06:35.702 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.695+0000 7f063586b640 1 -- 172.21.15.88:0/3825684817 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06300b2490 con 0x7f063006b100 2023-10-16T20:06:35.702 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.695+0000 7f063586b640 1 -- 172.21.15.88:0/3825684817 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06300b1b60 con 0x7f063006b100 2023-10-16T20:06:35.703 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.695+0000 7f063586b640 1 -- 172.21.15.88:0/3825684817 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f063004e9c0 con 0x7f063006b100 2023-10-16T20:06:35.704 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.699+0000 7f062cff9640 1 -- 172.21.15.88:0/3825684817 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0620013470 con 0x7f063006b100 2023-10-16T20:06:35.704 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.699+0000 7f062cff9640 1 --2- 172.21.15.88:0/3825684817 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f06100664a0 0x7f0610068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:35.704 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.699+0000 7f062cff9640 1 -- 172.21.15.88:0/3825684817 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f0620084570 con 0x7f063006b100 2023-10-16T20:06:35.704 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.699+0000 7f06267fc640 1 --2- 172.21.15.88:0/3825684817 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f06100664a0 0x7f0610068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:35.705 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.699+0000 7f06267fc640 1 --2- 172.21.15.88:0/3825684817 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f06100664a0 0x7f0610068960 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f06300c5a40 tx=0x7f0618009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:35.705 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.703+0000 7f062cff9640 1 -- 172.21.15.88:0/3825684817 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0620051690 con 0x7f063006b100 2023-10-16T20:06:35.855 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.851+0000 7f063586b640 1 -- 172.21.15.88:0/3825684817 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0630065e70 con 0x7f063006b100 2023-10-16T20:06:35.856 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.855+0000 7f062cff9640 1 -- 172.21.15.88:0/3825684817 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+18506 (secure 0 0 0) 0x7f0620054cb0 con 0x7f063006b100 2023-10-16T20:06:35.857 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:06:35.857 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":44,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:06:33.228336+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T20:06:14.743500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"testcephfs1_data","create_time":"2023-10-16T20:06:16.091837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-10-16T20:06:31.184462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:06:35.859 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.855+0000 7f063586b640 1 -- 172.21.15.88:0/3825684817 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f06100664a0 msgr2=0x7f0610068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:35.859 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.855+0000 7f063586b640 1 --2- 172.21.15.88:0/3825684817 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f06100664a0 0x7f0610068960 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f06300c5a40 tx=0x7f0618009040 comp rx=0 tx=0).stop 2023-10-16T20:06:35.859 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.855+0000 7f063586b640 1 -- 172.21.15.88:0/3825684817 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f063006b100 msgr2=0x7f0630152420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:35.859 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.855+0000 7f063586b640 1 --2- 172.21.15.88:0/3825684817 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f063006b100 0x7f0630152420 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f062000e930 tx=0x7f062000edf0 comp rx=0 tx=0).stop 2023-10-16T20:06:35.859 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.855+0000 7f063586b640 1 -- 172.21.15.88:0/3825684817 shutdown_connections 2023-10-16T20:06:35.860 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.855+0000 7f063586b640 1 --2- 172.21.15.88:0/3825684817 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f06100664a0 0x7f0610068960 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:35.860 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.855+0000 7f063586b640 1 --2- 172.21.15.88:0/3825684817 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f063006b4e0 0x7f06301549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:35.860 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.855+0000 7f063586b640 1 --2- 172.21.15.88:0/3825684817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06300715c0 0x7f0630153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:35.860 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.855+0000 7f063586b640 1 --2- 172.21.15.88:0/3825684817 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f063006b100 0x7f0630152420 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:35.860 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.855+0000 7f063586b640 1 -- 172.21.15.88:0/3825684817 >> 172.21.15.88:0/3825684817 conn(0x7f0630070aa0 msgr2=0x7f06300ab8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:35.861 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.855+0000 7f063586b640 1 -- 172.21.15.88:0/3825684817 shutdown_connections 2023-10-16T20:06:35.861 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:35.855+0000 7f063586b640 1 -- 172.21.15.88:0/3825684817 wait complete. 2023-10-16T20:06:35.878 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'testcephfs1_metadata', 'create_time': '2023-10-16T20:06:14.743500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'testcephfs1_data', 'create_time': '2023-10-16T20:06:16.091837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2023-10-16T20:06:31.184462+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T20:06:35.879 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:06:36.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.015+0000 7fc7132d6640 1 Processor -- start 2023-10-16T20:06:36.019 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.015+0000 7fc7132d6640 1 -- start start 2023-10-16T20:06:36.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc7132d6640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fc70c0d0760 0x7fc70c0b5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:36.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc7132d6640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc70c0d2a00 0x7fc70c0d2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:36.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc7132d6640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc70c0cda20 con 0x7fc70c15c930 2023-10-16T20:06:36.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc7132d6640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc70c0d33b0 con 0x7fc70c0d0760 2023-10-16T20:06:36.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc7132d6640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c0cdc80 con 0x7fc70c0d2a00 2023-10-16T20:06:36.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc711072640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fc70c15c930 0x7fc70c0cd310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:41478/0 (socket says 172.21.15.88:41478) 2023-10-16T20:06:36.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc711072640 1 -- 172.21.15.88:0/2955145199 learned_addr learned my addr 172.21.15.88:0/2955145199 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:36.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc711873640 1 --2- 172.21.15.88:0/2955145199 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc70c0d2a00 0x7fc70c0d2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:36.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc710871640 1 --2- 172.21.15.88:0/2955145199 >> v2:172.21.15.94:3300/0 conn(0x7fc70c0d0760 0x7fc70c0b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:36.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc703fff640 1 -- 172.21.15.88:0/2955145199 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2741472327 0 0) 0x7fc70c0cda20 con 0x7fc70c15c930 2023-10-16T20:06:36.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc703fff640 1 -- 172.21.15.88:0/2955145199 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6f8003530 con 0x7fc70c15c930 2023-10-16T20:06:36.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc703fff640 1 -- 172.21.15.88:0/2955145199 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3737617545 0 0) 0x7fc6f8003530 con 0x7fc70c15c930 2023-10-16T20:06:36.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc703fff640 1 -- 172.21.15.88:0/2955145199 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc70c0d2a00 msgr2=0x7fc70c0d2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:36.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc703fff640 1 --2- 172.21.15.88:0/2955145199 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc70c0d2a00 0x7fc70c0d2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:36.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc703fff640 1 -- 172.21.15.88:0/2955145199 >> v2:172.21.15.94:3300/0 conn(0x7fc70c0d0760 msgr2=0x7fc70c0b5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:36.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc703fff640 1 --2- 172.21.15.88:0/2955145199 >> v2:172.21.15.94:3300/0 conn(0x7fc70c0d0760 0x7fc70c0b5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:36.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc703fff640 1 -- 172.21.15.88:0/2955145199 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70c0cdf60 con 0x7fc70c15c930 2023-10-16T20:06:36.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc703fff640 1 -- 172.21.15.88:0/2955145199 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7fc7080016a0 con 0x7fc70c15c930 2023-10-16T20:06:36.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc703fff640 1 -- 172.21.15.88:0/2955145199 >> v1:172.21.15.88:6789/0 conn(0x7fc70c15c930 legacy=0x7fc70c0cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:36.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc703fff640 1 --2- 172.21.15.88:0/2955145199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6f8003980 0x7fc6f8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:36.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc703fff640 1 --2- 172.21.15.88:0/2955145199 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc6f80047d0 0x7fc6f8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:36.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc703fff640 1 --2- 172.21.15.88:0/2955145199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6f8005190 0x7fc6f800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:36.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc703fff640 1 -- 172.21.15.88:0/2955145199 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c0d33b0 con 0x7fc6f8003980 2023-10-16T20:06:36.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc703fff640 1 -- 172.21.15.88:0/2955145199 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c0cdc80 con 0x7fc6f8005190 2023-10-16T20:06:36.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc703fff640 1 -- 172.21.15.88:0/2955145199 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f800bed0 con 0x7fc6f80047d0 2023-10-16T20:06:36.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc710871640 1 --2- 172.21.15.88:0/2955145199 >> v2:172.21.15.94:3300/0 conn(0x7fc70c0d0760 0x7fc70c0b5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:06:36.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc711873640 1 --2- 172.21.15.88:0/2955145199 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc70c0d2a00 0x7fc70c0d2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:06:36.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc711072640 1 --2- 172.21.15.88:0/2955145199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6f8003980 0x7fc6f8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:36.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc710871640 1 --2- 172.21.15.88:0/2955145199 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc6f80047d0 0x7fc6f8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:36.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc711873640 1 --2- 172.21.15.88:0/2955145199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6f8005190 0x7fc6f800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:36.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc711072640 1 -- 172.21.15.88:0/2955145199 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc6f80047d0 msgr2=0x7fc6f8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:36.029 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc711072640 1 --2- 172.21.15.88:0/2955145199 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc6f80047d0 0x7fc6f8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:36.029 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc711072640 1 -- 172.21.15.88:0/2955145199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6f8005190 msgr2=0x7fc6f800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:36.029 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc711072640 1 --2- 172.21.15.88:0/2955145199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6f8005190 0x7fc6f800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:36.029 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.019+0000 7fc711072640 1 -- 172.21.15.88:0/2955145199 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f8003530 con 0x7fc6f8003980 2023-10-16T20:06:36.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.023+0000 7fc711072640 1 --2- 172.21.15.88:0/2955145199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6f8003980 0x7fc6f8003da0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc70800c240 tx=0x7fc70801cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=661135e24f8dea44 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:36.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.023+0000 7fc703fff640 1 -- 172.21.15.88:0/2955145199 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7080058e0 con 0x7fc6f8003980 2023-10-16T20:06:36.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.023+0000 7fc703fff640 1 -- 172.21.15.88:0/2955145199 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc708005a40 con 0x7fc6f8003980 2023-10-16T20:06:36.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.023+0000 7fc703fff640 1 -- 172.21.15.88:0/2955145199 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc708005df0 con 0x7fc6f8003980 2023-10-16T20:06:36.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.023+0000 7fc7132d6640 1 -- 172.21.15.88:0/2955145199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6f8003980 msgr2=0x7fc6f8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:36.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.023+0000 7fc7132d6640 1 --2- 172.21.15.88:0/2955145199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6f8003980 0x7fc6f8003da0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc70800c240 tx=0x7fc70801cbf0 comp rx=0 tx=0).stop 2023-10-16T20:06:36.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.023+0000 7fc711873640 1 -- 172.21.15.88:0/2955145199 reap_dead start 2023-10-16T20:06:36.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.023+0000 7fc7132d6640 1 -- 172.21.15.88:0/2955145199 shutdown_connections 2023-10-16T20:06:36.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.023+0000 7fc7132d6640 1 -- 172.21.15.88:0/2955145199 >> 172.21.15.88:0/2955145199 conn(0x7fc70c0719c0 msgr2=0x7fc70c071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:36.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.023+0000 7fc7132d6640 1 -- 172.21.15.88:0/2955145199 shutdown_connections 2023-10-16T20:06:36.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.023+0000 7fc7132d6640 1 -- 172.21.15.88:0/2955145199 wait complete. 2023-10-16T20:06:36.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.023+0000 7fc7132d6640 1 Processor -- start 2023-10-16T20:06:36.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.023+0000 7fc7132d6640 1 -- start start 2023-10-16T20:06:36.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.027+0000 7fc7132d6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6f8003980 0x7fc70c1f5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:36.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.027+0000 7fc7132d6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc70c157230 0x7fc70c0d13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:36.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.027+0000 7fc7132d6640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc70c0c6590 0x7fc70c0c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:36.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.027+0000 7fc7132d6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c1f8880 con 0x7fc6f8003980 2023-10-16T20:06:36.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.027+0000 7fc7132d6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c1f5760 con 0x7fc70c157230 2023-10-16T20:06:36.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.027+0000 7fc7132d6640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c1f5a40 con 0x7fc70c0c6590 2023-10-16T20:06:36.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.027+0000 7fc711072640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6f8003980 0x7fc70c1f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:36.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.027+0000 7fc711072640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6f8003980 0x7fc70c1f5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43214/0 (socket says 172.21.15.88:43214) 2023-10-16T20:06:36.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.027+0000 7fc711072640 1 -- 172.21.15.88:0/4105649529 learned_addr learned my addr 172.21.15.88:0/4105649529 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:36.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.027+0000 7fc710871640 1 --2- 172.21.15.88:0/4105649529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc70c157230 0x7fc70c0d13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:36.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.027+0000 7fc711873640 1 --2- 172.21.15.88:0/4105649529 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc70c0c6590 0x7fc70c0c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:36.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.027+0000 7fc711072640 1 -- 172.21.15.88:0/4105649529 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc70c0c6590 msgr2=0x7fc70c0c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:36.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.027+0000 7fc711072640 1 --2- 172.21.15.88:0/4105649529 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc70c0c6590 0x7fc70c0c6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:36.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.027+0000 7fc711072640 1 -- 172.21.15.88:0/4105649529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc70c157230 msgr2=0x7fc70c0d13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:36.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.027+0000 7fc711072640 1 --2- 172.21.15.88:0/4105649529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc70c157230 0x7fc70c0d13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:36.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.027+0000 7fc711072640 1 -- 172.21.15.88:0/4105649529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70c1f5d20 con 0x7fc6f8003980 2023-10-16T20:06:36.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.027+0000 7fc711072640 1 --2- 172.21.15.88:0/4105649529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6f8003980 0x7fc70c1f5180 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc7080ab040 tx=0x7fc6f8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:36.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.027+0000 7fc701ffb640 1 -- 172.21.15.88:0/4105649529 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7080047f0 con 0x7fc6f8003980 2023-10-16T20:06:36.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.027+0000 7fc701ffb640 1 -- 172.21.15.88:0/4105649529 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc708004950 con 0x7fc6f8003980 2023-10-16T20:06:36.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.027+0000 7fc701ffb640 1 -- 172.21.15.88:0/4105649529 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc708004ce0 con 0x7fc6f8003980 2023-10-16T20:06:36.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.027+0000 7fc7132d6640 1 -- 172.21.15.88:0/4105649529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc70c1f6000 con 0x7fc6f8003980 2023-10-16T20:06:36.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.027+0000 7fc7132d6640 1 -- 172.21.15.88:0/4105649529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc70c0d1900 con 0x7fc6f8003980 2023-10-16T20:06:36.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.027+0000 7fc7132d6640 1 -- 172.21.15.88:0/4105649529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc70c1f5b80 con 0x7fc6f8003980 2023-10-16T20:06:36.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.031+0000 7fc701ffb640 1 -- 172.21.15.88:0/4105649529 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc7080a57e0 con 0x7fc6f8003980 2023-10-16T20:06:36.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.031+0000 7fc701ffb640 1 --2- 172.21.15.88:0/4105649529 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc6f00664a0 0x7fc6f0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:36.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.031+0000 7fc701ffb640 1 -- 172.21.15.88:0/4105649529 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6449+0+0 (secure 0 0 0) 0x7fc70811db10 con 0x7fc6f8003980 2023-10-16T20:06:36.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.031+0000 7fc710871640 1 --2- 172.21.15.88:0/4105649529 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc6f00664a0 0x7fc6f0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:36.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.031+0000 7fc710871640 1 --2- 172.21.15.88:0/4105649529 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc6f00664a0 0x7fc6f0068960 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc70c0d0f80 tx=0x7fc6fc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:36.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.035+0000 7fc701ffb640 1 -- 172.21.15.88:0/4105649529 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc70c1f5b80 con 0x7fc6f8003980 2023-10-16T20:06:36.183 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.183+0000 7fc7132d6640 1 -- 172.21.15.88:0/4105649529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc70c1f5b80 con 0x7fc6f8003980 2023-10-16T20:06:36.184 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.183+0000 7fc701ffb640 1 -- 172.21.15.88:0/4105649529 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc70c1f5b80 con 0x7fc6f8003980 2023-10-16T20:06:36.184 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:06:36.185 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.183+0000 7fc7132d6640 1 -- 172.21.15.88:0/4105649529 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc6f00664a0 msgr2=0x7fc6f0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:36.186 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.183+0000 7fc7132d6640 1 --2- 172.21.15.88:0/4105649529 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc6f00664a0 0x7fc6f0068960 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc70c0d0f80 tx=0x7fc6fc008040 comp rx=0 tx=0).stop 2023-10-16T20:06:36.186 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.183+0000 7fc7132d6640 1 -- 172.21.15.88:0/4105649529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6f8003980 msgr2=0x7fc70c1f5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:36.186 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.183+0000 7fc7132d6640 1 --2- 172.21.15.88:0/4105649529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6f8003980 0x7fc70c1f5180 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc7080ab040 tx=0x7fc6f8005800 comp rx=0 tx=0).stop 2023-10-16T20:06:36.186 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.183+0000 7fc7132d6640 1 -- 172.21.15.88:0/4105649529 shutdown_connections 2023-10-16T20:06:36.187 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.183+0000 7fc7132d6640 1 --2- 172.21.15.88:0/4105649529 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc6f00664a0 0x7fc6f0068960 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:36.187 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.183+0000 7fc7132d6640 1 --2- 172.21.15.88:0/4105649529 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc70c0c6590 0x7fc70c0c6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:36.187 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.183+0000 7fc7132d6640 1 --2- 172.21.15.88:0/4105649529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc70c157230 0x7fc70c0d13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:36.187 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.183+0000 7fc7132d6640 1 --2- 172.21.15.88:0/4105649529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6f8003980 0x7fc70c1f5180 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:36.187 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.183+0000 7fc7132d6640 1 -- 172.21.15.88:0/4105649529 >> 172.21.15.88:0/4105649529 conn(0x7fc70c0719c0 msgr2=0x7fc70c15c200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:36.188 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.183+0000 7fc7132d6640 1 -- 172.21.15.88:0/4105649529 shutdown_connections 2023-10-16T20:06:36.188 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.183+0000 7fc7132d6640 1 -- 172.21.15.88:0/4105649529 wait complete. 2023-10-16T20:06:36.205 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_metadata pg_num 2023-10-16T20:06:36.346 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.342+0000 7f0f81f80640 1 Processor -- start 2023-10-16T20:06:36.346 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.342+0000 7f0f81f80640 1 -- start start 2023-10-16T20:06:36.347 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.346+0000 7f0f81f80640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f7c0b0da0 0x7f0f7c159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:36.347 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.346+0000 7f0f81f80640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f0f7c15a1f0 0x7f0f7c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:36.348 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.346+0000 7f0f81f80640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f7c15ad30 con 0x7f0f7c0b4260 2023-10-16T20:06:36.348 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.346+0000 7f0f81f80640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f7c0b4640 con 0x7f0f7c15a1f0 2023-10-16T20:06:36.348 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.346+0000 7f0f81f80640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c15a770 con 0x7f0f7c0b0da0 2023-10-16T20:06:36.348 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.346+0000 7f0f7bfff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f0f7c15a1f0 0x7f0f7c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:36.349 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.346+0000 7f0f7bfff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f0f7c15a1f0 0x7f0f7c06d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43218/0 (socket says 172.21.15.88:43218) 2023-10-16T20:06:36.349 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.346+0000 7f0f7bfff640 1 -- 172.21.15.88:0/1555390708 learned_addr learned my addr 172.21.15.88:0/1555390708 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:36.349 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.346+0000 7f0f7affd640 1 --2- 172.21.15.88:0/1555390708 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f7c0b0da0 0x7f0f7c159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:36.350 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.346+0000 7f0f7bfff640 1 -- 172.21.15.88:0/1555390708 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f7c0b0da0 msgr2=0x7f0f7c159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:36.350 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.346+0000 7f0f7bfff640 1 --2- 172.21.15.88:0/1555390708 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f7c0b0da0 0x7f0f7c159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:36.350 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.346+0000 7f0f7bfff640 1 -- 172.21.15.88:0/1555390708 >> v1:172.21.15.94:6789/0 conn(0x7f0f7c0b4260 legacy=0x7f0f7c15dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:36.350 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.346+0000 7f0f7bfff640 1 -- 172.21.15.88:0/1555390708 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7c15aa50 con 0x7f0f7c15a1f0 2023-10-16T20:06:36.351 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.346+0000 7f0f7bfff640 1 --2- 172.21.15.88:0/1555390708 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f7c15a1f0 0x7f0f7c06d3f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0f7400d150 tx=0x7f0f7401eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b8bde30a922c214 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:36.351 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.346+0000 7f0f7a7fc640 1 -- 172.21.15.88:0/1555390708 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7400b2d0 con 0x7f0f7c15a1f0 2023-10-16T20:06:36.351 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.350+0000 7f0f7a7fc640 1 -- 172.21.15.88:0/1555390708 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f7400b430 con 0x7f0f7c15a1f0 2023-10-16T20:06:36.352 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.350+0000 7f0f7a7fc640 1 -- 172.21.15.88:0/1555390708 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f740a7a70 con 0x7f0f7c15a1f0 2023-10-16T20:06:36.352 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.350+0000 7f0f81f80640 1 -- 172.21.15.88:0/1555390708 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f7c15a1f0 msgr2=0x7f0f7c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:36.353 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.350+0000 7f0f81f80640 1 --2- 172.21.15.88:0/1555390708 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f7c15a1f0 0x7f0f7c06d3f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0f7400d150 tx=0x7f0f7401eb80 comp rx=0 tx=0).stop 2023-10-16T20:06:36.353 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.350+0000 7f0f81f80640 1 -- 172.21.15.88:0/1555390708 shutdown_connections 2023-10-16T20:06:36.353 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.350+0000 7f0f81f80640 1 --2- 172.21.15.88:0/1555390708 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f7c15a1f0 0x7f0f7c06d3f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:36.353 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.350+0000 7f0f81f80640 1 --2- 172.21.15.88:0/1555390708 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f7c0b0da0 0x7f0f7c159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:36.354 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.350+0000 7f0f81f80640 1 -- 172.21.15.88:0/1555390708 >> 172.21.15.88:0/1555390708 conn(0x7f0f7c0cb080 msgr2=0x7f0f7c0cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:36.354 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.350+0000 7f0f81f80640 1 -- 172.21.15.88:0/1555390708 shutdown_connections 2023-10-16T20:06:36.354 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.350+0000 7f0f81f80640 1 -- 172.21.15.88:0/1555390708 wait complete. 2023-10-16T20:06:36.356 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.350+0000 7f0f81f80640 1 Processor -- start 2023-10-16T20:06:36.356 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.350+0000 7f0f81f80640 1 -- start start 2023-10-16T20:06:36.356 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.350+0000 7f0f81f80640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f7c0b0da0 0x7f0f7c14efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:36.357 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.350+0000 7f0f81f80640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f7c0b4260 0x7f0f7c156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:36.357 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.350+0000 7f0f81f80640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f7c15a1f0 0x7f0f7c157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:36.357 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.350+0000 7f0f81f80640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c0cc5b0 con 0x7f0f7c15a1f0 2023-10-16T20:06:36.357 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.350+0000 7f0f81f80640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c0cbff0 con 0x7f0f7c0b4260 2023-10-16T20:06:36.357 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.350+0000 7f0f81f80640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c0cc2d0 con 0x7f0f7c0b0da0 2023-10-16T20:06:36.358 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.350+0000 7f0f7bfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f7c15a1f0 0x7f0f7c157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:36.358 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.350+0000 7f0f7bfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f7c15a1f0 0x7f0f7c157130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43234/0 (socket says 172.21.15.88:43234) 2023-10-16T20:06:36.358 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.354+0000 7f0f7bfff640 1 -- 172.21.15.88:0/3320632923 learned_addr learned my addr 172.21.15.88:0/3320632923 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:36.358 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.354+0000 7f0f7b7fe640 1 --2- 172.21.15.88:0/3320632923 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f7c0b0da0 0x7f0f7c14efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:36.358 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.354+0000 7f0f7affd640 1 --2- 172.21.15.88:0/3320632923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f7c0b4260 0x7f0f7c156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:36.359 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.354+0000 7f0f7bfff640 1 -- 172.21.15.88:0/3320632923 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f7c0b0da0 msgr2=0x7f0f7c14efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:36.359 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.354+0000 7f0f7bfff640 1 --2- 172.21.15.88:0/3320632923 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f7c0b0da0 0x7f0f7c14efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:36.359 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.354+0000 7f0f7bfff640 1 -- 172.21.15.88:0/3320632923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f7c0b4260 msgr2=0x7f0f7c156bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:36.359 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.354+0000 7f0f7bfff640 1 --2- 172.21.15.88:0/3320632923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f7c0b4260 0x7f0f7c156bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:36.359 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.354+0000 7f0f7bfff640 1 -- 172.21.15.88:0/3320632923 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7400bd40 con 0x7f0f7c15a1f0 2023-10-16T20:06:36.360 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.354+0000 7f0f7bfff640 1 --2- 172.21.15.88:0/3320632923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f7c15a1f0 0x7f0f7c157130 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0f7401c7b0 tx=0x7f0f7401b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:36.360 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.354+0000 7f0f63fff640 1 -- 172.21.15.88:0/3320632923 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f74009800 con 0x7f0f7c15a1f0 2023-10-16T20:06:36.360 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.354+0000 7f0f63fff640 1 -- 172.21.15.88:0/3320632923 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f74009960 con 0x7f0f7c15a1f0 2023-10-16T20:06:36.360 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.354+0000 7f0f63fff640 1 -- 172.21.15.88:0/3320632923 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f74009d10 con 0x7f0f7c15a1f0 2023-10-16T20:06:36.360 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.354+0000 7f0f81f80640 1 -- 172.21.15.88:0/3320632923 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f7c14f980 con 0x7f0f7c15a1f0 2023-10-16T20:06:36.361 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.354+0000 7f0f81f80640 1 -- 172.21.15.88:0/3320632923 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f7c14cab0 con 0x7f0f7c15a1f0 2023-10-16T20:06:36.362 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.354+0000 7f0f81f80640 1 -- 172.21.15.88:0/3320632923 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f7c0cc410 con 0x7f0f7c15a1f0 2023-10-16T20:06:36.362 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.354+0000 7f0f63fff640 1 -- 172.21.15.88:0/3320632923 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0f740ba470 con 0x7f0f7c15a1f0 2023-10-16T20:06:36.362 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.354+0000 7f0f63fff640 1 --2- 172.21.15.88:0/3320632923 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0f680664a0 0x7f0f68068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:36.362 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.354+0000 7f0f63fff640 1 -- 172.21.15.88:0/3320632923 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f0f74121510 con 0x7f0f7c15a1f0 2023-10-16T20:06:36.362 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.358+0000 7f0f7b7fe640 1 --2- 172.21.15.88:0/3320632923 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0f680664a0 0x7f0f68068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:36.363 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.358+0000 7f0f7b7fe640 1 --2- 172.21.15.88:0/3320632923 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0f680664a0 0x7f0f68068960 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c158990 tx=0x7f0f6c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:36.363 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.362+0000 7f0f63fff640 1 -- 172.21.15.88:0/3320632923 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0f7c0cc410 con 0x7f0f7c15a1f0 2023-10-16T20:06:36.512 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.510+0000 7f0f81f80640 1 -- 172.21.15.88:0/3320632923 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0f7c14fea0 con 0x7f0f7c15a1f0 2023-10-16T20:06:36.512 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.510+0000 7f0f63fff640 1 -- 172.21.15.88:0/3320632923 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"}]=0 v44) v1 ==== 109+0+11 (secure 0 0 0) 0x7f0f7c14fea0 con 0x7f0f7c15a1f0 2023-10-16T20:06:36.513 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:06:36.514 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.510+0000 7f0f81f80640 1 -- 172.21.15.88:0/3320632923 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0f680664a0 msgr2=0x7f0f68068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:36.514 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.510+0000 7f0f81f80640 1 --2- 172.21.15.88:0/3320632923 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0f680664a0 0x7f0f68068960 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c158990 tx=0x7f0f6c009040 comp rx=0 tx=0).stop 2023-10-16T20:06:36.515 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.510+0000 7f0f81f80640 1 -- 172.21.15.88:0/3320632923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f7c15a1f0 msgr2=0x7f0f7c157130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:36.515 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.510+0000 7f0f81f80640 1 --2- 172.21.15.88:0/3320632923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f7c15a1f0 0x7f0f7c157130 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0f7401c7b0 tx=0x7f0f7401b920 comp rx=0 tx=0).stop 2023-10-16T20:06:36.515 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.510+0000 7f0f81f80640 1 -- 172.21.15.88:0/3320632923 shutdown_connections 2023-10-16T20:06:36.515 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.510+0000 7f0f81f80640 1 --2- 172.21.15.88:0/3320632923 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0f680664a0 0x7f0f68068960 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:36.515 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.510+0000 7f0f81f80640 1 --2- 172.21.15.88:0/3320632923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f7c15a1f0 0x7f0f7c157130 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:36.516 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.510+0000 7f0f81f80640 1 --2- 172.21.15.88:0/3320632923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f7c0b4260 0x7f0f7c156bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:36.516 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.510+0000 7f0f81f80640 1 --2- 172.21.15.88:0/3320632923 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f7c0b0da0 0x7f0f7c14efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:36.516 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.510+0000 7f0f81f80640 1 -- 172.21.15.88:0/3320632923 >> 172.21.15.88:0/3320632923 conn(0x7f0f7c0cb080 msgr2=0x7f0f7c0afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:36.516 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.510+0000 7f0f81f80640 1 -- 172.21.15.88:0/3320632923 shutdown_connections 2023-10-16T20:06:36.517 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.510+0000 7f0f81f80640 1 -- 172.21.15.88:0/3320632923 wait complete. 2023-10-16T20:06:36.534 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_data pg_num 2023-10-16T20:06:36.674 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.670+0000 7efed8ff8640 1 Processor -- start 2023-10-16T20:06:36.674 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.670+0000 7efed8ff8640 1 -- start start 2023-10-16T20:06:36.676 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.670+0000 7efed8ff8640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efed40cd370 0x7efed406a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:36.676 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.670+0000 7efed8ff8640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7efed415cea0 0x7efed406ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:36.676 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.670+0000 7efed8ff8640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efed406a3a0 con 0x7efed415cac0 2023-10-16T20:06:36.676 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.670+0000 7efed8ff8640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efed40cac10 con 0x7efed415cea0 2023-10-16T20:06:36.676 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.670+0000 7efed8ff8640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed40ca650 con 0x7efed40cd370 2023-10-16T20:06:36.677 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.674+0000 7efed2d9d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7efed415cea0 0x7efed406ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:36.677 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.674+0000 7efed2d9d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7efed415cea0 0x7efed406ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43236/0 (socket says 172.21.15.88:43236) 2023-10-16T20:06:36.677 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.674+0000 7efed2d9d640 1 -- 172.21.15.88:0/3294681070 learned_addr learned my addr 172.21.15.88:0/3294681070 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:36.677 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.674+0000 7efec9d9b640 1 --2- 172.21.15.88:0/3294681070 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efed40cd370 0x7efed406a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:36.677 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.674+0000 7efed2d9d640 1 -- 172.21.15.88:0/3294681070 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efed40cd370 msgr2=0x7efed406a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:36.678 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.674+0000 7efed2d9d640 1 --2- 172.21.15.88:0/3294681070 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efed40cd370 0x7efed406a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:36.678 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.674+0000 7efed2d9d640 1 -- 172.21.15.88:0/3294681070 >> v1:172.21.15.94:6789/0 conn(0x7efed415cac0 legacy=0x7efed40ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:36.678 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.674+0000 7efed2d9d640 1 -- 172.21.15.88:0/3294681070 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed40ca930 con 0x7efed415cea0 2023-10-16T20:06:36.679 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.674+0000 7efed2d9d640 1 --2- 172.21.15.88:0/3294681070 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efed415cea0 0x7efed406ad70 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7efecc009870 tx=0x7efecc01ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43286e299a105ffb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:36.679 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.674+0000 7efed1d9b640 1 -- 172.21.15.88:0/3294681070 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc0a8a30 con 0x7efed415cea0 2023-10-16T20:06:36.679 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.674+0000 7efed1d9b640 1 -- 172.21.15.88:0/3294681070 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efecc0a8b90 con 0x7efed415cea0 2023-10-16T20:06:36.679 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.674+0000 7efed1d9b640 1 -- 172.21.15.88:0/3294681070 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc0b0440 con 0x7efed415cea0 2023-10-16T20:06:36.680 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.678+0000 7efed8ff8640 1 -- 172.21.15.88:0/3294681070 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efed415cea0 msgr2=0x7efed406ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:36.680 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.678+0000 7efed8ff8640 1 --2- 172.21.15.88:0/3294681070 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efed415cea0 0x7efed406ad70 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7efecc009870 tx=0x7efecc01ed30 comp rx=0 tx=0).stop 2023-10-16T20:06:36.681 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.678+0000 7efed8ff8640 1 -- 172.21.15.88:0/3294681070 shutdown_connections 2023-10-16T20:06:36.681 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.678+0000 7efed8ff8640 1 --2- 172.21.15.88:0/3294681070 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efed415cea0 0x7efed406ad70 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:36.681 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.678+0000 7efed8ff8640 1 --2- 172.21.15.88:0/3294681070 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efed40cd370 0x7efed406a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:36.681 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.678+0000 7efed8ff8640 1 -- 172.21.15.88:0/3294681070 >> 172.21.15.88:0/3294681070 conn(0x7efed40b0970 msgr2=0x7efed40b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:36.682 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.678+0000 7efed8ff8640 1 -- 172.21.15.88:0/3294681070 shutdown_connections 2023-10-16T20:06:36.682 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.678+0000 7efed8ff8640 1 -- 172.21.15.88:0/3294681070 wait complete. 2023-10-16T20:06:36.684 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.678+0000 7efed8ff8640 1 Processor -- start 2023-10-16T20:06:36.684 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.678+0000 7efed8ff8640 1 -- start start 2023-10-16T20:06:36.684 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.678+0000 7efed8ff8640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efed40cd370 0x7efed4156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:36.684 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.678+0000 7efed8ff8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efed415cac0 0x7efed414e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:36.685 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.678+0000 7efed8ff8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efed415cea0 0x7efed414e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:36.685 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.678+0000 7efed8ff8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed40c6a20 con 0x7efed415cac0 2023-10-16T20:06:36.685 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.678+0000 7efed8ff8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed415f610 con 0x7efed415cea0 2023-10-16T20:06:36.685 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.678+0000 7efed8ff8640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed40c6e40 con 0x7efed40cd370 2023-10-16T20:06:36.685 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.678+0000 7efec9d9b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efed415cac0 0x7efed414e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:36.686 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.678+0000 7efec9d9b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efed415cac0 0x7efed414e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43250/0 (socket says 172.21.15.88:43250) 2023-10-16T20:06:36.686 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.678+0000 7efec9d9b640 1 -- 172.21.15.88:0/2395664635 learned_addr learned my addr 172.21.15.88:0/2395664635 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:36.686 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.678+0000 7efed2d9d640 1 --2- 172.21.15.88:0/2395664635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efed415cea0 0x7efed414e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:36.686 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.678+0000 7efed259c640 1 --2- 172.21.15.88:0/2395664635 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efed40cd370 0x7efed4156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:36.686 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.682+0000 7efec9d9b640 1 -- 172.21.15.88:0/2395664635 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efed40cd370 msgr2=0x7efed4156e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:36.687 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.682+0000 7efec9d9b640 1 --2- 172.21.15.88:0/2395664635 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efed40cd370 0x7efed4156e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:36.687 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.682+0000 7efec9d9b640 1 -- 172.21.15.88:0/2395664635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efed415cea0 msgr2=0x7efed414e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:36.687 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.682+0000 7efec9d9b640 1 --2- 172.21.15.88:0/2395664635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efed415cea0 0x7efed414e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:36.687 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.682+0000 7efec9d9b640 1 -- 172.21.15.88:0/2395664635 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efecc0094d0 con 0x7efed415cac0 2023-10-16T20:06:36.687 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.682+0000 7efec9d9b640 1 --2- 172.21.15.88:0/2395664635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efed415cac0 0x7efed414e0a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7efebc007ec0 tx=0x7efebc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:36.688 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.682+0000 7efecb7fe640 1 -- 172.21.15.88:0/2395664635 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc014620 con 0x7efed415cac0 2023-10-16T20:06:36.688 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.682+0000 7efecb7fe640 1 -- 172.21.15.88:0/2395664635 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efebc0147a0 con 0x7efed415cac0 2023-10-16T20:06:36.688 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.682+0000 7efecb7fe640 1 -- 172.21.15.88:0/2395664635 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc014b50 con 0x7efed415cac0 2023-10-16T20:06:36.688 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.682+0000 7efed8ff8640 1 -- 172.21.15.88:0/2395664635 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed40c6080 con 0x7efed415cac0 2023-10-16T20:06:36.688 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.682+0000 7efed8ff8640 1 -- 172.21.15.88:0/2395664635 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed40c6640 con 0x7efed415cac0 2023-10-16T20:06:36.690 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.682+0000 7efed8ff8640 1 -- 172.21.15.88:0/2395664635 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efed40c6b60 con 0x7efed415cac0 2023-10-16T20:06:36.690 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.682+0000 7efecb7fe640 1 -- 172.21.15.88:0/2395664635 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7efebc014cb0 con 0x7efed415cac0 2023-10-16T20:06:36.690 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.682+0000 7efecb7fe640 1 --2- 172.21.15.88:0/2395664635 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7efeb40664a0 0x7efeb4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:36.690 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.682+0000 7efecb7fe640 1 -- 172.21.15.88:0/2395664635 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6449+0+0 (secure 0 0 0) 0x7efebc0849e0 con 0x7efed415cac0 2023-10-16T20:06:36.690 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.682+0000 7efed259c640 1 --2- 172.21.15.88:0/2395664635 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7efeb40664a0 0x7efeb4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:36.691 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.686+0000 7efed259c640 1 --2- 172.21.15.88:0/2395664635 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7efeb40664a0 0x7efeb4068960 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efec4001650 tx=0x7efec4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:36.691 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.686+0000 7efecb7fe640 1 -- 172.21.15.88:0/2395664635 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7efed40c6b60 con 0x7efed415cac0 2023-10-16T20:06:36.837 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.834+0000 7efed8ff8640 1 -- 172.21.15.88:0/2395664635 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"} v 0) v1 -- 0x7efed40c6b60 con 0x7efed415cac0 2023-10-16T20:06:36.838 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.834+0000 7efecb7fe640 1 -- 172.21.15.88:0/2395664635 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"}]=0 v44) v1 ==== 105+0+11 (secure 0 0 0) 0x7efed40c6b60 con 0x7efed415cac0 2023-10-16T20:06:36.838 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2023-10-16T20:06:36.840 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.838+0000 7efed8ff8640 1 -- 172.21.15.88:0/2395664635 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7efeb40664a0 msgr2=0x7efeb4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:36.840 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.838+0000 7efed8ff8640 1 --2- 172.21.15.88:0/2395664635 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7efeb40664a0 0x7efeb4068960 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efec4001650 tx=0x7efec4009040 comp rx=0 tx=0).stop 2023-10-16T20:06:36.840 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.838+0000 7efed8ff8640 1 -- 172.21.15.88:0/2395664635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efed415cac0 msgr2=0x7efed414e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:36.840 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.838+0000 7efed8ff8640 1 --2- 172.21.15.88:0/2395664635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efed415cac0 0x7efed414e0a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7efebc007ec0 tx=0x7efebc00e4b0 comp rx=0 tx=0).stop 2023-10-16T20:06:36.841 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.838+0000 7efed8ff8640 1 -- 172.21.15.88:0/2395664635 shutdown_connections 2023-10-16T20:06:36.841 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.838+0000 7efed8ff8640 1 --2- 172.21.15.88:0/2395664635 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7efeb40664a0 0x7efeb4068960 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:36.841 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.838+0000 7efed8ff8640 1 --2- 172.21.15.88:0/2395664635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efed415cea0 0x7efed414e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:36.841 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.838+0000 7efed8ff8640 1 --2- 172.21.15.88:0/2395664635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efed415cac0 0x7efed414e0a0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:36.841 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.838+0000 7efed8ff8640 1 --2- 172.21.15.88:0/2395664635 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efed40cd370 0x7efed4156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:36.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.838+0000 7efed8ff8640 1 -- 172.21.15.88:0/2395664635 >> 172.21.15.88:0/2395664635 conn(0x7efed40b0970 msgr2=0x7efed40af460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:36.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.838+0000 7efed8ff8640 1 -- 172.21.15.88:0/2395664635 shutdown_connections 2023-10-16T20:06:36.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.838+0000 7efed8ff8640 1 -- 172.21.15.88:0/2395664635 wait complete. 2023-10-16T20:06:36.859 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-10-16T20:06:36.998 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.994+0000 7fb2bcaea640 1 Processor -- start 2023-10-16T20:06:36.999 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.994+0000 7fb2bcaea640 1 -- start start 2023-10-16T20:06:36.999 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2bcaea640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb2b81500a0 0x7fb2b8159f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:37.000 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2bcaea640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2b806c2e0 0x7fb2b8070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:37.000 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2bcaea640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2b815aa10 con 0x7fb2b8071020 2023-10-16T20:06:37.000 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2bcaea640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2b8150480 con 0x7fb2b81500a0 2023-10-16T20:06:37.000 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2bcaea640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b815a490 con 0x7fb2b806c2e0 2023-10-16T20:06:37.002 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2b6d9d640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fb2b8071020 0x7fb2b815ca90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:41480/0 (socket says 172.21.15.88:41480) 2023-10-16T20:06:37.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2b6d9d640 1 -- 172.21.15.88:0/3776957787 learned_addr learned my addr 172.21.15.88:0/3776957787 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:37.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2b5d9b640 1 --2- 172.21.15.88:0/3776957787 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2b806c2e0 0x7fb2b8070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:37.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2b659c640 1 --2- 172.21.15.88:0/3776957787 >> v2:172.21.15.94:3300/0 conn(0x7fb2b81500a0 0x7fb2b8159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:37.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2b559a640 1 -- 172.21.15.88:0/3776957787 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 439087926 0 0) 0x7fb2b815aa10 con 0x7fb2b8071020 2023-10-16T20:06:37.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2b559a640 1 -- 172.21.15.88:0/3776957787 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2a0003450 con 0x7fb2b8071020 2023-10-16T20:06:37.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2b559a640 1 -- 172.21.15.88:0/3776957787 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 800324546 0 0) 0x7fb2a0003450 con 0x7fb2b8071020 2023-10-16T20:06:37.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2b559a640 1 -- 172.21.15.88:0/3776957787 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2b806c2e0 msgr2=0x7fb2b8070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:37.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2b559a640 1 --2- 172.21.15.88:0/3776957787 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2b806c2e0 0x7fb2b8070ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:37.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2b559a640 1 -- 172.21.15.88:0/3776957787 >> v2:172.21.15.94:3300/0 conn(0x7fb2b81500a0 msgr2=0x7fb2b8159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:37.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2b559a640 1 --2- 172.21.15.88:0/3776957787 >> v2:172.21.15.94:3300/0 conn(0x7fb2b81500a0 0x7fb2b8159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:37.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2b559a640 1 -- 172.21.15.88:0/3776957787 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b806ca10 con 0x7fb2b8071020 2023-10-16T20:06:37.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2b559a640 1 -- 172.21.15.88:0/3776957787 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7fb2b0001a00 con 0x7fb2b8071020 2023-10-16T20:06:37.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2b5d9b640 1 --2- 172.21.15.88:0/3776957787 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2b806c2e0 0x7fb2b8070ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:06:37.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2b559a640 1 -- 172.21.15.88:0/3776957787 >> v1:172.21.15.88:6789/0 conn(0x7fb2b8071020 legacy=0x7fb2b815ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:37.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2b559a640 1 --2- 172.21.15.88:0/3776957787 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2a0003a70 0x7fb2a0003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:37.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2b559a640 1 --2- 172.21.15.88:0/3776957787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2a0008860 0x7fb2a000ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:37.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2b559a640 1 --2- 172.21.15.88:0/3776957787 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2b806c2e0 0x7fb2a000d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:37.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2b559a640 1 -- 172.21.15.88:0/3776957787 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b8150480 con 0x7fb2a0003a70 2023-10-16T20:06:37.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2b559a640 1 -- 172.21.15.88:0/3776957787 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b815a490 con 0x7fb2a0008860 2023-10-16T20:06:37.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2b559a640 1 -- 172.21.15.88:0/3776957787 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a000dae0 con 0x7fb2b806c2e0 2023-10-16T20:06:37.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2b659c640 1 --2- 172.21.15.88:0/3776957787 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2a0003a70 0x7fb2a0003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:37.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2b5d9b640 1 --2- 172.21.15.88:0/3776957787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2a0008860 0x7fb2a000ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:37.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2b6d9d640 1 --2- 172.21.15.88:0/3776957787 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2b806c2e0 0x7fb2a000d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:37.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2b659c640 1 -- 172.21.15.88:0/3776957787 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2b806c2e0 msgr2=0x7fb2a000d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:37.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2b659c640 1 --2- 172.21.15.88:0/3776957787 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2b806c2e0 0x7fb2a000d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:37.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2b659c640 1 -- 172.21.15.88:0/3776957787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2a0008860 msgr2=0x7fb2a000ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:37.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2b659c640 1 --2- 172.21.15.88:0/3776957787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2a0008860 0x7fb2a000ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:37.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2b659c640 1 -- 172.21.15.88:0/3776957787 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ac009420 con 0x7fb2a0003a70 2023-10-16T20:06:37.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:36.998+0000 7fb2b5d9b640 1 --2- 172.21.15.88:0/3776957787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2a0008860 0x7fb2a000ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:37.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.002+0000 7fb2b659c640 1 --2- 172.21.15.88:0/3776957787 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2a0003a70 0x7fb2a0003e90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8009ae0 tx=0x7fb2a801ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e285006389a8c1c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:37.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.002+0000 7fb2b559a640 1 -- 172.21.15.88:0/3776957787 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a80a6470 con 0x7fb2a0003a70 2023-10-16T20:06:37.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.002+0000 7fb2b559a640 1 -- 172.21.15.88:0/3776957787 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a80a65d0 con 0x7fb2a0003a70 2023-10-16T20:06:37.012 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.002+0000 7fb2b559a640 1 -- 172.21.15.88:0/3776957787 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a80a69c0 con 0x7fb2a0003a70 2023-10-16T20:06:37.012 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.002+0000 7fb2bcaea640 1 -- 172.21.15.88:0/3776957787 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2a0003a70 msgr2=0x7fb2a0003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:37.012 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.002+0000 7fb2bcaea640 1 --2- 172.21.15.88:0/3776957787 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2a0003a70 0x7fb2a0003e90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8009ae0 tx=0x7fb2a801ca90 comp rx=0 tx=0).stop 2023-10-16T20:06:37.012 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.002+0000 7fb2b6d9d640 1 -- 172.21.15.88:0/3776957787 reap_dead start 2023-10-16T20:06:37.012 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.002+0000 7fb2bcaea640 1 -- 172.21.15.88:0/3776957787 shutdown_connections 2023-10-16T20:06:37.013 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.002+0000 7fb2bcaea640 1 -- 172.21.15.88:0/3776957787 >> 172.21.15.88:0/3776957787 conn(0x7fb2b80cb080 msgr2=0x7fb2b80cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:37.013 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.002+0000 7fb2bcaea640 1 -- 172.21.15.88:0/3776957787 shutdown_connections 2023-10-16T20:06:37.013 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.002+0000 7fb2bcaea640 1 -- 172.21.15.88:0/3776957787 wait complete. 2023-10-16T20:06:37.013 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.006+0000 7fb2bcaea640 1 Processor -- start 2023-10-16T20:06:37.013 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.006+0000 7fb2bcaea640 1 -- start start 2023-10-16T20:06:37.013 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.006+0000 7fb2bcaea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2a0003a70 0x7fb2b8153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:37.014 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.006+0000 7fb2bcaea640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2b814d420 0x7fb2b81546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:37.014 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.006+0000 7fb2bcaea640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2b814d8a0 0x7fb2b81505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:37.014 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.006+0000 7fb2bcaea640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b814c0f0 con 0x7fb2b814d420 2023-10-16T20:06:37.014 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.006+0000 7fb2bcaea640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b81540a0 con 0x7fb2a0003a70 2023-10-16T20:06:37.014 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.006+0000 7fb2bcaea640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b814c3a0 con 0x7fb2b814d8a0 2023-10-16T20:06:37.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.006+0000 7fb2b5d9b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2b814d420 0x7fb2b81546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:37.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.006+0000 7fb2b5d9b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2b814d420 0x7fb2b81546d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43258/0 (socket says 172.21.15.88:43258) 2023-10-16T20:06:37.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.006+0000 7fb2b5d9b640 1 -- 172.21.15.88:0/2733745835 learned_addr learned my addr 172.21.15.88:0/2733745835 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:37.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.006+0000 7fb2b659c640 1 --2- 172.21.15.88:0/2733745835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2a0003a70 0x7fb2b8153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:37.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.006+0000 7fb2b6d9d640 1 --2- 172.21.15.88:0/2733745835 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2b814d8a0 0x7fb2b81505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:37.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.006+0000 7fb2b5d9b640 1 -- 172.21.15.88:0/2733745835 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2b814d8a0 msgr2=0x7fb2b81505c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:37.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.006+0000 7fb2b5d9b640 1 --2- 172.21.15.88:0/2733745835 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2b814d8a0 0x7fb2b81505c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:37.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.006+0000 7fb2b5d9b640 1 -- 172.21.15.88:0/2733745835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2a0003a70 msgr2=0x7fb2b8153b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:37.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.006+0000 7fb2b5d9b640 1 --2- 172.21.15.88:0/2733745835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2a0003a70 0x7fb2b8153b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:37.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.006+0000 7fb2b5d9b640 1 -- 172.21.15.88:0/2733745835 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b814c680 con 0x7fb2b814d420 2023-10-16T20:06:37.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.006+0000 7fb2b6d9d640 1 --2- 172.21.15.88:0/2733745835 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2b814d8a0 0x7fb2b81505c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:37.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.006+0000 7fb2b5d9b640 1 --2- 172.21.15.88:0/2733745835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2b814d420 0x7fb2b81546d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac013d20 tx=0x7fb2ac00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:37.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.006+0000 7fb2a6ffd640 1 -- 172.21.15.88:0/2733745835 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac018070 con 0x7fb2b814d420 2023-10-16T20:06:37.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.006+0000 7fb2a6ffd640 1 -- 172.21.15.88:0/2733745835 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ac00cde0 con 0x7fb2b814d420 2023-10-16T20:06:37.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.006+0000 7fb2a6ffd640 1 -- 172.21.15.88:0/2733745835 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac0132d0 con 0x7fb2b814d420 2023-10-16T20:06:37.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.006+0000 7fb2bcaea640 1 -- 172.21.15.88:0/2733745835 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b814c960 con 0x7fb2b814d420 2023-10-16T20:06:37.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.006+0000 7fb2bcaea640 1 -- 172.21.15.88:0/2733745835 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b814cf20 con 0x7fb2b814d420 2023-10-16T20:06:37.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.006+0000 7fb2bcaea640 1 -- 172.21.15.88:0/2733745835 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2b804e9c0 con 0x7fb2b814d420 2023-10-16T20:06:37.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.010+0000 7fb2a6ffd640 1 -- 172.21.15.88:0/2733745835 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb2ac0115d0 con 0x7fb2b814d420 2023-10-16T20:06:37.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.010+0000 7fb2a6ffd640 1 --2- 172.21.15.88:0/2733745835 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb2980664a0 0x7fb298068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:37.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.010+0000 7fb2a6ffd640 1 -- 172.21.15.88:0/2733745835 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6449+0+0 (secure 0 0 0) 0x7fb2ac088290 con 0x7fb2b814d420 2023-10-16T20:06:37.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.010+0000 7fb2b659c640 1 --2- 172.21.15.88:0/2733745835 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb2980664a0 0x7fb298068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:37.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.010+0000 7fb2b659c640 1 --2- 172.21.15.88:0/2733745835 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb2980664a0 0x7fb298068960 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80ad000 tx=0x7fb2a80b3040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:37.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.014+0000 7fb2a6ffd640 1 -- 172.21.15.88:0/2733745835 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb2ac055380 con 0x7fb2b814d420 2023-10-16T20:06:37.164 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.162+0000 7fb2bcaea640 1 -- 172.21.15.88:0/2733745835 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7fb2b8153820 con 0x7fb2b814d420 2023-10-16T20:06:37.165 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.162+0000 7fb2a6ffd640 1 -- 172.21.15.88:0/2733745835 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v44) v1 ==== 97+0+11 (secure 0 0 0) 0x7fb2ac0589a0 con 0x7fb2b814d420 2023-10-16T20:06:37.165 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:06:37.166 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.162+0000 7fb2bcaea640 1 -- 172.21.15.88:0/2733745835 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb2980664a0 msgr2=0x7fb298068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:37.167 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.162+0000 7fb2bcaea640 1 --2- 172.21.15.88:0/2733745835 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb2980664a0 0x7fb298068960 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80ad000 tx=0x7fb2a80b3040 comp rx=0 tx=0).stop 2023-10-16T20:06:37.167 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.162+0000 7fb2bcaea640 1 -- 172.21.15.88:0/2733745835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2b814d420 msgr2=0x7fb2b81546d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:37.167 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.162+0000 7fb2bcaea640 1 --2- 172.21.15.88:0/2733745835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2b814d420 0x7fb2b81546d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac013d20 tx=0x7fb2ac00c520 comp rx=0 tx=0).stop 2023-10-16T20:06:37.167 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.162+0000 7fb2bcaea640 1 -- 172.21.15.88:0/2733745835 shutdown_connections 2023-10-16T20:06:37.167 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.162+0000 7fb2bcaea640 1 --2- 172.21.15.88:0/2733745835 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb2980664a0 0x7fb298068960 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:37.168 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.162+0000 7fb2bcaea640 1 --2- 172.21.15.88:0/2733745835 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2b814d8a0 0x7fb2b81505c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:37.168 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.162+0000 7fb2bcaea640 1 --2- 172.21.15.88:0/2733745835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2b814d420 0x7fb2b81546d0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:37.168 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.162+0000 7fb2bcaea640 1 --2- 172.21.15.88:0/2733745835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2a0003a70 0x7fb2b8153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:37.168 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.162+0000 7fb2bcaea640 1 -- 172.21.15.88:0/2733745835 >> 172.21.15.88:0/2733745835 conn(0x7fb2b80cb080 msgr2=0x7fb2b814f370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:37.168 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.162+0000 7fb2bcaea640 1 -- 172.21.15.88:0/2733745835 shutdown_connections 2023-10-16T20:06:37.169 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.162+0000 7fb2bcaea640 1 -- 172.21.15.88:0/2733745835 wait complete. 2023-10-16T20:06:37.185 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T20:06:37.329 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.326+0000 7f295bf80640 1 Processor -- start 2023-10-16T20:06:37.329 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.326+0000 7f295bf80640 1 -- start start 2023-10-16T20:06:37.330 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.326+0000 7f295bf80640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f295415e5a0 0x7f29540b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:37.330 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.326+0000 7f295bf80640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29540cc5d0 0x7f29540cca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:37.331 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.326+0000 7f295bf80640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29540ccf40 con 0x7f29541584a0 2023-10-16T20:06:37.331 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.326+0000 7f295bf80640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f295415f4d0 con 0x7f295415e5a0 2023-10-16T20:06:37.331 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.326+0000 7f295bf80640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295415afb0 con 0x7f29540cc5d0 2023-10-16T20:06:37.331 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.326+0000 7f2959d1c640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f29541584a0 0x7f295415b6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:41492/0 (socket says 172.21.15.88:41492) 2023-10-16T20:06:37.331 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.326+0000 7f2959d1c640 1 -- 172.21.15.88:0/2854323684 learned_addr learned my addr 172.21.15.88:0/2854323684 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:37.332 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.326+0000 7f295a51d640 1 --2- 172.21.15.88:0/2854323684 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29540cc5d0 0x7f29540cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:37.333 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.326+0000 7f295951b640 1 --2- 172.21.15.88:0/2854323684 >> v2:172.21.15.94:3300/0 conn(0x7f295415e5a0 0x7f29540b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:37.333 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.326+0000 7f2958d1a640 1 -- 172.21.15.88:0/2854323684 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1363745618 0 0) 0x7f29540ccf40 con 0x7f29541584a0 2023-10-16T20:06:37.333 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.326+0000 7f2958d1a640 1 -- 172.21.15.88:0/2854323684 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2940003530 con 0x7f29541584a0 2023-10-16T20:06:37.333 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.330+0000 7f295a51d640 1 -- 172.21.15.88:0/2854323684 >> v2:172.21.15.94:3300/0 conn(0x7f295415e5a0 msgr2=0x7f29540b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:37.334 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.330+0000 7f295a51d640 1 --2- 172.21.15.88:0/2854323684 >> v2:172.21.15.94:3300/0 conn(0x7f295415e5a0 0x7f29540b21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:37.334 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.330+0000 7f2958d1a640 1 -- 172.21.15.88:0/2854323684 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3322667711 0 0) 0x7f2940003530 con 0x7f29541584a0 2023-10-16T20:06:37.334 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.330+0000 7f295a51d640 1 -- 172.21.15.88:0/2854323684 >> v1:172.21.15.88:6789/0 conn(0x7f29541584a0 legacy=0x7f295415b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:37.334 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.330+0000 7f295a51d640 1 -- 172.21.15.88:0/2854323684 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29540b2720 con 0x7f29540cc5d0 2023-10-16T20:06:37.335 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.330+0000 7f295a51d640 1 --2- 172.21.15.88:0/2854323684 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29540cc5d0 0x7f29540cca00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f294c009d90 tx=0x7f294c01c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3fbf8f8d740dd52 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:37.336 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.330+0000 7f2958d1a640 1 -- 172.21.15.88:0/2854323684 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29500016a0 con 0x7f29540cc5d0 2023-10-16T20:06:37.336 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.330+0000 7f2958d1a640 1 -- 172.21.15.88:0/2854323684 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294c0a8450 con 0x7f29540cc5d0 2023-10-16T20:06:37.336 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.330+0000 7f2958d1a640 1 -- 172.21.15.88:0/2854323684 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c0a87e0 con 0x7f29540cc5d0 2023-10-16T20:06:37.336 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.330+0000 7f295bf80640 1 -- 172.21.15.88:0/2854323684 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29540cc5d0 msgr2=0x7f29540cca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:37.336 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.330+0000 7f295bf80640 1 --2- 172.21.15.88:0/2854323684 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29540cc5d0 0x7f29540cca00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f294c009d90 tx=0x7f294c01c7e0 comp rx=0 tx=0).stop 2023-10-16T20:06:37.337 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.334+0000 7f295bf80640 1 -- 172.21.15.88:0/2854323684 shutdown_connections 2023-10-16T20:06:37.337 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.334+0000 7f295bf80640 1 --2- 172.21.15.88:0/2854323684 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29540cc5d0 0x7f29540cca00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:37.337 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.334+0000 7f295bf80640 1 --2- 172.21.15.88:0/2854323684 >> v2:172.21.15.94:3300/0 conn(0x7f295415e5a0 0x7f29540b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:37.337 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.334+0000 7f295bf80640 1 -- 172.21.15.88:0/2854323684 >> 172.21.15.88:0/2854323684 conn(0x7f29540ce5e0 msgr2=0x7f29540ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:37.337 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.334+0000 7f295bf80640 1 -- 172.21.15.88:0/2854323684 shutdown_connections 2023-10-16T20:06:37.338 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.334+0000 7f295bf80640 1 -- 172.21.15.88:0/2854323684 wait complete. 2023-10-16T20:06:37.340 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.334+0000 7f295bf80640 1 Processor -- start 2023-10-16T20:06:37.340 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.334+0000 7f295bf80640 1 -- start start 2023-10-16T20:06:37.340 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.334+0000 7f295bf80640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29540cc5d0 0x7f2954154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:37.340 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.334+0000 7f295bf80640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29541584a0 0x7f2954152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:37.341 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.334+0000 7f295bf80640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f295415e5a0 0x7f295414b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:37.341 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.334+0000 7f295bf80640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29540ae2f0 con 0x7f29541584a0 2023-10-16T20:06:37.341 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.334+0000 7f295bf80640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29540ae8b0 con 0x7f295415e5a0 2023-10-16T20:06:37.341 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.334+0000 7f295bf80640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295414c360 con 0x7f29540cc5d0 2023-10-16T20:06:37.341 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.334+0000 7f295951b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29541584a0 0x7f2954152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:37.342 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.334+0000 7f295951b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29541584a0 0x7f2954152750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43270/0 (socket says 172.21.15.88:43270) 2023-10-16T20:06:37.342 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.334+0000 7f295951b640 1 -- 172.21.15.88:0/3930905135 learned_addr learned my addr 172.21.15.88:0/3930905135 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:37.342 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.334+0000 7f295a51d640 1 --2- 172.21.15.88:0/3930905135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f295415e5a0 0x7f295414b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:37.342 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.334+0000 7f2959d1c640 1 --2- 172.21.15.88:0/3930905135 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29540cc5d0 0x7f2954154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:37.342 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.334+0000 7f295951b640 1 -- 172.21.15.88:0/3930905135 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29540cc5d0 msgr2=0x7f2954154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:37.342 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.334+0000 7f295951b640 1 --2- 172.21.15.88:0/3930905135 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29540cc5d0 0x7f2954154fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:37.343 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.334+0000 7f295951b640 1 -- 172.21.15.88:0/3930905135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f295415e5a0 msgr2=0x7f295414b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:37.343 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.334+0000 7f295951b640 1 --2- 172.21.15.88:0/3930905135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f295415e5a0 0x7f295414b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:37.343 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.334+0000 7f295951b640 1 -- 172.21.15.88:0/3930905135 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294c009a70 con 0x7f29541584a0 2023-10-16T20:06:37.343 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.334+0000 7f295951b640 1 --2- 172.21.15.88:0/3930905135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29541584a0 0x7f2954152750 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f294400e960 tx=0x7f294400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:37.343 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.338+0000 7f294a7fc640 1 -- 172.21.15.88:0/3930905135 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2944009690 con 0x7f29541584a0 2023-10-16T20:06:37.344 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.338+0000 7f294a7fc640 1 -- 172.21.15.88:0/3930905135 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294400ca00 con 0x7f29541584a0 2023-10-16T20:06:37.344 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.338+0000 7f294a7fc640 1 -- 172.21.15.88:0/3930905135 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294400cd70 con 0x7f29541584a0 2023-10-16T20:06:37.344 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.338+0000 7f295bf80640 1 -- 172.21.15.88:0/3930905135 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29540ac130 con 0x7f29541584a0 2023-10-16T20:06:37.344 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.338+0000 7f295bf80640 1 -- 172.21.15.88:0/3930905135 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29540ae540 con 0x7f29541584a0 2023-10-16T20:06:37.345 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.338+0000 7f295bf80640 1 -- 172.21.15.88:0/3930905135 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f295406d180 con 0x7f29541584a0 2023-10-16T20:06:37.345 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.338+0000 7f294a7fc640 1 -- 172.21.15.88:0/3930905135 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2944013470 con 0x7f29541584a0 2023-10-16T20:06:37.346 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.338+0000 7f294a7fc640 1 --2- 172.21.15.88:0/3930905135 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f29380664a0 0x7f2938068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:37.346 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.338+0000 7f294a7fc640 1 -- 172.21.15.88:0/3930905135 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f2944084270 con 0x7f29541584a0 2023-10-16T20:06:37.346 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.338+0000 7f2959d1c640 1 --2- 172.21.15.88:0/3930905135 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f29380664a0 0x7f2938068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:37.346 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.342+0000 7f2959d1c640 1 --2- 172.21.15.88:0/3930905135 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f29380664a0 0x7f2938068960 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2950001ae0 tx=0x7f2950001a70 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:37.347 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.342+0000 7f294a7fc640 1 -- 172.21.15.88:0/3930905135 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2944051530 con 0x7f29541584a0 2023-10-16T20:06:37.507 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.502+0000 7f295bf80640 1 -- 172.21.15.88:0/3930905135 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f295404e9c0 con 0x7f29541584a0 2023-10-16T20:06:37.507 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.502+0000 7f294a7fc640 1 -- 172.21.15.88:0/3930905135 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2944054b50 con 0x7f29541584a0 2023-10-16T20:06:37.507 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2023-10-16T20:06:37.509 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.506+0000 7f295bf80640 1 -- 172.21.15.88:0/3930905135 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f29380664a0 msgr2=0x7f2938068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:37.509 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.506+0000 7f295bf80640 1 --2- 172.21.15.88:0/3930905135 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f29380664a0 0x7f2938068960 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2950001ae0 tx=0x7f2950001a70 comp rx=0 tx=0).stop 2023-10-16T20:06:37.509 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.506+0000 7f295bf80640 1 -- 172.21.15.88:0/3930905135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29541584a0 msgr2=0x7f2954152750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:37.509 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.506+0000 7f295bf80640 1 --2- 172.21.15.88:0/3930905135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29541584a0 0x7f2954152750 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f294400e960 tx=0x7f294400ee20 comp rx=0 tx=0).stop 2023-10-16T20:06:37.509 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.506+0000 7f295bf80640 1 -- 172.21.15.88:0/3930905135 shutdown_connections 2023-10-16T20:06:37.510 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.506+0000 7f295bf80640 1 --2- 172.21.15.88:0/3930905135 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f29380664a0 0x7f2938068960 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:37.510 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.506+0000 7f295bf80640 1 --2- 172.21.15.88:0/3930905135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f295415e5a0 0x7f295414b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:37.510 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.506+0000 7f295bf80640 1 --2- 172.21.15.88:0/3930905135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29541584a0 0x7f2954152750 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:37.510 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.506+0000 7f295bf80640 1 --2- 172.21.15.88:0/3930905135 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29540cc5d0 0x7f2954154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:37.510 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.506+0000 7f295bf80640 1 -- 172.21.15.88:0/3930905135 >> 172.21.15.88:0/3930905135 conn(0x7f29540ce5e0 msgr2=0x7f29540cc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:37.511 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.506+0000 7f295bf80640 1 -- 172.21.15.88:0/3930905135 shutdown_connections 2023-10-16T20:06:37.511 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:37.506+0000 7f295bf80640 1 -- 172.21.15.88:0/3930905135 wait complete. 2023-10-16T20:06:37.527 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi088.front.sepia.ceph.com... 2023-10-16T20:06:37.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:37.528 DEBUG:teuthology.orchestra.run.smithi088:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-16T20:06:37.547 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[49414]: fuse finished with error 0 and tester_r 0 2023-10-16T20:06:37.550 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:06:43.652 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2023-10-16T20:06:43.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:43.653 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T20:06:43.664 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T20:06:43.855 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.850+0000 7f46d1c9e640 1 Processor -- start 2023-10-16T20:06:43.855 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.850+0000 7f46d1c9e640 1 -- start start 2023-10-16T20:06:43.856 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.854+0000 7f46d1c9e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46cc15d9f0 0x7f46cc161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:43.856 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.854+0000 7f46d1c9e640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46cc0b87e0 con 0x7f46cc0d26c0 2023-10-16T20:06:43.856 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.854+0000 7f46d1c9e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46cc15df10 con 0x7f46cc0d22e0 2023-10-16T20:06:43.857 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.854+0000 7f46d1c9e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc15e1f0 con 0x7f46cc15d9f0 2023-10-16T20:06:43.857 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.854+0000 7f46cbfff640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f46cc0d26c0 0x7f46cc0cf2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:38002/0 (socket says 172.21.15.88:38002) 2023-10-16T20:06:43.857 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.854+0000 7f46cbfff640 1 -- 172.21.15.88:0/357954860 learned_addr learned my addr 172.21.15.88:0/357954860 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:43.857 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.854+0000 7f46caffd640 1 --2- 172.21.15.88:0/357954860 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46cc15d9f0 0x7f46cc161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:43.858 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.854+0000 7f46ca7fc640 1 -- 172.21.15.88:0/357954860 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3147719771 0 0) 0x7f46cc0b87e0 con 0x7f46cc0d26c0 2023-10-16T20:06:43.858 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.854+0000 7f46ca7fc640 1 -- 172.21.15.88:0/357954860 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46b4003530 con 0x7f46cc0d26c0 2023-10-16T20:06:43.859 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.854+0000 7f46ca7fc640 1 -- 172.21.15.88:0/357954860 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3867542788 0 0) 0x7f46cc15df10 con 0x7f46cc0d22e0 2023-10-16T20:06:43.859 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.854+0000 7f46ca7fc640 1 -- 172.21.15.88:0/357954860 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46cc0b87e0 con 0x7f46cc0d22e0 2023-10-16T20:06:43.859 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.854+0000 7f46caffd640 1 -- 172.21.15.88:0/357954860 >> v1:172.21.15.94:6789/0 conn(0x7f46cc0d22e0 legacy=0x7f46cc15d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:43.859 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.854+0000 7f46caffd640 1 -- 172.21.15.88:0/357954860 >> v1:172.21.15.88:6789/0 conn(0x7f46cc0d26c0 legacy=0x7f46cc0cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:43.859 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.854+0000 7f46caffd640 1 -- 172.21.15.88:0/357954860 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46cc160970 con 0x7f46cc15d9f0 2023-10-16T20:06:43.861 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.858+0000 7f46caffd640 1 --2- 172.21.15.88:0/357954860 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46cc15d9f0 0x7f46cc161450 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f46c00097f0 tx=0x7f46c001c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61aa650a6cf6ff50 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:43.861 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.858+0000 7f46ca7fc640 1 -- 172.21.15.88:0/357954860 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c00ac640 con 0x7f46cc15d9f0 2023-10-16T20:06:43.861 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.858+0000 7f46ca7fc640 1 -- 172.21.15.88:0/357954860 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c00ac7a0 con 0x7f46cc15d9f0 2023-10-16T20:06:43.861 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.858+0000 7f46ca7fc640 1 -- 172.21.15.88:0/357954860 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c00acb30 con 0x7f46cc15d9f0 2023-10-16T20:06:43.861 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.858+0000 7f46d1c9e640 1 -- 172.21.15.88:0/357954860 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46cc15d9f0 msgr2=0x7f46cc161450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:43.862 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.858+0000 7f46d1c9e640 1 --2- 172.21.15.88:0/357954860 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46cc15d9f0 0x7f46cc161450 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f46c00097f0 tx=0x7f46c001c400 comp rx=0 tx=0).stop 2023-10-16T20:06:43.862 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.858+0000 7f46d1c9e640 1 -- 172.21.15.88:0/357954860 shutdown_connections 2023-10-16T20:06:43.862 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.858+0000 7f46d1c9e640 1 --2- 172.21.15.88:0/357954860 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46cc15d9f0 0x7f46cc161450 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:43.862 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.858+0000 7f46d1c9e640 1 -- 172.21.15.88:0/357954860 >> 172.21.15.88:0/357954860 conn(0x7f46cc0715c0 msgr2=0x7f46cc0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:43.862 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.858+0000 7f46d1c9e640 1 -- 172.21.15.88:0/357954860 shutdown_connections 2023-10-16T20:06:43.863 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.858+0000 7f46d1c9e640 1 -- 172.21.15.88:0/357954860 wait complete. 2023-10-16T20:06:43.865 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.858+0000 7f46d1c9e640 1 Processor -- start 2023-10-16T20:06:43.865 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.858+0000 7f46d1c9e640 1 -- start start 2023-10-16T20:06:43.865 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.862+0000 7f46d1c9e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46cc0d22e0 0x7f46cc14edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:43.865 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.862+0000 7f46d1c9e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46cc0d26c0 0x7f46cc1f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:43.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.862+0000 7f46d1c9e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46cc15d9f0 0x7f46cc0c33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:43.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.862+0000 7f46d1c9e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc1f61a0 con 0x7f46cc0d22e0 2023-10-16T20:06:43.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.862+0000 7f46d1c9e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc0c38e0 con 0x7f46cc0d26c0 2023-10-16T20:06:43.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.862+0000 7f46d1c9e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc0c3b80 con 0x7f46cc15d9f0 2023-10-16T20:06:43.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.862+0000 7f46cb7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46cc0d22e0 0x7f46cc14edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:43.867 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.862+0000 7f46cb7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46cc0d22e0 0x7f46cc14edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50984/0 (socket says 172.21.15.88:50984) 2023-10-16T20:06:43.867 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.862+0000 7f46cb7fe640 1 -- 172.21.15.88:0/3897522454 learned_addr learned my addr 172.21.15.88:0/3897522454 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:43.867 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.862+0000 7f46cbfff640 1 --2- 172.21.15.88:0/3897522454 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46cc15d9f0 0x7f46cc0c33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:43.867 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.862+0000 7f46caffd640 1 --2- 172.21.15.88:0/3897522454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46cc0d26c0 0x7f46cc1f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:43.867 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.862+0000 7f46cb7fe640 1 -- 172.21.15.88:0/3897522454 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46cc15d9f0 msgr2=0x7f46cc0c33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:43.868 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.862+0000 7f46cb7fe640 1 --2- 172.21.15.88:0/3897522454 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46cc15d9f0 0x7f46cc0c33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:43.868 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.862+0000 7f46cb7fe640 1 -- 172.21.15.88:0/3897522454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46cc0d26c0 msgr2=0x7f46cc1f5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:43.868 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.862+0000 7f46cb7fe640 1 --2- 172.21.15.88:0/3897522454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46cc0d26c0 0x7f46cc1f5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:43.868 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.862+0000 7f46cb7fe640 1 -- 172.21.15.88:0/3897522454 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c0009420 con 0x7f46cc0d22e0 2023-10-16T20:06:43.868 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.862+0000 7f46cbfff640 1 --2- 172.21.15.88:0/3897522454 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46cc15d9f0 0x7f46cc0c33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:43.869 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.862+0000 7f46caffd640 1 --2- 172.21.15.88:0/3897522454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46cc0d26c0 0x7f46cc1f5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:43.869 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.862+0000 7f46cb7fe640 1 --2- 172.21.15.88:0/3897522454 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46cc0d22e0 0x7f46cc14edb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00c490 tx=0x7f46bc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:43.869 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.862+0000 7f46b3fff640 1 -- 172.21.15.88:0/3897522454 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc0052d0 con 0x7f46cc0d22e0 2023-10-16T20:06:43.869 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.862+0000 7f46b3fff640 1 -- 172.21.15.88:0/3897522454 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46bc005430 con 0x7f46cc0d22e0 2023-10-16T20:06:43.869 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.862+0000 7f46b3fff640 1 -- 172.21.15.88:0/3897522454 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc012870 con 0x7f46cc0d22e0 2023-10-16T20:06:43.869 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.862+0000 7f46d1c9e640 1 -- 172.21.15.88:0/3897522454 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46cc0c3e60 con 0x7f46cc0d22e0 2023-10-16T20:06:43.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.862+0000 7f46d1c9e640 1 -- 172.21.15.88:0/3897522454 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46cc0c4420 con 0x7f46cc0d22e0 2023-10-16T20:06:43.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.862+0000 7f46d1c9e640 1 -- 172.21.15.88:0/3897522454 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46cc04e9c0 con 0x7f46cc0d22e0 2023-10-16T20:06:43.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.862+0000 7f46b3fff640 1 -- 172.21.15.88:0/3897522454 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f46bc0129d0 con 0x7f46cc0d22e0 2023-10-16T20:06:43.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.866+0000 7f46b3fff640 1 --2- 172.21.15.88:0/3897522454 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f46b80664a0 0x7f46b8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:43.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.866+0000 7f46b3fff640 1 -- 172.21.15.88:0/3897522454 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f46bc083d60 con 0x7f46cc0d22e0 2023-10-16T20:06:43.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.866+0000 7f46caffd640 1 --2- 172.21.15.88:0/3897522454 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f46b80664a0 0x7f46b8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:43.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.866+0000 7f46caffd640 1 --2- 172.21.15.88:0/3897522454 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f46b80664a0 0x7f46b8068960 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f46c00ae000 tx=0x7f46c00a6070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:43.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:43.870+0000 7f46b3fff640 1 -- 172.21.15.88:0/3897522454 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f46bc050ed0 con 0x7f46cc0d22e0 2023-10-16T20:06:44.054 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:44.050+0000 7f46d1c9e640 1 -- 172.21.15.88:0/3897522454 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f46cc0d0840 con 0x7f46cc0d22e0 2023-10-16T20:06:44.055 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:44.050+0000 7f46b3fff640 1 -- 172.21.15.88:0/3897522454 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v19) v1 ==== 53+0+97 (secure 0 0 0) 0x7f46bc0544f0 con 0x7f46cc0d22e0 2023-10-16T20:06:44.055 INFO:teuthology.orchestra.run.smithi088.stdout:name: testcephfs1, metadata pool: testcephfs1_metadata, data pools: [testcephfs1_data data_new ] 2023-10-16T20:06:44.057 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:44.054+0000 7f46d1c9e640 1 -- 172.21.15.88:0/3897522454 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f46b80664a0 msgr2=0x7f46b8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:44.057 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:44.054+0000 7f46d1c9e640 1 --2- 172.21.15.88:0/3897522454 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f46b80664a0 0x7f46b8068960 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f46c00ae000 tx=0x7f46c00a6070 comp rx=0 tx=0).stop 2023-10-16T20:06:44.057 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:44.054+0000 7f46d1c9e640 1 -- 172.21.15.88:0/3897522454 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46cc0d22e0 msgr2=0x7f46cc14edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:44.057 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:44.054+0000 7f46d1c9e640 1 --2- 172.21.15.88:0/3897522454 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46cc0d22e0 0x7f46cc14edb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00c490 tx=0x7f46bc00c950 comp rx=0 tx=0).stop 2023-10-16T20:06:44.058 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:44.054+0000 7f46d1c9e640 1 -- 172.21.15.88:0/3897522454 shutdown_connections 2023-10-16T20:06:44.058 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:44.054+0000 7f46d1c9e640 1 --2- 172.21.15.88:0/3897522454 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f46b80664a0 0x7f46b8068960 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:44.058 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:44.054+0000 7f46d1c9e640 1 --2- 172.21.15.88:0/3897522454 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46cc15d9f0 0x7f46cc0c33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:44.058 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:44.054+0000 7f46d1c9e640 1 --2- 172.21.15.88:0/3897522454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46cc0d26c0 0x7f46cc1f5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:44.058 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:44.054+0000 7f46d1c9e640 1 --2- 172.21.15.88:0/3897522454 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46cc0d22e0 0x7f46cc14edb0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:44.059 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:44.054+0000 7f46d1c9e640 1 -- 172.21.15.88:0/3897522454 >> 172.21.15.88:0/3897522454 conn(0x7f46cc0715c0 msgr2=0x7f46cc0b6860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:44.059 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:44.054+0000 7f46d1c9e640 1 -- 172.21.15.88:0/3897522454 shutdown_connections 2023-10-16T20:06:44.059 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:44.054+0000 7f46d1c9e640 1 -- 172.21.15.88:0/3897522454 wait complete. 2023-10-16T20:06:44.075 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T20:06:44.076 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T20:06:44.076 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi088.front.sepia.ceph.com 2023-10-16T20:06:44.077 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T20:06:44.077 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T20:06:44.077 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T20:06:44.078 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T20:06:44.078 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T20:06:44.078 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T20:06:44.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:44.079 DEBUG:teuthology.orchestra.run.smithi088:> ip addr 2023-10-16T20:06:44.089 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T20:06:44.089 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T20:06:44.089 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T20:06:44.090 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:06:44.090 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2023-10-16T20:06:44.090 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:06:44.090 INFO:teuthology.orchestra.run.smithi088.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:06:44.090 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f0 brd ff:ff:ff:ff:ff:ff 2023-10-16T20:06:44.091 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f0 2023-10-16T20:06:44.091 INFO:teuthology.orchestra.run.smithi088.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:06:44.091 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f1 brd ff:ff:ff:ff:ff:ff 2023-10-16T20:06:44.091 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f1 2023-10-16T20:06:44.091 INFO:teuthology.orchestra.run.smithi088.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:06:44.091 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2c brd ff:ff:ff:ff:ff:ff 2023-10-16T20:06:44.092 INFO:teuthology.orchestra.run.smithi088.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T20:06:44.092 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2d brd ff:ff:ff:ff:ff:ff 2023-10-16T20:06:44.092 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.21.15.88/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-16T20:06:44.092 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft 42328sec preferred_lft 42328sec 2023-10-16T20:06:44.092 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link 2023-10-16T20:06:44.093 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:06:44.093 INFO:teuthology.orchestra.run.smithi088.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-16T20:06:44.093 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 02:42:d9:3b:fb:39 brd ff:ff:ff:ff:ff:ff 2023-10-16T20:06:44.093 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-16T20:06:44.093 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:06:44.094 INFO:teuthology.orchestra.run.smithi088.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T20:06:44.094 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-16T20:06:44.094 INFO:teuthology.orchestra.run.smithi088.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T20:06:44.094 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:06:44.094 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-16T20:06:44.095 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:06:44.095 INFO:teuthology.orchestra.run.smithi088.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T20:06:44.095 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:06:44.095 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-16T20:06:44.095 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:06:44.095 INFO:teuthology.orchestra.run.smithi088.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T20:06:44.096 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:06:44.096 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-16T20:06:44.096 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:06:44.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:44.097 DEBUG:teuthology.orchestra.run.smithi088:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T20:06:44.149 INFO:teuthology.orchestra.run.smithi088.stdout:1 2023-10-16T20:06:44.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:44.152 DEBUG:teuthology.orchestra.run.smithi088:> route 2023-10-16T20:06:44.206 INFO:teuthology.orchestra.run.smithi088.stdout:Kernel IP routing table 2023-10-16T20:06:44.206 INFO:teuthology.orchestra.run.smithi088.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T20:06:44.206 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-16T20:06:44.206 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T20:06:44.207 INFO:teuthology.orchestra.run.smithi088.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-16T20:06:44.207 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T20:06:44.207 INFO:teuthology.orchestra.run.smithi088.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:06:44.207 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:06:44.208 INFO:teuthology.orchestra.run.smithi088.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:06:44.208 INFO:teuthology.orchestra.run.smithi088.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T20:06:44.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:44.209 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec bash -c ' 2023-10-16T20:06:44.209 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2023-10-16T20:06:44.209 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T20:06:44.210 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T20:06:44.210 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T20:06:44.210 DEBUG:teuthology.orchestra.run.smithi088:> ') 2023-10-16T20:06:44.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:44.300 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2023-10-16T20:06:44.310 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T20:06:44.311 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T20:06:44.311 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T20:06:44.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:44.312 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set brx.0 up 2023-10-16T20:06:44.371 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-16T20:06:44.371 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T20:06:44.372 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T20:06:44.423 INFO:teuthology.orchestra.run.smithi088.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T20:06:44.424 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T20:06:44.424 DEBUG:teuthology.orchestra.run.smithi088:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T20:06:44.475 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2023-10-16T20:06:44.540 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T20:06:44.592 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2023-10-16T20:06:44.593 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2023-10-16T20:06:44.593 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2023-10-16T20:06:44.593 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2023-10-16T20:06:44.593 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2023-10-16T20:06:44.593 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2023-10-16T20:06:44.594 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2023-10-16T20:06:44.594 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2023-10-16T20:06:44.594 INFO:teuthology.orchestra.run.smithi088.stdout:/run/lock 2023-10-16T20:06:44.594 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2023-10-16T20:06:44.594 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2023-10-16T20:06:44.595 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2023-10-16T20:06:44.595 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T20:06:44.595 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2023-10-16T20:06:44.595 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2023-10-16T20:06:44.595 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2023-10-16T20:06:44.595 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2023-10-16T20:06:44.596 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/fs/nfsd 2023-10-16T20:06:44.596 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2023-10-16T20:06:44.596 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2023-10-16T20:06:44.596 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T20:06:44.596 INFO:teuthology.orchestra.run.smithi088.stdout:/run/rpc_pipefs 2023-10-16T20:06:44.597 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2023-10-16T20:06:44.597 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2023-10-16T20:06:44.597 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T20:06:44.597 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T20:06:44.597 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T20:06:44.597 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T20:06:44.598 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2023-10-16T20:06:44.598 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:06:44.598 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:06:44.598 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:06:44.598 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:06:44.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:44.600 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2023-10-16T20:06:44.652 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-16T20:06:44.653 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-16T20:06:44.698 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2023-10-16T20:06:44.721 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T20:06:44.773 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2023-10-16T20:06:44.773 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2023-10-16T20:06:44.773 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2023-10-16T20:06:44.774 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2023-10-16T20:06:44.774 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2023-10-16T20:06:44.774 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2023-10-16T20:06:44.774 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2023-10-16T20:06:44.774 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2023-10-16T20:06:44.775 INFO:teuthology.orchestra.run.smithi088.stdout:/run/lock 2023-10-16T20:06:44.775 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2023-10-16T20:06:44.775 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2023-10-16T20:06:44.775 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2023-10-16T20:06:44.776 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T20:06:44.776 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2023-10-16T20:06:44.776 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2023-10-16T20:06:44.777 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2023-10-16T20:06:44.777 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2023-10-16T20:06:44.777 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/fs/nfsd 2023-10-16T20:06:44.777 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2023-10-16T20:06:44.777 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2023-10-16T20:06:44.778 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T20:06:44.778 INFO:teuthology.orchestra.run.smithi088.stdout:/run/rpc_pipefs 2023-10-16T20:06:44.778 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2023-10-16T20:06:44.779 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2023-10-16T20:06:44.779 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T20:06:44.779 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T20:06:44.780 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T20:06:44.780 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T20:06:44.780 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2023-10-16T20:06:44.781 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:06:44.781 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:06:44.781 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:06:44.782 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:06:44.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:44.784 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2023-10-16T20:06:44.792 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:2023-10-16T20:06:44.790+0000 7fa66a168580 -1 init, newargv = 0x55d8e457fc50 newargc=15 2023-10-16T20:06:44.793 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[49978]: starting ceph client 2023-10-16T20:06:44.800 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[49978]: starting fuse 2023-10-16T20:06:44.801 INFO:teuthology.orchestra.run.smithi088.stdout:43 2023-10-16T20:06:44.802 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-10-16T20:06:44.802 DEBUG:teuthology.orchestra.run.smithi088:> sudo stdin-killer -- python3 -c ' 2023-10-16T20:06:44.803 DEBUG:teuthology.orchestra.run.smithi088:> import glob 2023-10-16T20:06:44.803 DEBUG:teuthology.orchestra.run.smithi088:> import re 2023-10-16T20:06:44.803 DEBUG:teuthology.orchestra.run.smithi088:> import os 2023-10-16T20:06:44.804 DEBUG:teuthology.orchestra.run.smithi088:> import subprocess 2023-10-16T20:06:44.804 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:06:44.805 DEBUG:teuthology.orchestra.run.smithi088:> def _find_admin_socket(client_name): 2023-10-16T20:06:44.805 DEBUG:teuthology.orchestra.run.smithi088:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-16T20:06:44.806 DEBUG:teuthology.orchestra.run.smithi088:> files = glob.glob(asok_path) 2023-10-16T20:06:44.806 DEBUG:teuthology.orchestra.run.smithi088:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-16T20:06:44.806 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:06:44.807 DEBUG:teuthology.orchestra.run.smithi088:> # Given a non-glob path, it better be there 2023-10-16T20:06:44.807 DEBUG:teuthology.orchestra.run.smithi088:> if "*" not in asok_path: 2023-10-16T20:06:44.808 DEBUG:teuthology.orchestra.run.smithi088:> assert(len(files) == 1) 2023-10-16T20:06:44.808 DEBUG:teuthology.orchestra.run.smithi088:> return files[0] 2023-10-16T20:06:44.808 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:06:44.809 DEBUG:teuthology.orchestra.run.smithi088:> for f in files: 2023-10-16T20:06:44.809 DEBUG:teuthology.orchestra.run.smithi088:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-16T20:06:44.810 DEBUG:teuthology.orchestra.run.smithi088:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-16T20:06:44.810 DEBUG:teuthology.orchestra.run.smithi088:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-16T20:06:44.810 DEBUG:teuthology.orchestra.run.smithi088:> contents = proc_f.read() 2023-10-16T20:06:44.811 DEBUG:teuthology.orchestra.run.smithi088:> if mountpoint in contents: 2023-10-16T20:06:44.811 DEBUG:teuthology.orchestra.run.smithi088:> return f 2023-10-16T20:06:44.813 DEBUG:teuthology.orchestra.run.smithi088:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-16T20:06:44.813 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:06:44.813 DEBUG:teuthology.orchestra.run.smithi088:> print(_find_admin_socket("client.0")) 2023-10-16T20:06:44.814 DEBUG:teuthology.orchestra.run.smithi088:> ' 2023-10-16T20:06:44.923 INFO:teuthology.orchestra.run.smithi088.stdout:/var/run/ceph/ceph-client.0.49978.asok 2023-10-16T20:06:44.926 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-16T20:06:44.934 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.49978.asok 2023-10-16T20:06:44.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:44.935 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.49978.asok status 2023-10-16T20:06:45.106 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2023-10-16T20:06:45.107 INFO:teuthology.orchestra.run.smithi088.stdout: "metadata": { 2023-10-16T20:06:45.107 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_sha1": "4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0", 2023-10-16T20:06:45.107 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_version": "ceph version 18.0.0-6730-g4ba8a190 (4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0) reef (dev)", 2023-10-16T20:06:45.108 INFO:teuthology.orchestra.run.smithi088.stdout: "entity_id": "0", 2023-10-16T20:06:45.108 INFO:teuthology.orchestra.run.smithi088.stdout: "hostname": "smithi088", 2023-10-16T20:06:45.108 INFO:teuthology.orchestra.run.smithi088.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-16T20:06:45.108 INFO:teuthology.orchestra.run.smithi088.stdout: "pid": "49978", 2023-10-16T20:06:45.109 INFO:teuthology.orchestra.run.smithi088.stdout: "root": "/" 2023-10-16T20:06:45.109 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:45.109 INFO:teuthology.orchestra.run.smithi088.stdout: "dentry_count": 0, 2023-10-16T20:06:45.109 INFO:teuthology.orchestra.run.smithi088.stdout: "dentry_pinned_count": 0, 2023-10-16T20:06:45.109 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 5874, 2023-10-16T20:06:45.110 INFO:teuthology.orchestra.run.smithi088.stdout: "inst": { 2023-10-16T20:06:45.110 INFO:teuthology.orchestra.run.smithi088.stdout: "name": { 2023-10-16T20:06:45.110 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "client", 2023-10-16T20:06:45.110 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 5874 2023-10-16T20:06:45.111 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:45.111 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": { 2023-10-16T20:06:45.111 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2023-10-16T20:06:45.111 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "192.168.0.1:0", 2023-10-16T20:06:45.112 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 2138968541 2023-10-16T20:06:45.112 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:06:45.112 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:45.113 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": { 2023-10-16T20:06:45.113 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2023-10-16T20:06:45.113 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "192.168.0.1:0", 2023-10-16T20:06:45.113 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 2138968541 2023-10-16T20:06:45.114 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:45.114 INFO:teuthology.orchestra.run.smithi088.stdout: "inst_str": "client.5874 192.168.0.1:0/2138968541", 2023-10-16T20:06:45.114 INFO:teuthology.orchestra.run.smithi088.stdout: "addr_str": "192.168.0.1:0/2138968541", 2023-10-16T20:06:45.115 INFO:teuthology.orchestra.run.smithi088.stdout: "inode_count": 1, 2023-10-16T20:06:45.115 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_epoch": 19, 2023-10-16T20:06:45.115 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_epoch": 46, 2023-10-16T20:06:45.115 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_epoch_barrier": 0, 2023-10-16T20:06:45.116 INFO:teuthology.orchestra.run.smithi088.stdout: "blocklisted": false, 2023-10-16T20:06:45.116 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "testcephfs1" 2023-10-16T20:06:45.116 INFO:teuthology.orchestra.run.smithi088.stdout:} 2023-10-16T20:06:45.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:45.123 DEBUG:teuthology.orchestra.run.smithi088:> stat --file-system '--printf=%T 2023-10-16T20:06:45.123 DEBUG:teuthology.orchestra.run.smithi088:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-16T20:06:45.133 INFO:teuthology.orchestra.run.smithi088.stdout:fuseblk 2023-10-16T20:06:45.133 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-16T20:06:45.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:45.134 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T20:06:45.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:45.195 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.pool -v data_new layoutfile) 2023-10-16T20:06:45.250 INFO:teuthology.orchestra.run.smithi088.stderr:setfattr: layoutfile: Permission denied 2023-10-16T20:06:45.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T20:06:45.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:45.253 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v data_new layoutdir) 2023-10-16T20:06:45.310 INFO:teuthology.orchestra.run.smithi088.stderr:setfattr: layoutdir: Permission denied 2023-10-16T20:06:45.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T20:06:45.312 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:06:45.312 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:06:45.363 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:06:45.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.538+0000 7f32e946b640 1 Processor -- start 2023-10-16T20:06:45.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.542+0000 7f32e946b640 1 -- start start 2023-10-16T20:06:45.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.542+0000 7f32e946b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32e415cc60 0x7f32e40caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:45.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.542+0000 7f32e946b640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32e40ca710 con 0x7f32e4071450 2023-10-16T20:06:45.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.542+0000 7f32e946b640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32e40c9de0 con 0x7f32e40c40d0 2023-10-16T20:06:45.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.542+0000 7f32e946b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e40ca0c0 con 0x7f32e415cc60 2023-10-16T20:06:45.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.542+0000 7f32da7fc640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f32e4071450 0x7f32e40c39c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:38012/0 (socket says 172.21.15.88:38012) 2023-10-16T20:06:45.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.542+0000 7f32da7fc640 1 -- 172.21.15.88:0/2559568730 learned_addr learned my addr 172.21.15.88:0/2559568730 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:45.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.542+0000 7f32e2ffd640 1 --2- 172.21.15.88:0/2559568730 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32e415cc60 0x7f32e40caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:45.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.542+0000 7f32e27fc640 1 -- 172.21.15.88:0/2559568730 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 98515322 0 0) 0x7f32e40ca710 con 0x7f32e4071450 2023-10-16T20:06:45.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.542+0000 7f32e27fc640 1 -- 172.21.15.88:0/2559568730 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32d0003530 con 0x7f32e4071450 2023-10-16T20:06:45.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.542+0000 7f32e27fc640 1 -- 172.21.15.88:0/2559568730 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1320461044 0 0) 0x7f32d0003530 con 0x7f32e4071450 2023-10-16T20:06:45.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.542+0000 7f32e27fc640 1 -- 172.21.15.88:0/2559568730 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32e415cc60 msgr2=0x7f32e40caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:45.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.542+0000 7f32e27fc640 1 --2- 172.21.15.88:0/2559568730 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32e415cc60 0x7f32e40caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:45.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.542+0000 7f32e27fc640 1 -- 172.21.15.88:0/2559568730 >> v1:172.21.15.94:6789/0 conn(0x7f32e40c40d0 legacy=0x7f32e4070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:45.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.542+0000 7f32e27fc640 1 -- 172.21.15.88:0/2559568730 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e40ca3a0 con 0x7f32e4071450 2023-10-16T20:06:45.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.542+0000 7f32e27fc640 1 -- 172.21.15.88:0/2559568730 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f32d4001680 con 0x7f32e4071450 2023-10-16T20:06:45.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.542+0000 7f32e27fc640 1 -- 172.21.15.88:0/2559568730 >> v1:172.21.15.88:6789/0 conn(0x7f32e4071450 legacy=0x7f32e40c39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:45.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.542+0000 7f32e27fc640 1 --2- 172.21.15.88:0/2559568730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32d00039b0 0x7f32d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:45.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.542+0000 7f32e27fc640 1 --2- 172.21.15.88:0/2559568730 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32e415cc60 0x7f32d00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:45.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.542+0000 7f32e27fc640 1 --2- 172.21.15.88:0/2559568730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32d0004d10 0x7f32d0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:45.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.542+0000 7f32e27fc640 1 -- 172.21.15.88:0/2559568730 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e40ca0c0 con 0x7f32d0004d10 2023-10-16T20:06:45.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.542+0000 7f32e27fc640 1 -- 172.21.15.88:0/2559568730 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d000ba40 con 0x7f32d00039b0 2023-10-16T20:06:45.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.542+0000 7f32e27fc640 1 -- 172.21.15.88:0/2559568730 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d000bc00 con 0x7f32e415cc60 2023-10-16T20:06:45.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.542+0000 7f32e37fe640 1 --2- 172.21.15.88:0/2559568730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32d0004d10 0x7f32d0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:45.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.542+0000 7f32da7fc640 1 --2- 172.21.15.88:0/2559568730 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32e415cc60 0x7f32d00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:45.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.542+0000 7f32e2ffd640 1 --2- 172.21.15.88:0/2559568730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32d00039b0 0x7f32d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:45.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.542+0000 7f32e37fe640 1 -- 172.21.15.88:0/2559568730 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32e415cc60 msgr2=0x7f32d00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:45.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.542+0000 7f32e37fe640 1 --2- 172.21.15.88:0/2559568730 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32e415cc60 0x7f32d00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:45.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.542+0000 7f32e37fe640 1 -- 172.21.15.88:0/2559568730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32d00039b0 msgr2=0x7f32d0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:45.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.542+0000 7f32e37fe640 1 --2- 172.21.15.88:0/2559568730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32d00039b0 0x7f32d0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:45.552 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.542+0000 7f32e37fe640 1 -- 172.21.15.88:0/2559568730 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d0003530 con 0x7f32d0004d10 2023-10-16T20:06:45.554 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.546+0000 7f32e37fe640 1 --2- 172.21.15.88:0/2559568730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32d0004d10 0x7f32d0005190 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f32dc001e40 tx=0x7f32dc01c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f8b4bbe66120b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:45.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.546+0000 7f32e27fc640 1 -- 172.21.15.88:0/2559568730 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc003b80 con 0x7f32d0004d10 2023-10-16T20:06:45.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.546+0000 7f32e27fc640 1 -- 172.21.15.88:0/2559568730 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32dc003ce0 con 0x7f32d0004d10 2023-10-16T20:06:45.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.546+0000 7f32e27fc640 1 -- 172.21.15.88:0/2559568730 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0052c0 con 0x7f32d0004d10 2023-10-16T20:06:45.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.546+0000 7f32e946b640 1 -- 172.21.15.88:0/2559568730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32d0004d10 msgr2=0x7f32d0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:45.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.546+0000 7f32e946b640 1 --2- 172.21.15.88:0/2559568730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32d0004d10 0x7f32d0005190 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f32dc001e40 tx=0x7f32dc01c740 comp rx=0 tx=0).stop 2023-10-16T20:06:45.556 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.546+0000 7f32e37fe640 1 -- 172.21.15.88:0/2559568730 reap_dead start 2023-10-16T20:06:45.556 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.546+0000 7f32e946b640 1 -- 172.21.15.88:0/2559568730 shutdown_connections 2023-10-16T20:06:45.556 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.546+0000 7f32e946b640 1 -- 172.21.15.88:0/2559568730 >> 172.21.15.88:0/2559568730 conn(0x7f32e40ceb20 msgr2=0x7f32e40cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:45.556 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.546+0000 7f32e946b640 1 -- 172.21.15.88:0/2559568730 shutdown_connections 2023-10-16T20:06:45.556 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.546+0000 7f32e946b640 1 -- 172.21.15.88:0/2559568730 wait complete. 2023-10-16T20:06:45.557 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.550+0000 7f32e946b640 1 Processor -- start 2023-10-16T20:06:45.557 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.550+0000 7f32e946b640 1 -- start start 2023-10-16T20:06:45.557 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.550+0000 7f32e946b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32d0004d10 0x7f32e414fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:45.557 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.550+0000 7f32e946b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32e406c810 0x7f32e406d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:45.557 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.550+0000 7f32e946b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32e40c6620 0x7f32e40c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:45.558 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.550+0000 7f32e946b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e40c6fe0 con 0x7f32e406c810 2023-10-16T20:06:45.558 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.550+0000 7f32e946b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e41586e0 con 0x7f32e40c6620 2023-10-16T20:06:45.558 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.550+0000 7f32e946b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e406df80 con 0x7f32d0004d10 2023-10-16T20:06:45.558 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.550+0000 7f32da7fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32e406c810 0x7f32e406d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:45.558 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.550+0000 7f32da7fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32e406c810 0x7f32e406d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50994/0 (socket says 172.21.15.88:50994) 2023-10-16T20:06:45.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.550+0000 7f32da7fc640 1 -- 172.21.15.88:0/2911834900 learned_addr learned my addr 172.21.15.88:0/2911834900 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:45.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.550+0000 7f32e2ffd640 1 --2- 172.21.15.88:0/2911834900 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32d0004d10 0x7f32e414fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:45.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.550+0000 7f32e37fe640 1 --2- 172.21.15.88:0/2911834900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32e40c6620 0x7f32e40c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:45.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.550+0000 7f32da7fc640 1 -- 172.21.15.88:0/2911834900 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32d0004d10 msgr2=0x7f32e414fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:45.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.550+0000 7f32da7fc640 1 --2- 172.21.15.88:0/2911834900 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32d0004d10 0x7f32e414fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:45.560 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.550+0000 7f32da7fc640 1 -- 172.21.15.88:0/2911834900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32e40c6620 msgr2=0x7f32e40c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:45.560 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.550+0000 7f32da7fc640 1 --2- 172.21.15.88:0/2911834900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32e40c6620 0x7f32e40c6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:45.560 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.550+0000 7f32da7fc640 1 -- 172.21.15.88:0/2911834900 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e4158930 con 0x7f32e406c810 2023-10-16T20:06:45.560 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.550+0000 7f32da7fc640 1 --2- 172.21.15.88:0/2911834900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32e406c810 0x7f32e406d700 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f32d40072c0 tx=0x7f32d40076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:45.560 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.550+0000 7f32dbfff640 1 -- 172.21.15.88:0/2911834900 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d400f9c0 con 0x7f32e406c810 2023-10-16T20:06:45.560 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.550+0000 7f32dbfff640 1 -- 172.21.15.88:0/2911834900 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d400fb20 con 0x7f32e406c810 2023-10-16T20:06:45.561 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.550+0000 7f32e946b640 1 -- 172.21.15.88:0/2911834900 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32e4158c10 con 0x7f32e406c810 2023-10-16T20:06:45.561 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.550+0000 7f32dbfff640 1 -- 172.21.15.88:0/2911834900 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d400fed0 con 0x7f32e406c810 2023-10-16T20:06:45.561 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.550+0000 7f32e946b640 1 -- 172.21.15.88:0/2911834900 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32e41502a0 con 0x7f32e406c810 2023-10-16T20:06:45.562 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.550+0000 7f32e946b640 1 -- 172.21.15.88:0/2911834900 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32e404e9c0 con 0x7f32e406c810 2023-10-16T20:06:45.563 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.554+0000 7f32dbfff640 1 -- 172.21.15.88:0/2911834900 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f32d401d460 con 0x7f32e406c810 2023-10-16T20:06:45.563 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.554+0000 7f32dbfff640 1 --2- 172.21.15.88:0/2911834900 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f32c40664a0 0x7f32c4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:45.563 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.554+0000 7f32dbfff640 1 -- 172.21.15.88:0/2911834900 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f32d40842f0 con 0x7f32e406c810 2023-10-16T20:06:45.563 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.554+0000 7f32e2ffd640 1 --2- 172.21.15.88:0/2911834900 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f32c40664a0 0x7f32c4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:45.564 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.554+0000 7f32e2ffd640 1 --2- 172.21.15.88:0/2911834900 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f32c40664a0 0x7f32c4068960 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f32cc009be0 tx=0x7f32cc009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:45.564 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.558+0000 7f32dbfff640 1 -- 172.21.15.88:0/2911834900 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f32d40513d0 con 0x7f32e406c810 2023-10-16T20:06:45.711 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.706+0000 7f32e946b640 1 -- 172.21.15.88:0/2911834900 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32e41590c0 con 0x7f32e406c810 2023-10-16T20:06:45.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.710+0000 7f32dbfff640 1 -- 172.21.15.88:0/2911834900 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+18499 (secure 0 0 0) 0x7f32d40549f0 con 0x7f32e406c810 2023-10-16T20:06:45.712 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:06:45.713 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":46,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:06:38.250351+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T20:06:14.743500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"testcephfs1_data","create_time":"2023-10-16T20:06:16.091837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-10-16T20:06:31.184462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.39","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:06:45.714 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.710+0000 7f32e946b640 1 -- 172.21.15.88:0/2911834900 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f32c40664a0 msgr2=0x7f32c4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:45.715 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.710+0000 7f32e946b640 1 --2- 172.21.15.88:0/2911834900 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f32c40664a0 0x7f32c4068960 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f32cc009be0 tx=0x7f32cc009190 comp rx=0 tx=0).stop 2023-10-16T20:06:45.715 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.710+0000 7f32e946b640 1 -- 172.21.15.88:0/2911834900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32e406c810 msgr2=0x7f32e406d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:45.715 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.710+0000 7f32e946b640 1 --2- 172.21.15.88:0/2911834900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32e406c810 0x7f32e406d700 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f32d40072c0 tx=0x7f32d40076c0 comp rx=0 tx=0).stop 2023-10-16T20:06:45.715 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.710+0000 7f32e946b640 1 -- 172.21.15.88:0/2911834900 shutdown_connections 2023-10-16T20:06:45.716 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.710+0000 7f32e946b640 1 --2- 172.21.15.88:0/2911834900 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f32c40664a0 0x7f32c4068960 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:45.716 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.710+0000 7f32e946b640 1 --2- 172.21.15.88:0/2911834900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32e40c6620 0x7f32e40c6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:45.716 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.710+0000 7f32e946b640 1 --2- 172.21.15.88:0/2911834900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32e406c810 0x7f32e406d700 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:45.716 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.710+0000 7f32e946b640 1 --2- 172.21.15.88:0/2911834900 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f32d0004d10 0x7f32e414fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:45.716 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.710+0000 7f32e946b640 1 -- 172.21.15.88:0/2911834900 >> 172.21.15.88:0/2911834900 conn(0x7f32e40ceb20 msgr2=0x7f32e40cd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:45.716 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.710+0000 7f32e946b640 1 -- 172.21.15.88:0/2911834900 shutdown_connections 2023-10-16T20:06:45.717 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.710+0000 7f32e946b640 1 -- 172.21.15.88:0/2911834900 wait complete. 2023-10-16T20:06:45.734 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'testcephfs1_metadata', 'create_time': '2023-10-16T20:06:14.743500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'testcephfs1_data', 'create_time': '2023-10-16T20:06:16.091837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2023-10-16T20:06:31.184462+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T20:06:45.735 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:06:45.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.874+0000 7fb508f9d640 1 Processor -- start 2023-10-16T20:06:45.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.874+0000 7fb508f9d640 1 -- start start 2023-10-16T20:06:45.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.874+0000 7fb508f9d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb50415cb70 0x7fb5040b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:45.878 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.874+0000 7fb508f9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb5040b4530 0x7fb5040b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:45.878 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.874+0000 7fb508f9d640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5040b4ed0 con 0x7fb5040d0bd0 2023-10-16T20:06:45.878 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.874+0000 7fb508f9d640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5040b7d20 con 0x7fb5040b4530 2023-10-16T20:06:45.878 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.874+0000 7fb508f9d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5040b7110 con 0x7fb50415cb70 2023-10-16T20:06:45.879 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.874+0000 7fb50259c640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb50415cb70 0x7fb5040b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:45.879 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.874+0000 7fb50259c640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb50415cb70 0x7fb5040b83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:44562/0 (socket says 172.21.15.88:44562) 2023-10-16T20:06:45.879 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.874+0000 7fb50259c640 1 -- 172.21.15.88:0/3987583638 learned_addr learned my addr 172.21.15.88:0/3987583638 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:45.879 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.874+0000 7fb4f9d9b640 1 --2- 172.21.15.88:0/3987583638 >> v2:172.21.15.94:3300/0 conn(0x7fb5040b4530 0x7fb5040b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:45.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.874+0000 7fb50259c640 1 -- 172.21.15.88:0/3987583638 >> v2:172.21.15.94:3300/0 conn(0x7fb5040b4530 msgr2=0x7fb5040b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:45.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.874+0000 7fb50259c640 1 --2- 172.21.15.88:0/3987583638 >> v2:172.21.15.94:3300/0 conn(0x7fb5040b4530 0x7fb5040b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:45.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.874+0000 7fb50259c640 1 -- 172.21.15.88:0/3987583638 >> v1:172.21.15.94:6789/0 conn(0x7fb5040d0bd0 legacy=0x7fb5040d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:45.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.874+0000 7fb50259c640 1 -- 172.21.15.88:0/3987583638 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5040b73f0 con 0x7fb50415cb70 2023-10-16T20:06:45.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.874+0000 7fb501d9b640 1 -- 172.21.15.88:0/3987583638 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3571706294 0 0) 0x7fb5040b4ed0 con 0x7fb5040d0bd0 2023-10-16T20:06:45.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.878+0000 7fb50259c640 1 --2- 172.21.15.88:0/3987583638 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb50415cb70 0x7fb5040b83d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009870 tx=0x7fb4f401ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3962a825129047db server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:45.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.878+0000 7fb501d9b640 1 -- 172.21.15.88:0/3987583638 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f40a8a30 con 0x7fb50415cb70 2023-10-16T20:06:45.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.878+0000 7fb501d9b640 1 -- 172.21.15.88:0/3987583638 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f40a8b90 con 0x7fb50415cb70 2023-10-16T20:06:45.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.878+0000 7fb501d9b640 1 -- 172.21.15.88:0/3987583638 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f40b0440 con 0x7fb50415cb70 2023-10-16T20:06:45.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.878+0000 7fb508f9d640 1 -- 172.21.15.88:0/3987583638 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb50415cb70 msgr2=0x7fb5040b83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:45.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.878+0000 7fb508f9d640 1 --2- 172.21.15.88:0/3987583638 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb50415cb70 0x7fb5040b83d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009870 tx=0x7fb4f401ed30 comp rx=0 tx=0).stop 2023-10-16T20:06:45.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.878+0000 7fb508f9d640 1 -- 172.21.15.88:0/3987583638 shutdown_connections 2023-10-16T20:06:45.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.878+0000 7fb508f9d640 1 --2- 172.21.15.88:0/3987583638 >> v2:172.21.15.94:3300/0 conn(0x7fb5040b4530 0x7fb5040b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:45.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.878+0000 7fb508f9d640 1 --2- 172.21.15.88:0/3987583638 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb50415cb70 0x7fb5040b83d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:45.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.878+0000 7fb508f9d640 1 -- 172.21.15.88:0/3987583638 >> 172.21.15.88:0/3987583638 conn(0x7fb50406b6e0 msgr2=0x7fb50406baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:45.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.878+0000 7fb508f9d640 1 -- 172.21.15.88:0/3987583638 shutdown_connections 2023-10-16T20:06:45.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.878+0000 7fb508f9d640 1 -- 172.21.15.88:0/3987583638 wait complete. 2023-10-16T20:06:45.887 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.882+0000 7fb508f9d640 1 Processor -- start 2023-10-16T20:06:45.887 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.882+0000 7fb508f9d640 1 -- start start 2023-10-16T20:06:45.887 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.882+0000 7fb508f9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5040b4530 0x7fb50415fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:45.887 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.882+0000 7fb508f9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5040d0bd0 0x7fb50414fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:45.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.882+0000 7fb508f9d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb50415cb70 0x7fb504150400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:45.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.882+0000 7fb508f9d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50414ed30 con 0x7fb5040b4530 2023-10-16T20:06:45.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.882+0000 7fb508f9d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb504150940 con 0x7fb5040d0bd0 2023-10-16T20:06:45.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.882+0000 7fb508f9d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50414f150 con 0x7fb50415cb70 2023-10-16T20:06:45.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.882+0000 7fb50259c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5040b4530 0x7fb50415fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:45.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.882+0000 7fb50259c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5040b4530 0x7fb50415fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51006/0 (socket says 172.21.15.88:51006) 2023-10-16T20:06:45.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.882+0000 7fb50259c640 1 -- 172.21.15.88:0/2510248913 learned_addr learned my addr 172.21.15.88:0/2510248913 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:45.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.882+0000 7fb4f9d9b640 1 --2- 172.21.15.88:0/2510248913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5040d0bd0 0x7fb50414fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:45.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.882+0000 7fb502d9d640 1 --2- 172.21.15.88:0/2510248913 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb50415cb70 0x7fb504150400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:45.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.882+0000 7fb50259c640 1 -- 172.21.15.88:0/2510248913 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb50415cb70 msgr2=0x7fb504150400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:45.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.882+0000 7fb50259c640 1 --2- 172.21.15.88:0/2510248913 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb50415cb70 0x7fb504150400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:45.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.882+0000 7fb50259c640 1 -- 172.21.15.88:0/2510248913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5040d0bd0 msgr2=0x7fb50414fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:45.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.882+0000 7fb50259c640 1 --2- 172.21.15.88:0/2510248913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5040d0bd0 0x7fb50414fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:45.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.882+0000 7fb50259c640 1 -- 172.21.15.88:0/2510248913 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f40094d0 con 0x7fb5040b4530 2023-10-16T20:06:45.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.882+0000 7fb50259c640 1 --2- 172.21.15.88:0/2510248913 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5040b4530 0x7fb50415fb20 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009960 tx=0x7fb4f401d570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:45.891 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.882+0000 7fb4fb7fe640 1 -- 172.21.15.88:0/2510248913 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f40b1de0 con 0x7fb5040b4530 2023-10-16T20:06:45.891 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.882+0000 7fb4fb7fe640 1 -- 172.21.15.88:0/2510248913 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f4004030 con 0x7fb5040b4530 2023-10-16T20:06:45.891 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.882+0000 7fb4fb7fe640 1 -- 172.21.15.88:0/2510248913 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f40043e0 con 0x7fb5040b4530 2023-10-16T20:06:45.891 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.882+0000 7fb508f9d640 1 -- 172.21.15.88:0/2510248913 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb50414e0b0 con 0x7fb5040b4530 2023-10-16T20:06:45.891 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.882+0000 7fb508f9d640 1 -- 172.21.15.88:0/2510248913 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb50414e670 con 0x7fb5040b4530 2023-10-16T20:06:45.892 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.886+0000 7fb508f9d640 1 -- 172.21.15.88:0/2510248913 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb50404e9c0 con 0x7fb5040b4530 2023-10-16T20:06:45.893 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.886+0000 7fb4fb7fe640 1 -- 172.21.15.88:0/2510248913 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb4f40b7040 con 0x7fb5040b4530 2023-10-16T20:06:45.893 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.886+0000 7fb4fb7fe640 1 --2- 172.21.15.88:0/2510248913 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb4e40664a0 0x7fb4e4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:45.893 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.886+0000 7fb4fb7fe640 1 -- 172.21.15.88:0/2510248913 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7fb4f4121350 con 0x7fb5040b4530 2023-10-16T20:06:45.893 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.886+0000 7fb4f9d9b640 1 --2- 172.21.15.88:0/2510248913 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb4e40664a0 0x7fb4e4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:45.893 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.886+0000 7fb4f9d9b640 1 --2- 172.21.15.88:0/2510248913 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb4e40664a0 0x7fb4e4068960 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb50414fe40 tx=0x7fb4ec009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:45.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:45.890+0000 7fb4fb7fe640 1 -- 172.21.15.88:0/2510248913 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb4f40ee4e0 con 0x7fb5040b4530 2023-10-16T20:06:46.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.038+0000 7fb508f9d640 1 -- 172.21.15.88:0/2510248913 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb5040b6700 con 0x7fb5040b4530 2023-10-16T20:06:46.044 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.042+0000 7fb4fb7fe640 1 -- 172.21.15.88:0/2510248913 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb4f40f1b00 con 0x7fb5040b4530 2023-10-16T20:06:46.044 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:06:46.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.042+0000 7fb508f9d640 1 -- 172.21.15.88:0/2510248913 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb4e40664a0 msgr2=0x7fb4e4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:46.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.042+0000 7fb508f9d640 1 --2- 172.21.15.88:0/2510248913 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb4e40664a0 0x7fb4e4068960 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb50414fe40 tx=0x7fb4ec009190 comp rx=0 tx=0).stop 2023-10-16T20:06:46.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.042+0000 7fb508f9d640 1 -- 172.21.15.88:0/2510248913 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5040b4530 msgr2=0x7fb50415fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:46.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.042+0000 7fb508f9d640 1 --2- 172.21.15.88:0/2510248913 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5040b4530 0x7fb50415fb20 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009960 tx=0x7fb4f401d570 comp rx=0 tx=0).stop 2023-10-16T20:06:46.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.042+0000 7fb508f9d640 1 -- 172.21.15.88:0/2510248913 shutdown_connections 2023-10-16T20:06:46.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.042+0000 7fb508f9d640 1 --2- 172.21.15.88:0/2510248913 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb4e40664a0 0x7fb4e4068960 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:46.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.042+0000 7fb508f9d640 1 --2- 172.21.15.88:0/2510248913 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb50415cb70 0x7fb504150400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:46.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.042+0000 7fb508f9d640 1 --2- 172.21.15.88:0/2510248913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5040d0bd0 0x7fb50414fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:46.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.042+0000 7fb508f9d640 1 --2- 172.21.15.88:0/2510248913 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5040b4530 0x7fb50415fb20 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:46.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.042+0000 7fb508f9d640 1 -- 172.21.15.88:0/2510248913 >> 172.21.15.88:0/2510248913 conn(0x7fb50406b6e0 msgr2=0x7fb504072790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:46.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.042+0000 7fb508f9d640 1 -- 172.21.15.88:0/2510248913 shutdown_connections 2023-10-16T20:06:46.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.042+0000 7fb508f9d640 1 -- 172.21.15.88:0/2510248913 wait complete. 2023-10-16T20:06:46.064 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_metadata pg_num 2023-10-16T20:06:46.206 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.202+0000 7fe8cd5aa640 1 Processor -- start 2023-10-16T20:06:46.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.202+0000 7fe8cd5aa640 1 -- start start 2023-10-16T20:06:46.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8cd5aa640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe8c815cb70 0x7fe8c80b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:46.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8cd5aa640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8c80b7d20 con 0x7fe8c80b6a80 2023-10-16T20:06:46.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8cd5aa640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8c80b6eb0 con 0x7fe8c80b4530 2023-10-16T20:06:46.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8cd5aa640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8c80b7110 con 0x7fe8c815cb70 2023-10-16T20:06:46.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8c77fe640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fe8c80b6a80 0x7fe8c80d12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:38016/0 (socket says 172.21.15.88:38016) 2023-10-16T20:06:46.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8c77fe640 1 -- 172.21.15.88:0/1368367627 learned_addr learned my addr 172.21.15.88:0/1368367627 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:46.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8c6ffd640 1 --2- 172.21.15.88:0/1368367627 >> v2:172.21.15.94:3300/0 conn(0x7fe8c815cb70 0x7fe8c80b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:46.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8c67fc640 1 -- 172.21.15.88:0/1368367627 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2708596545 0 0) 0x7fe8c80b7d20 con 0x7fe8c80b6a80 2023-10-16T20:06:46.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8c67fc640 1 -- 172.21.15.88:0/1368367627 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8b4003530 con 0x7fe8c80b6a80 2023-10-16T20:06:46.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8c67fc640 1 -- 172.21.15.88:0/1368367627 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3266307470 0 0) 0x7fe8c80b6eb0 con 0x7fe8c80b4530 2023-10-16T20:06:46.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8c67fc640 1 -- 172.21.15.88:0/1368367627 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8c80b7d20 con 0x7fe8c80b4530 2023-10-16T20:06:46.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8c67fc640 1 -- 172.21.15.88:0/1368367627 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3941575261 0 0) 0x7fe8b4003530 con 0x7fe8c80b6a80 2023-10-16T20:06:46.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8c67fc640 1 -- 172.21.15.88:0/1368367627 >> v2:172.21.15.94:3300/0 conn(0x7fe8c815cb70 msgr2=0x7fe8c80b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:46.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8c67fc640 1 --2- 172.21.15.88:0/1368367627 >> v2:172.21.15.94:3300/0 conn(0x7fe8c815cb70 0x7fe8c80b83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:46.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8c67fc640 1 -- 172.21.15.88:0/1368367627 >> v1:172.21.15.94:6789/0 conn(0x7fe8c80b4530 legacy=0x7fe8c80d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:46.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8c67fc640 1 -- 172.21.15.88:0/1368367627 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c80b73f0 con 0x7fe8c80b6a80 2023-10-16T20:06:46.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8c67fc640 1 -- 172.21.15.88:0/1368367627 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7fe8c0001a00 con 0x7fe8c80b6a80 2023-10-16T20:06:46.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8c67fc640 1 -- 172.21.15.88:0/1368367627 >> v1:172.21.15.88:6789/0 conn(0x7fe8c80b6a80 legacy=0x7fe8c80d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:46.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8c67fc640 1 --2- 172.21.15.88:0/1368367627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8b4003b10 0x7fe8b4003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:46.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8c67fc640 1 --2- 172.21.15.88:0/1368367627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8b40047d0 0x7fe8b4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:46.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8c67fc640 1 --2- 172.21.15.88:0/1368367627 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe8b4005190 0x7fe8b400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:46.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8c67fc640 1 -- 172.21.15.88:0/1368367627 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c80b7110 con 0x7fe8b4003b10 2023-10-16T20:06:46.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8c67fc640 1 -- 172.21.15.88:0/1368367627 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b400bf80 con 0x7fe8b40047d0 2023-10-16T20:06:46.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8c67fc640 1 -- 172.21.15.88:0/1368367627 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b400c140 con 0x7fe8b4005190 2023-10-16T20:06:46.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8c6ffd640 1 --2- 172.21.15.88:0/1368367627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8b4003b10 0x7fe8b4003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:46.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8be7fc640 1 --2- 172.21.15.88:0/1368367627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8b40047d0 0x7fe8b4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:46.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8c77fe640 1 --2- 172.21.15.88:0/1368367627 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe8b4005190 0x7fe8b400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:46.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8c6ffd640 1 -- 172.21.15.88:0/1368367627 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe8b4005190 msgr2=0x7fe8b400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:46.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8c6ffd640 1 --2- 172.21.15.88:0/1368367627 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe8b4005190 0x7fe8b400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:46.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8c6ffd640 1 -- 172.21.15.88:0/1368367627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8b40047d0 msgr2=0x7fe8b4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:46.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8c6ffd640 1 --2- 172.21.15.88:0/1368367627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8b40047d0 0x7fe8b4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:46.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8c6ffd640 1 -- 172.21.15.88:0/1368367627 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b4003530 con 0x7fe8b4003b10 2023-10-16T20:06:46.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.206+0000 7fe8c77fe640 1 -- 172.21.15.88:0/1368367627 reap_dead start 2023-10-16T20:06:46.218 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.210+0000 7fe8c6ffd640 1 --2- 172.21.15.88:0/1368367627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8b4003b10 0x7fe8b4003f10 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0009ab0 tx=0x7fe8b001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d2734335d0cc720 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:46.218 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.210+0000 7fe8c67fc640 1 -- 172.21.15.88:0/1368367627 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b00a5620 con 0x7fe8b4003b10 2023-10-16T20:06:46.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.210+0000 7fe8c67fc640 1 -- 172.21.15.88:0/1368367627 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b00a5780 con 0x7fe8b4003b10 2023-10-16T20:06:46.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.210+0000 7fe8c67fc640 1 -- 172.21.15.88:0/1368367627 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b00a5b30 con 0x7fe8b4003b10 2023-10-16T20:06:46.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.210+0000 7fe8cd5aa640 1 -- 172.21.15.88:0/1368367627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8b4003b10 msgr2=0x7fe8b4003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:46.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.210+0000 7fe8cd5aa640 1 --2- 172.21.15.88:0/1368367627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8b4003b10 0x7fe8b4003f10 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0009ab0 tx=0x7fe8b001ca30 comp rx=0 tx=0).stop 2023-10-16T20:06:46.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.210+0000 7fe8cd5aa640 1 -- 172.21.15.88:0/1368367627 shutdown_connections 2023-10-16T20:06:46.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.210+0000 7fe8cd5aa640 1 --2- 172.21.15.88:0/1368367627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8b4003b10 0x7fe8b4003f10 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:46.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.210+0000 7fe8cd5aa640 1 -- 172.21.15.88:0/1368367627 >> 172.21.15.88:0/1368367627 conn(0x7fe8c806b6e0 msgr2=0x7fe8c806baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:46.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.210+0000 7fe8cd5aa640 1 -- 172.21.15.88:0/1368367627 shutdown_connections 2023-10-16T20:06:46.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.210+0000 7fe8cd5aa640 1 -- 172.21.15.88:0/1368367627 wait complete. 2023-10-16T20:06:46.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.210+0000 7fe8cd5aa640 1 Processor -- start 2023-10-16T20:06:46.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.210+0000 7fe8cd5aa640 1 -- start start 2023-10-16T20:06:46.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.214+0000 7fe8cd5aa640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe8b4003b10 0x7fe8c814d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:46.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.214+0000 7fe8cd5aa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c81f20c0 0x7fe8c814e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:46.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.214+0000 7fe8cd5aa640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8c8150750 0x7fe8c8150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:46.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.214+0000 7fe8cd5aa640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c80d17c0 con 0x7fe8c8150750 2023-10-16T20:06:46.222 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.214+0000 7fe8cd5aa640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c81f2610 con 0x7fe8c81f20c0 2023-10-16T20:06:46.222 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.214+0000 7fe8cd5aa640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c81f28f0 con 0x7fe8b4003b10 2023-10-16T20:06:46.222 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.214+0000 7fe8c77fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8c8150750 0x7fe8c8150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:46.222 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.214+0000 7fe8c77fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8c8150750 0x7fe8c8150bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51028/0 (socket says 172.21.15.88:51028) 2023-10-16T20:06:46.222 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.214+0000 7fe8c77fe640 1 -- 172.21.15.88:0/3730607375 learned_addr learned my addr 172.21.15.88:0/3730607375 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:46.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.214+0000 7fe8c6ffd640 1 --2- 172.21.15.88:0/3730607375 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe8b4003b10 0x7fe8c814d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:46.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.214+0000 7fe8be7fc640 1 --2- 172.21.15.88:0/3730607375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c81f20c0 0x7fe8c814e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:46.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.214+0000 7fe8c77fe640 1 -- 172.21.15.88:0/3730607375 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe8b4003b10 msgr2=0x7fe8c814d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:46.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.214+0000 7fe8c77fe640 1 --2- 172.21.15.88:0/3730607375 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe8b4003b10 0x7fe8c814d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:46.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.214+0000 7fe8c77fe640 1 -- 172.21.15.88:0/3730607375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c81f20c0 msgr2=0x7fe8c814e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:46.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.214+0000 7fe8c77fe640 1 --2- 172.21.15.88:0/3730607375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c81f20c0 0x7fe8c814e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:46.224 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.214+0000 7fe8c77fe640 1 -- 172.21.15.88:0/3730607375 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c81f2bd0 con 0x7fe8c8150750 2023-10-16T20:06:46.224 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.214+0000 7fe8c77fe640 1 --2- 172.21.15.88:0/3730607375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8c8150750 0x7fe8c8150bd0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe8c808bb70 tx=0x7fe8c0007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:46.224 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.214+0000 7fe8bffff640 1 -- 172.21.15.88:0/3730607375 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c000ea80 con 0x7fe8c8150750 2023-10-16T20:06:46.224 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.214+0000 7fe8bffff640 1 -- 172.21.15.88:0/3730607375 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c000ebe0 con 0x7fe8c8150750 2023-10-16T20:06:46.224 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.214+0000 7fe8cd5aa640 1 -- 172.21.15.88:0/3730607375 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c81f2eb0 con 0x7fe8c8150750 2023-10-16T20:06:46.225 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.214+0000 7fe8bffff640 1 -- 172.21.15.88:0/3730607375 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0013420 con 0x7fe8c8150750 2023-10-16T20:06:46.225 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.214+0000 7fe8cd5aa640 1 -- 172.21.15.88:0/3730607375 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c8151110 con 0x7fe8c8150750 2023-10-16T20:06:46.225 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.214+0000 7fe8cd5aa640 1 -- 172.21.15.88:0/3730607375 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c80d1930 con 0x7fe8c8150750 2023-10-16T20:06:46.226 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.214+0000 7fe8bffff640 1 -- 172.21.15.88:0/3730607375 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe8c0013580 con 0x7fe8c8150750 2023-10-16T20:06:46.226 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.218+0000 7fe8bffff640 1 --2- 172.21.15.88:0/3730607375 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe8a80664d0 0x7fe8a8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:46.226 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.218+0000 7fe8bffff640 1 -- 172.21.15.88:0/3730607375 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7fe8c00843f0 con 0x7fe8c8150750 2023-10-16T20:06:46.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.218+0000 7fe8c6ffd640 1 --2- 172.21.15.88:0/3730607375 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe8a80664d0 0x7fe8a8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:46.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.218+0000 7fe8c6ffd640 1 --2- 172.21.15.88:0/3730607375 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe8a80664d0 0x7fe8a8068990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0009ab0 tx=0x7fe8b00046d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:46.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.222+0000 7fe8bffff640 1 -- 172.21.15.88:0/3730607375 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe8c80d1930 con 0x7fe8c8150750 2023-10-16T20:06:46.375 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.370+0000 7fe8cd5aa640 1 -- 172.21.15.88:0/3730607375 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe8c81f30a0 con 0x7fe8c8150750 2023-10-16T20:06:46.375 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.370+0000 7fe8bffff640 1 -- 172.21.15.88:0/3730607375 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"}]=0 v46) v1 ==== 109+0+11 (secure 0 0 0) 0x7fe8c0051600 con 0x7fe8c8150750 2023-10-16T20:06:46.375 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:06:46.377 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.374+0000 7fe8cd5aa640 1 -- 172.21.15.88:0/3730607375 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe8a80664d0 msgr2=0x7fe8a8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:46.377 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.374+0000 7fe8cd5aa640 1 --2- 172.21.15.88:0/3730607375 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe8a80664d0 0x7fe8a8068990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0009ab0 tx=0x7fe8b00046d0 comp rx=0 tx=0).stop 2023-10-16T20:06:46.377 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.374+0000 7fe8cd5aa640 1 -- 172.21.15.88:0/3730607375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8c8150750 msgr2=0x7fe8c8150bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:46.377 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.374+0000 7fe8cd5aa640 1 --2- 172.21.15.88:0/3730607375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8c8150750 0x7fe8c8150bd0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe8c808bb70 tx=0x7fe8c0007bc0 comp rx=0 tx=0).stop 2023-10-16T20:06:46.377 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.374+0000 7fe8cd5aa640 1 -- 172.21.15.88:0/3730607375 shutdown_connections 2023-10-16T20:06:46.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.374+0000 7fe8cd5aa640 1 --2- 172.21.15.88:0/3730607375 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe8a80664d0 0x7fe8a8068990 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:46.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.374+0000 7fe8cd5aa640 1 --2- 172.21.15.88:0/3730607375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8c8150750 0x7fe8c8150bd0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:46.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.374+0000 7fe8cd5aa640 1 --2- 172.21.15.88:0/3730607375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c81f20c0 0x7fe8c814e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:46.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.374+0000 7fe8cd5aa640 1 --2- 172.21.15.88:0/3730607375 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe8b4003b10 0x7fe8c814d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:46.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.374+0000 7fe8cd5aa640 1 -- 172.21.15.88:0/3730607375 >> 172.21.15.88:0/3730607375 conn(0x7fe8c806b6e0 msgr2=0x7fe8c815b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:46.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.374+0000 7fe8cd5aa640 1 -- 172.21.15.88:0/3730607375 shutdown_connections 2023-10-16T20:06:46.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.374+0000 7fe8cd5aa640 1 -- 172.21.15.88:0/3730607375 wait complete. 2023-10-16T20:06:46.396 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_data pg_num 2023-10-16T20:06:46.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.534+0000 7f032f172640 1 Processor -- start 2023-10-16T20:06:46.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.534+0000 7f032f172640 1 -- start start 2023-10-16T20:06:46.539 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.534+0000 7f032f172640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f03280b4260 0x7f032815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:46.540 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.534+0000 7f032f172640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f03280b0da0 0x7f0328159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:46.540 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.534+0000 7f032f172640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f032815a1f0 0x7f032806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:46.540 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.534+0000 7f032f172640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f032815ad30 con 0x7f03280b0da0 2023-10-16T20:06:46.540 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.534+0000 7f032f172640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03280b4640 con 0x7f032815a1f0 2023-10-16T20:06:46.540 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.534+0000 7f032f172640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f032815a770 con 0x7f03280b4260 2023-10-16T20:06:46.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.534+0000 7f031ffff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f03280b0da0 0x7f0328159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:46.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.534+0000 7f031ffff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f03280b0da0 0x7f0328159cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51030/0 (socket says 172.21.15.88:51030) 2023-10-16T20:06:46.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.534+0000 7f031ffff640 1 -- 172.21.15.88:0/3998889240 learned_addr learned my addr 172.21.15.88:0/3998889240 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:46.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.538+0000 7f031ffff640 1 -- 172.21.15.88:0/3998889240 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f03280b4260 msgr2=0x7f032815dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:46.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.538+0000 7f032cf0e640 1 --2- 172.21.15.88:0/3998889240 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f03280b4260 0x7f032815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:46.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.538+0000 7f032d70f640 1 --2- 172.21.15.88:0/3998889240 >> v2:172.21.15.94:3300/0 conn(0x7f032815a1f0 0x7f032806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:46.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.538+0000 7f031ffff640 1 --2- 172.21.15.88:0/3998889240 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f03280b4260 0x7f032815dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:46.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.538+0000 7f031ffff640 1 -- 172.21.15.88:0/3998889240 >> v2:172.21.15.94:3300/0 conn(0x7f032815a1f0 msgr2=0x7f032806d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:46.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.538+0000 7f031ffff640 1 --2- 172.21.15.88:0/3998889240 >> v2:172.21.15.94:3300/0 conn(0x7f032815a1f0 0x7f032806d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:46.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.538+0000 7f031ffff640 1 -- 172.21.15.88:0/3998889240 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032815aa50 con 0x7f03280b0da0 2023-10-16T20:06:46.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.538+0000 7f032cf0e640 1 --2- 172.21.15.88:0/3998889240 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f03280b4260 0x7f032815dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:06:46.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.538+0000 7f032d70f640 1 --2- 172.21.15.88:0/3998889240 >> v2:172.21.15.94:3300/0 conn(0x7f032815a1f0 0x7f032806d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:06:46.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.538+0000 7f031ffff640 1 --2- 172.21.15.88:0/3998889240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03280b0da0 0x7f0328159cb0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f03280b1200 tx=0x7f031801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44a740ed6ec5186 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:46.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.538+0000 7f031f7fe640 1 -- 172.21.15.88:0/3998889240 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03180ac640 con 0x7f03280b0da0 2023-10-16T20:06:46.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.538+0000 7f031f7fe640 1 -- 172.21.15.88:0/3998889240 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03180ac7a0 con 0x7f03280b0da0 2023-10-16T20:06:46.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.538+0000 7f031f7fe640 1 -- 172.21.15.88:0/3998889240 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03180acb30 con 0x7f03280b0da0 2023-10-16T20:06:46.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.538+0000 7f032f172640 1 -- 172.21.15.88:0/3998889240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03280b0da0 msgr2=0x7f0328159cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:46.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.538+0000 7f032f172640 1 --2- 172.21.15.88:0/3998889240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03280b0da0 0x7f0328159cb0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f03280b1200 tx=0x7f031801c440 comp rx=0 tx=0).stop 2023-10-16T20:06:46.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.538+0000 7f032f172640 1 -- 172.21.15.88:0/3998889240 shutdown_connections 2023-10-16T20:06:46.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.542+0000 7f032f172640 1 --2- 172.21.15.88:0/3998889240 >> v2:172.21.15.94:3300/0 conn(0x7f032815a1f0 0x7f032806d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:46.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.542+0000 7f032f172640 1 --2- 172.21.15.88:0/3998889240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03280b0da0 0x7f0328159cb0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:46.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.542+0000 7f032f172640 1 --2- 172.21.15.88:0/3998889240 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f03280b4260 0x7f032815dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:46.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.542+0000 7f032f172640 1 -- 172.21.15.88:0/3998889240 >> 172.21.15.88:0/3998889240 conn(0x7f03280cb080 msgr2=0x7f03280cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:46.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.542+0000 7f032f172640 1 -- 172.21.15.88:0/3998889240 shutdown_connections 2023-10-16T20:06:46.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.542+0000 7f032f172640 1 -- 172.21.15.88:0/3998889240 wait complete. 2023-10-16T20:06:46.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.542+0000 7f032f172640 1 Processor -- start 2023-10-16T20:06:46.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.542+0000 7f032f172640 1 -- start start 2023-10-16T20:06:46.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.542+0000 7f032f172640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f03280b0da0 0x7f0328158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:46.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.542+0000 7f032f172640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03280b4260 0x7f032813d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:46.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.542+0000 7f032f172640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f032815a1f0 0x7f032814fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:46.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.542+0000 7f032f172640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0328156e70 con 0x7f032815a1f0 2023-10-16T20:06:46.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.542+0000 7f032f172640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032814f0c0 con 0x7f03280b4260 2023-10-16T20:06:46.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.542+0000 7f032f172640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f032814f3a0 con 0x7f03280b0da0 2023-10-16T20:06:46.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.542+0000 7f032d70f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f032815a1f0 0x7f032814fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:46.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.542+0000 7f032d70f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f032815a1f0 0x7f032814fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51046/0 (socket says 172.21.15.88:51046) 2023-10-16T20:06:46.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.542+0000 7f032d70f640 1 -- 172.21.15.88:0/1267080346 learned_addr learned my addr 172.21.15.88:0/1267080346 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:46.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.542+0000 7f032cf0e640 1 --2- 172.21.15.88:0/1267080346 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f03280b0da0 0x7f0328158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:46.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.542+0000 7f031ffff640 1 --2- 172.21.15.88:0/1267080346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03280b4260 0x7f032813d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:46.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.542+0000 7f032d70f640 1 -- 172.21.15.88:0/1267080346 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f03280b0da0 msgr2=0x7f0328158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:46.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.542+0000 7f032d70f640 1 --2- 172.21.15.88:0/1267080346 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f03280b0da0 0x7f0328158640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:46.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.542+0000 7f032d70f640 1 -- 172.21.15.88:0/1267080346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03280b4260 msgr2=0x7f032813d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:46.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.542+0000 7f032d70f640 1 --2- 172.21.15.88:0/1267080346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03280b4260 0x7f032813d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:46.552 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.542+0000 7f032d70f640 1 -- 172.21.15.88:0/1267080346 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03180094d0 con 0x7f032815a1f0 2023-10-16T20:06:46.552 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.542+0000 7f032cf0e640 1 --2- 172.21.15.88:0/1267080346 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f03280b0da0 0x7f0328158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:46.552 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.542+0000 7f031ffff640 1 --2- 172.21.15.88:0/1267080346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03280b4260 0x7f032813d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:46.552 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.546+0000 7f032d70f640 1 --2- 172.21.15.88:0/1267080346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f032815a1f0 0x7f032814fe80 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f032000cef0 tx=0x7f032000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:46.552 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.546+0000 7f031d7fa640 1 -- 172.21.15.88:0/1267080346 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032000d840 con 0x7f032815a1f0 2023-10-16T20:06:46.553 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.546+0000 7f031d7fa640 1 -- 172.21.15.88:0/1267080346 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032000d9c0 con 0x7f032815a1f0 2023-10-16T20:06:46.553 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.546+0000 7f032f172640 1 -- 172.21.15.88:0/1267080346 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f032814f680 con 0x7f032815a1f0 2023-10-16T20:06:46.553 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.546+0000 7f031d7fa640 1 -- 172.21.15.88:0/1267080346 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032000dd70 con 0x7f032815a1f0 2023-10-16T20:06:46.553 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.546+0000 7f032f172640 1 -- 172.21.15.88:0/1267080346 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0328158b80 con 0x7f032815a1f0 2023-10-16T20:06:46.553 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.546+0000 7f032f172640 1 -- 172.21.15.88:0/1267080346 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f032804e9c0 con 0x7f032815a1f0 2023-10-16T20:06:46.554 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.546+0000 7f031d7fa640 1 -- 172.21.15.88:0/1267080346 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f032001e3b0 con 0x7f032815a1f0 2023-10-16T20:06:46.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.546+0000 7f031d7fa640 1 --2- 172.21.15.88:0/1267080346 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f03140664a0 0x7f0314068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:46.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.546+0000 7f031d7fa640 1 -- 172.21.15.88:0/1267080346 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f0320084c50 con 0x7f032815a1f0 2023-10-16T20:06:46.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.546+0000 7f032cf0e640 1 --2- 172.21.15.88:0/1267080346 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f03140664a0 0x7f0314068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:46.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.550+0000 7f032cf0e640 1 --2- 172.21.15.88:0/1267080346 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f03140664a0 0x7f0314068960 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0324006b20 tx=0x7f0324008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:46.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.550+0000 7f031d7fa640 1 -- 172.21.15.88:0/1267080346 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0320051e70 con 0x7f032815a1f0 2023-10-16T20:06:46.703 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.698+0000 7f032f172640 1 -- 172.21.15.88:0/1267080346 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"} v 0) v1 -- 0x7f032814fb30 con 0x7f032815a1f0 2023-10-16T20:06:46.703 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.698+0000 7f031d7fa640 1 -- 172.21.15.88:0/1267080346 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"}]=0 v46) v1 ==== 105+0+11 (secure 0 0 0) 0x7f032001a020 con 0x7f032815a1f0 2023-10-16T20:06:46.704 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2023-10-16T20:06:46.705 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.702+0000 7f032f172640 1 -- 172.21.15.88:0/1267080346 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f03140664a0 msgr2=0x7f0314068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:46.705 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.702+0000 7f032f172640 1 --2- 172.21.15.88:0/1267080346 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f03140664a0 0x7f0314068960 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0324006b20 tx=0x7f0324008040 comp rx=0 tx=0).stop 2023-10-16T20:06:46.706 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.702+0000 7f032f172640 1 -- 172.21.15.88:0/1267080346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f032815a1f0 msgr2=0x7f032814fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:46.706 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.702+0000 7f032f172640 1 --2- 172.21.15.88:0/1267080346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f032815a1f0 0x7f032814fe80 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f032000cef0 tx=0x7f032000a710 comp rx=0 tx=0).stop 2023-10-16T20:06:46.706 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.702+0000 7f032f172640 1 -- 172.21.15.88:0/1267080346 shutdown_connections 2023-10-16T20:06:46.706 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.702+0000 7f032f172640 1 --2- 172.21.15.88:0/1267080346 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f03140664a0 0x7f0314068960 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:46.706 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.702+0000 7f032f172640 1 --2- 172.21.15.88:0/1267080346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f032815a1f0 0x7f032814fe80 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:46.707 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.702+0000 7f032f172640 1 --2- 172.21.15.88:0/1267080346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03280b4260 0x7f032813d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:46.707 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.702+0000 7f032f172640 1 --2- 172.21.15.88:0/1267080346 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f03280b0da0 0x7f0328158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:46.707 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.702+0000 7f032f172640 1 -- 172.21.15.88:0/1267080346 >> 172.21.15.88:0/1267080346 conn(0x7f03280cb080 msgr2=0x7f0328157b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:46.707 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.702+0000 7f032f172640 1 -- 172.21.15.88:0/1267080346 shutdown_connections 2023-10-16T20:06:46.707 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.702+0000 7f032f172640 1 -- 172.21.15.88:0/1267080346 wait complete. 2023-10-16T20:06:46.724 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-10-16T20:06:46.865 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.862+0000 7f19e631d640 1 Processor -- start 2023-10-16T20:06:46.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.862+0000 7f19e631d640 1 -- start start 2023-10-16T20:06:46.868 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.862+0000 7f19e631d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19e015cac0 0x7f19e00ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:46.868 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.862+0000 7f19e631d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f19e00ae130 0x7f19e00cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:46.868 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.862+0000 7f19e631d640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19e006b840 con 0x7f19e015cea0 2023-10-16T20:06:46.868 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.862+0000 7f19e631d640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19e00cac10 con 0x7f19e00ae130 2023-10-16T20:06:46.869 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.862+0000 7f19e631d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e006a3a0 con 0x7f19e015cac0 2023-10-16T20:06:46.869 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.862+0000 7f19df7fe640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f19e015cea0 0x7f19e01607d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:45620/0 (socket says 172.21.15.88:45620) 2023-10-16T20:06:46.869 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.862+0000 7f19e48ba640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f19e00ae130 0x7f19e00cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:46.869 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.862+0000 7f19df7fe640 1 -- 172.21.15.88:0/3549979256 learned_addr learned my addr 172.21.15.88:0/3549979256 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:46.869 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.862+0000 7f19dffff640 1 --2- 172.21.15.88:0/3549979256 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19e015cac0 0x7f19e00ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:46.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.866+0000 7f19deffd640 1 -- 172.21.15.88:0/3549979256 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 635665270 0 0) 0x7f19e006b840 con 0x7f19e015cea0 2023-10-16T20:06:46.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.866+0000 7f19deffd640 1 -- 172.21.15.88:0/3549979256 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19c8003530 con 0x7f19e015cea0 2023-10-16T20:06:46.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.866+0000 7f19e48ba640 1 -- 172.21.15.88:0/3549979256 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19e015cac0 msgr2=0x7f19e00ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:46.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.866+0000 7f19e48ba640 1 --2- 172.21.15.88:0/3549979256 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19e015cac0 0x7f19e00ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:46.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.866+0000 7f19e48ba640 1 -- 172.21.15.88:0/3549979256 >> v1:172.21.15.94:6789/0 conn(0x7f19e015cea0 legacy=0x7f19e01607d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:46.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.866+0000 7f19e48ba640 1 -- 172.21.15.88:0/3549979256 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e00ca650 con 0x7f19e00ae130 2023-10-16T20:06:46.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.866+0000 7f19dffff640 1 --2- 172.21.15.88:0/3549979256 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19e015cac0 0x7f19e00ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:06:46.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.866+0000 7f19e48ba640 1 --2- 172.21.15.88:0/3549979256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e00ae130 0x7f19e00cd1a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009d90 tx=0x7f19d801f130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=713e6b1a2cdaffeb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:46.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.866+0000 7f19deffd640 1 -- 172.21.15.88:0/3549979256 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d80af600 con 0x7f19e00ae130 2023-10-16T20:06:46.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.866+0000 7f19deffd640 1 -- 172.21.15.88:0/3549979256 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d80ab050 con 0x7f19e00ae130 2023-10-16T20:06:46.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.866+0000 7f19deffd640 1 -- 172.21.15.88:0/3549979256 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d80af980 con 0x7f19e00ae130 2023-10-16T20:06:46.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.870+0000 7f19e631d640 1 -- 172.21.15.88:0/3549979256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e00ae130 msgr2=0x7f19e00cd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:46.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.870+0000 7f19e631d640 1 --2- 172.21.15.88:0/3549979256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e00ae130 0x7f19e00cd1a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009d90 tx=0x7f19d801f130 comp rx=0 tx=0).stop 2023-10-16T20:06:46.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.870+0000 7f19e631d640 1 -- 172.21.15.88:0/3549979256 shutdown_connections 2023-10-16T20:06:46.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.870+0000 7f19e631d640 1 --2- 172.21.15.88:0/3549979256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e00ae130 0x7f19e00cd1a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:46.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.870+0000 7f19e631d640 1 --2- 172.21.15.88:0/3549979256 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19e015cac0 0x7f19e00ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:46.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.870+0000 7f19e631d640 1 -- 172.21.15.88:0/3549979256 >> 172.21.15.88:0/3549979256 conn(0x7f19e00b0970 msgr2=0x7f19e00b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:46.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.870+0000 7f19e631d640 1 -- 172.21.15.88:0/3549979256 shutdown_connections 2023-10-16T20:06:46.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.870+0000 7f19e631d640 1 -- 172.21.15.88:0/3549979256 wait complete. 2023-10-16T20:06:46.879 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.870+0000 7f19e631d640 1 Processor -- start 2023-10-16T20:06:46.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.870+0000 7f19e631d640 1 -- start start 2023-10-16T20:06:46.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.870+0000 7f19e631d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19e00ae130 0x7f19e01f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:46.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.870+0000 7f19e631d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e015cac0 0x7f19e015ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:46.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.870+0000 7f19e631d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19e015cea0 0x7f19e00acb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:46.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.870+0000 7f19e631d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e01f61b0 con 0x7f19e00ae130 2023-10-16T20:06:46.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.870+0000 7f19e631d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e015f740 con 0x7f19e015cac0 2023-10-16T20:06:46.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.870+0000 7f19e631d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e01f5af0 con 0x7f19e015cea0 2023-10-16T20:06:46.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.870+0000 7f19dffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19e00ae130 0x7f19e01f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:46.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.870+0000 7f19dffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19e00ae130 0x7f19e01f65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51060/0 (socket says 172.21.15.88:51060) 2023-10-16T20:06:46.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.870+0000 7f19dffff640 1 -- 172.21.15.88:0/3206500298 learned_addr learned my addr 172.21.15.88:0/3206500298 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:46.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.870+0000 7f19e48ba640 1 --2- 172.21.15.88:0/3206500298 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19e015cea0 0x7f19e00acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:46.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.870+0000 7f19df7fe640 1 --2- 172.21.15.88:0/3206500298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e015cac0 0x7f19e015ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:46.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.870+0000 7f19dffff640 1 -- 172.21.15.88:0/3206500298 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19e015cea0 msgr2=0x7f19e00acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:46.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.874+0000 7f19dffff640 1 --2- 172.21.15.88:0/3206500298 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19e015cea0 0x7f19e00acb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:46.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.874+0000 7f19dffff640 1 -- 172.21.15.88:0/3206500298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e015cac0 msgr2=0x7f19e015ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:46.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.874+0000 7f19dffff640 1 --2- 172.21.15.88:0/3206500298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e015cac0 0x7f19e015ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:46.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.874+0000 7f19dffff640 1 -- 172.21.15.88:0/3206500298 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d0009420 con 0x7f19e00ae130 2023-10-16T20:06:46.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.874+0000 7f19dffff640 1 --2- 172.21.15.88:0/3206500298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19e00ae130 0x7f19e01f65d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f19d000ebe0 tx=0x7f19d000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:46.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.874+0000 7f19dcff9640 1 -- 172.21.15.88:0/3206500298 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d000eee0 con 0x7f19e00ae130 2023-10-16T20:06:46.887 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.874+0000 7f19dcff9640 1 -- 172.21.15.88:0/3206500298 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d000cc30 con 0x7f19e00ae130 2023-10-16T20:06:46.887 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.874+0000 7f19e631d640 1 -- 172.21.15.88:0/3206500298 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19d8009a70 con 0x7f19e00ae130 2023-10-16T20:06:46.887 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.874+0000 7f19dcff9640 1 -- 172.21.15.88:0/3206500298 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d0013470 con 0x7f19e00ae130 2023-10-16T20:06:46.887 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.874+0000 7f19e631d640 1 -- 172.21.15.88:0/3206500298 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e015fa20 con 0x7f19e00ae130 2023-10-16T20:06:46.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.874+0000 7f19e631d640 1 -- 172.21.15.88:0/3206500298 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e01f6420 con 0x7f19e00ae130 2023-10-16T20:06:46.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.874+0000 7f19dcff9640 1 -- 172.21.15.88:0/3206500298 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f19d00135d0 con 0x7f19e00ae130 2023-10-16T20:06:46.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.874+0000 7f19dcff9640 1 --2- 172.21.15.88:0/3206500298 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f19cc0664a0 0x7f19cc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:46.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.874+0000 7f19dcff9640 1 -- 172.21.15.88:0/3206500298 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f19d0084400 con 0x7f19e00ae130 2023-10-16T20:06:46.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.874+0000 7f19df7fe640 1 --2- 172.21.15.88:0/3206500298 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f19cc0664a0 0x7f19cc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:46.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.878+0000 7f19df7fe640 1 --2- 172.21.15.88:0/3206500298 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f19cc0664a0 0x7f19cc068960 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f19e0157fa0 tx=0x7f19d4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:46.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:46.878+0000 7f19dcff9640 1 -- 172.21.15.88:0/3206500298 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f19d0051590 con 0x7f19e00ae130 2023-10-16T20:06:47.030 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.026+0000 7f19e631d640 1 -- 172.21.15.88:0/3206500298 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f19e004e9c0 con 0x7f19e00ae130 2023-10-16T20:06:47.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.026+0000 7f19dcff9640 1 -- 172.21.15.88:0/3206500298 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v46) v1 ==== 97+0+11 (secure 0 0 0) 0x7f19d0054bb0 con 0x7f19e00ae130 2023-10-16T20:06:47.031 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:06:47.032 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.030+0000 7f19e631d640 1 -- 172.21.15.88:0/3206500298 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f19cc0664a0 msgr2=0x7f19cc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:47.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.030+0000 7f19e631d640 1 --2- 172.21.15.88:0/3206500298 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f19cc0664a0 0x7f19cc068960 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f19e0157fa0 tx=0x7f19d4009040 comp rx=0 tx=0).stop 2023-10-16T20:06:47.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.030+0000 7f19e631d640 1 -- 172.21.15.88:0/3206500298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19e00ae130 msgr2=0x7f19e01f65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:47.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.030+0000 7f19e631d640 1 --2- 172.21.15.88:0/3206500298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19e00ae130 0x7f19e01f65d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f19d000ebe0 tx=0x7f19d000c520 comp rx=0 tx=0).stop 2023-10-16T20:06:47.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.030+0000 7f19e631d640 1 -- 172.21.15.88:0/3206500298 shutdown_connections 2023-10-16T20:06:47.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.030+0000 7f19e631d640 1 --2- 172.21.15.88:0/3206500298 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f19cc0664a0 0x7f19cc068960 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.030+0000 7f19e631d640 1 --2- 172.21.15.88:0/3206500298 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f19e015cea0 0x7f19e00acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.030+0000 7f19e631d640 1 --2- 172.21.15.88:0/3206500298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e015cac0 0x7f19e015ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.030+0000 7f19e631d640 1 --2- 172.21.15.88:0/3206500298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19e00ae130 0x7f19e01f65d0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.030+0000 7f19e631d640 1 -- 172.21.15.88:0/3206500298 >> 172.21.15.88:0/3206500298 conn(0x7f19e00b0970 msgr2=0x7f19e00cca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:47.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.030+0000 7f19e631d640 1 -- 172.21.15.88:0/3206500298 shutdown_connections 2023-10-16T20:06:47.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.030+0000 7f19e631d640 1 -- 172.21.15.88:0/3206500298 wait complete. 2023-10-16T20:06:47.052 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T20:06:47.190 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.186+0000 7f25331d9640 1 Processor -- start 2023-10-16T20:06:47.191 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.186+0000 7f25331d9640 1 -- start start 2023-10-16T20:06:47.194 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.190+0000 7f25331d9640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f252c06b5b0 0x7f252c0c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:47.194 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.190+0000 7f25331d9640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f252c071de0 con 0x7f252c1616f0 2023-10-16T20:06:47.194 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.190+0000 7f25331d9640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f252c06e090 con 0x7f252c161310 2023-10-16T20:06:47.194 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.190+0000 7f25331d9640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f252c0cda50 con 0x7f252c06b5b0 2023-10-16T20:06:47.195 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.190+0000 7f2531776640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f252c1616f0 0x7f252c0ca520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:38018/0 (socket says 172.21.15.88:38018) 2023-10-16T20:06:47.195 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.190+0000 7f2531776640 1 -- 172.21.15.88:0/2798571418 learned_addr learned my addr 172.21.15.88:0/2798571418 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:47.195 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.190+0000 7f25237fe640 1 -- 172.21.15.88:0/2798571418 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3992488392 0 0) 0x7f252c071de0 con 0x7f252c1616f0 2023-10-16T20:06:47.195 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.190+0000 7f25237fe640 1 -- 172.21.15.88:0/2798571418 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2514003530 con 0x7f252c1616f0 2023-10-16T20:06:47.195 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.190+0000 7f2523fff640 1 --2- 172.21.15.88:0/2798571418 >> v2:172.21.15.94:3300/0 conn(0x7f252c06b5b0 0x7f252c0c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:47.195 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.190+0000 7f25237fe640 1 -- 172.21.15.88:0/2798571418 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2795512860 0 0) 0x7f2514003530 con 0x7f252c1616f0 2023-10-16T20:06:47.196 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.190+0000 7f25237fe640 1 -- 172.21.15.88:0/2798571418 >> v2:172.21.15.94:3300/0 conn(0x7f252c06b5b0 msgr2=0x7f252c0c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:47.196 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.190+0000 7f25237fe640 1 --2- 172.21.15.88:0/2798571418 >> v2:172.21.15.94:3300/0 conn(0x7f252c06b5b0 0x7f252c0c9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.196 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.190+0000 7f25237fe640 1 -- 172.21.15.88:0/2798571418 >> v1:172.21.15.94:6789/0 conn(0x7f252c161310 legacy=0x7f252c06aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:47.196 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.190+0000 7f25237fe640 1 -- 172.21.15.88:0/2798571418 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f252c071760 con 0x7f252c1616f0 2023-10-16T20:06:47.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.190+0000 7f25237fe640 1 -- 172.21.15.88:0/2798571418 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f2524001a00 con 0x7f252c1616f0 2023-10-16T20:06:47.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.190+0000 7f25237fe640 1 -- 172.21.15.88:0/2798571418 >> v1:172.21.15.88:6789/0 conn(0x7f252c1616f0 legacy=0x7f252c0ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:47.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.190+0000 7f25237fe640 1 --2- 172.21.15.88:0/2798571418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2514003980 0x7f2514003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:47.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.190+0000 7f25237fe640 1 --2- 172.21.15.88:0/2798571418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25140047d0 0x7f2514004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:47.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.190+0000 7f25237fe640 1 --2- 172.21.15.88:0/2798571418 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2514005190 0x7f251400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:47.198 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.190+0000 7f25237fe640 1 -- 172.21.15.88:0/2798571418 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252c0cda50 con 0x7f25140047d0 2023-10-16T20:06:47.198 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.190+0000 7f25237fe640 1 -- 172.21.15.88:0/2798571418 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251400c030 con 0x7f2514003980 2023-10-16T20:06:47.198 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.190+0000 7f25237fe640 1 -- 172.21.15.88:0/2798571418 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251400c1f0 con 0x7f2514005190 2023-10-16T20:06:47.198 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.190+0000 7f2523fff640 1 --2- 172.21.15.88:0/2798571418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25140047d0 0x7f2514004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:47.198 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.190+0000 7f2531776640 1 --2- 172.21.15.88:0/2798571418 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2514005190 0x7f251400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:47.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.190+0000 7f2530f75640 1 --2- 172.21.15.88:0/2798571418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2514003980 0x7f2514003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:47.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.190+0000 7f2523fff640 1 -- 172.21.15.88:0/2798571418 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2514005190 msgr2=0x7f251400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:47.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.190+0000 7f2523fff640 1 --2- 172.21.15.88:0/2798571418 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2514005190 0x7f251400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.190+0000 7f2523fff640 1 -- 172.21.15.88:0/2798571418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2514003980 msgr2=0x7f2514003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:47.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.190+0000 7f2523fff640 1 --2- 172.21.15.88:0/2798571418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2514003980 0x7f2514003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.190+0000 7f2523fff640 1 -- 172.21.15.88:0/2798571418 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2514003530 con 0x7f25140047d0 2023-10-16T20:06:47.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.190+0000 7f2531776640 1 -- 172.21.15.88:0/2798571418 reap_dead start 2023-10-16T20:06:47.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.194+0000 7f2523fff640 1 --2- 172.21.15.88:0/2798571418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25140047d0 0x7f2514004c50 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f251c009ae0 tx=0x7f251c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a477f7d6ee9d873 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:47.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.194+0000 7f25237fe640 1 -- 172.21.15.88:0/2798571418 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c0a5530 con 0x7f25140047d0 2023-10-16T20:06:47.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.194+0000 7f25237fe640 1 -- 172.21.15.88:0/2798571418 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f251c0a5690 con 0x7f25140047d0 2023-10-16T20:06:47.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.194+0000 7f25237fe640 1 -- 172.21.15.88:0/2798571418 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c0a5a40 con 0x7f25140047d0 2023-10-16T20:06:47.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.194+0000 7f25331d9640 1 -- 172.21.15.88:0/2798571418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25140047d0 msgr2=0x7f2514004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:47.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.194+0000 7f25331d9640 1 --2- 172.21.15.88:0/2798571418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25140047d0 0x7f2514004c50 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f251c009ae0 tx=0x7f251c01ca50 comp rx=0 tx=0).stop 2023-10-16T20:06:47.206 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.194+0000 7f25331d9640 1 -- 172.21.15.88:0/2798571418 shutdown_connections 2023-10-16T20:06:47.206 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.194+0000 7f25331d9640 1 --2- 172.21.15.88:0/2798571418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25140047d0 0x7f2514004c50 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.194+0000 7f25331d9640 1 -- 172.21.15.88:0/2798571418 >> 172.21.15.88:0/2798571418 conn(0x7f252c070aa0 msgr2=0x7f252c070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:47.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.194+0000 7f25331d9640 1 -- 172.21.15.88:0/2798571418 shutdown_connections 2023-10-16T20:06:47.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.194+0000 7f25331d9640 1 -- 172.21.15.88:0/2798571418 wait complete. 2023-10-16T20:06:47.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.194+0000 7f25331d9640 1 Processor -- start 2023-10-16T20:06:47.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.194+0000 7f25331d9640 1 -- start start 2023-10-16T20:06:47.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.198+0000 7f25331d9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25140047d0 0x7f252c0c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:47.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.198+0000 7f25331d9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f252c0b42d0 0x7f252c15d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:47.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.198+0000 7f25331d9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f252c0c4840 0x7f252c0c4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:47.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.198+0000 7f25331d9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252c0c5200 con 0x7f252c0b42d0 2023-10-16T20:06:47.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.198+0000 7f25331d9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252c0b1a80 con 0x7f25140047d0 2023-10-16T20:06:47.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.198+0000 7f25331d9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f252c0c7a60 con 0x7f252c0c4840 2023-10-16T20:06:47.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.198+0000 7f2523fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f252c0b42d0 0x7f252c15d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:47.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.198+0000 7f2523fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f252c0b42d0 0x7f252c15d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51074/0 (socket says 172.21.15.88:51074) 2023-10-16T20:06:47.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.198+0000 7f2523fff640 1 -- 172.21.15.88:0/966515988 learned_addr learned my addr 172.21.15.88:0/966515988 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:47.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.198+0000 7f2530f75640 1 --2- 172.21.15.88:0/966515988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25140047d0 0x7f252c0c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:47.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.198+0000 7f2531776640 1 --2- 172.21.15.88:0/966515988 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f252c0c4840 0x7f252c0c4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:47.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.198+0000 7f2523fff640 1 -- 172.21.15.88:0/966515988 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f252c0c4840 msgr2=0x7f252c0c4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:47.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.198+0000 7f2523fff640 1 --2- 172.21.15.88:0/966515988 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f252c0c4840 0x7f252c0c4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.198+0000 7f2523fff640 1 -- 172.21.15.88:0/966515988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25140047d0 msgr2=0x7f252c0c8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:47.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.198+0000 7f2523fff640 1 --2- 172.21.15.88:0/966515988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25140047d0 0x7f252c0c8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.198+0000 7f2523fff640 1 -- 172.21.15.88:0/966515988 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f252c0c7d40 con 0x7f252c0b42d0 2023-10-16T20:06:47.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.198+0000 7f2531776640 1 --2- 172.21.15.88:0/966515988 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f252c0c4840 0x7f252c0c4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:06:47.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.198+0000 7f2523fff640 1 --2- 172.21.15.88:0/966515988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f252c0b42d0 0x7f252c15d230 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f251c01c880 tx=0x7f251c0197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:47.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.198+0000 7f25217fa640 1 -- 172.21.15.88:0/966515988 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c004020 con 0x7f252c0b42d0 2023-10-16T20:06:47.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.198+0000 7f25217fa640 1 -- 172.21.15.88:0/966515988 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f251c0041c0 con 0x7f252c0b42d0 2023-10-16T20:06:47.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.198+0000 7f25217fa640 1 -- 172.21.15.88:0/966515988 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c0b8660 con 0x7f252c0b42d0 2023-10-16T20:06:47.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.198+0000 7f25331d9640 1 -- 172.21.15.88:0/966515988 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f252c0c8020 con 0x7f252c0b42d0 2023-10-16T20:06:47.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.198+0000 7f25331d9640 1 -- 172.21.15.88:0/966515988 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f252c15d770 con 0x7f252c0b42d0 2023-10-16T20:06:47.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.198+0000 7f25331d9640 1 -- 172.21.15.88:0/966515988 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f252c0c81b0 con 0x7f252c0b42d0 2023-10-16T20:06:47.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.202+0000 7f25217fa640 1 -- 172.21.15.88:0/966515988 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f251c0b4070 con 0x7f252c0b42d0 2023-10-16T20:06:47.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.202+0000 7f25217fa640 1 --2- 172.21.15.88:0/966515988 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f25180664d0 0x7f2518068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:47.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.202+0000 7f25217fa640 1 -- 172.21.15.88:0/966515988 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f251c11e7f0 con 0x7f252c0b42d0 2023-10-16T20:06:47.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.202+0000 7f2530f75640 1 --2- 172.21.15.88:0/966515988 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f25180664d0 0x7f2518068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:47.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.202+0000 7f2530f75640 1 --2- 172.21.15.88:0/966515988 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f25180664d0 0x7f2518068990 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2528005510 tx=0x7f2528009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:47.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.206+0000 7f25217fa640 1 -- 172.21.15.88:0/966515988 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f251c0eb8d0 con 0x7f252c0b42d0 2023-10-16T20:06:47.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.366+0000 7f25331d9640 1 -- 172.21.15.88:0/966515988 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f252c04e9c0 con 0x7f252c0b42d0 2023-10-16T20:06:47.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.370+0000 7f25217fa640 1 -- 172.21.15.88:0/966515988 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v46) v1 ==== 80+0+0 (secure 0 0 0) 0x7f251c0eeef0 con 0x7f252c0b42d0 2023-10-16T20:06:47.372 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2023-10-16T20:06:47.373 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.370+0000 7f25331d9640 1 -- 172.21.15.88:0/966515988 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f25180664d0 msgr2=0x7f2518068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:47.373 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.370+0000 7f25331d9640 1 --2- 172.21.15.88:0/966515988 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f25180664d0 0x7f2518068990 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2528005510 tx=0x7f2528009040 comp rx=0 tx=0).stop 2023-10-16T20:06:47.374 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.370+0000 7f25331d9640 1 -- 172.21.15.88:0/966515988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f252c0b42d0 msgr2=0x7f252c15d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:47.374 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.370+0000 7f25331d9640 1 --2- 172.21.15.88:0/966515988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f252c0b42d0 0x7f252c15d230 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f251c01c880 tx=0x7f251c0197f0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.374 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.370+0000 7f25331d9640 1 -- 172.21.15.88:0/966515988 shutdown_connections 2023-10-16T20:06:47.374 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.370+0000 7f25331d9640 1 --2- 172.21.15.88:0/966515988 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f25180664d0 0x7f2518068990 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.375 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.370+0000 7f25331d9640 1 --2- 172.21.15.88:0/966515988 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f252c0c4840 0x7f252c0c4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.375 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.370+0000 7f25331d9640 1 --2- 172.21.15.88:0/966515988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f252c0b42d0 0x7f252c15d230 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.375 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.370+0000 7f25331d9640 1 --2- 172.21.15.88:0/966515988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25140047d0 0x7f252c0c8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.375 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.370+0000 7f25331d9640 1 -- 172.21.15.88:0/966515988 >> 172.21.15.88:0/966515988 conn(0x7f252c070aa0 msgr2=0x7f252c0c9400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:47.375 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.370+0000 7f25331d9640 1 -- 172.21.15.88:0/966515988 shutdown_connections 2023-10-16T20:06:47.375 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.370+0000 7f25331d9640 1 -- 172.21.15.88:0/966515988 wait complete. 2023-10-16T20:06:47.392 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:06:47.393 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:06:47.402 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:06:47.579 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.574+0000 7f740c971640 1 Processor -- start 2023-10-16T20:06:47.579 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.574+0000 7f740c971640 1 -- start start 2023-10-16T20:06:47.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.578+0000 7f740c971640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f740815c9a0 0x7f74080cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:47.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.578+0000 7f740c971640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f74080d3270 0x7f74080d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:47.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.578+0000 7f740c971640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f74080d1610 0x7f74080cf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:47.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.578+0000 7f740c971640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74080ce7e0 con 0x7f74080d1610 2023-10-16T20:06:47.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.578+0000 7f740c971640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74080d0c40 con 0x7f740815c9a0 2023-10-16T20:06:47.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.578+0000 7f740c971640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74080d0630 con 0x7f74080d3270 2023-10-16T20:06:47.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.578+0000 7f740759e640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f74080d1610 0x7f74080cf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:47.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.578+0000 7f740759e640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f74080d1610 0x7f74080cf230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51088/0 (socket says 172.21.15.88:51088) 2023-10-16T20:06:47.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.578+0000 7f740759e640 1 -- 172.21.15.88:0/4207792925 learned_addr learned my addr 172.21.15.88:0/4207792925 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:47.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.578+0000 7f7406d9d640 1 --2- 172.21.15.88:0/4207792925 >> v2:172.21.15.94:3300/0 conn(0x7f740815c9a0 0x7f74080cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:47.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.578+0000 7f73fffff640 1 --2- 172.21.15.88:0/4207792925 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f74080d3270 0x7f74080d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:47.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.578+0000 7f740759e640 1 -- 172.21.15.88:0/4207792925 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f74080d3270 msgr2=0x7f74080d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:47.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.578+0000 7f740759e640 1 --2- 172.21.15.88:0/4207792925 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f74080d3270 0x7f74080d10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.584 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.578+0000 7f740759e640 1 -- 172.21.15.88:0/4207792925 >> v2:172.21.15.94:3300/0 conn(0x7f740815c9a0 msgr2=0x7f74080cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:47.584 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.578+0000 7f740759e640 1 --2- 172.21.15.88:0/4207792925 >> v2:172.21.15.94:3300/0 conn(0x7f740815c9a0 0x7f74080cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.584 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.578+0000 7f740759e640 1 -- 172.21.15.88:0/4207792925 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74080ce520 con 0x7f74080d1610 2023-10-16T20:06:47.584 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.578+0000 7f7406d9d640 1 --2- 172.21.15.88:0/4207792925 >> v2:172.21.15.94:3300/0 conn(0x7f740815c9a0 0x7f74080cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:47.585 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.582+0000 7f740759e640 1 --2- 172.21.15.88:0/4207792925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74080d1610 0x7f74080cf230 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f740808c070 tx=0x7f740001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec0bf0df57b9c35e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:47.585 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.582+0000 7f740659c640 1 -- 172.21.15.88:0/4207792925 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74000a5840 con 0x7f74080d1610 2023-10-16T20:06:47.586 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.582+0000 7f740659c640 1 -- 172.21.15.88:0/4207792925 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74000a59a0 con 0x7f74080d1610 2023-10-16T20:06:47.586 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.582+0000 7f740659c640 1 -- 172.21.15.88:0/4207792925 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74000a5d30 con 0x7f74080d1610 2023-10-16T20:06:47.586 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.582+0000 7f740c971640 1 -- 172.21.15.88:0/4207792925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74080d1610 msgr2=0x7f74080cf230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:47.586 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.582+0000 7f740c971640 1 --2- 172.21.15.88:0/4207792925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74080d1610 0x7f74080cf230 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f740808c070 tx=0x7f740001c9a0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.586 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.582+0000 7f740c971640 1 -- 172.21.15.88:0/4207792925 shutdown_connections 2023-10-16T20:06:47.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.582+0000 7f740c971640 1 --2- 172.21.15.88:0/4207792925 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f74080d3270 0x7f74080d10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.582+0000 7f740c971640 1 --2- 172.21.15.88:0/4207792925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74080d1610 0x7f74080cf230 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.582+0000 7f740c971640 1 --2- 172.21.15.88:0/4207792925 >> v2:172.21.15.94:3300/0 conn(0x7f740815c9a0 0x7f74080cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.582+0000 7f740c971640 1 -- 172.21.15.88:0/4207792925 >> 172.21.15.88:0/4207792925 conn(0x7f740809bc30 msgr2=0x7f74080d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:47.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.582+0000 7f740c971640 1 -- 172.21.15.88:0/4207792925 shutdown_connections 2023-10-16T20:06:47.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.582+0000 7f740c971640 1 -- 172.21.15.88:0/4207792925 wait complete. 2023-10-16T20:06:47.589 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.582+0000 7f740c971640 1 Processor -- start 2023-10-16T20:06:47.590 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.582+0000 7f740c971640 1 -- start start 2023-10-16T20:06:47.590 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.586+0000 7f740c971640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74080d1610 0x7f740815ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:47.590 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.586+0000 7f740c971640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f74080d3270 0x7f74080ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:47.590 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.586+0000 7f740c971640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f740815c9a0 0x7f74080ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:47.591 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.586+0000 7f740c971640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740815c5d0 con 0x7f74080d1610 2023-10-16T20:06:47.591 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.586+0000 7f740c971640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740815c140 con 0x7f740815c9a0 2023-10-16T20:06:47.591 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.586+0000 7f740c971640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740815b530 con 0x7f74080d3270 2023-10-16T20:06:47.591 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.586+0000 7f7406d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74080d1610 0x7f740815ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:47.591 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.586+0000 7f7406d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74080d1610 0x7f740815ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51096/0 (socket says 172.21.15.88:51096) 2023-10-16T20:06:47.592 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.586+0000 7f7406d9d640 1 -- 172.21.15.88:0/1626380462 learned_addr learned my addr 172.21.15.88:0/1626380462 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:47.592 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.586+0000 7f740759e640 1 --2- 172.21.15.88:0/1626380462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f740815c9a0 0x7f74080ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:47.592 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.586+0000 7f73fffff640 1 --2- 172.21.15.88:0/1626380462 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f74080d3270 0x7f74080ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:47.592 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.586+0000 7f7406d9d640 1 -- 172.21.15.88:0/1626380462 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f74080d3270 msgr2=0x7f74080ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:47.592 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.586+0000 7f7406d9d640 1 --2- 172.21.15.88:0/1626380462 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f74080d3270 0x7f74080ca280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.592 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.586+0000 7f7406d9d640 1 -- 172.21.15.88:0/1626380462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f740815c9a0 msgr2=0x7f74080ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:47.593 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.586+0000 7f7406d9d640 1 --2- 172.21.15.88:0/1626380462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f740815c9a0 0x7f74080ca7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.593 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.586+0000 7f7406d9d640 1 -- 172.21.15.88:0/1626380462 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7400009b30 con 0x7f74080d1610 2023-10-16T20:06:47.593 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.586+0000 7f73fffff640 1 --2- 172.21.15.88:0/1626380462 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f74080d3270 0x7f74080ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:47.593 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.586+0000 7f7406d9d640 1 --2- 172.21.15.88:0/1626380462 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74080d1610 0x7f740815ae70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f73f800e7e0 tx=0x7f73f800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:47.593 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.586+0000 7f73ff7fe640 1 -- 172.21.15.88:0/1626380462 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f80059a0 con 0x7f74080d1610 2023-10-16T20:06:47.593 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.586+0000 7f73ff7fe640 1 -- 172.21.15.88:0/1626380462 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f8005b00 con 0x7f74080d1610 2023-10-16T20:06:47.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.586+0000 7f740c971640 1 -- 172.21.15.88:0/1626380462 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f740815cf00 con 0x7f74080d1610 2023-10-16T20:06:47.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.586+0000 7f73ff7fe640 1 -- 172.21.15.88:0/1626380462 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f8005e90 con 0x7f74080d1610 2023-10-16T20:06:47.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.586+0000 7f740c971640 1 -- 172.21.15.88:0/1626380462 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f740815baf0 con 0x7f74080d1610 2023-10-16T20:06:47.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.586+0000 7f740c971640 1 -- 172.21.15.88:0/1626380462 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74080674f0 con 0x7f74080d1610 2023-10-16T20:06:47.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.586+0000 7f73ff7fe640 1 -- 172.21.15.88:0/1626380462 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f73f8004030 con 0x7f74080d1610 2023-10-16T20:06:47.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.590+0000 7f73ff7fe640 1 --2- 172.21.15.88:0/1626380462 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f73e80664a0 0x7f73e8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:47.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.590+0000 7f73ff7fe640 1 -- 172.21.15.88:0/1626380462 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f73f8084280 con 0x7f74080d1610 2023-10-16T20:06:47.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.590+0000 7f73fffff640 1 --2- 172.21.15.88:0/1626380462 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f73e80664a0 0x7f73e8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:47.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.590+0000 7f73fffff640 1 --2- 172.21.15.88:0/1626380462 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f73e80664a0 0x7f73e8068960 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f74080c9ab0 tx=0x7f73f0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:47.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.594+0000 7f73ff7fe640 1 -- 172.21.15.88:0/1626380462 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f73f8051360 con 0x7f74080d1610 2023-10-16T20:06:47.747 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.742+0000 7f740c971640 1 -- 172.21.15.88:0/1626380462 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74080cff40 con 0x7f74080d1610 2023-10-16T20:06:47.747 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.746+0000 7f73ff7fe640 1 -- 172.21.15.88:0/1626380462 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+18499 (secure 0 0 0) 0x7f73f8054980 con 0x7f74080d1610 2023-10-16T20:06:47.748 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:06:47.748 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":46,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:06:38.250351+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T20:06:14.743500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"testcephfs1_data","create_time":"2023-10-16T20:06:16.091837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-10-16T20:06:31.184462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.39","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:06:47.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.746+0000 7f740c971640 1 -- 172.21.15.88:0/1626380462 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f73e80664a0 msgr2=0x7f73e8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:47.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.746+0000 7f740c971640 1 --2- 172.21.15.88:0/1626380462 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f73e80664a0 0x7f73e8068960 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f74080c9ab0 tx=0x7f73f0008040 comp rx=0 tx=0).stop 2023-10-16T20:06:47.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.746+0000 7f740c971640 1 -- 172.21.15.88:0/1626380462 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74080d1610 msgr2=0x7f740815ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:47.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.746+0000 7f740c971640 1 --2- 172.21.15.88:0/1626380462 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74080d1610 0x7f740815ae70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f73f800e7e0 tx=0x7f73f800eca0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.746+0000 7f740c971640 1 -- 172.21.15.88:0/1626380462 shutdown_connections 2023-10-16T20:06:47.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.746+0000 7f740c971640 1 --2- 172.21.15.88:0/1626380462 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f73e80664a0 0x7f73e8068960 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.746+0000 7f740c971640 1 --2- 172.21.15.88:0/1626380462 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f74080d3270 0x7f74080ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.746+0000 7f740c971640 1 --2- 172.21.15.88:0/1626380462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f740815c9a0 0x7f74080ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.746+0000 7f740c971640 1 --2- 172.21.15.88:0/1626380462 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74080d1610 0x7f740815ae70 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.752 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.746+0000 7f740c971640 1 -- 172.21.15.88:0/1626380462 >> 172.21.15.88:0/1626380462 conn(0x7f740809bc30 msgr2=0x7f74080b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:47.752 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.746+0000 7f740c971640 1 -- 172.21.15.88:0/1626380462 shutdown_connections 2023-10-16T20:06:47.752 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.746+0000 7f740c971640 1 -- 172.21.15.88:0/1626380462 wait complete. 2023-10-16T20:06:47.769 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'testcephfs1_metadata', 'create_time': '2023-10-16T20:06:14.743500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'testcephfs1_data', 'create_time': '2023-10-16T20:06:16.091837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2023-10-16T20:06:31.184462+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T20:06:47.770 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:06:47.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.906+0000 7f2960eea640 1 Processor -- start 2023-10-16T20:06:47.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.906+0000 7f2960eea640 1 -- start start 2023-10-16T20:06:47.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.906+0000 7f2960eea640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f295c0b0da0 0x7f295c159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:47.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.906+0000 7f2960eea640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f295c15a1f0 0x7f295c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:47.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.906+0000 7f2960eea640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f295c15ad30 con 0x7f295c0b4260 2023-10-16T20:06:47.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.906+0000 7f2960eea640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f295c0b4640 con 0x7f295c15a1f0 2023-10-16T20:06:47.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.906+0000 7f2960eea640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295c15a770 con 0x7f295c0b0da0 2023-10-16T20:06:47.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.910+0000 7f295a59c640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f295c0b4260 0x7f295c15dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:38034/0 (socket says 172.21.15.88:38034) 2023-10-16T20:06:47.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.910+0000 7f295a59c640 1 -- 172.21.15.88:0/2379794162 learned_addr learned my addr 172.21.15.88:0/2379794162 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:47.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.910+0000 7f295ad9d640 1 --2- 172.21.15.88:0/2379794162 >> v2:172.21.15.94:3300/0 conn(0x7f295c15a1f0 0x7f295c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:47.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.910+0000 7f2951d9b640 1 --2- 172.21.15.88:0/2379794162 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f295c0b0da0 0x7f295c159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:47.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.910+0000 7f2959d9b640 1 -- 172.21.15.88:0/2379794162 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1913067183 0 0) 0x7f295c15ad30 con 0x7f295c0b4260 2023-10-16T20:06:47.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.910+0000 7f2959d9b640 1 -- 172.21.15.88:0/2379794162 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2948003530 con 0x7f295c0b4260 2023-10-16T20:06:47.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.910+0000 7f295ad9d640 1 -- 172.21.15.88:0/2379794162 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f295c0b0da0 msgr2=0x7f295c159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:47.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.910+0000 7f295ad9d640 1 --2- 172.21.15.88:0/2379794162 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f295c0b0da0 0x7f295c159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.910+0000 7f295ad9d640 1 -- 172.21.15.88:0/2379794162 >> v1:172.21.15.88:6789/0 conn(0x7f295c0b4260 legacy=0x7f295c15dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:47.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.910+0000 7f295ad9d640 1 -- 172.21.15.88:0/2379794162 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295c15aa50 con 0x7f295c15a1f0 2023-10-16T20:06:47.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.910+0000 7f295ad9d640 1 --2- 172.21.15.88:0/2379794162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f295c15a1f0 0x7f295c06d3f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f29540097c0 tx=0x7f295401c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b09bcc6e46a865ca server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:47.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.914+0000 7f2959d9b640 1 -- 172.21.15.88:0/2379794162 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29540ab600 con 0x7f295c15a1f0 2023-10-16T20:06:47.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.914+0000 7f2959d9b640 1 -- 172.21.15.88:0/2379794162 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29540a7050 con 0x7f295c15a1f0 2023-10-16T20:06:47.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.914+0000 7f2959d9b640 1 -- 172.21.15.88:0/2379794162 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29540ab980 con 0x7f295c15a1f0 2023-10-16T20:06:47.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.914+0000 7f2960eea640 1 -- 172.21.15.88:0/2379794162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f295c15a1f0 msgr2=0x7f295c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:47.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.914+0000 7f2960eea640 1 --2- 172.21.15.88:0/2379794162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f295c15a1f0 0x7f295c06d3f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f29540097c0 tx=0x7f295401c3f0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.914+0000 7f2960eea640 1 -- 172.21.15.88:0/2379794162 shutdown_connections 2023-10-16T20:06:47.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.914+0000 7f2960eea640 1 --2- 172.21.15.88:0/2379794162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f295c15a1f0 0x7f295c06d3f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.914+0000 7f2960eea640 1 --2- 172.21.15.88:0/2379794162 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f295c0b0da0 0x7f295c159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.914+0000 7f2960eea640 1 -- 172.21.15.88:0/2379794162 >> 172.21.15.88:0/2379794162 conn(0x7f295c0cb080 msgr2=0x7f295c0cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:47.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.914+0000 7f2960eea640 1 -- 172.21.15.88:0/2379794162 shutdown_connections 2023-10-16T20:06:47.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.914+0000 7f2960eea640 1 -- 172.21.15.88:0/2379794162 wait complete. 2023-10-16T20:06:47.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.914+0000 7f2960eea640 1 Processor -- start 2023-10-16T20:06:47.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.914+0000 7f2960eea640 1 -- start start 2023-10-16T20:06:47.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.914+0000 7f2960eea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f295c0b0da0 0x7f295c14ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:47.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.914+0000 7f2960eea640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f295c0b4260 0x7f295c152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:47.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.914+0000 7f2960eea640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f295c15a1f0 0x7f295c150b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:47.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.914+0000 7f2960eea640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c154330 con 0x7f295c15a1f0 2023-10-16T20:06:47.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.914+0000 7f2960eea640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c14ef50 con 0x7f295c0b0da0 2023-10-16T20:06:47.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.914+0000 7f2960eea640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295c1545d0 con 0x7f295c0b4260 2023-10-16T20:06:47.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.918+0000 7f295ad9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f295c15a1f0 0x7f295c150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:47.922 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.918+0000 7f295ad9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f295c15a1f0 0x7f295c150b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51110/0 (socket says 172.21.15.88:51110) 2023-10-16T20:06:47.922 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.918+0000 7f295ad9d640 1 -- 172.21.15.88:0/2725270861 learned_addr learned my addr 172.21.15.88:0/2725270861 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:47.922 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.918+0000 7f2951d9b640 1 --2- 172.21.15.88:0/2725270861 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f295c0b4260 0x7f295c152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:47.922 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.918+0000 7f295a59c640 1 --2- 172.21.15.88:0/2725270861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f295c0b0da0 0x7f295c14ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:47.922 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.918+0000 7f295ad9d640 1 -- 172.21.15.88:0/2725270861 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f295c0b4260 msgr2=0x7f295c152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:47.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.918+0000 7f295ad9d640 1 --2- 172.21.15.88:0/2725270861 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f295c0b4260 0x7f295c152a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.918+0000 7f295ad9d640 1 -- 172.21.15.88:0/2725270861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f295c0b0da0 msgr2=0x7f295c14ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:47.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.918+0000 7f295ad9d640 1 --2- 172.21.15.88:0/2725270861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f295c0b0da0 0x7f295c14ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:47.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.918+0000 7f295ad9d640 1 -- 172.21.15.88:0/2725270861 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2954009420 con 0x7f295c15a1f0 2023-10-16T20:06:47.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.918+0000 7f295ad9d640 1 --2- 172.21.15.88:0/2725270861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f295c15a1f0 0x7f295c150b90 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f295401a020 tx=0x7f2954019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:47.925 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.918+0000 7f29537fe640 1 -- 172.21.15.88:0/2725270861 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29540abe90 con 0x7f295c15a1f0 2023-10-16T20:06:47.925 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.918+0000 7f29537fe640 1 -- 172.21.15.88:0/2725270861 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29540b4430 con 0x7f295c15a1f0 2023-10-16T20:06:47.925 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.918+0000 7f2960eea640 1 -- 172.21.15.88:0/2725270861 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f295c1548b0 con 0x7f295c15a1f0 2023-10-16T20:06:47.926 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.918+0000 7f29537fe640 1 -- 172.21.15.88:0/2725270861 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29540b47c0 con 0x7f295c15a1f0 2023-10-16T20:06:47.926 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.918+0000 7f2960eea640 1 -- 172.21.15.88:0/2725270861 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f295c152fc0 con 0x7f295c15a1f0 2023-10-16T20:06:47.926 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.918+0000 7f2960eea640 1 -- 172.21.15.88:0/2725270861 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f295c04e9c0 con 0x7f295c15a1f0 2023-10-16T20:06:47.926 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.918+0000 7f29537fe640 1 -- 172.21.15.88:0/2725270861 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f29540b2040 con 0x7f295c15a1f0 2023-10-16T20:06:47.926 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.922+0000 7f29537fe640 1 --2- 172.21.15.88:0/2725270861 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f293c0664d0 0x7f293c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:47.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.922+0000 7f29537fe640 1 -- 172.21.15.88:0/2725270861 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f295411dde0 con 0x7f295c15a1f0 2023-10-16T20:06:47.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.922+0000 7f295a59c640 1 --2- 172.21.15.88:0/2725270861 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f293c0664d0 0x7f293c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:47.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.922+0000 7f295a59c640 1 --2- 172.21.15.88:0/2725270861 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f293c0664d0 0x7f293c068990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f294c001570 tx=0x7f294c001aa0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:47.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:47.926+0000 7f29537fe640 1 -- 172.21.15.88:0/2725270861 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f29540eb000 con 0x7f295c15a1f0 2023-10-16T20:06:48.077 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.074+0000 7f2960eea640 1 -- 172.21.15.88:0/2725270861 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f295c14cd40 con 0x7f295c15a1f0 2023-10-16T20:06:48.077 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.074+0000 7f29537fe640 1 -- 172.21.15.88:0/2725270861 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f29540b0020 con 0x7f295c15a1f0 2023-10-16T20:06:48.078 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:06:48.079 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.074+0000 7f2960eea640 1 -- 172.21.15.88:0/2725270861 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f293c0664d0 msgr2=0x7f293c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:48.079 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.074+0000 7f2960eea640 1 --2- 172.21.15.88:0/2725270861 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f293c0664d0 0x7f293c068990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f294c001570 tx=0x7f294c001aa0 comp rx=0 tx=0).stop 2023-10-16T20:06:48.080 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.074+0000 7f2960eea640 1 -- 172.21.15.88:0/2725270861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f295c15a1f0 msgr2=0x7f295c150b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:48.080 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.074+0000 7f2960eea640 1 --2- 172.21.15.88:0/2725270861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f295c15a1f0 0x7f295c150b90 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f295401a020 tx=0x7f2954019190 comp rx=0 tx=0).stop 2023-10-16T20:06:48.080 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.074+0000 7f2960eea640 1 -- 172.21.15.88:0/2725270861 shutdown_connections 2023-10-16T20:06:48.080 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.074+0000 7f2960eea640 1 --2- 172.21.15.88:0/2725270861 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f293c0664d0 0x7f293c068990 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:48.080 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.074+0000 7f2960eea640 1 --2- 172.21.15.88:0/2725270861 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f295c0b4260 0x7f295c152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:48.081 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.074+0000 7f2960eea640 1 --2- 172.21.15.88:0/2725270861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f295c15a1f0 0x7f295c150b90 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:48.081 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.074+0000 7f2960eea640 1 --2- 172.21.15.88:0/2725270861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f295c0b0da0 0x7f295c14ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:48.081 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.074+0000 7f2960eea640 1 -- 172.21.15.88:0/2725270861 >> 172.21.15.88:0/2725270861 conn(0x7f295c0cb080 msgr2=0x7f295c0afaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:48.081 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.078+0000 7f2960eea640 1 -- 172.21.15.88:0/2725270861 shutdown_connections 2023-10-16T20:06:48.081 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.078+0000 7f2960eea640 1 -- 172.21.15.88:0/2725270861 wait complete. 2023-10-16T20:06:48.098 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_metadata pg_num 2023-10-16T20:06:48.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.238+0000 7f58adf3e640 1 Processor -- start 2023-10-16T20:06:48.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.238+0000 7f58adf3e640 1 -- start start 2023-10-16T20:06:48.242 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.238+0000 7f58adf3e640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f58a80d1be0 0x7f58a80b4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:48.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.238+0000 7f58adf3e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f58a80b4a70 0x7f58a80b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:48.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.238+0000 7f58adf3e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58a80b7d20 con 0x7f58a815cb70 2023-10-16T20:06:48.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.238+0000 7f58adf3e640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a80b4ea0 con 0x7f58a80d1be0 2023-10-16T20:06:48.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.238+0000 7f58adf3e640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a80b7110 con 0x7f58a80b4a70 2023-10-16T20:06:48.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.238+0000 7f58a6ffd640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f58a80d1be0 0x7f58a80b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:48.244 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.238+0000 7f58a6ffd640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f58a80d1be0 0x7f58a80b4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51116/0 (socket says 172.21.15.88:51116) 2023-10-16T20:06:48.244 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.238+0000 7f58a6ffd640 1 -- 172.21.15.88:0/1657364561 learned_addr learned my addr 172.21.15.88:0/1657364561 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:48.244 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.238+0000 7f58a7fff640 1 --2- 172.21.15.88:0/1657364561 >> v2:172.21.15.94:3300/0 conn(0x7f58a80b4a70 0x7f58a80b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:48.244 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.238+0000 7f58a6ffd640 1 -- 172.21.15.88:0/1657364561 >> v2:172.21.15.94:3300/0 conn(0x7f58a80b4a70 msgr2=0x7f58a80b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:48.244 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.238+0000 7f58a6ffd640 1 --2- 172.21.15.88:0/1657364561 >> v2:172.21.15.94:3300/0 conn(0x7f58a80b4a70 0x7f58a80b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:48.245 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.238+0000 7f58a6ffd640 1 -- 172.21.15.88:0/1657364561 >> v1:172.21.15.94:6789/0 conn(0x7f58a815cb70 legacy=0x7f58a80b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:48.245 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.238+0000 7f58a6ffd640 1 -- 172.21.15.88:0/1657364561 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a80b73f0 con 0x7f58a80d1be0 2023-10-16T20:06:48.245 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.238+0000 7f58a7fff640 1 --2- 172.21.15.88:0/1657364561 >> v2:172.21.15.94:3300/0 conn(0x7f58a80b4a70 0x7f58a80b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:48.246 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.242+0000 7f58a6ffd640 1 --2- 172.21.15.88:0/1657364561 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58a80d1be0 0x7f58a80b4530 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f58a80d03c0 tx=0x7f589c01ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df99bd8e4e98550 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:48.247 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.242+0000 7f58a67fc640 1 -- 172.21.15.88:0/1657364561 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c0a89f0 con 0x7f58a80d1be0 2023-10-16T20:06:48.247 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.242+0000 7f58a67fc640 1 -- 172.21.15.88:0/1657364561 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589c0a8b50 con 0x7f58a80d1be0 2023-10-16T20:06:48.247 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.242+0000 7f58a67fc640 1 -- 172.21.15.88:0/1657364561 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c0b0440 con 0x7f58a80d1be0 2023-10-16T20:06:48.247 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.242+0000 7f58adf3e640 1 -- 172.21.15.88:0/1657364561 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58a80d1be0 msgr2=0x7f58a80b4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:48.247 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.242+0000 7f58adf3e640 1 --2- 172.21.15.88:0/1657364561 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58a80d1be0 0x7f58a80b4530 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f58a80d03c0 tx=0x7f589c01ecf0 comp rx=0 tx=0).stop 2023-10-16T20:06:48.248 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.242+0000 7f58adf3e640 1 -- 172.21.15.88:0/1657364561 shutdown_connections 2023-10-16T20:06:48.248 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.242+0000 7f58adf3e640 1 --2- 172.21.15.88:0/1657364561 >> v2:172.21.15.94:3300/0 conn(0x7f58a80b4a70 0x7f58a80b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:48.248 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.242+0000 7f58adf3e640 1 --2- 172.21.15.88:0/1657364561 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58a80d1be0 0x7f58a80b4530 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:48.248 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.242+0000 7f58adf3e640 1 -- 172.21.15.88:0/1657364561 >> 172.21.15.88:0/1657364561 conn(0x7f58a806b6e0 msgr2=0x7f58a806baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:48.248 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.242+0000 7f58adf3e640 1 -- 172.21.15.88:0/1657364561 shutdown_connections 2023-10-16T20:06:48.249 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.242+0000 7f58adf3e640 1 -- 172.21.15.88:0/1657364561 wait complete. 2023-10-16T20:06:48.251 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.246+0000 7f58adf3e640 1 Processor -- start 2023-10-16T20:06:48.251 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.246+0000 7f58adf3e640 1 -- start start 2023-10-16T20:06:48.251 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.246+0000 7f58adf3e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f58a80b4a70 0x7f58a8151920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:48.251 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.246+0000 7f58adf3e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58a80d1be0 0x7f58a80c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:48.251 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.246+0000 7f58adf3e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58a815cb70 0x7f58a8150b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:48.252 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.246+0000 7f58adf3e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a8152550 con 0x7f58a80d1be0 2023-10-16T20:06:48.252 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.246+0000 7f58adf3e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a8152270 con 0x7f58a815cb70 2023-10-16T20:06:48.252 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.246+0000 7f58adf3e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a8153c10 con 0x7f58a80b4a70 2023-10-16T20:06:48.252 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.246+0000 7f58a6ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58a80d1be0 0x7f58a80c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:48.252 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.246+0000 7f58a6ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58a80d1be0 0x7f58a80c89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51128/0 (socket says 172.21.15.88:51128) 2023-10-16T20:06:48.252 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.246+0000 7f58a6ffd640 1 -- 172.21.15.88:0/1760501833 learned_addr learned my addr 172.21.15.88:0/1760501833 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:48.253 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.246+0000 7f58a77fe640 1 --2- 172.21.15.88:0/1760501833 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f58a80b4a70 0x7f58a8151920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:48.253 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.246+0000 7f58a7fff640 1 --2- 172.21.15.88:0/1760501833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58a815cb70 0x7f58a8150b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:48.253 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.246+0000 7f58a6ffd640 1 -- 172.21.15.88:0/1760501833 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f58a80b4a70 msgr2=0x7f58a8151920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:48.253 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.246+0000 7f58a6ffd640 1 --2- 172.21.15.88:0/1760501833 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f58a80b4a70 0x7f58a8151920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:48.253 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.246+0000 7f58a6ffd640 1 -- 172.21.15.88:0/1760501833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58a815cb70 msgr2=0x7f58a8150b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:48.254 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.246+0000 7f58a6ffd640 1 --2- 172.21.15.88:0/1760501833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58a815cb70 0x7f58a8150b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:48.254 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.246+0000 7f58a6ffd640 1 -- 172.21.15.88:0/1760501833 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f589c0094d0 con 0x7f58a80d1be0 2023-10-16T20:06:48.254 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.246+0000 7f58a6ffd640 1 --2- 172.21.15.88:0/1760501833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58a80d1be0 0x7f58a80c89f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f589c01b900 tx=0x7f589c01ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:48.254 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.246+0000 7f588ffff640 1 -- 172.21.15.88:0/1760501833 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c0ba600 con 0x7f58a80d1be0 2023-10-16T20:06:48.254 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.246+0000 7f588ffff640 1 -- 172.21.15.88:0/1760501833 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589c0ba760 con 0x7f58a80d1be0 2023-10-16T20:06:48.255 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.246+0000 7f588ffff640 1 -- 172.21.15.88:0/1760501833 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c0bab10 con 0x7f58a80d1be0 2023-10-16T20:06:48.255 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.246+0000 7f58adf3e640 1 -- 172.21.15.88:0/1760501833 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a8153ef0 con 0x7f58a80d1be0 2023-10-16T20:06:48.255 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.246+0000 7f58adf3e640 1 -- 172.21.15.88:0/1760501833 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a8151e60 con 0x7f58a80d1be0 2023-10-16T20:06:48.256 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.250+0000 7f58adf3e640 1 -- 172.21.15.88:0/1760501833 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a8153a50 con 0x7f58a80d1be0 2023-10-16T20:06:48.256 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.250+0000 7f588ffff640 1 -- 172.21.15.88:0/1760501833 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f589c0bac70 con 0x7f58a80d1be0 2023-10-16T20:06:48.256 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.250+0000 7f588ffff640 1 --2- 172.21.15.88:0/1760501833 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f58940664d0 0x7f5894068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:48.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.250+0000 7f588ffff640 1 -- 172.21.15.88:0/1760501833 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f589c1225a0 con 0x7f58a80d1be0 2023-10-16T20:06:48.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.250+0000 7f58a77fe640 1 --2- 172.21.15.88:0/1760501833 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f58940664d0 0x7f5894068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:48.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.250+0000 7f58a77fe640 1 --2- 172.21.15.88:0/1760501833 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f58940664d0 0x7f5894068990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f58a80c9350 tx=0x7f5898009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:48.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.254+0000 7f588ffff640 1 -- 172.21.15.88:0/1760501833 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f589c0ef7c0 con 0x7f58a80d1be0 2023-10-16T20:06:48.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.406+0000 7f58adf3e640 1 -- 172.21.15.88:0/1760501833 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"} v 0) v1 -- 0x7f58a804e9c0 con 0x7f58a80d1be0 2023-10-16T20:06:48.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.406+0000 7f588ffff640 1 -- 172.21.15.88:0/1760501833 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"}]=0 v46) v1 ==== 109+0+11 (secure 0 0 0) 0x7f589c0b5020 con 0x7f58a80d1be0 2023-10-16T20:06:48.409 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:06:48.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.406+0000 7f58adf3e640 1 -- 172.21.15.88:0/1760501833 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f58940664d0 msgr2=0x7f5894068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:48.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.406+0000 7f58adf3e640 1 --2- 172.21.15.88:0/1760501833 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f58940664d0 0x7f5894068990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f58a80c9350 tx=0x7f5898009040 comp rx=0 tx=0).stop 2023-10-16T20:06:48.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.406+0000 7f58adf3e640 1 -- 172.21.15.88:0/1760501833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58a80d1be0 msgr2=0x7f58a80c89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:48.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.406+0000 7f58adf3e640 1 --2- 172.21.15.88:0/1760501833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58a80d1be0 0x7f58a80c89f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f589c01b900 tx=0x7f589c01ba90 comp rx=0 tx=0).stop 2023-10-16T20:06:48.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.406+0000 7f58adf3e640 1 -- 172.21.15.88:0/1760501833 shutdown_connections 2023-10-16T20:06:48.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.406+0000 7f58adf3e640 1 --2- 172.21.15.88:0/1760501833 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f58940664d0 0x7f5894068990 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:48.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.406+0000 7f58adf3e640 1 --2- 172.21.15.88:0/1760501833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58a815cb70 0x7f58a8150b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:48.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.406+0000 7f58adf3e640 1 --2- 172.21.15.88:0/1760501833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58a80d1be0 0x7f58a80c89f0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:48.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.406+0000 7f58adf3e640 1 --2- 172.21.15.88:0/1760501833 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f58a80b4a70 0x7f58a8151920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:48.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.406+0000 7f58adf3e640 1 -- 172.21.15.88:0/1760501833 >> 172.21.15.88:0/1760501833 conn(0x7f58a806b6e0 msgr2=0x7f58a8157720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:48.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.406+0000 7f58adf3e640 1 -- 172.21.15.88:0/1760501833 shutdown_connections 2023-10-16T20:06:48.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.406+0000 7f58adf3e640 1 -- 172.21.15.88:0/1760501833 wait complete. 2023-10-16T20:06:48.429 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_data pg_num 2023-10-16T20:06:48.571 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.566+0000 7ff4f4565640 1 Processor -- start 2023-10-16T20:06:48.571 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.566+0000 7ff4f4565640 1 -- start start 2023-10-16T20:06:48.573 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.570+0000 7ff4f4565640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff4ec159f50 0x7ff4ec06c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:48.573 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.570+0000 7ff4f4565640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4ec15a400 con 0x7ff4ec0711f0 2023-10-16T20:06:48.573 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.570+0000 7ff4f4565640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4ec15aa10 con 0x7ff4ec1500a0 2023-10-16T20:06:48.573 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.570+0000 7ff4f4565640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4ec150480 con 0x7ff4ec159f50 2023-10-16T20:06:48.573 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.570+0000 7ff4f1b00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7ff4ec0711f0 0x7ff4ec15aef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:38036/0 (socket says 172.21.15.88:38036) 2023-10-16T20:06:48.573 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.570+0000 7ff4f1b00640 1 -- 172.21.15.88:0/3514778043 learned_addr learned my addr 172.21.15.88:0/3514778043 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:48.574 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.570+0000 7ff4f2b02640 1 --2- 172.21.15.88:0/3514778043 >> v2:172.21.15.88:3300/0 conn(0x7ff4ec159f50 0x7ff4ec06c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:48.574 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.570+0000 7ff4f12ff640 1 -- 172.21.15.88:0/3514778043 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1306116723 0 0) 0x7ff4ec15a400 con 0x7ff4ec0711f0 2023-10-16T20:06:48.575 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.570+0000 7ff4f2b02640 1 -- 172.21.15.88:0/3514778043 >> v1:172.21.15.94:6789/0 conn(0x7ff4ec1500a0 legacy=0x7ff4ec070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:48.575 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.570+0000 7ff4f2b02640 1 -- 172.21.15.88:0/3514778043 >> v1:172.21.15.88:6789/0 conn(0x7ff4ec0711f0 legacy=0x7ff4ec15aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:48.575 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.570+0000 7ff4f2b02640 1 -- 172.21.15.88:0/3514778043 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ec06ca10 con 0x7ff4ec159f50 2023-10-16T20:06:48.576 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.574+0000 7ff4f2b02640 1 --2- 172.21.15.88:0/3514778043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4ec159f50 0x7ff4ec06c2e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff4e400aea0 tx=0x7ff4e401d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a93dd0a78e2a50 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:48.576 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.574+0000 7ff4f12ff640 1 -- 172.21.15.88:0/3514778043 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e400a2d0 con 0x7ff4ec159f50 2023-10-16T20:06:48.577 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.574+0000 7ff4f12ff640 1 -- 172.21.15.88:0/3514778043 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e40a6620 con 0x7ff4ec159f50 2023-10-16T20:06:48.577 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.574+0000 7ff4f12ff640 1 -- 172.21.15.88:0/3514778043 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e40a69b0 con 0x7ff4ec159f50 2023-10-16T20:06:48.577 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.574+0000 7ff4f4565640 1 -- 172.21.15.88:0/3514778043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4ec159f50 msgr2=0x7ff4ec06c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:48.578 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.574+0000 7ff4f4565640 1 --2- 172.21.15.88:0/3514778043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4ec159f50 0x7ff4ec06c2e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff4e400aea0 tx=0x7ff4e401d9c0 comp rx=0 tx=0).stop 2023-10-16T20:06:48.578 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.574+0000 7ff4f4565640 1 -- 172.21.15.88:0/3514778043 shutdown_connections 2023-10-16T20:06:48.578 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.574+0000 7ff4f4565640 1 --2- 172.21.15.88:0/3514778043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4ec159f50 0x7ff4ec06c2e0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:48.578 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.574+0000 7ff4f4565640 1 -- 172.21.15.88:0/3514778043 >> 172.21.15.88:0/3514778043 conn(0x7ff4ec0cb080 msgr2=0x7ff4ec0cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:48.579 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.574+0000 7ff4f4565640 1 -- 172.21.15.88:0/3514778043 shutdown_connections 2023-10-16T20:06:48.579 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.574+0000 7ff4f4565640 1 -- 172.21.15.88:0/3514778043 wait complete. 2023-10-16T20:06:48.579 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.574+0000 7ff4f4565640 1 Processor -- start 2023-10-16T20:06:48.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.574+0000 7ff4f4565640 1 -- start start 2023-10-16T20:06:48.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.578+0000 7ff4f4565640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4ec0711f0 0x7ff4ec155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:48.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.578+0000 7ff4f4565640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4ec1500a0 0x7ff4ec1452b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:48.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.578+0000 7ff4f4565640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4ec159f50 0x7ff4ec1475f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:48.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.578+0000 7ff4f4565640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec13bb60 con 0x7ff4ec1500a0 2023-10-16T20:06:48.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.578+0000 7ff4f4565640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec14a190 con 0x7ff4ec159f50 2023-10-16T20:06:48.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.578+0000 7ff4f4565640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec156ff0 con 0x7ff4ec0711f0 2023-10-16T20:06:48.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.578+0000 7ff4f1b00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4ec1500a0 0x7ff4ec1452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:48.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.578+0000 7ff4f1b00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4ec1500a0 0x7ff4ec1452b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51142/0 (socket says 172.21.15.88:51142) 2023-10-16T20:06:48.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.578+0000 7ff4f1b00640 1 -- 172.21.15.88:0/2772323049 learned_addr learned my addr 172.21.15.88:0/2772323049 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:48.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.578+0000 7ff4f2b02640 1 --2- 172.21.15.88:0/2772323049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4ec159f50 0x7ff4ec1475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:48.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.578+0000 7ff4f2301640 1 --2- 172.21.15.88:0/2772323049 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4ec0711f0 0x7ff4ec155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:48.584 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.578+0000 7ff4f1b00640 1 -- 172.21.15.88:0/2772323049 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4ec0711f0 msgr2=0x7ff4ec155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:48.584 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.578+0000 7ff4f1b00640 1 --2- 172.21.15.88:0/2772323049 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4ec0711f0 0x7ff4ec155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:48.584 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.578+0000 7ff4f1b00640 1 -- 172.21.15.88:0/2772323049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4ec159f50 msgr2=0x7ff4ec1475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:48.584 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.578+0000 7ff4f1b00640 1 --2- 172.21.15.88:0/2772323049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4ec159f50 0x7ff4ec1475f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:48.584 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.578+0000 7ff4f1b00640 1 -- 172.21.15.88:0/2772323049 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e400ab80 con 0x7ff4ec1500a0 2023-10-16T20:06:48.585 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.578+0000 7ff4f2301640 1 --2- 172.21.15.88:0/2772323049 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4ec0711f0 0x7ff4ec155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:48.585 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.578+0000 7ff4f1b00640 1 --2- 172.21.15.88:0/2772323049 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4ec1500a0 0x7ff4ec1452b0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc00c480 tx=0x7ff4dc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:48.585 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.578+0000 7ff4e2ffd640 1 -- 172.21.15.88:0/2772323049 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc0074e0 con 0x7ff4ec1500a0 2023-10-16T20:06:48.585 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.578+0000 7ff4e2ffd640 1 -- 172.21.15.88:0/2772323049 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4dc007640 con 0x7ff4ec1500a0 2023-10-16T20:06:48.585 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.578+0000 7ff4f4565640 1 -- 172.21.15.88:0/2772323049 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4ec14a430 con 0x7ff4ec1500a0 2023-10-16T20:06:48.586 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.578+0000 7ff4e2ffd640 1 -- 172.21.15.88:0/2772323049 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc004a10 con 0x7ff4ec1500a0 2023-10-16T20:06:48.586 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.578+0000 7ff4f4565640 1 -- 172.21.15.88:0/2772323049 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4ec14a9f0 con 0x7ff4ec1500a0 2023-10-16T20:06:48.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.578+0000 7ff4f4565640 1 -- 172.21.15.88:0/2772323049 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4ec1571b0 con 0x7ff4ec1500a0 2023-10-16T20:06:48.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.582+0000 7ff4e2ffd640 1 -- 172.21.15.88:0/2772323049 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff4dc004b70 con 0x7ff4ec1500a0 2023-10-16T20:06:48.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.582+0000 7ff4e2ffd640 1 --2- 172.21.15.88:0/2772323049 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff4d00664a0 0x7ff4d0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:48.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.582+0000 7ff4e2ffd640 1 -- 172.21.15.88:0/2772323049 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7ff4dc083600 con 0x7ff4ec1500a0 2023-10-16T20:06:48.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.582+0000 7ff4f2301640 1 --2- 172.21.15.88:0/2772323049 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff4d00664a0 0x7ff4d0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:48.588 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.582+0000 7ff4f2301640 1 --2- 172.21.15.88:0/2772323049 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff4d00664a0 0x7ff4d0068960 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8000f70 tx=0x7ff4e8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:48.588 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.586+0000 7ff4e2ffd640 1 -- 172.21.15.88:0/2772323049 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff4dc050820 con 0x7ff4ec1500a0 2023-10-16T20:06:48.738 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.734+0000 7ff4f4565640 1 -- 172.21.15.88:0/2772323049 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"} v 0) v1 -- 0x7ff4ec04e9c0 con 0x7ff4ec1500a0 2023-10-16T20:06:48.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.734+0000 7ff4e2ffd640 1 -- 172.21.15.88:0/2772323049 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"}]=0 v46) v1 ==== 105+0+11 (secure 0 0 0) 0x7ff4dc053e40 con 0x7ff4ec1500a0 2023-10-16T20:06:48.739 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2023-10-16T20:06:48.740 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.738+0000 7ff4f4565640 1 -- 172.21.15.88:0/2772323049 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff4d00664a0 msgr2=0x7ff4d0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:48.741 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.738+0000 7ff4f4565640 1 --2- 172.21.15.88:0/2772323049 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff4d00664a0 0x7ff4d0068960 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8000f70 tx=0x7ff4e8009040 comp rx=0 tx=0).stop 2023-10-16T20:06:48.741 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.738+0000 7ff4f4565640 1 -- 172.21.15.88:0/2772323049 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4ec1500a0 msgr2=0x7ff4ec1452b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:48.741 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.738+0000 7ff4f4565640 1 --2- 172.21.15.88:0/2772323049 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4ec1500a0 0x7ff4ec1452b0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc00c480 tx=0x7ff4dc00c940 comp rx=0 tx=0).stop 2023-10-16T20:06:48.741 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.738+0000 7ff4f4565640 1 -- 172.21.15.88:0/2772323049 shutdown_connections 2023-10-16T20:06:48.742 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.738+0000 7ff4f4565640 1 --2- 172.21.15.88:0/2772323049 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff4d00664a0 0x7ff4d0068960 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:48.742 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.738+0000 7ff4f4565640 1 --2- 172.21.15.88:0/2772323049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4ec159f50 0x7ff4ec1475f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:48.742 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.738+0000 7ff4f4565640 1 --2- 172.21.15.88:0/2772323049 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4ec1500a0 0x7ff4ec1452b0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:48.742 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.738+0000 7ff4f4565640 1 --2- 172.21.15.88:0/2772323049 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4ec0711f0 0x7ff4ec155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:48.742 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.738+0000 7ff4f4565640 1 -- 172.21.15.88:0/2772323049 >> 172.21.15.88:0/2772323049 conn(0x7ff4ec0cb080 msgr2=0x7ff4ec14bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:48.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.738+0000 7ff4f4565640 1 -- 172.21.15.88:0/2772323049 shutdown_connections 2023-10-16T20:06:48.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.738+0000 7ff4f4565640 1 -- 172.21.15.88:0/2772323049 wait complete. 2023-10-16T20:06:48.759 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-10-16T20:06:48.900 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.898+0000 7ff75ac17640 1 Processor -- start 2023-10-16T20:06:48.900 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.898+0000 7ff75ac17640 1 -- start start 2023-10-16T20:06:48.901 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.898+0000 7ff75ac17640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff75406d3f0 0x7ff75406d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:48.902 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.898+0000 7ff75ac17640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff75415ad30 con 0x7ff7540b4260 2023-10-16T20:06:48.902 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.898+0000 7ff75ac17640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff75406dd60 con 0x7ff7540b0da0 2023-10-16T20:06:48.902 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.898+0000 7ff75ac17640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7540b4640 con 0x7ff75406d3f0 2023-10-16T20:06:48.902 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.898+0000 7ff7589b3640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7ff7540b4260 0x7ff75415dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:38052/0 (socket says 172.21.15.88:38052) 2023-10-16T20:06:48.902 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.898+0000 7ff7589b3640 1 -- 172.21.15.88:0/1752899809 learned_addr learned my addr 172.21.15.88:0/1752899809 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:48.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.898+0000 7ff7591b4640 1 --2- 172.21.15.88:0/1752899809 >> v2:172.21.15.94:3300/0 conn(0x7ff75406d3f0 0x7ff75406d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:48.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.898+0000 7ff7537fe640 1 -- 172.21.15.88:0/1752899809 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 251797160 0 0) 0x7ff75415ad30 con 0x7ff7540b4260 2023-10-16T20:06:48.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.898+0000 7ff7537fe640 1 -- 172.21.15.88:0/1752899809 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff744005740 con 0x7ff7540b4260 2023-10-16T20:06:48.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.898+0000 7ff7537fe640 1 -- 172.21.15.88:0/1752899809 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2462583797 0 0) 0x7ff744005740 con 0x7ff7540b4260 2023-10-16T20:06:48.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.898+0000 7ff7537fe640 1 -- 172.21.15.88:0/1752899809 >> v2:172.21.15.94:3300/0 conn(0x7ff75406d3f0 msgr2=0x7ff75406d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:48.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.898+0000 7ff7537fe640 1 --2- 172.21.15.88:0/1752899809 >> v2:172.21.15.94:3300/0 conn(0x7ff75406d3f0 0x7ff75406d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:48.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.898+0000 7ff7537fe640 1 -- 172.21.15.88:0/1752899809 >> v1:172.21.15.94:6789/0 conn(0x7ff7540b0da0 legacy=0x7ff754159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:48.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.898+0000 7ff7537fe640 1 -- 172.21.15.88:0/1752899809 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75415a770 con 0x7ff7540b4260 2023-10-16T20:06:48.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.898+0000 7ff7537fe640 1 -- 172.21.15.88:0/1752899809 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7ff748001680 con 0x7ff7540b4260 2023-10-16T20:06:48.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.898+0000 7ff7537fe640 1 -- 172.21.15.88:0/1752899809 >> v1:172.21.15.88:6789/0 conn(0x7ff7540b4260 legacy=0x7ff75415dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:48.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.898+0000 7ff7537fe640 1 --2- 172.21.15.88:0/1752899809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff744005b90 0x7ff744005f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:48.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.898+0000 7ff7537fe640 1 --2- 172.21.15.88:0/1752899809 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff744006590 0x7ff744006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:48.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.898+0000 7ff7537fe640 1 --2- 172.21.15.88:0/1752899809 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff744007300 0x7ff74400dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:48.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.898+0000 7ff7537fe640 1 -- 172.21.15.88:0/1752899809 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7540b4640 con 0x7ff744007300 2023-10-16T20:06:48.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.898+0000 7ff7537fe640 1 -- 172.21.15.88:0/1752899809 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74400e240 con 0x7ff744005b90 2023-10-16T20:06:48.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.898+0000 7ff7537fe640 1 -- 172.21.15.88:0/1752899809 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff74400e400 con 0x7ff744006590 2023-10-16T20:06:48.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.898+0000 7ff7591b4640 1 --2- 172.21.15.88:0/1752899809 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff744007300 0x7ff74400dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:48.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.898+0000 7ff7589b3640 1 --2- 172.21.15.88:0/1752899809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff744005b90 0x7ff744005f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:48.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.898+0000 7ff753fff640 1 --2- 172.21.15.88:0/1752899809 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff744006590 0x7ff744006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:48.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.898+0000 7ff7591b4640 1 -- 172.21.15.88:0/1752899809 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff744006590 msgr2=0x7ff744006a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:48.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.898+0000 7ff7591b4640 1 --2- 172.21.15.88:0/1752899809 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff744006590 0x7ff744006a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:48.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.898+0000 7ff7591b4640 1 -- 172.21.15.88:0/1752899809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff744005b90 msgr2=0x7ff744005f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:48.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.898+0000 7ff7591b4640 1 --2- 172.21.15.88:0/1752899809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff744005b90 0x7ff744005f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:48.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.898+0000 7ff7591b4640 1 -- 172.21.15.88:0/1752899809 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff744005740 con 0x7ff744007300 2023-10-16T20:06:48.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.902+0000 7ff7591b4640 1 -- 172.21.15.88:0/1752899809 reap_dead start 2023-10-16T20:06:48.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.902+0000 7ff7591b4640 1 --2- 172.21.15.88:0/1752899809 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff744007300 0x7ff74400dba0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff74c009fc0 tx=0x7ff74c01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16e0b5cf9a25103 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:48.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.902+0000 7ff7537fe640 1 -- 172.21.15.88:0/1752899809 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74c0a5cb0 con 0x7ff744007300 2023-10-16T20:06:48.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.902+0000 7ff7537fe640 1 -- 172.21.15.88:0/1752899809 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74c00f040 con 0x7ff744007300 2023-10-16T20:06:48.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.902+0000 7ff7537fe640 1 -- 172.21.15.88:0/1752899809 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74c003be0 con 0x7ff744007300 2023-10-16T20:06:48.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.902+0000 7ff75ac17640 1 -- 172.21.15.88:0/1752899809 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff744007300 msgr2=0x7ff74400dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:48.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.902+0000 7ff75ac17640 1 --2- 172.21.15.88:0/1752899809 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff744007300 0x7ff74400dba0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff74c009fc0 tx=0x7ff74c01cca0 comp rx=0 tx=0).stop 2023-10-16T20:06:48.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.902+0000 7ff75ac17640 1 -- 172.21.15.88:0/1752899809 shutdown_connections 2023-10-16T20:06:48.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.902+0000 7ff75ac17640 1 --2- 172.21.15.88:0/1752899809 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff744007300 0x7ff74400dba0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:48.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.902+0000 7ff75ac17640 1 -- 172.21.15.88:0/1752899809 >> 172.21.15.88:0/1752899809 conn(0x7ff7540cb080 msgr2=0x7ff7540cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:48.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.902+0000 7ff75ac17640 1 -- 172.21.15.88:0/1752899809 shutdown_connections 2023-10-16T20:06:48.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.902+0000 7ff75ac17640 1 -- 172.21.15.88:0/1752899809 wait complete. 2023-10-16T20:06:48.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.906+0000 7ff75ac17640 1 Processor -- start 2023-10-16T20:06:48.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.906+0000 7ff75ac17640 1 -- start start 2023-10-16T20:06:48.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.906+0000 7ff75ac17640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff744007300 0x7ff75414cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:48.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.906+0000 7ff75ac17640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff7540b0730 0x7ff7541516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:48.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.906+0000 7ff75ac17640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff75414c220 0x7ff75414c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:48.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.906+0000 7ff75ac17640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75414e790 con 0x7ff75414c220 2023-10-16T20:06:48.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.906+0000 7ff75ac17640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7541ee820 con 0x7ff744007300 2023-10-16T20:06:48.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.906+0000 7ff75ac17640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff75414cbe0 con 0x7ff7540b0730 2023-10-16T20:06:48.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.906+0000 7ff7591b4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff75414c220 0x7ff75414c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:48.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.906+0000 7ff7591b4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff75414c220 0x7ff75414c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51146/0 (socket says 172.21.15.88:51146) 2023-10-16T20:06:48.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.906+0000 7ff7591b4640 1 -- 172.21.15.88:0/1333538757 learned_addr learned my addr 172.21.15.88:0/1333538757 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:48.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.906+0000 7ff753fff640 1 --2- 172.21.15.88:0/1333538757 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff7540b0730 0x7ff7541516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:48.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.906+0000 7ff7589b3640 1 --2- 172.21.15.88:0/1333538757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff744007300 0x7ff75414cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:48.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.906+0000 7ff7591b4640 1 -- 172.21.15.88:0/1333538757 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff7540b0730 msgr2=0x7ff7541516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:48.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.906+0000 7ff7591b4640 1 --2- 172.21.15.88:0/1333538757 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff7540b0730 0x7ff7541516d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:48.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.906+0000 7ff7591b4640 1 -- 172.21.15.88:0/1333538757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff744007300 msgr2=0x7ff75414cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:48.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.906+0000 7ff7591b4640 1 --2- 172.21.15.88:0/1333538757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff744007300 0x7ff75414cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:48.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.906+0000 7ff7591b4640 1 -- 172.21.15.88:0/1333538757 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7540b0bb0 con 0x7ff75414c220 2023-10-16T20:06:48.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.906+0000 7ff7589b3640 1 --2- 172.21.15.88:0/1333538757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff744007300 0x7ff75414cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:48.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.906+0000 7ff7591b4640 1 --2- 172.21.15.88:0/1333538757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff75414c220 0x7ff75414c6a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff74c009f90 tx=0x7ff74c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:48.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.906+0000 7ff7517fa640 1 -- 172.21.15.88:0/1333538757 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74c004220 con 0x7ff75414c220 2023-10-16T20:06:48.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.906+0000 7ff7517fa640 1 -- 172.21.15.88:0/1333538757 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74c003e90 con 0x7ff75414c220 2023-10-16T20:06:48.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.906+0000 7ff7517fa640 1 -- 172.21.15.88:0/1333538757 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74c0045a0 con 0x7ff75414c220 2023-10-16T20:06:48.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.906+0000 7ff75ac17640 1 -- 172.21.15.88:0/1333538757 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff75414dc10 con 0x7ff75414c220 2023-10-16T20:06:48.919 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.906+0000 7ff75ac17640 1 -- 172.21.15.88:0/1333538757 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75414e1d0 con 0x7ff75414c220 2023-10-16T20:06:48.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.910+0000 7ff75ac17640 1 -- 172.21.15.88:0/1333538757 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff75404e9c0 con 0x7ff75414c220 2023-10-16T20:06:48.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.910+0000 7ff7517fa640 1 -- 172.21.15.88:0/1333538757 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff74c004700 con 0x7ff75414c220 2023-10-16T20:06:48.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.910+0000 7ff7517fa640 1 --2- 172.21.15.88:0/1333538757 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff7380664a0 0x7ff738068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:48.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.910+0000 7ff7517fa640 1 -- 172.21.15.88:0/1333538757 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7ff74c0a6e40 con 0x7ff75414c220 2023-10-16T20:06:48.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.910+0000 7ff7589b3640 1 --2- 172.21.15.88:0/1333538757 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff7380664a0 0x7ff738068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:48.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.910+0000 7ff7589b3640 1 --2- 172.21.15.88:0/1333538757 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff7380664a0 0x7ff738068960 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff7480059a0 tx=0x7ff748009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:48.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:48.914+0000 7ff7517fa640 1 -- 172.21.15.88:0/1333538757 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff74c122030 con 0x7ff75414c220 2023-10-16T20:06:49.065 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.062+0000 7ff75ac17640 1 -- 172.21.15.88:0/1333538757 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7ff754071500 con 0x7ff75414c220 2023-10-16T20:06:49.066 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.062+0000 7ff7517fa640 1 -- 172.21.15.88:0/1333538757 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v46) v1 ==== 97+0+11 (secure 0 0 0) 0x7ff74c0bf0a0 con 0x7ff75414c220 2023-10-16T20:06:49.066 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:06:49.068 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.066+0000 7ff75ac17640 1 -- 172.21.15.88:0/1333538757 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff7380664a0 msgr2=0x7ff738068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:49.068 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.066+0000 7ff75ac17640 1 --2- 172.21.15.88:0/1333538757 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff7380664a0 0x7ff738068960 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff7480059a0 tx=0x7ff748009040 comp rx=0 tx=0).stop 2023-10-16T20:06:49.068 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.066+0000 7ff75ac17640 1 -- 172.21.15.88:0/1333538757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff75414c220 msgr2=0x7ff75414c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:49.069 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.066+0000 7ff75ac17640 1 --2- 172.21.15.88:0/1333538757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff75414c220 0x7ff75414c6a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff74c009f90 tx=0x7ff74c009750 comp rx=0 tx=0).stop 2023-10-16T20:06:49.069 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.066+0000 7ff75ac17640 1 -- 172.21.15.88:0/1333538757 shutdown_connections 2023-10-16T20:06:49.069 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.066+0000 7ff75ac17640 1 --2- 172.21.15.88:0/1333538757 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff7380664a0 0x7ff738068960 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:49.069 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.066+0000 7ff75ac17640 1 --2- 172.21.15.88:0/1333538757 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff7540b0730 0x7ff7541516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:49.069 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.066+0000 7ff75ac17640 1 --2- 172.21.15.88:0/1333538757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff75414c220 0x7ff75414c6a0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:49.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.066+0000 7ff75ac17640 1 --2- 172.21.15.88:0/1333538757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff744007300 0x7ff75414cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:49.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.066+0000 7ff75ac17640 1 -- 172.21.15.88:0/1333538757 >> 172.21.15.88:0/1333538757 conn(0x7ff7540cb080 msgr2=0x7ff7540b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:49.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.066+0000 7ff75ac17640 1 -- 172.21.15.88:0/1333538757 shutdown_connections 2023-10-16T20:06:49.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.066+0000 7ff75ac17640 1 -- 172.21.15.88:0/1333538757 wait complete. 2023-10-16T20:06:49.087 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T20:06:49.228 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.226+0000 7fcd852dc640 1 Processor -- start 2023-10-16T20:06:49.228 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.226+0000 7fcd852dc640 1 -- start start 2023-10-16T20:06:49.230 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.226+0000 7fcd852dc640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fcd800d0e20 0x7fcd800cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:49.230 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.226+0000 7fcd852dc640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd800d0430 0x7fcd800d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:49.230 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.226+0000 7fcd852dc640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fcd800cd370 0x7fcd800cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:49.230 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.226+0000 7fcd852dc640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd800d1740 con 0x7fcd800cd370 2023-10-16T20:06:49.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.226+0000 7fcd852dc640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd800d0bd0 con 0x7fcd800d0e20 2023-10-16T20:06:49.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.226+0000 7fcd852dc640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd800d08f0 con 0x7fcd800d0430 2023-10-16T20:06:49.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.226+0000 7fcd84adb640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fcd800cd370 0x7fcd800cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:49.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.226+0000 7fcd7ffff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fcd800d0e20 0x7fcd800cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:49.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.226+0000 7fcd7f7fe640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd800d0430 0x7fcd800d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:49.232 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.226+0000 7fcd84adb640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fcd800cd370 0x7fcd800cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51154/0 (socket says 172.21.15.88:51154) 2023-10-16T20:06:49.232 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.226+0000 7fcd84adb640 1 -- 172.21.15.88:0/1719819952 learned_addr learned my addr 172.21.15.88:0/1719819952 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:49.232 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.226+0000 7fcd84adb640 1 -- 172.21.15.88:0/1719819952 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd800d0430 msgr2=0x7fcd800d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:49.232 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.226+0000 7fcd84adb640 1 --2- 172.21.15.88:0/1719819952 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd800d0430 0x7fcd800d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:49.232 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.226+0000 7fcd84adb640 1 -- 172.21.15.88:0/1719819952 >> v2:172.21.15.94:3300/0 conn(0x7fcd800d0e20 msgr2=0x7fcd800cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:49.232 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.226+0000 7fcd84adb640 1 --2- 172.21.15.88:0/1719819952 >> v2:172.21.15.94:3300/0 conn(0x7fcd800d0e20 0x7fcd800cfa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:49.233 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.226+0000 7fcd84adb640 1 -- 172.21.15.88:0/1719819952 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8006b160 con 0x7fcd800cd370 2023-10-16T20:06:49.234 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.230+0000 7fcd84adb640 1 --2- 172.21.15.88:0/1719819952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd800cd370 0x7fcd800cd820 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fcd800b5450 tx=0x7fcd7801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883bc2cff5f91769 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:49.234 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.230+0000 7fcd7effd640 1 -- 172.21.15.88:0/1719819952 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd780a5820 con 0x7fcd800cd370 2023-10-16T20:06:49.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.230+0000 7fcd7effd640 1 -- 172.21.15.88:0/1719819952 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd780a5980 con 0x7fcd800cd370 2023-10-16T20:06:49.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.230+0000 7fcd7effd640 1 -- 172.21.15.88:0/1719819952 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd780a5d10 con 0x7fcd800cd370 2023-10-16T20:06:49.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.230+0000 7fcd852dc640 1 -- 172.21.15.88:0/1719819952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd800cd370 msgr2=0x7fcd800cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:49.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.230+0000 7fcd852dc640 1 --2- 172.21.15.88:0/1719819952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd800cd370 0x7fcd800cd820 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fcd800b5450 tx=0x7fcd7801c9a0 comp rx=0 tx=0).stop 2023-10-16T20:06:49.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.230+0000 7fcd852dc640 1 -- 172.21.15.88:0/1719819952 shutdown_connections 2023-10-16T20:06:49.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.230+0000 7fcd852dc640 1 --2- 172.21.15.88:0/1719819952 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd800d0430 0x7fcd800d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:49.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.230+0000 7fcd852dc640 1 --2- 172.21.15.88:0/1719819952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd800cd370 0x7fcd800cd820 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:49.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.230+0000 7fcd852dc640 1 --2- 172.21.15.88:0/1719819952 >> v2:172.21.15.94:3300/0 conn(0x7fcd800d0e20 0x7fcd800cfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:49.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.230+0000 7fcd852dc640 1 -- 172.21.15.88:0/1719819952 >> 172.21.15.88:0/1719819952 conn(0x7fcd8009bbf0 msgr2=0x7fcd800d2860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:49.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.230+0000 7fcd852dc640 1 -- 172.21.15.88:0/1719819952 shutdown_connections 2023-10-16T20:06:49.237 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.230+0000 7fcd852dc640 1 -- 172.21.15.88:0/1719819952 wait complete. 2023-10-16T20:06:49.239 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.234+0000 7fcd852dc640 1 Processor -- start 2023-10-16T20:06:49.239 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.234+0000 7fcd852dc640 1 -- start start 2023-10-16T20:06:49.239 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.234+0000 7fcd852dc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd800cd370 0x7fcd8014c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:49.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.234+0000 7fcd852dc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd800d0430 0x7fcd800b8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:49.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.234+0000 7fcd852dc640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd800d0e20 0x7fcd800b56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:49.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.234+0000 7fcd852dc640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd80159ee0 con 0x7fcd800cd370 2023-10-16T20:06:49.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.234+0000 7fcd852dc640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd800c6f80 con 0x7fcd800d0430 2023-10-16T20:06:49.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.234+0000 7fcd852dc640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd800c6090 con 0x7fcd800d0e20 2023-10-16T20:06:49.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.234+0000 7fcd7ffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd800cd370 0x7fcd8014c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:49.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.234+0000 7fcd7ffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd800cd370 0x7fcd8014c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51158/0 (socket says 172.21.15.88:51158) 2023-10-16T20:06:49.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.234+0000 7fcd7ffff640 1 -- 172.21.15.88:0/890721773 learned_addr learned my addr 172.21.15.88:0/890721773 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:49.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.234+0000 7fcd7f7fe640 1 --2- 172.21.15.88:0/890721773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd800d0430 0x7fcd800b8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:49.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.234+0000 7fcd84adb640 1 --2- 172.21.15.88:0/890721773 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd800d0e20 0x7fcd800b56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:49.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.234+0000 7fcd7ffff640 1 -- 172.21.15.88:0/890721773 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd800d0e20 msgr2=0x7fcd800b56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:49.242 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.234+0000 7fcd7ffff640 1 --2- 172.21.15.88:0/890721773 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd800d0e20 0x7fcd800b56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:49.242 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.234+0000 7fcd7ffff640 1 -- 172.21.15.88:0/890721773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd800d0430 msgr2=0x7fcd800b8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:49.242 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.234+0000 7fcd7ffff640 1 --2- 172.21.15.88:0/890721773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd800d0430 0x7fcd800b8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:49.242 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.234+0000 7fcd7ffff640 1 -- 172.21.15.88:0/890721773 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd78009b30 con 0x7fcd800cd370 2023-10-16T20:06:49.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.234+0000 7fcd7ffff640 1 --2- 172.21.15.88:0/890721773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd800cd370 0x7fcd8014c2d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fcd7000e870 tx=0x7fcd7000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:49.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.234+0000 7fcd7cff9640 1 -- 172.21.15.88:0/890721773 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd700095d0 con 0x7fcd800cd370 2023-10-16T20:06:49.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.234+0000 7fcd7cff9640 1 -- 172.21.15.88:0/890721773 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd7000c8e0 con 0x7fcd800cd370 2023-10-16T20:06:49.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.234+0000 7fcd852dc640 1 -- 172.21.15.88:0/890721773 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd800ce2f0 con 0x7fcd800cd370 2023-10-16T20:06:49.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.234+0000 7fcd7cff9640 1 -- 172.21.15.88:0/890721773 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7000cc50 con 0x7fcd800cd370 2023-10-16T20:06:49.244 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.234+0000 7fcd852dc640 1 -- 172.21.15.88:0/890721773 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd800c6650 con 0x7fcd800cd370 2023-10-16T20:06:49.244 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.234+0000 7fcd852dc640 1 -- 172.21.15.88:0/890721773 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd800674f0 con 0x7fcd800cd370 2023-10-16T20:06:49.245 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.238+0000 7fcd7cff9640 1 -- 172.21.15.88:0/890721773 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcd70013470 con 0x7fcd800cd370 2023-10-16T20:06:49.245 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.238+0000 7fcd7cff9640 1 --2- 172.21.15.88:0/890721773 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcd6c0664a0 0x7fcd6c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:49.245 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.238+0000 7fcd7cff9640 1 -- 172.21.15.88:0/890721773 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7fcd70084680 con 0x7fcd800cd370 2023-10-16T20:06:49.245 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.238+0000 7fcd7f7fe640 1 --2- 172.21.15.88:0/890721773 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcd6c0664a0 0x7fcd6c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:49.245 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.238+0000 7fcd7f7fe640 1 --2- 172.21.15.88:0/890721773 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcd6c0664a0 0x7fcd6c068960 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcd800ce730 tx=0x7fcd74009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:49.246 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.242+0000 7fcd7cff9640 1 -- 172.21.15.88:0/890721773 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcd700518d0 con 0x7fcd800cd370 2023-10-16T20:06:49.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.402+0000 7fcd852dc640 1 -- 172.21.15.88:0/890721773 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcd800b1fa0 con 0x7fcd800cd370 2023-10-16T20:06:49.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.402+0000 7fcd7cff9640 1 -- 172.21.15.88:0/890721773 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v46) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcd70054ef0 con 0x7fcd800cd370 2023-10-16T20:06:49.407 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2023-10-16T20:06:49.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.406+0000 7fcd852dc640 1 -- 172.21.15.88:0/890721773 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcd6c0664a0 msgr2=0x7fcd6c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:49.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.406+0000 7fcd852dc640 1 --2- 172.21.15.88:0/890721773 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcd6c0664a0 0x7fcd6c068960 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcd800ce730 tx=0x7fcd74009190 comp rx=0 tx=0).stop 2023-10-16T20:06:49.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.406+0000 7fcd852dc640 1 -- 172.21.15.88:0/890721773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd800cd370 msgr2=0x7fcd8014c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:49.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.406+0000 7fcd852dc640 1 --2- 172.21.15.88:0/890721773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd800cd370 0x7fcd8014c2d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fcd7000e870 tx=0x7fcd7000ed30 comp rx=0 tx=0).stop 2023-10-16T20:06:49.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.406+0000 7fcd852dc640 1 -- 172.21.15.88:0/890721773 shutdown_connections 2023-10-16T20:06:49.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.406+0000 7fcd852dc640 1 --2- 172.21.15.88:0/890721773 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcd6c0664a0 0x7fcd6c068960 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:49.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.406+0000 7fcd852dc640 1 --2- 172.21.15.88:0/890721773 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcd800d0e20 0x7fcd800b56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:49.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.406+0000 7fcd852dc640 1 --2- 172.21.15.88:0/890721773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd800d0430 0x7fcd800b8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:49.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.406+0000 7fcd852dc640 1 --2- 172.21.15.88:0/890721773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd800cd370 0x7fcd8014c2d0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:49.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.406+0000 7fcd852dc640 1 -- 172.21.15.88:0/890721773 >> 172.21.15.88:0/890721773 conn(0x7fcd8009bbf0 msgr2=0x7fcd800af660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:49.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.406+0000 7fcd852dc640 1 -- 172.21.15.88:0/890721773 shutdown_connections 2023-10-16T20:06:49.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:49.406+0000 7fcd852dc640 1 -- 172.21.15.88:0/890721773 wait complete. 2023-10-16T20:06:49.428 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi088.front.sepia.ceph.com... 2023-10-16T20:06:49.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:49.429 DEBUG:teuthology.orchestra.run.smithi088:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-16T20:06:49.449 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[49978]: fuse finished with error 0 and tester_r 0 2023-10-16T20:06:49.451 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:06:55.553 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2023-10-16T20:06:55.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:55.554 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T20:06:55.565 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds 'allow rwp' mon 'allow r' osd 'allow rw pool=testcephfs1_data,allow rw pool=data_new' 2023-10-16T20:06:55.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.754+0000 7fb034854640 1 Processor -- start 2023-10-16T20:06:55.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.754+0000 7fb034854640 1 -- start start 2023-10-16T20:06:55.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.758+0000 7fb034854640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb03015cac0 0x7fb0300ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:55.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.758+0000 7fb034854640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb0300ae130 0x7fb0300cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:55.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.758+0000 7fb034854640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb03006b840 con 0x7fb03015cea0 2023-10-16T20:06:55.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.758+0000 7fb034854640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0300cac10 con 0x7fb0300ae130 2023-10-16T20:06:55.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.758+0000 7fb034854640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03006a3a0 con 0x7fb03015cac0 2023-10-16T20:06:55.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.758+0000 7fb025d9b640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fb03015cea0 0x7fb0301607d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:57288/0 (socket says 172.21.15.88:57288) 2023-10-16T20:06:55.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.758+0000 7fb025d9b640 1 -- 172.21.15.88:0/3840376213 learned_addr learned my addr 172.21.15.88:0/3840376213 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:55.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.758+0000 7fb02ed9d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb0300ae130 0x7fb0300cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:55.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.758+0000 7fb02e59c640 1 --2- 172.21.15.88:0/3840376213 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb03015cac0 0x7fb0300ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:55.763 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.758+0000 7fb02dd9b640 1 -- 172.21.15.88:0/3840376213 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1612533333 0 0) 0x7fb03006b840 con 0x7fb03015cea0 2023-10-16T20:06:55.763 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.758+0000 7fb02dd9b640 1 -- 172.21.15.88:0/3840376213 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb01c003530 con 0x7fb03015cea0 2023-10-16T20:06:55.763 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.758+0000 7fb02ed9d640 1 -- 172.21.15.88:0/3840376213 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb03015cac0 msgr2=0x7fb0300ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:55.763 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.758+0000 7fb02ed9d640 1 --2- 172.21.15.88:0/3840376213 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb03015cac0 0x7fb0300ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:55.763 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.758+0000 7fb02ed9d640 1 -- 172.21.15.88:0/3840376213 >> v1:172.21.15.88:6789/0 conn(0x7fb03015cea0 legacy=0x7fb0301607d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:55.764 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.758+0000 7fb02ed9d640 1 -- 172.21.15.88:0/3840376213 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0300ca650 con 0x7fb0300ae130 2023-10-16T20:06:55.765 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.762+0000 7fb02ed9d640 1 --2- 172.21.15.88:0/3840376213 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0300ae130 0x7fb0300cd1a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb0280097c0 tx=0x7fb02801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53c602cc8aea1b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:55.765 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.762+0000 7fb02dd9b640 1 -- 172.21.15.88:0/3840376213 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0280a8970 con 0x7fb0300ae130 2023-10-16T20:06:55.765 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.762+0000 7fb02dd9b640 1 -- 172.21.15.88:0/3840376213 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0280a8ad0 con 0x7fb0300ae130 2023-10-16T20:06:55.765 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.762+0000 7fb02dd9b640 1 -- 172.21.15.88:0/3840376213 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0280a8e40 con 0x7fb0300ae130 2023-10-16T20:06:55.766 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.762+0000 7fb034854640 1 -- 172.21.15.88:0/3840376213 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0300ae130 msgr2=0x7fb0300cd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:55.766 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.762+0000 7fb034854640 1 --2- 172.21.15.88:0/3840376213 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0300ae130 0x7fb0300cd1a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb0280097c0 tx=0x7fb02801ec70 comp rx=0 tx=0).stop 2023-10-16T20:06:55.766 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.762+0000 7fb034854640 1 -- 172.21.15.88:0/3840376213 shutdown_connections 2023-10-16T20:06:55.767 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.762+0000 7fb034854640 1 --2- 172.21.15.88:0/3840376213 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0300ae130 0x7fb0300cd1a0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:55.767 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.762+0000 7fb034854640 1 --2- 172.21.15.88:0/3840376213 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb03015cac0 0x7fb0300ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:55.767 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.762+0000 7fb034854640 1 -- 172.21.15.88:0/3840376213 >> 172.21.15.88:0/3840376213 conn(0x7fb0300b0970 msgr2=0x7fb0300b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:55.767 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.762+0000 7fb034854640 1 -- 172.21.15.88:0/3840376213 shutdown_connections 2023-10-16T20:06:55.767 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.762+0000 7fb034854640 1 -- 172.21.15.88:0/3840376213 wait complete. 2023-10-16T20:06:55.769 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.762+0000 7fb034854640 1 Processor -- start 2023-10-16T20:06:55.769 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.762+0000 7fb034854640 1 -- start start 2023-10-16T20:06:55.770 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.766+0000 7fb034854640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0300ae130 0x7fb0300b2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:55.770 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.766+0000 7fb034854640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb03015cac0 0x7fb03015e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:55.770 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.766+0000 7fb034854640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb03015cea0 0x7fb0300b2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:55.770 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.766+0000 7fb034854640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03015ed30 con 0x7fb03015cea0 2023-10-16T20:06:55.771 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.766+0000 7fb034854640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03015ef40 con 0x7fb0300ae130 2023-10-16T20:06:55.771 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.766+0000 7fb034854640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03015f220 con 0x7fb03015cac0 2023-10-16T20:06:55.771 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.766+0000 7fb02ed9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb03015cea0 0x7fb0300b2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:55.771 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.766+0000 7fb02ed9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb03015cea0 0x7fb0300b2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53228/0 (socket says 172.21.15.88:53228) 2023-10-16T20:06:55.771 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.766+0000 7fb02ed9d640 1 -- 172.21.15.88:0/3713351366 learned_addr learned my addr 172.21.15.88:0/3713351366 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:55.771 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.766+0000 7fb025d9b640 1 --2- 172.21.15.88:0/3713351366 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb03015cac0 0x7fb03015e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:55.772 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.766+0000 7fb02e59c640 1 --2- 172.21.15.88:0/3713351366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0300ae130 0x7fb0300b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:55.772 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.766+0000 7fb02ed9d640 1 -- 172.21.15.88:0/3713351366 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb03015cac0 msgr2=0x7fb03015e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:55.772 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.766+0000 7fb02ed9d640 1 --2- 172.21.15.88:0/3713351366 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb03015cac0 0x7fb03015e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:55.772 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.766+0000 7fb02ed9d640 1 -- 172.21.15.88:0/3713351366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0300ae130 msgr2=0x7fb0300b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:55.772 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.766+0000 7fb02ed9d640 1 --2- 172.21.15.88:0/3713351366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0300ae130 0x7fb0300b2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:55.773 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.766+0000 7fb02ed9d640 1 -- 172.21.15.88:0/3713351366 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb028009420 con 0x7fb03015cea0 2023-10-16T20:06:55.773 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.766+0000 7fb02e59c640 1 --2- 172.21.15.88:0/3713351366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0300ae130 0x7fb0300b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:55.773 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.766+0000 7fb025d9b640 1 --2- 172.21.15.88:0/3713351366 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb03015cac0 0x7fb03015e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:55.773 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.766+0000 7fb02ed9d640 1 --2- 172.21.15.88:0/3713351366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb03015cea0 0x7fb0300b2f90 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb02801c8a0 tx=0x7fb02801ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:55.773 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.766+0000 7fb0277fe640 1 -- 172.21.15.88:0/3713351366 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0280b1d30 con 0x7fb03015cea0 2023-10-16T20:06:55.773 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.766+0000 7fb0277fe640 1 -- 172.21.15.88:0/3713351366 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0280b1e90 con 0x7fb03015cea0 2023-10-16T20:06:55.774 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.766+0000 7fb034854640 1 -- 172.21.15.88:0/3713351366 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb03015f500 con 0x7fb03015cea0 2023-10-16T20:06:55.774 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.766+0000 7fb0277fe640 1 -- 172.21.15.88:0/3713351366 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0280b9650 con 0x7fb03015cea0 2023-10-16T20:06:55.774 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.766+0000 7fb034854640 1 -- 172.21.15.88:0/3713351366 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb03015fac0 con 0x7fb03015cea0 2023-10-16T20:06:55.775 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.766+0000 7fb034854640 1 -- 172.21.15.88:0/3713351366 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb03015fcb0 con 0x7fb03015cea0 2023-10-16T20:06:55.775 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.770+0000 7fb0277fe640 1 -- 172.21.15.88:0/3713351366 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb0280b7040 con 0x7fb03015cea0 2023-10-16T20:06:55.775 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.770+0000 7fb0277fe640 1 --2- 172.21.15.88:0/3713351366 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb0100664a0 0x7fb010068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:55.776 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.770+0000 7fb0277fe640 1 -- 172.21.15.88:0/3713351366 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7fb028121cd0 con 0x7fb03015cea0 2023-10-16T20:06:55.776 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.770+0000 7fb02e59c640 1 --2- 172.21.15.88:0/3713351366 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb0100664a0 0x7fb010068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:55.777 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.770+0000 7fb02e59c640 1 --2- 172.21.15.88:0/3713351366 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb0100664a0 0x7fb010068960 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb0200074c0 tx=0x7fb02000b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:55.777 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.774+0000 7fb0277fe640 1 -- 172.21.15.88:0/3713351366 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb0280eee30 con 0x7fb03015cea0 2023-10-16T20:06:55.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.974+0000 7fb034854640 1 -- 172.21.15.88:0/3713351366 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rwp", "mon", "allow r", "osd", "allow rw pool=testcephfs1_data,allow rw pool=data_new"]} v 0) v1 -- 0x7fb03004e9c0 con 0x7fb03015cea0 2023-10-16T20:06:55.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.978+0000 7fb0277fe640 1 -- 172.21.15.88:0/3713351366 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rwp", "mon", "allow r", "osd", "allow rw pool=testcephfs1_data,allow rw pool=data_new"]}]=0 updated caps for client.0 v15) v1 ==== 216+0+173 (secure 0 0 0) 0x7fb0280b5020 con 0x7fb03015cea0 2023-10-16T20:06:55.981 INFO:teuthology.orchestra.run.smithi088.stdout:[client.0] 2023-10-16T20:06:55.981 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQAvly1looeRAhAAeLshHOVIhQqoTtlPehorIg== 2023-10-16T20:06:55.981 INFO:teuthology.orchestra.run.smithi088.stdout: caps mds = "allow rwp" 2023-10-16T20:06:55.981 INFO:teuthology.orchestra.run.smithi088.stdout: caps mon = "allow r" 2023-10-16T20:06:55.981 INFO:teuthology.orchestra.run.smithi088.stdout: caps osd = "allow rw pool=testcephfs1_data,allow rw pool=data_new" 2023-10-16T20:06:55.982 INFO:teuthology.orchestra.run.smithi088.stderr:updated caps for client.0 2023-10-16T20:06:55.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.978+0000 7fb034854640 1 -- 172.21.15.88:0/3713351366 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb0100664a0 msgr2=0x7fb010068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:55.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.978+0000 7fb034854640 1 --2- 172.21.15.88:0/3713351366 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb0100664a0 0x7fb010068960 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb0200074c0 tx=0x7fb02000b040 comp rx=0 tx=0).stop 2023-10-16T20:06:55.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.978+0000 7fb034854640 1 -- 172.21.15.88:0/3713351366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb03015cea0 msgr2=0x7fb0300b2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:55.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.978+0000 7fb034854640 1 --2- 172.21.15.88:0/3713351366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb03015cea0 0x7fb0300b2f90 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb02801c8a0 tx=0x7fb02801ba10 comp rx=0 tx=0).stop 2023-10-16T20:06:55.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.978+0000 7fb034854640 1 -- 172.21.15.88:0/3713351366 shutdown_connections 2023-10-16T20:06:55.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.978+0000 7fb034854640 1 --2- 172.21.15.88:0/3713351366 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb0100664a0 0x7fb010068960 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:55.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.978+0000 7fb034854640 1 --2- 172.21.15.88:0/3713351366 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb03015cac0 0x7fb03015e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:55.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.978+0000 7fb034854640 1 --2- 172.21.15.88:0/3713351366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb03015cea0 0x7fb0300b2f90 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:55.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.978+0000 7fb034854640 1 --2- 172.21.15.88:0/3713351366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0300ae130 0x7fb0300b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:55.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.978+0000 7fb034854640 1 -- 172.21.15.88:0/3713351366 >> 172.21.15.88:0/3713351366 conn(0x7fb0300b0970 msgr2=0x7fb03006b180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:55.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.978+0000 7fb034854640 1 -- 172.21.15.88:0/3713351366 shutdown_connections 2023-10-16T20:06:55.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:55.978+0000 7fb034854640 1 -- 172.21.15.88:0/3713351366 wait complete. 2023-10-16T20:06:56.002 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T20:06:56.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.138+0000 7f4897c9e640 1 Processor -- start 2023-10-16T20:06:56.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.138+0000 7f4897c9e640 1 -- start start 2023-10-16T20:06:56.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.138+0000 7f4897c9e640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f48900d22e0 0x7f4890161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:56.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.138+0000 7f4897c9e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f48900d26c0 0x7f489015d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:56.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.138+0000 7f4897c9e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48900b87e0 con 0x7f489015d820 2023-10-16T20:06:56.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.138+0000 7f4897c9e640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f489015dc70 con 0x7f48900d22e0 2023-10-16T20:06:56.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.138+0000 7f4897c9e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f489015df10 con 0x7f48900d26c0 2023-10-16T20:06:56.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.138+0000 7f4895a3a640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f48900d22e0 0x7f4890161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:56.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.138+0000 7f4895a3a640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f48900d22e0 0x7f4890161450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53236/0 (socket says 172.21.15.88:53236) 2023-10-16T20:06:56.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.138+0000 7f4895a3a640 1 -- 172.21.15.88:0/110720489 learned_addr learned my addr 172.21.15.88:0/110720489 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:56.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.138+0000 7f4895239640 1 --2- 172.21.15.88:0/110720489 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f48900d26c0 0x7f489015d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:56.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.138+0000 7f4895a3a640 1 -- 172.21.15.88:0/110720489 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f48900d26c0 msgr2=0x7f489015d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:56.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.138+0000 7f4895a3a640 1 --2- 172.21.15.88:0/110720489 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f48900d26c0 0x7f489015d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:56.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.138+0000 7f4895a3a640 1 -- 172.21.15.88:0/110720489 >> v1:172.21.15.94:6789/0 conn(0x7f489015d820 legacy=0x7f48900cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:56.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.138+0000 7f4895a3a640 1 -- 172.21.15.88:0/110720489 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489015e1f0 con 0x7f48900d22e0 2023-10-16T20:06:56.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.142+0000 7f4895a3a640 1 --2- 172.21.15.88:0/110720489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48900d22e0 0x7f4890161450 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f488c009870 tx=0x7f488c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a70b6dcba27f97dc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:56.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.142+0000 7f4894a38640 1 -- 172.21.15.88:0/110720489 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c0ad640 con 0x7f48900d22e0 2023-10-16T20:06:56.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.142+0000 7f4894a38640 1 -- 172.21.15.88:0/110720489 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488c0ad7a0 con 0x7f48900d22e0 2023-10-16T20:06:56.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.142+0000 7f4894a38640 1 -- 172.21.15.88:0/110720489 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c0adb30 con 0x7f48900d22e0 2023-10-16T20:06:56.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.142+0000 7f4897c9e640 1 -- 172.21.15.88:0/110720489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48900d22e0 msgr2=0x7f4890161450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:56.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.142+0000 7f4897c9e640 1 --2- 172.21.15.88:0/110720489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48900d22e0 0x7f4890161450 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f488c009870 tx=0x7f488c01c480 comp rx=0 tx=0).stop 2023-10-16T20:06:56.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.142+0000 7f4897c9e640 1 -- 172.21.15.88:0/110720489 shutdown_connections 2023-10-16T20:06:56.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.142+0000 7f4897c9e640 1 --2- 172.21.15.88:0/110720489 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f48900d26c0 0x7f489015d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:56.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.142+0000 7f4897c9e640 1 --2- 172.21.15.88:0/110720489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48900d22e0 0x7f4890161450 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:56.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.142+0000 7f4897c9e640 1 -- 172.21.15.88:0/110720489 >> 172.21.15.88:0/110720489 conn(0x7f48900715c0 msgr2=0x7f48900719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:56.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.142+0000 7f4897c9e640 1 -- 172.21.15.88:0/110720489 shutdown_connections 2023-10-16T20:06:56.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.142+0000 7f4897c9e640 1 -- 172.21.15.88:0/110720489 wait complete. 2023-10-16T20:06:56.150 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.146+0000 7f4897c9e640 1 Processor -- start 2023-10-16T20:06:56.150 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.146+0000 7f4897c9e640 1 -- start start 2023-10-16T20:06:56.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.146+0000 7f4897c9e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48900d22e0 0x7f48900c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:56.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.146+0000 7f4897c9e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48900d26c0 0x7f4890156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:56.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.146+0000 7f4897c9e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f489015d820 0x7f4890152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:56.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.146+0000 7f4897c9e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48901536b0 con 0x7f48900d22e0 2023-10-16T20:06:56.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.146+0000 7f4897c9e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4890156100 con 0x7f48900d26c0 2023-10-16T20:06:56.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.146+0000 7f4897c9e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48901533d0 con 0x7f489015d820 2023-10-16T20:06:56.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.146+0000 7f4895a3a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48900d22e0 0x7f48900c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:56.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.146+0000 7f4895a3a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48900d22e0 0x7f48900c58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53250/0 (socket says 172.21.15.88:53250) 2023-10-16T20:06:56.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.146+0000 7f4895a3a640 1 -- 172.21.15.88:0/2594983619 learned_addr learned my addr 172.21.15.88:0/2594983619 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:56.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.146+0000 7f4895239640 1 --2- 172.21.15.88:0/2594983619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48900d26c0 0x7f4890156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:56.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.146+0000 7f489623b640 1 --2- 172.21.15.88:0/2594983619 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f489015d820 0x7f4890152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:56.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.146+0000 7f4895a3a640 1 -- 172.21.15.88:0/2594983619 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f489015d820 msgr2=0x7f4890152800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:56.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.146+0000 7f4895a3a640 1 --2- 172.21.15.88:0/2594983619 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f489015d820 0x7f4890152800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:56.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.146+0000 7f4895a3a640 1 -- 172.21.15.88:0/2594983619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48900d26c0 msgr2=0x7f4890156590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:56.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.146+0000 7f4895a3a640 1 --2- 172.21.15.88:0/2594983619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48900d26c0 0x7f4890156590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:56.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.146+0000 7f4895a3a640 1 -- 172.21.15.88:0/2594983619 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f488c0094d0 con 0x7f48900d22e0 2023-10-16T20:06:56.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.146+0000 7f489623b640 1 --2- 172.21.15.88:0/2594983619 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f489015d820 0x7f4890152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:06:56.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.146+0000 7f4895239640 1 --2- 172.21.15.88:0/2594983619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48900d26c0 0x7f4890156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:06:56.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.146+0000 7f4895a3a640 1 --2- 172.21.15.88:0/2594983619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48900d22e0 0x7f48900c58d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f488c0b0040 tx=0x7f488c019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:56.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.146+0000 7f48867fc640 1 -- 172.21.15.88:0/2594983619 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c0b8670 con 0x7f48900d22e0 2023-10-16T20:06:56.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.146+0000 7f48867fc640 1 -- 172.21.15.88:0/2594983619 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488c0a6040 con 0x7f48900d22e0 2023-10-16T20:06:56.155 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.146+0000 7f4897c9e640 1 -- 172.21.15.88:0/2594983619 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4890155ab0 con 0x7f48900d22e0 2023-10-16T20:06:56.155 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.146+0000 7f48867fc640 1 -- 172.21.15.88:0/2594983619 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c0b8a10 con 0x7f48900d22e0 2023-10-16T20:06:56.155 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.146+0000 7f4897c9e640 1 -- 172.21.15.88:0/2594983619 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48900c5e10 con 0x7f48900d22e0 2023-10-16T20:06:56.156 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.146+0000 7f4897c9e640 1 -- 172.21.15.88:0/2594983619 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4890155910 con 0x7f48900d22e0 2023-10-16T20:06:56.156 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.150+0000 7f48867fc640 1 -- 172.21.15.88:0/2594983619 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f488c0b4070 con 0x7f48900d22e0 2023-10-16T20:06:56.157 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.150+0000 7f48867fc640 1 --2- 172.21.15.88:0/2594983619 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f48740664a0 0x7f4874068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:56.157 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.150+0000 7f48867fc640 1 -- 172.21.15.88:0/2594983619 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f488c11eb70 con 0x7f48900d22e0 2023-10-16T20:06:56.157 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.150+0000 7f4895239640 1 --2- 172.21.15.88:0/2594983619 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f48740664a0 0x7f4874068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:56.157 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.150+0000 7f4895239640 1 --2- 172.21.15.88:0/2594983619 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f48740664a0 0x7f4874068960 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f48901f8ec0 tx=0x7f4880008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:56.158 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.154+0000 7f48867fc640 1 -- 172.21.15.88:0/2594983619 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4890155910 con 0x7f48900d22e0 2023-10-16T20:06:56.344 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.342+0000 7f4897c9e640 1 -- 172.21.15.88:0/2594983619 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4890155910 con 0x7f48900d22e0 2023-10-16T20:06:56.344 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.342+0000 7f48867fc640 1 -- 172.21.15.88:0/2594983619 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v19) v1 ==== 53+0+97 (secure 0 0 0) 0x7f4890155910 con 0x7f48900d22e0 2023-10-16T20:06:56.345 INFO:teuthology.orchestra.run.smithi088.stdout:name: testcephfs1, metadata pool: testcephfs1_metadata, data pools: [testcephfs1_data data_new ] 2023-10-16T20:06:56.346 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.342+0000 7f4897c9e640 1 -- 172.21.15.88:0/2594983619 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f48740664a0 msgr2=0x7f4874068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:56.346 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.342+0000 7f4897c9e640 1 --2- 172.21.15.88:0/2594983619 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f48740664a0 0x7f4874068960 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f48901f8ec0 tx=0x7f4880008040 comp rx=0 tx=0).stop 2023-10-16T20:06:56.347 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.342+0000 7f4897c9e640 1 -- 172.21.15.88:0/2594983619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48900d22e0 msgr2=0x7f48900c58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:56.347 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.342+0000 7f4897c9e640 1 --2- 172.21.15.88:0/2594983619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48900d22e0 0x7f48900c58d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f488c0b0040 tx=0x7f488c019220 comp rx=0 tx=0).stop 2023-10-16T20:06:56.347 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.342+0000 7f4897c9e640 1 -- 172.21.15.88:0/2594983619 shutdown_connections 2023-10-16T20:06:56.347 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.342+0000 7f4897c9e640 1 --2- 172.21.15.88:0/2594983619 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f48740664a0 0x7f4874068960 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:56.347 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.342+0000 7f4897c9e640 1 --2- 172.21.15.88:0/2594983619 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f489015d820 0x7f4890152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:56.348 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.342+0000 7f4897c9e640 1 --2- 172.21.15.88:0/2594983619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48900d26c0 0x7f4890156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:56.348 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.342+0000 7f4897c9e640 1 --2- 172.21.15.88:0/2594983619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48900d22e0 0x7f48900c58d0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:56.348 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.342+0000 7f4897c9e640 1 -- 172.21.15.88:0/2594983619 >> 172.21.15.88:0/2594983619 conn(0x7f48900715c0 msgr2=0x7f48900719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:56.348 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.342+0000 7f4897c9e640 1 -- 172.21.15.88:0/2594983619 shutdown_connections 2023-10-16T20:06:56.348 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:56.342+0000 7f4897c9e640 1 -- 172.21.15.88:0/2594983619 wait complete. 2023-10-16T20:06:56.365 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T20:06:56.365 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T20:06:56.366 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi088.front.sepia.ceph.com 2023-10-16T20:06:56.366 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T20:06:56.366 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T20:06:56.367 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T20:06:56.367 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T20:06:56.367 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T20:06:56.368 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T20:06:56.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:56.369 DEBUG:teuthology.orchestra.run.smithi088:> ip addr 2023-10-16T20:06:56.379 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T20:06:56.379 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T20:06:56.379 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T20:06:56.379 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:06:56.379 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2023-10-16T20:06:56.380 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:06:56.380 INFO:teuthology.orchestra.run.smithi088.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:06:56.380 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f0 brd ff:ff:ff:ff:ff:ff 2023-10-16T20:06:56.380 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f0 2023-10-16T20:06:56.381 INFO:teuthology.orchestra.run.smithi088.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:06:56.381 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f1 brd ff:ff:ff:ff:ff:ff 2023-10-16T20:06:56.381 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f1 2023-10-16T20:06:56.381 INFO:teuthology.orchestra.run.smithi088.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:06:56.381 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2c brd ff:ff:ff:ff:ff:ff 2023-10-16T20:06:56.382 INFO:teuthology.orchestra.run.smithi088.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T20:06:56.382 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2d brd ff:ff:ff:ff:ff:ff 2023-10-16T20:06:56.382 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.21.15.88/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-16T20:06:56.382 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft 42315sec preferred_lft 42315sec 2023-10-16T20:06:56.382 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link 2023-10-16T20:06:56.383 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:06:56.383 INFO:teuthology.orchestra.run.smithi088.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-16T20:06:56.383 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 02:42:d9:3b:fb:39 brd ff:ff:ff:ff:ff:ff 2023-10-16T20:06:56.383 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-16T20:06:56.383 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:06:56.383 INFO:teuthology.orchestra.run.smithi088.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T20:06:56.384 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-16T20:06:56.384 INFO:teuthology.orchestra.run.smithi088.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T20:06:56.384 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:06:56.384 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-16T20:06:56.384 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:06:56.385 INFO:teuthology.orchestra.run.smithi088.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T20:06:56.385 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:06:56.385 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-16T20:06:56.385 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:06:56.385 INFO:teuthology.orchestra.run.smithi088.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T20:06:56.386 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:06:56.386 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-16T20:06:56.386 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:06:56.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:56.387 DEBUG:teuthology.orchestra.run.smithi088:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T20:06:56.442 INFO:teuthology.orchestra.run.smithi088.stdout:1 2023-10-16T20:06:56.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:56.444 DEBUG:teuthology.orchestra.run.smithi088:> route 2023-10-16T20:06:56.503 INFO:teuthology.orchestra.run.smithi088.stdout:Kernel IP routing table 2023-10-16T20:06:56.503 INFO:teuthology.orchestra.run.smithi088.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T20:06:56.504 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-16T20:06:56.504 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T20:06:56.504 INFO:teuthology.orchestra.run.smithi088.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-16T20:06:56.504 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T20:06:56.505 INFO:teuthology.orchestra.run.smithi088.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:06:56.505 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:06:56.505 INFO:teuthology.orchestra.run.smithi088.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:06:56.505 INFO:teuthology.orchestra.run.smithi088.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T20:06:56.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:56.506 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec bash -c ' 2023-10-16T20:06:56.507 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2023-10-16T20:06:56.508 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T20:06:56.508 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T20:06:56.509 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T20:06:56.510 DEBUG:teuthology.orchestra.run.smithi088:> ') 2023-10-16T20:06:56.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:56.599 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2023-10-16T20:06:56.599 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T20:06:56.605 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T20:06:56.609 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T20:06:56.610 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T20:06:56.610 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T20:06:56.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:56.611 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set brx.0 up 2023-10-16T20:06:56.710 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-16T20:06:56.711 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T20:06:56.712 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T20:06:56.760 INFO:teuthology.orchestra.run.smithi088.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T20:06:56.760 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T20:06:56.761 DEBUG:teuthology.orchestra.run.smithi088:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T20:06:56.811 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2023-10-16T20:06:56.880 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T20:06:56.933 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2023-10-16T20:06:56.934 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2023-10-16T20:06:56.934 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2023-10-16T20:06:56.934 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2023-10-16T20:06:56.934 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2023-10-16T20:06:56.934 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2023-10-16T20:06:56.935 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2023-10-16T20:06:56.935 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2023-10-16T20:06:56.935 INFO:teuthology.orchestra.run.smithi088.stdout:/run/lock 2023-10-16T20:06:56.935 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2023-10-16T20:06:56.935 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2023-10-16T20:06:56.936 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2023-10-16T20:06:56.936 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T20:06:56.936 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2023-10-16T20:06:56.936 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2023-10-16T20:06:56.936 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2023-10-16T20:06:56.936 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2023-10-16T20:06:56.937 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/fs/nfsd 2023-10-16T20:06:56.937 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2023-10-16T20:06:56.937 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2023-10-16T20:06:56.937 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T20:06:56.937 INFO:teuthology.orchestra.run.smithi088.stdout:/run/rpc_pipefs 2023-10-16T20:06:56.938 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2023-10-16T20:06:56.938 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2023-10-16T20:06:56.938 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T20:06:56.938 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T20:06:56.938 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T20:06:56.939 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T20:06:56.939 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2023-10-16T20:06:56.939 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:06:56.939 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:06:56.939 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:06:56.939 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:06:56.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:56.941 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2023-10-16T20:06:56.991 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-16T20:06:56.991 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-16T20:06:57.042 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2023-10-16T20:06:57.064 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T20:06:57.118 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2023-10-16T20:06:57.119 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2023-10-16T20:06:57.119 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2023-10-16T20:06:57.120 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2023-10-16T20:06:57.120 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2023-10-16T20:06:57.120 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2023-10-16T20:06:57.121 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2023-10-16T20:06:57.121 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2023-10-16T20:06:57.121 INFO:teuthology.orchestra.run.smithi088.stdout:/run/lock 2023-10-16T20:06:57.122 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2023-10-16T20:06:57.122 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2023-10-16T20:06:57.123 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2023-10-16T20:06:57.123 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T20:06:57.123 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2023-10-16T20:06:57.124 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2023-10-16T20:06:57.124 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2023-10-16T20:06:57.124 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2023-10-16T20:06:57.125 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/fs/nfsd 2023-10-16T20:06:57.125 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2023-10-16T20:06:57.125 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2023-10-16T20:06:57.126 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T20:06:57.126 INFO:teuthology.orchestra.run.smithi088.stdout:/run/rpc_pipefs 2023-10-16T20:06:57.126 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2023-10-16T20:06:57.127 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2023-10-16T20:06:57.127 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T20:06:57.127 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T20:06:57.128 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T20:06:57.128 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T20:06:57.128 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2023-10-16T20:06:57.128 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:06:57.128 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:06:57.129 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:06:57.129 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:06:57.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:57.131 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2023-10-16T20:06:57.139 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:2023-10-16T20:06:57.134+0000 7f9218045580 -1 init, newargv = 0x55b341d8be90 newargc=15 2023-10-16T20:06:57.140 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[50434]: starting ceph client 2023-10-16T20:06:57.147 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[50434]: starting fuse 2023-10-16T20:06:57.149 INFO:teuthology.orchestra.run.smithi088.stdout:43 2023-10-16T20:06:57.149 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-10-16T20:06:57.150 DEBUG:teuthology.orchestra.run.smithi088:> sudo stdin-killer -- python3 -c ' 2023-10-16T20:06:57.150 DEBUG:teuthology.orchestra.run.smithi088:> import glob 2023-10-16T20:06:57.151 DEBUG:teuthology.orchestra.run.smithi088:> import re 2023-10-16T20:06:57.151 DEBUG:teuthology.orchestra.run.smithi088:> import os 2023-10-16T20:06:57.151 DEBUG:teuthology.orchestra.run.smithi088:> import subprocess 2023-10-16T20:06:57.152 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:06:57.152 DEBUG:teuthology.orchestra.run.smithi088:> def _find_admin_socket(client_name): 2023-10-16T20:06:57.153 DEBUG:teuthology.orchestra.run.smithi088:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-16T20:06:57.153 DEBUG:teuthology.orchestra.run.smithi088:> files = glob.glob(asok_path) 2023-10-16T20:06:57.153 DEBUG:teuthology.orchestra.run.smithi088:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-16T20:06:57.154 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:06:57.154 DEBUG:teuthology.orchestra.run.smithi088:> # Given a non-glob path, it better be there 2023-10-16T20:06:57.154 DEBUG:teuthology.orchestra.run.smithi088:> if "*" not in asok_path: 2023-10-16T20:06:57.155 DEBUG:teuthology.orchestra.run.smithi088:> assert(len(files) == 1) 2023-10-16T20:06:57.155 DEBUG:teuthology.orchestra.run.smithi088:> return files[0] 2023-10-16T20:06:57.155 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:06:57.156 DEBUG:teuthology.orchestra.run.smithi088:> for f in files: 2023-10-16T20:06:57.156 DEBUG:teuthology.orchestra.run.smithi088:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-16T20:06:57.157 DEBUG:teuthology.orchestra.run.smithi088:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-16T20:06:57.157 DEBUG:teuthology.orchestra.run.smithi088:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-16T20:06:57.157 DEBUG:teuthology.orchestra.run.smithi088:> contents = proc_f.read() 2023-10-16T20:06:57.158 DEBUG:teuthology.orchestra.run.smithi088:> if mountpoint in contents: 2023-10-16T20:06:57.158 DEBUG:teuthology.orchestra.run.smithi088:> return f 2023-10-16T20:06:57.158 DEBUG:teuthology.orchestra.run.smithi088:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-16T20:06:57.159 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:06:57.159 DEBUG:teuthology.orchestra.run.smithi088:> print(_find_admin_socket("client.0")) 2023-10-16T20:06:57.160 DEBUG:teuthology.orchestra.run.smithi088:> ' 2023-10-16T20:06:57.275 INFO:teuthology.orchestra.run.smithi088.stdout:/var/run/ceph/ceph-client.0.50434.asok 2023-10-16T20:06:57.279 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-16T20:06:57.287 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.50434.asok 2023-10-16T20:06:57.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:57.288 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.50434.asok status 2023-10-16T20:06:57.462 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2023-10-16T20:06:57.463 INFO:teuthology.orchestra.run.smithi088.stdout: "metadata": { 2023-10-16T20:06:57.463 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_sha1": "4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0", 2023-10-16T20:06:57.463 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_version": "ceph version 18.0.0-6730-g4ba8a190 (4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0) reef (dev)", 2023-10-16T20:06:57.464 INFO:teuthology.orchestra.run.smithi088.stdout: "entity_id": "0", 2023-10-16T20:06:57.464 INFO:teuthology.orchestra.run.smithi088.stdout: "hostname": "smithi088", 2023-10-16T20:06:57.464 INFO:teuthology.orchestra.run.smithi088.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-16T20:06:57.464 INFO:teuthology.orchestra.run.smithi088.stdout: "pid": "50434", 2023-10-16T20:06:57.464 INFO:teuthology.orchestra.run.smithi088.stdout: "root": "/" 2023-10-16T20:06:57.465 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:57.465 INFO:teuthology.orchestra.run.smithi088.stdout: "dentry_count": 0, 2023-10-16T20:06:57.465 INFO:teuthology.orchestra.run.smithi088.stdout: "dentry_pinned_count": 0, 2023-10-16T20:06:57.465 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 5964, 2023-10-16T20:06:57.465 INFO:teuthology.orchestra.run.smithi088.stdout: "inst": { 2023-10-16T20:06:57.465 INFO:teuthology.orchestra.run.smithi088.stdout: "name": { 2023-10-16T20:06:57.466 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "client", 2023-10-16T20:06:57.466 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 5964 2023-10-16T20:06:57.466 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:57.466 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": { 2023-10-16T20:06:57.466 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2023-10-16T20:06:57.467 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "192.168.0.1:0", 2023-10-16T20:06:57.467 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 2014671712 2023-10-16T20:06:57.467 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:06:57.467 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:57.467 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": { 2023-10-16T20:06:57.467 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2023-10-16T20:06:57.468 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "192.168.0.1:0", 2023-10-16T20:06:57.468 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 2014671712 2023-10-16T20:06:57.468 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:06:57.468 INFO:teuthology.orchestra.run.smithi088.stdout: "inst_str": "client.5964 192.168.0.1:0/2014671712", 2023-10-16T20:06:57.468 INFO:teuthology.orchestra.run.smithi088.stdout: "addr_str": "192.168.0.1:0/2014671712", 2023-10-16T20:06:57.469 INFO:teuthology.orchestra.run.smithi088.stdout: "inode_count": 1, 2023-10-16T20:06:57.469 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_epoch": 19, 2023-10-16T20:06:57.469 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_epoch": 46, 2023-10-16T20:06:57.469 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_epoch_barrier": 0, 2023-10-16T20:06:57.469 INFO:teuthology.orchestra.run.smithi088.stdout: "blocklisted": false, 2023-10-16T20:06:57.470 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "testcephfs1" 2023-10-16T20:06:57.470 INFO:teuthology.orchestra.run.smithi088.stdout:} 2023-10-16T20:06:57.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:57.478 DEBUG:teuthology.orchestra.run.smithi088:> stat --file-system '--printf=%T 2023-10-16T20:06:57.479 DEBUG:teuthology.orchestra.run.smithi088:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-16T20:06:57.488 INFO:teuthology.orchestra.run.smithi088.stdout:fuseblk 2023-10-16T20:06:57.489 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-16T20:06:57.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:57.490 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T20:06:57.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:57.551 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.pool -v data_new layoutfile) 2023-10-16T20:06:57.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:06:57.604 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v data_new layoutdir) 2023-10-16T20:06:57.667 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:06:57.667 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:06:57.716 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:06:57.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.890+0000 7f79a88ea640 1 Processor -- start 2023-10-16T20:06:57.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.890+0000 7f79a88ea640 1 -- start start 2023-10-16T20:06:57.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.894+0000 7f79a88ea640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f79a41599f0 0x7f79a40b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:57.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.894+0000 7f79a88ea640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f79a4069e90 0x7f79a406a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:57.897 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.894+0000 7f79a88ea640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79a406a830 con 0x7f79a40b05a0 2023-10-16T20:06:57.897 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.894+0000 7f79a88ea640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79a40b1260 con 0x7f79a41599f0 2023-10-16T20:06:57.897 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.894+0000 7f79a88ea640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a40ae960 con 0x7f79a4069e90 2023-10-16T20:06:57.898 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.894+0000 7f7999d9b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f79a4069e90 0x7f79a406a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:57.898 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.894+0000 7f79a2d9d640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f79a40b05a0 0x7f79a40b0a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:60840/0 (socket says 172.21.15.88:60840) 2023-10-16T20:06:57.898 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.894+0000 7f79a2d9d640 1 -- 172.21.15.88:0/224726496 learned_addr learned my addr 172.21.15.88:0/224726496 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:57.899 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.894+0000 7f79a259c640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f79a41599f0 0x7f79a40b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:57.899 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.894+0000 7f79a1d9b640 1 -- 172.21.15.88:0/224726496 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3274760602 0 0) 0x7f79a406a830 con 0x7f79a40b05a0 2023-10-16T20:06:57.899 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.894+0000 7f7999d9b640 1 -- 172.21.15.88:0/224726496 >> v2:172.21.15.94:3300/0 conn(0x7f79a41599f0 msgr2=0x7f79a40b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:57.899 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.894+0000 7f7999d9b640 1 --2- 172.21.15.88:0/224726496 >> v2:172.21.15.94:3300/0 conn(0x7f79a41599f0 0x7f79a40b2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:57.899 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.894+0000 7f7999d9b640 1 -- 172.21.15.88:0/224726496 >> v1:172.21.15.94:6789/0 conn(0x7f79a40b05a0 legacy=0x7f79a40b0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:57.900 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.894+0000 7f7999d9b640 1 -- 172.21.15.88:0/224726496 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a40aec40 con 0x7f79a4069e90 2023-10-16T20:06:57.900 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.894+0000 7f79a259c640 1 --2- 172.21.15.88:0/224726496 >> v2:172.21.15.94:3300/0 conn(0x7f79a41599f0 0x7f79a40b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:06:57.901 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.898+0000 7f7999d9b640 1 --2- 172.21.15.88:0/224726496 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f79a4069e90 0x7f79a406a2f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f798c0097c0 tx=0x7f798c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c2a8930c881d4d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:57.901 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.898+0000 7f79a1d9b640 1 -- 172.21.15.88:0/224726496 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c0ab650 con 0x7f79a4069e90 2023-10-16T20:06:57.901 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.898+0000 7f79a1d9b640 1 -- 172.21.15.88:0/224726496 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798c0ab7b0 con 0x7f79a4069e90 2023-10-16T20:06:57.901 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.898+0000 7f79a1d9b640 1 -- 172.21.15.88:0/224726496 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c0abb40 con 0x7f79a4069e90 2023-10-16T20:06:57.901 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.898+0000 7f79a88ea640 1 -- 172.21.15.88:0/224726496 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f79a4069e90 msgr2=0x7f79a406a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:57.902 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.898+0000 7f79a88ea640 1 --2- 172.21.15.88:0/224726496 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f79a4069e90 0x7f79a406a2f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f798c0097c0 tx=0x7f798c01c450 comp rx=0 tx=0).stop 2023-10-16T20:06:57.902 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.898+0000 7f79a88ea640 1 -- 172.21.15.88:0/224726496 shutdown_connections 2023-10-16T20:06:57.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.898+0000 7f79a88ea640 1 --2- 172.21.15.88:0/224726496 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f79a4069e90 0x7f79a406a2f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:57.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.898+0000 7f79a88ea640 1 --2- 172.21.15.88:0/224726496 >> v2:172.21.15.94:3300/0 conn(0x7f79a41599f0 0x7f79a40b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:57.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.898+0000 7f79a88ea640 1 -- 172.21.15.88:0/224726496 >> 172.21.15.88:0/224726496 conn(0x7f79a40ce5e0 msgr2=0x7f79a40ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:57.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.898+0000 7f79a88ea640 1 -- 172.21.15.88:0/224726496 shutdown_connections 2023-10-16T20:06:57.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.898+0000 7f79a88ea640 1 -- 172.21.15.88:0/224726496 wait complete. 2023-10-16T20:06:57.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.898+0000 7f79a88ea640 1 Processor -- start 2023-10-16T20:06:57.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.898+0000 7f79a88ea640 1 -- start start 2023-10-16T20:06:57.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.902+0000 7f79a88ea640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f79a4069e90 0x7f79a4146c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:57.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.902+0000 7f79a88ea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79a40b05a0 0x7f79a4155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:57.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.902+0000 7f79a88ea640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79a41599f0 0x7f79a40ac210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:57.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.902+0000 7f79a88ea640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a41592b0 con 0x7f79a41599f0 2023-10-16T20:06:57.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.902+0000 7f79a88ea640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a414ead0 con 0x7f79a40b05a0 2023-10-16T20:06:57.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.902+0000 7f79a88ea640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a414ec60 con 0x7f79a4069e90 2023-10-16T20:06:57.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.902+0000 7f79a2d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79a41599f0 0x7f79a40ac210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:57.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.902+0000 7f79a2d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79a41599f0 0x7f79a40ac210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53262/0 (socket says 172.21.15.88:53262) 2023-10-16T20:06:57.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.902+0000 7f79a2d9d640 1 -- 172.21.15.88:0/538231289 learned_addr learned my addr 172.21.15.88:0/538231289 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:57.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.902+0000 7f79a259c640 1 --2- 172.21.15.88:0/538231289 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f79a4069e90 0x7f79a4146c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:57.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.902+0000 7f7999d9b640 1 --2- 172.21.15.88:0/538231289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79a40b05a0 0x7f79a4155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:57.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.902+0000 7f79a2d9d640 1 -- 172.21.15.88:0/538231289 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f79a4069e90 msgr2=0x7f79a4146c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:57.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.902+0000 7f79a2d9d640 1 --2- 172.21.15.88:0/538231289 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f79a4069e90 0x7f79a4146c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:57.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.902+0000 7f79a2d9d640 1 -- 172.21.15.88:0/538231289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79a40b05a0 msgr2=0x7f79a4155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:57.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.902+0000 7f79a2d9d640 1 --2- 172.21.15.88:0/538231289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79a40b05a0 0x7f79a4155230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:57.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.902+0000 7f79a2d9d640 1 -- 172.21.15.88:0/538231289 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7994009420 con 0x7f79a41599f0 2023-10-16T20:06:57.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.902+0000 7f79a2d9d640 1 --2- 172.21.15.88:0/538231289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79a41599f0 0x7f79a40ac210 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f799c001570 tx=0x7f799c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:57.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.902+0000 7f799b7fe640 1 -- 172.21.15.88:0/538231289 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c00a640 con 0x7f79a41599f0 2023-10-16T20:06:57.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.902+0000 7f799b7fe640 1 -- 172.21.15.88:0/538231289 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f799c00a7c0 con 0x7f79a41599f0 2023-10-16T20:06:57.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.902+0000 7f79a88ea640 1 -- 172.21.15.88:0/538231289 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f798c009420 con 0x7f79a41599f0 2023-10-16T20:06:57.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.902+0000 7f799b7fe640 1 -- 172.21.15.88:0/538231289 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c00ab70 con 0x7f79a41599f0 2023-10-16T20:06:57.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.902+0000 7f79a88ea640 1 -- 172.21.15.88:0/538231289 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a4157840 con 0x7f79a41599f0 2023-10-16T20:06:57.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.902+0000 7f79a88ea640 1 -- 172.21.15.88:0/538231289 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79a414edd0 con 0x7f79a41599f0 2023-10-16T20:06:57.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.906+0000 7f799b7fe640 1 -- 172.21.15.88:0/538231289 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f799c00acd0 con 0x7f79a41599f0 2023-10-16T20:06:57.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.906+0000 7f799b7fe640 1 --2- 172.21.15.88:0/538231289 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f79840664a0 0x7f7984068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:57.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.906+0000 7f799b7fe640 1 -- 172.21.15.88:0/538231289 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f799c083720 con 0x7f79a41599f0 2023-10-16T20:06:57.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.906+0000 7f79a259c640 1 --2- 172.21.15.88:0/538231289 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f79840664a0 0x7f7984068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:57.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.906+0000 7f79a259c640 1 --2- 172.21.15.88:0/538231289 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f79840664a0 0x7f7984068960 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7994009c50 tx=0x7f7994009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:57.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:57.910+0000 7f799b7fe640 1 -- 172.21.15.88:0/538231289 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f79a414edd0 con 0x7f79a41599f0 2023-10-16T20:06:58.062 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.058+0000 7f79a88ea640 1 -- 172.21.15.88:0/538231289 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79a414edd0 con 0x7f79a41599f0 2023-10-16T20:06:58.064 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.062+0000 7f799b7fe640 1 -- 172.21.15.88:0/538231289 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+18499 (secure 0 0 0) 0x7f79a414edd0 con 0x7f79a41599f0 2023-10-16T20:06:58.064 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:06:58.064 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":46,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:06:38.250351+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T20:06:14.743500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"testcephfs1_data","create_time":"2023-10-16T20:06:16.091837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-10-16T20:06:31.184462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.39","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:06:58.066 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.062+0000 7f79a88ea640 1 -- 172.21.15.88:0/538231289 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f79840664a0 msgr2=0x7f7984068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:58.066 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.062+0000 7f79a88ea640 1 --2- 172.21.15.88:0/538231289 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f79840664a0 0x7f7984068960 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7994009c50 tx=0x7f7994009190 comp rx=0 tx=0).stop 2023-10-16T20:06:58.066 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.062+0000 7f79a88ea640 1 -- 172.21.15.88:0/538231289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79a41599f0 msgr2=0x7f79a40ac210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:58.066 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.062+0000 7f79a88ea640 1 --2- 172.21.15.88:0/538231289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79a41599f0 0x7f79a40ac210 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f799c001570 tx=0x7f799c00cca0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.066 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.062+0000 7f79a88ea640 1 -- 172.21.15.88:0/538231289 shutdown_connections 2023-10-16T20:06:58.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.062+0000 7f79a88ea640 1 --2- 172.21.15.88:0/538231289 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f79840664a0 0x7f7984068960 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.062+0000 7f79a88ea640 1 --2- 172.21.15.88:0/538231289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79a41599f0 0x7f79a40ac210 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.062+0000 7f79a88ea640 1 --2- 172.21.15.88:0/538231289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79a40b05a0 0x7f79a4155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.062+0000 7f79a88ea640 1 --2- 172.21.15.88:0/538231289 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f79a4069e90 0x7f79a4146c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.062+0000 7f79a88ea640 1 -- 172.21.15.88:0/538231289 >> 172.21.15.88:0/538231289 conn(0x7f79a40ce5e0 msgr2=0x7f79a4071320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:58.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.062+0000 7f79a88ea640 1 -- 172.21.15.88:0/538231289 shutdown_connections 2023-10-16T20:06:58.068 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.062+0000 7f79a88ea640 1 -- 172.21.15.88:0/538231289 wait complete. 2023-10-16T20:06:58.084 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'testcephfs1_metadata', 'create_time': '2023-10-16T20:06:14.743500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'testcephfs1_data', 'create_time': '2023-10-16T20:06:16.091837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2023-10-16T20:06:31.184462+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T20:06:58.085 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:06:58.226 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.222+0000 7fe79e90f640 1 Processor -- start 2023-10-16T20:06:58.226 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.222+0000 7fe79e90f640 1 -- start start 2023-10-16T20:06:58.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.222+0000 7fe79e90f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7981500a0 0x7fe798159f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:58.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.222+0000 7fe79e90f640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe7980711f0 0x7fe79815aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:58.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.222+0000 7fe79e90f640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe79815aa10 con 0x7fe79806c2e0 2023-10-16T20:06:58.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.222+0000 7fe79e90f640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe798150480 con 0x7fe7980711f0 2023-10-16T20:06:58.228 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.222+0000 7fe79e90f640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe79815a490 con 0x7fe7981500a0 2023-10-16T20:06:58.229 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.226+0000 7fe797fff640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7981500a0 0x7fe798159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:58.229 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.226+0000 7fe797fff640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7981500a0 0x7fe798159f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:34572/0 (socket says 172.21.15.88:34572) 2023-10-16T20:06:58.229 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.226+0000 7fe797fff640 1 -- 172.21.15.88:0/1349958478 learned_addr learned my addr 172.21.15.88:0/1349958478 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:58.229 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.226+0000 7fe79ceac640 1 --2- 172.21.15.88:0/1349958478 >> v2:172.21.15.94:3300/0 conn(0x7fe7980711f0 0x7fe79815aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:58.229 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.226+0000 7fe7977fe640 1 --1- 172.21.15.88:0/1349958478 >> v1:172.21.15.94:6789/0 conn(0x7fe79806c2e0 0x7fe798070ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:60844/0 (socket says 172.21.15.88:60844) 2023-10-16T20:06:58.230 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.226+0000 7fe797fff640 1 -- 172.21.15.88:0/1349958478 >> v2:172.21.15.94:3300/0 conn(0x7fe7980711f0 msgr2=0x7fe79815aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:58.230 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.226+0000 7fe797fff640 1 --2- 172.21.15.88:0/1349958478 >> v2:172.21.15.94:3300/0 conn(0x7fe7980711f0 0x7fe79815aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.230 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.226+0000 7fe797fff640 1 -- 172.21.15.88:0/1349958478 >> v1:172.21.15.94:6789/0 conn(0x7fe79806c2e0 legacy=0x7fe798070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:58.230 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.226+0000 7fe797fff640 1 -- 172.21.15.88:0/1349958478 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe79806ca10 con 0x7fe7981500a0 2023-10-16T20:06:58.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.226+0000 7fe797fff640 1 --2- 172.21.15.88:0/1349958478 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7981500a0 0x7fe798159f50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe788006fd0 tx=0x7fe78801e9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7697a4c4e12831d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:58.232 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.226+0000 7fe796ffd640 1 -- 172.21.15.88:0/1349958478 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe788004230 con 0x7fe7981500a0 2023-10-16T20:06:58.232 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.230+0000 7fe796ffd640 1 -- 172.21.15.88:0/1349958478 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7880ab070 con 0x7fe7981500a0 2023-10-16T20:06:58.232 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.230+0000 7fe796ffd640 1 -- 172.21.15.88:0/1349958478 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7880a7710 con 0x7fe7981500a0 2023-10-16T20:06:58.232 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.230+0000 7fe79e90f640 1 -- 172.21.15.88:0/1349958478 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7981500a0 msgr2=0x7fe798159f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:58.233 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.230+0000 7fe79e90f640 1 --2- 172.21.15.88:0/1349958478 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7981500a0 0x7fe798159f50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe788006fd0 tx=0x7fe78801e9d0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.233 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.230+0000 7fe79e90f640 1 -- 172.21.15.88:0/1349958478 shutdown_connections 2023-10-16T20:06:58.233 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.230+0000 7fe79e90f640 1 --2- 172.21.15.88:0/1349958478 >> v2:172.21.15.94:3300/0 conn(0x7fe7980711f0 0x7fe79815aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.233 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.230+0000 7fe79e90f640 1 --2- 172.21.15.88:0/1349958478 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7981500a0 0x7fe798159f50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.233 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.230+0000 7fe79e90f640 1 -- 172.21.15.88:0/1349958478 >> 172.21.15.88:0/1349958478 conn(0x7fe7980cb080 msgr2=0x7fe7980cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:58.233 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.230+0000 7fe79e90f640 1 -- 172.21.15.88:0/1349958478 shutdown_connections 2023-10-16T20:06:58.234 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.230+0000 7fe79e90f640 1 -- 172.21.15.88:0/1349958478 wait complete. 2023-10-16T20:06:58.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.230+0000 7fe79e90f640 1 Processor -- start 2023-10-16T20:06:58.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.230+0000 7fe79e90f640 1 -- start start 2023-10-16T20:06:58.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.230+0000 7fe79e90f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe79806c2e0 0x7fe79814f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:58.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.230+0000 7fe79e90f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7980711f0 0x7fe7980cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:58.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.230+0000 7fe79e90f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7981500a0 0x7fe7980ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:58.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.230+0000 7fe79e90f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe798146ad0 con 0x7fe79806c2e0 2023-10-16T20:06:58.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.230+0000 7fe79e90f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79814f920 con 0x7fe7981500a0 2023-10-16T20:06:58.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.230+0000 7fe79e90f640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe79814b120 con 0x7fe7980711f0 2023-10-16T20:06:58.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.230+0000 7fe797fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe79806c2e0 0x7fe79814f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:58.237 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.230+0000 7fe797fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe79806c2e0 0x7fe79814f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53272/0 (socket says 172.21.15.88:53272) 2023-10-16T20:06:58.237 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.230+0000 7fe797fff640 1 -- 172.21.15.88:0/874988200 learned_addr learned my addr 172.21.15.88:0/874988200 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:58.237 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.230+0000 7fe7977fe640 1 --2- 172.21.15.88:0/874988200 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7980711f0 0x7fe7980cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:58.237 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.230+0000 7fe79ceac640 1 --2- 172.21.15.88:0/874988200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7981500a0 0x7fe7980ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:58.239 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.234+0000 7fe797fff640 1 -- 172.21.15.88:0/874988200 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7980711f0 msgr2=0x7fe7980cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:58.239 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.234+0000 7fe797fff640 1 --2- 172.21.15.88:0/874988200 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7980711f0 0x7fe7980cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.239 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.234+0000 7fe797fff640 1 -- 172.21.15.88:0/874988200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7981500a0 msgr2=0x7fe7980ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:58.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.234+0000 7fe797fff640 1 --2- 172.21.15.88:0/874988200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7981500a0 0x7fe7980ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.234+0000 7fe797fff640 1 -- 172.21.15.88:0/874988200 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe78800b780 con 0x7fe79806c2e0 2023-10-16T20:06:58.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.234+0000 7fe79ceac640 1 --2- 172.21.15.88:0/874988200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7981500a0 0x7fe7980ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:58.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.234+0000 7fe797fff640 1 --2- 172.21.15.88:0/874988200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe79806c2e0 0x7fe79814f3e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe78801c600 tx=0x7fe78801b770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:58.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.234+0000 7fe794ff9640 1 -- 172.21.15.88:0/874988200 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7880a79a0 con 0x7fe79806c2e0 2023-10-16T20:06:58.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.234+0000 7fe794ff9640 1 -- 172.21.15.88:0/874988200 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7880a7b20 con 0x7fe79806c2e0 2023-10-16T20:06:58.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.234+0000 7fe794ff9640 1 -- 172.21.15.88:0/874988200 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe788003620 con 0x7fe79806c2e0 2023-10-16T20:06:58.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.234+0000 7fe79e90f640 1 -- 172.21.15.88:0/874988200 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe79814b3a0 con 0x7fe79806c2e0 2023-10-16T20:06:58.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.234+0000 7fe79e90f640 1 -- 172.21.15.88:0/874988200 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe798146d40 con 0x7fe79806c2e0 2023-10-16T20:06:58.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.234+0000 7fe79e90f640 1 -- 172.21.15.88:0/874988200 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe798146f30 con 0x7fe79806c2e0 2023-10-16T20:06:58.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.234+0000 7fe794ff9640 1 -- 172.21.15.88:0/874988200 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe7880b9070 con 0x7fe79806c2e0 2023-10-16T20:06:58.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.234+0000 7fe794ff9640 1 --2- 172.21.15.88:0/874988200 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe7840664a0 0x7fe784068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:58.242 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.234+0000 7fe794ff9640 1 -- 172.21.15.88:0/874988200 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7fe78811ff60 con 0x7fe79806c2e0 2023-10-16T20:06:58.242 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.234+0000 7fe7977fe640 1 --2- 172.21.15.88:0/874988200 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe7840664a0 0x7fe784068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:58.242 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.238+0000 7fe7977fe640 1 --2- 172.21.15.88:0/874988200 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe7840664a0 0x7fe784068960 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe78c000f70 tx=0x7fe78c007c90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:58.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.238+0000 7fe794ff9640 1 -- 172.21.15.88:0/874988200 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe7880ed040 con 0x7fe79806c2e0 2023-10-16T20:06:58.394 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.390+0000 7fe79e90f640 1 -- 172.21.15.88:0/874988200 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe79804e9c0 con 0x7fe79806c2e0 2023-10-16T20:06:58.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.390+0000 7fe794ff9640 1 -- 172.21.15.88:0/874988200 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe7880ad0a0 con 0x7fe79806c2e0 2023-10-16T20:06:58.395 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:06:58.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.394+0000 7fe79e90f640 1 -- 172.21.15.88:0/874988200 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe7840664a0 msgr2=0x7fe784068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:58.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.394+0000 7fe79e90f640 1 --2- 172.21.15.88:0/874988200 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe7840664a0 0x7fe784068960 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe78c000f70 tx=0x7fe78c007c90 comp rx=0 tx=0).stop 2023-10-16T20:06:58.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.394+0000 7fe79e90f640 1 -- 172.21.15.88:0/874988200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe79806c2e0 msgr2=0x7fe79814f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:58.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.394+0000 7fe79e90f640 1 --2- 172.21.15.88:0/874988200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe79806c2e0 0x7fe79814f3e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe78801c600 tx=0x7fe78801b770 comp rx=0 tx=0).stop 2023-10-16T20:06:58.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.394+0000 7fe79e90f640 1 -- 172.21.15.88:0/874988200 shutdown_connections 2023-10-16T20:06:58.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.394+0000 7fe79e90f640 1 --2- 172.21.15.88:0/874988200 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe7840664a0 0x7fe784068960 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.394+0000 7fe79e90f640 1 --2- 172.21.15.88:0/874988200 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7980711f0 0x7fe7980cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.394+0000 7fe79e90f640 1 --2- 172.21.15.88:0/874988200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7981500a0 0x7fe7980ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.394+0000 7fe79e90f640 1 --2- 172.21.15.88:0/874988200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe79806c2e0 0x7fe79814f3e0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.394+0000 7fe79e90f640 1 -- 172.21.15.88:0/874988200 >> 172.21.15.88:0/874988200 conn(0x7fe7980cb080 msgr2=0x7fe7981589f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:58.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.394+0000 7fe79e90f640 1 -- 172.21.15.88:0/874988200 shutdown_connections 2023-10-16T20:06:58.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.394+0000 7fe79e90f640 1 -- 172.21.15.88:0/874988200 wait complete. 2023-10-16T20:06:58.415 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_metadata pg_num 2023-10-16T20:06:58.557 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.554+0000 7f2d35d1d640 1 Processor -- start 2023-10-16T20:06:58.557 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.554+0000 7f2d35d1d640 1 -- start start 2023-10-16T20:06:58.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.554+0000 7f2d35d1d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2d3015cb70 0x7f2d300b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:58.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.554+0000 7f2d35d1d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2d300b4530 0x7f2d300b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:58.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.554+0000 7f2d35d1d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f2d300d0bd0 0x7f2d300d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:58.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.554+0000 7f2d35d1d640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d300b4ed0 con 0x7f2d300d0bd0 2023-10-16T20:06:58.560 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.554+0000 7f2d35d1d640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d300b7d20 con 0x7f2d300b4530 2023-10-16T20:06:58.560 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.554+0000 7f2d35d1d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d300b7110 con 0x7f2d3015cb70 2023-10-16T20:06:58.560 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.554+0000 7f2d2ffff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f2d300d0bd0 0x7f2d300d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:58.560 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.554+0000 7f2d2ffff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f2d300d0bd0 0x7f2d300d1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53288/0 (socket says 172.21.15.88:53288) 2023-10-16T20:06:58.560 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.554+0000 7f2d2ffff640 1 -- 172.21.15.88:0/3370563868 learned_addr learned my addr 172.21.15.88:0/3370563868 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:58.561 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.554+0000 7f2d2f7fe640 1 --2- 172.21.15.88:0/3370563868 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2d3015cb70 0x7f2d300b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:58.561 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.554+0000 7f2d2effd640 1 --2- 172.21.15.88:0/3370563868 >> v2:172.21.15.94:3300/0 conn(0x7f2d300b4530 0x7f2d300b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:58.561 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.554+0000 7f2d2ffff640 1 -- 172.21.15.88:0/3370563868 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2d3015cb70 msgr2=0x7f2d300b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:58.561 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.554+0000 7f2d2ffff640 1 --2- 172.21.15.88:0/3370563868 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2d3015cb70 0x7f2d300b83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.561 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.554+0000 7f2d2ffff640 1 -- 172.21.15.88:0/3370563868 >> v2:172.21.15.94:3300/0 conn(0x7f2d300b4530 msgr2=0x7f2d300b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:58.562 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.554+0000 7f2d2ffff640 1 --2- 172.21.15.88:0/3370563868 >> v2:172.21.15.94:3300/0 conn(0x7f2d300b4530 0x7f2d300b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.562 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.554+0000 7f2d2ffff640 1 -- 172.21.15.88:0/3370563868 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d300b73f0 con 0x7f2d300d0bd0 2023-10-16T20:06:58.562 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.554+0000 7f2d2effd640 1 --2- 172.21.15.88:0/3370563868 >> v2:172.21.15.94:3300/0 conn(0x7f2d300b4530 0x7f2d300b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:58.563 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.558+0000 7f2d2ffff640 1 --2- 172.21.15.88:0/3370563868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d300d0bd0 0x7f2d300d1060 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f2d3008bbf0 tx=0x7f2d2801f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c5be8643a7dffe3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:58.563 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.558+0000 7f2d2e7fc640 1 -- 172.21.15.88:0/3370563868 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d280af400 con 0x7f2d300d0bd0 2023-10-16T20:06:58.564 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.558+0000 7f2d2e7fc640 1 -- 172.21.15.88:0/3370563868 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d280ab050 con 0x7f2d300d0bd0 2023-10-16T20:06:58.564 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.558+0000 7f2d2e7fc640 1 -- 172.21.15.88:0/3370563868 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d280af780 con 0x7f2d300d0bd0 2023-10-16T20:06:58.564 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.558+0000 7f2d35d1d640 1 -- 172.21.15.88:0/3370563868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d300d0bd0 msgr2=0x7f2d300d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:58.564 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.558+0000 7f2d35d1d640 1 --2- 172.21.15.88:0/3370563868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d300d0bd0 0x7f2d300d1060 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f2d3008bbf0 tx=0x7f2d2801f1c0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.564 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.558+0000 7f2d35d1d640 1 -- 172.21.15.88:0/3370563868 shutdown_connections 2023-10-16T20:06:58.565 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.558+0000 7f2d35d1d640 1 --2- 172.21.15.88:0/3370563868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d300d0bd0 0x7f2d300d1060 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.565 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.558+0000 7f2d35d1d640 1 --2- 172.21.15.88:0/3370563868 >> v2:172.21.15.94:3300/0 conn(0x7f2d300b4530 0x7f2d300b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.565 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.558+0000 7f2d35d1d640 1 --2- 172.21.15.88:0/3370563868 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2d3015cb70 0x7f2d300b83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.565 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.558+0000 7f2d35d1d640 1 -- 172.21.15.88:0/3370563868 >> 172.21.15.88:0/3370563868 conn(0x7f2d3006b6e0 msgr2=0x7f2d3006baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:58.565 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.558+0000 7f2d35d1d640 1 -- 172.21.15.88:0/3370563868 shutdown_connections 2023-10-16T20:06:58.566 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.558+0000 7f2d35d1d640 1 -- 172.21.15.88:0/3370563868 wait complete. 2023-10-16T20:06:58.567 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.562+0000 7f2d35d1d640 1 Processor -- start 2023-10-16T20:06:58.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.562+0000 7f2d35d1d640 1 -- start start 2023-10-16T20:06:58.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.562+0000 7f2d35d1d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d300b4530 0x7f2d3014ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:58.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.562+0000 7f2d35d1d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d300d0bd0 0x7f2d3014fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:58.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.562+0000 7f2d35d1d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2d3015cb70 0x7f2d3014c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:58.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.562+0000 7f2d35d1d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d30154a00 con 0x7f2d300d0bd0 2023-10-16T20:06:58.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.562+0000 7f2d35d1d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d30154720 con 0x7f2d300b4530 2023-10-16T20:06:58.569 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.562+0000 7f2d35d1d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d300c65b0 con 0x7f2d3015cb70 2023-10-16T20:06:58.569 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.562+0000 7f2d2effd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d300d0bd0 0x7f2d3014fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:58.569 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.562+0000 7f2d2effd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d300d0bd0 0x7f2d3014fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53302/0 (socket says 172.21.15.88:53302) 2023-10-16T20:06:58.569 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.562+0000 7f2d2effd640 1 -- 172.21.15.88:0/2415913981 learned_addr learned my addr 172.21.15.88:0/2415913981 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:58.569 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.562+0000 7f2d2effd640 1 -- 172.21.15.88:0/2415913981 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2d3015cb70 msgr2=0x7f2d3014c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:58.570 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.562+0000 7f2d2effd640 1 --2- 172.21.15.88:0/2415913981 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2d3015cb70 0x7f2d3014c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.570 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.562+0000 7f2d2effd640 1 -- 172.21.15.88:0/2415913981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d300b4530 msgr2=0x7f2d3014ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:58.570 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.562+0000 7f2d2f7fe640 1 --2- 172.21.15.88:0/2415913981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d300b4530 0x7f2d3014ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:58.570 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.562+0000 7f2d2effd640 1 --2- 172.21.15.88:0/2415913981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d300b4530 0x7f2d3014ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.570 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.562+0000 7f2d2effd640 1 -- 172.21.15.88:0/2415913981 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d28009b30 con 0x7f2d300d0bd0 2023-10-16T20:06:58.571 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.562+0000 7f2d2f7fe640 1 --2- 172.21.15.88:0/2415913981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d300b4530 0x7f2d3014ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:06:58.571 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.562+0000 7f2d2effd640 1 --2- 172.21.15.88:0/2415913981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d300d0bd0 0x7f2d3014fa30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400e810 tx=0x7f2d2400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:58.571 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.562+0000 7f2d17fff640 1 -- 172.21.15.88:0/2415913981 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d24004030 con 0x7f2d300d0bd0 2023-10-16T20:06:58.571 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.562+0000 7f2d17fff640 1 -- 172.21.15.88:0/2415913981 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d24004190 con 0x7f2d300d0bd0 2023-10-16T20:06:58.571 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.562+0000 7f2d17fff640 1 -- 172.21.15.88:0/2415913981 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d24004520 con 0x7f2d300d0bd0 2023-10-16T20:06:58.572 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.562+0000 7f2d35d1d640 1 -- 172.21.15.88:0/2415913981 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d300c6890 con 0x7f2d300d0bd0 2023-10-16T20:06:58.572 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.566+0000 7f2d35d1d640 1 -- 172.21.15.88:0/2415913981 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d3014f140 con 0x7f2d300d0bd0 2023-10-16T20:06:58.573 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.566+0000 7f2d35d1d640 1 -- 172.21.15.88:0/2415913981 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d3004e9c0 con 0x7f2d300d0bd0 2023-10-16T20:06:58.573 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.566+0000 7f2d17fff640 1 -- 172.21.15.88:0/2415913981 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2d2400c6d0 con 0x7f2d300d0bd0 2023-10-16T20:06:58.573 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.566+0000 7f2d17fff640 1 --2- 172.21.15.88:0/2415913981 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2d1c0664a0 0x7f2d1c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:58.573 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.566+0000 7f2d17fff640 1 -- 172.21.15.88:0/2415913981 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f2d24083d90 con 0x7f2d300d0bd0 2023-10-16T20:06:58.573 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.566+0000 7f2d2f7fe640 1 --2- 172.21.15.88:0/2415913981 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2d1c0664a0 0x7f2d1c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:58.574 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.566+0000 7f2d2f7fe640 1 --2- 172.21.15.88:0/2415913981 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2d1c0664a0 0x7f2d1c068960 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2d20009a40 tx=0x7f2d20008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:58.574 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.570+0000 7f2d17fff640 1 -- 172.21.15.88:0/2415913981 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2d24050f00 con 0x7f2d300d0bd0 2023-10-16T20:06:58.721 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.718+0000 7f2d35d1d640 1 -- 172.21.15.88:0/2415913981 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2d3006a220 con 0x7f2d300d0bd0 2023-10-16T20:06:58.722 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.718+0000 7f2d17fff640 1 -- 172.21.15.88:0/2415913981 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"}]=0 v46) v1 ==== 109+0+11 (secure 0 0 0) 0x7f2d24054520 con 0x7f2d300d0bd0 2023-10-16T20:06:58.722 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:06:58.724 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.718+0000 7f2d35d1d640 1 -- 172.21.15.88:0/2415913981 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2d1c0664a0 msgr2=0x7f2d1c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:58.724 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.718+0000 7f2d35d1d640 1 --2- 172.21.15.88:0/2415913981 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2d1c0664a0 0x7f2d1c068960 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2d20009a40 tx=0x7f2d20008040 comp rx=0 tx=0).stop 2023-10-16T20:06:58.724 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.722+0000 7f2d35d1d640 1 -- 172.21.15.88:0/2415913981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d300d0bd0 msgr2=0x7f2d3014fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:58.724 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.722+0000 7f2d35d1d640 1 --2- 172.21.15.88:0/2415913981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d300d0bd0 0x7f2d3014fa30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400e810 tx=0x7f2d2400ecd0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.725 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.722+0000 7f2d35d1d640 1 -- 172.21.15.88:0/2415913981 shutdown_connections 2023-10-16T20:06:58.725 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.722+0000 7f2d35d1d640 1 --2- 172.21.15.88:0/2415913981 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2d1c0664a0 0x7f2d1c068960 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.725 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.722+0000 7f2d35d1d640 1 --2- 172.21.15.88:0/2415913981 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2d3015cb70 0x7f2d3014c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.725 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.722+0000 7f2d35d1d640 1 --2- 172.21.15.88:0/2415913981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d300d0bd0 0x7f2d3014fa30 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.725 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.722+0000 7f2d35d1d640 1 --2- 172.21.15.88:0/2415913981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d300b4530 0x7f2d3014ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.725 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.722+0000 7f2d35d1d640 1 -- 172.21.15.88:0/2415913981 >> 172.21.15.88:0/2415913981 conn(0x7f2d3006b6e0 msgr2=0x7f2d3015b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:58.726 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.722+0000 7f2d35d1d640 1 -- 172.21.15.88:0/2415913981 shutdown_connections 2023-10-16T20:06:58.726 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.722+0000 7f2d35d1d640 1 -- 172.21.15.88:0/2415913981 wait complete. 2023-10-16T20:06:58.743 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_data pg_num 2023-10-16T20:06:58.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.878+0000 7f5430baa640 1 Processor -- start 2023-10-16T20:06:58.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.878+0000 7f5430baa640 1 -- start start 2023-10-16T20:06:58.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.878+0000 7f5430baa640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f542c15cac0 0x7f542c0ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:58.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.882+0000 7f5430baa640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f542c15cea0 0x7f542c0cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:58.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.882+0000 7f5430baa640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f542c06b840 con 0x7f542c0ae130 2023-10-16T20:06:58.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.882+0000 7f5430baa640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f542c0cac10 con 0x7f542c15cac0 2023-10-16T20:06:58.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.882+0000 7f5430baa640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542c06a3a0 con 0x7f542c15cea0 2023-10-16T20:06:58.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.882+0000 7f542ad9d640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f542c0ae130 0x7f542c1607d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:57300/0 (socket says 172.21.15.88:57300) 2023-10-16T20:06:58.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.882+0000 7f542ad9d640 1 -- 172.21.15.88:0/1022755702 learned_addr learned my addr 172.21.15.88:0/1022755702 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:58.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.882+0000 7f5421d9b640 1 --2- 172.21.15.88:0/1022755702 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f542c15cea0 0x7f542c0cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:58.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.882+0000 7f542a59c640 1 --2- 172.21.15.88:0/1022755702 >> v2:172.21.15.94:3300/0 conn(0x7f542c15cac0 0x7f542c0ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:58.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.882+0000 7f5429d9b640 1 -- 172.21.15.88:0/1022755702 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 350708279 0 0) 0x7f542c06b840 con 0x7f542c0ae130 2023-10-16T20:06:58.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.882+0000 7f5429d9b640 1 -- 172.21.15.88:0/1022755702 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5418003530 con 0x7f542c0ae130 2023-10-16T20:06:58.887 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.882+0000 7f5421d9b640 1 -- 172.21.15.88:0/1022755702 >> v2:172.21.15.94:3300/0 conn(0x7f542c15cac0 msgr2=0x7f542c0ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:58.887 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.882+0000 7f5421d9b640 1 --2- 172.21.15.88:0/1022755702 >> v2:172.21.15.94:3300/0 conn(0x7f542c15cac0 0x7f542c0ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.887 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.882+0000 7f5421d9b640 1 -- 172.21.15.88:0/1022755702 >> v1:172.21.15.88:6789/0 conn(0x7f542c0ae130 legacy=0x7f542c1607d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:58.887 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.882+0000 7f5429d9b640 1 -- 172.21.15.88:0/1022755702 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4071690161 0 0) 0x7f5418003530 con 0x7f542c0ae130 2023-10-16T20:06:58.887 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.882+0000 7f5421d9b640 1 -- 172.21.15.88:0/1022755702 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542c0ca650 con 0x7f542c15cea0 2023-10-16T20:06:58.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.882+0000 7f542a59c640 1 --2- 172.21.15.88:0/1022755702 >> v2:172.21.15.94:3300/0 conn(0x7f542c15cac0 0x7f542c0ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:06:58.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.886+0000 7f5421d9b640 1 --2- 172.21.15.88:0/1022755702 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f542c15cea0 0x7f542c0cd1a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f541c0097c0 tx=0x7f541c01eb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79645d85f2e05f7f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:58.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.886+0000 7f5429d9b640 1 -- 172.21.15.88:0/1022755702 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5424001a00 con 0x7f542c15cea0 2023-10-16T20:06:58.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.886+0000 7f5429d9b640 1 -- 172.21.15.88:0/1022755702 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541c0a85d0 con 0x7f542c15cea0 2023-10-16T20:06:58.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.886+0000 7f5429d9b640 1 -- 172.21.15.88:0/1022755702 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c0a8960 con 0x7f542c15cea0 2023-10-16T20:06:58.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.886+0000 7f5430baa640 1 -- 172.21.15.88:0/1022755702 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f542c15cea0 msgr2=0x7f542c0cd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:58.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.886+0000 7f5430baa640 1 --2- 172.21.15.88:0/1022755702 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f542c15cea0 0x7f542c0cd1a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f541c0097c0 tx=0x7f541c01eb90 comp rx=0 tx=0).stop 2023-10-16T20:06:58.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.886+0000 7f5430baa640 1 -- 172.21.15.88:0/1022755702 shutdown_connections 2023-10-16T20:06:58.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.886+0000 7f5430baa640 1 --2- 172.21.15.88:0/1022755702 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f542c15cea0 0x7f542c0cd1a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.891 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.886+0000 7f5430baa640 1 --2- 172.21.15.88:0/1022755702 >> v2:172.21.15.94:3300/0 conn(0x7f542c15cac0 0x7f542c0ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.891 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.886+0000 7f5430baa640 1 -- 172.21.15.88:0/1022755702 >> 172.21.15.88:0/1022755702 conn(0x7f542c0b0970 msgr2=0x7f542c0b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:58.891 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.886+0000 7f5430baa640 1 -- 172.21.15.88:0/1022755702 shutdown_connections 2023-10-16T20:06:58.891 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.886+0000 7f5430baa640 1 -- 172.21.15.88:0/1022755702 wait complete. 2023-10-16T20:06:58.893 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.886+0000 7f5430baa640 1 Processor -- start 2023-10-16T20:06:58.893 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.886+0000 7f5430baa640 1 -- start start 2023-10-16T20:06:58.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.890+0000 7f5430baa640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f542c0ae130 0x7f542c14d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:58.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.890+0000 7f5430baa640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f542c15cac0 0x7f542c0ac260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:58.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.890+0000 7f5430baa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f542c15cea0 0x7f542c155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:58.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.890+0000 7f5430baa640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c0acdc0 con 0x7f542c0ae130 2023-10-16T20:06:58.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.890+0000 7f5430baa640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c0ac7a0 con 0x7f542c15cea0 2023-10-16T20:06:58.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.890+0000 7f5430baa640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542c0c7680 con 0x7f542c15cac0 2023-10-16T20:06:58.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.890+0000 7f542a59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f542c0ae130 0x7f542c14d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:58.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.890+0000 7f542a59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f542c0ae130 0x7f542c14d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53306/0 (socket says 172.21.15.88:53306) 2023-10-16T20:06:58.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.890+0000 7f542a59c640 1 -- 172.21.15.88:0/2486878158 learned_addr learned my addr 172.21.15.88:0/2486878158 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:58.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.890+0000 7f5421d9b640 1 --2- 172.21.15.88:0/2486878158 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f542c15cac0 0x7f542c0ac260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:58.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.890+0000 7f542ad9d640 1 --2- 172.21.15.88:0/2486878158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f542c15cea0 0x7f542c155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:58.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.890+0000 7f542a59c640 1 -- 172.21.15.88:0/2486878158 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f542c15cac0 msgr2=0x7f542c0ac260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:58.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.890+0000 7f542a59c640 1 --2- 172.21.15.88:0/2486878158 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f542c15cac0 0x7f542c0ac260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.890+0000 7f542a59c640 1 -- 172.21.15.88:0/2486878158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f542c15cea0 msgr2=0x7f542c155a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:58.897 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.890+0000 7f542a59c640 1 --2- 172.21.15.88:0/2486878158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f542c15cea0 0x7f542c155a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:58.897 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.890+0000 7f542a59c640 1 -- 172.21.15.88:0/2486878158 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5414009420 con 0x7f542c0ae130 2023-10-16T20:06:58.897 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.890+0000 7f542ad9d640 1 --2- 172.21.15.88:0/2486878158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f542c15cea0 0x7f542c155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:58.897 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.890+0000 7f542a59c640 1 --2- 172.21.15.88:0/2486878158 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f542c0ae130 0x7f542c14d280 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f541400ebe0 tx=0x7f541400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:58.897 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.890+0000 7f54237fe640 1 -- 172.21.15.88:0/2486878158 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541400eee0 con 0x7f542c0ae130 2023-10-16T20:06:58.898 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.890+0000 7f54237fe640 1 -- 172.21.15.88:0/2486878158 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541400cc30 con 0x7f542c0ae130 2023-10-16T20:06:58.898 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.890+0000 7f54237fe640 1 -- 172.21.15.88:0/2486878158 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5414013470 con 0x7f542c0ae130 2023-10-16T20:06:58.898 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.890+0000 7f5430baa640 1 -- 172.21.15.88:0/2486878158 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f541c009420 con 0x7f542c0ae130 2023-10-16T20:06:58.898 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.890+0000 7f5430baa640 1 -- 172.21.15.88:0/2486878158 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f542c0c7c40 con 0x7f542c0ae130 2023-10-16T20:06:58.899 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.890+0000 7f5430baa640 1 -- 172.21.15.88:0/2486878158 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f542c04e9c0 con 0x7f542c0ae130 2023-10-16T20:06:58.900 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.890+0000 7f54237fe640 1 -- 172.21.15.88:0/2486878158 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f54140135d0 con 0x7f542c0ae130 2023-10-16T20:06:58.900 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.894+0000 7f54237fe640 1 --2- 172.21.15.88:0/2486878158 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f540c0664a0 0x7f540c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:58.900 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.894+0000 7f54237fe640 1 -- 172.21.15.88:0/2486878158 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f54140843e0 con 0x7f542c0ae130 2023-10-16T20:06:58.900 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.894+0000 7f5421d9b640 1 --2- 172.21.15.88:0/2486878158 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f540c0664a0 0x7f540c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:58.900 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.894+0000 7f5421d9b640 1 --2- 172.21.15.88:0/2486878158 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f540c0664a0 0x7f540c068960 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f541c0b2000 tx=0x7f541c0aa070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:58.900 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:58.898+0000 7f54237fe640 1 -- 172.21.15.88:0/2486878158 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5414051600 con 0x7f542c0ae130 2023-10-16T20:06:59.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.042+0000 7f5430baa640 1 -- 172.21.15.88:0/2486878158 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"} v 0) v1 -- 0x7f542c071770 con 0x7f542c0ae130 2023-10-16T20:06:59.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.042+0000 7f54237fe640 1 -- 172.21.15.88:0/2486878158 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"}]=0 v46) v1 ==== 105+0+11 (secure 0 0 0) 0x7f5414054c20 con 0x7f542c0ae130 2023-10-16T20:06:59.047 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2023-10-16T20:06:59.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.046+0000 7f5430baa640 1 -- 172.21.15.88:0/2486878158 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f540c0664a0 msgr2=0x7f540c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:59.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.046+0000 7f5430baa640 1 --2- 172.21.15.88:0/2486878158 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f540c0664a0 0x7f540c068960 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f541c0b2000 tx=0x7f541c0aa070 comp rx=0 tx=0).stop 2023-10-16T20:06:59.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.046+0000 7f5430baa640 1 -- 172.21.15.88:0/2486878158 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f542c0ae130 msgr2=0x7f542c14d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:59.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.046+0000 7f5430baa640 1 --2- 172.21.15.88:0/2486878158 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f542c0ae130 0x7f542c14d280 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f541400ebe0 tx=0x7f541400c520 comp rx=0 tx=0).stop 2023-10-16T20:06:59.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.046+0000 7f5430baa640 1 -- 172.21.15.88:0/2486878158 shutdown_connections 2023-10-16T20:06:59.050 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.046+0000 7f5430baa640 1 --2- 172.21.15.88:0/2486878158 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f540c0664a0 0x7f540c068960 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:59.050 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.046+0000 7f5430baa640 1 --2- 172.21.15.88:0/2486878158 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f542c15cac0 0x7f542c0ac260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:59.050 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.046+0000 7f5430baa640 1 --2- 172.21.15.88:0/2486878158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f542c15cea0 0x7f542c155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:59.050 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.046+0000 7f5430baa640 1 --2- 172.21.15.88:0/2486878158 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f542c0ae130 0x7f542c14d280 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:59.051 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.046+0000 7f5430baa640 1 -- 172.21.15.88:0/2486878158 >> 172.21.15.88:0/2486878158 conn(0x7f542c0b0970 msgr2=0x7f542c15bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:59.051 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.046+0000 7f5430baa640 1 -- 172.21.15.88:0/2486878158 shutdown_connections 2023-10-16T20:06:59.051 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.046+0000 7f5430baa640 1 -- 172.21.15.88:0/2486878158 wait complete. 2023-10-16T20:06:59.068 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-10-16T20:06:59.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f178c965640 1 Processor -- start 2023-10-16T20:06:59.208 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f178c965640 1 -- start start 2023-10-16T20:06:59.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f178c965640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f17880ca0b0 0x7f1788161540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:59.209 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f178c965640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1788071360 0x7f17880717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:59.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f178c965640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1788161a80 con 0x7f178815cf90 2023-10-16T20:06:59.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f178c965640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1788071d30 con 0x7f17880ca0b0 2023-10-16T20:06:59.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f178c965640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17880ca840 con 0x7f1788071360 2023-10-16T20:06:59.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f178659c640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f178815cf90 0x7f17880cb8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:57314/0 (socket says 172.21.15.88:57314) 2023-10-16T20:06:59.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f178659c640 1 -- 172.21.15.88:0/2693785028 learned_addr learned my addr 172.21.15.88:0/2693785028 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:06:59.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f1786d9d640 1 --2- 172.21.15.88:0/2693785028 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1788071360 0x7f17880717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:59.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f177dd9b640 1 --2- 172.21.15.88:0/2693785028 >> v2:172.21.15.94:3300/0 conn(0x7f17880ca0b0 0x7f1788161540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:59.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f1785d9b640 1 -- 172.21.15.88:0/2693785028 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060019944 0 0) 0x7f1788161a80 con 0x7f178815cf90 2023-10-16T20:06:59.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f1785d9b640 1 -- 172.21.15.88:0/2693785028 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1774003530 con 0x7f178815cf90 2023-10-16T20:06:59.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f1785d9b640 1 -- 172.21.15.88:0/2693785028 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1952032175 0 0) 0x7f1774003530 con 0x7f178815cf90 2023-10-16T20:06:59.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f1785d9b640 1 -- 172.21.15.88:0/2693785028 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1788071360 msgr2=0x7f17880717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:59.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f1785d9b640 1 --2- 172.21.15.88:0/2693785028 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1788071360 0x7f17880717f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:59.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f1785d9b640 1 -- 172.21.15.88:0/2693785028 >> v2:172.21.15.94:3300/0 conn(0x7f17880ca0b0 msgr2=0x7f1788161540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:59.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f1785d9b640 1 --2- 172.21.15.88:0/2693785028 >> v2:172.21.15.94:3300/0 conn(0x7f17880ca0b0 0x7f1788161540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:59.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f1785d9b640 1 -- 172.21.15.88:0/2693785028 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17880cab20 con 0x7f178815cf90 2023-10-16T20:06:59.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f1785d9b640 1 -- 172.21.15.88:0/2693785028 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f1778001680 con 0x7f178815cf90 2023-10-16T20:06:59.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f1785d9b640 1 -- 172.21.15.88:0/2693785028 >> v1:172.21.15.88:6789/0 conn(0x7f178815cf90 legacy=0x7f17880cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:59.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f1785d9b640 1 --2- 172.21.15.88:0/2693785028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1774003980 0x7f1774003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:59.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f1785d9b640 1 --2- 172.21.15.88:0/2693785028 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1788071360 0x7f17740047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:59.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f1785d9b640 1 --2- 172.21.15.88:0/2693785028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1774004d10 0x7f1774005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:59.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f1785d9b640 1 -- 172.21.15.88:0/2693785028 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1788071d30 con 0x7f1774004d10 2023-10-16T20:06:59.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f1785d9b640 1 -- 172.21.15.88:0/2693785028 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17880ca840 con 0x7f1774003980 2023-10-16T20:06:59.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f1785d9b640 1 -- 172.21.15.88:0/2693785028 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f177400b990 con 0x7f1788071360 2023-10-16T20:06:59.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f1786d9d640 1 --2- 172.21.15.88:0/2693785028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1774004d10 0x7f1774005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:59.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f177dd9b640 1 --2- 172.21.15.88:0/2693785028 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1788071360 0x7f17740047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:59.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f178659c640 1 --2- 172.21.15.88:0/2693785028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1774003980 0x7f1774003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:59.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f1786d9d640 1 -- 172.21.15.88:0/2693785028 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1788071360 msgr2=0x7f17740047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:59.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f1786d9d640 1 --2- 172.21.15.88:0/2693785028 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1788071360 0x7f17740047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:59.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f1786d9d640 1 -- 172.21.15.88:0/2693785028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1774003980 msgr2=0x7f1774003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:59.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f1786d9d640 1 --2- 172.21.15.88:0/2693785028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1774003980 0x7f1774003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:59.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.206+0000 7f1786d9d640 1 -- 172.21.15.88:0/2693785028 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1774003530 con 0x7f1774004d10 2023-10-16T20:06:59.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.210+0000 7f1786d9d640 1 --2- 172.21.15.88:0/2693785028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1774004d10 0x7f1774005190 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1780009ae0 tx=0x7f178001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cfaf321caf71a65 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:59.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.210+0000 7f1785d9b640 1 -- 172.21.15.88:0/2693785028 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17800a5640 con 0x7f1774004d10 2023-10-16T20:06:59.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.210+0000 7f1785d9b640 1 -- 172.21.15.88:0/2693785028 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17800a57a0 con 0x7f1774004d10 2023-10-16T20:06:59.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.210+0000 7f1785d9b640 1 -- 172.21.15.88:0/2693785028 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17800a5b50 con 0x7f1774004d10 2023-10-16T20:06:59.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.210+0000 7f178c965640 1 -- 172.21.15.88:0/2693785028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1774004d10 msgr2=0x7f1774005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:59.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.210+0000 7f178c965640 1 --2- 172.21.15.88:0/2693785028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1774004d10 0x7f1774005190 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1780009ae0 tx=0x7f178001ca50 comp rx=0 tx=0).stop 2023-10-16T20:06:59.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.210+0000 7f1786d9d640 1 -- 172.21.15.88:0/2693785028 reap_dead start 2023-10-16T20:06:59.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.210+0000 7f178c965640 1 -- 172.21.15.88:0/2693785028 shutdown_connections 2023-10-16T20:06:59.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.210+0000 7f178c965640 1 -- 172.21.15.88:0/2693785028 >> 172.21.15.88:0/2693785028 conn(0x7f17880cf710 msgr2=0x7f17880cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:59.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.210+0000 7f178c965640 1 -- 172.21.15.88:0/2693785028 shutdown_connections 2023-10-16T20:06:59.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.210+0000 7f178c965640 1 -- 172.21.15.88:0/2693785028 wait complete. 2023-10-16T20:06:59.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.214+0000 7f178c965640 1 Processor -- start 2023-10-16T20:06:59.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.214+0000 7f178c965640 1 -- start start 2023-10-16T20:06:59.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.214+0000 7f178c965640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1774004d10 0x7f17880aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:59.222 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.214+0000 7f178c965640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17880b0840 0x7f178814d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:59.222 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.214+0000 7f178c965640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f178815b5e0 0x7f178815ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:59.222 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.214+0000 7f178c965640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178815c830 con 0x7f1774004d10 2023-10-16T20:06:59.222 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.214+0000 7f178c965640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178815bfa0 con 0x7f17880b0840 2023-10-16T20:06:59.222 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.214+0000 7f178c965640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17880afe60 con 0x7f178815b5e0 2023-10-16T20:06:59.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.214+0000 7f178659c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1774004d10 0x7f17880aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:59.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.214+0000 7f178659c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1774004d10 0x7f17880aeae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53322/0 (socket says 172.21.15.88:53322) 2023-10-16T20:06:59.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.214+0000 7f178659c640 1 -- 172.21.15.88:0/1062759513 learned_addr learned my addr 172.21.15.88:0/1062759513 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:59.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.214+0000 7f177dd9b640 1 --2- 172.21.15.88:0/1062759513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17880b0840 0x7f178814d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:59.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.214+0000 7f1786d9d640 1 --2- 172.21.15.88:0/1062759513 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f178815b5e0 0x7f178815ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:59.223 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.214+0000 7f178659c640 1 -- 172.21.15.88:0/1062759513 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f178815b5e0 msgr2=0x7f178815ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:59.224 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.214+0000 7f178659c640 1 --2- 172.21.15.88:0/1062759513 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f178815b5e0 0x7f178815ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:59.224 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.214+0000 7f178659c640 1 -- 172.21.15.88:0/1062759513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17880b0840 msgr2=0x7f178814d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:59.224 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.214+0000 7f178659c640 1 --2- 172.21.15.88:0/1062759513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17880b0840 0x7f178814d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:59.224 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.214+0000 7f178659c640 1 -- 172.21.15.88:0/1062759513 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17880af020 con 0x7f1774004d10 2023-10-16T20:06:59.224 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.214+0000 7f177dd9b640 1 --2- 172.21.15.88:0/1062759513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17880b0840 0x7f178814d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:59.224 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.214+0000 7f178659c640 1 --2- 172.21.15.88:0/1062759513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1774004d10 0x7f17880aeae0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1778007340 tx=0x7f1778007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:59.225 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.214+0000 7f177f7fe640 1 -- 172.21.15.88:0/1062759513 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1778007c20 con 0x7f1774004d10 2023-10-16T20:06:59.225 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.214+0000 7f177f7fe640 1 -- 172.21.15.88:0/1062759513 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f177800f9c0 con 0x7f1774004d10 2023-10-16T20:06:59.225 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.214+0000 7f178c965640 1 -- 172.21.15.88:0/1062759513 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17880af250 con 0x7f1774004d10 2023-10-16T20:06:59.225 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.214+0000 7f177f7fe640 1 -- 172.21.15.88:0/1062759513 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177800fd50 con 0x7f1774004d10 2023-10-16T20:06:59.225 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.214+0000 7f178c965640 1 -- 172.21.15.88:0/1062759513 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17880af810 con 0x7f1774004d10 2023-10-16T20:06:59.226 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.218+0000 7f178c965640 1 -- 172.21.15.88:0/1062759513 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1788067100 con 0x7f1774004d10 2023-10-16T20:06:59.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.218+0000 7f177f7fe640 1 -- 172.21.15.88:0/1062759513 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f177801e3b0 con 0x7f1774004d10 2023-10-16T20:06:59.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.218+0000 7f177f7fe640 1 --2- 172.21.15.88:0/1062759513 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f17680664d0 0x7f1768068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:59.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.218+0000 7f177f7fe640 1 -- 172.21.15.88:0/1062759513 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f1778083cc0 con 0x7f1774004d10 2023-10-16T20:06:59.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.218+0000 7f177dd9b640 1 --2- 172.21.15.88:0/1062759513 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f17680664d0 0x7f1768068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:59.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.218+0000 7f177dd9b640 1 --2- 172.21.15.88:0/1062759513 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f17680664d0 0x7f1768068990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1788151660 tx=0x7f177000b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:59.228 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.222+0000 7f177f7fe640 1 -- 172.21.15.88:0/1062759513 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1778050ed0 con 0x7f1774004d10 2023-10-16T20:06:59.375 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.370+0000 7f178c965640 1 -- 172.21.15.88:0/1062759513 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f178804e9c0 con 0x7f1774004d10 2023-10-16T20:06:59.375 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.370+0000 7f177f7fe640 1 -- 172.21.15.88:0/1062759513 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v46) v1 ==== 97+0+11 (secure 0 0 0) 0x7f177801a020 con 0x7f1774004d10 2023-10-16T20:06:59.376 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:06:59.377 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.374+0000 7f178c965640 1 -- 172.21.15.88:0/1062759513 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f17680664d0 msgr2=0x7f1768068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:59.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.374+0000 7f178c965640 1 --2- 172.21.15.88:0/1062759513 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f17680664d0 0x7f1768068990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1788151660 tx=0x7f177000b040 comp rx=0 tx=0).stop 2023-10-16T20:06:59.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.374+0000 7f178c965640 1 -- 172.21.15.88:0/1062759513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1774004d10 msgr2=0x7f17880aeae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:59.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.374+0000 7f178c965640 1 --2- 172.21.15.88:0/1062759513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1774004d10 0x7f17880aeae0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1778007340 tx=0x7f1778007800 comp rx=0 tx=0).stop 2023-10-16T20:06:59.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.374+0000 7f178c965640 1 -- 172.21.15.88:0/1062759513 shutdown_connections 2023-10-16T20:06:59.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.374+0000 7f178c965640 1 --2- 172.21.15.88:0/1062759513 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f17680664d0 0x7f1768068990 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:59.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.374+0000 7f178c965640 1 --2- 172.21.15.88:0/1062759513 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f178815b5e0 0x7f178815ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:59.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.374+0000 7f178c965640 1 --2- 172.21.15.88:0/1062759513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17880b0840 0x7f178814d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:59.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.374+0000 7f178c965640 1 --2- 172.21.15.88:0/1062759513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1774004d10 0x7f17880aeae0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:59.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.374+0000 7f178c965640 1 -- 172.21.15.88:0/1062759513 >> 172.21.15.88:0/1062759513 conn(0x7f17880cf710 msgr2=0x7f1788158cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:59.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.374+0000 7f178c965640 1 -- 172.21.15.88:0/1062759513 shutdown_connections 2023-10-16T20:06:59.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.374+0000 7f178c965640 1 -- 172.21.15.88:0/1062759513 wait complete. 2023-10-16T20:06:59.396 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T20:06:59.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.530+0000 7f495733e640 1 Processor -- start 2023-10-16T20:06:59.535 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.530+0000 7f495733e640 1 -- start start 2023-10-16T20:06:59.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.534+0000 7f495733e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4950069e90 0x7f495006a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:59.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.534+0000 7f495733e640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f49500cc120 0x7f49500cc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:59.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.534+0000 7f495733e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4950162580 con 0x7f49500afdf0 2023-10-16T20:06:59.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.534+0000 7f495733e640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f495006a860 con 0x7f49500cc120 2023-10-16T20:06:59.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.534+0000 7f495733e640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49500ae830 con 0x7f4950069e90 2023-10-16T20:06:59.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.534+0000 7f49558db640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f49500cc120 0x7f49500cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:59.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.534+0000 7f49558db640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f49500cc120 0x7f49500cc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53334/0 (socket says 172.21.15.88:53334) 2023-10-16T20:06:59.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.534+0000 7f49558db640 1 -- 172.21.15.88:0/2386553612 learned_addr learned my addr 172.21.15.88:0/2386553612 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:59.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.534+0000 7f49548d9640 1 --2- 172.21.15.88:0/2386553612 >> v2:172.21.15.94:3300/0 conn(0x7f4950069e90 0x7f495006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:59.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.534+0000 7f49558db640 1 -- 172.21.15.88:0/2386553612 >> v2:172.21.15.94:3300/0 conn(0x7f4950069e90 msgr2=0x7f495006a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:59.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.534+0000 7f49558db640 1 --2- 172.21.15.88:0/2386553612 >> v2:172.21.15.94:3300/0 conn(0x7f4950069e90 0x7f495006a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:59.539 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.534+0000 7f49558db640 1 -- 172.21.15.88:0/2386553612 >> v1:172.21.15.94:6789/0 conn(0x7f49500afdf0 legacy=0x7f49500aecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:59.539 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.534+0000 7f49558db640 1 -- 172.21.15.88:0/2386553612 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49500adf00 con 0x7f49500cc120 2023-10-16T20:06:59.539 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.538+0000 7f49558db640 1 --2- 172.21.15.88:0/2386553612 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49500cc120 0x7f49500cc570 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f495006e710 tx=0x7f494801c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8d7de534534224 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:59.540 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.538+0000 7f4947fff640 1 -- 172.21.15.88:0/2386553612 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49480a5700 con 0x7f49500cc120 2023-10-16T20:06:59.540 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.538+0000 7f4947fff640 1 -- 172.21.15.88:0/2386553612 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49480a5860 con 0x7f49500cc120 2023-10-16T20:06:59.540 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.538+0000 7f4947fff640 1 -- 172.21.15.88:0/2386553612 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49480a5bf0 con 0x7f49500cc120 2023-10-16T20:06:59.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.538+0000 7f495733e640 1 -- 172.21.15.88:0/2386553612 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49500cc120 msgr2=0x7f49500cc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:59.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.538+0000 7f495733e640 1 --2- 172.21.15.88:0/2386553612 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49500cc120 0x7f49500cc570 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f495006e710 tx=0x7f494801c840 comp rx=0 tx=0).stop 2023-10-16T20:06:59.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.538+0000 7f495733e640 1 -- 172.21.15.88:0/2386553612 shutdown_connections 2023-10-16T20:06:59.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.538+0000 7f495733e640 1 --2- 172.21.15.88:0/2386553612 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49500cc120 0x7f49500cc570 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:59.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.538+0000 7f495733e640 1 --2- 172.21.15.88:0/2386553612 >> v2:172.21.15.94:3300/0 conn(0x7f4950069e90 0x7f495006a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:59.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.538+0000 7f495733e640 1 -- 172.21.15.88:0/2386553612 >> 172.21.15.88:0/2386553612 conn(0x7f49500ce5e0 msgr2=0x7f49500ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:59.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.538+0000 7f495733e640 1 -- 172.21.15.88:0/2386553612 shutdown_connections 2023-10-16T20:06:59.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.538+0000 7f495733e640 1 -- 172.21.15.88:0/2386553612 wait complete. 2023-10-16T20:06:59.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.538+0000 7f495733e640 1 Processor -- start 2023-10-16T20:06:59.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.538+0000 7f495733e640 1 -- start start 2023-10-16T20:06:59.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.538+0000 7f495733e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4950069e90 0x7f4950158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:59.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.538+0000 7f495733e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49500afdf0 0x7f495015c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:59.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.538+0000 7f495733e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49500cc120 0x7f495015c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:59.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.542+0000 7f495733e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49500b34f0 con 0x7f49500afdf0 2023-10-16T20:06:59.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.542+0000 7f495733e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49500b28e0 con 0x7f4950069e90 2023-10-16T20:06:59.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.542+0000 7f495733e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49500b2bc0 con 0x7f49500cc120 2023-10-16T20:06:59.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.542+0000 7f49548d9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49500afdf0 0x7f495015c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:59.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.542+0000 7f49548d9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49500afdf0 0x7f495015c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53338/0 (socket says 172.21.15.88:53338) 2023-10-16T20:06:59.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.542+0000 7f49548d9640 1 -- 172.21.15.88:0/3573765743 learned_addr learned my addr 172.21.15.88:0/3573765743 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:59.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.542+0000 7f49550da640 1 --2- 172.21.15.88:0/3573765743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4950069e90 0x7f4950158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:59.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.542+0000 7f49558db640 1 --2- 172.21.15.88:0/3573765743 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49500cc120 0x7f495015c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:59.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.542+0000 7f49548d9640 1 -- 172.21.15.88:0/3573765743 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49500cc120 msgr2=0x7f495015c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:59.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.542+0000 7f49548d9640 1 --2- 172.21.15.88:0/3573765743 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49500cc120 0x7f495015c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:59.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.542+0000 7f49548d9640 1 -- 172.21.15.88:0/3573765743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4950069e90 msgr2=0x7f4950158c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:59.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.542+0000 7f49548d9640 1 --2- 172.21.15.88:0/3573765743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4950069e90 0x7f4950158c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:59.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.542+0000 7f49548d9640 1 -- 172.21.15.88:0/3573765743 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4948009b30 con 0x7f49500afdf0 2023-10-16T20:06:59.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.542+0000 7f49550da640 1 --2- 172.21.15.88:0/3573765743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4950069e90 0x7f4950158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:06:59.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.542+0000 7f49548d9640 1 --2- 172.21.15.88:0/3573765743 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49500afdf0 0x7f495015c490 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f494000d800 tx=0x7f494000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:59.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.542+0000 7f4945ffb640 1 -- 172.21.15.88:0/3573765743 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494000b6c0 con 0x7f49500afdf0 2023-10-16T20:06:59.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.542+0000 7f4945ffb640 1 -- 172.21.15.88:0/3573765743 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494000b820 con 0x7f49500afdf0 2023-10-16T20:06:59.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.542+0000 7f4945ffb640 1 -- 172.21.15.88:0/3573765743 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494000bb90 con 0x7f49500afdf0 2023-10-16T20:06:59.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.542+0000 7f495733e640 1 -- 172.21.15.88:0/3573765743 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49500b2ea0 con 0x7f49500afdf0 2023-10-16T20:06:59.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.542+0000 7f495733e640 1 -- 172.21.15.88:0/3573765743 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4950159180 con 0x7f49500afdf0 2023-10-16T20:06:59.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.542+0000 7f495733e640 1 -- 172.21.15.88:0/3573765743 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49500cdc30 con 0x7f49500afdf0 2023-10-16T20:06:59.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.542+0000 7f4945ffb640 1 -- 172.21.15.88:0/3573765743 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f494000bcf0 con 0x7f49500afdf0 2023-10-16T20:06:59.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.546+0000 7f4945ffb640 1 --2- 172.21.15.88:0/3573765743 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f49340664a0 0x7f4934068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:59.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.546+0000 7f4945ffb640 1 -- 172.21.15.88:0/3573765743 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f49400846a0 con 0x7f49500afdf0 2023-10-16T20:06:59.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.546+0000 7f49550da640 1 --2- 172.21.15.88:0/3573765743 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f49340664a0 0x7f4934068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:59.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.546+0000 7f49550da640 1 --2- 172.21.15.88:0/3573765743 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f49340664a0 0x7f4934068960 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f494c003ec0 tx=0x7f494c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:59.552 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.550+0000 7f4945ffb640 1 -- 172.21.15.88:0/3573765743 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4940051820 con 0x7f49500afdf0 2023-10-16T20:06:59.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.710+0000 7f495733e640 1 -- 172.21.15.88:0/3573765743 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f49500b3630 con 0x7f49500afdf0 2023-10-16T20:06:59.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.710+0000 7f4945ffb640 1 -- 172.21.15.88:0/3573765743 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v46) v1 ==== 80+0+0 (secure 0 0 0) 0x7f49500b3630 con 0x7f49500afdf0 2023-10-16T20:06:59.713 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2023-10-16T20:06:59.714 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.710+0000 7f495733e640 1 -- 172.21.15.88:0/3573765743 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f49340664a0 msgr2=0x7f4934068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:59.714 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.710+0000 7f495733e640 1 --2- 172.21.15.88:0/3573765743 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f49340664a0 0x7f4934068960 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f494c003ec0 tx=0x7f494c009040 comp rx=0 tx=0).stop 2023-10-16T20:06:59.715 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.710+0000 7f495733e640 1 -- 172.21.15.88:0/3573765743 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49500afdf0 msgr2=0x7f495015c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:59.715 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.710+0000 7f495733e640 1 --2- 172.21.15.88:0/3573765743 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49500afdf0 0x7f495015c490 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f494000d800 tx=0x7f494000dcc0 comp rx=0 tx=0).stop 2023-10-16T20:06:59.715 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.710+0000 7f495733e640 1 -- 172.21.15.88:0/3573765743 shutdown_connections 2023-10-16T20:06:59.715 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.710+0000 7f495733e640 1 --2- 172.21.15.88:0/3573765743 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f49340664a0 0x7f4934068960 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:59.715 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.710+0000 7f495733e640 1 --2- 172.21.15.88:0/3573765743 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49500cc120 0x7f495015c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:59.716 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.710+0000 7f495733e640 1 --2- 172.21.15.88:0/3573765743 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49500afdf0 0x7f495015c490 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:59.716 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.710+0000 7f495733e640 1 --2- 172.21.15.88:0/3573765743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4950069e90 0x7f4950158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:59.716 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.710+0000 7f495733e640 1 -- 172.21.15.88:0/3573765743 >> 172.21.15.88:0/3573765743 conn(0x7f49500ce5e0 msgr2=0x7f49500719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:59.716 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.710+0000 7f495733e640 1 -- 172.21.15.88:0/3573765743 shutdown_connections 2023-10-16T20:06:59.716 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.710+0000 7f495733e640 1 -- 172.21.15.88:0/3573765743 wait complete. 2023-10-16T20:06:59.733 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:06:59.733 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:06:59.743 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:06:59.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.918+0000 7f4226246640 1 Processor -- start 2023-10-16T20:06:59.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.918+0000 7f4226246640 1 -- start start 2023-10-16T20:06:59.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.918+0000 7f4226246640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f422015d9f0 0x7f4220161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:59.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.918+0000 7f4226246640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f42200d26c0 0x7f42200cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:59.922 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.918+0000 7f4226246640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42200b87e0 con 0x7f42200d22e0 2023-10-16T20:06:59.922 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.918+0000 7f4226246640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f422015df10 con 0x7f42200d26c0 2023-10-16T20:06:59.922 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.918+0000 7f4226246640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422015e1f0 con 0x7f422015d9f0 2023-10-16T20:06:59.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.918+0000 7f421effd640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f422015d9f0 0x7f4220161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:59.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.918+0000 7f421effd640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f422015d9f0 0x7f4220161450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:34680/0 (socket says 172.21.15.88:34680) 2023-10-16T20:06:59.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.918+0000 7f421effd640 1 -- 172.21.15.88:0/2857620815 learned_addr learned my addr 172.21.15.88:0/2857620815 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:59.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.918+0000 7f421f7fe640 1 --1- 172.21.15.88:0/2857620815 >> v1:172.21.15.94:6789/0 conn(0x7f42200d22e0 0x7f422015d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:60854/0 (socket says 172.21.15.88:60854) 2023-10-16T20:06:59.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.918+0000 7f421ffff640 1 --2- 172.21.15.88:0/2857620815 >> v2:172.21.15.94:3300/0 conn(0x7f42200d26c0 0x7f42200cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:59.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.918+0000 7f421effd640 1 -- 172.21.15.88:0/2857620815 >> v2:172.21.15.94:3300/0 conn(0x7f42200d26c0 msgr2=0x7f42200cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:06:59.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.918+0000 7f421e7fc640 1 -- 172.21.15.88:0/2857620815 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3539224375 0 0) 0x7f42200b87e0 con 0x7f42200d22e0 2023-10-16T20:06:59.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.918+0000 7f421effd640 1 --2- 172.21.15.88:0/2857620815 >> v2:172.21.15.94:3300/0 conn(0x7f42200d26c0 0x7f42200cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:59.925 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.918+0000 7f421effd640 1 -- 172.21.15.88:0/2857620815 >> v1:172.21.15.94:6789/0 conn(0x7f42200d22e0 legacy=0x7f422015d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:59.925 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.918+0000 7f421effd640 1 -- 172.21.15.88:0/2857620815 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4220160970 con 0x7f422015d9f0 2023-10-16T20:06:59.926 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.922+0000 7f421effd640 1 --2- 172.21.15.88:0/2857620815 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f422015d9f0 0x7f4220161450 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4214009870 tx=0x7f421401c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5038286c6cec8dbf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:59.926 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.922+0000 7f421e7fc640 1 -- 172.21.15.88:0/2857620815 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42140ab680 con 0x7f422015d9f0 2023-10-16T20:06:59.926 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.922+0000 7f421e7fc640 1 -- 172.21.15.88:0/2857620815 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42140ab7e0 con 0x7f422015d9f0 2023-10-16T20:06:59.926 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.922+0000 7f421e7fc640 1 -- 172.21.15.88:0/2857620815 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42140abb70 con 0x7f422015d9f0 2023-10-16T20:06:59.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.922+0000 7f4226246640 1 -- 172.21.15.88:0/2857620815 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f422015d9f0 msgr2=0x7f4220161450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:59.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.922+0000 7f4226246640 1 --2- 172.21.15.88:0/2857620815 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f422015d9f0 0x7f4220161450 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4214009870 tx=0x7f421401c480 comp rx=0 tx=0).stop 2023-10-16T20:06:59.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.922+0000 7f4226246640 1 -- 172.21.15.88:0/2857620815 shutdown_connections 2023-10-16T20:06:59.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.922+0000 7f4226246640 1 --2- 172.21.15.88:0/2857620815 >> v2:172.21.15.94:3300/0 conn(0x7f42200d26c0 0x7f42200cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:59.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.922+0000 7f4226246640 1 --2- 172.21.15.88:0/2857620815 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f422015d9f0 0x7f4220161450 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:59.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.922+0000 7f4226246640 1 -- 172.21.15.88:0/2857620815 >> 172.21.15.88:0/2857620815 conn(0x7f42200715c0 msgr2=0x7f42200719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:06:59.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.922+0000 7f4226246640 1 -- 172.21.15.88:0/2857620815 shutdown_connections 2023-10-16T20:06:59.929 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.922+0000 7f4226246640 1 -- 172.21.15.88:0/2857620815 wait complete. 2023-10-16T20:06:59.931 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.926+0000 7f4226246640 1 Processor -- start 2023-10-16T20:06:59.931 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.926+0000 7f4226246640 1 -- start start 2023-10-16T20:06:59.931 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.926+0000 7f4226246640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42200d22e0 0x7f42201565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:59.931 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.926+0000 7f4226246640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42200d26c0 0x7f422014e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:59.932 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.926+0000 7f4226246640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f422015d9f0 0x7f4220152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:59.932 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.926+0000 7f4226246640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422014f740 con 0x7f42200d26c0 2023-10-16T20:06:59.932 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.926+0000 7f4226246640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422014f180 con 0x7f42200d22e0 2023-10-16T20:06:59.932 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.926+0000 7f4226246640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422014f460 con 0x7f422015d9f0 2023-10-16T20:06:59.932 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.926+0000 7f421effd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42200d26c0 0x7f422014e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:59.933 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.926+0000 7f421effd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42200d26c0 0x7f422014e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53354/0 (socket says 172.21.15.88:53354) 2023-10-16T20:06:59.933 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.926+0000 7f421effd640 1 -- 172.21.15.88:0/2109812663 learned_addr learned my addr 172.21.15.88:0/2109812663 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:06:59.933 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.926+0000 7f421f7fe640 1 --2- 172.21.15.88:0/2109812663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42200d22e0 0x7f42201565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:59.933 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.926+0000 7f421ffff640 1 --2- 172.21.15.88:0/2109812663 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f422015d9f0 0x7f4220152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:59.933 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.926+0000 7f421effd640 1 -- 172.21.15.88:0/2109812663 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f422015d9f0 msgr2=0x7f4220152040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:59.934 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.926+0000 7f421effd640 1 --2- 172.21.15.88:0/2109812663 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f422015d9f0 0x7f4220152040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:59.934 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.926+0000 7f421effd640 1 -- 172.21.15.88:0/2109812663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42200d22e0 msgr2=0x7f42201565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:06:59.934 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.926+0000 7f421effd640 1 --2- 172.21.15.88:0/2109812663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42200d22e0 0x7f42201565e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:06:59.934 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.926+0000 7f421effd640 1 -- 172.21.15.88:0/2109812663 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42140094d0 con 0x7f42200d26c0 2023-10-16T20:06:59.934 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.926+0000 7f421f7fe640 1 --2- 172.21.15.88:0/2109812663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42200d22e0 0x7f42201565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:06:59.935 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.926+0000 7f421effd640 1 --2- 172.21.15.88:0/2109812663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42200d26c0 0x7f422014e830 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f42140ad040 tx=0x7f4214019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:59.935 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.926+0000 7f4207fff640 1 -- 172.21.15.88:0/2109812663 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42140b5810 con 0x7f42200d26c0 2023-10-16T20:06:59.935 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.926+0000 7f4207fff640 1 -- 172.21.15.88:0/2109812663 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42140b59b0 con 0x7f42200d26c0 2023-10-16T20:06:59.935 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.926+0000 7f4207fff640 1 -- 172.21.15.88:0/2109812663 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42140b5d60 con 0x7f42200d26c0 2023-10-16T20:06:59.935 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.926+0000 7f4226246640 1 -- 172.21.15.88:0/2109812663 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4220155de0 con 0x7f42200d26c0 2023-10-16T20:06:59.935 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.926+0000 7f4226246640 1 -- 172.21.15.88:0/2109812663 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4220156b20 con 0x7f42200d26c0 2023-10-16T20:06:59.936 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.930+0000 7f4226246640 1 -- 172.21.15.88:0/2109812663 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4220155c40 con 0x7f42200d26c0 2023-10-16T20:06:59.937 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.930+0000 7f4207fff640 1 -- 172.21.15.88:0/2109812663 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f42140bf410 con 0x7f42200d26c0 2023-10-16T20:06:59.937 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.930+0000 7f4207fff640 1 --2- 172.21.15.88:0/2109812663 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f420c0664a0 0x7f420c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:06:59.937 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.930+0000 7f4207fff640 1 -- 172.21.15.88:0/2109812663 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f421411f1c0 con 0x7f42200d26c0 2023-10-16T20:06:59.937 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.930+0000 7f421f7fe640 1 --2- 172.21.15.88:0/2109812663 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f420c0664a0 0x7f420c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:06:59.937 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.930+0000 7f421f7fe640 1 --2- 172.21.15.88:0/2109812663 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f420c0664a0 0x7f420c068960 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4210000f70 tx=0x7f4210009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:06:59.938 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:06:59.934+0000 7f4207fff640 1 -- 172.21.15.88:0/2109812663 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4220155c40 con 0x7f42200d26c0 2023-10-16T20:07:00.089 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.086+0000 7f4226246640 1 -- 172.21.15.88:0/2109812663 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4220155c40 con 0x7f42200d26c0 2023-10-16T20:07:00.091 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.086+0000 7f4207fff640 1 -- 172.21.15.88:0/2109812663 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+18499 (secure 0 0 0) 0x7f4220155c40 con 0x7f42200d26c0 2023-10-16T20:07:00.091 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:07:00.091 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":46,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:06:38.250351+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T20:06:14.743500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"testcephfs1_data","create_time":"2023-10-16T20:06:16.091837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-10-16T20:06:31.184462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.39","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:07:00.093 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.090+0000 7f4226246640 1 -- 172.21.15.88:0/2109812663 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f420c0664a0 msgr2=0x7f420c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:00.093 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.090+0000 7f4226246640 1 --2- 172.21.15.88:0/2109812663 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f420c0664a0 0x7f420c068960 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4210000f70 tx=0x7f4210009040 comp rx=0 tx=0).stop 2023-10-16T20:07:00.093 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.090+0000 7f4226246640 1 -- 172.21.15.88:0/2109812663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42200d26c0 msgr2=0x7f422014e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:00.093 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.090+0000 7f4226246640 1 --2- 172.21.15.88:0/2109812663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42200d26c0 0x7f422014e830 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f42140ad040 tx=0x7f4214019220 comp rx=0 tx=0).stop 2023-10-16T20:07:00.094 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.090+0000 7f4226246640 1 -- 172.21.15.88:0/2109812663 shutdown_connections 2023-10-16T20:07:00.094 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.090+0000 7f4226246640 1 --2- 172.21.15.88:0/2109812663 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f420c0664a0 0x7f420c068960 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:00.094 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.090+0000 7f4226246640 1 --2- 172.21.15.88:0/2109812663 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f422015d9f0 0x7f4220152040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:00.094 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.090+0000 7f4226246640 1 --2- 172.21.15.88:0/2109812663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42200d26c0 0x7f422014e830 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:00.094 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.090+0000 7f4226246640 1 --2- 172.21.15.88:0/2109812663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42200d22e0 0x7f42201565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:00.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.090+0000 7f4226246640 1 -- 172.21.15.88:0/2109812663 >> 172.21.15.88:0/2109812663 conn(0x7f42200715c0 msgr2=0x7f422015d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:00.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.090+0000 7f4226246640 1 -- 172.21.15.88:0/2109812663 shutdown_connections 2023-10-16T20:07:00.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.090+0000 7f4226246640 1 -- 172.21.15.88:0/2109812663 wait complete. 2023-10-16T20:07:00.112 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'testcephfs1_metadata', 'create_time': '2023-10-16T20:06:14.743500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'testcephfs1_data', 'create_time': '2023-10-16T20:06:16.091837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2023-10-16T20:06:31.184462+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T20:07:00.113 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:07:00.254 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.250+0000 7f63c1646640 1 Processor -- start 2023-10-16T20:07:00.254 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.250+0000 7f63c1646640 1 -- start start 2023-10-16T20:07:00.256 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.250+0000 7f63c1646640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63bc15cb70 0x7f63bc0b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:00.256 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.250+0000 7f63c1646640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f63bc0b4530 0x7f63bc0b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:00.256 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.250+0000 7f63c1646640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63bc0b4ed0 con 0x7f63bc0d0bd0 2023-10-16T20:07:00.256 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.250+0000 7f63c1646640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63bc0b7d20 con 0x7f63bc0b4530 2023-10-16T20:07:00.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.250+0000 7f63c1646640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc0b7110 con 0x7f63bc15cb70 2023-10-16T20:07:00.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.250+0000 7f63bb7fe640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f63bc0d0bd0 0x7f63bc0d1060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:57330/0 (socket says 172.21.15.88:57330) 2023-10-16T20:07:00.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.250+0000 7f63bb7fe640 1 -- 172.21.15.88:0/2869832562 learned_addr learned my addr 172.21.15.88:0/2869832562 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:00.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.250+0000 7f63b27fc640 1 --2- 172.21.15.88:0/2869832562 >> v2:172.21.15.94:3300/0 conn(0x7f63bc0b4530 0x7f63bc0b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:00.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.250+0000 7f63baffd640 1 --2- 172.21.15.88:0/2869832562 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63bc15cb70 0x7f63bc0b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:00.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.254+0000 7f63ba7fc640 1 -- 172.21.15.88:0/2869832562 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 952357446 0 0) 0x7f63bc0b4ed0 con 0x7f63bc0d0bd0 2023-10-16T20:07:00.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.254+0000 7f63ba7fc640 1 -- 172.21.15.88:0/2869832562 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63a8003530 con 0x7f63bc0d0bd0 2023-10-16T20:07:00.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.254+0000 7f63b27fc640 1 -- 172.21.15.88:0/2869832562 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63bc15cb70 msgr2=0x7f63bc0b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:00.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.254+0000 7f63b27fc640 1 --2- 172.21.15.88:0/2869832562 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63bc15cb70 0x7f63bc0b83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:00.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.254+0000 7f63b27fc640 1 -- 172.21.15.88:0/2869832562 >> v1:172.21.15.88:6789/0 conn(0x7f63bc0d0bd0 legacy=0x7f63bc0d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:00.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.254+0000 7f63b27fc640 1 -- 172.21.15.88:0/2869832562 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63bc0b73f0 con 0x7f63bc0b4530 2023-10-16T20:07:00.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.254+0000 7f63baffd640 1 --2- 172.21.15.88:0/2869832562 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63bc15cb70 0x7f63bc0b83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:07:00.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.254+0000 7f63b27fc640 1 --2- 172.21.15.88:0/2869832562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63bc0b4530 0x7f63bc0b4990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f63a40097c0 tx=0x7f63a401ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c554bb8866782836 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:00.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.254+0000 7f63ba7fc640 1 -- 172.21.15.88:0/2869832562 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a40a8970 con 0x7f63bc0b4530 2023-10-16T20:07:00.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.254+0000 7f63ba7fc640 1 -- 172.21.15.88:0/2869832562 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a40a8ad0 con 0x7f63bc0b4530 2023-10-16T20:07:00.261 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.254+0000 7f63ba7fc640 1 -- 172.21.15.88:0/2869832562 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a40a8e40 con 0x7f63bc0b4530 2023-10-16T20:07:00.261 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.258+0000 7f63c1646640 1 -- 172.21.15.88:0/2869832562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63bc0b4530 msgr2=0x7f63bc0b4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:00.261 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.258+0000 7f63c1646640 1 --2- 172.21.15.88:0/2869832562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63bc0b4530 0x7f63bc0b4990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f63a40097c0 tx=0x7f63a401ec70 comp rx=0 tx=0).stop 2023-10-16T20:07:00.261 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.258+0000 7f63c1646640 1 -- 172.21.15.88:0/2869832562 shutdown_connections 2023-10-16T20:07:00.261 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.258+0000 7f63c1646640 1 --2- 172.21.15.88:0/2869832562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63bc0b4530 0x7f63bc0b4990 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:00.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.258+0000 7f63c1646640 1 --2- 172.21.15.88:0/2869832562 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63bc15cb70 0x7f63bc0b83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:00.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.258+0000 7f63c1646640 1 -- 172.21.15.88:0/2869832562 >> 172.21.15.88:0/2869832562 conn(0x7f63bc06b6e0 msgr2=0x7f63bc06baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:00.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.258+0000 7f63c1646640 1 -- 172.21.15.88:0/2869832562 shutdown_connections 2023-10-16T20:07:00.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.258+0000 7f63c1646640 1 -- 172.21.15.88:0/2869832562 wait complete. 2023-10-16T20:07:00.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.258+0000 7f63c1646640 1 Processor -- start 2023-10-16T20:07:00.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.258+0000 7f63c1646640 1 -- start start 2023-10-16T20:07:00.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.258+0000 7f63c1646640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63bc0b4530 0x7f63bc14cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:00.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.258+0000 7f63c1646640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63bc0d0bd0 0x7f63bc0c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:00.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.258+0000 7f63c1646640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63bc15cb70 0x7f63bc158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:00.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.258+0000 7f63c1646640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc14d160 con 0x7f63bc0b4530 2023-10-16T20:07:00.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.258+0000 7f63c1646640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc1f3020 con 0x7f63bc15cb70 2023-10-16T20:07:00.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.258+0000 7f63c1646640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc0c92d0 con 0x7f63bc0d0bd0 2023-10-16T20:07:00.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.258+0000 7f63baffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63bc0b4530 0x7f63bc14cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:00.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.258+0000 7f63baffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63bc0b4530 0x7f63bc14cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53358/0 (socket says 172.21.15.88:53358) 2023-10-16T20:07:00.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.258+0000 7f63baffd640 1 -- 172.21.15.88:0/68264716 learned_addr learned my addr 172.21.15.88:0/68264716 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:00.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.258+0000 7f63b27fc640 1 --2- 172.21.15.88:0/68264716 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63bc0d0bd0 0x7f63bc0c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:00.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.258+0000 7f63bb7fe640 1 --2- 172.21.15.88:0/68264716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63bc15cb70 0x7f63bc158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:00.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.258+0000 7f63baffd640 1 -- 172.21.15.88:0/68264716 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63bc0d0bd0 msgr2=0x7f63bc0c89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:00.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.258+0000 7f63baffd640 1 --2- 172.21.15.88:0/68264716 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63bc0d0bd0 0x7f63bc0c89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:00.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.258+0000 7f63baffd640 1 -- 172.21.15.88:0/68264716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63bc15cb70 msgr2=0x7f63bc158a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:00.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.258+0000 7f63baffd640 1 --2- 172.21.15.88:0/68264716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63bc15cb70 0x7f63bc158a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:00.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.262+0000 7f63baffd640 1 -- 172.21.15.88:0/68264716 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ac009a70 con 0x7f63bc0b4530 2023-10-16T20:07:00.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.262+0000 7f63baffd640 1 --2- 172.21.15.88:0/68264716 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63bc0b4530 0x7f63bc14cc20 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f63ac00efc0 tx=0x7f63ac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:00.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.262+0000 7f63b3fff640 1 -- 172.21.15.88:0/68264716 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac014680 con 0x7f63bc0b4530 2023-10-16T20:07:00.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.262+0000 7f63b3fff640 1 -- 172.21.15.88:0/68264716 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63ac00f040 con 0x7f63bc0b4530 2023-10-16T20:07:00.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.262+0000 7f63b3fff640 1 -- 172.21.15.88:0/68264716 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac014a00 con 0x7f63bc0b4530 2023-10-16T20:07:00.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.262+0000 7f63c1646640 1 -- 172.21.15.88:0/68264716 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a4009420 con 0x7f63bc0b4530 2023-10-16T20:07:00.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.262+0000 7f63c1646640 1 -- 172.21.15.88:0/68264716 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63bc1f3570 con 0x7f63bc0b4530 2023-10-16T20:07:00.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.262+0000 7f63c1646640 1 -- 172.21.15.88:0/68264716 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63bc04e9c0 con 0x7f63bc0b4530 2023-10-16T20:07:00.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.262+0000 7f63b3fff640 1 -- 172.21.15.88:0/68264716 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f63ac014b60 con 0x7f63bc0b4530 2023-10-16T20:07:00.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.262+0000 7f63b3fff640 1 --2- 172.21.15.88:0/68264716 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f639c0664a0 0x7f639c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:00.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.262+0000 7f63b3fff640 1 -- 172.21.15.88:0/68264716 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f63ac085080 con 0x7f63bc0b4530 2023-10-16T20:07:00.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.262+0000 7f63b27fc640 1 --2- 172.21.15.88:0/68264716 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f639c0664a0 0x7f639c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:00.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.266+0000 7f63b27fc640 1 --2- 172.21.15.88:0/68264716 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f639c0664a0 0x7f639c068960 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f63a40b2000 tx=0x7f63a40aa070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:00.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.266+0000 7f63b3fff640 1 -- 172.21.15.88:0/68264716 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f63ac0b6090 con 0x7f63bc0b4530 2023-10-16T20:07:00.421 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.418+0000 7f63c1646640 1 -- 172.21.15.88:0/68264716 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f63bc157b30 con 0x7f63bc0b4530 2023-10-16T20:07:00.422 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.418+0000 7f63b3fff640 1 -- 172.21.15.88:0/68264716 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f63ac052190 con 0x7f63bc0b4530 2023-10-16T20:07:00.422 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:07:00.424 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.422+0000 7f63c1646640 1 -- 172.21.15.88:0/68264716 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f639c0664a0 msgr2=0x7f639c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:00.424 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.422+0000 7f63c1646640 1 --2- 172.21.15.88:0/68264716 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f639c0664a0 0x7f639c068960 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f63a40b2000 tx=0x7f63a40aa070 comp rx=0 tx=0).stop 2023-10-16T20:07:00.424 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.422+0000 7f63c1646640 1 -- 172.21.15.88:0/68264716 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63bc0b4530 msgr2=0x7f63bc14cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:00.424 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.422+0000 7f63c1646640 1 --2- 172.21.15.88:0/68264716 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63bc0b4530 0x7f63bc14cc20 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f63ac00efc0 tx=0x7f63ac00c490 comp rx=0 tx=0).stop 2023-10-16T20:07:00.425 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.422+0000 7f63c1646640 1 -- 172.21.15.88:0/68264716 shutdown_connections 2023-10-16T20:07:00.425 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.422+0000 7f63c1646640 1 --2- 172.21.15.88:0/68264716 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f639c0664a0 0x7f639c068960 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:00.425 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.422+0000 7f63c1646640 1 --2- 172.21.15.88:0/68264716 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63bc0d0bd0 0x7f63bc0c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:00.425 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.422+0000 7f63c1646640 1 --2- 172.21.15.88:0/68264716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63bc15cb70 0x7f63bc158a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:00.425 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.422+0000 7f63c1646640 1 --2- 172.21.15.88:0/68264716 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63bc0b4530 0x7f63bc14cc20 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:00.425 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.422+0000 7f63c1646640 1 -- 172.21.15.88:0/68264716 >> 172.21.15.88:0/68264716 conn(0x7f63bc06b6e0 msgr2=0x7f63bc15bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:00.426 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.422+0000 7f63c1646640 1 -- 172.21.15.88:0/68264716 shutdown_connections 2023-10-16T20:07:00.426 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.422+0000 7f63c1646640 1 -- 172.21.15.88:0/68264716 wait complete. 2023-10-16T20:07:00.443 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_metadata pg_num 2023-10-16T20:07:00.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.582+0000 7fb50de46640 1 Processor -- start 2023-10-16T20:07:00.584 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.582+0000 7fb50de46640 1 -- start start 2023-10-16T20:07:00.585 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.582+0000 7fb50de46640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb50815cac0 0x7fb5080ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:00.586 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.582+0000 7fb50de46640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb50815cea0 0x7fb5080cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:00.586 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.582+0000 7fb50de46640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb50806b840 con 0x7fb5080ae130 2023-10-16T20:07:00.586 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.582+0000 7fb50de46640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5080cac10 con 0x7fb50815cac0 2023-10-16T20:07:00.586 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.582+0000 7fb50de46640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50806a3a0 con 0x7fb50815cea0 2023-10-16T20:07:00.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.582+0000 7fb5077fe640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb50815cac0 0x7fb5080ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:00.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.582+0000 7fb507fff640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fb5080ae130 0x7fb5081607d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:57334/0 (socket says 172.21.15.88:57334) 2023-10-16T20:07:00.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.582+0000 7fb507fff640 1 -- 172.21.15.88:0/2690102882 learned_addr learned my addr 172.21.15.88:0/2690102882 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:00.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.582+0000 7fb506ffd640 1 --2- 172.21.15.88:0/2690102882 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb50815cea0 0x7fb5080cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:00.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.582+0000 7fb5077fe640 1 -- 172.21.15.88:0/2690102882 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb50815cea0 msgr2=0x7fb5080cd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:00.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.582+0000 7fb5077fe640 1 --2- 172.21.15.88:0/2690102882 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb50815cea0 0x7fb5080cd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:00.588 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.582+0000 7fb5077fe640 1 -- 172.21.15.88:0/2690102882 >> v1:172.21.15.88:6789/0 conn(0x7fb5080ae130 legacy=0x7fb5081607d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:00.588 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.582+0000 7fb5077fe640 1 -- 172.21.15.88:0/2690102882 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5080ca650 con 0x7fb50815cac0 2023-10-16T20:07:00.588 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.582+0000 7fb5067fc640 1 -- 172.21.15.88:0/2690102882 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3336228506 0 0) 0x7fb50806b840 con 0x7fb5080ae130 2023-10-16T20:07:00.589 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.586+0000 7fb5077fe640 1 --2- 172.21.15.88:0/2690102882 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb50815cac0 0x7fb5080ccc60 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb4f80097c0 tx=0x7fb4f801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23739f54e9f4f396 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:00.590 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.586+0000 7fb5067fc640 1 -- 172.21.15.88:0/2690102882 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f80a8970 con 0x7fb50815cac0 2023-10-16T20:07:00.590 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.586+0000 7fb5067fc640 1 -- 172.21.15.88:0/2690102882 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f80a8ad0 con 0x7fb50815cac0 2023-10-16T20:07:00.590 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.586+0000 7fb5067fc640 1 -- 172.21.15.88:0/2690102882 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f80a8e60 con 0x7fb50815cac0 2023-10-16T20:07:00.590 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.586+0000 7fb50de46640 1 -- 172.21.15.88:0/2690102882 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb50815cac0 msgr2=0x7fb5080ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:00.591 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.586+0000 7fb50de46640 1 --2- 172.21.15.88:0/2690102882 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb50815cac0 0x7fb5080ccc60 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb4f80097c0 tx=0x7fb4f801ec70 comp rx=0 tx=0).stop 2023-10-16T20:07:00.591 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.586+0000 7fb50de46640 1 -- 172.21.15.88:0/2690102882 shutdown_connections 2023-10-16T20:07:00.591 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.586+0000 7fb50de46640 1 --2- 172.21.15.88:0/2690102882 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb50815cea0 0x7fb5080cd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:00.591 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.586+0000 7fb50de46640 1 --2- 172.21.15.88:0/2690102882 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb50815cac0 0x7fb5080ccc60 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:00.591 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.586+0000 7fb50de46640 1 -- 172.21.15.88:0/2690102882 >> 172.21.15.88:0/2690102882 conn(0x7fb5080b0970 msgr2=0x7fb5080b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:00.592 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.586+0000 7fb50de46640 1 -- 172.21.15.88:0/2690102882 shutdown_connections 2023-10-16T20:07:00.592 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.586+0000 7fb50de46640 1 -- 172.21.15.88:0/2690102882 wait complete. 2023-10-16T20:07:00.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.586+0000 7fb50de46640 1 Processor -- start 2023-10-16T20:07:00.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.586+0000 7fb50de46640 1 -- start start 2023-10-16T20:07:00.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.590+0000 7fb50de46640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb5080ae130 0x7fb5080b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:00.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.590+0000 7fb50de46640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb50815cac0 0x7fb5080b4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:00.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.590+0000 7fb50de46640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb50815cea0 0x7fb508145e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:00.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.590+0000 7fb50de46640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5080b3290 con 0x7fb50815cea0 2023-10-16T20:07:00.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.590+0000 7fb50de46640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5080b4a50 con 0x7fb50815cac0 2023-10-16T20:07:00.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.590+0000 7fb50de46640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5080b4080 con 0x7fb5080ae130 2023-10-16T20:07:00.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.590+0000 7fb507fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb50815cea0 0x7fb508145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:00.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.590+0000 7fb507fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb50815cea0 0x7fb508145e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53376/0 (socket says 172.21.15.88:53376) 2023-10-16T20:07:00.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.590+0000 7fb507fff640 1 -- 172.21.15.88:0/2734560251 learned_addr learned my addr 172.21.15.88:0/2734560251 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:00.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.590+0000 7fb506ffd640 1 --2- 172.21.15.88:0/2734560251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb50815cac0 0x7fb5080b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:00.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.590+0000 7fb5077fe640 1 --2- 172.21.15.88:0/2734560251 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb5080ae130 0x7fb5080b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:00.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.590+0000 7fb507fff640 1 -- 172.21.15.88:0/2734560251 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb5080ae130 msgr2=0x7fb5080b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:00.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.590+0000 7fb507fff640 1 --2- 172.21.15.88:0/2734560251 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb5080ae130 0x7fb5080b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:00.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.590+0000 7fb507fff640 1 -- 172.21.15.88:0/2734560251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb50815cac0 msgr2=0x7fb5080b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:00.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.590+0000 7fb507fff640 1 --2- 172.21.15.88:0/2734560251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb50815cac0 0x7fb5080b4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:00.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.590+0000 7fb507fff640 1 -- 172.21.15.88:0/2734560251 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f8009420 con 0x7fb50815cea0 2023-10-16T20:07:00.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.590+0000 7fb506ffd640 1 --2- 172.21.15.88:0/2734560251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb50815cac0 0x7fb5080b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:07:00.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.590+0000 7fb507fff640 1 --2- 172.21.15.88:0/2734560251 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb50815cea0 0x7fb508145e10 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb50000c810 tx=0x7fb50000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:00.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.590+0000 7fb4effff640 1 -- 172.21.15.88:0/2734560251 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50000a640 con 0x7fb50815cea0 2023-10-16T20:07:00.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.590+0000 7fb4effff640 1 -- 172.21.15.88:0/2734560251 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50000a7c0 con 0x7fb50815cea0 2023-10-16T20:07:00.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.590+0000 7fb50de46640 1 -- 172.21.15.88:0/2734560251 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb508146350 con 0x7fb50815cea0 2023-10-16T20:07:00.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.590+0000 7fb4effff640 1 -- 172.21.15.88:0/2734560251 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50000ab70 con 0x7fb50815cea0 2023-10-16T20:07:00.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.590+0000 7fb50de46640 1 -- 172.21.15.88:0/2734560251 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5080b3750 con 0x7fb50815cea0 2023-10-16T20:07:00.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.590+0000 7fb50de46640 1 -- 172.21.15.88:0/2734560251 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5080b41f0 con 0x7fb50815cea0 2023-10-16T20:07:00.600 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.594+0000 7fb4effff640 1 -- 172.21.15.88:0/2734560251 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb50000acd0 con 0x7fb50815cea0 2023-10-16T20:07:00.601 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.594+0000 7fb4effff640 1 --2- 172.21.15.88:0/2734560251 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb4f40664a0 0x7fb4f4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:00.601 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.594+0000 7fb4effff640 1 -- 172.21.15.88:0/2734560251 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7fb500083a80 con 0x7fb50815cea0 2023-10-16T20:07:00.601 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.594+0000 7fb5077fe640 1 --2- 172.21.15.88:0/2734560251 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb4f40664a0 0x7fb4f4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:00.601 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.594+0000 7fb5077fe640 1 --2- 172.21.15.88:0/2734560251 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb4f40664a0 0x7fb4f4068960 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb4f80b2000 tx=0x7fb4f8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:00.601 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.598+0000 7fb4effff640 1 -- 172.21.15.88:0/2734560251 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb500050b60 con 0x7fb50815cea0 2023-10-16T20:07:00.747 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.742+0000 7fb50de46640 1 -- 172.21.15.88:0/2734560251 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb50804e9c0 con 0x7fb50815cea0 2023-10-16T20:07:00.748 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.746+0000 7fb4effff640 1 -- 172.21.15.88:0/2734560251 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"}]=0 v46) v1 ==== 109+0+11 (secure 0 0 0) 0x7fb500018020 con 0x7fb50815cea0 2023-10-16T20:07:00.748 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:07:00.749 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.746+0000 7fb50de46640 1 -- 172.21.15.88:0/2734560251 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb4f40664a0 msgr2=0x7fb4f4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:00.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.746+0000 7fb50de46640 1 --2- 172.21.15.88:0/2734560251 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb4f40664a0 0x7fb4f4068960 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb4f80b2000 tx=0x7fb4f8008040 comp rx=0 tx=0).stop 2023-10-16T20:07:00.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.746+0000 7fb50de46640 1 -- 172.21.15.88:0/2734560251 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb50815cea0 msgr2=0x7fb508145e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:00.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.746+0000 7fb50de46640 1 --2- 172.21.15.88:0/2734560251 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb50815cea0 0x7fb508145e10 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb50000c810 tx=0x7fb50000ccd0 comp rx=0 tx=0).stop 2023-10-16T20:07:00.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.746+0000 7fb50de46640 1 -- 172.21.15.88:0/2734560251 shutdown_connections 2023-10-16T20:07:00.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.746+0000 7fb50de46640 1 --2- 172.21.15.88:0/2734560251 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb4f40664a0 0x7fb4f4068960 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:00.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.746+0000 7fb50de46640 1 --2- 172.21.15.88:0/2734560251 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb50815cea0 0x7fb508145e10 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:00.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.746+0000 7fb50de46640 1 --2- 172.21.15.88:0/2734560251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb50815cac0 0x7fb5080b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:00.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.746+0000 7fb50de46640 1 --2- 172.21.15.88:0/2734560251 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb5080ae130 0x7fb5080b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:00.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.746+0000 7fb50de46640 1 -- 172.21.15.88:0/2734560251 >> 172.21.15.88:0/2734560251 conn(0x7fb5080b0970 msgr2=0x7fb508065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:00.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.746+0000 7fb50de46640 1 -- 172.21.15.88:0/2734560251 shutdown_connections 2023-10-16T20:07:00.752 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.746+0000 7fb50de46640 1 -- 172.21.15.88:0/2734560251 wait complete. 2023-10-16T20:07:00.769 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_data pg_num 2023-10-16T20:07:00.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.906+0000 7f09b491d640 1 Processor -- start 2023-10-16T20:07:00.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.906+0000 7f09b491d640 1 -- start start 2023-10-16T20:07:00.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.906+0000 7f09b491d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f09b015b700 0x7f09b01653f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:00.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.906+0000 7f09b491d640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09b006d2f0 con 0x7f09b00b0c10 2023-10-16T20:07:00.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.906+0000 7f09b491d640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09b015bd20 con 0x7f09b0154f30 2023-10-16T20:07:00.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.906+0000 7f09b491d640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09b0155310 con 0x7f09b015b700 2023-10-16T20:07:00.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.906+0000 7f09add9b640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f09b00b0c10 0x7f09b015aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:57344/0 (socket says 172.21.15.88:57344) 2023-10-16T20:07:00.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.906+0000 7f09add9b640 1 -- 172.21.15.88:0/3080907063 learned_addr learned my addr 172.21.15.88:0/3080907063 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:00.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.906+0000 7f09aed9d640 1 --2- 172.21.15.88:0/3080907063 >> v2:172.21.15.94:3300/0 conn(0x7f09b015b700 0x7f09b01653f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:00.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.906+0000 7f09ad59a640 1 -- 172.21.15.88:0/3080907063 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2962752967 0 0) 0x7f09b006d2f0 con 0x7f09b00b0c10 2023-10-16T20:07:00.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.906+0000 7f09ad59a640 1 -- 172.21.15.88:0/3080907063 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0998003530 con 0x7f09b00b0c10 2023-10-16T20:07:00.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.906+0000 7f09ad59a640 1 -- 172.21.15.88:0/3080907063 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1888506034 0 0) 0x7f09b015bd20 con 0x7f09b0154f30 2023-10-16T20:07:00.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.906+0000 7f09ad59a640 1 -- 172.21.15.88:0/3080907063 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09b006d2f0 con 0x7f09b0154f30 2023-10-16T20:07:00.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.906+0000 7f09aed9d640 1 -- 172.21.15.88:0/3080907063 >> v1:172.21.15.94:6789/0 conn(0x7f09b0154f30 legacy=0x7f09b00b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:00.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.906+0000 7f09aed9d640 1 -- 172.21.15.88:0/3080907063 >> v1:172.21.15.88:6789/0 conn(0x7f09b00b0c10 legacy=0x7f09b015aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:00.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.906+0000 7f09ad59a640 1 -- 172.21.15.88:0/3080907063 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3355961115 0 0) 0x7f0998003530 con 0x7f09b00b0c10 2023-10-16T20:07:00.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.906+0000 7f09aed9d640 1 -- 172.21.15.88:0/3080907063 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b00702f0 con 0x7f09b015b700 2023-10-16T20:07:00.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.910+0000 7f09aed9d640 1 --2- 172.21.15.88:0/3080907063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09b015b700 0x7f09b01653f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f09a8009d90 tx=0x7f09a801c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc9b16f081951562 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:00.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.910+0000 7f09ad59a640 1 -- 172.21.15.88:0/3080907063 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a4001680 con 0x7f09b015b700 2023-10-16T20:07:00.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.910+0000 7f09ad59a640 1 -- 172.21.15.88:0/3080907063 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a80a5680 con 0x7f09b015b700 2023-10-16T20:07:00.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.910+0000 7f09ad59a640 1 -- 172.21.15.88:0/3080907063 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a80a5a10 con 0x7f09b015b700 2023-10-16T20:07:00.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.910+0000 7f09b491d640 1 -- 172.21.15.88:0/3080907063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09b015b700 msgr2=0x7f09b01653f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:00.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.910+0000 7f09b491d640 1 --2- 172.21.15.88:0/3080907063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09b015b700 0x7f09b01653f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f09a8009d90 tx=0x7f09a801c840 comp rx=0 tx=0).stop 2023-10-16T20:07:00.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.910+0000 7f09b491d640 1 -- 172.21.15.88:0/3080907063 shutdown_connections 2023-10-16T20:07:00.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.910+0000 7f09b491d640 1 --2- 172.21.15.88:0/3080907063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09b015b700 0x7f09b01653f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:00.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.910+0000 7f09b491d640 1 -- 172.21.15.88:0/3080907063 >> 172.21.15.88:0/3080907063 conn(0x7f09b00cb080 msgr2=0x7f09b00cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:00.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.910+0000 7f09b491d640 1 -- 172.21.15.88:0/3080907063 shutdown_connections 2023-10-16T20:07:00.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.910+0000 7f09b491d640 1 -- 172.21.15.88:0/3080907063 wait complete. 2023-10-16T20:07:00.919 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.914+0000 7f09b491d640 1 Processor -- start 2023-10-16T20:07:00.919 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.914+0000 7f09b491d640 1 -- start start 2023-10-16T20:07:00.919 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.914+0000 7f09b491d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f09b00b0c10 0x7f09b0135650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:00.919 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.914+0000 7f09b491d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09b0154f30 0x7f09b015a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:00.919 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.914+0000 7f09b491d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09b015b700 0x7f09b0146140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:00.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.914+0000 7f09b491d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b0132fc0 con 0x7f09b0154f30 2023-10-16T20:07:00.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.914+0000 7f09b491d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b0135b90 con 0x7f09b015b700 2023-10-16T20:07:00.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.914+0000 7f09b491d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b0132c90 con 0x7f09b00b0c10 2023-10-16T20:07:00.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.914+0000 7f09add9b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09b0154f30 0x7f09b015a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:00.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.914+0000 7f09add9b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09b0154f30 0x7f09b015a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53386/0 (socket says 172.21.15.88:53386) 2023-10-16T20:07:00.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.914+0000 7f09add9b640 1 -- 172.21.15.88:0/2618660186 learned_addr learned my addr 172.21.15.88:0/2618660186 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:00.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.914+0000 7f09aed9d640 1 --2- 172.21.15.88:0/2618660186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09b015b700 0x7f09b0146140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:00.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.914+0000 7f09ae59c640 1 --2- 172.21.15.88:0/2618660186 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f09b00b0c10 0x7f09b0135650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:00.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.914+0000 7f09add9b640 1 -- 172.21.15.88:0/2618660186 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f09b00b0c10 msgr2=0x7f09b0135650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:00.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.914+0000 7f09add9b640 1 --2- 172.21.15.88:0/2618660186 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f09b00b0c10 0x7f09b0135650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:00.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.914+0000 7f09add9b640 1 -- 172.21.15.88:0/2618660186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09b015b700 msgr2=0x7f09b0146140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:00.922 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.914+0000 7f09add9b640 1 --2- 172.21.15.88:0/2618660186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09b015b700 0x7f09b0146140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:00.922 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.914+0000 7f09add9b640 1 -- 172.21.15.88:0/2618660186 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a8009a70 con 0x7f09b0154f30 2023-10-16T20:07:00.922 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.914+0000 7f09add9b640 1 --2- 172.21.15.88:0/2618660186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09b0154f30 0x7f09b015a090 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f09a4005890 tx=0x7f09a400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:00.922 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.914+0000 7f099effd640 1 -- 172.21.15.88:0/2618660186 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a4005ae0 con 0x7f09b0154f30 2023-10-16T20:07:00.922 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.914+0000 7f099effd640 1 -- 172.21.15.88:0/2618660186 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a4005c40 con 0x7f09b0154f30 2023-10-16T20:07:00.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.914+0000 7f09b491d640 1 -- 172.21.15.88:0/2618660186 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b014fe20 con 0x7f09b0154f30 2023-10-16T20:07:00.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.914+0000 7f099effd640 1 -- 172.21.15.88:0/2618660186 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a4007070 con 0x7f09b0154f30 2023-10-16T20:07:00.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.914+0000 7f09b491d640 1 -- 172.21.15.88:0/2618660186 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b01503e0 con 0x7f09b0154f30 2023-10-16T20:07:00.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.918+0000 7f09b491d640 1 -- 172.21.15.88:0/2618660186 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b004e9c0 con 0x7f09b0154f30 2023-10-16T20:07:00.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.918+0000 7f099effd640 1 -- 172.21.15.88:0/2618660186 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f09a40071d0 con 0x7f09b0154f30 2023-10-16T20:07:00.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.918+0000 7f099effd640 1 --2- 172.21.15.88:0/2618660186 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f09900664d0 0x7f0990068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:00.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.918+0000 7f099effd640 1 -- 172.21.15.88:0/2618660186 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f09a40838e0 con 0x7f09b0154f30 2023-10-16T20:07:00.925 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.918+0000 7f09ae59c640 1 --2- 172.21.15.88:0/2618660186 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f09900664d0 0x7f0990068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:00.925 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.918+0000 7f09ae59c640 1 --2- 172.21.15.88:0/2618660186 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f09900664d0 0x7f0990068990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f09a00011f0 tx=0x7f09a0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:00.925 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:00.922+0000 7f099effd640 1 -- 172.21.15.88:0/2618660186 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f09a40509e0 con 0x7f09b0154f30 2023-10-16T20:07:01.074 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.070+0000 7f09b491d640 1 -- 172.21.15.88:0/2618660186 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"} v 0) v1 -- 0x7f09b00ca5f0 con 0x7f09b0154f30 2023-10-16T20:07:01.074 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.070+0000 7f099effd640 1 -- 172.21.15.88:0/2618660186 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"}]=0 v46) v1 ==== 105+0+11 (secure 0 0 0) 0x7f09a4017020 con 0x7f09b0154f30 2023-10-16T20:07:01.075 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2023-10-16T20:07:01.076 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.074+0000 7f09b491d640 1 -- 172.21.15.88:0/2618660186 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f09900664d0 msgr2=0x7f0990068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:01.076 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.074+0000 7f09b491d640 1 --2- 172.21.15.88:0/2618660186 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f09900664d0 0x7f0990068990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f09a00011f0 tx=0x7f09a0009040 comp rx=0 tx=0).stop 2023-10-16T20:07:01.077 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.074+0000 7f09b491d640 1 -- 172.21.15.88:0/2618660186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09b0154f30 msgr2=0x7f09b015a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:01.077 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.074+0000 7f09b491d640 1 --2- 172.21.15.88:0/2618660186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09b0154f30 0x7f09b015a090 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f09a4005890 tx=0x7f09a400cbe0 comp rx=0 tx=0).stop 2023-10-16T20:07:01.077 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.074+0000 7f09b491d640 1 -- 172.21.15.88:0/2618660186 shutdown_connections 2023-10-16T20:07:01.077 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.074+0000 7f09b491d640 1 --2- 172.21.15.88:0/2618660186 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f09900664d0 0x7f0990068990 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:01.077 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.074+0000 7f09b491d640 1 --2- 172.21.15.88:0/2618660186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09b015b700 0x7f09b0146140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:01.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.074+0000 7f09b491d640 1 --2- 172.21.15.88:0/2618660186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09b0154f30 0x7f09b015a090 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:01.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.074+0000 7f09b491d640 1 --2- 172.21.15.88:0/2618660186 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f09b00b0c10 0x7f09b0135650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:01.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.074+0000 7f09b491d640 1 -- 172.21.15.88:0/2618660186 >> 172.21.15.88:0/2618660186 conn(0x7f09b00cb080 msgr2=0x7f09b00cc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:01.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.074+0000 7f09b491d640 1 -- 172.21.15.88:0/2618660186 shutdown_connections 2023-10-16T20:07:01.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.074+0000 7f09b491d640 1 -- 172.21.15.88:0/2618660186 wait complete. 2023-10-16T20:07:01.095 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-10-16T20:07:01.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.234+0000 7fef780ea640 1 Processor -- start 2023-10-16T20:07:01.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.234+0000 7fef780ea640 1 -- start start 2023-10-16T20:07:01.238 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.234+0000 7fef780ea640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef700c9de0 0x7fef70065a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:01.238 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.234+0000 7fef780ea640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fef7015ca70 0x7fef7015ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:01.238 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.234+0000 7fef780ea640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fef7006d290 0x7fef7006d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:01.238 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.234+0000 7fef780ea640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef7015d410 con 0x7fef7006d290 2023-10-16T20:07:01.239 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.234+0000 7fef780ea640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef700cb1b0 con 0x7fef7015ca70 2023-10-16T20:07:01.239 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.234+0000 7fef780ea640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef700ca630 con 0x7fef700c9de0 2023-10-16T20:07:01.239 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.234+0000 7fef76687640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fef7006d290 0x7fef7006d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:01.239 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.234+0000 7fef76687640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fef7006d290 0x7fef7006d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53400/0 (socket says 172.21.15.88:53400) 2023-10-16T20:07:01.239 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.234+0000 7fef76687640 1 -- 172.21.15.88:0/3545946303 learned_addr learned my addr 172.21.15.88:0/3545946303 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:01.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.234+0000 7fef75685640 1 --2- 172.21.15.88:0/3545946303 >> v2:172.21.15.94:3300/0 conn(0x7fef7015ca70 0x7fef7015ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:01.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.234+0000 7fef75e86640 1 --2- 172.21.15.88:0/3545946303 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef700c9de0 0x7fef70065a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:01.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.234+0000 7fef76687640 1 -- 172.21.15.88:0/3545946303 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef700c9de0 msgr2=0x7fef70065a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:01.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.234+0000 7fef76687640 1 --2- 172.21.15.88:0/3545946303 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef700c9de0 0x7fef70065a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:01.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.234+0000 7fef76687640 1 -- 172.21.15.88:0/3545946303 >> v2:172.21.15.94:3300/0 conn(0x7fef7015ca70 msgr2=0x7fef7015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:01.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.234+0000 7fef76687640 1 --2- 172.21.15.88:0/3545946303 >> v2:172.21.15.94:3300/0 conn(0x7fef7015ca70 0x7fef7015ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:01.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.234+0000 7fef76687640 1 -- 172.21.15.88:0/3545946303 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef700ca910 con 0x7fef7006d290 2023-10-16T20:07:01.242 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.238+0000 7fef76687640 1 --2- 172.21.15.88:0/3545946303 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef7006d290 0x7fef7006d720 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fef70072380 tx=0x7fef6801c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d4d34eaba664f1c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:01.242 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.238+0000 7fef74e84640 1 -- 172.21.15.88:0/3545946303 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef680a5640 con 0x7fef7006d290 2023-10-16T20:07:01.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.238+0000 7fef74e84640 1 -- 172.21.15.88:0/3545946303 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef680a57a0 con 0x7fef7006d290 2023-10-16T20:07:01.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.238+0000 7fef74e84640 1 -- 172.21.15.88:0/3545946303 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef680a5b30 con 0x7fef7006d290 2023-10-16T20:07:01.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.238+0000 7fef780ea640 1 -- 172.21.15.88:0/3545946303 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef7006d290 msgr2=0x7fef7006d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:01.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.238+0000 7fef780ea640 1 --2- 172.21.15.88:0/3545946303 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef7006d290 0x7fef7006d720 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fef70072380 tx=0x7fef6801c7c0 comp rx=0 tx=0).stop 2023-10-16T20:07:01.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.238+0000 7fef780ea640 1 -- 172.21.15.88:0/3545946303 shutdown_connections 2023-10-16T20:07:01.244 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.238+0000 7fef780ea640 1 --2- 172.21.15.88:0/3545946303 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef7006d290 0x7fef7006d720 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:01.244 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.238+0000 7fef780ea640 1 --2- 172.21.15.88:0/3545946303 >> v2:172.21.15.94:3300/0 conn(0x7fef7015ca70 0x7fef7015ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:01.244 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.238+0000 7fef780ea640 1 --2- 172.21.15.88:0/3545946303 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef700c9de0 0x7fef70065a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:01.244 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.238+0000 7fef780ea640 1 -- 172.21.15.88:0/3545946303 >> 172.21.15.88:0/3545946303 conn(0x7fef700b0970 msgr2=0x7fef700b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:01.244 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.238+0000 7fef780ea640 1 -- 172.21.15.88:0/3545946303 shutdown_connections 2023-10-16T20:07:01.245 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.238+0000 7fef780ea640 1 -- 172.21.15.88:0/3545946303 wait complete. 2023-10-16T20:07:01.247 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.242+0000 7fef780ea640 1 Processor -- start 2023-10-16T20:07:01.247 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.242+0000 7fef780ea640 1 -- start start 2023-10-16T20:07:01.247 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.242+0000 7fef780ea640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef7006d290 0x7fef7014deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:01.247 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.242+0000 7fef780ea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef700c9de0 0x7fef701627f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:01.248 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.242+0000 7fef780ea640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef7015ca70 0x7fef700b1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:01.248 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.242+0000 7fef780ea640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef700c29b0 con 0x7fef7015ca70 2023-10-16T20:07:01.248 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.242+0000 7fef780ea640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef700c2c90 con 0x7fef700c9de0 2023-10-16T20:07:01.248 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.242+0000 7fef780ea640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef700c2f70 con 0x7fef7006d290 2023-10-16T20:07:01.249 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.242+0000 7fef76687640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef7015ca70 0x7fef700b1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:01.249 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.242+0000 7fef76687640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef7015ca70 0x7fef700b1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53406/0 (socket says 172.21.15.88:53406) 2023-10-16T20:07:01.249 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.242+0000 7fef76687640 1 -- 172.21.15.88:0/1449372711 learned_addr learned my addr 172.21.15.88:0/1449372711 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:01.249 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.242+0000 7fef75e86640 1 --2- 172.21.15.88:0/1449372711 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef7006d290 0x7fef7014deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:01.249 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.242+0000 7fef75685640 1 --2- 172.21.15.88:0/1449372711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef700c9de0 0x7fef701627f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:01.250 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.242+0000 7fef76687640 1 -- 172.21.15.88:0/1449372711 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef7006d290 msgr2=0x7fef7014deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:01.250 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.242+0000 7fef76687640 1 --2- 172.21.15.88:0/1449372711 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef7006d290 0x7fef7014deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:01.250 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.242+0000 7fef76687640 1 -- 172.21.15.88:0/1449372711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef700c9de0 msgr2=0x7fef701627f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:01.250 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.242+0000 7fef76687640 1 --2- 172.21.15.88:0/1449372711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef700c9de0 0x7fef701627f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:01.250 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.242+0000 7fef76687640 1 -- 172.21.15.88:0/1449372711 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef68009b30 con 0x7fef7015ca70 2023-10-16T20:07:01.251 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.242+0000 7fef75685640 1 --2- 172.21.15.88:0/1449372711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef700c9de0 0x7fef701627f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:07:01.251 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.242+0000 7fef76687640 1 --2- 172.21.15.88:0/1449372711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef7015ca70 0x7fef700b1c80 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fef680af040 tx=0x7fef68019560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:01.251 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.242+0000 7fef667fc640 1 -- 172.21.15.88:0/1449372711 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef680a5640 con 0x7fef7015ca70 2023-10-16T20:07:01.251 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.242+0000 7fef667fc640 1 -- 172.21.15.88:0/1449372711 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef680b8690 con 0x7fef7015ca70 2023-10-16T20:07:01.251 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.242+0000 7fef780ea640 1 -- 172.21.15.88:0/1449372711 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef700c3250 con 0x7fef7015ca70 2023-10-16T20:07:01.252 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.242+0000 7fef667fc640 1 -- 172.21.15.88:0/1449372711 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef680b8a40 con 0x7fef7015ca70 2023-10-16T20:07:01.252 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.242+0000 7fef780ea640 1 -- 172.21.15.88:0/1449372711 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef700c3810 con 0x7fef7015ca70 2023-10-16T20:07:01.252 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.242+0000 7fef780ea640 1 -- 172.21.15.88:0/1449372711 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef700c3a00 con 0x7fef7015ca70 2023-10-16T20:07:01.253 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.246+0000 7fef667fc640 1 -- 172.21.15.88:0/1449372711 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fef680b8ba0 con 0x7fef7015ca70 2023-10-16T20:07:01.253 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.246+0000 7fef667fc640 1 --2- 172.21.15.88:0/1449372711 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fef540664a0 0x7fef54068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:01.253 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.246+0000 7fef667fc640 1 -- 172.21.15.88:0/1449372711 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7fef6811f460 con 0x7fef7015ca70 2023-10-16T20:07:01.253 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.246+0000 7fef75e86640 1 --2- 172.21.15.88:0/1449372711 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fef540664a0 0x7fef54068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:01.254 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.246+0000 7fef75e86640 1 --2- 172.21.15.88:0/1449372711 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fef540664a0 0x7fef54068960 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fef6c009ac0 tx=0x7fef6c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:01.254 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.250+0000 7fef667fc640 1 -- 172.21.15.88:0/1449372711 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fef680ec540 con 0x7fef7015ca70 2023-10-16T20:07:01.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.402+0000 7fef780ea640 1 -- 172.21.15.88:0/1449372711 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7fef70065f40 con 0x7fef7015ca70 2023-10-16T20:07:01.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.402+0000 7fef667fc640 1 -- 172.21.15.88:0/1449372711 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v46) v1 ==== 97+0+11 (secure 0 0 0) 0x7fef68009510 con 0x7fef7015ca70 2023-10-16T20:07:01.408 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:07:01.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.406+0000 7fef780ea640 1 -- 172.21.15.88:0/1449372711 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fef540664a0 msgr2=0x7fef54068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:01.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.406+0000 7fef780ea640 1 --2- 172.21.15.88:0/1449372711 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fef540664a0 0x7fef54068960 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fef6c009ac0 tx=0x7fef6c009190 comp rx=0 tx=0).stop 2023-10-16T20:07:01.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.406+0000 7fef780ea640 1 -- 172.21.15.88:0/1449372711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef7015ca70 msgr2=0x7fef700b1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:01.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.406+0000 7fef780ea640 1 --2- 172.21.15.88:0/1449372711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef7015ca70 0x7fef700b1c80 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fef680af040 tx=0x7fef68019560 comp rx=0 tx=0).stop 2023-10-16T20:07:01.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.406+0000 7fef780ea640 1 -- 172.21.15.88:0/1449372711 shutdown_connections 2023-10-16T20:07:01.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.406+0000 7fef780ea640 1 --2- 172.21.15.88:0/1449372711 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fef540664a0 0x7fef54068960 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:01.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.406+0000 7fef780ea640 1 --2- 172.21.15.88:0/1449372711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef7015ca70 0x7fef700b1c80 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:01.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.406+0000 7fef780ea640 1 --2- 172.21.15.88:0/1449372711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef700c9de0 0x7fef701627f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:01.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.406+0000 7fef780ea640 1 --2- 172.21.15.88:0/1449372711 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef7006d290 0x7fef7014deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:01.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.406+0000 7fef780ea640 1 -- 172.21.15.88:0/1449372711 >> 172.21.15.88:0/1449372711 conn(0x7fef700b0970 msgr2=0x7fef7006a720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:01.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.406+0000 7fef780ea640 1 -- 172.21.15.88:0/1449372711 shutdown_connections 2023-10-16T20:07:01.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.406+0000 7fef780ea640 1 -- 172.21.15.88:0/1449372711 wait complete. 2023-10-16T20:07:01.428 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T20:07:01.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.566+0000 7fa7a7254640 1 Processor -- start 2023-10-16T20:07:01.569 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.566+0000 7fa7a7254640 1 -- start start 2023-10-16T20:07:01.569 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.566+0000 7fa7a7254640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa7a00cd720 0x7fa7a015da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:01.570 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.566+0000 7fa7a7254640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fa7a00ccb10 0x7fa7a015df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:01.570 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.566+0000 7fa7a7254640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7a015d5b0 con 0x7fa7a00cc730 2023-10-16T20:07:01.570 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.566+0000 7fa7a7254640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7a00cdc20 con 0x7fa7a00ccb10 2023-10-16T20:07:01.570 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.566+0000 7fa7a7254640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a00cdf00 con 0x7fa7a00cd720 2023-10-16T20:07:01.571 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.566+0000 7fa7a57f1640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fa7a00ccb10 0x7fa7a015df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:01.571 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.566+0000 7fa7a4ff0640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fa7a00cc730 0x7fa7a01609c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:57346/0 (socket says 172.21.15.88:57346) 2023-10-16T20:07:01.571 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.566+0000 7fa7a4ff0640 1 -- 172.21.15.88:0/69011452 learned_addr learned my addr 172.21.15.88:0/69011452 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:01.572 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.566+0000 7fa79ffff640 1 --2- 172.21.15.88:0/69011452 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa7a00cd720 0x7fa7a015da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:01.572 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.566+0000 7fa7a57f1640 1 -- 172.21.15.88:0/69011452 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa7a00cd720 msgr2=0x7fa7a015da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:01.572 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.566+0000 7fa7a57f1640 1 --2- 172.21.15.88:0/69011452 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa7a00cd720 0x7fa7a015da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:01.572 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.566+0000 7fa7a57f1640 1 -- 172.21.15.88:0/69011452 >> v1:172.21.15.88:6789/0 conn(0x7fa7a00cc730 legacy=0x7fa7a01609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:01.573 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.566+0000 7fa79f7fe640 1 -- 172.21.15.88:0/69011452 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 709073895 0 0) 0x7fa7a015d5b0 con 0x7fa7a00cc730 2023-10-16T20:07:01.573 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.566+0000 7fa7a57f1640 1 -- 172.21.15.88:0/69011452 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a00ce1e0 con 0x7fa7a00ccb10 2023-10-16T20:07:01.574 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.570+0000 7fa7a57f1640 1 --2- 172.21.15.88:0/69011452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7a00ccb10 0x7fa7a015df80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa798009d90 tx=0x7fa79801c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5c764bed43c7cc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:01.574 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.570+0000 7fa79f7fe640 1 -- 172.21.15.88:0/69011452 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7980a58b0 con 0x7fa7a00ccb10 2023-10-16T20:07:01.574 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.570+0000 7fa79f7fe640 1 -- 172.21.15.88:0/69011452 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7980a5a10 con 0x7fa7a00ccb10 2023-10-16T20:07:01.575 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.570+0000 7fa79f7fe640 1 -- 172.21.15.88:0/69011452 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7980a5da0 con 0x7fa7a00ccb10 2023-10-16T20:07:01.575 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.570+0000 7fa7a7254640 1 -- 172.21.15.88:0/69011452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7a00ccb10 msgr2=0x7fa7a015df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:01.575 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.570+0000 7fa7a7254640 1 --2- 172.21.15.88:0/69011452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7a00ccb10 0x7fa7a015df80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa798009d90 tx=0x7fa79801c890 comp rx=0 tx=0).stop 2023-10-16T20:07:01.575 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.570+0000 7fa7a7254640 1 -- 172.21.15.88:0/69011452 shutdown_connections 2023-10-16T20:07:01.575 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.570+0000 7fa7a7254640 1 --2- 172.21.15.88:0/69011452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7a00ccb10 0x7fa7a015df80 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:01.576 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.570+0000 7fa7a7254640 1 --2- 172.21.15.88:0/69011452 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa7a00cd720 0x7fa7a015da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:01.576 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.570+0000 7fa7a7254640 1 -- 172.21.15.88:0/69011452 >> 172.21.15.88:0/69011452 conn(0x7fa7a006b170 msgr2=0x7fa7a006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:01.576 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.570+0000 7fa7a7254640 1 -- 172.21.15.88:0/69011452 shutdown_connections 2023-10-16T20:07:01.576 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.570+0000 7fa7a7254640 1 -- 172.21.15.88:0/69011452 wait complete. 2023-10-16T20:07:01.578 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.574+0000 7fa7a7254640 1 Processor -- start 2023-10-16T20:07:01.578 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.574+0000 7fa7a7254640 1 -- start start 2023-10-16T20:07:01.579 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.574+0000 7fa7a7254640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7a00cc730 0x7fa7a0154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:01.579 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.574+0000 7fa7a7254640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a00ccb10 0x7fa7a00b2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:01.579 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.574+0000 7fa7a7254640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa7a00cd720 0x7fa7a00b2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:01.579 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.574+0000 7fa7a7254640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a0159d60 con 0x7fa7a00cc730 2023-10-16T20:07:01.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.574+0000 7fa7a7254640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a00c4910 con 0x7fa7a00ccb10 2023-10-16T20:07:01.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.574+0000 7fa7a7254640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a0159fd0 con 0x7fa7a00cd720 2023-10-16T20:07:01.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.574+0000 7fa7a4ff0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7a00cc730 0x7fa7a0154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:01.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.574+0000 7fa7a4ff0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7a00cc730 0x7fa7a0154210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53416/0 (socket says 172.21.15.88:53416) 2023-10-16T20:07:01.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.574+0000 7fa7a4ff0640 1 -- 172.21.15.88:0/1386373485 learned_addr learned my addr 172.21.15.88:0/1386373485 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:01.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.574+0000 7fa7a4ff0640 1 -- 172.21.15.88:0/1386373485 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa7a00cd720 msgr2=0x7fa7a00b2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:01.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.574+0000 7fa7a4ff0640 1 --2- 172.21.15.88:0/1386373485 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa7a00cd720 0x7fa7a00b2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:01.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.574+0000 7fa79ffff640 1 --2- 172.21.15.88:0/1386373485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a00ccb10 0x7fa7a00b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:01.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.574+0000 7fa7a4ff0640 1 -- 172.21.15.88:0/1386373485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a00ccb10 msgr2=0x7fa7a00b2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:01.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.574+0000 7fa7a4ff0640 1 --2- 172.21.15.88:0/1386373485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a00ccb10 0x7fa7a00b2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:01.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.574+0000 7fa7a4ff0640 1 -- 172.21.15.88:0/1386373485 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa798009a70 con 0x7fa7a00cc730 2023-10-16T20:07:01.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.574+0000 7fa79ffff640 1 --2- 172.21.15.88:0/1386373485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a00ccb10 0x7fa7a00b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:07:01.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.574+0000 7fa7a4ff0640 1 --2- 172.21.15.88:0/1386373485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7a00cc730 0x7fa7a0154210 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fa7940011f0 tx=0x7fa79400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:01.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.574+0000 7fa79d7fa640 1 -- 172.21.15.88:0/1386373485 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7940074e0 con 0x7fa7a00cc730 2023-10-16T20:07:01.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.574+0000 7fa79d7fa640 1 -- 172.21.15.88:0/1386373485 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa794007640 con 0x7fa7a00cc730 2023-10-16T20:07:01.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.574+0000 7fa79d7fa640 1 -- 172.21.15.88:0/1386373485 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa794004a10 con 0x7fa7a00cc730 2023-10-16T20:07:01.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.574+0000 7fa7a7254640 1 -- 172.21.15.88:0/1386373485 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a015a2b0 con 0x7fa7a00cc730 2023-10-16T20:07:01.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.574+0000 7fa7a7254640 1 -- 172.21.15.88:0/1386373485 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a0154750 con 0x7fa7a00cc730 2023-10-16T20:07:01.584 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.574+0000 7fa7a7254640 1 -- 172.21.15.88:0/1386373485 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a015a4d0 con 0x7fa7a00cc730 2023-10-16T20:07:01.584 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.578+0000 7fa79d7fa640 1 -- 172.21.15.88:0/1386373485 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa794004b70 con 0x7fa7a00cc730 2023-10-16T20:07:01.584 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.578+0000 7fa79d7fa640 1 --2- 172.21.15.88:0/1386373485 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa7840664a0 0x7fa784068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:01.584 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.578+0000 7fa79d7fa640 1 -- 172.21.15.88:0/1386373485 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7fa794083490 con 0x7fa7a00cc730 2023-10-16T20:07:01.584 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.578+0000 7fa79ffff640 1 --2- 172.21.15.88:0/1386373485 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa7840664a0 0x7fa784068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:01.585 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.578+0000 7fa79ffff640 1 --2- 172.21.15.88:0/1386373485 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa7840664a0 0x7fa784068960 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa7a00b40e0 tx=0x7fa78c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:01.585 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.582+0000 7fa79d7fa640 1 -- 172.21.15.88:0/1386373485 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa794050600 con 0x7fa7a00cc730 2023-10-16T20:07:01.745 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.742+0000 7fa7a7254640 1 -- 172.21.15.88:0/1386373485 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa7a004e9c0 con 0x7fa7a00cc730 2023-10-16T20:07:01.745 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.742+0000 7fa79d7fa640 1 -- 172.21.15.88:0/1386373485 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v46) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa794018080 con 0x7fa7a00cc730 2023-10-16T20:07:01.746 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2023-10-16T20:07:01.747 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.742+0000 7fa7a7254640 1 -- 172.21.15.88:0/1386373485 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa7840664a0 msgr2=0x7fa784068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:01.747 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.742+0000 7fa7a7254640 1 --2- 172.21.15.88:0/1386373485 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa7840664a0 0x7fa784068960 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa7a00b40e0 tx=0x7fa78c008040 comp rx=0 tx=0).stop 2023-10-16T20:07:01.748 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.742+0000 7fa7a7254640 1 -- 172.21.15.88:0/1386373485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7a00cc730 msgr2=0x7fa7a0154210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:01.748 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.742+0000 7fa7a7254640 1 --2- 172.21.15.88:0/1386373485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7a00cc730 0x7fa7a0154210 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fa7940011f0 tx=0x7fa79400c9b0 comp rx=0 tx=0).stop 2023-10-16T20:07:01.749 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.742+0000 7fa7a7254640 1 -- 172.21.15.88:0/1386373485 shutdown_connections 2023-10-16T20:07:01.749 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.742+0000 7fa7a7254640 1 --2- 172.21.15.88:0/1386373485 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa7840664a0 0x7fa784068960 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:01.749 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.742+0000 7fa7a7254640 1 --2- 172.21.15.88:0/1386373485 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa7a00cd720 0x7fa7a00b2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:01.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.742+0000 7fa7a7254640 1 --2- 172.21.15.88:0/1386373485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a00ccb10 0x7fa7a00b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:01.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.742+0000 7fa7a7254640 1 --2- 172.21.15.88:0/1386373485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7a00cc730 0x7fa7a0154210 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:01.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.742+0000 7fa7a7254640 1 -- 172.21.15.88:0/1386373485 >> 172.21.15.88:0/1386373485 conn(0x7fa7a006b170 msgr2=0x7fa7a0155610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:01.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.742+0000 7fa7a7254640 1 -- 172.21.15.88:0/1386373485 shutdown_connections 2023-10-16T20:07:01.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:01.742+0000 7fa7a7254640 1 -- 172.21.15.88:0/1386373485 wait complete. 2023-10-16T20:07:01.766 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi088.front.sepia.ceph.com... 2023-10-16T20:07:01.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:01.767 DEBUG:teuthology.orchestra.run.smithi088:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-16T20:07:01.786 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[50434]: fuse finished with error 0 and tester_r 0 2023-10-16T20:07:01.789 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:07:07.891 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2023-10-16T20:07:07.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:07.892 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T20:07:07.904 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=testcephfs1_data' 2023-10-16T20:07:08.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.097+0000 7fa2e0a08640 1 Processor -- start 2023-10-16T20:07:08.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.097+0000 7fa2e0a08640 1 -- start start 2023-10-16T20:07:08.104 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.097+0000 7fa2e0a08640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2dc0d12d0 0x7fa2dc0cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:08.104 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.097+0000 7fa2e0a08640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fa2dc160560 0x7fa2dc0cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:08.104 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.097+0000 7fa2e0a08640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2dc0b3df0 con 0x7fa2dc160180 2023-10-16T20:07:08.105 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.097+0000 7fa2e0a08640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2dc0b4280 con 0x7fa2dc160560 2023-10-16T20:07:08.105 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.097+0000 7fa2e0a08640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc0d0220 con 0x7fa2dc0d12d0 2023-10-16T20:07:08.105 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.097+0000 7fa2dad9d640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fa2dc160180 0x7fa2dc0d0bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:50762/0 (socket says 172.21.15.88:50762) 2023-10-16T20:07:08.105 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.097+0000 7fa2dad9d640 1 -- 172.21.15.88:0/3631193591 learned_addr learned my addr 172.21.15.88:0/3631193591 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:08.105 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.097+0000 7fa2db59e640 1 --2- 172.21.15.88:0/3631193591 >> v2:172.21.15.94:3300/0 conn(0x7fa2dc160560 0x7fa2dc0cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:08.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.097+0000 7fa2da59c640 1 --2- 172.21.15.88:0/3631193591 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2dc0d12d0 0x7fa2dc0cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:08.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.097+0000 7fa2d9d9b640 1 -- 172.21.15.88:0/3631193591 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3531293092 0 0) 0x7fa2dc0b3df0 con 0x7fa2dc160180 2023-10-16T20:07:08.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.097+0000 7fa2d9d9b640 1 -- 172.21.15.88:0/3631193591 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2c4003530 con 0x7fa2dc160180 2023-10-16T20:07:08.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.097+0000 7fa2d9d9b640 1 -- 172.21.15.88:0/3631193591 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3347626272 0 0) 0x7fa2c4003530 con 0x7fa2dc160180 2023-10-16T20:07:08.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.097+0000 7fa2d9d9b640 1 -- 172.21.15.88:0/3631193591 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2dc0d12d0 msgr2=0x7fa2dc0cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:08.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.097+0000 7fa2d9d9b640 1 --2- 172.21.15.88:0/3631193591 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2dc0d12d0 0x7fa2dc0cfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:08.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.097+0000 7fa2d9d9b640 1 -- 172.21.15.88:0/3631193591 >> v2:172.21.15.94:3300/0 conn(0x7fa2dc160560 msgr2=0x7fa2dc0cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:08.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.097+0000 7fa2d9d9b640 1 --2- 172.21.15.88:0/3631193591 >> v2:172.21.15.94:3300/0 conn(0x7fa2dc160560 0x7fa2dc0cd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:08.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.097+0000 7fa2d9d9b640 1 -- 172.21.15.88:0/3631193591 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2dc0b3af0 con 0x7fa2dc160180 2023-10-16T20:07:08.108 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.097+0000 7fa2d9d9b640 1 -- 172.21.15.88:0/3631193591 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7fa2cc001680 con 0x7fa2dc160180 2023-10-16T20:07:08.108 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.097+0000 7fa2d9d9b640 1 -- 172.21.15.88:0/3631193591 >> v1:172.21.15.88:6789/0 conn(0x7fa2dc160180 legacy=0x7fa2dc0d0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:08.108 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.097+0000 7fa2d9d9b640 1 --2- 172.21.15.88:0/3631193591 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2dc0d12d0 0x7fa2c4003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:08.108 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.097+0000 7fa2d9d9b640 1 --2- 172.21.15.88:0/3631193591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2c4003f00 0x7fa2c40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:08.108 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.097+0000 7fa2d9d9b640 1 --2- 172.21.15.88:0/3631193591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2c4004d10 0x7fa2c4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:08.108 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.097+0000 7fa2d9d9b640 1 -- 172.21.15.88:0/3631193591 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc0b4280 con 0x7fa2c4004d10 2023-10-16T20:07:08.109 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.097+0000 7fa2d9d9b640 1 -- 172.21.15.88:0/3631193591 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc0d0220 con 0x7fa2c4003f00 2023-10-16T20:07:08.109 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.097+0000 7fa2d9d9b640 1 -- 172.21.15.88:0/3631193591 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c400b9e0 con 0x7fa2dc0d12d0 2023-10-16T20:07:08.109 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.101+0000 7fa2db59e640 1 --2- 172.21.15.88:0/3631193591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2c4004d10 0x7fa2c4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:08.109 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.101+0000 7fa2dad9d640 1 --2- 172.21.15.88:0/3631193591 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2dc0d12d0 0x7fa2c4003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:08.109 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.101+0000 7fa2da59c640 1 --2- 172.21.15.88:0/3631193591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2c4003f00 0x7fa2c40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:08.110 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.101+0000 7fa2db59e640 1 -- 172.21.15.88:0/3631193591 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2dc0d12d0 msgr2=0x7fa2c4003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:08.110 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.101+0000 7fa2db59e640 1 --2- 172.21.15.88:0/3631193591 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2dc0d12d0 0x7fa2c4003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:08.110 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.101+0000 7fa2db59e640 1 -- 172.21.15.88:0/3631193591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2c4003f00 msgr2=0x7fa2c40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:08.110 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.101+0000 7fa2db59e640 1 --2- 172.21.15.88:0/3631193591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2c4003f00 0x7fa2c40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:08.111 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.101+0000 7fa2db59e640 1 -- 172.21.15.88:0/3631193591 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c4003530 con 0x7fa2c4004d10 2023-10-16T20:07:08.113 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.101+0000 7fa2db59e640 1 --2- 172.21.15.88:0/3631193591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2c4004d10 0x7fa2c4005190 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4009f90 tx=0x7fa2d401cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da24a5cccf734482 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:08.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.101+0000 7fa2d9d9b640 1 -- 172.21.15.88:0/3631193591 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d40a5c30 con 0x7fa2c4004d10 2023-10-16T20:07:08.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.101+0000 7fa2d9d9b640 1 -- 172.21.15.88:0/3631193591 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d400f040 con 0x7fa2c4004d10 2023-10-16T20:07:08.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.101+0000 7fa2d9d9b640 1 -- 172.21.15.88:0/3631193591 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d40ad440 con 0x7fa2c4004d10 2023-10-16T20:07:08.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.105+0000 7fa2e0a08640 1 -- 172.21.15.88:0/3631193591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2c4004d10 msgr2=0x7fa2c4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:08.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.105+0000 7fa2e0a08640 1 --2- 172.21.15.88:0/3631193591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2c4004d10 0x7fa2c4005190 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4009f90 tx=0x7fa2d401cc20 comp rx=0 tx=0).stop 2023-10-16T20:07:08.115 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.105+0000 7fa2db59e640 1 -- 172.21.15.88:0/3631193591 reap_dead start 2023-10-16T20:07:08.115 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.105+0000 7fa2e0a08640 1 -- 172.21.15.88:0/3631193591 shutdown_connections 2023-10-16T20:07:08.115 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.105+0000 7fa2e0a08640 1 -- 172.21.15.88:0/3631193591 >> 172.21.15.88:0/3631193591 conn(0x7fa2dc089fb0 msgr2=0x7fa2dc0d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:08.115 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.105+0000 7fa2e0a08640 1 -- 172.21.15.88:0/3631193591 shutdown_connections 2023-10-16T20:07:08.115 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.105+0000 7fa2e0a08640 1 -- 172.21.15.88:0/3631193591 wait complete. 2023-10-16T20:07:08.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.105+0000 7fa2e0a08640 1 Processor -- start 2023-10-16T20:07:08.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.105+0000 7fa2e0a08640 1 -- start start 2023-10-16T20:07:08.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.105+0000 7fa2e0a08640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2dc0cf260 0x7fa2dc0b76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:08.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.105+0000 7fa2e0a08640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2dc0cf640 0x7fa2dc0b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:08.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.105+0000 7fa2e0a08640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2dc14d9b0 0x7fa2dc14de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:08.117 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.105+0000 7fa2e0a08640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc14e7c0 con 0x7fa2dc14d9b0 2023-10-16T20:07:08.117 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.105+0000 7fa2e0a08640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc14eaa0 con 0x7fa2dc0cf640 2023-10-16T20:07:08.117 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.105+0000 7fa2e0a08640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc14ed80 con 0x7fa2dc0cf260 2023-10-16T20:07:08.117 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.105+0000 7fa2db59e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2dc14d9b0 0x7fa2dc14de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:08.117 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.105+0000 7fa2db59e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2dc14d9b0 0x7fa2dc14de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57900/0 (socket says 172.21.15.88:57900) 2023-10-16T20:07:08.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.105+0000 7fa2db59e640 1 -- 172.21.15.88:0/2175999538 learned_addr learned my addr 172.21.15.88:0/2175999538 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:08.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.105+0000 7fa2dad9d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2dc0cf260 0x7fa2dc0b76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:08.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.105+0000 7fa2da59c640 1 --2- 172.21.15.88:0/2175999538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2dc0cf640 0x7fa2dc0b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:08.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.109+0000 7fa2db59e640 1 -- 172.21.15.88:0/2175999538 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2dc0cf260 msgr2=0x7fa2dc0b76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:08.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.109+0000 7fa2db59e640 1 --2- 172.21.15.88:0/2175999538 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2dc0cf260 0x7fa2dc0b76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:08.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.109+0000 7fa2db59e640 1 -- 172.21.15.88:0/2175999538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2dc0cf640 msgr2=0x7fa2dc0b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:08.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.109+0000 7fa2db59e640 1 --2- 172.21.15.88:0/2175999538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2dc0cf640 0x7fa2dc0b7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:08.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.109+0000 7fa2db59e640 1 -- 172.21.15.88:0/2175999538 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2dc14f060 con 0x7fa2dc14d9b0 2023-10-16T20:07:08.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.109+0000 7fa2db59e640 1 --2- 172.21.15.88:0/2175999538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2dc14d9b0 0x7fa2dc14de30 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4002240 tx=0x7fa2d4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:08.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.109+0000 7fa2c37fe640 1 -- 172.21.15.88:0/2175999538 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d40ad650 con 0x7fa2dc14d9b0 2023-10-16T20:07:08.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.109+0000 7fa2c37fe640 1 -- 172.21.15.88:0/2175999538 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d40ad7b0 con 0x7fa2dc14d9b0 2023-10-16T20:07:08.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.109+0000 7fa2c37fe640 1 -- 172.21.15.88:0/2175999538 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d40adb40 con 0x7fa2dc14d9b0 2023-10-16T20:07:08.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.109+0000 7fa2e0a08640 1 -- 172.21.15.88:0/2175999538 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2dc14f340 con 0x7fa2dc14d9b0 2023-10-16T20:07:08.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.109+0000 7fa2e0a08640 1 -- 172.21.15.88:0/2175999538 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2dc14e540 con 0x7fa2dc14d9b0 2023-10-16T20:07:08.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.109+0000 7fa2e0a08640 1 -- 172.21.15.88:0/2175999538 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2dc0674f0 con 0x7fa2dc14d9b0 2023-10-16T20:07:08.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.109+0000 7fa2c37fe640 1 -- 172.21.15.88:0/2175999538 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa2d40b4040 con 0x7fa2dc14d9b0 2023-10-16T20:07:08.122 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.109+0000 7fa2c37fe640 1 --2- 172.21.15.88:0/2175999538 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa2c80664a0 0x7fa2c8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:08.122 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.109+0000 7fa2c37fe640 1 -- 172.21.15.88:0/2175999538 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7fa2d411e5c0 con 0x7fa2dc14d9b0 2023-10-16T20:07:08.122 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.109+0000 7fa2dad9d640 1 --2- 172.21.15.88:0/2175999538 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa2c80664a0 0x7fa2c8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:08.122 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.113+0000 7fa2dad9d640 1 --2- 172.21.15.88:0/2175999538 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa2c80664a0 0x7fa2c8068960 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc004850 tx=0x7fa2cc007900 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:08.122 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.113+0000 7fa2c37fe640 1 -- 172.21.15.88:0/2175999538 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa2d40eb6d0 con 0x7fa2dc14d9b0 2023-10-16T20:07:08.320 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.317+0000 7fa2e0a08640 1 -- 172.21.15.88:0/2175999538 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=testcephfs1_data"]} v 0) v1 -- 0x7fa2dc08a500 con 0x7fa2dc14d9b0 2023-10-16T20:07:08.323 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.317+0000 7fa2c37fe640 1 -- 172.21.15.88:0/2175999538 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=testcephfs1_data"]}]=0 updated caps for client.0 v16) v1 ==== 189+0+146 (secure 0 0 0) 0x7fa2d40b20e0 con 0x7fa2dc14d9b0 2023-10-16T20:07:08.323 INFO:teuthology.orchestra.run.smithi088.stdout:[client.0] 2023-10-16T20:07:08.323 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQAvly1looeRAhAAeLshHOVIhQqoTtlPehorIg== 2023-10-16T20:07:08.324 INFO:teuthology.orchestra.run.smithi088.stdout: caps mds = "allow" 2023-10-16T20:07:08.324 INFO:teuthology.orchestra.run.smithi088.stdout: caps mon = "allow r" 2023-10-16T20:07:08.324 INFO:teuthology.orchestra.run.smithi088.stdout: caps osd = "allow rw pool=testcephfs1_data" 2023-10-16T20:07:08.325 INFO:teuthology.orchestra.run.smithi088.stderr:updated caps for client.0 2023-10-16T20:07:08.325 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.321+0000 7fa2e0a08640 1 -- 172.21.15.88:0/2175999538 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa2c80664a0 msgr2=0x7fa2c8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:08.326 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.321+0000 7fa2e0a08640 1 --2- 172.21.15.88:0/2175999538 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa2c80664a0 0x7fa2c8068960 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc004850 tx=0x7fa2cc007900 comp rx=0 tx=0).stop 2023-10-16T20:07:08.326 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.321+0000 7fa2e0a08640 1 -- 172.21.15.88:0/2175999538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2dc14d9b0 msgr2=0x7fa2dc14de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:08.326 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.321+0000 7fa2e0a08640 1 --2- 172.21.15.88:0/2175999538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2dc14d9b0 0x7fa2dc14de30 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4002240 tx=0x7fa2d4009750 comp rx=0 tx=0).stop 2023-10-16T20:07:08.326 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.321+0000 7fa2e0a08640 1 -- 172.21.15.88:0/2175999538 shutdown_connections 2023-10-16T20:07:08.327 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.321+0000 7fa2e0a08640 1 --2- 172.21.15.88:0/2175999538 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa2c80664a0 0x7fa2c8068960 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:08.327 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.321+0000 7fa2e0a08640 1 --2- 172.21.15.88:0/2175999538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2dc14d9b0 0x7fa2dc14de30 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:08.327 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.321+0000 7fa2e0a08640 1 --2- 172.21.15.88:0/2175999538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2dc0cf640 0x7fa2dc0b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:08.327 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.321+0000 7fa2e0a08640 1 --2- 172.21.15.88:0/2175999538 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa2dc0cf260 0x7fa2dc0b76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:08.327 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.321+0000 7fa2e0a08640 1 -- 172.21.15.88:0/2175999538 >> 172.21.15.88:0/2175999538 conn(0x7fa2dc089fb0 msgr2=0x7fa2dc0b3170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:08.327 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.321+0000 7fa2e0a08640 1 -- 172.21.15.88:0/2175999538 shutdown_connections 2023-10-16T20:07:08.328 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.321+0000 7fa2e0a08640 1 -- 172.21.15.88:0/2175999538 wait complete. 2023-10-16T20:07:08.344 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T20:07:08.344 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T20:07:08.426 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T20:07:08.510 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T20:07:08.510 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:07:08.511 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:07:08.520 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354668k,nr_inodes=4088667,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9372 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T20:07:08.521 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2023-10-16T20:07:08.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:08.522 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T20:07:08.575 INFO:teuthology.orchestra.run.smithi088.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-16T20:07:08.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T20:07:08.576 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:07:08.577 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:07:08.586 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354624k,nr_inodes=4088656,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5605 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T20:07:08.586 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2023-10-16T20:07:08.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:08.587 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-16T20:07:08.635 INFO:teuthology.orchestra.run.smithi094.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-16T20:07:08.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T20:07:08.637 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:07:08.637 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:07:08.646 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354668k,nr_inodes=4088667,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9372 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T20:07:08.646 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2023-10-16T20:07:08.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:08.647 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-16T20:07:08.699 INFO:teuthology.orchestra.run.smithi088.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-16T20:07:08.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T20:07:08.702 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:07:08.702 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:07:08.713 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354624k,nr_inodes=4088656,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5605 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T20:07:08.713 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2023-10-16T20:07:08.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:08.715 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-16T20:07:08.763 INFO:teuthology.orchestra.run.smithi094.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-16T20:07:08.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T20:07:08.765 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T20:07:08.901 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.897+0000 7ff8239f8640 1 Processor -- start 2023-10-16T20:07:08.902 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.897+0000 7ff8239f8640 1 -- start start 2023-10-16T20:07:08.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.897+0000 7ff8239f8640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff81c15cc60 0x7ff81c0caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:08.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.897+0000 7ff8239f8640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff81c0c40d0 0x7ff81c070710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:08.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.897+0000 7ff8239f8640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff81c0ca710 con 0x7ff81c071450 2023-10-16T20:07:08.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.897+0000 7ff8239f8640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff81c0c9de0 con 0x7ff81c0c40d0 2023-10-16T20:07:08.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.897+0000 7ff8239f8640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c0ca0c0 con 0x7ff81c15cc60 2023-10-16T20:07:08.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.897+0000 7ff821f95640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff81c0c40d0 0x7ff81c070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:08.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.897+0000 7ff821f95640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff81c0c40d0 0x7ff81c070710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57916/0 (socket says 172.21.15.88:57916) 2023-10-16T20:07:08.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.897+0000 7ff821f95640 1 -- 172.21.15.88:0/2699312514 learned_addr learned my addr 172.21.15.88:0/2699312514 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:08.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.897+0000 7ff821794640 1 --2- 172.21.15.88:0/2699312514 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff81c15cc60 0x7ff81c0caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:08.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.897+0000 7ff821f95640 1 -- 172.21.15.88:0/2699312514 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff81c15cc60 msgr2=0x7ff81c0caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:08.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.897+0000 7ff821f95640 1 --2- 172.21.15.88:0/2699312514 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff81c15cc60 0x7ff81c0caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:08.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.897+0000 7ff821f95640 1 -- 172.21.15.88:0/2699312514 >> v1:172.21.15.94:6789/0 conn(0x7ff81c071450 legacy=0x7ff81c0c39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:08.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.897+0000 7ff821f95640 1 -- 172.21.15.88:0/2699312514 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81c0ca3a0 con 0x7ff81c0c40d0 2023-10-16T20:07:08.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.897+0000 7ff821794640 1 --2- 172.21.15.88:0/2699312514 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff81c15cc60 0x7ff81c0caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:07:08.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.901+0000 7ff821f95640 1 --2- 172.21.15.88:0/2699312514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff81c0c40d0 0x7ff81c070710 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff814009e50 tx=0x7ff81401ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60cb8c8b23c36cb6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:08.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.901+0000 7ff813fff640 1 -- 172.21.15.88:0/2699312514 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8140a57f0 con 0x7ff81c0c40d0 2023-10-16T20:07:08.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.901+0000 7ff813fff640 1 -- 172.21.15.88:0/2699312514 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8140a5950 con 0x7ff81c0c40d0 2023-10-16T20:07:08.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.901+0000 7ff813fff640 1 -- 172.21.15.88:0/2699312514 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8140a5ce0 con 0x7ff81c0c40d0 2023-10-16T20:07:08.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.901+0000 7ff8239f8640 1 -- 172.21.15.88:0/2699312514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff81c0c40d0 msgr2=0x7ff81c070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:08.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.901+0000 7ff8239f8640 1 --2- 172.21.15.88:0/2699312514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff81c0c40d0 0x7ff81c070710 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff814009e50 tx=0x7ff81401ca10 comp rx=0 tx=0).stop 2023-10-16T20:07:08.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.901+0000 7ff8239f8640 1 -- 172.21.15.88:0/2699312514 shutdown_connections 2023-10-16T20:07:08.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.901+0000 7ff8239f8640 1 --2- 172.21.15.88:0/2699312514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff81c0c40d0 0x7ff81c070710 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:08.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.901+0000 7ff8239f8640 1 --2- 172.21.15.88:0/2699312514 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff81c15cc60 0x7ff81c0caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:08.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.901+0000 7ff8239f8640 1 -- 172.21.15.88:0/2699312514 >> 172.21.15.88:0/2699312514 conn(0x7ff81c0ceb20 msgr2=0x7ff81c0cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:08.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.901+0000 7ff8239f8640 1 -- 172.21.15.88:0/2699312514 shutdown_connections 2023-10-16T20:07:08.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.901+0000 7ff8239f8640 1 -- 172.21.15.88:0/2699312514 wait complete. 2023-10-16T20:07:08.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.905+0000 7ff8239f8640 1 Processor -- start 2023-10-16T20:07:08.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.905+0000 7ff8239f8640 1 -- start start 2023-10-16T20:07:08.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.905+0000 7ff8239f8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff81c071450 0x7ff81c143130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:08.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.905+0000 7ff8239f8640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff81c15cc60 0x7ff81c143860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:08.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.905+0000 7ff8239f8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff81c143da0 0x7ff81c0c4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:08.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.905+0000 7ff8239f8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c14ef30 con 0x7ff81c143da0 2023-10-16T20:07:08.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.905+0000 7ff8239f8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c0b0370 con 0x7ff81c071450 2023-10-16T20:07:08.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.905+0000 7ff8239f8640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c143670 con 0x7ff81c15cc60 2023-10-16T20:07:08.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.905+0000 7ff821794640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff81c071450 0x7ff81c143130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:08.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.905+0000 7ff821f95640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff81c143da0 0x7ff81c0c4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:08.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.905+0000 7ff820f93640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff81c15cc60 0x7ff81c143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:08.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.905+0000 7ff821f95640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff81c143da0 0x7ff81c0c4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57922/0 (socket says 172.21.15.88:57922) 2023-10-16T20:07:08.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.905+0000 7ff821794640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff81c071450 0x7ff81c143130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:34018/0 (socket says 172.21.15.88:34018) 2023-10-16T20:07:08.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.905+0000 7ff821f95640 1 -- 172.21.15.88:0/1099458602 learned_addr learned my addr 172.21.15.88:0/1099458602 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:08.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.905+0000 7ff821f95640 1 -- 172.21.15.88:0/1099458602 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff81c15cc60 msgr2=0x7ff81c143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:08.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.905+0000 7ff821f95640 1 --2- 172.21.15.88:0/1099458602 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff81c15cc60 0x7ff81c143860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:08.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.905+0000 7ff821f95640 1 -- 172.21.15.88:0/1099458602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff81c071450 msgr2=0x7ff81c143130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:08.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.905+0000 7ff821f95640 1 --2- 172.21.15.88:0/1099458602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff81c071450 0x7ff81c143130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:08.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.905+0000 7ff821f95640 1 -- 172.21.15.88:0/1099458602 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff814009b30 con 0x7ff81c143da0 2023-10-16T20:07:08.919 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.905+0000 7ff821794640 1 --2- 172.21.15.88:0/1099458602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff81c071450 0x7ff81c143130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:07:08.919 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.905+0000 7ff821f95640 1 --2- 172.21.15.88:0/1099458602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff81c143da0 0x7ff81c0c4750 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff8140af040 tx=0x7ff8140197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:08.919 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.905+0000 7ff811ffb640 1 -- 172.21.15.88:0/1099458602 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8140b6680 con 0x7ff81c143da0 2023-10-16T20:07:08.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.909+0000 7ff811ffb640 1 -- 172.21.15.88:0/1099458602 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8140b67e0 con 0x7ff81c143da0 2023-10-16T20:07:08.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.909+0000 7ff811ffb640 1 -- 172.21.15.88:0/1099458602 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8140b6b90 con 0x7ff81c143da0 2023-10-16T20:07:08.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.909+0000 7ff8239f8640 1 -- 172.21.15.88:0/1099458602 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff81c14e600 con 0x7ff81c143da0 2023-10-16T20:07:08.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.909+0000 7ff8239f8640 1 -- 172.21.15.88:0/1099458602 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81c14ebc0 con 0x7ff81c143da0 2023-10-16T20:07:08.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.909+0000 7ff8239f8640 1 -- 172.21.15.88:0/1099458602 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff81c14f0e0 con 0x7ff81c143da0 2023-10-16T20:07:08.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.909+0000 7ff811ffb640 1 -- 172.21.15.88:0/1099458602 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff8140b6cf0 con 0x7ff81c143da0 2023-10-16T20:07:08.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.909+0000 7ff811ffb640 1 --2- 172.21.15.88:0/1099458602 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff8000664a0 0x7ff800068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:08.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.909+0000 7ff811ffb640 1 -- 172.21.15.88:0/1099458602 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7ff81411f1b0 con 0x7ff81c143da0 2023-10-16T20:07:08.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.909+0000 7ff821794640 1 --2- 172.21.15.88:0/1099458602 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff8000664a0 0x7ff800068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:08.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.909+0000 7ff821794640 1 --2- 172.21.15.88:0/1099458602 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff8000664a0 0x7ff800068960 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff818004630 tx=0x7ff818008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:08.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:08.913+0000 7ff811ffb640 1 -- 172.21.15.88:0/1099458602 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff81c14f0e0 con 0x7ff81c143da0 2023-10-16T20:07:09.098 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.093+0000 7ff8239f8640 1 -- 172.21.15.88:0/1099458602 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff81c14f0e0 con 0x7ff81c143da0 2023-10-16T20:07:09.099 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.093+0000 7ff811ffb640 1 -- 172.21.15.88:0/1099458602 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4917 (secure 0 0 0) 0x7ff81c14f0e0 con 0x7ff81c143da0 2023-10-16T20:07:09.099 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:07:09.100 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/696292235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":696292235},{"type":"v1","addr":"172.21.15.94:6834","nonce":696292235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/2152672724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2152672724},{"type":"v1","addr":"172.21.15.88:6835","nonce":2152672724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3107974207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":3107974207},{"type":"v1","addr":"172.21.15.88:6837","nonce":3107974207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T20:06:17.088386+0000","modified":"2023-10-16T20:06:31.579567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5535":{"gid":5535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6835/2190281591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":2190281591},{"type":"v1","addr":"172.21.15.94:6835","nonce":2190281591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"testcephfs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-16T20:07:09.100 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 19 2023-10-16T20:07:09.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.097+0000 7ff8239f8640 1 -- 172.21.15.88:0/1099458602 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff8000664a0 msgr2=0x7ff800068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:09.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.097+0000 7ff8239f8640 1 --2- 172.21.15.88:0/1099458602 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff8000664a0 0x7ff800068960 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff818004630 tx=0x7ff818008040 comp rx=0 tx=0).stop 2023-10-16T20:07:09.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.097+0000 7ff8239f8640 1 -- 172.21.15.88:0/1099458602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff81c143da0 msgr2=0x7ff81c0c4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:09.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.097+0000 7ff8239f8640 1 --2- 172.21.15.88:0/1099458602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff81c143da0 0x7ff81c0c4750 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff8140af040 tx=0x7ff8140197b0 comp rx=0 tx=0).stop 2023-10-16T20:07:09.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.097+0000 7ff8239f8640 1 -- 172.21.15.88:0/1099458602 shutdown_connections 2023-10-16T20:07:09.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.097+0000 7ff8239f8640 1 --2- 172.21.15.88:0/1099458602 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff8000664a0 0x7ff800068960 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:09.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.097+0000 7ff8239f8640 1 --2- 172.21.15.88:0/1099458602 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff81c15cc60 0x7ff81c143860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:09.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.097+0000 7ff8239f8640 1 --2- 172.21.15.88:0/1099458602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff81c143da0 0x7ff81c0c4750 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:09.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.097+0000 7ff8239f8640 1 --2- 172.21.15.88:0/1099458602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff81c071450 0x7ff81c143130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:09.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.097+0000 7ff8239f8640 1 -- 172.21.15.88:0/1099458602 >> 172.21.15.88:0/1099458602 conn(0x7ff81c0ceb20 msgr2=0x7ff81c0adb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:09.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.097+0000 7ff8239f8640 1 -- 172.21.15.88:0/1099458602 shutdown_connections 2023-10-16T20:07:09.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.097+0000 7ff8239f8640 1 -- 172.21.15.88:0/1099458602 wait complete. 2023-10-16T20:07:09.120 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:07:09.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.253+0000 7f901cb3e640 1 Processor -- start 2023-10-16T20:07:09.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.253+0000 7f901cb3e640 1 -- start start 2023-10-16T20:07:09.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.253+0000 7f901cb3e640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f90180d22e0 0x7f9018161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:09.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.253+0000 7f901cb3e640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90180b87e0 con 0x7f90180d26c0 2023-10-16T20:07:09.261 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.253+0000 7f901cb3e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f901815df10 con 0x7f901815d9f0 2023-10-16T20:07:09.261 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.253+0000 7f901cb3e640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f901815e1f0 con 0x7f90180d22e0 2023-10-16T20:07:09.261 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.253+0000 7f901659c640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f90180d22e0 0x7f9018161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:09.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.253+0000 7f901659c640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f90180d22e0 0x7f9018161450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57932/0 (socket says 172.21.15.88:57932) 2023-10-16T20:07:09.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.253+0000 7f901659c640 1 -- 172.21.15.88:0/619809779 learned_addr learned my addr 172.21.15.88:0/619809779 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:09.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.253+0000 7f901659c640 1 -- 172.21.15.88:0/619809779 >> v1:172.21.15.94:6789/0 conn(0x7f901815d9f0 legacy=0x7f90180cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:09.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.253+0000 7f901659c640 1 -- 172.21.15.88:0/619809779 >> v1:172.21.15.88:6789/0 conn(0x7f90180d26c0 legacy=0x7f901815d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:09.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.253+0000 7f901659c640 1 -- 172.21.15.88:0/619809779 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9018160970 con 0x7f90180d22e0 2023-10-16T20:07:09.263 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.253+0000 7f9015d9b640 1 -- 172.21.15.88:0/619809779 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 794296006 0 0) 0x7f90180b87e0 con 0x7f90180d26c0 2023-10-16T20:07:09.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.257+0000 7f901659c640 1 --2- 172.21.15.88:0/619809779 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90180d22e0 0x7f9018161450 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9008009e50 tx=0x7f900801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2c40afbfe582df server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:09.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.257+0000 7f9015d9b640 1 -- 172.21.15.88:0/619809779 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90080a5720 con 0x7f90180d22e0 2023-10-16T20:07:09.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.257+0000 7f9015d9b640 1 -- 172.21.15.88:0/619809779 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90080a5880 con 0x7f90180d22e0 2023-10-16T20:07:09.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.257+0000 7f9015d9b640 1 -- 172.21.15.88:0/619809779 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90080a5c10 con 0x7f90180d22e0 2023-10-16T20:07:09.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.257+0000 7f901cb3e640 1 -- 172.21.15.88:0/619809779 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90180d22e0 msgr2=0x7f9018161450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:09.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.257+0000 7f901cb3e640 1 --2- 172.21.15.88:0/619809779 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90180d22e0 0x7f9018161450 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9008009e50 tx=0x7f900801c950 comp rx=0 tx=0).stop 2023-10-16T20:07:09.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.257+0000 7f901cb3e640 1 -- 172.21.15.88:0/619809779 shutdown_connections 2023-10-16T20:07:09.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.257+0000 7f901cb3e640 1 --2- 172.21.15.88:0/619809779 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90180d22e0 0x7f9018161450 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:09.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.257+0000 7f901cb3e640 1 -- 172.21.15.88:0/619809779 >> 172.21.15.88:0/619809779 conn(0x7f90180715c0 msgr2=0x7f90180719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:09.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.257+0000 7f901cb3e640 1 -- 172.21.15.88:0/619809779 shutdown_connections 2023-10-16T20:07:09.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.257+0000 7f901cb3e640 1 -- 172.21.15.88:0/619809779 wait complete. 2023-10-16T20:07:09.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.261+0000 7f901cb3e640 1 Processor -- start 2023-10-16T20:07:09.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.261+0000 7f901cb3e640 1 -- start start 2023-10-16T20:07:09.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.261+0000 7f901cb3e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90180d22e0 0x7f9018151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:09.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.261+0000 7f901cb3e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f90180d26c0 0x7f90181579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:09.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.261+0000 7f901cb3e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f901815d9f0 0x7f9018157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:09.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.261+0000 7f901cb3e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9018156cf0 con 0x7f901815d9f0 2023-10-16T20:07:09.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.261+0000 7f901cb3e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9018152550 con 0x7f90180d22e0 2023-10-16T20:07:09.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.261+0000 7f901cb3e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9018156f40 con 0x7f90180d26c0 2023-10-16T20:07:09.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.261+0000 7f9016d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f901815d9f0 0x7f9018157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:09.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.261+0000 7f9016d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f901815d9f0 0x7f9018157ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57936/0 (socket says 172.21.15.88:57936) 2023-10-16T20:07:09.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.261+0000 7f9016d9d640 1 -- 172.21.15.88:0/964797397 learned_addr learned my addr 172.21.15.88:0/964797397 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:09.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.261+0000 7f901659c640 1 --2- 172.21.15.88:0/964797397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90180d22e0 0x7f9018151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:09.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.261+0000 7f900dd9b640 1 --2- 172.21.15.88:0/964797397 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f90180d26c0 0x7f90181579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:09.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.261+0000 7f9016d9d640 1 -- 172.21.15.88:0/964797397 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f90180d26c0 msgr2=0x7f90181579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:09.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.261+0000 7f9016d9d640 1 --2- 172.21.15.88:0/964797397 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f90180d26c0 0x7f90181579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:09.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.261+0000 7f9016d9d640 1 -- 172.21.15.88:0/964797397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90180d22e0 msgr2=0x7f9018151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:09.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.261+0000 7f9016d9d640 1 --2- 172.21.15.88:0/964797397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90180d22e0 0x7f9018151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:09.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.261+0000 7f9016d9d640 1 -- 172.21.15.88:0/964797397 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9008009b30 con 0x7f901815d9f0 2023-10-16T20:07:09.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.261+0000 7f900dd9b640 1 --2- 172.21.15.88:0/964797397 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f90180d26c0 0x7f90181579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:07:09.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.261+0000 7f9016d9d640 1 --2- 172.21.15.88:0/964797397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f901815d9f0 0x7f9018157ee0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f901000c340 tx=0x7f901000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:09.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.261+0000 7f900f7fe640 1 -- 172.21.15.88:0/964797397 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9010005220 con 0x7f901815d9f0 2023-10-16T20:07:09.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.261+0000 7f900f7fe640 1 -- 172.21.15.88:0/964797397 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9010005380 con 0x7f901815d9f0 2023-10-16T20:07:09.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.261+0000 7f900f7fe640 1 -- 172.21.15.88:0/964797397 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9010007270 con 0x7f901815d9f0 2023-10-16T20:07:09.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.261+0000 7f901cb3e640 1 -- 172.21.15.88:0/964797397 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f901815e330 con 0x7f901815d9f0 2023-10-16T20:07:09.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.261+0000 7f901cb3e640 1 -- 172.21.15.88:0/964797397 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f901815e8f0 con 0x7f901815d9f0 2023-10-16T20:07:09.273 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.265+0000 7f901cb3e640 1 -- 172.21.15.88:0/964797397 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f901804e9c0 con 0x7f901815d9f0 2023-10-16T20:07:09.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.265+0000 7f900f7fe640 1 -- 172.21.15.88:0/964797397 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f90100073d0 con 0x7f901815d9f0 2023-10-16T20:07:09.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.265+0000 7f900f7fe640 1 --2- 172.21.15.88:0/964797397 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f8ff80664a0 0x7f8ff8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:09.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.265+0000 7f900f7fe640 1 -- 172.21.15.88:0/964797397 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f9010083610 con 0x7f901815d9f0 2023-10-16T20:07:09.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.265+0000 7f901659c640 1 --2- 172.21.15.88:0/964797397 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f8ff80664a0 0x7f8ff8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:09.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.265+0000 7f901659c640 1 --2- 172.21.15.88:0/964797397 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f8ff80664a0 0x7f8ff8068960 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f90080ae000 tx=0x7f90080a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:09.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.269+0000 7f900f7fe640 1 -- 172.21.15.88:0/964797397 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9010050780 con 0x7f901815d9f0 2023-10-16T20:07:09.426 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.421+0000 7f901cb3e640 1 -- 172.21.15.88:0/964797397 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90180b84a0 con 0x7f901815d9f0 2023-10-16T20:07:09.427 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.421+0000 7f900f7fe640 1 -- 172.21.15.88:0/964797397 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+18499 (secure 0 0 0) 0x7f9010053da0 con 0x7f901815d9f0 2023-10-16T20:07:09.427 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:07:09.428 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":46,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:06:38.250351+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T20:06:14.743500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"testcephfs1_data","create_time":"2023-10-16T20:06:16.091837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-10-16T20:06:31.184462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.39","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:07:09.429 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.425+0000 7f901cb3e640 1 -- 172.21.15.88:0/964797397 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f8ff80664a0 msgr2=0x7f8ff8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:09.429 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.425+0000 7f901cb3e640 1 --2- 172.21.15.88:0/964797397 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f8ff80664a0 0x7f8ff8068960 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f90080ae000 tx=0x7f90080a7070 comp rx=0 tx=0).stop 2023-10-16T20:07:09.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.425+0000 7f901cb3e640 1 -- 172.21.15.88:0/964797397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f901815d9f0 msgr2=0x7f9018157ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:09.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.425+0000 7f901cb3e640 1 --2- 172.21.15.88:0/964797397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f901815d9f0 0x7f9018157ee0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f901000c340 tx=0x7f901000c800 comp rx=0 tx=0).stop 2023-10-16T20:07:09.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.425+0000 7f901cb3e640 1 -- 172.21.15.88:0/964797397 shutdown_connections 2023-10-16T20:07:09.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.425+0000 7f901cb3e640 1 --2- 172.21.15.88:0/964797397 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f8ff80664a0 0x7f8ff8068960 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:09.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.425+0000 7f901cb3e640 1 --2- 172.21.15.88:0/964797397 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f90180d26c0 0x7f90181579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:09.431 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.425+0000 7f901cb3e640 1 --2- 172.21.15.88:0/964797397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f901815d9f0 0x7f9018157ee0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:09.431 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.425+0000 7f901cb3e640 1 --2- 172.21.15.88:0/964797397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90180d22e0 0x7f9018151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:09.431 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.425+0000 7f901cb3e640 1 -- 172.21.15.88:0/964797397 >> 172.21.15.88:0/964797397 conn(0x7f90180715c0 msgr2=0x7f90180719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:09.431 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.425+0000 7f901cb3e640 1 -- 172.21.15.88:0/964797397 shutdown_connections 2023-10-16T20:07:09.431 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.425+0000 7f901cb3e640 1 -- 172.21.15.88:0/964797397 wait complete. 2023-10-16T20:07:09.449 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'testcephfs1_metadata', 'create_time': '2023-10-16T20:06:14.743500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'testcephfs1_data', 'create_time': '2023-10-16T20:06:16.091837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2023-10-16T20:06:31.184462+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T20:07:09.450 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:07:09.592 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.589+0000 7f36440d6640 1 Processor -- start 2023-10-16T20:07:09.592 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.589+0000 7f36440d6640 1 -- start start 2023-10-16T20:07:09.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.589+0000 7f36440d6640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f363c0cac00 0x7f363c0cd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:09.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.589+0000 7f36440d6640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f363c070730 0x7f363c070bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:09.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.589+0000 7f36440d6640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f363c0c4870 con 0x7f363c15cc80 2023-10-16T20:07:09.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.589+0000 7f36440d6640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f363c0cd8b0 con 0x7f363c070730 2023-10-16T20:07:09.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.589+0000 7f36440d6640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f363c0cb060 con 0x7f363c0cac00 2023-10-16T20:07:09.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.589+0000 7f3642673640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f363c070730 0x7f363c070bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:09.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.589+0000 7f3642673640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f363c070730 0x7f363c070bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57946/0 (socket says 172.21.15.88:57946) 2023-10-16T20:07:09.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.589+0000 7f3642673640 1 -- 172.21.15.88:0/2749338766 learned_addr learned my addr 172.21.15.88:0/2749338766 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:09.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.589+0000 7f3641671640 1 --2- 172.21.15.88:0/2749338766 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f363c0cac00 0x7f363c0cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:09.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.589+0000 7f3642673640 1 -- 172.21.15.88:0/2749338766 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f363c0cac00 msgr2=0x7f363c0cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:09.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.589+0000 7f3642673640 1 --2- 172.21.15.88:0/2749338766 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f363c0cac00 0x7f363c0cd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:09.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.589+0000 7f3642673640 1 -- 172.21.15.88:0/2749338766 >> v1:172.21.15.94:6789/0 conn(0x7f363c15cc80 legacy=0x7f363c06c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:09.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.589+0000 7f3642673640 1 -- 172.21.15.88:0/2749338766 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363c0717c0 con 0x7f363c070730 2023-10-16T20:07:09.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.593+0000 7f3642673640 1 --2- 172.21.15.88:0/2749338766 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f363c070730 0x7f363c070bc0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f3634009e50 tx=0x7f363401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b9fb61c76899fa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:09.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.593+0000 7f3640e70640 1 -- 172.21.15.88:0/2749338766 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36340a56b0 con 0x7f363c070730 2023-10-16T20:07:09.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.593+0000 7f3640e70640 1 -- 172.21.15.88:0/2749338766 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36340a5810 con 0x7f363c070730 2023-10-16T20:07:09.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.593+0000 7f3640e70640 1 -- 172.21.15.88:0/2749338766 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36340a5ba0 con 0x7f363c070730 2023-10-16T20:07:09.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.593+0000 7f36440d6640 1 -- 172.21.15.88:0/2749338766 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f363c070730 msgr2=0x7f363c070bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:09.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.593+0000 7f36440d6640 1 --2- 172.21.15.88:0/2749338766 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f363c070730 0x7f363c070bc0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f3634009e50 tx=0x7f363401c940 comp rx=0 tx=0).stop 2023-10-16T20:07:09.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.593+0000 7f36440d6640 1 -- 172.21.15.88:0/2749338766 shutdown_connections 2023-10-16T20:07:09.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.593+0000 7f36440d6640 1 --2- 172.21.15.88:0/2749338766 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f363c070730 0x7f363c070bc0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:09.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.593+0000 7f36440d6640 1 --2- 172.21.15.88:0/2749338766 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f363c0cac00 0x7f363c0cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:09.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.593+0000 7f36440d6640 1 -- 172.21.15.88:0/2749338766 >> 172.21.15.88:0/2749338766 conn(0x7f363c0ceb40 msgr2=0x7f363c0cef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:09.600 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.593+0000 7f36440d6640 1 -- 172.21.15.88:0/2749338766 shutdown_connections 2023-10-16T20:07:09.600 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.593+0000 7f36440d6640 1 -- 172.21.15.88:0/2749338766 wait complete. 2023-10-16T20:07:09.601 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.597+0000 7f36440d6640 1 Processor -- start 2023-10-16T20:07:09.602 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.597+0000 7f36440d6640 1 -- start start 2023-10-16T20:07:09.602 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.597+0000 7f36440d6640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f363c070730 0x7f363c158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:09.602 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.597+0000 7f36440d6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f363c0cac00 0x7f363c14d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:09.602 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.597+0000 7f36440d6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f363c15cc80 0x7f363c1f5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:09.602 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.597+0000 7f36440d6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c158b40 con 0x7f363c15cc80 2023-10-16T20:07:09.603 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.597+0000 7f36440d6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c14d7c0 con 0x7f363c0cac00 2023-10-16T20:07:09.603 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.597+0000 7f36440d6640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f363c157580 con 0x7f363c070730 2023-10-16T20:07:09.603 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.597+0000 7f3642673640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f363c15cc80 0x7f363c1f5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:09.603 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.597+0000 7f3642673640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f363c15cc80 0x7f363c1f5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57956/0 (socket says 172.21.15.88:57956) 2023-10-16T20:07:09.603 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.597+0000 7f3642673640 1 -- 172.21.15.88:0/1659243006 learned_addr learned my addr 172.21.15.88:0/1659243006 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:09.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.597+0000 7f3641e72640 1 --2- 172.21.15.88:0/1659243006 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f363c070730 0x7f363c158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:09.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.597+0000 7f3641671640 1 --2- 172.21.15.88:0/1659243006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f363c0cac00 0x7f363c14d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:09.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.597+0000 7f3642673640 1 -- 172.21.15.88:0/1659243006 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f363c070730 msgr2=0x7f363c158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:09.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.597+0000 7f3642673640 1 --2- 172.21.15.88:0/1659243006 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f363c070730 0x7f363c158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:09.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.597+0000 7f3642673640 1 -- 172.21.15.88:0/1659243006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f363c0cac00 msgr2=0x7f363c14d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:09.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.597+0000 7f3642673640 1 --2- 172.21.15.88:0/1659243006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f363c0cac00 0x7f363c14d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:09.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.597+0000 7f3642673640 1 -- 172.21.15.88:0/1659243006 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3634009b30 con 0x7f363c15cc80 2023-10-16T20:07:09.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.597+0000 7f3642673640 1 --2- 172.21.15.88:0/1659243006 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f363c15cc80 0x7f363c1f5390 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f36340af040 tx=0x7f36340196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:09.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.597+0000 7f36327fc640 1 -- 172.21.15.88:0/1659243006 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36340b8670 con 0x7f363c15cc80 2023-10-16T20:07:09.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.597+0000 7f36327fc640 1 -- 172.21.15.88:0/1659243006 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36340b87d0 con 0x7f363c15cc80 2023-10-16T20:07:09.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.597+0000 7f36440d6640 1 -- 172.21.15.88:0/1659243006 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f363c157840 con 0x7f363c15cc80 2023-10-16T20:07:09.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.597+0000 7f36327fc640 1 -- 172.21.15.88:0/1659243006 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36340b8b80 con 0x7f363c15cc80 2023-10-16T20:07:09.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.597+0000 7f36440d6640 1 -- 172.21.15.88:0/1659243006 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f363c157e00 con 0x7f363c15cc80 2023-10-16T20:07:09.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.597+0000 7f36440d6640 1 -- 172.21.15.88:0/1659243006 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f363c04e9c0 con 0x7f363c15cc80 2023-10-16T20:07:09.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.601+0000 7f36327fc640 1 -- 172.21.15.88:0/1659243006 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f36340b8ce0 con 0x7f363c15cc80 2023-10-16T20:07:09.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.601+0000 7f36327fc640 1 --2- 172.21.15.88:0/1659243006 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f36200664a0 0x7f3620068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:09.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.601+0000 7f36327fc640 1 -- 172.21.15.88:0/1659243006 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f363411f6e0 con 0x7f363c15cc80 2023-10-16T20:07:09.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.601+0000 7f3641e72640 1 --2- 172.21.15.88:0/1659243006 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f36200664a0 0x7f3620068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:09.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.601+0000 7f3641e72640 1 --2- 172.21.15.88:0/1659243006 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f36200664a0 0x7f3620068960 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f36380098e0 tx=0x7f3638009060 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:09.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.605+0000 7f36327fc640 1 -- 172.21.15.88:0/1659243006 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f36340ec990 con 0x7f363c15cc80 2023-10-16T20:07:09.758 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.753+0000 7f36440d6640 1 -- 172.21.15.88:0/1659243006 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f363c0cbc00 con 0x7f363c15cc80 2023-10-16T20:07:09.758 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.753+0000 7f36327fc640 1 -- 172.21.15.88:0/1659243006 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f36340effb0 con 0x7f363c15cc80 2023-10-16T20:07:09.759 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:07:09.760 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.757+0000 7f36440d6640 1 -- 172.21.15.88:0/1659243006 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f36200664a0 msgr2=0x7f3620068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:09.760 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.757+0000 7f36440d6640 1 --2- 172.21.15.88:0/1659243006 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f36200664a0 0x7f3620068960 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f36380098e0 tx=0x7f3638009060 comp rx=0 tx=0).stop 2023-10-16T20:07:09.760 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.757+0000 7f36440d6640 1 -- 172.21.15.88:0/1659243006 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f363c15cc80 msgr2=0x7f363c1f5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:09.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.757+0000 7f36440d6640 1 --2- 172.21.15.88:0/1659243006 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f363c15cc80 0x7f363c1f5390 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f36340af040 tx=0x7f36340196e0 comp rx=0 tx=0).stop 2023-10-16T20:07:09.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.757+0000 7f36440d6640 1 -- 172.21.15.88:0/1659243006 shutdown_connections 2023-10-16T20:07:09.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.757+0000 7f36440d6640 1 --2- 172.21.15.88:0/1659243006 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f36200664a0 0x7f3620068960 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:09.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.757+0000 7f36440d6640 1 --2- 172.21.15.88:0/1659243006 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f363c15cc80 0x7f363c1f5390 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:09.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.757+0000 7f36440d6640 1 --2- 172.21.15.88:0/1659243006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f363c0cac00 0x7f363c14d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:09.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.757+0000 7f36440d6640 1 --2- 172.21.15.88:0/1659243006 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f363c070730 0x7f363c158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:09.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.757+0000 7f36440d6640 1 -- 172.21.15.88:0/1659243006 >> 172.21.15.88:0/1659243006 conn(0x7f363c0ceb40 msgr2=0x7f363c0c39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:09.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.757+0000 7f36440d6640 1 -- 172.21.15.88:0/1659243006 shutdown_connections 2023-10-16T20:07:09.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.757+0000 7f36440d6640 1 -- 172.21.15.88:0/1659243006 wait complete. 2023-10-16T20:07:09.779 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_metadata pg_num 2023-10-16T20:07:09.919 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.913+0000 7f22e9de4640 1 Processor -- start 2023-10-16T20:07:09.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.913+0000 7f22e9de4640 1 -- start start 2023-10-16T20:07:09.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.917+0000 7f22e9de4640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f22e415c8d0 0x7f22e40b2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:09.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.917+0000 7f22e9de4640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f22e40cd050 0x7f22e40cd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:09.922 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.917+0000 7f22e9de4640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e415d510 con 0x7f22e415ccb0 2023-10-16T20:07:09.922 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.917+0000 7f22e9de4640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e406cde0 con 0x7f22e40cd050 2023-10-16T20:07:09.922 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.917+0000 7f22e9de4640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e40b3060 con 0x7f22e415c8d0 2023-10-16T20:07:09.922 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.917+0000 7f22e95e3640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f22e40cd050 0x7f22e40cd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:09.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.917+0000 7f22e95e3640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f22e40cd050 0x7f22e40cd4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57972/0 (socket says 172.21.15.88:57972) 2023-10-16T20:07:09.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.917+0000 7f22e95e3640 1 -- 172.21.15.88:0/522780822 learned_addr learned my addr 172.21.15.88:0/522780822 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:09.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.917+0000 7f22e8de2640 1 --2- 172.21.15.88:0/522780822 >> v2:172.21.15.94:3300/0 conn(0x7f22e415c8d0 0x7f22e40b2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:09.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.917+0000 7f22e95e3640 1 -- 172.21.15.88:0/522780822 >> v2:172.21.15.94:3300/0 conn(0x7f22e415c8d0 msgr2=0x7f22e40b2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:09.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.917+0000 7f22e95e3640 1 --2- 172.21.15.88:0/522780822 >> v2:172.21.15.94:3300/0 conn(0x7f22e415c8d0 0x7f22e40b2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:09.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.917+0000 7f22e95e3640 1 -- 172.21.15.88:0/522780822 >> v1:172.21.15.94:6789/0 conn(0x7f22e415ccb0 legacy=0x7f22e40d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:09.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.917+0000 7f22e95e3640 1 -- 172.21.15.88:0/522780822 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e415d7f0 con 0x7f22e40cd050 2023-10-16T20:07:09.925 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.921+0000 7f22e95e3640 1 --2- 172.21.15.88:0/522780822 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22e40cd050 0x7f22e40cd4c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f22e40cdd50 tx=0x7f22dc01c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977c63b570fd19e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:09.925 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.921+0000 7f22db7fe640 1 -- 172.21.15.88:0/522780822 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc0a5780 con 0x7f22e40cd050 2023-10-16T20:07:09.926 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.921+0000 7f22db7fe640 1 -- 172.21.15.88:0/522780822 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22dc0a58e0 con 0x7f22e40cd050 2023-10-16T20:07:09.926 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.921+0000 7f22db7fe640 1 -- 172.21.15.88:0/522780822 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc0a5c70 con 0x7f22e40cd050 2023-10-16T20:07:09.926 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.921+0000 7f22e9de4640 1 -- 172.21.15.88:0/522780822 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22e40cd050 msgr2=0x7f22e40cd4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:09.926 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.921+0000 7f22e9de4640 1 --2- 172.21.15.88:0/522780822 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22e40cd050 0x7f22e40cd4c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f22e40cdd50 tx=0x7f22dc01c9d0 comp rx=0 tx=0).stop 2023-10-16T20:07:09.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.921+0000 7f22e9de4640 1 -- 172.21.15.88:0/522780822 shutdown_connections 2023-10-16T20:07:09.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.921+0000 7f22e9de4640 1 --2- 172.21.15.88:0/522780822 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22e40cd050 0x7f22e40cd4c0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:09.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.921+0000 7f22e9de4640 1 --2- 172.21.15.88:0/522780822 >> v2:172.21.15.94:3300/0 conn(0x7f22e415c8d0 0x7f22e40b2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:09.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.921+0000 7f22e9de4640 1 -- 172.21.15.88:0/522780822 >> 172.21.15.88:0/522780822 conn(0x7f22e4069e90 msgr2=0x7f22e406a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:09.927 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.921+0000 7f22e9de4640 1 -- 172.21.15.88:0/522780822 shutdown_connections 2023-10-16T20:07:09.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.921+0000 7f22e9de4640 1 -- 172.21.15.88:0/522780822 wait complete. 2023-10-16T20:07:09.930 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.921+0000 7f22e9de4640 1 Processor -- start 2023-10-16T20:07:09.930 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.921+0000 7f22e9de4640 1 -- start start 2023-10-16T20:07:09.930 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.925+0000 7f22e9de4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22e40cd050 0x7f22e41599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:09.931 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.925+0000 7f22e9de4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22e415c8d0 0x7f22e40b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:09.931 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.925+0000 7f22e9de4640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f22e415ccb0 0x7f22e40b5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:09.931 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.925+0000 7f22e9de4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e415b0f0 con 0x7f22e415c8d0 2023-10-16T20:07:09.931 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.925+0000 7f22e9de4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e415a470 con 0x7f22e40cd050 2023-10-16T20:07:09.932 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.925+0000 7f22e9de4640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e415a750 con 0x7f22e415ccb0 2023-10-16T20:07:09.932 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.925+0000 7f22dbfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22e415c8d0 0x7f22e40b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:09.932 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.925+0000 7f22dbfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22e415c8d0 0x7f22e40b50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57976/0 (socket says 172.21.15.88:57976) 2023-10-16T20:07:09.932 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.925+0000 7f22dbfff640 1 -- 172.21.15.88:0/4007676514 learned_addr learned my addr 172.21.15.88:0/4007676514 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:09.932 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.925+0000 7f22e8de2640 1 --2- 172.21.15.88:0/4007676514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22e40cd050 0x7f22e41599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:09.933 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.925+0000 7f22e95e3640 1 --2- 172.21.15.88:0/4007676514 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f22e415ccb0 0x7f22e40b5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:09.933 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.925+0000 7f22dbfff640 1 -- 172.21.15.88:0/4007676514 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f22e415ccb0 msgr2=0x7f22e40b5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:09.933 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.925+0000 7f22dbfff640 1 --2- 172.21.15.88:0/4007676514 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f22e415ccb0 0x7f22e40b5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:09.933 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.925+0000 7f22dbfff640 1 -- 172.21.15.88:0/4007676514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22e40cd050 msgr2=0x7f22e41599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:09.933 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.925+0000 7f22dbfff640 1 --2- 172.21.15.88:0/4007676514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22e40cd050 0x7f22e41599f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:09.934 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.925+0000 7f22dbfff640 1 -- 172.21.15.88:0/4007676514 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22dc009b30 con 0x7f22e415c8d0 2023-10-16T20:07:09.934 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.925+0000 7f22dbfff640 1 --2- 172.21.15.88:0/4007676514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22e415c8d0 0x7f22e40b50d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f22d400c3c0 tx=0x7f22d400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:09.934 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.925+0000 7f22d97fa640 1 -- 172.21.15.88:0/4007676514 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d40052d0 con 0x7f22e415c8d0 2023-10-16T20:07:09.934 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.925+0000 7f22d97fa640 1 -- 172.21.15.88:0/4007676514 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d4005430 con 0x7f22e415c8d0 2023-10-16T20:07:09.935 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.925+0000 7f22d97fa640 1 -- 172.21.15.88:0/4007676514 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d4003840 con 0x7f22e415c8d0 2023-10-16T20:07:09.935 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.925+0000 7f22e9de4640 1 -- 172.21.15.88:0/4007676514 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e415aa30 con 0x7f22e415c8d0 2023-10-16T20:07:09.935 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.925+0000 7f22e9de4640 1 -- 172.21.15.88:0/4007676514 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e41454d0 con 0x7f22e415c8d0 2023-10-16T20:07:09.935 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.925+0000 7f22e9de4640 1 -- 172.21.15.88:0/4007676514 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e406a400 con 0x7f22e415c8d0 2023-10-16T20:07:09.936 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.925+0000 7f22d97fa640 1 -- 172.21.15.88:0/4007676514 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f22d40039a0 con 0x7f22e415c8d0 2023-10-16T20:07:09.937 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.929+0000 7f22d97fa640 1 --2- 172.21.15.88:0/4007676514 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f22d00664a0 0x7f22d0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:09.937 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.929+0000 7f22d97fa640 1 -- 172.21.15.88:0/4007676514 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f22d40835c0 con 0x7f22e415c8d0 2023-10-16T20:07:09.937 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.929+0000 7f22e8de2640 1 --2- 172.21.15.88:0/4007676514 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f22d00664a0 0x7f22d0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:09.937 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.929+0000 7f22e8de2640 1 --2- 172.21.15.88:0/4007676514 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f22d00664a0 0x7f22d0068960 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f22e41563f0 tx=0x7f22e0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:09.937 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:09.933+0000 7f22d97fa640 1 -- 172.21.15.88:0/4007676514 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f22d4050880 con 0x7f22e415c8d0 2023-10-16T20:07:10.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.081+0000 7f22e9de4640 1 -- 172.21.15.88:0/4007676514 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"} v 0) v1 -- 0x7f22e40d1000 con 0x7f22e415c8d0 2023-10-16T20:07:10.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.081+0000 7f22d97fa640 1 -- 172.21.15.88:0/4007676514 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"}]=0 v46) v1 ==== 109+0+11 (secure 0 0 0) 0x7f22d4053ea0 con 0x7f22e415c8d0 2023-10-16T20:07:10.084 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:07:10.086 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.081+0000 7f22e9de4640 1 -- 172.21.15.88:0/4007676514 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f22d00664a0 msgr2=0x7f22d0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:10.086 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.081+0000 7f22e9de4640 1 --2- 172.21.15.88:0/4007676514 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f22d00664a0 0x7f22d0068960 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f22e41563f0 tx=0x7f22e0008040 comp rx=0 tx=0).stop 2023-10-16T20:07:10.086 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.081+0000 7f22e9de4640 1 -- 172.21.15.88:0/4007676514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22e415c8d0 msgr2=0x7f22e40b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:10.087 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.081+0000 7f22e9de4640 1 --2- 172.21.15.88:0/4007676514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22e415c8d0 0x7f22e40b50d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f22d400c3c0 tx=0x7f22d400c880 comp rx=0 tx=0).stop 2023-10-16T20:07:10.087 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.081+0000 7f22e9de4640 1 -- 172.21.15.88:0/4007676514 shutdown_connections 2023-10-16T20:07:10.087 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.081+0000 7f22e9de4640 1 --2- 172.21.15.88:0/4007676514 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f22d00664a0 0x7f22d0068960 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:10.087 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.081+0000 7f22e9de4640 1 --2- 172.21.15.88:0/4007676514 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f22e415ccb0 0x7f22e40b5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:10.087 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.081+0000 7f22e9de4640 1 --2- 172.21.15.88:0/4007676514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22e415c8d0 0x7f22e40b50d0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:10.088 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.081+0000 7f22e9de4640 1 --2- 172.21.15.88:0/4007676514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22e40cd050 0x7f22e41599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:10.088 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.081+0000 7f22e9de4640 1 -- 172.21.15.88:0/4007676514 >> 172.21.15.88:0/4007676514 conn(0x7f22e4069e90 msgr2=0x7f22e40b1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:10.088 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.081+0000 7f22e9de4640 1 -- 172.21.15.88:0/4007676514 shutdown_connections 2023-10-16T20:07:10.088 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.081+0000 7f22e9de4640 1 -- 172.21.15.88:0/4007676514 wait complete. 2023-10-16T20:07:10.105 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_data pg_num 2023-10-16T20:07:10.246 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.241+0000 7f9f675f8640 1 Processor -- start 2023-10-16T20:07:10.246 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.241+0000 7f9f675f8640 1 -- start start 2023-10-16T20:07:10.247 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.241+0000 7f9f675f8640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9f600b0da0 0x7f9f60159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:10.247 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.241+0000 7f9f675f8640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f9f6015a1f0 0x7f9f6006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:10.247 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.241+0000 7f9f675f8640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f6015ad30 con 0x7f9f600b4260 2023-10-16T20:07:10.248 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.241+0000 7f9f675f8640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f600b4640 con 0x7f9f6015a1f0 2023-10-16T20:07:10.248 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.241+0000 7f9f675f8640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6015a770 con 0x7f9f600b0da0 2023-10-16T20:07:10.249 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.245+0000 7f9f64b93640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9f600b0da0 0x7f9f60159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:10.249 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.245+0000 7f9f65b95640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f9f6015a1f0 0x7f9f6006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:10.249 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.245+0000 7f9f64b93640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9f600b0da0 0x7f9f60159cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:35500/0 (socket says 172.21.15.88:35500) 2023-10-16T20:07:10.249 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.245+0000 7f9f64b93640 1 -- 172.21.15.88:0/1987901640 learned_addr learned my addr 172.21.15.88:0/1987901640 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:10.249 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.245+0000 7f9f65394640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f9f600b4260 0x7f9f6015dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:60200/0 (socket says 172.21.15.88:60200) 2023-10-16T20:07:10.250 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.245+0000 7f9f64b93640 1 -- 172.21.15.88:0/1987901640 >> v2:172.21.15.94:3300/0 conn(0x7f9f6015a1f0 msgr2=0x7f9f6006d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:10.250 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.245+0000 7f9f64b93640 1 --2- 172.21.15.88:0/1987901640 >> v2:172.21.15.94:3300/0 conn(0x7f9f6015a1f0 0x7f9f6006d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:10.250 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.245+0000 7f9f64b93640 1 -- 172.21.15.88:0/1987901640 >> v1:172.21.15.94:6789/0 conn(0x7f9f600b4260 legacy=0x7f9f6015dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:10.250 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.245+0000 7f9f64b93640 1 -- 172.21.15.88:0/1987901640 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f6015aa50 con 0x7f9f600b0da0 2023-10-16T20:07:10.251 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.245+0000 7f9f64b93640 1 --2- 172.21.15.88:0/1987901640 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9f600b0da0 0x7f9f60159cb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9f50009870 tx=0x7f9f5001e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ff8a628bae8ce5d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:10.251 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.245+0000 7f9f57fff640 1 -- 172.21.15.88:0/1987901640 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f500ac430 con 0x7f9f600b0da0 2023-10-16T20:07:10.252 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.249+0000 7f9f57fff640 1 -- 172.21.15.88:0/1987901640 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f500ac590 con 0x7f9f600b0da0 2023-10-16T20:07:10.252 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.249+0000 7f9f57fff640 1 -- 172.21.15.88:0/1987901640 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f500ac920 con 0x7f9f600b0da0 2023-10-16T20:07:10.253 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.249+0000 7f9f675f8640 1 -- 172.21.15.88:0/1987901640 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9f600b0da0 msgr2=0x7f9f60159cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:10.253 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.249+0000 7f9f675f8640 1 --2- 172.21.15.88:0/1987901640 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9f600b0da0 0x7f9f60159cb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9f50009870 tx=0x7f9f5001e6c0 comp rx=0 tx=0).stop 2023-10-16T20:07:10.253 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.249+0000 7f9f675f8640 1 -- 172.21.15.88:0/1987901640 shutdown_connections 2023-10-16T20:07:10.253 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.249+0000 7f9f675f8640 1 --2- 172.21.15.88:0/1987901640 >> v2:172.21.15.94:3300/0 conn(0x7f9f6015a1f0 0x7f9f6006d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:10.253 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.249+0000 7f9f675f8640 1 --2- 172.21.15.88:0/1987901640 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9f600b0da0 0x7f9f60159cb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:10.254 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.249+0000 7f9f675f8640 1 -- 172.21.15.88:0/1987901640 >> 172.21.15.88:0/1987901640 conn(0x7f9f600cb080 msgr2=0x7f9f600cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:10.254 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.249+0000 7f9f675f8640 1 -- 172.21.15.88:0/1987901640 shutdown_connections 2023-10-16T20:07:10.254 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.249+0000 7f9f675f8640 1 -- 172.21.15.88:0/1987901640 wait complete. 2023-10-16T20:07:10.255 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.249+0000 7f9f675f8640 1 Processor -- start 2023-10-16T20:07:10.255 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.249+0000 7f9f675f8640 1 -- start start 2023-10-16T20:07:10.256 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.249+0000 7f9f675f8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f600b0da0 0x7f9f6014aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:10.256 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.249+0000 7f9f675f8640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9f600b4260 0x7f9f601545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:10.256 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.249+0000 7f9f675f8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f6015a1f0 0x7f9f6006c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:10.256 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.249+0000 7f9f675f8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f601639c0 con 0x7f9f6015a1f0 2023-10-16T20:07:10.256 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.249+0000 7f9f675f8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6014afa0 con 0x7f9f600b0da0 2023-10-16T20:07:10.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.249+0000 7f9f675f8640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f60154b00 con 0x7f9f600b4260 2023-10-16T20:07:10.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.249+0000 7f9f65b95640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f6015a1f0 0x7f9f6006c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:10.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.249+0000 7f9f65b95640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f6015a1f0 0x7f9f6006c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57980/0 (socket says 172.21.15.88:57980) 2023-10-16T20:07:10.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.249+0000 7f9f65b95640 1 -- 172.21.15.88:0/1445308700 learned_addr learned my addr 172.21.15.88:0/1445308700 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:10.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.253+0000 7f9f65394640 1 --2- 172.21.15.88:0/1445308700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f600b0da0 0x7f9f6014aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:10.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.253+0000 7f9f64b93640 1 --2- 172.21.15.88:0/1445308700 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9f600b4260 0x7f9f601545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:10.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.253+0000 7f9f65b95640 1 -- 172.21.15.88:0/1445308700 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9f600b4260 msgr2=0x7f9f601545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:10.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.253+0000 7f9f65b95640 1 --2- 172.21.15.88:0/1445308700 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9f600b4260 0x7f9f601545c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:10.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.253+0000 7f9f65b95640 1 -- 172.21.15.88:0/1445308700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f600b0da0 msgr2=0x7f9f6014aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:10.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.253+0000 7f9f65b95640 1 --2- 172.21.15.88:0/1445308700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f600b0da0 0x7f9f6014aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:10.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.253+0000 7f9f65b95640 1 -- 172.21.15.88:0/1445308700 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f500094d0 con 0x7f9f6015a1f0 2023-10-16T20:07:10.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.253+0000 7f9f65b95640 1 --2- 172.21.15.88:0/1445308700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f6015a1f0 0x7f9f6006c6a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f9f5800efc0 tx=0x7f9f5800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:10.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.253+0000 7f9f55ffb640 1 -- 172.21.15.88:0/1445308700 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f58009ba0 con 0x7f9f6015a1f0 2023-10-16T20:07:10.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.253+0000 7f9f55ffb640 1 -- 172.21.15.88:0/1445308700 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5800f040 con 0x7f9f6015a1f0 2023-10-16T20:07:10.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.253+0000 7f9f675f8640 1 -- 172.21.15.88:0/1445308700 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f600cd8c0 con 0x7f9f6015a1f0 2023-10-16T20:07:10.261 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.253+0000 7f9f55ffb640 1 -- 172.21.15.88:0/1445308700 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f580146c0 con 0x7f9f6015a1f0 2023-10-16T20:07:10.261 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.253+0000 7f9f675f8640 1 -- 172.21.15.88:0/1445308700 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f601636e0 con 0x7f9f6015a1f0 2023-10-16T20:07:10.261 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.253+0000 7f9f675f8640 1 -- 172.21.15.88:0/1445308700 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f6004e9c0 con 0x7f9f6015a1f0 2023-10-16T20:07:10.261 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.253+0000 7f9f55ffb640 1 -- 172.21.15.88:0/1445308700 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9f58014820 con 0x7f9f6015a1f0 2023-10-16T20:07:10.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.257+0000 7f9f55ffb640 1 --2- 172.21.15.88:0/1445308700 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9f440664a0 0x7f9f44068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:10.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.257+0000 7f9f55ffb640 1 -- 172.21.15.88:0/1445308700 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f9f58084b60 con 0x7f9f6015a1f0 2023-10-16T20:07:10.263 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.257+0000 7f9f65394640 1 --2- 172.21.15.88:0/1445308700 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9f440664a0 0x7f9f44068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:10.263 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.257+0000 7f9f65394640 1 --2- 172.21.15.88:0/1445308700 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9f440664a0 0x7f9f44068960 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c000f70 tx=0x7f9f5c007e50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:10.263 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.257+0000 7f9f55ffb640 1 -- 172.21.15.88:0/1445308700 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9f58051cf0 con 0x7f9f6015a1f0 2023-10-16T20:07:10.414 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.409+0000 7f9f675f8640 1 -- 172.21.15.88:0/1445308700 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"} v 0) v1 -- 0x7f9f60071500 con 0x7f9f6015a1f0 2023-10-16T20:07:10.415 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.409+0000 7f9f55ffb640 1 -- 172.21.15.88:0/1445308700 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"}]=0 v46) v1 ==== 105+0+11 (secure 0 0 0) 0x7f9f58018020 con 0x7f9f6015a1f0 2023-10-16T20:07:10.415 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2023-10-16T20:07:10.416 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.413+0000 7f9f675f8640 1 -- 172.21.15.88:0/1445308700 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9f440664a0 msgr2=0x7f9f44068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:10.416 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.413+0000 7f9f675f8640 1 --2- 172.21.15.88:0/1445308700 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9f440664a0 0x7f9f44068960 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c000f70 tx=0x7f9f5c007e50 comp rx=0 tx=0).stop 2023-10-16T20:07:10.417 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.413+0000 7f9f675f8640 1 -- 172.21.15.88:0/1445308700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f6015a1f0 msgr2=0x7f9f6006c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:10.417 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.413+0000 7f9f675f8640 1 --2- 172.21.15.88:0/1445308700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f6015a1f0 0x7f9f6006c6a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f9f5800efc0 tx=0x7f9f5800c490 comp rx=0 tx=0).stop 2023-10-16T20:07:10.417 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.413+0000 7f9f675f8640 1 -- 172.21.15.88:0/1445308700 shutdown_connections 2023-10-16T20:07:10.417 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.413+0000 7f9f675f8640 1 --2- 172.21.15.88:0/1445308700 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9f440664a0 0x7f9f44068960 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:10.417 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.413+0000 7f9f675f8640 1 --2- 172.21.15.88:0/1445308700 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9f600b4260 0x7f9f601545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:10.418 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.413+0000 7f9f675f8640 1 --2- 172.21.15.88:0/1445308700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f6015a1f0 0x7f9f6006c6a0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:10.418 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.413+0000 7f9f675f8640 1 --2- 172.21.15.88:0/1445308700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f600b0da0 0x7f9f6014aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:10.418 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.413+0000 7f9f675f8640 1 -- 172.21.15.88:0/1445308700 >> 172.21.15.88:0/1445308700 conn(0x7f9f600cb080 msgr2=0x7f9f60156000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:10.418 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.413+0000 7f9f675f8640 1 -- 172.21.15.88:0/1445308700 shutdown_connections 2023-10-16T20:07:10.418 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.413+0000 7f9f675f8640 1 -- 172.21.15.88:0/1445308700 wait complete. 2023-10-16T20:07:10.435 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-10-16T20:07:10.576 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f2b90f640 1 Processor -- start 2023-10-16T20:07:10.576 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f2b90f640 1 -- start start 2023-10-16T20:07:10.577 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f2b90f640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1f2415b700 0x7f1f241653f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:10.577 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f2b90f640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f2406d2f0 con 0x7f1f240b0c10 2023-10-16T20:07:10.577 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f2b90f640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f2415bd20 con 0x7f1f24154f30 2023-10-16T20:07:10.578 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f2b90f640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f24155310 con 0x7f1f2415b700 2023-10-16T20:07:10.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f28eaa640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f1f240b0c10 0x7f1f2415aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:50776/0 (socket says 172.21.15.88:50776) 2023-10-16T20:07:10.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f28eaa640 1 -- 172.21.15.88:0/3636484847 learned_addr learned my addr 172.21.15.88:0/3636484847 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:10.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f29eac640 1 --2- 172.21.15.88:0/3636484847 >> v2:172.21.15.94:3300/0 conn(0x7f1f2415b700 0x7f1f241653f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:10.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f1bfff640 1 -- 172.21.15.88:0/3636484847 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 122863706 0 0) 0x7f1f2406d2f0 con 0x7f1f240b0c10 2023-10-16T20:07:10.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f1bfff640 1 -- 172.21.15.88:0/3636484847 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f10003530 con 0x7f1f240b0c10 2023-10-16T20:07:10.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f1bfff640 1 -- 172.21.15.88:0/3636484847 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1191519697 0 0) 0x7f1f2415bd20 con 0x7f1f24154f30 2023-10-16T20:07:10.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f1bfff640 1 -- 172.21.15.88:0/3636484847 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f2406d2f0 con 0x7f1f24154f30 2023-10-16T20:07:10.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f1bfff640 1 -- 172.21.15.88:0/3636484847 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1844622409 0 0) 0x7f1f10003530 con 0x7f1f240b0c10 2023-10-16T20:07:10.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f1bfff640 1 -- 172.21.15.88:0/3636484847 >> v2:172.21.15.94:3300/0 conn(0x7f1f2415b700 msgr2=0x7f1f241653f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:10.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f1bfff640 1 --2- 172.21.15.88:0/3636484847 >> v2:172.21.15.94:3300/0 conn(0x7f1f2415b700 0x7f1f241653f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:10.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f1bfff640 1 -- 172.21.15.88:0/3636484847 >> v1:172.21.15.94:6789/0 conn(0x7f1f24154f30 legacy=0x7f1f240b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:10.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f1bfff640 1 -- 172.21.15.88:0/3636484847 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f240702f0 con 0x7f1f240b0c10 2023-10-16T20:07:10.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f1bfff640 1 -- 172.21.15.88:0/3636484847 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f1f14001680 con 0x7f1f240b0c10 2023-10-16T20:07:10.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f1bfff640 1 -- 172.21.15.88:0/3636484847 >> v1:172.21.15.88:6789/0 conn(0x7f1f240b0c10 legacy=0x7f1f2415aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:10.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f1bfff640 1 --2- 172.21.15.88:0/3636484847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f10003b10 0x7f1f10003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:10.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f1bfff640 1 --2- 172.21.15.88:0/3636484847 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1f100047d0 0x7f1f10004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:10.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f1bfff640 1 --2- 172.21.15.88:0/3636484847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f10005190 0x7f1f1000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:10.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f1bfff640 1 -- 172.21.15.88:0/3636484847 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f24155310 con 0x7f1f10003b10 2023-10-16T20:07:10.584 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f1bfff640 1 -- 172.21.15.88:0/3636484847 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1000c010 con 0x7f1f10005190 2023-10-16T20:07:10.584 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f1bfff640 1 -- 172.21.15.88:0/3636484847 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1000c1d0 con 0x7f1f100047d0 2023-10-16T20:07:10.584 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f29eac640 1 --2- 172.21.15.88:0/3636484847 >> v2:172.21.15.94:3300/0 conn(0x7f1f2415b700 0x7f1f241653f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:07:10.584 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f296ab640 1 --2- 172.21.15.88:0/3636484847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f10003b10 0x7f1f10003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:10.584 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f28eaa640 1 --2- 172.21.15.88:0/3636484847 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1f100047d0 0x7f1f10004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:10.584 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f29eac640 1 --2- 172.21.15.88:0/3636484847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f10005190 0x7f1f1000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:10.585 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f296ab640 1 -- 172.21.15.88:0/3636484847 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1f100047d0 msgr2=0x7f1f10004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:10.585 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f296ab640 1 --2- 172.21.15.88:0/3636484847 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1f100047d0 0x7f1f10004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:10.585 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f296ab640 1 -- 172.21.15.88:0/3636484847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f10005190 msgr2=0x7f1f1000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:10.585 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f296ab640 1 --2- 172.21.15.88:0/3636484847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f10005190 0x7f1f1000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:10.585 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f296ab640 1 -- 172.21.15.88:0/3636484847 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f10003530 con 0x7f1f10003b10 2023-10-16T20:07:10.586 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f29eac640 1 --2- 172.21.15.88:0/3636484847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f10005190 0x7f1f1000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:07:10.586 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.573+0000 7f1f29eac640 1 -- 172.21.15.88:0/3636484847 reap_dead start 2023-10-16T20:07:10.589 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.577+0000 7f1f296ab640 1 --2- 172.21.15.88:0/3636484847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f10003b10 0x7f1f10003f10 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1f200018c0 tx=0x7f1f2001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e6116c6d6df73e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:10.589 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.577+0000 7f1f1bfff640 1 -- 172.21.15.88:0/3636484847 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f200052d0 con 0x7f1f10003b10 2023-10-16T20:07:10.589 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.577+0000 7f1f1bfff640 1 -- 172.21.15.88:0/3636484847 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f200a7050 con 0x7f1f10003b10 2023-10-16T20:07:10.589 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.577+0000 7f1f1bfff640 1 -- 172.21.15.88:0/3636484847 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f20003820 con 0x7f1f10003b10 2023-10-16T20:07:10.590 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.577+0000 7f1f2b90f640 1 -- 172.21.15.88:0/3636484847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f10003b10 msgr2=0x7f1f10003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:10.590 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.577+0000 7f1f2b90f640 1 --2- 172.21.15.88:0/3636484847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f10003b10 0x7f1f10003f10 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1f200018c0 tx=0x7f1f2001c4d0 comp rx=0 tx=0).stop 2023-10-16T20:07:10.590 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.577+0000 7f1f2b90f640 1 -- 172.21.15.88:0/3636484847 shutdown_connections 2023-10-16T20:07:10.590 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.577+0000 7f1f2b90f640 1 --2- 172.21.15.88:0/3636484847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f10003b10 0x7f1f10003f10 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:10.590 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.577+0000 7f1f2b90f640 1 -- 172.21.15.88:0/3636484847 >> 172.21.15.88:0/3636484847 conn(0x7f1f240cb080 msgr2=0x7f1f240cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:10.591 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.577+0000 7f1f2b90f640 1 -- 172.21.15.88:0/3636484847 shutdown_connections 2023-10-16T20:07:10.591 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.577+0000 7f1f2b90f640 1 -- 172.21.15.88:0/3636484847 wait complete. 2023-10-16T20:07:10.591 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.581+0000 7f1f2b90f640 1 Processor -- start 2023-10-16T20:07:10.591 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.581+0000 7f1f2b90f640 1 -- start start 2023-10-16T20:07:10.591 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.581+0000 7f1f2b90f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f10003b10 0x7f1f24157d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:10.592 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.581+0000 7f1f2b90f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f24142fd0 0x7f1f24155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:10.592 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.581+0000 7f1f2b90f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1f24156040 0x7f1f2414e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:10.592 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.581+0000 7f1f2b90f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f241564c0 con 0x7f1f24142fd0 2023-10-16T20:07:10.592 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.581+0000 7f1f2b90f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2414f6c0 con 0x7f1f10003b10 2023-10-16T20:07:10.592 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.581+0000 7f1f2b90f640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2414c450 con 0x7f1f24156040 2023-10-16T20:07:10.593 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.581+0000 7f1f28eaa640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f24142fd0 0x7f1f24155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:10.593 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.581+0000 7f1f28eaa640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f24142fd0 0x7f1f24155b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58004/0 (socket says 172.21.15.88:58004) 2023-10-16T20:07:10.593 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.581+0000 7f1f28eaa640 1 -- 172.21.15.88:0/350324529 learned_addr learned my addr 172.21.15.88:0/350324529 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:10.593 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.581+0000 7f1f296ab640 1 --2- 172.21.15.88:0/350324529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f10003b10 0x7f1f24157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:10.593 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.581+0000 7f1f29eac640 1 --2- 172.21.15.88:0/350324529 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1f24156040 0x7f1f2414e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:10.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.581+0000 7f1f28eaa640 1 -- 172.21.15.88:0/350324529 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1f24156040 msgr2=0x7f1f2414e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:10.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.581+0000 7f1f28eaa640 1 --2- 172.21.15.88:0/350324529 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1f24156040 0x7f1f2414e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:10.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.581+0000 7f1f28eaa640 1 -- 172.21.15.88:0/350324529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f10003b10 msgr2=0x7f1f24157d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:10.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.581+0000 7f1f28eaa640 1 --2- 172.21.15.88:0/350324529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f10003b10 0x7f1f24157d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:10.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.581+0000 7f1f28eaa640 1 -- 172.21.15.88:0/350324529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2414c730 con 0x7f1f24142fd0 2023-10-16T20:07:10.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.581+0000 7f1f28eaa640 1 --2- 172.21.15.88:0/350324529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f24142fd0 0x7f1f24155b00 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1f14001060 tx=0x7f1f140077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:10.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.581+0000 7f1f19ffb640 1 -- 172.21.15.88:0/350324529 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1400f990 con 0x7f1f24142fd0 2023-10-16T20:07:10.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.581+0000 7f1f19ffb640 1 -- 172.21.15.88:0/350324529 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f1400faf0 con 0x7f1f24142fd0 2023-10-16T20:07:10.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.581+0000 7f1f2b90f640 1 -- 172.21.15.88:0/350324529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f2414ca10 con 0x7f1f24142fd0 2023-10-16T20:07:10.595 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.581+0000 7f1f19ffb640 1 -- 172.21.15.88:0/350324529 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1400fea0 con 0x7f1f24142fd0 2023-10-16T20:07:10.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.585+0000 7f1f2b90f640 1 -- 172.21.15.88:0/350324529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f24158240 con 0x7f1f24142fd0 2023-10-16T20:07:10.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.585+0000 7f1f2b90f640 1 -- 172.21.15.88:0/350324529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f2404e9c0 con 0x7f1f24142fd0 2023-10-16T20:07:10.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.585+0000 7f1f19ffb640 1 -- 172.21.15.88:0/350324529 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1f1401d460 con 0x7f1f24142fd0 2023-10-16T20:07:10.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.585+0000 7f1f19ffb640 1 --2- 172.21.15.88:0/350324529 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1f080664d0 0x7f1f08068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:10.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.585+0000 7f1f19ffb640 1 -- 172.21.15.88:0/350324529 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f1f14084240 con 0x7f1f24142fd0 2023-10-16T20:07:10.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.585+0000 7f1f296ab640 1 --2- 172.21.15.88:0/350324529 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1f080664d0 0x7f1f08068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:10.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.585+0000 7f1f296ab640 1 --2- 172.21.15.88:0/350324529 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1f080664d0 0x7f1f08068990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1f200ac000 tx=0x7f1f20009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:10.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.589+0000 7f1f19ffb640 1 -- 172.21.15.88:0/350324529 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1f14051460 con 0x7f1f24142fd0 2023-10-16T20:07:10.742 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.737+0000 7f1f2b90f640 1 -- 172.21.15.88:0/350324529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f1f2406eb00 con 0x7f1f24142fd0 2023-10-16T20:07:10.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.737+0000 7f1f19ffb640 1 -- 172.21.15.88:0/350324529 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v46) v1 ==== 97+0+11 (secure 0 0 0) 0x7f1f14054a80 con 0x7f1f24142fd0 2023-10-16T20:07:10.743 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:07:10.745 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.741+0000 7f1f2b90f640 1 -- 172.21.15.88:0/350324529 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1f080664d0 msgr2=0x7f1f08068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:10.745 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.741+0000 7f1f2b90f640 1 --2- 172.21.15.88:0/350324529 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1f080664d0 0x7f1f08068990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1f200ac000 tx=0x7f1f20009040 comp rx=0 tx=0).stop 2023-10-16T20:07:10.745 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.741+0000 7f1f2b90f640 1 -- 172.21.15.88:0/350324529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f24142fd0 msgr2=0x7f1f24155b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:10.745 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.741+0000 7f1f2b90f640 1 --2- 172.21.15.88:0/350324529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f24142fd0 0x7f1f24155b00 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1f14001060 tx=0x7f1f140077a0 comp rx=0 tx=0).stop 2023-10-16T20:07:10.746 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.741+0000 7f1f2b90f640 1 -- 172.21.15.88:0/350324529 shutdown_connections 2023-10-16T20:07:10.746 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.741+0000 7f1f2b90f640 1 --2- 172.21.15.88:0/350324529 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1f080664d0 0x7f1f08068990 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:10.746 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.741+0000 7f1f2b90f640 1 --2- 172.21.15.88:0/350324529 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1f24156040 0x7f1f2414e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:10.746 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.741+0000 7f1f2b90f640 1 --2- 172.21.15.88:0/350324529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f24142fd0 0x7f1f24155b00 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:10.746 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.741+0000 7f1f2b90f640 1 --2- 172.21.15.88:0/350324529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f10003b10 0x7f1f24157d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:10.747 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.741+0000 7f1f2b90f640 1 -- 172.21.15.88:0/350324529 >> 172.21.15.88:0/350324529 conn(0x7f1f240cb080 msgr2=0x7f1f240cc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:10.747 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.741+0000 7f1f2b90f640 1 -- 172.21.15.88:0/350324529 shutdown_connections 2023-10-16T20:07:10.747 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.741+0000 7f1f2b90f640 1 -- 172.21.15.88:0/350324529 wait complete. 2023-10-16T20:07:10.764 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T20:07:10.901 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.897+0000 7f81d7fab640 1 Processor -- start 2023-10-16T20:07:10.901 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.897+0000 7f81d7fab640 1 -- start start 2023-10-16T20:07:10.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.897+0000 7f81d7fab640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f81d00d22e0 0x7f81d0161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:10.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.897+0000 7f81d7fab640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f81d00d26c0 0x7f81d015d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:10.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.897+0000 7f81d7fab640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81d00b87e0 con 0x7f81d015d820 2023-10-16T20:07:10.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.897+0000 7f81d7fab640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81d015dc70 con 0x7f81d00d22e0 2023-10-16T20:07:10.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.897+0000 7f81d7fab640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d015df10 con 0x7f81d00d26c0 2023-10-16T20:07:10.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.897+0000 7f81d5d47640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f81d00d22e0 0x7f81d0161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:10.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.897+0000 7f81d5d47640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f81d00d22e0 0x7f81d0161450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58018/0 (socket says 172.21.15.88:58018) 2023-10-16T20:07:10.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.897+0000 7f81d6548640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f81d015d820 0x7f81d00cf2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:50792/0 (socket says 172.21.15.88:50792) 2023-10-16T20:07:10.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.897+0000 7f81d5d47640 1 -- 172.21.15.88:0/444959221 learned_addr learned my addr 172.21.15.88:0/444959221 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:10.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.897+0000 7f81d5546640 1 --2- 172.21.15.88:0/444959221 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f81d00d26c0 0x7f81d015d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:10.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.897+0000 7f81d5d47640 1 -- 172.21.15.88:0/444959221 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f81d00d26c0 msgr2=0x7f81d015d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:10.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.897+0000 7f81d5d47640 1 --2- 172.21.15.88:0/444959221 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f81d00d26c0 0x7f81d015d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:10.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.897+0000 7f81d5d47640 1 -- 172.21.15.88:0/444959221 >> v1:172.21.15.88:6789/0 conn(0x7f81d015d820 legacy=0x7f81d00cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:10.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.897+0000 7f81d5d47640 1 -- 172.21.15.88:0/444959221 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d015e1f0 con 0x7f81d00d22e0 2023-10-16T20:07:10.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.897+0000 7f81d4d45640 1 -- 172.21.15.88:0/444959221 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3134958634 0 0) 0x7f81d00b87e0 con 0x7f81d015d820 2023-10-16T20:07:10.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.901+0000 7f81d5d47640 1 --2- 172.21.15.88:0/444959221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81d00d22e0 0x7f81d0161450 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f81cc009870 tx=0x7f81cc01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=987772f2b25bfb2e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:10.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.901+0000 7f81d4d45640 1 -- 172.21.15.88:0/444959221 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc0ad640 con 0x7f81d00d22e0 2023-10-16T20:07:10.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.901+0000 7f81d4d45640 1 -- 172.21.15.88:0/444959221 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81cc0ad7a0 con 0x7f81d00d22e0 2023-10-16T20:07:10.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.901+0000 7f81d4d45640 1 -- 172.21.15.88:0/444959221 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc0adb30 con 0x7f81d00d22e0 2023-10-16T20:07:10.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.901+0000 7f81d7fab640 1 -- 172.21.15.88:0/444959221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81d00d22e0 msgr2=0x7f81d0161450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:10.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.901+0000 7f81d7fab640 1 --2- 172.21.15.88:0/444959221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81d00d22e0 0x7f81d0161450 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f81cc009870 tx=0x7f81cc01c480 comp rx=0 tx=0).stop 2023-10-16T20:07:10.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.901+0000 7f81d7fab640 1 -- 172.21.15.88:0/444959221 shutdown_connections 2023-10-16T20:07:10.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.901+0000 7f81d7fab640 1 --2- 172.21.15.88:0/444959221 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f81d00d26c0 0x7f81d015d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:10.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.901+0000 7f81d7fab640 1 --2- 172.21.15.88:0/444959221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81d00d22e0 0x7f81d0161450 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:10.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.901+0000 7f81d7fab640 1 -- 172.21.15.88:0/444959221 >> 172.21.15.88:0/444959221 conn(0x7f81d00715c0 msgr2=0x7f81d00719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:10.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.901+0000 7f81d7fab640 1 -- 172.21.15.88:0/444959221 shutdown_connections 2023-10-16T20:07:10.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.901+0000 7f81d7fab640 1 -- 172.21.15.88:0/444959221 wait complete. 2023-10-16T20:07:10.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.905+0000 7f81d7fab640 1 Processor -- start 2023-10-16T20:07:10.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.905+0000 7f81d7fab640 1 -- start start 2023-10-16T20:07:10.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.905+0000 7f81d7fab640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81d00d22e0 0x7f81d01499f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:10.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.905+0000 7f81d7fab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81d00d26c0 0x7f81d0148310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:10.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.905+0000 7f81d7fab640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f81d015d820 0x7f81d0156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:10.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.905+0000 7f81d7fab640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d014a360 con 0x7f81d00d22e0 2023-10-16T20:07:10.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.905+0000 7f81d7fab640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d0150330 con 0x7f81d00d26c0 2023-10-16T20:07:10.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.905+0000 7f81d7fab640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d0148850 con 0x7f81d015d820 2023-10-16T20:07:10.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.905+0000 7f81d5d47640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81d00d22e0 0x7f81d01499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:10.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.905+0000 7f81d5d47640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81d00d22e0 0x7f81d01499f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58032/0 (socket says 172.21.15.88:58032) 2023-10-16T20:07:10.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.905+0000 7f81d5d47640 1 -- 172.21.15.88:0/1652122278 learned_addr learned my addr 172.21.15.88:0/1652122278 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:10.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.905+0000 7f81d5546640 1 --2- 172.21.15.88:0/1652122278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81d00d26c0 0x7f81d0148310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:10.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.905+0000 7f81d5d47640 1 -- 172.21.15.88:0/1652122278 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f81d015d820 msgr2=0x7f81d0156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:10.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.905+0000 7f81d6548640 1 --2- 172.21.15.88:0/1652122278 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f81d015d820 0x7f81d0156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:10.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.905+0000 7f81d5d47640 1 --2- 172.21.15.88:0/1652122278 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f81d015d820 0x7f81d0156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:10.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.905+0000 7f81d5d47640 1 -- 172.21.15.88:0/1652122278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81d00d26c0 msgr2=0x7f81d0148310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:10.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.905+0000 7f81d5d47640 1 --2- 172.21.15.88:0/1652122278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81d00d26c0 0x7f81d0148310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:10.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.905+0000 7f81d5d47640 1 -- 172.21.15.88:0/1652122278 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81cc0094d0 con 0x7f81d00d22e0 2023-10-16T20:07:10.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.905+0000 7f81d5546640 1 --2- 172.21.15.88:0/1652122278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81d00d26c0 0x7f81d0148310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:07:10.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.905+0000 7f81d6548640 1 --2- 172.21.15.88:0/1652122278 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f81d015d820 0x7f81d0156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:07:10.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.905+0000 7f81d5d47640 1 --2- 172.21.15.88:0/1652122278 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81d00d22e0 0x7f81d01499f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f81cc01a0b0 tx=0x7f81cc019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:10.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.905+0000 7f81c67fc640 1 -- 172.21.15.88:0/1652122278 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc0b6850 con 0x7f81d00d22e0 2023-10-16T20:07:10.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.905+0000 7f81c67fc640 1 -- 172.21.15.88:0/1652122278 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81cc0a6040 con 0x7f81d00d22e0 2023-10-16T20:07:10.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.905+0000 7f81c67fc640 1 -- 172.21.15.88:0/1652122278 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc0b6bf0 con 0x7f81d00d22e0 2023-10-16T20:07:10.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.905+0000 7f81d7fab640 1 -- 172.21.15.88:0/1652122278 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81d0148c30 con 0x7f81d00d22e0 2023-10-16T20:07:10.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.905+0000 7f81d7fab640 1 -- 172.21.15.88:0/1652122278 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81d01491f0 con 0x7f81d00d22e0 2023-10-16T20:07:10.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.909+0000 7f81d7fab640 1 -- 172.21.15.88:0/1652122278 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81d004e9c0 con 0x7f81d00d22e0 2023-10-16T20:07:10.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.909+0000 7f81c67fc640 1 -- 172.21.15.88:0/1652122278 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f81cc0b4070 con 0x7f81d00d22e0 2023-10-16T20:07:10.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.909+0000 7f81c67fc640 1 --2- 172.21.15.88:0/1652122278 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f81b40664d0 0x7f81b4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:10.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.909+0000 7f81c67fc640 1 -- 172.21.15.88:0/1652122278 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f81cc11e3f0 con 0x7f81d00d22e0 2023-10-16T20:07:10.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.909+0000 7f81d5546640 1 --2- 172.21.15.88:0/1652122278 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f81b40664d0 0x7f81b4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:10.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.909+0000 7f81d5546640 1 --2- 172.21.15.88:0/1652122278 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f81b40664d0 0x7f81b4068990 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f81d014aa20 tx=0x7f81c0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:10.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:10.913+0000 7f81c67fc640 1 -- 172.21.15.88:0/1652122278 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f81cc0eb570 con 0x7f81d00d22e0 2023-10-16T20:07:11.097 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.093+0000 7f81d7fab640 1 -- 172.21.15.88:0/1652122278 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81d00b58b0 con 0x7f81d00d22e0 2023-10-16T20:07:11.098 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.093+0000 7f81c67fc640 1 -- 172.21.15.88:0/1652122278 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4917 (secure 0 0 0) 0x7f81cc0eeb90 con 0x7f81d00d22e0 2023-10-16T20:07:11.098 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:07:11.098 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/696292235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":696292235},{"type":"v1","addr":"172.21.15.94:6834","nonce":696292235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/2152672724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2152672724},{"type":"v1","addr":"172.21.15.88:6835","nonce":2152672724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3107974207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":3107974207},{"type":"v1","addr":"172.21.15.88:6837","nonce":3107974207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T20:06:17.088386+0000","modified":"2023-10-16T20:06:31.579567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5535":{"gid":5535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6835/2190281591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":2190281591},{"type":"v1","addr":"172.21.15.94:6835","nonce":2190281591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"testcephfs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-16T20:07:11.099 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 19 2023-10-16T20:07:11.100 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.093+0000 7f81d7fab640 1 -- 172.21.15.88:0/1652122278 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f81b40664d0 msgr2=0x7f81b4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:11.100 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.093+0000 7f81d7fab640 1 --2- 172.21.15.88:0/1652122278 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f81b40664d0 0x7f81b4068990 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f81d014aa20 tx=0x7f81c0008040 comp rx=0 tx=0).stop 2023-10-16T20:07:11.100 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.093+0000 7f81d7fab640 1 -- 172.21.15.88:0/1652122278 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81d00d22e0 msgr2=0x7f81d01499f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:11.100 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.093+0000 7f81d7fab640 1 --2- 172.21.15.88:0/1652122278 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81d00d22e0 0x7f81d01499f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f81cc01a0b0 tx=0x7f81cc019220 comp rx=0 tx=0).stop 2023-10-16T20:07:11.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.093+0000 7f81d7fab640 1 -- 172.21.15.88:0/1652122278 shutdown_connections 2023-10-16T20:07:11.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.093+0000 7f81d7fab640 1 --2- 172.21.15.88:0/1652122278 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f81b40664d0 0x7f81b4068990 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:11.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.093+0000 7f81d7fab640 1 --2- 172.21.15.88:0/1652122278 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f81d015d820 0x7f81d0156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:11.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.093+0000 7f81d7fab640 1 --2- 172.21.15.88:0/1652122278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81d00d26c0 0x7f81d0148310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:11.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.093+0000 7f81d7fab640 1 --2- 172.21.15.88:0/1652122278 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81d00d22e0 0x7f81d01499f0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:11.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.093+0000 7f81d7fab640 1 -- 172.21.15.88:0/1652122278 >> 172.21.15.88:0/1652122278 conn(0x7f81d00715c0 msgr2=0x7f81d00b7910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:11.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.097+0000 7f81d7fab640 1 -- 172.21.15.88:0/1652122278 shutdown_connections 2023-10-16T20:07:11.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.097+0000 7f81d7fab640 1 -- 172.21.15.88:0/1652122278 wait complete. 2023-10-16T20:07:11.118 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:07:11.256 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.253+0000 7f538bc09640 1 Processor -- start 2023-10-16T20:07:11.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.253+0000 7f538bc09640 1 -- start start 2023-10-16T20:07:11.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.253+0000 7f538bc09640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f53840ada40 0x7f53840b1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:11.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.253+0000 7f538bc09640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5384143eb0 0x7f53840b0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:11.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.253+0000 7f538bc09640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53840b2570 con 0x7f53840ade20 2023-10-16T20:07:11.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.253+0000 7f538bc09640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53840cca50 con 0x7f53840ada40 2023-10-16T20:07:11.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.253+0000 7f538bc09640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5384144320 con 0x7f5384143eb0 2023-10-16T20:07:11.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.253+0000 7f53899a5640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f53840ada40 0x7f53840b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:11.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.253+0000 7f538a1a6640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5384143eb0 0x7f53840b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:11.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.253+0000 7f53899a5640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f53840ada40 0x7f53840b1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:34088/0 (socket says 172.21.15.88:34088) 2023-10-16T20:07:11.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.253+0000 7f53891a4640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f53840ade20 0x7f538415adb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:60206/0 (socket says 172.21.15.88:60206) 2023-10-16T20:07:11.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.253+0000 7f53899a5640 1 -- 172.21.15.88:0/2311692388 learned_addr learned my addr 172.21.15.88:0/2311692388 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:11.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.253+0000 7f538a1a6640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5384143eb0 0x7f53840b0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:35540/0 (socket says 172.21.15.88:35540) 2023-10-16T20:07:11.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.253+0000 7f53899a5640 1 -- 172.21.15.88:0/2311692388 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5384143eb0 msgr2=0x7f53840b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:11.261 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.253+0000 7f53899a5640 1 --2- 172.21.15.88:0/2311692388 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5384143eb0 0x7f53840b0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:11.261 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.253+0000 7f53899a5640 1 -- 172.21.15.88:0/2311692388 >> v1:172.21.15.94:6789/0 conn(0x7f53840ade20 legacy=0x7f538415adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:11.261 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.253+0000 7f53899a5640 1 -- 172.21.15.88:0/2311692388 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53840b1fb0 con 0x7f53840ada40 2023-10-16T20:07:11.261 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.253+0000 7f538a1a6640 1 --2- 172.21.15.88:0/2311692388 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5384143eb0 0x7f53840b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:07:11.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.257+0000 7f53899a5640 1 --2- 172.21.15.88:0/2311692388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53840ada40 0x7f53840b1920 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5380009870 tx=0x7f538001c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2f2df39e272ae0c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:11.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.257+0000 7f53889a3640 1 -- 172.21.15.88:0/2311692388 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53800ad640 con 0x7f53840ada40 2023-10-16T20:07:11.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.257+0000 7f53889a3640 1 -- 172.21.15.88:0/2311692388 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53800ad7a0 con 0x7f53840ada40 2023-10-16T20:07:11.263 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.257+0000 7f53889a3640 1 -- 172.21.15.88:0/2311692388 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53800adb30 con 0x7f53840ada40 2023-10-16T20:07:11.263 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.257+0000 7f538bc09640 1 -- 172.21.15.88:0/2311692388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53840ada40 msgr2=0x7f53840b1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:11.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.257+0000 7f538bc09640 1 --2- 172.21.15.88:0/2311692388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53840ada40 0x7f53840b1920 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5380009870 tx=0x7f538001c480 comp rx=0 tx=0).stop 2023-10-16T20:07:11.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.257+0000 7f538bc09640 1 -- 172.21.15.88:0/2311692388 shutdown_connections 2023-10-16T20:07:11.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.257+0000 7f538bc09640 1 --2- 172.21.15.88:0/2311692388 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5384143eb0 0x7f53840b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:11.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.257+0000 7f538bc09640 1 --2- 172.21.15.88:0/2311692388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53840ada40 0x7f53840b1920 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:11.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.257+0000 7f538bc09640 1 -- 172.21.15.88:0/2311692388 >> 172.21.15.88:0/2311692388 conn(0x7f53840ce5e0 msgr2=0x7f53840ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:11.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.257+0000 7f538bc09640 1 -- 172.21.15.88:0/2311692388 shutdown_connections 2023-10-16T20:07:11.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.257+0000 7f538bc09640 1 -- 172.21.15.88:0/2311692388 wait complete. 2023-10-16T20:07:11.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.261+0000 7f538bc09640 1 Processor -- start 2023-10-16T20:07:11.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.261+0000 7f538bc09640 1 -- start start 2023-10-16T20:07:11.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.261+0000 7f538bc09640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53840ada40 0x7f5384159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:11.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.261+0000 7f538bc09640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53840ade20 0x7f538415ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:11.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.261+0000 7f538bc09640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5384143eb0 0x7f5384141ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:11.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.261+0000 7f538bc09640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538415a280 con 0x7f53840ade20 2023-10-16T20:07:11.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.261+0000 7f538bc09640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53840ae4d0 con 0x7f5384143eb0 2023-10-16T20:07:11.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.261+0000 7f538bc09640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53840ae7b0 con 0x7f53840ada40 2023-10-16T20:07:11.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.261+0000 7f53891a4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53840ade20 0x7f538415ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:11.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.261+0000 7f53891a4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53840ade20 0x7f538415ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58044/0 (socket says 172.21.15.88:58044) 2023-10-16T20:07:11.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.261+0000 7f53891a4640 1 -- 172.21.15.88:0/2875238417 learned_addr learned my addr 172.21.15.88:0/2875238417 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:11.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.261+0000 7f538a1a6640 1 --2- 172.21.15.88:0/2875238417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5384143eb0 0x7f5384141ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:11.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.261+0000 7f53899a5640 1 --2- 172.21.15.88:0/2875238417 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53840ada40 0x7f5384159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:11.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.261+0000 7f53891a4640 1 -- 172.21.15.88:0/2875238417 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53840ada40 msgr2=0x7f5384159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:11.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.261+0000 7f53891a4640 1 --2- 172.21.15.88:0/2875238417 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53840ada40 0x7f5384159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:11.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.261+0000 7f53891a4640 1 -- 172.21.15.88:0/2875238417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5384143eb0 msgr2=0x7f5384141ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:11.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.261+0000 7f53891a4640 1 --2- 172.21.15.88:0/2875238417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5384143eb0 0x7f5384141ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:11.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.261+0000 7f53891a4640 1 -- 172.21.15.88:0/2875238417 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537c009a70 con 0x7f53840ade20 2023-10-16T20:07:11.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.261+0000 7f53891a4640 1 --2- 172.21.15.88:0/2875238417 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53840ade20 0x7f538415ba40 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f537400c370 tx=0x7f537400c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:11.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.261+0000 7f537a7fc640 1 -- 172.21.15.88:0/2875238417 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53740074e0 con 0x7f53840ade20 2023-10-16T20:07:11.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.261+0000 7f537a7fc640 1 -- 172.21.15.88:0/2875238417 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5374007640 con 0x7f53840ade20 2023-10-16T20:07:11.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.261+0000 7f537a7fc640 1 -- 172.21.15.88:0/2875238417 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5374004a30 con 0x7f53840ade20 2023-10-16T20:07:11.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.261+0000 7f538bc09640 1 -- 172.21.15.88:0/2875238417 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53800094d0 con 0x7f53840ade20 2023-10-16T20:07:11.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.261+0000 7f538bc09640 1 -- 172.21.15.88:0/2875238417 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53840aed70 con 0x7f53840ade20 2023-10-16T20:07:11.273 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.265+0000 7f538bc09640 1 -- 172.21.15.88:0/2875238417 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f538404e9c0 con 0x7f53840ade20 2023-10-16T20:07:11.273 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.265+0000 7f537a7fc640 1 -- 172.21.15.88:0/2875238417 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5374004b90 con 0x7f53840ade20 2023-10-16T20:07:11.273 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.265+0000 7f537a7fc640 1 --2- 172.21.15.88:0/2875238417 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f53680664a0 0x7f5368068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:11.273 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.265+0000 7f537a7fc640 1 -- 172.21.15.88:0/2875238417 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f5374083a80 con 0x7f53840ade20 2023-10-16T20:07:11.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.265+0000 7f53899a5640 1 --2- 172.21.15.88:0/2875238417 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f53680664a0 0x7f5368068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:11.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.265+0000 7f53899a5640 1 --2- 172.21.15.88:0/2875238417 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f53680664a0 0x7f5368068960 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f53800af000 tx=0x7f53800a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:11.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.269+0000 7f537a7fc640 1 -- 172.21.15.88:0/2875238417 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5374050b60 con 0x7f53840ade20 2023-10-16T20:07:11.429 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.425+0000 7f538bc09640 1 -- 172.21.15.88:0/2875238417 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5384140cd0 con 0x7f53840ade20 2023-10-16T20:07:11.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.425+0000 7f537a7fc640 1 -- 172.21.15.88:0/2875238417 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+18499 (secure 0 0 0) 0x7f5374054180 con 0x7f53840ade20 2023-10-16T20:07:11.431 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:07:11.431 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":46,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:06:38.250351+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T20:06:14.743500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"testcephfs1_data","create_time":"2023-10-16T20:06:16.091837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-10-16T20:06:31.184462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.39","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:07:11.433 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.429+0000 7f538bc09640 1 -- 172.21.15.88:0/2875238417 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f53680664a0 msgr2=0x7f5368068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:11.433 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.429+0000 7f538bc09640 1 --2- 172.21.15.88:0/2875238417 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f53680664a0 0x7f5368068960 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f53800af000 tx=0x7f53800a7070 comp rx=0 tx=0).stop 2023-10-16T20:07:11.433 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.429+0000 7f538bc09640 1 -- 172.21.15.88:0/2875238417 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53840ade20 msgr2=0x7f538415ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:11.433 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.429+0000 7f538bc09640 1 --2- 172.21.15.88:0/2875238417 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53840ade20 0x7f538415ba40 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f537400c370 tx=0x7f537400c770 comp rx=0 tx=0).stop 2023-10-16T20:07:11.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.429+0000 7f538bc09640 1 -- 172.21.15.88:0/2875238417 shutdown_connections 2023-10-16T20:07:11.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.429+0000 7f538bc09640 1 --2- 172.21.15.88:0/2875238417 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f53680664a0 0x7f5368068960 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:11.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.429+0000 7f538bc09640 1 --2- 172.21.15.88:0/2875238417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5384143eb0 0x7f5384141ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:11.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.429+0000 7f538bc09640 1 --2- 172.21.15.88:0/2875238417 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53840ade20 0x7f538415ba40 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:11.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.429+0000 7f538bc09640 1 --2- 172.21.15.88:0/2875238417 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53840ada40 0x7f5384159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:11.435 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.429+0000 7f538bc09640 1 -- 172.21.15.88:0/2875238417 >> 172.21.15.88:0/2875238417 conn(0x7f53840ce5e0 msgr2=0x7f53840b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:11.435 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.429+0000 7f538bc09640 1 -- 172.21.15.88:0/2875238417 shutdown_connections 2023-10-16T20:07:11.435 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.429+0000 7f538bc09640 1 -- 172.21.15.88:0/2875238417 wait complete. 2023-10-16T20:07:11.452 INFO:tasks.cephfs.filesystem:Destroying file system testcephfs1 and related pools 2023-10-16T20:07:11.453 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T20:07:11.591 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.585+0000 7fb950365640 1 Processor -- start 2023-10-16T20:07:11.591 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.585+0000 7fb950365640 1 -- start start 2023-10-16T20:07:11.592 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb950365640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb9480ade20 0x7fb9480b1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:11.593 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb950365640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9480b2570 con 0x7fb948143eb0 2023-10-16T20:07:11.593 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb950365640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9480cca50 con 0x7fb9480ada40 2023-10-16T20:07:11.593 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb950365640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb948144340 con 0x7fb9480ade20 2023-10-16T20:07:11.593 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb94e902640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fb948143eb0 0x7fb9480b0500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:50800/0 (socket says 172.21.15.88:50800) 2023-10-16T20:07:11.593 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb94e902640 1 -- 172.21.15.88:0/558510038 learned_addr learned my addr 172.21.15.88:0/558510038 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:11.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb94d900640 1 --2- 172.21.15.88:0/558510038 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb9480ade20 0x7fb9480b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:11.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb94d0ff640 1 -- 172.21.15.88:0/558510038 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3429020336 0 0) 0x7fb9480b2570 con 0x7fb948143eb0 2023-10-16T20:07:11.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb94d0ff640 1 -- 172.21.15.88:0/558510038 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb934003530 con 0x7fb948143eb0 2023-10-16T20:07:11.596 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb94d0ff640 1 -- 172.21.15.88:0/558510038 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1825474966 0 0) 0x7fb9480cca50 con 0x7fb9480ada40 2023-10-16T20:07:11.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb94d0ff640 1 -- 172.21.15.88:0/558510038 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9480b2570 con 0x7fb9480ada40 2023-10-16T20:07:11.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb94d0ff640 1 -- 172.21.15.88:0/558510038 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 542416908 0 0) 0x7fb934003530 con 0x7fb948143eb0 2023-10-16T20:07:11.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb94d0ff640 1 -- 172.21.15.88:0/558510038 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb9480ade20 msgr2=0x7fb9480b1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:11.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb94d0ff640 1 --2- 172.21.15.88:0/558510038 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb9480ade20 0x7fb9480b1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:11.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb94d0ff640 1 -- 172.21.15.88:0/558510038 >> v1:172.21.15.94:6789/0 conn(0x7fb9480ada40 legacy=0x7fb94815adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:11.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb94d0ff640 1 -- 172.21.15.88:0/558510038 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9480b1fb0 con 0x7fb948143eb0 2023-10-16T20:07:11.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb94d0ff640 1 -- 172.21.15.88:0/558510038 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7fb940001a00 con 0x7fb948143eb0 2023-10-16T20:07:11.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb94d900640 1 --2- 172.21.15.88:0/558510038 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb9480ade20 0x7fb9480b1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:07:11.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb94d0ff640 1 -- 172.21.15.88:0/558510038 >> v1:172.21.15.88:6789/0 conn(0x7fb948143eb0 legacy=0x7fb9480b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:11.598 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb94d0ff640 1 --2- 172.21.15.88:0/558510038 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb934003b40 0x7fb934003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:11.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb94d0ff640 1 --2- 172.21.15.88:0/558510038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9340047d0 0x7fb934004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:11.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb94d0ff640 1 --2- 172.21.15.88:0/558510038 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb9480ade20 0x7fb934005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:11.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb94d0ff640 1 -- 172.21.15.88:0/558510038 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb948144340 con 0x7fb934003b40 2023-10-16T20:07:11.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb94d0ff640 1 -- 172.21.15.88:0/558510038 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93400ba70 con 0x7fb9340047d0 2023-10-16T20:07:11.599 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb94d0ff640 1 -- 172.21.15.88:0/558510038 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb93400bc30 con 0x7fb9480ade20 2023-10-16T20:07:11.600 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb94e101640 1 --2- 172.21.15.88:0/558510038 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb934003b40 0x7fb934003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:11.600 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb94d900640 1 --2- 172.21.15.88:0/558510038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9340047d0 0x7fb934004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:11.600 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb94e902640 1 --2- 172.21.15.88:0/558510038 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb9480ade20 0x7fb934005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:11.600 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb94e101640 1 -- 172.21.15.88:0/558510038 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb9480ade20 msgr2=0x7fb934005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:11.600 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb94e101640 1 --2- 172.21.15.88:0/558510038 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb9480ade20 0x7fb934005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:11.601 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb94e101640 1 -- 172.21.15.88:0/558510038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9340047d0 msgr2=0x7fb934004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:11.601 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb94e101640 1 --2- 172.21.15.88:0/558510038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9340047d0 0x7fb934004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:11.601 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb94e101640 1 -- 172.21.15.88:0/558510038 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb938009420 con 0x7fb934003b40 2023-10-16T20:07:11.601 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.589+0000 7fb94d900640 1 --2- 172.21.15.88:0/558510038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9340047d0 0x7fb934004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:07:11.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.593+0000 7fb94e101640 1 --2- 172.21.15.88:0/558510038 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb934003b40 0x7fb934003f40 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb944007fd0 tx=0x7fb94401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4556d7d97bc4911 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:11.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.593+0000 7fb94d0ff640 1 -- 172.21.15.88:0/558510038 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9440052d0 con 0x7fb934003b40 2023-10-16T20:07:11.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.593+0000 7fb94d0ff640 1 -- 172.21.15.88:0/558510038 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb944005450 con 0x7fb934003b40 2023-10-16T20:07:11.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.593+0000 7fb94d0ff640 1 -- 172.21.15.88:0/558510038 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb944003840 con 0x7fb934003b40 2023-10-16T20:07:11.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.593+0000 7fb950365640 1 -- 172.21.15.88:0/558510038 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb934003b40 msgr2=0x7fb934003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:11.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.593+0000 7fb950365640 1 --2- 172.21.15.88:0/558510038 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb934003b40 0x7fb934003f40 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb944007fd0 tx=0x7fb94401c4f0 comp rx=0 tx=0).stop 2023-10-16T20:07:11.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.593+0000 7fb94e902640 1 -- 172.21.15.88:0/558510038 reap_dead start 2023-10-16T20:07:11.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.593+0000 7fb950365640 1 -- 172.21.15.88:0/558510038 shutdown_connections 2023-10-16T20:07:11.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.593+0000 7fb950365640 1 -- 172.21.15.88:0/558510038 >> 172.21.15.88:0/558510038 conn(0x7fb9480ce5e0 msgr2=0x7fb9480ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:11.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.593+0000 7fb950365640 1 -- 172.21.15.88:0/558510038 shutdown_connections 2023-10-16T20:07:11.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.593+0000 7fb950365640 1 -- 172.21.15.88:0/558510038 wait complete. 2023-10-16T20:07:11.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.597+0000 7fb950365640 1 Processor -- start 2023-10-16T20:07:11.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.597+0000 7fb950365640 1 -- start start 2023-10-16T20:07:11.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.597+0000 7fb950365640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb934003b40 0x7fb94815a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:11.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.597+0000 7fb950365640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb9481418f0 0x7fb948141d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:11.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.597+0000 7fb950365640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb94815c810 0x7fb94815cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:11.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.597+0000 7fb950365640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9481422b0 con 0x7fb94815c810 2023-10-16T20:07:11.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.597+0000 7fb950365640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94815b4f0 con 0x7fb934003b40 2023-10-16T20:07:11.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.597+0000 7fb950365640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94815bd90 con 0x7fb9481418f0 2023-10-16T20:07:11.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.597+0000 7fb94e902640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb94815c810 0x7fb94815cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:11.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.597+0000 7fb94e902640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb94815c810 0x7fb94815cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58066/0 (socket says 172.21.15.88:58066) 2023-10-16T20:07:11.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.597+0000 7fb94e902640 1 -- 172.21.15.88:0/1408957647 learned_addr learned my addr 172.21.15.88:0/1408957647 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:11.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.597+0000 7fb94e101640 1 --2- 172.21.15.88:0/1408957647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb934003b40 0x7fb94815a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:11.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.597+0000 7fb94d900640 1 --2- 172.21.15.88:0/1408957647 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb9481418f0 0x7fb948141d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:11.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.597+0000 7fb94e902640 1 -- 172.21.15.88:0/1408957647 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb9481418f0 msgr2=0x7fb948141d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:11.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.597+0000 7fb94e902640 1 --2- 172.21.15.88:0/1408957647 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb9481418f0 0x7fb948141d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:11.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.597+0000 7fb94e902640 1 -- 172.21.15.88:0/1408957647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb934003b40 msgr2=0x7fb94815a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:11.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.597+0000 7fb94e902640 1 --2- 172.21.15.88:0/1408957647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb934003b40 0x7fb94815a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:11.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.597+0000 7fb94e902640 1 -- 172.21.15.88:0/1408957647 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94815d580 con 0x7fb94815c810 2023-10-16T20:07:11.610 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.597+0000 7fb94e902640 1 --2- 172.21.15.88:0/1408957647 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb94815c810 0x7fb94815cc70 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb934004750 tx=0x7fb940007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:11.610 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.597+0000 7fb93effd640 1 -- 172.21.15.88:0/1408957647 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb940001440 con 0x7fb94815c810 2023-10-16T20:07:11.610 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.597+0000 7fb93effd640 1 -- 172.21.15.88:0/1408957647 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94000abf0 con 0x7fb94815c810 2023-10-16T20:07:11.610 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.597+0000 7fb93effd640 1 -- 172.21.15.88:0/1408957647 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9400133f0 con 0x7fb94815c810 2023-10-16T20:07:11.610 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.597+0000 7fb950365640 1 -- 172.21.15.88:0/1408957647 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94815b740 con 0x7fb94815c810 2023-10-16T20:07:11.611 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.597+0000 7fb950365640 1 -- 172.21.15.88:0/1408957647 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb94815a580 con 0x7fb94815c810 2023-10-16T20:07:11.611 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.597+0000 7fb950365640 1 -- 172.21.15.88:0/1408957647 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb94815bf20 con 0x7fb94815c810 2023-10-16T20:07:11.611 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.601+0000 7fb93effd640 1 -- 172.21.15.88:0/1408957647 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb940013550 con 0x7fb94815c810 2023-10-16T20:07:11.612 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.601+0000 7fb93effd640 1 --2- 172.21.15.88:0/1408957647 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb92c0664a0 0x7fb92c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:11.612 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.601+0000 7fb93effd640 1 -- 172.21.15.88:0/1408957647 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7fb940084650 con 0x7fb94815c810 2023-10-16T20:07:11.612 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.601+0000 7fb94e101640 1 --2- 172.21.15.88:0/1408957647 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb92c0664a0 0x7fb92c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:11.612 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.601+0000 7fb94e101640 1 --2- 172.21.15.88:0/1408957647 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb92c0664a0 0x7fb92c068960 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb9440ad000 tx=0x7fb9440a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:11.613 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.605+0000 7fb93effd640 1 -- 172.21.15.88:0/1408957647 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb9400517e0 con 0x7fb94815c810 2023-10-16T20:07:11.789 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.785+0000 7fb950365640 1 -- 172.21.15.88:0/1408957647 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb94804e9c0 con 0x7fb94815c810 2023-10-16T20:07:11.790 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.785+0000 7fb93effd640 1 -- 172.21.15.88:0/1408957647 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4917 (secure 0 0 0) 0x7fb940054e00 con 0x7fb94815c810 2023-10-16T20:07:11.791 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:07:11.791 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/696292235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":696292235},{"type":"v1","addr":"172.21.15.94:6834","nonce":696292235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/2152672724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2152672724},{"type":"v1","addr":"172.21.15.88:6835","nonce":2152672724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3107974207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":3107974207},{"type":"v1","addr":"172.21.15.88:6837","nonce":3107974207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T20:06:17.088386+0000","modified":"2023-10-16T20:06:31.579567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5535":{"gid":5535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6835/2190281591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":2190281591},{"type":"v1","addr":"172.21.15.94:6835","nonce":2190281591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"testcephfs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-16T20:07:11.791 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 19 2023-10-16T20:07:11.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.789+0000 7fb950365640 1 -- 172.21.15.88:0/1408957647 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb92c0664a0 msgr2=0x7fb92c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:11.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.789+0000 7fb950365640 1 --2- 172.21.15.88:0/1408957647 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb92c0664a0 0x7fb92c068960 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb9440ad000 tx=0x7fb9440a7000 comp rx=0 tx=0).stop 2023-10-16T20:07:11.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.789+0000 7fb950365640 1 -- 172.21.15.88:0/1408957647 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb94815c810 msgr2=0x7fb94815cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:11.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.789+0000 7fb950365640 1 --2- 172.21.15.88:0/1408957647 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb94815c810 0x7fb94815cc70 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb934004750 tx=0x7fb940007bd0 comp rx=0 tx=0).stop 2023-10-16T20:07:11.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.789+0000 7fb950365640 1 -- 172.21.15.88:0/1408957647 shutdown_connections 2023-10-16T20:07:11.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.789+0000 7fb950365640 1 --2- 172.21.15.88:0/1408957647 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb92c0664a0 0x7fb92c068960 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:11.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.789+0000 7fb950365640 1 --2- 172.21.15.88:0/1408957647 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb9481418f0 0x7fb948141d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:11.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.789+0000 7fb950365640 1 --2- 172.21.15.88:0/1408957647 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb94815c810 0x7fb94815cc70 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:11.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.789+0000 7fb950365640 1 --2- 172.21.15.88:0/1408957647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb934003b40 0x7fb94815a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:11.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.789+0000 7fb950365640 1 -- 172.21.15.88:0/1408957647 >> 172.21.15.88:0/1408957647 conn(0x7fb9480ce5e0 msgr2=0x7fb9480ae2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:11.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.789+0000 7fb950365640 1 -- 172.21.15.88:0/1408957647 shutdown_connections 2023-10-16T20:07:11.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.789+0000 7fb950365640 1 -- 172.21.15.88:0/1408957647 wait complete. 2023-10-16T20:07:11.811 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T20:07:11.948 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.945+0000 7fbb2369c640 1 Processor -- start 2023-10-16T20:07:11.949 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.945+0000 7fbb2369c640 1 -- start start 2023-10-16T20:07:11.950 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.945+0000 7fbb2369c640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb1c15d9f0 0x7fbb1c161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:11.950 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.945+0000 7fbb2369c640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fbb1c0d26c0 0x7fbb1c0cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:11.951 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.945+0000 7fbb2369c640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb1c0b87e0 con 0x7fbb1c0d22e0 2023-10-16T20:07:11.951 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.945+0000 7fbb2369c640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb1c15df10 con 0x7fbb1c0d26c0 2023-10-16T20:07:11.951 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.945+0000 7fbb2369c640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c15e1f0 con 0x7fbb1c15d9f0 2023-10-16T20:07:11.951 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.945+0000 7fbb21438640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fbb1c0d22e0 0x7fbb1c15d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:50802/0 (socket says 172.21.15.88:50802) 2023-10-16T20:07:11.952 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.945+0000 7fbb21438640 1 -- 172.21.15.88:0/1007289430 learned_addr learned my addr 172.21.15.88:0/1007289430 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:11.952 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.945+0000 7fbb21c39640 1 --2- 172.21.15.88:0/1007289430 >> v2:172.21.15.88:3300/0 conn(0x7fbb1c0d26c0 0x7fbb1c0cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:11.952 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.945+0000 7fbb20c37640 1 --2- 172.21.15.88:0/1007289430 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb1c15d9f0 0x7fbb1c161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:11.952 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.945+0000 7fbb13fff640 1 -- 172.21.15.88:0/1007289430 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 40728295 0 0) 0x7fbb1c0b87e0 con 0x7fbb1c0d22e0 2023-10-16T20:07:11.952 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.945+0000 7fbb21c39640 1 -- 172.21.15.88:0/1007289430 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb1c15d9f0 msgr2=0x7fbb1c161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:11.952 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.945+0000 7fbb21c39640 1 --2- 172.21.15.88:0/1007289430 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb1c15d9f0 0x7fbb1c161450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:11.953 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.945+0000 7fbb21c39640 1 -- 172.21.15.88:0/1007289430 >> v1:172.21.15.88:6789/0 conn(0x7fbb1c0d22e0 legacy=0x7fbb1c15d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:11.953 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.945+0000 7fbb21c39640 1 -- 172.21.15.88:0/1007289430 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1c160970 con 0x7fbb1c0d26c0 2023-10-16T20:07:11.953 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.945+0000 7fbb20c37640 1 --2- 172.21.15.88:0/1007289430 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb1c15d9f0 0x7fbb1c161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:07:11.955 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.949+0000 7fbb21c39640 1 --2- 172.21.15.88:0/1007289430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb1c0d26c0 0x7fbb1c0cf2e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fbb14009d90 tx=0x7fbb1401c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53ccd66e0b189cf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:11.955 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.949+0000 7fbb13fff640 1 -- 172.21.15.88:0/1007289430 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb140a5950 con 0x7fbb1c0d26c0 2023-10-16T20:07:11.955 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.949+0000 7fbb13fff640 1 -- 172.21.15.88:0/1007289430 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb140a5ab0 con 0x7fbb1c0d26c0 2023-10-16T20:07:11.955 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.949+0000 7fbb13fff640 1 -- 172.21.15.88:0/1007289430 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb140a5e40 con 0x7fbb1c0d26c0 2023-10-16T20:07:11.955 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.949+0000 7fbb2369c640 1 -- 172.21.15.88:0/1007289430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb1c0d26c0 msgr2=0x7fbb1c0cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:11.956 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.949+0000 7fbb2369c640 1 --2- 172.21.15.88:0/1007289430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb1c0d26c0 0x7fbb1c0cf2e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fbb14009d90 tx=0x7fbb1401c890 comp rx=0 tx=0).stop 2023-10-16T20:07:11.956 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.949+0000 7fbb2369c640 1 -- 172.21.15.88:0/1007289430 shutdown_connections 2023-10-16T20:07:11.956 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.949+0000 7fbb2369c640 1 --2- 172.21.15.88:0/1007289430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb1c0d26c0 0x7fbb1c0cf2e0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:11.956 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.949+0000 7fbb2369c640 1 --2- 172.21.15.88:0/1007289430 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb1c15d9f0 0x7fbb1c161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:11.957 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.949+0000 7fbb2369c640 1 -- 172.21.15.88:0/1007289430 >> 172.21.15.88:0/1007289430 conn(0x7fbb1c0715c0 msgr2=0x7fbb1c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:11.957 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.949+0000 7fbb2369c640 1 -- 172.21.15.88:0/1007289430 shutdown_connections 2023-10-16T20:07:11.957 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.949+0000 7fbb2369c640 1 -- 172.21.15.88:0/1007289430 wait complete. 2023-10-16T20:07:11.959 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.953+0000 7fbb2369c640 1 Processor -- start 2023-10-16T20:07:11.959 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.953+0000 7fbb2369c640 1 -- start start 2023-10-16T20:07:11.960 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.953+0000 7fbb2369c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb1c0d22e0 0x7fbb1c0ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:11.960 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.953+0000 7fbb2369c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb1c0d26c0 0x7fbb1c153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:11.960 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.953+0000 7fbb2369c640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb1c15d9f0 0x7fbb1c0cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:11.960 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.953+0000 7fbb2369c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c0c7160 con 0x7fbb1c0d26c0 2023-10-16T20:07:11.960 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.953+0000 7fbb2369c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c153690 con 0x7fbb1c0d22e0 2023-10-16T20:07:11.961 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.953+0000 7fbb2369c640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c0cbe40 con 0x7fbb1c15d9f0 2023-10-16T20:07:11.961 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.953+0000 7fbb20c37640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb1c0d26c0 0x7fbb1c153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:11.961 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.953+0000 7fbb20c37640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb1c0d26c0 0x7fbb1c153150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58084/0 (socket says 172.21.15.88:58084) 2023-10-16T20:07:11.961 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.953+0000 7fbb20c37640 1 -- 172.21.15.88:0/1546890153 learned_addr learned my addr 172.21.15.88:0/1546890153 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:11.961 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.953+0000 7fbb21438640 1 --2- 172.21.15.88:0/1546890153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb1c0d22e0 0x7fbb1c0ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:11.962 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.953+0000 7fbb21c39640 1 --2- 172.21.15.88:0/1546890153 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb1c15d9f0 0x7fbb1c0cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:11.962 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.953+0000 7fbb20c37640 1 -- 172.21.15.88:0/1546890153 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb1c15d9f0 msgr2=0x7fbb1c0cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:11.962 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.953+0000 7fbb20c37640 1 --2- 172.21.15.88:0/1546890153 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb1c15d9f0 0x7fbb1c0cb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:11.962 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.953+0000 7fbb20c37640 1 -- 172.21.15.88:0/1546890153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb1c0d22e0 msgr2=0x7fbb1c0ca8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:11.962 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.953+0000 7fbb20c37640 1 --2- 172.21.15.88:0/1546890153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb1c0d22e0 0x7fbb1c0ca8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:11.963 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.953+0000 7fbb20c37640 1 -- 172.21.15.88:0/1546890153 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb14009a70 con 0x7fbb1c0d26c0 2023-10-16T20:07:11.963 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.953+0000 7fbb20c37640 1 --2- 172.21.15.88:0/1546890153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb1c0d26c0 0x7fbb1c153150 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c00d830 tx=0x7fbb0c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:11.963 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.953+0000 7fbb11ffb640 1 -- 172.21.15.88:0/1546890153 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c0096a0 con 0x7fbb1c0d26c0 2023-10-16T20:07:11.963 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.953+0000 7fbb11ffb640 1 -- 172.21.15.88:0/1546890153 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb0c009800 con 0x7fbb1c0d26c0 2023-10-16T20:07:11.963 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.953+0000 7fbb2369c640 1 -- 172.21.15.88:0/1546890153 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb1c0c6850 con 0x7fbb1c0d26c0 2023-10-16T20:07:11.964 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.953+0000 7fbb11ffb640 1 -- 172.21.15.88:0/1546890153 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c009bb0 con 0x7fbb1c0d26c0 2023-10-16T20:07:11.964 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.953+0000 7fbb2369c640 1 -- 172.21.15.88:0/1546890153 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb1c0c6df0 con 0x7fbb1c0d26c0 2023-10-16T20:07:11.964 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.957+0000 7fbb2369c640 1 -- 172.21.15.88:0/1546890153 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1c04e9c0 con 0x7fbb1c0d26c0 2023-10-16T20:07:11.965 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.957+0000 7fbb11ffb640 1 -- 172.21.15.88:0/1546890153 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbb0c01e460 con 0x7fbb1c0d26c0 2023-10-16T20:07:11.965 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.957+0000 7fbb11ffb640 1 --2- 172.21.15.88:0/1546890153 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbb000664a0 0x7fbb00068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:11.966 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.957+0000 7fbb11ffb640 1 -- 172.21.15.88:0/1546890153 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7fbb0c0848f0 con 0x7fbb1c0d26c0 2023-10-16T20:07:11.966 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.957+0000 7fbb21438640 1 --2- 172.21.15.88:0/1546890153 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbb000664a0 0x7fbb00068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:11.966 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.957+0000 7fbb21438640 1 --2- 172.21.15.88:0/1546890153 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbb000664a0 0x7fbb00068960 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbb18000f70 tx=0x7fbb18009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:11.966 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:11.961+0000 7fbb11ffb640 1 -- 172.21.15.88:0/1546890153 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbb0c051a60 con 0x7fbb1c0d26c0 2023-10-16T20:07:12.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.141+0000 7fbb2369c640 1 -- 172.21.15.88:0/1546890153 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb1c0d3ad0 con 0x7fbb1c0d26c0 2023-10-16T20:07:12.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.145+0000 7fbb11ffb640 1 -- 172.21.15.88:0/1546890153 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4917 (secure 0 0 0) 0x7fbb0c055080 con 0x7fbb1c0d26c0 2023-10-16T20:07:12.148 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:07:12.148 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/696292235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":696292235},{"type":"v1","addr":"172.21.15.94:6834","nonce":696292235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/2152672724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2152672724},{"type":"v1","addr":"172.21.15.88:6835","nonce":2152672724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3107974207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":3107974207},{"type":"v1","addr":"172.21.15.88:6837","nonce":3107974207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T20:06:17.088386+0000","modified":"2023-10-16T20:06:31.579567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5535":{"gid":5535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6835/2190281591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":2190281591},{"type":"v1","addr":"172.21.15.94:6835","nonce":2190281591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"testcephfs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-16T20:07:12.149 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 19 2023-10-16T20:07:12.150 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.145+0000 7fbb2369c640 1 -- 172.21.15.88:0/1546890153 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbb000664a0 msgr2=0x7fbb00068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:12.150 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.145+0000 7fbb2369c640 1 --2- 172.21.15.88:0/1546890153 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbb000664a0 0x7fbb00068960 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbb18000f70 tx=0x7fbb18009040 comp rx=0 tx=0).stop 2023-10-16T20:07:12.150 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.145+0000 7fbb2369c640 1 -- 172.21.15.88:0/1546890153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb1c0d26c0 msgr2=0x7fbb1c153150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:12.150 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.145+0000 7fbb2369c640 1 --2- 172.21.15.88:0/1546890153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb1c0d26c0 0x7fbb1c153150 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c00d830 tx=0x7fbb0c00dcf0 comp rx=0 tx=0).stop 2023-10-16T20:07:12.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.145+0000 7fbb2369c640 1 -- 172.21.15.88:0/1546890153 shutdown_connections 2023-10-16T20:07:12.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.145+0000 7fbb2369c640 1 --2- 172.21.15.88:0/1546890153 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbb000664a0 0x7fbb00068960 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:12.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.145+0000 7fbb2369c640 1 --2- 172.21.15.88:0/1546890153 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb1c15d9f0 0x7fbb1c0cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:12.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.145+0000 7fbb2369c640 1 --2- 172.21.15.88:0/1546890153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb1c0d26c0 0x7fbb1c153150 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:12.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.145+0000 7fbb2369c640 1 --2- 172.21.15.88:0/1546890153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb1c0d22e0 0x7fbb1c0ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:12.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.145+0000 7fbb2369c640 1 -- 172.21.15.88:0/1546890153 >> 172.21.15.88:0/1546890153 conn(0x7fbb1c0715c0 msgr2=0x7fbb1c15fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:12.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.145+0000 7fbb2369c640 1 -- 172.21.15.88:0/1546890153 shutdown_connections 2023-10-16T20:07:12.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.145+0000 7fbb2369c640 1 -- 172.21.15.88:0/1546890153 wait complete. 2023-10-16T20:07:12.169 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:07:12.311 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.301+0000 7fce66f1d640 1 Processor -- start 2023-10-16T20:07:12.311 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.301+0000 7fce66f1d640 1 -- start start 2023-10-16T20:07:12.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.305+0000 7fce66f1d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce60069e90 0x7fce6006a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:12.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.305+0000 7fce66f1d640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce6006a830 con 0x7fce601599f0 2023-10-16T20:07:12.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.305+0000 7fce66f1d640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce600b1260 con 0x7fce600b05a0 2023-10-16T20:07:12.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.305+0000 7fce66f1d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce600b3140 con 0x7fce60069e90 2023-10-16T20:07:12.313 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.305+0000 7fce64cb9640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fce601599f0 0x7fce600b2a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:39518/0 (socket says 172.21.15.88:39518) 2023-10-16T20:07:12.313 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.305+0000 7fce64cb9640 1 -- 172.21.15.88:0/3522154840 learned_addr learned my addr 172.21.15.88:0/3522154840 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:12.313 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.305+0000 7fce57fff640 1 --2- 172.21.15.88:0/3522154840 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce60069e90 0x7fce6006a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:12.313 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.305+0000 7fce577fe640 1 -- 172.21.15.88:0/3522154840 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 80682386 0 0) 0x7fce6006a830 con 0x7fce601599f0 2023-10-16T20:07:12.313 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.305+0000 7fce577fe640 1 -- 172.21.15.88:0/3522154840 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce48003530 con 0x7fce601599f0 2023-10-16T20:07:12.314 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.305+0000 7fce577fe640 1 -- 172.21.15.88:0/3522154840 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 686796589 0 0) 0x7fce580014a0 con 0x7fce600b05a0 2023-10-16T20:07:12.314 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.305+0000 7fce577fe640 1 -- 172.21.15.88:0/3522154840 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce6006a830 con 0x7fce600b05a0 2023-10-16T20:07:12.314 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.305+0000 7fce577fe640 1 -- 172.21.15.88:0/3522154840 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1216897701 0 0) 0x7fce48003530 con 0x7fce601599f0 2023-10-16T20:07:12.314 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.305+0000 7fce577fe640 1 -- 172.21.15.88:0/3522154840 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce60069e90 msgr2=0x7fce6006a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:12.314 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.305+0000 7fce577fe640 1 --2- 172.21.15.88:0/3522154840 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce60069e90 0x7fce6006a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:12.315 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.305+0000 7fce577fe640 1 -- 172.21.15.88:0/3522154840 >> v1:172.21.15.94:6789/0 conn(0x7fce600b05a0 legacy=0x7fce600b0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:12.315 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.305+0000 7fce577fe640 1 -- 172.21.15.88:0/3522154840 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce600ae960 con 0x7fce601599f0 2023-10-16T20:07:12.315 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.305+0000 7fce577fe640 1 -- 172.21.15.88:0/3522154840 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7fce5c001680 con 0x7fce601599f0 2023-10-16T20:07:12.315 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.305+0000 7fce577fe640 1 -- 172.21.15.88:0/3522154840 >> v1:172.21.15.88:6789/0 conn(0x7fce601599f0 legacy=0x7fce600b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:12.315 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.305+0000 7fce577fe640 1 --2- 172.21.15.88:0/3522154840 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce48003b40 0x7fce48003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:12.316 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.305+0000 7fce577fe640 1 --2- 172.21.15.88:0/3522154840 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce60069e90 0x7fce480047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:12.316 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.305+0000 7fce577fe640 1 --2- 172.21.15.88:0/3522154840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce48004d10 0x7fce48005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:12.316 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.305+0000 7fce577fe640 1 -- 172.21.15.88:0/3522154840 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce600b3140 con 0x7fce48003b40 2023-10-16T20:07:12.316 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.305+0000 7fce577fe640 1 -- 172.21.15.88:0/3522154840 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4800ba40 con 0x7fce48004d10 2023-10-16T20:07:12.316 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.305+0000 7fce577fe640 1 -- 172.21.15.88:0/3522154840 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4800bc00 con 0x7fce60069e90 2023-10-16T20:07:12.316 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.305+0000 7fce64cb9640 1 --2- 172.21.15.88:0/3522154840 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce48003b40 0x7fce48003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:12.317 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.305+0000 7fce57fff640 1 --2- 172.21.15.88:0/3522154840 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce60069e90 0x7fce480047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:12.317 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.305+0000 7fce654ba640 1 --2- 172.21.15.88:0/3522154840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce48004d10 0x7fce48005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:12.317 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.305+0000 7fce64cb9640 1 -- 172.21.15.88:0/3522154840 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce60069e90 msgr2=0x7fce480047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:12.317 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.305+0000 7fce64cb9640 1 --2- 172.21.15.88:0/3522154840 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce60069e90 0x7fce480047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:12.317 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.305+0000 7fce64cb9640 1 -- 172.21.15.88:0/3522154840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce48004d10 msgr2=0x7fce48005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:12.318 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.305+0000 7fce64cb9640 1 --2- 172.21.15.88:0/3522154840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce48004d10 0x7fce48005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:12.318 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.305+0000 7fce64cb9640 1 -- 172.21.15.88:0/3522154840 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce48003530 con 0x7fce48003b40 2023-10-16T20:07:12.321 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.309+0000 7fce64cb9640 1 --2- 172.21.15.88:0/3522154840 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce48003b40 0x7fce48003f40 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fce5c007fb0 tx=0x7fce5c01caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22202d5cd484188a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:12.321 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.309+0000 7fce577fe640 1 -- 172.21.15.88:0/3522154840 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c005ae0 con 0x7fce48003b40 2023-10-16T20:07:12.322 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.309+0000 7fce577fe640 1 -- 172.21.15.88:0/3522154840 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5c005c40 con 0x7fce48003b40 2023-10-16T20:07:12.322 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.309+0000 7fce577fe640 1 -- 172.21.15.88:0/3522154840 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c007070 con 0x7fce48003b40 2023-10-16T20:07:12.322 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.309+0000 7fce66f1d640 1 -- 172.21.15.88:0/3522154840 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce48003b40 msgr2=0x7fce48003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:12.322 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.309+0000 7fce66f1d640 1 --2- 172.21.15.88:0/3522154840 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce48003b40 0x7fce48003f40 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fce5c007fb0 tx=0x7fce5c01caf0 comp rx=0 tx=0).stop 2023-10-16T20:07:12.323 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.309+0000 7fce654ba640 1 -- 172.21.15.88:0/3522154840 reap_dead start 2023-10-16T20:07:12.323 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.309+0000 7fce66f1d640 1 -- 172.21.15.88:0/3522154840 shutdown_connections 2023-10-16T20:07:12.323 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.309+0000 7fce66f1d640 1 -- 172.21.15.88:0/3522154840 >> 172.21.15.88:0/3522154840 conn(0x7fce600ce5e0 msgr2=0x7fce600ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:12.323 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.309+0000 7fce66f1d640 1 -- 172.21.15.88:0/3522154840 shutdown_connections 2023-10-16T20:07:12.323 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.309+0000 7fce66f1d640 1 -- 172.21.15.88:0/3522154840 wait complete. 2023-10-16T20:07:12.324 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.313+0000 7fce66f1d640 1 Processor -- start 2023-10-16T20:07:12.324 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.313+0000 7fce66f1d640 1 -- start start 2023-10-16T20:07:12.324 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.313+0000 7fce66f1d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce48003b40 0x7fce601f5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:12.324 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.313+0000 7fce66f1d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce6013ed40 0x7fce6014c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:12.324 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.313+0000 7fce66f1d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce6013f1c0 0x7fce600cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:12.324 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.313+0000 7fce66f1d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce60155de0 con 0x7fce6013ed40 2023-10-16T20:07:12.325 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.313+0000 7fce66f1d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce601560c0 con 0x7fce48003b40 2023-10-16T20:07:12.325 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.313+0000 7fce66f1d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6014cf90 con 0x7fce6013f1c0 2023-10-16T20:07:12.325 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.313+0000 7fce57fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce6013ed40 0x7fce6014c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:12.325 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.313+0000 7fce57fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce6013ed40 0x7fce6014c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:33930/0 (socket says 172.21.15.88:33930) 2023-10-16T20:07:12.325 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.313+0000 7fce57fff640 1 -- 172.21.15.88:0/355116001 learned_addr learned my addr 172.21.15.88:0/355116001 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:12.326 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.313+0000 7fce64cb9640 1 --2- 172.21.15.88:0/355116001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce48003b40 0x7fce601f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:12.326 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.313+0000 7fce654ba640 1 --2- 172.21.15.88:0/355116001 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce6013f1c0 0x7fce600cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:12.326 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.313+0000 7fce57fff640 1 -- 172.21.15.88:0/355116001 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce6013f1c0 msgr2=0x7fce600cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:12.326 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.313+0000 7fce57fff640 1 --2- 172.21.15.88:0/355116001 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce6013f1c0 0x7fce600cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:12.327 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.313+0000 7fce57fff640 1 -- 172.21.15.88:0/355116001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce48003b40 msgr2=0x7fce601f5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:12.327 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.313+0000 7fce57fff640 1 --2- 172.21.15.88:0/355116001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce48003b40 0x7fce601f5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:12.327 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.313+0000 7fce57fff640 1 -- 172.21.15.88:0/355116001 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6014d270 con 0x7fce6013ed40 2023-10-16T20:07:12.327 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.313+0000 7fce57fff640 1 --2- 172.21.15.88:0/355116001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce6013ed40 0x7fce6014c620 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fce50011b90 tx=0x7fce50010810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:12.327 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.313+0000 7fce557fa640 1 -- 172.21.15.88:0/355116001 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5000f7e0 con 0x7fce6013ed40 2023-10-16T20:07:12.328 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.313+0000 7fce557fa640 1 -- 172.21.15.88:0/355116001 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5000f940 con 0x7fce6013ed40 2023-10-16T20:07:12.328 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.313+0000 7fce557fa640 1 -- 172.21.15.88:0/355116001 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5000fcd0 con 0x7fce6013ed40 2023-10-16T20:07:12.328 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.313+0000 7fce66f1d640 1 -- 172.21.15.88:0/355116001 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce6014d550 con 0x7fce6013ed40 2023-10-16T20:07:12.328 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.313+0000 7fce66f1d640 1 -- 172.21.15.88:0/355116001 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce6014cd30 con 0x7fce6013ed40 2023-10-16T20:07:12.328 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.313+0000 7fce66f1d640 1 -- 172.21.15.88:0/355116001 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce6004e9c0 con 0x7fce6013ed40 2023-10-16T20:07:12.328 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.317+0000 7fce557fa640 1 -- 172.21.15.88:0/355116001 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fce50023440 con 0x7fce6013ed40 2023-10-16T20:07:12.329 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.317+0000 7fce557fa640 1 --2- 172.21.15.88:0/355116001 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fce4c0664d0 0x7fce4c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:12.329 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.317+0000 7fce557fa640 1 -- 172.21.15.88:0/355116001 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7fce50084fe0 con 0x7fce6013ed40 2023-10-16T20:07:12.329 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.317+0000 7fce64cb9640 1 --2- 172.21.15.88:0/355116001 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fce4c0664d0 0x7fce4c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:12.329 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.317+0000 7fce64cb9640 1 --2- 172.21.15.88:0/355116001 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fce4c0664d0 0x7fce4c068990 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fce5c007f80 tx=0x7fce5c0a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:12.330 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.321+0000 7fce557fa640 1 -- 172.21.15.88:0/355116001 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fce500522b0 con 0x7fce6013ed40 2023-10-16T20:07:12.476 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.473+0000 7fce66f1d640 1 -- 172.21.15.88:0/355116001 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce6006aa90 con 0x7fce6013ed40 2023-10-16T20:07:12.477 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.473+0000 7fce557fa640 1 -- 172.21.15.88:0/355116001 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+18499 (secure 0 0 0) 0x7fce500558d0 con 0x7fce6013ed40 2023-10-16T20:07:12.478 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:07:12.478 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":46,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:06:38.250351+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T20:06:14.743500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"testcephfs1_data","create_time":"2023-10-16T20:06:16.091837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-10-16T20:06:31.184462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.39","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:07:12.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.477+0000 7fce66f1d640 1 -- 172.21.15.88:0/355116001 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fce4c0664d0 msgr2=0x7fce4c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:12.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.477+0000 7fce66f1d640 1 --2- 172.21.15.88:0/355116001 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fce4c0664d0 0x7fce4c068990 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fce5c007f80 tx=0x7fce5c0a7070 comp rx=0 tx=0).stop 2023-10-16T20:07:12.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.477+0000 7fce66f1d640 1 -- 172.21.15.88:0/355116001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce6013ed40 msgr2=0x7fce6014c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:12.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.477+0000 7fce66f1d640 1 --2- 172.21.15.88:0/355116001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce6013ed40 0x7fce6014c620 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fce50011b90 tx=0x7fce50010810 comp rx=0 tx=0).stop 2023-10-16T20:07:12.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.477+0000 7fce66f1d640 1 -- 172.21.15.88:0/355116001 shutdown_connections 2023-10-16T20:07:12.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.477+0000 7fce66f1d640 1 --2- 172.21.15.88:0/355116001 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fce4c0664d0 0x7fce4c068990 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:12.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.477+0000 7fce66f1d640 1 --2- 172.21.15.88:0/355116001 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fce6013f1c0 0x7fce600cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:12.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.477+0000 7fce66f1d640 1 --2- 172.21.15.88:0/355116001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce6013ed40 0x7fce6014c620 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:12.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.477+0000 7fce66f1d640 1 --2- 172.21.15.88:0/355116001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce48003b40 0x7fce601f5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:12.482 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.477+0000 7fce66f1d640 1 -- 172.21.15.88:0/355116001 >> 172.21.15.88:0/355116001 conn(0x7fce600ce5e0 msgr2=0x7fce60071780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:12.482 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.477+0000 7fce66f1d640 1 -- 172.21.15.88:0/355116001 shutdown_connections 2023-10-16T20:07:12.482 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.477+0000 7fce66f1d640 1 -- 172.21.15.88:0/355116001 wait complete. 2023-10-16T20:07:12.500 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail testcephfs1 2023-10-16T20:07:12.638 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.633+0000 7f4e74046640 1 Processor -- start 2023-10-16T20:07:12.639 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.633+0000 7f4e74046640 1 -- start start 2023-10-16T20:07:12.639 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.637+0000 7f4e74046640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4e6c0adf80 0x7f4e6c15a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:12.640 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.637+0000 7f4e74046640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4e6c15d480 0x7f4e6c0afa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:12.640 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.637+0000 7f4e74046640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e6c15e400 con 0x7f4e6c069e90 2023-10-16T20:07:12.640 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.637+0000 7f4e74046640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e6c159e40 con 0x7f4e6c15d480 2023-10-16T20:07:12.640 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.637+0000 7f4e74046640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c15de40 con 0x7f4e6c0adf80 2023-10-16T20:07:12.641 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.637+0000 7f4e715e1640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4e6c0adf80 0x7f4e6c15a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:12.642 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.637+0000 7f4e725e3640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4e6c15d480 0x7f4e6c0afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:12.642 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.637+0000 7f4e715e1640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4e6c0adf80 0x7f4e6c15a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:56986/0 (socket says 172.21.15.88:56986) 2023-10-16T20:07:12.642 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.637+0000 7f4e71de2640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f4e6c069e90 0x7f4e6c06a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:44178/0 (socket says 172.21.15.88:44178) 2023-10-16T20:07:12.642 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.637+0000 7f4e715e1640 1 -- 172.21.15.88:0/577730494 learned_addr learned my addr 172.21.15.88:0/577730494 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:12.642 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.637+0000 7f4e70de0640 1 -- 172.21.15.88:0/577730494 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1493591418 0 0) 0x7f4e6c15e400 con 0x7f4e6c069e90 2023-10-16T20:07:12.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.637+0000 7f4e70de0640 1 -- 172.21.15.88:0/577730494 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e58003530 con 0x7f4e6c069e90 2023-10-16T20:07:12.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.637+0000 7f4e715e1640 1 -- 172.21.15.88:0/577730494 >> v2:172.21.15.94:3300/0 conn(0x7f4e6c15d480 msgr2=0x7f4e6c0afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:12.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.637+0000 7f4e715e1640 1 --2- 172.21.15.88:0/577730494 >> v2:172.21.15.94:3300/0 conn(0x7f4e6c15d480 0x7f4e6c0afa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:12.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.637+0000 7f4e715e1640 1 -- 172.21.15.88:0/577730494 >> v1:172.21.15.94:6789/0 conn(0x7f4e6c069e90 legacy=0x7f4e6c06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:12.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.637+0000 7f4e715e1640 1 -- 172.21.15.88:0/577730494 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6c15e120 con 0x7f4e6c0adf80 2023-10-16T20:07:12.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.637+0000 7f4e725e3640 1 --2- 172.21.15.88:0/577730494 >> v2:172.21.15.94:3300/0 conn(0x7f4e6c15d480 0x7f4e6c0afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:07:12.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.641+0000 7f4e715e1640 1 --2- 172.21.15.88:0/577730494 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4e6c0adf80 0x7f4e6c15a830 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c009870 tx=0x7f4e5c01c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f942652f06fc1e66 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:12.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.641+0000 7f4e70de0640 1 -- 172.21.15.88:0/577730494 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c0a6710 con 0x7f4e6c0adf80 2023-10-16T20:07:12.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.641+0000 7f4e70de0640 1 -- 172.21.15.88:0/577730494 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5c0a6870 con 0x7f4e6c0adf80 2023-10-16T20:07:12.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.641+0000 7f4e70de0640 1 -- 172.21.15.88:0/577730494 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c0a6c00 con 0x7f4e6c0adf80 2023-10-16T20:07:12.646 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.641+0000 7f4e74046640 1 -- 172.21.15.88:0/577730494 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4e6c0adf80 msgr2=0x7f4e6c15a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:12.646 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.641+0000 7f4e74046640 1 --2- 172.21.15.88:0/577730494 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4e6c0adf80 0x7f4e6c15a830 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c009870 tx=0x7f4e5c01c740 comp rx=0 tx=0).stop 2023-10-16T20:07:12.646 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.641+0000 7f4e74046640 1 -- 172.21.15.88:0/577730494 shutdown_connections 2023-10-16T20:07:12.646 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.641+0000 7f4e74046640 1 --2- 172.21.15.88:0/577730494 >> v2:172.21.15.94:3300/0 conn(0x7f4e6c15d480 0x7f4e6c0afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:12.646 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.641+0000 7f4e74046640 1 --2- 172.21.15.88:0/577730494 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4e6c0adf80 0x7f4e6c15a830 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:12.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.641+0000 7f4e74046640 1 -- 172.21.15.88:0/577730494 >> 172.21.15.88:0/577730494 conn(0x7f4e6c0ce5e0 msgr2=0x7f4e6c0ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:12.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.641+0000 7f4e74046640 1 -- 172.21.15.88:0/577730494 shutdown_connections 2023-10-16T20:07:12.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.641+0000 7f4e74046640 1 -- 172.21.15.88:0/577730494 wait complete. 2023-10-16T20:07:12.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.641+0000 7f4e74046640 1 Processor -- start 2023-10-16T20:07:12.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.641+0000 7f4e74046640 1 -- start start 2023-10-16T20:07:12.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.641+0000 7f4e74046640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e6c069e90 0x7f4e6c1494a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:12.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.641+0000 7f4e74046640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4e6c0adf80 0x7f4e6c158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:12.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.641+0000 7f4e74046640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e6c15d480 0x7f4e6c1592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:12.651 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.645+0000 7f4e74046640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c14d2b0 con 0x7f4e6c15d480 2023-10-16T20:07:12.651 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.645+0000 7f4e74046640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c14d590 con 0x7f4e6c069e90 2023-10-16T20:07:12.651 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.645+0000 7f4e74046640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c14d870 con 0x7f4e6c0adf80 2023-10-16T20:07:12.651 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.645+0000 7f4e71de2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e6c069e90 0x7f4e6c1494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:12.651 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.645+0000 7f4e715e1640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4e6c0adf80 0x7f4e6c158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:12.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.645+0000 7f4e71de2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e6c069e90 0x7f4e6c1494a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:48444/0 (socket says 172.21.15.88:48444) 2023-10-16T20:07:12.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.645+0000 7f4e71de2640 1 -- 172.21.15.88:0/607177603 learned_addr learned my addr 172.21.15.88:0/607177603 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:12.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.645+0000 7f4e725e3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e6c15d480 0x7f4e6c1592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:12.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.645+0000 7f4e71de2640 1 -- 172.21.15.88:0/607177603 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4e6c0adf80 msgr2=0x7f4e6c158d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:12.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.645+0000 7f4e71de2640 1 --2- 172.21.15.88:0/607177603 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4e6c0adf80 0x7f4e6c158d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:12.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.645+0000 7f4e71de2640 1 -- 172.21.15.88:0/607177603 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e6c15d480 msgr2=0x7f4e6c1592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:12.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.645+0000 7f4e71de2640 1 --2- 172.21.15.88:0/607177603 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e6c15d480 0x7f4e6c1592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:12.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.645+0000 7f4e71de2640 1 -- 172.21.15.88:0/607177603 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e64009a70 con 0x7f4e6c069e90 2023-10-16T20:07:12.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.645+0000 7f4e725e3640 1 --2- 172.21.15.88:0/607177603 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e6c15d480 0x7f4e6c1592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:07:12.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.645+0000 7f4e71de2640 1 --2- 172.21.15.88:0/607177603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e6c069e90 0x7f4e6c1494a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800c430 tx=0x7f4e6800c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:12.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.645+0000 7f4e627fc640 1 -- 172.21.15.88:0/607177603 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e680074e0 con 0x7f4e6c069e90 2023-10-16T20:07:12.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.645+0000 7f4e627fc640 1 -- 172.21.15.88:0/607177603 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e68007660 con 0x7f4e6c069e90 2023-10-16T20:07:12.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.645+0000 7f4e627fc640 1 -- 172.21.15.88:0/607177603 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e68004a30 con 0x7f4e6c069e90 2023-10-16T20:07:12.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.645+0000 7f4e74046640 1 -- 172.21.15.88:0/607177603 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e5c0094d0 con 0x7f4e6c069e90 2023-10-16T20:07:12.654 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.645+0000 7f4e74046640 1 -- 172.21.15.88:0/607177603 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e6c149f50 con 0x7f4e6c069e90 2023-10-16T20:07:12.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.645+0000 7f4e74046640 1 -- 172.21.15.88:0/607177603 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e6c06d180 con 0x7f4e6c069e90 2023-10-16T20:07:12.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.645+0000 7f4e627fc640 1 -- 172.21.15.88:0/607177603 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4e68004b90 con 0x7f4e6c069e90 2023-10-16T20:07:12.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.649+0000 7f4e627fc640 1 --2- 172.21.15.88:0/607177603 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4e500664a0 0x7f4e50068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:12.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.649+0000 7f4e627fc640 1 -- 172.21.15.88:0/607177603 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f4e68083aa0 con 0x7f4e6c069e90 2023-10-16T20:07:12.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.649+0000 7f4e715e1640 1 --2- 172.21.15.88:0/607177603 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4e500664a0 0x7f4e50068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:12.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.649+0000 7f4e715e1640 1 --2- 172.21.15.88:0/607177603 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4e500664a0 0x7f4e50068960 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0af000 tx=0x7f4e5c0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:12.657 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.653+0000 7f4e627fc640 1 -- 172.21.15.88:0/607177603 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4e68050c90 con 0x7f4e6c069e90 2023-10-16T20:07:12.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.837+0000 7f4e74046640 1 -- 172.21.15.88:0/607177603 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "testcephfs1"} v 0) v1 -- 0x7f4e6c14d6d0 con 0x7f4e6c069e90 2023-10-16T20:07:12.843 INFO:tasks.ceph.mon.a.smithi088.stderr:2023-10-16T20:07:12.837+0000 7f41ee15b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T20:07:12.852 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.849+0000 7f4e627fc640 1 -- 172.21.15.88:0/607177603 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "testcephfs1"}]=0 testcephfs1 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v20) v1 ==== 171+0+0 (secure 0 0 0) 0x7f4e6c14d6d0 con 0x7f4e6c069e90 2023-10-16T20:07:12.852 INFO:teuthology.orchestra.run.smithi088.stderr:testcephfs1 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T20:07:12.854 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.849+0000 7f4e74046640 1 -- 172.21.15.88:0/607177603 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4e500664a0 msgr2=0x7f4e50068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:12.854 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.849+0000 7f4e74046640 1 --2- 172.21.15.88:0/607177603 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4e500664a0 0x7f4e50068960 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0af000 tx=0x7f4e5c0a7000 comp rx=0 tx=0).stop 2023-10-16T20:07:12.855 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.849+0000 7f4e74046640 1 -- 172.21.15.88:0/607177603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e6c069e90 msgr2=0x7f4e6c1494a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:12.855 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.849+0000 7f4e74046640 1 --2- 172.21.15.88:0/607177603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e6c069e90 0x7f4e6c1494a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800c430 tx=0x7f4e6800c830 comp rx=0 tx=0).stop 2023-10-16T20:07:12.855 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.849+0000 7f4e74046640 1 -- 172.21.15.88:0/607177603 shutdown_connections 2023-10-16T20:07:12.855 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.849+0000 7f4e74046640 1 --2- 172.21.15.88:0/607177603 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4e500664a0 0x7f4e50068960 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:12.856 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.849+0000 7f4e74046640 1 --2- 172.21.15.88:0/607177603 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4e6c0adf80 0x7f4e6c158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:12.856 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.849+0000 7f4e74046640 1 --2- 172.21.15.88:0/607177603 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e6c15d480 0x7f4e6c1592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:12.856 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.849+0000 7f4e74046640 1 --2- 172.21.15.88:0/607177603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e6c069e90 0x7f4e6c1494a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:12.856 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.849+0000 7f4e74046640 1 -- 172.21.15.88:0/607177603 >> 172.21.15.88:0/607177603 conn(0x7f4e6c0ce5e0 msgr2=0x7f4e6c0b1180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:12.856 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.849+0000 7f4e74046640 1 -- 172.21.15.88:0/607177603 shutdown_connections 2023-10-16T20:07:12.857 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:12.849+0000 7f4e74046640 1 -- 172.21.15.88:0/607177603 wait complete. 2023-10-16T20:07:12.858 INFO:tasks.ceph.mds.b.smithi094.stderr: -3818> 2023-10-16T20:06:21.674+0000 7fa95b9d6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T20:07:12.873 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm testcephfs1 --yes-i-really-mean-it 2023-10-16T20:07:12.908 INFO:tasks.ceph.mds.b.smithi094.stdout:starting mds.b at 2023-10-16T20:07:13.013 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.009+0000 7fdac3980640 1 Processor -- start 2023-10-16T20:07:13.013 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.009+0000 7fdac3980640 1 -- start start 2023-10-16T20:07:13.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.009+0000 7fdac3980640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdabc06d3f0 0x7fdabc06d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:13.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.009+0000 7fdac3980640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdabc15ad30 con 0x7fdabc0b4260 2023-10-16T20:07:13.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.009+0000 7fdac3980640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdabc0b4640 con 0x7fdabc0b0da0 2023-10-16T20:07:13.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.009+0000 7fdac3980640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc06dda0 con 0x7fdabc06d3f0 2023-10-16T20:07:13.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.009+0000 7fdac171c640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fdabc0b4260 0x7fdabc15dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:39528/0 (socket says 172.21.15.88:39528) 2023-10-16T20:07:13.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.009+0000 7fdac171c640 1 -- 172.21.15.88:0/1197593395 learned_addr learned my addr 172.21.15.88:0/1197593395 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:13.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.009+0000 7fdac1f1d640 1 --2- 172.21.15.88:0/1197593395 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdabc06d3f0 0x7fdabc06d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:13.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.009+0000 7fdab3fff640 1 -- 172.21.15.88:0/1197593395 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2409402941 0 0) 0x7fdabc15ad30 con 0x7fdabc0b4260 2023-10-16T20:07:13.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.009+0000 7fdab3fff640 1 -- 172.21.15.88:0/1197593395 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdaa8003530 con 0x7fdabc0b4260 2023-10-16T20:07:13.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.009+0000 7fdab3fff640 1 -- 172.21.15.88:0/1197593395 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1382476632 0 0) 0x7fdabc0b4640 con 0x7fdabc0b0da0 2023-10-16T20:07:13.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.009+0000 7fdab3fff640 1 -- 172.21.15.88:0/1197593395 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdabc15ad30 con 0x7fdabc0b0da0 2023-10-16T20:07:13.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.009+0000 7fdac1f1d640 1 -- 172.21.15.88:0/1197593395 >> v1:172.21.15.94:6789/0 conn(0x7fdabc0b0da0 legacy=0x7fdabc159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:13.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.009+0000 7fdac1f1d640 1 -- 172.21.15.88:0/1197593395 >> v1:172.21.15.88:6789/0 conn(0x7fdabc0b4260 legacy=0x7fdabc15dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:13.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.009+0000 7fdac1f1d640 1 -- 172.21.15.88:0/1197593395 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdabc15a770 con 0x7fdabc06d3f0 2023-10-16T20:07:13.019 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.013+0000 7fdac1f1d640 1 --2- 172.21.15.88:0/1197593395 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdabc06d3f0 0x7fdabc06d860 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009dc0 tx=0x7fdab401c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e0092a79736cd67 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:13.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.013+0000 7fdab3fff640 1 -- 172.21.15.88:0/1197593395 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab40a5890 con 0x7fdabc06d3f0 2023-10-16T20:07:13.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.013+0000 7fdab3fff640 1 -- 172.21.15.88:0/1197593395 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab40a59f0 con 0x7fdabc06d3f0 2023-10-16T20:07:13.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.013+0000 7fdab3fff640 1 -- 172.21.15.88:0/1197593395 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab40a5d80 con 0x7fdabc06d3f0 2023-10-16T20:07:13.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.013+0000 7fdac3980640 1 -- 172.21.15.88:0/1197593395 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdabc06d3f0 msgr2=0x7fdabc06d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:13.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.013+0000 7fdac3980640 1 --2- 172.21.15.88:0/1197593395 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdabc06d3f0 0x7fdabc06d860 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009dc0 tx=0x7fdab401c8e0 comp rx=0 tx=0).stop 2023-10-16T20:07:13.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.013+0000 7fdac3980640 1 -- 172.21.15.88:0/1197593395 shutdown_connections 2023-10-16T20:07:13.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.013+0000 7fdac3980640 1 --2- 172.21.15.88:0/1197593395 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdabc06d3f0 0x7fdabc06d860 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:13.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.013+0000 7fdac3980640 1 -- 172.21.15.88:0/1197593395 >> 172.21.15.88:0/1197593395 conn(0x7fdabc0cb080 msgr2=0x7fdabc0cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:13.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.013+0000 7fdac3980640 1 -- 172.21.15.88:0/1197593395 shutdown_connections 2023-10-16T20:07:13.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.013+0000 7fdac3980640 1 -- 172.21.15.88:0/1197593395 wait complete. 2023-10-16T20:07:13.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.017+0000 7fdac3980640 1 Processor -- start 2023-10-16T20:07:13.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.017+0000 7fdac3980640 1 -- start start 2023-10-16T20:07:13.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.017+0000 7fdac3980640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdabc06d3f0 0x7fdabc0ac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:13.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.017+0000 7fdac3980640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdabc0b0da0 0x7fdabc1f4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:13.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.017+0000 7fdac3980640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdabc0b4260 0x7fdabc15aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:13.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.017+0000 7fdac3980640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc15ce80 con 0x7fdabc06d3f0 2023-10-16T20:07:13.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.017+0000 7fdac3980640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc1f4f90 con 0x7fdabc0b4260 2023-10-16T20:07:13.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.017+0000 7fdac3980640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc15b420 con 0x7fdabc0b0da0 2023-10-16T20:07:13.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.017+0000 7fdac171c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdabc06d3f0 0x7fdabc0ac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:13.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.017+0000 7fdac171c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdabc06d3f0 0x7fdabc0ac2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:33944/0 (socket says 172.21.15.88:33944) 2023-10-16T20:07:13.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.017+0000 7fdac171c640 1 -- 172.21.15.88:0/1237826876 learned_addr learned my addr 172.21.15.88:0/1237826876 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:13.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.017+0000 7fdac1f1d640 1 --2- 172.21.15.88:0/1237826876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdabc0b4260 0x7fdabc15aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:13.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.017+0000 7fdac0f1b640 1 --2- 172.21.15.88:0/1237826876 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdabc0b0da0 0x7fdabc1f4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:13.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.017+0000 7fdac171c640 1 -- 172.21.15.88:0/1237826876 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdabc0b0da0 msgr2=0x7fdabc1f4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:13.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.017+0000 7fdac171c640 1 --2- 172.21.15.88:0/1237826876 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdabc0b0da0 0x7fdabc1f4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:13.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.017+0000 7fdac171c640 1 -- 172.21.15.88:0/1237826876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdabc0b4260 msgr2=0x7fdabc15aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:13.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.017+0000 7fdac171c640 1 --2- 172.21.15.88:0/1237826876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdabc0b4260 0x7fdabc15aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:13.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.017+0000 7fdac171c640 1 -- 172.21.15.88:0/1237826876 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab4009a70 con 0x7fdabc06d3f0 2023-10-16T20:07:13.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.017+0000 7fdac0f1b640 1 --2- 172.21.15.88:0/1237826876 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdabc0b0da0 0x7fdabc1f4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:07:13.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.017+0000 7fdac171c640 1 --2- 172.21.15.88:0/1237826876 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdabc06d3f0 0x7fdabc0ac2a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fdab800c450 tx=0x7fdab800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:13.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.017+0000 7fdab1ffb640 1 -- 172.21.15.88:0/1237826876 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab8005cf0 con 0x7fdabc06d3f0 2023-10-16T20:07:13.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.017+0000 7fdab1ffb640 1 -- 172.21.15.88:0/1237826876 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab8005e50 con 0x7fdabc06d3f0 2023-10-16T20:07:13.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.017+0000 7fdac3980640 1 -- 172.21.15.88:0/1237826876 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdabc15a540 con 0x7fdabc06d3f0 2023-10-16T20:07:13.029 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.017+0000 7fdab1ffb640 1 -- 172.21.15.88:0/1237826876 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab8007310 con 0x7fdabc06d3f0 2023-10-16T20:07:13.029 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.021+0000 7fdac3980640 1 -- 172.21.15.88:0/1237826876 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdabc15cba0 con 0x7fdabc06d3f0 2023-10-16T20:07:13.030 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.021+0000 7fdac3980640 1 -- 172.21.15.88:0/1237826876 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdabc04e9c0 con 0x7fdabc06d3f0 2023-10-16T20:07:13.030 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.021+0000 7fdab1ffb640 1 -- 172.21.15.88:0/1237826876 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdab8007470 con 0x7fdabc06d3f0 2023-10-16T20:07:13.030 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.021+0000 7fdab1ffb640 1 --2- 172.21.15.88:0/1237826876 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fdaa00664a0 0x7fdaa0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:13.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.021+0000 7fdab1ffb640 1 -- 172.21.15.88:0/1237826876 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6564+0+0 (secure 0 0 0) 0x7fdab8083800 con 0x7fdabc06d3f0 2023-10-16T20:07:13.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.021+0000 7fdac0f1b640 1 --2- 172.21.15.88:0/1237826876 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fdaa00664a0 0x7fdaa0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:13.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.021+0000 7fdac0f1b640 1 --2- 172.21.15.88:0/1237826876 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fdaa00664a0 0x7fdaa0068960 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdabc0b0820 tx=0x7fdaac009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:13.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.025+0000 7fdab1ffb640 1 -- 172.21.15.88:0/1237826876 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdab8050930 con 0x7fdabc06d3f0 2023-10-16T20:07:13.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.209+0000 7fdac3980640 1 -- 172.21.15.88:0/1237826876 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "testcephfs1", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdabc071500 con 0x7fdabc06d3f0 2023-10-16T20:07:13.850 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.845+0000 7fdab1ffb640 1 -- 172.21.15.88:0/1237826876 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "testcephfs1", "yes_i_really_mean_it": true}]=0 v21) v1 ==== 109+0+0 (secure 0 0 0) 0x7fdab8053f50 con 0x7fdabc06d3f0 2023-10-16T20:07:13.852 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.849+0000 7fdac3980640 1 -- 172.21.15.88:0/1237826876 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fdaa00664a0 msgr2=0x7fdaa0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:13.852 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.849+0000 7fdac3980640 1 --2- 172.21.15.88:0/1237826876 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fdaa00664a0 0x7fdaa0068960 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdabc0b0820 tx=0x7fdaac009040 comp rx=0 tx=0).stop 2023-10-16T20:07:13.853 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.849+0000 7fdac3980640 1 -- 172.21.15.88:0/1237826876 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdabc06d3f0 msgr2=0x7fdabc0ac2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:13.853 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.849+0000 7fdac3980640 1 --2- 172.21.15.88:0/1237826876 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdabc06d3f0 0x7fdabc0ac2a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fdab800c450 tx=0x7fdab800c910 comp rx=0 tx=0).stop 2023-10-16T20:07:13.853 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.849+0000 7fdac3980640 1 -- 172.21.15.88:0/1237826876 shutdown_connections 2023-10-16T20:07:13.853 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.849+0000 7fdac3980640 1 --2- 172.21.15.88:0/1237826876 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fdaa00664a0 0x7fdaa0068960 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:13.853 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.849+0000 7fdac3980640 1 --2- 172.21.15.88:0/1237826876 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdabc0b0da0 0x7fdabc1f4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:13.854 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.849+0000 7fdac3980640 1 --2- 172.21.15.88:0/1237826876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdabc0b4260 0x7fdabc15aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:13.854 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.849+0000 7fdac3980640 1 --2- 172.21.15.88:0/1237826876 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdabc06d3f0 0x7fdabc0ac2a0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:13.854 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.849+0000 7fdac3980640 1 -- 172.21.15.88:0/1237826876 >> 172.21.15.88:0/1237826876 conn(0x7fdabc0cb080 msgr2=0x7fdabc155270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:13.854 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.849+0000 7fdac3980640 1 -- 172.21.15.88:0/1237826876 shutdown_connections 2023-10-16T20:07:13.854 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:13.849+0000 7fdac3980640 1 -- 172.21.15.88:0/1237826876 wait complete. 2023-10-16T20:07:13.873 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm testcephfs1_metadata testcephfs1_metadata --yes-i-really-really-mean-it 2023-10-16T20:07:14.014 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.009+0000 7fa4e51ab640 1 Processor -- start 2023-10-16T20:07:14.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.009+0000 7fa4e51ab640 1 -- start start 2023-10-16T20:07:14.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.009+0000 7fa4e51ab640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fa4e0159730 0x7fa4e00ad700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:14.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.009+0000 7fa4e51ab640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa4e006a5a0 0x7fa4e006d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:14.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.009+0000 7fa4e51ab640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4e015d660 con 0x7fa4e00c8cb0 2023-10-16T20:07:14.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.009+0000 7fa4e51ab640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4e015e2e0 con 0x7fa4e0159730 2023-10-16T20:07:14.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.009+0000 7fa4e51ab640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e015d940 con 0x7fa4e006a5a0 2023-10-16T20:07:14.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.013+0000 7fa4ded9d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fa4e0159730 0x7fa4e00ad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:14.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.013+0000 7fa4ded9d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fa4e0159730 0x7fa4e00ad700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:33958/0 (socket says 172.21.15.88:33958) 2023-10-16T20:07:14.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.013+0000 7fa4ded9d640 1 -- 172.21.15.88:0/594316160 learned_addr learned my addr 172.21.15.88:0/594316160 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:14.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.013+0000 7fa4df59e640 1 --2- 172.21.15.88:0/594316160 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa4e006a5a0 0x7fa4e006d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:14.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.013+0000 7fa4ded9d640 1 -- 172.21.15.88:0/594316160 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa4e006a5a0 msgr2=0x7fa4e006d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:14.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.013+0000 7fa4ded9d640 1 --2- 172.21.15.88:0/594316160 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa4e006a5a0 0x7fa4e006d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:14.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.013+0000 7fa4ded9d640 1 -- 172.21.15.88:0/594316160 >> v1:172.21.15.94:6789/0 conn(0x7fa4e00c8cb0 legacy=0x7fa4e0069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:14.019 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.013+0000 7fa4ded9d640 1 -- 172.21.15.88:0/594316160 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e015dc20 con 0x7fa4e0159730 2023-10-16T20:07:14.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.013+0000 7fa4ded9d640 1 --2- 172.21.15.88:0/594316160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4e0159730 0x7fa4e00ad700 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0009870 tx=0x7fa4d001e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea544213da2f1848 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:14.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.013+0000 7fa4de59c640 1 -- 172.21.15.88:0/594316160 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d00a86d0 con 0x7fa4e0159730 2023-10-16T20:07:14.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.013+0000 7fa4de59c640 1 -- 172.21.15.88:0/594316160 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d00a8830 con 0x7fa4e0159730 2023-10-16T20:07:14.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.013+0000 7fa4de59c640 1 -- 172.21.15.88:0/594316160 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d00a8bc0 con 0x7fa4e0159730 2023-10-16T20:07:14.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.017+0000 7fa4e51ab640 1 -- 172.21.15.88:0/594316160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4e0159730 msgr2=0x7fa4e00ad700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:14.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.017+0000 7fa4e51ab640 1 --2- 172.21.15.88:0/594316160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4e0159730 0x7fa4e00ad700 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0009870 tx=0x7fa4d001e720 comp rx=0 tx=0).stop 2023-10-16T20:07:14.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.017+0000 7fa4e51ab640 1 -- 172.21.15.88:0/594316160 shutdown_connections 2023-10-16T20:07:14.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.017+0000 7fa4e51ab640 1 --2- 172.21.15.88:0/594316160 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa4e006a5a0 0x7fa4e006d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:14.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.017+0000 7fa4e51ab640 1 --2- 172.21.15.88:0/594316160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4e0159730 0x7fa4e00ad700 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:14.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.017+0000 7fa4e51ab640 1 -- 172.21.15.88:0/594316160 >> 172.21.15.88:0/594316160 conn(0x7fa4e00ce660 msgr2=0x7fa4e00cea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:14.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.017+0000 7fa4e51ab640 1 -- 172.21.15.88:0/594316160 shutdown_connections 2023-10-16T20:07:14.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.017+0000 7fa4e51ab640 1 -- 172.21.15.88:0/594316160 wait complete. 2023-10-16T20:07:14.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.017+0000 7fa4e51ab640 1 Processor -- start 2023-10-16T20:07:14.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.017+0000 7fa4e51ab640 1 -- start start 2023-10-16T20:07:14.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.017+0000 7fa4e51ab640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa4e006a5a0 0x7fa4e0152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:14.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.017+0000 7fa4e51ab640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4e00c8cb0 0x7fa4e00cc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:14.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.017+0000 7fa4e51ab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4e0159730 0x7fa4e00cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:14.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.017+0000 7fa4e51ab640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e015b690 con 0x7fa4e00c8cb0 2023-10-16T20:07:14.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.017+0000 7fa4e51ab640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e00c3fa0 con 0x7fa4e0159730 2023-10-16T20:07:14.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.017+0000 7fa4e51ab640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e015b8e0 con 0x7fa4e006a5a0 2023-10-16T20:07:14.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.017+0000 7fa4d659c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4e00c8cb0 0x7fa4e00cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:14.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.017+0000 7fa4d659c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4e00c8cb0 0x7fa4e00cc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:33972/0 (socket says 172.21.15.88:33972) 2023-10-16T20:07:14.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.017+0000 7fa4d659c640 1 -- 172.21.15.88:0/110175091 learned_addr learned my addr 172.21.15.88:0/110175091 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:14.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.017+0000 7fa4ded9d640 1 --2- 172.21.15.88:0/110175091 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa4e006a5a0 0x7fa4e0152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:14.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.017+0000 7fa4df59e640 1 --2- 172.21.15.88:0/110175091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4e0159730 0x7fa4e00cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:14.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.021+0000 7fa4d659c640 1 -- 172.21.15.88:0/110175091 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa4e006a5a0 msgr2=0x7fa4e0152290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:14.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.021+0000 7fa4d659c640 1 --2- 172.21.15.88:0/110175091 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa4e006a5a0 0x7fa4e0152290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:14.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.021+0000 7fa4d659c640 1 -- 172.21.15.88:0/110175091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4e0159730 msgr2=0x7fa4e00cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:14.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.021+0000 7fa4d659c640 1 --2- 172.21.15.88:0/110175091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4e0159730 0x7fa4e00cc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:14.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.021+0000 7fa4d659c640 1 -- 172.21.15.88:0/110175091 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d00094d0 con 0x7fa4e00c8cb0 2023-10-16T20:07:14.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.021+0000 7fa4d659c640 1 --2- 172.21.15.88:0/110175091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4e00c8cb0 0x7fa4e00cc370 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa4c800c500 tx=0x7fa4c800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:14.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.021+0000 7fa4d7fff640 1 -- 172.21.15.88:0/110175091 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8005220 con 0x7fa4e00c8cb0 2023-10-16T20:07:14.029 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.021+0000 7fa4d7fff640 1 -- 172.21.15.88:0/110175091 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c8005380 con 0x7fa4e00c8cb0 2023-10-16T20:07:14.029 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.021+0000 7fa4e51ab640 1 -- 172.21.15.88:0/110175091 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e015bbc0 con 0x7fa4e00c8cb0 2023-10-16T20:07:14.029 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.021+0000 7fa4d7fff640 1 -- 172.21.15.88:0/110175091 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8007270 con 0x7fa4e00c8cb0 2023-10-16T20:07:14.029 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.021+0000 7fa4e51ab640 1 -- 172.21.15.88:0/110175091 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e01527d0 con 0x7fa4e00c8cb0 2023-10-16T20:07:14.030 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.021+0000 7fa4e51ab640 1 -- 172.21.15.88:0/110175091 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4e00670c0 con 0x7fa4e00c8cb0 2023-10-16T20:07:14.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.021+0000 7fa4d7fff640 1 -- 172.21.15.88:0/110175091 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa4c80073d0 con 0x7fa4e00c8cb0 2023-10-16T20:07:14.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.021+0000 7fa4d7fff640 1 --2- 172.21.15.88:0/110175091 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa4c00664a0 0x7fa4c0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:14.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.021+0000 7fa4d7fff640 1 -- 172.21.15.88:0/110175091 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6564+0+0 (secure 0 0 0) 0x7fa4c80834f0 con 0x7fa4e00c8cb0 2023-10-16T20:07:14.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.021+0000 7fa4ded9d640 1 --2- 172.21.15.88:0/110175091 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa4c00664a0 0x7fa4c0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:14.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.025+0000 7fa4ded9d640 1 --2- 172.21.15.88:0/110175091 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa4c00664a0 0x7fa4c0068960 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa4d00b1000 tx=0x7fa4d0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:14.032 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.025+0000 7fa4d7fff640 1 -- 172.21.15.88:0/110175091 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa4c80506d0 con 0x7fa4e00c8cb0 2023-10-16T20:07:14.181 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.177+0000 7fa4e51ab640 1 -- 172.21.15.88:0/110175091 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "testcephfs1_metadata", "pool2": "testcephfs1_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa4e004e9c0 con 0x7fa4e00c8cb0 2023-10-16T20:07:14.853 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.849+0000 7fa4d7fff640 1 -- 172.21.15.88:0/110175091 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "testcephfs1_metadata", "pool2": "testcephfs1_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'testcephfs1_metadata' removed v48) v1 ==== 196+0+0 (secure 0 0 0) 0x7fa4c805a030 con 0x7fa4e00c8cb0 2023-10-16T20:07:14.853 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'testcephfs1_metadata' removed 2023-10-16T20:07:14.855 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.849+0000 7fa4e51ab640 1 -- 172.21.15.88:0/110175091 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa4c00664a0 msgr2=0x7fa4c0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:14.856 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.849+0000 7fa4e51ab640 1 --2- 172.21.15.88:0/110175091 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa4c00664a0 0x7fa4c0068960 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa4d00b1000 tx=0x7fa4d0008040 comp rx=0 tx=0).stop 2023-10-16T20:07:14.856 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.853+0000 7fa4e51ab640 1 -- 172.21.15.88:0/110175091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4e00c8cb0 msgr2=0x7fa4e00cc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:14.856 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.853+0000 7fa4e51ab640 1 --2- 172.21.15.88:0/110175091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4e00c8cb0 0x7fa4e00cc370 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa4c800c500 tx=0x7fa4c800c9c0 comp rx=0 tx=0).stop 2023-10-16T20:07:14.857 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.853+0000 7fa4e51ab640 1 -- 172.21.15.88:0/110175091 shutdown_connections 2023-10-16T20:07:14.857 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.853+0000 7fa4e51ab640 1 --2- 172.21.15.88:0/110175091 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa4c00664a0 0x7fa4c0068960 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:14.857 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.853+0000 7fa4e51ab640 1 --2- 172.21.15.88:0/110175091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4e0159730 0x7fa4e00cc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:14.857 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.853+0000 7fa4e51ab640 1 --2- 172.21.15.88:0/110175091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4e00c8cb0 0x7fa4e00cc370 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:14.858 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.853+0000 7fa4e51ab640 1 --2- 172.21.15.88:0/110175091 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa4e006a5a0 0x7fa4e0152290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:14.858 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.853+0000 7fa4e51ab640 1 -- 172.21.15.88:0/110175091 >> 172.21.15.88:0/110175091 conn(0x7fa4e00ce660 msgr2=0x7fa4e00b07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:14.858 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.853+0000 7fa4e51ab640 1 -- 172.21.15.88:0/110175091 shutdown_connections 2023-10-16T20:07:14.858 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:14.853+0000 7fa4e51ab640 1 -- 172.21.15.88:0/110175091 wait complete. 2023-10-16T20:07:14.882 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm testcephfs1_data testcephfs1_data --yes-i-really-really-mean-it 2023-10-16T20:07:15.030 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.025+0000 7fc677091640 1 Processor -- start 2023-10-16T20:07:15.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.025+0000 7fc677091640 1 -- start start 2023-10-16T20:07:15.032 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.029+0000 7fc677091640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc6700cc850 0x7fc670167ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:15.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.029+0000 7fc677091640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6700cb3b0 con 0x7fc67015cb80 2023-10-16T20:07:15.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.029+0000 7fc677091640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6700cecf0 con 0x7fc670161400 2023-10-16T20:07:15.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.029+0000 7fc677091640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6700cefd0 con 0x7fc6700cc850 2023-10-16T20:07:15.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.029+0000 7fc674e2d640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fc67015cb80 0x7fc6700ce410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:39538/0 (socket says 172.21.15.88:39538) 2023-10-16T20:07:15.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.029+0000 7fc674e2d640 1 -- 172.21.15.88:0/1633505209 learned_addr learned my addr 172.21.15.88:0/1633505209 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:15.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.029+0000 7fc67562e640 1 --2- 172.21.15.88:0/1633505209 >> v2:172.21.15.88:3300/0 conn(0x7fc6700cc850 0x7fc670167ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:15.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.029+0000 7fc6677fe640 1 -- 172.21.15.88:0/1633505209 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2110529495 0 0) 0x7fc6700cb3b0 con 0x7fc67015cb80 2023-10-16T20:07:15.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.029+0000 7fc6677fe640 1 -- 172.21.15.88:0/1633505209 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc658003530 con 0x7fc67015cb80 2023-10-16T20:07:15.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.029+0000 7fc67562e640 1 -- 172.21.15.88:0/1633505209 >> v1:172.21.15.94:6789/0 conn(0x7fc670161400 legacy=0x7fc6700cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:15.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.029+0000 7fc67562e640 1 -- 172.21.15.88:0/1633505209 >> v1:172.21.15.88:6789/0 conn(0x7fc67015cb80 legacy=0x7fc6700ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:15.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.029+0000 7fc67562e640 1 -- 172.21.15.88:0/1633505209 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6700cf2b0 con 0x7fc6700cc850 2023-10-16T20:07:15.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.029+0000 7fc67562e640 1 --2- 172.21.15.88:0/1633505209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6700cc850 0x7fc670167ae0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc668009d90 tx=0x7fc66801f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf53af81212a0da server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:15.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.029+0000 7fc6677fe640 1 -- 172.21.15.88:0/1633505209 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6680af600 con 0x7fc6700cc850 2023-10-16T20:07:15.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.029+0000 7fc6677fe640 1 -- 172.21.15.88:0/1633505209 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6680ab050 con 0x7fc6700cc850 2023-10-16T20:07:15.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.029+0000 7fc6677fe640 1 -- 172.21.15.88:0/1633505209 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6680af960 con 0x7fc6700cc850 2023-10-16T20:07:15.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.033+0000 7fc677091640 1 -- 172.21.15.88:0/1633505209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6700cc850 msgr2=0x7fc670167ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:15.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.033+0000 7fc677091640 1 --2- 172.21.15.88:0/1633505209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6700cc850 0x7fc670167ae0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc668009d90 tx=0x7fc66801f130 comp rx=0 tx=0).stop 2023-10-16T20:07:15.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.033+0000 7fc677091640 1 -- 172.21.15.88:0/1633505209 shutdown_connections 2023-10-16T20:07:15.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.033+0000 7fc677091640 1 --2- 172.21.15.88:0/1633505209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6700cc850 0x7fc670167ae0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:15.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.033+0000 7fc677091640 1 -- 172.21.15.88:0/1633505209 >> 172.21.15.88:0/1633505209 conn(0x7fc67006a060 msgr2=0x7fc67006a440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:15.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.033+0000 7fc677091640 1 -- 172.21.15.88:0/1633505209 shutdown_connections 2023-10-16T20:07:15.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.033+0000 7fc677091640 1 -- 172.21.15.88:0/1633505209 wait complete. 2023-10-16T20:07:15.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.033+0000 7fc677091640 1 Processor -- start 2023-10-16T20:07:15.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.033+0000 7fc677091640 1 -- start start 2023-10-16T20:07:15.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.033+0000 7fc677091640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc67015cb80 0x7fc670156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:15.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.033+0000 7fc677091640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc670161400 0x7fc670158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:15.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.033+0000 7fc677091640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc670158770 0x7fc6700b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:15.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.033+0000 7fc677091640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6700b6470 con 0x7fc67015cb80 2023-10-16T20:07:15.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.033+0000 7fc677091640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc670157040 con 0x7fc670161400 2023-10-16T20:07:15.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.033+0000 7fc677091640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6700b5eb0 con 0x7fc670158770 2023-10-16T20:07:15.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.033+0000 7fc674e2d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc67015cb80 0x7fc670156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:15.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.033+0000 7fc674e2d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc67015cb80 0x7fc670156b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:33994/0 (socket says 172.21.15.88:33994) 2023-10-16T20:07:15.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.033+0000 7fc674e2d640 1 -- 172.21.15.88:0/82843014 learned_addr learned my addr 172.21.15.88:0/82843014 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:15.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.033+0000 7fc667fff640 1 --2- 172.21.15.88:0/82843014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc670161400 0x7fc670158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:15.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.033+0000 7fc67562e640 1 --2- 172.21.15.88:0/82843014 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc670158770 0x7fc6700b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:15.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.037+0000 7fc674e2d640 1 -- 172.21.15.88:0/82843014 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc670158770 msgr2=0x7fc6700b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:15.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.037+0000 7fc674e2d640 1 --2- 172.21.15.88:0/82843014 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc670158770 0x7fc6700b53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:15.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.037+0000 7fc674e2d640 1 -- 172.21.15.88:0/82843014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc670161400 msgr2=0x7fc670158230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:15.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.037+0000 7fc674e2d640 1 --2- 172.21.15.88:0/82843014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc670161400 0x7fc670158230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:15.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.037+0000 7fc674e2d640 1 -- 172.21.15.88:0/82843014 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc668009a70 con 0x7fc67015cb80 2023-10-16T20:07:15.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.037+0000 7fc674e2d640 1 --2- 172.21.15.88:0/82843014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc67015cb80 0x7fc670156b00 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fc66c00c4d0 tx=0x7fc66c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:15.044 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.037+0000 7fc6657fa640 1 -- 172.21.15.88:0/82843014 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c005cf0 con 0x7fc67015cb80 2023-10-16T20:07:15.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.037+0000 7fc6657fa640 1 -- 172.21.15.88:0/82843014 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66c005e50 con 0x7fc67015cb80 2023-10-16T20:07:15.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.037+0000 7fc677091640 1 -- 172.21.15.88:0/82843014 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6700b6190 con 0x7fc67015cb80 2023-10-16T20:07:15.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.037+0000 7fc6657fa640 1 -- 172.21.15.88:0/82843014 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c007270 con 0x7fc67015cb80 2023-10-16T20:07:15.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.037+0000 7fc677091640 1 -- 172.21.15.88:0/82843014 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6700c7d90 con 0x7fc67015cb80 2023-10-16T20:07:15.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.037+0000 7fc677091640 1 -- 172.21.15.88:0/82843014 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6700c7910 con 0x7fc67015cb80 2023-10-16T20:07:15.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.037+0000 7fc6657fa640 1 -- 172.21.15.88:0/82843014 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc66c0073d0 con 0x7fc67015cb80 2023-10-16T20:07:15.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.037+0000 7fc6657fa640 1 --2- 172.21.15.88:0/82843014 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc65c0664a0 0x7fc65c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:15.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.037+0000 7fc6657fa640 1 -- 172.21.15.88:0/82843014 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fc66c0834e0 con 0x7fc67015cb80 2023-10-16T20:07:15.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.037+0000 7fc667fff640 1 --2- 172.21.15.88:0/82843014 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc65c0664a0 0x7fc65c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:15.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.041+0000 7fc667fff640 1 --2- 172.21.15.88:0/82843014 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc65c0664a0 0x7fc65c068960 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc670157e20 tx=0x7fc660009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:15.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.041+0000 7fc6657fa640 1 -- 172.21.15.88:0/82843014 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc6700c7910 con 0x7fc67015cb80 2023-10-16T20:07:15.198 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.193+0000 7fc677091640 1 -- 172.21.15.88:0/82843014 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "testcephfs1_data", "pool2": "testcephfs1_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc6700c7910 con 0x7fc67015cb80 2023-10-16T20:07:15.860 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.857+0000 7fc6657fa640 1 -- 172.21.15.88:0/82843014 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "testcephfs1_data", "pool2": "testcephfs1_data", "yes_i_really_really_mean_it": true}]=0 pool 'testcephfs1_data' removed v49) v1 ==== 184+0+0 (secure 0 0 0) 0x7fc6700c7910 con 0x7fc67015cb80 2023-10-16T20:07:15.860 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'testcephfs1_data' removed 2023-10-16T20:07:15.864 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.861+0000 7fc677091640 1 -- 172.21.15.88:0/82843014 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc65c0664a0 msgr2=0x7fc65c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:15.864 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.861+0000 7fc677091640 1 --2- 172.21.15.88:0/82843014 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc65c0664a0 0x7fc65c068960 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc670157e20 tx=0x7fc660009040 comp rx=0 tx=0).stop 2023-10-16T20:07:15.865 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.861+0000 7fc677091640 1 -- 172.21.15.88:0/82843014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc67015cb80 msgr2=0x7fc670156b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:15.865 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.861+0000 7fc677091640 1 --2- 172.21.15.88:0/82843014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc67015cb80 0x7fc670156b00 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fc66c00c4d0 tx=0x7fc66c00c990 comp rx=0 tx=0).stop 2023-10-16T20:07:15.865 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.861+0000 7fc677091640 1 -- 172.21.15.88:0/82843014 shutdown_connections 2023-10-16T20:07:15.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.861+0000 7fc677091640 1 --2- 172.21.15.88:0/82843014 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc65c0664a0 0x7fc65c068960 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:15.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.861+0000 7fc677091640 1 --2- 172.21.15.88:0/82843014 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc670158770 0x7fc6700b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:15.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.861+0000 7fc677091640 1 --2- 172.21.15.88:0/82843014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc670161400 0x7fc670158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:15.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.861+0000 7fc677091640 1 --2- 172.21.15.88:0/82843014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc67015cb80 0x7fc670156b00 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:15.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.861+0000 7fc677091640 1 -- 172.21.15.88:0/82843014 >> 172.21.15.88:0/82843014 conn(0x7fc67006a060 msgr2=0x7fc67006a440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:15.867 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.861+0000 7fc677091640 1 -- 172.21.15.88:0/82843014 shutdown_connections 2023-10-16T20:07:15.868 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:15.865+0000 7fc677091640 1 -- 172.21.15.88:0/82843014 wait complete. 2023-10-16T20:07:15.894 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm data_new data_new --yes-i-really-really-mean-it 2023-10-16T20:07:16.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.033+0000 7f5609e6b640 1 Processor -- start 2023-10-16T20:07:16.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.033+0000 7f5609e6b640 1 -- start start 2023-10-16T20:07:16.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.033+0000 7f5609e6b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f560406a5a0 0x7f56040ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:16.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.033+0000 7f5609e6b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f56040c91b0 0x7f56040b1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:16.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.033+0000 7f5609e6b640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56040c9600 con 0x7f560415ab20 2023-10-16T20:07:16.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.033+0000 7f5609e6b640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f560415af00 con 0x7f560406a5a0 2023-10-16T20:07:16.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.033+0000 7f5609e6b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56040b4630 con 0x7f56040c91b0 2023-10-16T20:07:16.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.033+0000 7f5603fff640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f56040c91b0 0x7f56040b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:16.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.033+0000 7f5602ffd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f560406a5a0 0x7f56040ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:16.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.033+0000 7f56037fe640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f560415ab20 0x7f5604069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:44218/0 (socket says 172.21.15.88:44218) 2023-10-16T20:07:16.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.033+0000 7f56037fe640 1 -- 172.21.15.88:0/4059724015 learned_addr learned my addr 172.21.15.88:0/4059724015 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:16.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.033+0000 7f5603fff640 1 -- 172.21.15.88:0/4059724015 >> v2:172.21.15.94:3300/0 conn(0x7f560406a5a0 msgr2=0x7f56040ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:16.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.033+0000 7f5603fff640 1 --2- 172.21.15.88:0/4059724015 >> v2:172.21.15.94:3300/0 conn(0x7f560406a5a0 0x7f56040ca7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:16.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.033+0000 7f5603fff640 1 -- 172.21.15.88:0/4059724015 >> v1:172.21.15.94:6789/0 conn(0x7f560415ab20 legacy=0x7f5604069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:16.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.033+0000 7f5603fff640 1 -- 172.21.15.88:0/4059724015 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56040c9840 con 0x7f56040c91b0 2023-10-16T20:07:16.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.037+0000 7f5603fff640 1 --2- 172.21.15.88:0/4059724015 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f56040c91b0 0x7f56040b1870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f55fc009d90 tx=0x7f55fc01c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1f42ee1449669f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:16.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.037+0000 7f56027fc640 1 -- 172.21.15.88:0/4059724015 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc0a5a10 con 0x7f56040c91b0 2023-10-16T20:07:16.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.037+0000 7f56027fc640 1 -- 172.21.15.88:0/4059724015 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55fc0a5b70 con 0x7f56040c91b0 2023-10-16T20:07:16.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.037+0000 7f56027fc640 1 -- 172.21.15.88:0/4059724015 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc0ad440 con 0x7f56040c91b0 2023-10-16T20:07:16.044 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.037+0000 7f5609e6b640 1 -- 172.21.15.88:0/4059724015 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f56040c91b0 msgr2=0x7f56040b1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:16.044 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.037+0000 7f5609e6b640 1 --2- 172.21.15.88:0/4059724015 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f56040c91b0 0x7f56040b1870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f55fc009d90 tx=0x7f55fc01c950 comp rx=0 tx=0).stop 2023-10-16T20:07:16.044 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.037+0000 7f5609e6b640 1 -- 172.21.15.88:0/4059724015 shutdown_connections 2023-10-16T20:07:16.044 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.037+0000 7f5609e6b640 1 --2- 172.21.15.88:0/4059724015 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f56040c91b0 0x7f56040b1870 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:16.044 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.037+0000 7f5609e6b640 1 --2- 172.21.15.88:0/4059724015 >> v2:172.21.15.94:3300/0 conn(0x7f560406a5a0 0x7f56040ca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:16.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.037+0000 7f5609e6b640 1 -- 172.21.15.88:0/4059724015 >> 172.21.15.88:0/4059724015 conn(0x7f56040ce660 msgr2=0x7f56040cea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:16.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.037+0000 7f5609e6b640 1 -- 172.21.15.88:0/4059724015 shutdown_connections 2023-10-16T20:07:16.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.037+0000 7f5609e6b640 1 -- 172.21.15.88:0/4059724015 wait complete. 2023-10-16T20:07:16.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.041+0000 7f5609e6b640 1 Processor -- start 2023-10-16T20:07:16.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.041+0000 7f5609e6b640 1 -- start start 2023-10-16T20:07:16.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.041+0000 7f5609e6b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f560406a5a0 0x7f560414b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:16.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.041+0000 7f5609e6b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56040c91b0 0x7f5604154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:16.048 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.041+0000 7f5609e6b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f560415ab20 0x7f56041581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:16.048 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.041+0000 7f5609e6b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560414bf70 con 0x7f56040c91b0 2023-10-16T20:07:16.048 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.041+0000 7f5609e6b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5604157680 con 0x7f560406a5a0 2023-10-16T20:07:16.048 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.041+0000 7f5609e6b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56041579e0 con 0x7f560415ab20 2023-10-16T20:07:16.048 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.041+0000 7f5602ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56040c91b0 0x7f5604154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:16.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.041+0000 7f5602ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56040c91b0 0x7f5604154700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34004/0 (socket says 172.21.15.88:34004) 2023-10-16T20:07:16.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.041+0000 7f5602ffd640 1 -- 172.21.15.88:0/2239630183 learned_addr learned my addr 172.21.15.88:0/2239630183 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:16.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.041+0000 7f5603fff640 1 --2- 172.21.15.88:0/2239630183 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f560415ab20 0x7f56041581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:16.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.041+0000 7f56037fe640 1 --2- 172.21.15.88:0/2239630183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f560406a5a0 0x7f560414b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:16.050 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.041+0000 7f5602ffd640 1 -- 172.21.15.88:0/2239630183 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f560415ab20 msgr2=0x7f56041581a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:16.050 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.041+0000 7f5602ffd640 1 --2- 172.21.15.88:0/2239630183 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f560415ab20 0x7f56041581a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:16.050 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.041+0000 7f5602ffd640 1 -- 172.21.15.88:0/2239630183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f560406a5a0 msgr2=0x7f560414b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:16.050 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.041+0000 7f5602ffd640 1 --2- 172.21.15.88:0/2239630183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f560406a5a0 0x7f560414b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:16.050 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.041+0000 7f5602ffd640 1 -- 172.21.15.88:0/2239630183 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55fc009a70 con 0x7f56040c91b0 2023-10-16T20:07:16.051 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.041+0000 7f56037fe640 1 --2- 172.21.15.88:0/2239630183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f560406a5a0 0x7f560414b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:07:16.051 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.041+0000 7f5602ffd640 1 --2- 172.21.15.88:0/2239630183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56040c91b0 0x7f5604154700 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f55f800e930 tx=0x7f55f800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:16.051 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.041+0000 7f55ebfff640 1 -- 172.21.15.88:0/2239630183 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f8009690 con 0x7f56040c91b0 2023-10-16T20:07:16.051 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.041+0000 7f55ebfff640 1 -- 172.21.15.88:0/2239630183 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f800ca00 con 0x7f56040c91b0 2023-10-16T20:07:16.052 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.041+0000 7f55ebfff640 1 -- 172.21.15.88:0/2239630183 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f800cd70 con 0x7f56040c91b0 2023-10-16T20:07:16.052 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.041+0000 7f5609e6b640 1 -- 172.21.15.88:0/2239630183 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56041570c0 con 0x7f56040c91b0 2023-10-16T20:07:16.053 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.045+0000 7f5609e6b640 1 -- 172.21.15.88:0/2239630183 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5604157c80 con 0x7f56040c91b0 2023-10-16T20:07:16.053 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.045+0000 7f5609e6b640 1 -- 172.21.15.88:0/2239630183 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f560404e9c0 con 0x7f56040c91b0 2023-10-16T20:07:16.053 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.045+0000 7f55ebfff640 1 -- 172.21.15.88:0/2239630183 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f55f8013470 con 0x7f56040c91b0 2023-10-16T20:07:16.053 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.045+0000 7f55ebfff640 1 --2- 172.21.15.88:0/2239630183 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f55f00664a0 0x7f55f0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:16.054 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.045+0000 7f55ebfff640 1 -- 172.21.15.88:0/2239630183 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5663+0+0 (secure 0 0 0) 0x7f55f8084030 con 0x7f56040c91b0 2023-10-16T20:07:16.054 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.045+0000 7f56037fe640 1 --2- 172.21.15.88:0/2239630183 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f55f00664a0 0x7f55f0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:16.054 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.045+0000 7f56037fe640 1 --2- 172.21.15.88:0/2239630183 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f55f00664a0 0x7f55f0068960 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f55f4000f70 tx=0x7f55f4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:16.055 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.049+0000 7f55ebfff640 1 -- 172.21.15.88:0/2239630183 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f55f80b5090 con 0x7f56040c91b0 2023-10-16T20:07:16.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:16.201+0000 7f5609e6b640 1 -- 172.21.15.88:0/2239630183 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f56040ab8a0 con 0x7f56040c91b0 2023-10-16T20:07:17.210 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.205+0000 7f55ebfff640 1 -- 172.21.15.88:0/2239630183 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true}]=0 pool 'data_new' removed v50) v1 ==== 160+0+0 (secure 0 0 0) 0x7f55f80170d0 con 0x7f56040c91b0 2023-10-16T20:07:17.211 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'data_new' removed 2023-10-16T20:07:17.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.209+0000 7f5609e6b640 1 -- 172.21.15.88:0/2239630183 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f55f00664a0 msgr2=0x7f55f0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:17.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.209+0000 7f5609e6b640 1 --2- 172.21.15.88:0/2239630183 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f55f00664a0 0x7f55f0068960 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f55f4000f70 tx=0x7f55f4009040 comp rx=0 tx=0).stop 2023-10-16T20:07:17.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.209+0000 7f5609e6b640 1 -- 172.21.15.88:0/2239630183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56040c91b0 msgr2=0x7f5604154700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:17.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.209+0000 7f5609e6b640 1 --2- 172.21.15.88:0/2239630183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56040c91b0 0x7f5604154700 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f55f800e930 tx=0x7f55f800edf0 comp rx=0 tx=0).stop 2023-10-16T20:07:17.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.209+0000 7f5609e6b640 1 -- 172.21.15.88:0/2239630183 shutdown_connections 2023-10-16T20:07:17.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.209+0000 7f5609e6b640 1 --2- 172.21.15.88:0/2239630183 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f55f00664a0 0x7f55f0068960 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:17.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.209+0000 7f5609e6b640 1 --2- 172.21.15.88:0/2239630183 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f560415ab20 0x7f56041581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:17.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.209+0000 7f5609e6b640 1 --2- 172.21.15.88:0/2239630183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56040c91b0 0x7f5604154700 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:17.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.209+0000 7f5609e6b640 1 --2- 172.21.15.88:0/2239630183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f560406a5a0 0x7f560414b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:17.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.209+0000 7f5609e6b640 1 -- 172.21.15.88:0/2239630183 >> 172.21.15.88:0/2239630183 conn(0x7f56040ce660 msgr2=0x7f5604069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:17.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.209+0000 7f5609e6b640 1 -- 172.21.15.88:0/2239630183 shutdown_connections 2023-10-16T20:07:17.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.209+0000 7f5609e6b640 1 -- 172.21.15.88:0/2239630183 wait complete. 2023-10-16T20:07:17.239 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T20:07:17.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.377+0000 7fc1a053e640 1 Processor -- start 2023-10-16T20:07:17.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.377+0000 7fc1a053e640 1 -- start start 2023-10-16T20:07:17.384 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.381+0000 7fc1a053e640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc1980b4260 0x7fc19815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:17.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.381+0000 7fc1a053e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fc19806d3f0 0x7fc19806d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:17.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.381+0000 7fc1a053e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc19815ad30 con 0x7fc1980b0da0 2023-10-16T20:07:17.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.381+0000 7fc1a053e640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc19806dd60 con 0x7fc1980b4260 2023-10-16T20:07:17.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.381+0000 7fc1a053e640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1980b4640 con 0x7fc19806d3f0 2023-10-16T20:07:17.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.381+0000 7fc19e2da640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc1980b4260 0x7fc19815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:17.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.381+0000 7fc19e2da640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc1980b4260 0x7fc19815dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34018/0 (socket says 172.21.15.88:34018) 2023-10-16T20:07:17.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.381+0000 7fc19e2da640 1 -- 172.21.15.88:0/2590615071 learned_addr learned my addr 172.21.15.88:0/2590615071 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:17.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.381+0000 7fc19eadb640 1 --2- 172.21.15.88:0/2590615071 >> v2:172.21.15.94:3300/0 conn(0x7fc19806d3f0 0x7fc19806d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:17.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.381+0000 7fc19e2da640 1 -- 172.21.15.88:0/2590615071 >> v2:172.21.15.94:3300/0 conn(0x7fc19806d3f0 msgr2=0x7fc19806d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:17.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.381+0000 7fc19e2da640 1 --2- 172.21.15.88:0/2590615071 >> v2:172.21.15.94:3300/0 conn(0x7fc19806d3f0 0x7fc19806d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:17.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.381+0000 7fc19e2da640 1 -- 172.21.15.88:0/2590615071 >> v1:172.21.15.94:6789/0 conn(0x7fc1980b0da0 legacy=0x7fc198159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:17.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.381+0000 7fc19e2da640 1 -- 172.21.15.88:0/2590615071 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19815a770 con 0x7fc1980b4260 2023-10-16T20:07:17.388 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.381+0000 7fc19e2da640 1 --2- 172.21.15.88:0/2590615071 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1980b4260 0x7fc19815dc90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fc1980b1230 tx=0x7fc19401ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7acbec25d8f2342 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:17.388 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.385+0000 7fc19d2d8640 1 -- 172.21.15.88:0/2590615071 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1940a7ba0 con 0x7fc1980b4260 2023-10-16T20:07:17.388 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.385+0000 7fc19d2d8640 1 -- 172.21.15.88:0/2590615071 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1940a7d00 con 0x7fc1980b4260 2023-10-16T20:07:17.388 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.385+0000 7fc19d2d8640 1 -- 172.21.15.88:0/2590615071 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1940af660 con 0x7fc1980b4260 2023-10-16T20:07:17.389 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.385+0000 7fc1a053e640 1 -- 172.21.15.88:0/2590615071 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1980b4260 msgr2=0x7fc19815dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:17.389 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.385+0000 7fc1a053e640 1 --2- 172.21.15.88:0/2590615071 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1980b4260 0x7fc19815dc90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fc1980b1230 tx=0x7fc19401ead0 comp rx=0 tx=0).stop 2023-10-16T20:07:17.390 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.385+0000 7fc1a053e640 1 -- 172.21.15.88:0/2590615071 shutdown_connections 2023-10-16T20:07:17.390 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.385+0000 7fc1a053e640 1 --2- 172.21.15.88:0/2590615071 >> v2:172.21.15.94:3300/0 conn(0x7fc19806d3f0 0x7fc19806d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:17.390 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.385+0000 7fc1a053e640 1 --2- 172.21.15.88:0/2590615071 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1980b4260 0x7fc19815dc90 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:17.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.385+0000 7fc1a053e640 1 -- 172.21.15.88:0/2590615071 >> 172.21.15.88:0/2590615071 conn(0x7fc1980cb080 msgr2=0x7fc1980cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:17.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.385+0000 7fc1a053e640 1 -- 172.21.15.88:0/2590615071 shutdown_connections 2023-10-16T20:07:17.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.385+0000 7fc1a053e640 1 -- 172.21.15.88:0/2590615071 wait complete. 2023-10-16T20:07:17.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.385+0000 7fc1a053e640 1 Processor -- start 2023-10-16T20:07:17.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.385+0000 7fc1a053e640 1 -- start start 2023-10-16T20:07:17.393 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.385+0000 7fc1a053e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc19806d3f0 0x7fc1981518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:17.393 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.385+0000 7fc1a053e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1980b0da0 0x7fc1980cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:17.394 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.385+0000 7fc1a053e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1980b4260 0x7fc19814de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:17.394 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.385+0000 7fc1a053e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19815b1f0 con 0x7fc19806d3f0 2023-10-16T20:07:17.394 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.385+0000 7fc1a053e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19815aee0 con 0x7fc1980b0da0 2023-10-16T20:07:17.394 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.385+0000 7fc1a053e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1980ccf60 con 0x7fc1980b4260 2023-10-16T20:07:17.394 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.385+0000 7fc19e2da640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc19806d3f0 0x7fc1981518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:17.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.385+0000 7fc19e2da640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc19806d3f0 0x7fc1981518b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34034/0 (socket says 172.21.15.88:34034) 2023-10-16T20:07:17.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.385+0000 7fc19e2da640 1 -- 172.21.15.88:0/4021446493 learned_addr learned my addr 172.21.15.88:0/4021446493 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:17.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.389+0000 7fc19eadb640 1 --2- 172.21.15.88:0/4021446493 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1980b4260 0x7fc19814de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:17.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.389+0000 7fc19dad9640 1 --2- 172.21.15.88:0/4021446493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1980b0da0 0x7fc1980cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:17.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.389+0000 7fc19e2da640 1 -- 172.21.15.88:0/4021446493 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1980b4260 msgr2=0x7fc19814de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:17.396 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.389+0000 7fc19e2da640 1 --2- 172.21.15.88:0/4021446493 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1980b4260 0x7fc19814de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:17.396 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.389+0000 7fc19e2da640 1 -- 172.21.15.88:0/4021446493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1980b0da0 msgr2=0x7fc1980cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:17.396 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.389+0000 7fc19e2da640 1 --2- 172.21.15.88:0/4021446493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1980b0da0 0x7fc1980cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:17.396 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.389+0000 7fc19e2da640 1 -- 172.21.15.88:0/4021446493 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19400b770 con 0x7fc19806d3f0 2023-10-16T20:07:17.396 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.389+0000 7fc19e2da640 1 --2- 172.21.15.88:0/4021446493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc19806d3f0 0x7fc1981518b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fc19400b2c0 tx=0x7fc19401b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:17.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.389+0000 7fc18effd640 1 -- 172.21.15.88:0/4021446493 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1940afe20 con 0x7fc19806d3f0 2023-10-16T20:07:17.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.389+0000 7fc18effd640 1 -- 172.21.15.88:0/4021446493 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1940ba440 con 0x7fc19806d3f0 2023-10-16T20:07:17.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.389+0000 7fc1a053e640 1 -- 172.21.15.88:0/4021446493 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc19814e370 con 0x7fc19806d3f0 2023-10-16T20:07:17.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.389+0000 7fc18effd640 1 -- 172.21.15.88:0/4021446493 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1940ba7f0 con 0x7fc19806d3f0 2023-10-16T20:07:17.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.389+0000 7fc1a053e640 1 -- 172.21.15.88:0/4021446493 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc198152270 con 0x7fc19806d3f0 2023-10-16T20:07:17.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.389+0000 7fc1a053e640 1 -- 172.21.15.88:0/4021446493 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc19804e9c0 con 0x7fc19806d3f0 2023-10-16T20:07:17.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.389+0000 7fc18effd640 1 -- 172.21.15.88:0/4021446493 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc1940b93e0 con 0x7fc19806d3f0 2023-10-16T20:07:17.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.389+0000 7fc18effd640 1 --2- 172.21.15.88:0/4021446493 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc17c0664a0 0x7fc17c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:17.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.389+0000 7fc18effd640 1 -- 172.21.15.88:0/4021446493 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fc19411e830 con 0x7fc19806d3f0 2023-10-16T20:07:17.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.393+0000 7fc19dad9640 1 --2- 172.21.15.88:0/4021446493 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc17c0664a0 0x7fc17c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:17.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.393+0000 7fc19dad9640 1 --2- 172.21.15.88:0/4021446493 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc17c0664a0 0x7fc17c068960 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc1980701c0 tx=0x7fc188009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:17.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.397+0000 7fc18effd640 1 -- 172.21.15.88:0/4021446493 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc1940ebdd0 con 0x7fc19806d3f0 2023-10-16T20:07:17.554 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.549+0000 7fc1a053e640 1 -- 172.21.15.88:0/4021446493 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc19804e9c0 con 0x7fc19806d3f0 2023-10-16T20:07:17.558 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.553+0000 7fc18effd640 1 -- 172.21.15.88:0/4021446493 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc1940ef3f0 con 0x7fc19806d3f0 2023-10-16T20:07:17.560 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.557+0000 7fc1a053e640 1 -- 172.21.15.88:0/4021446493 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc17c0664a0 msgr2=0x7fc17c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:17.561 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.557+0000 7fc1a053e640 1 --2- 172.21.15.88:0/4021446493 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc17c0664a0 0x7fc17c068960 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc1980701c0 tx=0x7fc188009040 comp rx=0 tx=0).stop 2023-10-16T20:07:17.561 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.557+0000 7fc1a053e640 1 -- 172.21.15.88:0/4021446493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc19806d3f0 msgr2=0x7fc1981518b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:17.561 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.557+0000 7fc1a053e640 1 --2- 172.21.15.88:0/4021446493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc19806d3f0 0x7fc1981518b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fc19400b2c0 tx=0x7fc19401b870 comp rx=0 tx=0).stop 2023-10-16T20:07:17.561 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.557+0000 7fc1a053e640 1 -- 172.21.15.88:0/4021446493 shutdown_connections 2023-10-16T20:07:17.561 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.557+0000 7fc1a053e640 1 --2- 172.21.15.88:0/4021446493 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc17c0664a0 0x7fc17c068960 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:17.562 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.557+0000 7fc1a053e640 1 --2- 172.21.15.88:0/4021446493 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1980b4260 0x7fc19814de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:17.562 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.557+0000 7fc1a053e640 1 --2- 172.21.15.88:0/4021446493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1980b0da0 0x7fc1980cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:17.562 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.557+0000 7fc1a053e640 1 --2- 172.21.15.88:0/4021446493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc19806d3f0 0x7fc1981518b0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:17.562 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.557+0000 7fc1a053e640 1 -- 172.21.15.88:0/4021446493 >> 172.21.15.88:0/4021446493 conn(0x7fc1980cb080 msgr2=0x7fc19806ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:17.563 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.557+0000 7fc1a053e640 1 -- 172.21.15.88:0/4021446493 shutdown_connections 2023-10-16T20:07:17.563 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.557+0000 7fc1a053e640 1 -- 172.21.15.88:0/4021446493 wait complete. 2023-10-16T20:07:17.584 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification' 2023-10-16T20:07:17.721 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.717+0000 7f5ef690f640 1 Processor -- start 2023-10-16T20:07:17.722 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.717+0000 7f5ef690f640 1 -- start start 2023-10-16T20:07:17.723 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.717+0000 7f5ef690f640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5ef015d9f0 0x7f5ef0161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:17.723 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.717+0000 7f5ef690f640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5ef00d26c0 0x7f5ef00cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:17.724 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.717+0000 7f5ef690f640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ef00b87e0 con 0x7f5ef00d22e0 2023-10-16T20:07:17.724 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.717+0000 7f5ef690f640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ef015df10 con 0x7f5ef015d9f0 2023-10-16T20:07:17.724 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.717+0000 7f5ef690f640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ef015e1f0 con 0x7f5ef00d26c0 2023-10-16T20:07:17.724 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.717+0000 7f5eeffff640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f5ef00d22e0 0x7f5ef015d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:39550/0 (socket says 172.21.15.88:39550) 2023-10-16T20:07:17.725 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.717+0000 7f5eeffff640 1 -- 172.21.15.88:0/3841263953 learned_addr learned my addr 172.21.15.88:0/3841263953 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:17.725 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.717+0000 7f5eef7fe640 1 --2- 172.21.15.88:0/3841263953 >> v2:172.21.15.88:3300/0 conn(0x7f5ef015d9f0 0x7f5ef0161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:17.725 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.717+0000 7f5ef4eac640 1 --2- 172.21.15.88:0/3841263953 >> v2:172.21.15.94:3300/0 conn(0x7f5ef00d26c0 0x7f5ef00cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:17.725 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.717+0000 7f5eeeffd640 1 -- 172.21.15.88:0/3841263953 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 746301977 0 0) 0x7f5ef00b87e0 con 0x7f5ef00d22e0 2023-10-16T20:07:17.726 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.717+0000 7f5eeeffd640 1 -- 172.21.15.88:0/3841263953 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ed8003530 con 0x7f5ef00d22e0 2023-10-16T20:07:17.726 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.717+0000 7f5eef7fe640 1 -- 172.21.15.88:0/3841263953 >> v2:172.21.15.94:3300/0 conn(0x7f5ef00d26c0 msgr2=0x7f5ef00cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:17.726 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.717+0000 7f5eef7fe640 1 --2- 172.21.15.88:0/3841263953 >> v2:172.21.15.94:3300/0 conn(0x7f5ef00d26c0 0x7f5ef00cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:17.726 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.717+0000 7f5eef7fe640 1 -- 172.21.15.88:0/3841263953 >> v1:172.21.15.88:6789/0 conn(0x7f5ef00d22e0 legacy=0x7f5ef015d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:17.727 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.717+0000 7f5eef7fe640 1 -- 172.21.15.88:0/3841263953 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef0160970 con 0x7f5ef015d9f0 2023-10-16T20:07:17.728 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.721+0000 7f5eef7fe640 1 --2- 172.21.15.88:0/3841263953 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ef015d9f0 0x7f5ef0161450 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0157930 tx=0x7f5ee401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b0cff36d861a12 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:17.728 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.721+0000 7f5eeeffd640 1 -- 172.21.15.88:0/3841263953 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee40ac600 con 0x7f5ef015d9f0 2023-10-16T20:07:17.729 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.721+0000 7f5eeeffd640 1 -- 172.21.15.88:0/3841263953 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee40a8050 con 0x7f5ef015d9f0 2023-10-16T20:07:17.729 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.721+0000 7f5eeeffd640 1 -- 172.21.15.88:0/3841263953 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee40ac980 con 0x7f5ef015d9f0 2023-10-16T20:07:17.729 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.721+0000 7f5ef690f640 1 -- 172.21.15.88:0/3841263953 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ef015d9f0 msgr2=0x7f5ef0161450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:17.730 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.721+0000 7f5ef690f640 1 --2- 172.21.15.88:0/3841263953 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ef015d9f0 0x7f5ef0161450 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0157930 tx=0x7f5ee401c3a0 comp rx=0 tx=0).stop 2023-10-16T20:07:17.730 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.721+0000 7f5ef690f640 1 -- 172.21.15.88:0/3841263953 shutdown_connections 2023-10-16T20:07:17.730 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.721+0000 7f5ef690f640 1 --2- 172.21.15.88:0/3841263953 >> v2:172.21.15.94:3300/0 conn(0x7f5ef00d26c0 0x7f5ef00cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:17.730 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.721+0000 7f5ef690f640 1 --2- 172.21.15.88:0/3841263953 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ef015d9f0 0x7f5ef0161450 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:17.731 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.721+0000 7f5ef690f640 1 -- 172.21.15.88:0/3841263953 >> 172.21.15.88:0/3841263953 conn(0x7f5ef00715c0 msgr2=0x7f5ef00719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:17.731 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.721+0000 7f5ef690f640 1 -- 172.21.15.88:0/3841263953 shutdown_connections 2023-10-16T20:07:17.731 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.721+0000 7f5ef690f640 1 -- 172.21.15.88:0/3841263953 wait complete. 2023-10-16T20:07:17.733 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.725+0000 7f5ef690f640 1 Processor -- start 2023-10-16T20:07:17.734 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.725+0000 7f5ef690f640 1 -- start start 2023-10-16T20:07:17.734 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.725+0000 7f5ef690f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ef00d22e0 0x7f5ef00b7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:17.734 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.725+0000 7f5ef690f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ef00d26c0 0x7f5ef00b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:17.734 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.725+0000 7f5ef690f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5ef015d9f0 0x7f5ef00b7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:17.734 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.725+0000 7f5ef690f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef015f0c0 con 0x7f5ef00d26c0 2023-10-16T20:07:17.735 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.725+0000 7f5ef690f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef015f860 con 0x7f5ef00d22e0 2023-10-16T20:07:17.735 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.725+0000 7f5ef690f640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef00b6ba0 con 0x7f5ef015d9f0 2023-10-16T20:07:17.735 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.725+0000 7f5eef7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ef00d26c0 0x7f5ef00b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:17.735 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.725+0000 7f5eef7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ef00d26c0 0x7f5ef00b7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34044/0 (socket says 172.21.15.88:34044) 2023-10-16T20:07:17.736 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.725+0000 7f5eef7fe640 1 -- 172.21.15.88:0/3812788724 learned_addr learned my addr 172.21.15.88:0/3812788724 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:17.736 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.725+0000 7f5eeffff640 1 --2- 172.21.15.88:0/3812788724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ef00d22e0 0x7f5ef00b7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:17.736 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.725+0000 7f5ef4eac640 1 --2- 172.21.15.88:0/3812788724 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5ef015d9f0 0x7f5ef00b7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:17.736 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.725+0000 7f5eef7fe640 1 -- 172.21.15.88:0/3812788724 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5ef015d9f0 msgr2=0x7f5ef00b7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:17.736 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.725+0000 7f5eef7fe640 1 --2- 172.21.15.88:0/3812788724 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5ef015d9f0 0x7f5ef00b7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:17.737 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.725+0000 7f5eef7fe640 1 -- 172.21.15.88:0/3812788724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ef00d22e0 msgr2=0x7f5ef00b7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:17.737 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.725+0000 7f5eef7fe640 1 --2- 172.21.15.88:0/3812788724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ef00d22e0 0x7f5ef00b7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:17.737 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.725+0000 7f5eef7fe640 1 -- 172.21.15.88:0/3812788724 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee4009420 con 0x7f5ef00d26c0 2023-10-16T20:07:17.737 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.725+0000 7f5eef7fe640 1 --2- 172.21.15.88:0/3812788724 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ef00d26c0 0x7f5ef00b7880 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4018fb0 tx=0x7f5ee401abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:17.737 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.725+0000 7f5eecff9640 1 -- 172.21.15.88:0/3812788724 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee40b7450 con 0x7f5ef00d26c0 2023-10-16T20:07:17.738 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.725+0000 7f5eecff9640 1 -- 172.21.15.88:0/3812788724 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee40b75d0 con 0x7f5ef00d26c0 2023-10-16T20:07:17.738 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.725+0000 7f5eecff9640 1 -- 172.21.15.88:0/3812788724 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee40b7980 con 0x7f5ef00d26c0 2023-10-16T20:07:17.738 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.725+0000 7f5ef690f640 1 -- 172.21.15.88:0/3812788724 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef015f2a0 con 0x7f5ef00d26c0 2023-10-16T20:07:17.738 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.729+0000 7f5ef690f640 1 -- 172.21.15.88:0/3812788724 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef00b6e40 con 0x7f5ef00d26c0 2023-10-16T20:07:17.738 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.729+0000 7f5ef690f640 1 -- 172.21.15.88:0/3812788724 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef004e9c0 con 0x7f5ef00d26c0 2023-10-16T20:07:17.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.729+0000 7f5eecff9640 1 -- 172.21.15.88:0/3812788724 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5ee40b3070 con 0x7f5ef00d26c0 2023-10-16T20:07:17.740 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.729+0000 7f5eecff9640 1 --2- 172.21.15.88:0/3812788724 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5edc0664a0 0x7f5edc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:17.740 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.729+0000 7f5eecff9640 1 -- 172.21.15.88:0/3812788724 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f5ee411e130 con 0x7f5ef00d26c0 2023-10-16T20:07:17.740 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.729+0000 7f5eeffff640 1 --2- 172.21.15.88:0/3812788724 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5edc0664a0 0x7f5edc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:17.740 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.729+0000 7f5eeffff640 1 --2- 172.21.15.88:0/3812788724 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5edc0664a0 0x7f5edc068960 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00011f0 tx=0x7f5ee0007d40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:17.740 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.733+0000 7f5eecff9640 1 -- 172.21.15.88:0/3812788724 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5ee40eb810 con 0x7f5ef00d26c0 2023-10-16T20:07:17.941 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:17.937+0000 7f5ef690f640 1 -- 172.21.15.88:0/3812788724 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]} v 0) v1 -- 0x7f5ef00d1b40 con 0x7f5ef00d26c0 2023-10-16T20:07:18.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.209+0000 7f5eecff9640 1 -- 172.21.15.88:0/3812788724 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]}]=0 v149) v1 ==== 147+0+0 (secure 0 0 0) 0x7f5ee40eee30 con 0x7f5ef00d26c0 2023-10-16T20:07:18.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.209+0000 7f5ef690f640 1 -- 172.21.15.88:0/3812788724 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5edc0664a0 msgr2=0x7f5edc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:18.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.209+0000 7f5ef690f640 1 --2- 172.21.15.88:0/3812788724 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5edc0664a0 0x7f5edc068960 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00011f0 tx=0x7f5ee0007d40 comp rx=0 tx=0).stop 2023-10-16T20:07:18.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.209+0000 7f5ef690f640 1 -- 172.21.15.88:0/3812788724 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ef00d26c0 msgr2=0x7f5ef00b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:18.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.209+0000 7f5ef690f640 1 --2- 172.21.15.88:0/3812788724 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ef00d26c0 0x7f5ef00b7880 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4018fb0 tx=0x7f5ee401abe0 comp rx=0 tx=0).stop 2023-10-16T20:07:18.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.209+0000 7f5ef690f640 1 -- 172.21.15.88:0/3812788724 shutdown_connections 2023-10-16T20:07:18.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.209+0000 7f5ef690f640 1 --2- 172.21.15.88:0/3812788724 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5edc0664a0 0x7f5edc068960 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:18.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.209+0000 7f5ef690f640 1 --2- 172.21.15.88:0/3812788724 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5ef015d9f0 0x7f5ef00b7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:18.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.209+0000 7f5ef690f640 1 --2- 172.21.15.88:0/3812788724 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ef00d26c0 0x7f5ef00b7880 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:18.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.209+0000 7f5ef690f640 1 --2- 172.21.15.88:0/3812788724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ef00d22e0 0x7f5ef00b7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:18.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.209+0000 7f5ef690f640 1 -- 172.21.15.88:0/3812788724 >> 172.21.15.88:0/3812788724 conn(0x7f5ef00715c0 msgr2=0x7f5ef015bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:18.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.209+0000 7f5ef690f640 1 -- 172.21.15.88:0/3812788724 shutdown_connections 2023-10-16T20:07:18.217 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.209+0000 7f5ef690f640 1 -- 172.21.15.88:0/3812788724 wait complete. 2023-10-16T20:07:18.235 INFO:tasks.cephfs_test_runner:That a client who does not have the capability for setting ... ok 2023-10-16T20:07:18.235 INFO:tasks.cephfs_test_runner:Starting test: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-10-16T20:07:18.236 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:07:18.374 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.369+0000 7f28c9e09640 1 Processor -- start 2023-10-16T20:07:18.374 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.369+0000 7f28c9e09640 1 -- start start 2023-10-16T20:07:18.376 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.369+0000 7f28c9e09640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f28c40ada40 0x7f28c40b1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:18.376 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.369+0000 7f28c9e09640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f28c4143eb0 0x7f28c40b0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:18.376 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.369+0000 7f28c9e09640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c40b2570 con 0x7f28c40ade20 2023-10-16T20:07:18.376 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.369+0000 7f28c9e09640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c40cca50 con 0x7f28c4143eb0 2023-10-16T20:07:18.377 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.369+0000 7f28c9e09640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c4144320 con 0x7f28c40ada40 2023-10-16T20:07:18.377 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.369+0000 7f28bbfff640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f28c40ade20 0x7f28c415adb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:39558/0 (socket says 172.21.15.88:39558) 2023-10-16T20:07:18.377 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.373+0000 7f28bbfff640 1 -- 172.21.15.88:0/1731497451 learned_addr learned my addr 172.21.15.88:0/1731497451 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:18.377 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.373+0000 7f28c3fff640 1 --2- 172.21.15.88:0/1731497451 >> v2:172.21.15.88:3300/0 conn(0x7f28c4143eb0 0x7f28c40b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:18.377 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.373+0000 7f28c37fe640 1 --2- 172.21.15.88:0/1731497451 >> v2:172.21.15.94:3300/0 conn(0x7f28c40ada40 0x7f28c40b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:18.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.373+0000 7f28c2ffd640 1 -- 172.21.15.88:0/1731497451 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2967792964 0 0) 0x7f28c40b2570 con 0x7f28c40ade20 2023-10-16T20:07:18.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.373+0000 7f28c2ffd640 1 -- 172.21.15.88:0/1731497451 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28b0003530 con 0x7f28c40ade20 2023-10-16T20:07:18.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.373+0000 7f28c3fff640 1 -- 172.21.15.88:0/1731497451 >> v2:172.21.15.94:3300/0 conn(0x7f28c40ada40 msgr2=0x7f28c40b1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:18.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.373+0000 7f28c3fff640 1 --2- 172.21.15.88:0/1731497451 >> v2:172.21.15.94:3300/0 conn(0x7f28c40ada40 0x7f28c40b1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:18.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.373+0000 7f28c3fff640 1 -- 172.21.15.88:0/1731497451 >> v1:172.21.15.88:6789/0 conn(0x7f28c40ade20 legacy=0x7f28c415adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:18.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.373+0000 7f28c3fff640 1 -- 172.21.15.88:0/1731497451 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c40b1fb0 con 0x7f28c4143eb0 2023-10-16T20:07:18.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.373+0000 7f28c3fff640 1 --2- 172.21.15.88:0/1731497451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28c4143eb0 0x7f28c40b0500 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f28c40b10f0 tx=0x7f28bc01c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23be08401327ca76 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:18.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.373+0000 7f28c2ffd640 1 -- 172.21.15.88:0/1731497451 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc0a5820 con 0x7f28c4143eb0 2023-10-16T20:07:18.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.373+0000 7f28c2ffd640 1 -- 172.21.15.88:0/1731497451 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28bc0a5980 con 0x7f28c4143eb0 2023-10-16T20:07:18.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.373+0000 7f28c2ffd640 1 -- 172.21.15.88:0/1731497451 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc0a5d10 con 0x7f28c4143eb0 2023-10-16T20:07:18.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.377+0000 7f28c9e09640 1 -- 172.21.15.88:0/1731497451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28c4143eb0 msgr2=0x7f28c40b0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:18.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.377+0000 7f28c9e09640 1 --2- 172.21.15.88:0/1731497451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28c4143eb0 0x7f28c40b0500 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f28c40b10f0 tx=0x7f28bc01c850 comp rx=0 tx=0).stop 2023-10-16T20:07:18.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.377+0000 7f28c9e09640 1 -- 172.21.15.88:0/1731497451 shutdown_connections 2023-10-16T20:07:18.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.377+0000 7f28c9e09640 1 --2- 172.21.15.88:0/1731497451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28c4143eb0 0x7f28c40b0500 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:18.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.377+0000 7f28c9e09640 1 --2- 172.21.15.88:0/1731497451 >> v2:172.21.15.94:3300/0 conn(0x7f28c40ada40 0x7f28c40b1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:18.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.377+0000 7f28c9e09640 1 -- 172.21.15.88:0/1731497451 >> 172.21.15.88:0/1731497451 conn(0x7f28c40ce5e0 msgr2=0x7f28c40ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:18.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.377+0000 7f28c9e09640 1 -- 172.21.15.88:0/1731497451 shutdown_connections 2023-10-16T20:07:18.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.377+0000 7f28c9e09640 1 -- 172.21.15.88:0/1731497451 wait complete. 2023-10-16T20:07:18.384 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.377+0000 7f28c9e09640 1 Processor -- start 2023-10-16T20:07:18.384 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.377+0000 7f28c9e09640 1 -- start start 2023-10-16T20:07:18.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.377+0000 7f28c9e09640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28c40ada40 0x7f28c40ad190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:18.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.377+0000 7f28c9e09640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28c40ade20 0x7f28c40b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:18.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.377+0000 7f28c9e09640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f28c4143eb0 0x7f28c415a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:18.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.377+0000 7f28c9e09640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c40ac9a0 con 0x7f28c40ada40 2023-10-16T20:07:18.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.377+0000 7f28c9e09640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c40abd90 con 0x7f28c40ade20 2023-10-16T20:07:18.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.377+0000 7f28c9e09640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c40ac070 con 0x7f28c4143eb0 2023-10-16T20:07:18.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.377+0000 7f28c37fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28c40ada40 0x7f28c40ad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:18.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.377+0000 7f28c37fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28c40ada40 0x7f28c40ad190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34058/0 (socket says 172.21.15.88:34058) 2023-10-16T20:07:18.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.377+0000 7f28c37fe640 1 -- 172.21.15.88:0/3788717897 learned_addr learned my addr 172.21.15.88:0/3788717897 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:18.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.377+0000 7f28bbfff640 1 --2- 172.21.15.88:0/3788717897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28c40ade20 0x7f28c40b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:18.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.377+0000 7f28c3fff640 1 --2- 172.21.15.88:0/3788717897 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f28c4143eb0 0x7f28c415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:18.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.377+0000 7f28c37fe640 1 -- 172.21.15.88:0/3788717897 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f28c4143eb0 msgr2=0x7f28c415a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:18.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.377+0000 7f28c37fe640 1 --2- 172.21.15.88:0/3788717897 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f28c4143eb0 0x7f28c415a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:18.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.377+0000 7f28c37fe640 1 -- 172.21.15.88:0/3788717897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28c40ade20 msgr2=0x7f28c40b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:18.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.377+0000 7f28c37fe640 1 --2- 172.21.15.88:0/3788717897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28c40ade20 0x7f28c40b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:18.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.377+0000 7f28c37fe640 1 -- 172.21.15.88:0/3788717897 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28bc009a70 con 0x7f28c40ada40 2023-10-16T20:07:18.388 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.377+0000 7f28c3fff640 1 --2- 172.21.15.88:0/3788717897 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f28c4143eb0 0x7f28c415a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:07:18.388 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.377+0000 7f28bbfff640 1 --2- 172.21.15.88:0/3788717897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28c40ade20 0x7f28c40b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:07:18.388 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.377+0000 7f28c37fe640 1 --2- 172.21.15.88:0/3788717897 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28c40ada40 0x7f28c40ad190 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f28b400e870 tx=0x7f28b400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:18.388 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.381+0000 7f28c0ff9640 1 -- 172.21.15.88:0/3788717897 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b4009510 con 0x7f28c40ada40 2023-10-16T20:07:18.388 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.381+0000 7f28c0ff9640 1 -- 172.21.15.88:0/3788717897 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b400eec0 con 0x7f28c40ada40 2023-10-16T20:07:18.389 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.381+0000 7f28c0ff9640 1 -- 172.21.15.88:0/3788717897 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b400cb40 con 0x7f28c40ada40 2023-10-16T20:07:18.389 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.381+0000 7f28c9e09640 1 -- 172.21.15.88:0/3788717897 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c40ac350 con 0x7f28c40ada40 2023-10-16T20:07:18.389 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.381+0000 7f28c9e09640 1 -- 172.21.15.88:0/3788717897 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c40ad6d0 con 0x7f28c40ada40 2023-10-16T20:07:18.390 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.381+0000 7f28c9e09640 1 -- 172.21.15.88:0/3788717897 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c404e9c0 con 0x7f28c40ada40 2023-10-16T20:07:18.390 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.381+0000 7f28c0ff9640 1 -- 172.21.15.88:0/3788717897 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f28b400cca0 con 0x7f28c40ada40 2023-10-16T20:07:18.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.381+0000 7f28c0ff9640 1 --2- 172.21.15.88:0/3788717897 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f28a40664d0 0x7f28a4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:18.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.381+0000 7f28c0ff9640 1 -- 172.21.15.88:0/3788717897 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f28b4083dd0 con 0x7f28c40ada40 2023-10-16T20:07:18.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.381+0000 7f28bbfff640 1 --2- 172.21.15.88:0/3788717897 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f28a40664d0 0x7f28a4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:18.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.381+0000 7f28bbfff640 1 --2- 172.21.15.88:0/3788717897 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f28a40664d0 0x7f28a4068990 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f28c4142f60 tx=0x7f28ac009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:18.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.385+0000 7f28c0ff9640 1 -- 172.21.15.88:0/3788717897 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f28b4051380 con 0x7f28c40ada40 2023-10-16T20:07:18.540 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.537+0000 7f28c9e09640 1 -- 172.21.15.88:0/3788717897 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28c4071b60 con 0x7f28c40ada40 2023-10-16T20:07:18.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.537+0000 7f28c0ff9640 1 -- 172.21.15.88:0/3788717897 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+13119 (secure 0 0 0) 0x7f28b4021090 con 0x7f28c40ada40 2023-10-16T20:07:18.541 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:07:18.541 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":50,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:07:17.203896+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6835/2190281591":"2023-10-17T20:07:12.842861+0000","172.21.15.94:6833/2190281591":"2023-10-17T20:07:12.842861+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:07:18.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.537+0000 7f28c9e09640 1 -- 172.21.15.88:0/3788717897 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f28a40664d0 msgr2=0x7f28a4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:18.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.537+0000 7f28c9e09640 1 --2- 172.21.15.88:0/3788717897 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f28a40664d0 0x7f28a4068990 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f28c4142f60 tx=0x7f28ac009040 comp rx=0 tx=0).stop 2023-10-16T20:07:18.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.537+0000 7f28c9e09640 1 -- 172.21.15.88:0/3788717897 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28c40ada40 msgr2=0x7f28c40ad190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:18.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.537+0000 7f28c9e09640 1 --2- 172.21.15.88:0/3788717897 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28c40ada40 0x7f28c40ad190 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f28b400e870 tx=0x7f28b400ec70 comp rx=0 tx=0).stop 2023-10-16T20:07:18.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.537+0000 7f28c9e09640 1 -- 172.21.15.88:0/3788717897 shutdown_connections 2023-10-16T20:07:18.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.537+0000 7f28c9e09640 1 --2- 172.21.15.88:0/3788717897 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f28a40664d0 0x7f28a4068990 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:18.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.537+0000 7f28c9e09640 1 --2- 172.21.15.88:0/3788717897 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f28c4143eb0 0x7f28c415a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:18.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.537+0000 7f28c9e09640 1 --2- 172.21.15.88:0/3788717897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28c40ade20 0x7f28c40b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:18.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.537+0000 7f28c9e09640 1 --2- 172.21.15.88:0/3788717897 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28c40ada40 0x7f28c40ad190 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:18.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.537+0000 7f28c9e09640 1 -- 172.21.15.88:0/3788717897 >> 172.21.15.88:0/3788717897 conn(0x7f28c40ce5e0 msgr2=0x7f28c41430d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:18.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.537+0000 7f28c9e09640 1 -- 172.21.15.88:0/3788717897 shutdown_connections 2023-10-16T20:07:18.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.537+0000 7f28c9e09640 1 -- 172.21.15.88:0/3788717897 wait complete. 2023-10-16T20:07:18.562 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T20:07:18.563 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:07:18.703 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.697+0000 7fe95623a640 1 Processor -- start 2023-10-16T20:07:18.703 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.697+0000 7fe95623a640 1 -- start start 2023-10-16T20:07:18.705 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.701+0000 7fe95623a640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe95015a730 0x7fe950070ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:18.705 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.701+0000 7fe95623a640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe9500b1360 0x7fe9500caee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:18.705 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.701+0000 7fe95623a640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fe9500ca7e0 0x7fe9500cb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:18.705 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.701+0000 7fe95623a640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9500b0cb0 con 0x7fe9500ca7e0 2023-10-16T20:07:18.706 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.701+0000 7fe95623a640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9500b0030 con 0x7fe9500b1360 2023-10-16T20:07:18.706 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.701+0000 7fe95623a640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9500b0310 con 0x7fe95015a730 2023-10-16T20:07:18.706 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.701+0000 7fe955a39640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fe9500ca7e0 0x7fe9500cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:18.707 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.701+0000 7fe955a39640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fe9500ca7e0 0x7fe9500cb420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34068/0 (socket says 172.21.15.88:34068) 2023-10-16T20:07:18.707 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.701+0000 7fe955a39640 1 -- 172.21.15.88:0/4248888843 learned_addr learned my addr 172.21.15.88:0/4248888843 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:18.707 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.701+0000 7fe955238640 1 --2- 172.21.15.88:0/4248888843 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe95015a730 0x7fe950070ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:18.707 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.701+0000 7fe954a37640 1 --2- 172.21.15.88:0/4248888843 >> v2:172.21.15.94:3300/0 conn(0x7fe9500b1360 0x7fe9500caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:18.707 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.701+0000 7fe955a39640 1 -- 172.21.15.88:0/4248888843 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe95015a730 msgr2=0x7fe950070ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:18.708 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.701+0000 7fe955a39640 1 --2- 172.21.15.88:0/4248888843 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe95015a730 0x7fe950070ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:18.708 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.701+0000 7fe955a39640 1 -- 172.21.15.88:0/4248888843 >> v2:172.21.15.94:3300/0 conn(0x7fe9500b1360 msgr2=0x7fe9500caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:18.708 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.701+0000 7fe955a39640 1 --2- 172.21.15.88:0/4248888843 >> v2:172.21.15.94:3300/0 conn(0x7fe9500b1360 0x7fe9500caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:18.708 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.701+0000 7fe955a39640 1 -- 172.21.15.88:0/4248888843 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9500b05f0 con 0x7fe9500ca7e0 2023-10-16T20:07:18.709 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.705+0000 7fe955a39640 1 --2- 172.21.15.88:0/4248888843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9500ca7e0 0x7fe9500cb420 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fe9500b1070 tx=0x7fe94801c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa47d65a7b770184 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:18.710 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.705+0000 7fe947fff640 1 -- 172.21.15.88:0/4248888843 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9480a56c0 con 0x7fe9500ca7e0 2023-10-16T20:07:18.710 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.705+0000 7fe947fff640 1 -- 172.21.15.88:0/4248888843 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9480a5820 con 0x7fe9500ca7e0 2023-10-16T20:07:18.710 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.705+0000 7fe947fff640 1 -- 172.21.15.88:0/4248888843 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9480a5bb0 con 0x7fe9500ca7e0 2023-10-16T20:07:18.710 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.705+0000 7fe95623a640 1 -- 172.21.15.88:0/4248888843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9500ca7e0 msgr2=0x7fe9500cb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:18.710 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.705+0000 7fe95623a640 1 --2- 172.21.15.88:0/4248888843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9500ca7e0 0x7fe9500cb420 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fe9500b1070 tx=0x7fe94801c880 comp rx=0 tx=0).stop 2023-10-16T20:07:18.711 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.705+0000 7fe95623a640 1 -- 172.21.15.88:0/4248888843 shutdown_connections 2023-10-16T20:07:18.711 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.705+0000 7fe95623a640 1 --2- 172.21.15.88:0/4248888843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9500ca7e0 0x7fe9500cb420 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:18.711 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.705+0000 7fe95623a640 1 --2- 172.21.15.88:0/4248888843 >> v2:172.21.15.94:3300/0 conn(0x7fe9500b1360 0x7fe9500caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:18.711 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.705+0000 7fe95623a640 1 --2- 172.21.15.88:0/4248888843 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe95015a730 0x7fe950070ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:18.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.705+0000 7fe95623a640 1 -- 172.21.15.88:0/4248888843 >> 172.21.15.88:0/4248888843 conn(0x7fe95006a4c0 msgr2=0x7fe95006c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:18.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.705+0000 7fe95623a640 1 -- 172.21.15.88:0/4248888843 shutdown_connections 2023-10-16T20:07:18.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.705+0000 7fe95623a640 1 -- 172.21.15.88:0/4248888843 wait complete. 2023-10-16T20:07:18.714 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.705+0000 7fe95623a640 1 Processor -- start 2023-10-16T20:07:18.714 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.705+0000 7fe95623a640 1 -- start start 2023-10-16T20:07:18.714 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.705+0000 7fe95623a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9500b1360 0x7fe95015d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:18.714 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.705+0000 7fe95623a640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe9500ca7e0 0x7fe9501f88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:18.715 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.705+0000 7fe95623a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe95015a730 0x7fe9501591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:18.715 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.705+0000 7fe95623a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe950157940 con 0x7fe9500b1360 2023-10-16T20:07:18.715 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.705+0000 7fe95623a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe950157610 con 0x7fe95015a730 2023-10-16T20:07:18.715 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.705+0000 7fe95623a640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9501f9130 con 0x7fe9500ca7e0 2023-10-16T20:07:18.715 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.709+0000 7fe955238640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9500b1360 0x7fe95015d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:18.716 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.709+0000 7fe955238640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9500b1360 0x7fe95015d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34078/0 (socket says 172.21.15.88:34078) 2023-10-16T20:07:18.716 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.709+0000 7fe955238640 1 -- 172.21.15.88:0/928190107 learned_addr learned my addr 172.21.15.88:0/928190107 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:18.716 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.709+0000 7fe955a39640 1 --2- 172.21.15.88:0/928190107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe95015a730 0x7fe9501591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:18.716 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.709+0000 7fe954a37640 1 --2- 172.21.15.88:0/928190107 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe9500ca7e0 0x7fe9501f88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:18.717 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.709+0000 7fe955238640 1 -- 172.21.15.88:0/928190107 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe9500ca7e0 msgr2=0x7fe9501f88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:18.717 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.709+0000 7fe955238640 1 --2- 172.21.15.88:0/928190107 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe9500ca7e0 0x7fe9501f88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:18.717 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.709+0000 7fe955238640 1 -- 172.21.15.88:0/928190107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe95015a730 msgr2=0x7fe9501591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:18.717 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.709+0000 7fe955238640 1 --2- 172.21.15.88:0/928190107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe95015a730 0x7fe9501591a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:18.718 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.709+0000 7fe955238640 1 -- 172.21.15.88:0/928190107 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe948009b30 con 0x7fe9500b1360 2023-10-16T20:07:18.718 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.709+0000 7fe955a39640 1 --2- 172.21.15.88:0/928190107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe95015a730 0x7fe9501591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:07:18.719 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.709+0000 7fe955238640 1 --2- 172.21.15.88:0/928190107 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9500b1360 0x7fe95015d3a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe94c00d5f0 tx=0x7fe94c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:18.719 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.709+0000 7fe945ffb640 1 -- 172.21.15.88:0/928190107 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c0125f0 con 0x7fe9500b1360 2023-10-16T20:07:18.719 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.709+0000 7fe945ffb640 1 -- 172.21.15.88:0/928190107 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94c012750 con 0x7fe9500b1360 2023-10-16T20:07:18.720 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.709+0000 7fe95623a640 1 -- 172.21.15.88:0/928190107 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9501f9410 con 0x7fe9500b1360 2023-10-16T20:07:18.720 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.709+0000 7fe945ffb640 1 -- 172.21.15.88:0/928190107 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c012ae0 con 0x7fe9500b1360 2023-10-16T20:07:18.720 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.709+0000 7fe95623a640 1 -- 172.21.15.88:0/928190107 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9501f8e20 con 0x7fe9500b1360 2023-10-16T20:07:18.721 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.709+0000 7fe95623a640 1 -- 172.21.15.88:0/928190107 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9500674f0 con 0x7fe9500b1360 2023-10-16T20:07:18.723 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.709+0000 7fe945ffb640 1 -- 172.21.15.88:0/928190107 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe94c012c40 con 0x7fe9500b1360 2023-10-16T20:07:18.723 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.709+0000 7fe945ffb640 1 --2- 172.21.15.88:0/928190107 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe9340664a0 0x7fe934068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:18.723 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.713+0000 7fe945ffb640 1 -- 172.21.15.88:0/928190107 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fe94c0841c0 con 0x7fe9500b1360 2023-10-16T20:07:18.723 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.713+0000 7fe954a37640 1 --2- 172.21.15.88:0/928190107 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe9340664a0 0x7fe934068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:18.724 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.713+0000 7fe954a37640 1 --2- 172.21.15.88:0/928190107 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe9340664a0 0x7fe934068960 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe950071420 tx=0x7fe940008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:18.724 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.713+0000 7fe945ffb640 1 -- 172.21.15.88:0/928190107 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe94c051800 con 0x7fe9500b1360 2023-10-16T20:07:18.869 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.865+0000 7fe95623a640 1 -- 172.21.15.88:0/928190107 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe95006cee0 con 0x7fe9500b1360 2023-10-16T20:07:18.869 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.865+0000 7fe945ffb640 1 -- 172.21.15.88:0/928190107 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe94c007380 con 0x7fe9500b1360 2023-10-16T20:07:18.870 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:07:18.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.865+0000 7fe95623a640 1 -- 172.21.15.88:0/928190107 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe9340664a0 msgr2=0x7fe934068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:18.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.865+0000 7fe95623a640 1 --2- 172.21.15.88:0/928190107 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe9340664a0 0x7fe934068960 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe950071420 tx=0x7fe940008040 comp rx=0 tx=0).stop 2023-10-16T20:07:18.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.865+0000 7fe95623a640 1 -- 172.21.15.88:0/928190107 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9500b1360 msgr2=0x7fe95015d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:18.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.865+0000 7fe95623a640 1 --2- 172.21.15.88:0/928190107 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9500b1360 0x7fe95015d3a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe94c00d5f0 tx=0x7fe94c00dab0 comp rx=0 tx=0).stop 2023-10-16T20:07:18.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.865+0000 7fe95623a640 1 -- 172.21.15.88:0/928190107 shutdown_connections 2023-10-16T20:07:18.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.865+0000 7fe95623a640 1 --2- 172.21.15.88:0/928190107 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe9340664a0 0x7fe934068960 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:18.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.865+0000 7fe95623a640 1 --2- 172.21.15.88:0/928190107 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe9500ca7e0 0x7fe9501f88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:18.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.865+0000 7fe95623a640 1 --2- 172.21.15.88:0/928190107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe95015a730 0x7fe9501591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:18.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.865+0000 7fe95623a640 1 --2- 172.21.15.88:0/928190107 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9500b1360 0x7fe95015d3a0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:18.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.865+0000 7fe95623a640 1 -- 172.21.15.88:0/928190107 >> 172.21.15.88:0/928190107 conn(0x7fe95006a4c0 msgr2=0x7fe9500c9070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:18.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.865+0000 7fe95623a640 1 -- 172.21.15.88:0/928190107 shutdown_connections 2023-10-16T20:07:18.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:18.865+0000 7fe95623a640 1 -- 172.21.15.88:0/928190107 wait complete. 2023-10-16T20:07:18.890 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm' 2023-10-16T20:07:19.029 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.025+0000 7f61fde54640 1 Processor -- start 2023-10-16T20:07:19.029 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.025+0000 7f61fde54640 1 -- start start 2023-10-16T20:07:19.030 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.025+0000 7f61fde54640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f61f815cb80 0x7f61f80b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:19.030 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.025+0000 7f61fde54640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f61f80cebd0 0x7f61f806bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:19.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.025+0000 7f61fde54640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f80b2cf0 con 0x7f61f80b28b0 2023-10-16T20:07:19.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.025+0000 7f61fde54640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61f806b490 con 0x7f61f80cebd0 2023-10-16T20:07:19.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.025+0000 7f61fde54640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f80cf180 con 0x7f61f815cb80 2023-10-16T20:07:19.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.025+0000 7f61f7fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f61f80cebd0 0x7f61f806bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:19.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.025+0000 7f61f7fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f61f80cebd0 0x7f61f806bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34090/0 (socket says 172.21.15.88:34090) 2023-10-16T20:07:19.032 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.025+0000 7f61f7fff640 1 -- 172.21.15.88:0/3017240371 learned_addr learned my addr 172.21.15.88:0/3017240371 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:19.032 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.025+0000 7f61f77fe640 1 --2- 172.21.15.88:0/3017240371 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f61f815cb80 0x7f61f80b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:19.032 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.025+0000 7f61f7fff640 1 -- 172.21.15.88:0/3017240371 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f61f815cb80 msgr2=0x7f61f80b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:19.032 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.025+0000 7f61f7fff640 1 --2- 172.21.15.88:0/3017240371 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f61f815cb80 0x7f61f80b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:19.032 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.025+0000 7f61f7fff640 1 -- 172.21.15.88:0/3017240371 >> v1:172.21.15.94:6789/0 conn(0x7f61f80b28b0 legacy=0x7f61f80ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:19.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.025+0000 7f61f7fff640 1 -- 172.21.15.88:0/3017240371 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f806b700 con 0x7f61f80cebd0 2023-10-16T20:07:19.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.025+0000 7f61f77fe640 1 --2- 172.21.15.88:0/3017240371 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f61f815cb80 0x7f61f80b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:07:19.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.029+0000 7f61f7fff640 1 --2- 172.21.15.88:0/3017240371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61f80cebd0 0x7f61f806bf00 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f61f0009e50 tx=0x7f61f001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38db3b189e9d9dd9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:19.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.029+0000 7f61f6ffd640 1 -- 172.21.15.88:0/3017240371 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f00a5750 con 0x7f61f80cebd0 2023-10-16T20:07:19.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.029+0000 7f61f6ffd640 1 -- 172.21.15.88:0/3017240371 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f00a58b0 con 0x7f61f80cebd0 2023-10-16T20:07:19.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.029+0000 7f61f6ffd640 1 -- 172.21.15.88:0/3017240371 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f00a5c40 con 0x7f61f80cebd0 2023-10-16T20:07:19.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.029+0000 7f61fde54640 1 -- 172.21.15.88:0/3017240371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61f80cebd0 msgr2=0x7f61f806bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:19.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.029+0000 7f61fde54640 1 --2- 172.21.15.88:0/3017240371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61f80cebd0 0x7f61f806bf00 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f61f0009e50 tx=0x7f61f001c9a0 comp rx=0 tx=0).stop 2023-10-16T20:07:19.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.029+0000 7f61fde54640 1 -- 172.21.15.88:0/3017240371 shutdown_connections 2023-10-16T20:07:19.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.029+0000 7f61fde54640 1 --2- 172.21.15.88:0/3017240371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61f80cebd0 0x7f61f806bf00 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:19.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.029+0000 7f61fde54640 1 --2- 172.21.15.88:0/3017240371 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f61f815cb80 0x7f61f80b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:19.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.029+0000 7f61fde54640 1 -- 172.21.15.88:0/3017240371 >> 172.21.15.88:0/3017240371 conn(0x7f61f80cd370 msgr2=0x7f61f80cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:19.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.029+0000 7f61fde54640 1 -- 172.21.15.88:0/3017240371 shutdown_connections 2023-10-16T20:07:19.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.029+0000 7f61fde54640 1 -- 172.21.15.88:0/3017240371 wait complete. 2023-10-16T20:07:19.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.033+0000 7f61fde54640 1 Processor -- start 2023-10-16T20:07:19.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.033+0000 7f61fde54640 1 -- start start 2023-10-16T20:07:19.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.033+0000 7f61fde54640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61f80b28b0 0x7f61f815c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:19.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.033+0000 7f61fde54640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61f80cebd0 0x7f61f81521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:19.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.033+0000 7f61fde54640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f61f815cb80 0x7f61f815e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:19.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.033+0000 7f61fde54640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f815bc50 con 0x7f61f80cebd0 2023-10-16T20:07:19.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.033+0000 7f61fde54640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f815d7a0 con 0x7f61f80b28b0 2023-10-16T20:07:19.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.033+0000 7f61fde54640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f81526f0 con 0x7f61f815cb80 2023-10-16T20:07:19.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.033+0000 7f61effff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61f80cebd0 0x7f61f81521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:19.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.033+0000 7f61effff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61f80cebd0 0x7f61f81521b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34098/0 (socket says 172.21.15.88:34098) 2023-10-16T20:07:19.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.033+0000 7f61effff640 1 -- 172.21.15.88:0/3451859249 learned_addr learned my addr 172.21.15.88:0/3451859249 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:19.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.033+0000 7f61f7fff640 1 --2- 172.21.15.88:0/3451859249 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f61f815cb80 0x7f61f815e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:19.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.033+0000 7f61f77fe640 1 --2- 172.21.15.88:0/3451859249 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61f80b28b0 0x7f61f815c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:19.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.033+0000 7f61effff640 1 -- 172.21.15.88:0/3451859249 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f61f815cb80 msgr2=0x7f61f815e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:19.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.033+0000 7f61effff640 1 --2- 172.21.15.88:0/3451859249 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f61f815cb80 0x7f61f815e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:19.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.033+0000 7f61effff640 1 -- 172.21.15.88:0/3451859249 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61f80b28b0 msgr2=0x7f61f815c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:19.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.033+0000 7f61effff640 1 --2- 172.21.15.88:0/3451859249 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61f80b28b0 0x7f61f815c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:19.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.033+0000 7f61effff640 1 -- 172.21.15.88:0/3451859249 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f0009b30 con 0x7f61f80cebd0 2023-10-16T20:07:19.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.033+0000 7f61effff640 1 --2- 172.21.15.88:0/3451859249 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61f80cebd0 0x7f61f81521b0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f61e000c450 tx=0x7f61e000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:19.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.033+0000 7f61f4ff9640 1 -- 172.21.15.88:0/3451859249 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e0012830 con 0x7f61f80cebd0 2023-10-16T20:07:19.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.033+0000 7f61f4ff9640 1 -- 172.21.15.88:0/3451859249 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e00129b0 con 0x7f61f80cebd0 2023-10-16T20:07:19.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.033+0000 7f61fde54640 1 -- 172.21.15.88:0/3451859249 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f815afd0 con 0x7f61f80cebd0 2023-10-16T20:07:19.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.033+0000 7f61f4ff9640 1 -- 172.21.15.88:0/3451859249 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e0012d60 con 0x7f61f80cebd0 2023-10-16T20:07:19.048 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.033+0000 7f61fde54640 1 -- 172.21.15.88:0/3451859249 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f815b590 con 0x7f61f80cebd0 2023-10-16T20:07:19.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.033+0000 7f61fde54640 1 -- 172.21.15.88:0/3451859249 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f815bd90 con 0x7f61f80cebd0 2023-10-16T20:07:19.050 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.037+0000 7f61f4ff9640 1 -- 172.21.15.88:0/3451859249 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f61e001c460 con 0x7f61f80cebd0 2023-10-16T20:07:19.050 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.037+0000 7f61f4ff9640 1 --2- 172.21.15.88:0/3451859249 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f61d80664a0 0x7f61d8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:19.050 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.037+0000 7f61f4ff9640 1 -- 172.21.15.88:0/3451859249 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f61e00839e0 con 0x7f61f80cebd0 2023-10-16T20:07:19.050 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.037+0000 7f61f77fe640 1 --2- 172.21.15.88:0/3451859249 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f61d80664a0 0x7f61d8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:19.051 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.037+0000 7f61f77fe640 1 --2- 172.21.15.88:0/3451859249 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f61d80664a0 0x7f61d8068960 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f61e8006fd0 tx=0x7f61e8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:19.051 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.041+0000 7f61f4ff9640 1 -- 172.21.15.88:0/3451859249 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f61f815bd90 con 0x7f61f80cebd0 2023-10-16T20:07:19.248 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:19.245+0000 7f61fde54640 1 -- 172.21.15.88:0/3451859249 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]} v 0) v1 -- 0x7f61f815bd90 con 0x7f61f80cebd0 2023-10-16T20:07:20.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.217+0000 7f61f4ff9640 1 -- 172.21.15.88:0/3451859249 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]}]=0 v151) v1 ==== 137+0+0 (secure 0 0 0) 0x7f61f815bd90 con 0x7f61f80cebd0 2023-10-16T20:07:20.224 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.217+0000 7f61fde54640 1 -- 172.21.15.88:0/3451859249 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f61d80664a0 msgr2=0x7f61d8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:20.224 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.217+0000 7f61fde54640 1 --2- 172.21.15.88:0/3451859249 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f61d80664a0 0x7f61d8068960 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f61e8006fd0 tx=0x7f61e8008040 comp rx=0 tx=0).stop 2023-10-16T20:07:20.225 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.217+0000 7f61fde54640 1 -- 172.21.15.88:0/3451859249 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61f80cebd0 msgr2=0x7f61f81521b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:20.225 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.217+0000 7f61fde54640 1 --2- 172.21.15.88:0/3451859249 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61f80cebd0 0x7f61f81521b0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f61e000c450 tx=0x7f61e000c910 comp rx=0 tx=0).stop 2023-10-16T20:07:20.225 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.221+0000 7f61fde54640 1 -- 172.21.15.88:0/3451859249 shutdown_connections 2023-10-16T20:07:20.226 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.221+0000 7f61fde54640 1 --2- 172.21.15.88:0/3451859249 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f61d80664a0 0x7f61d8068960 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:20.226 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.221+0000 7f61fde54640 1 --2- 172.21.15.88:0/3451859249 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f61f815cb80 0x7f61f815e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:20.226 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.221+0000 7f61fde54640 1 --2- 172.21.15.88:0/3451859249 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61f80cebd0 0x7f61f81521b0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:20.226 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.221+0000 7f61fde54640 1 --2- 172.21.15.88:0/3451859249 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61f80b28b0 0x7f61f815c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:20.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.221+0000 7f61fde54640 1 -- 172.21.15.88:0/3451859249 >> 172.21.15.88:0/3451859249 conn(0x7f61f80cd370 msgr2=0x7f61f80cbe30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:20.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.221+0000 7f61fde54640 1 -- 172.21.15.88:0/3451859249 shutdown_connections 2023-10-16T20:07:20.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.221+0000 7f61fde54640 1 -- 172.21.15.88:0/3451859249 wait complete. 2023-10-16T20:07:20.248 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T20:07:20.393 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.389+0000 7fb3678a9640 1 Processor -- start 2023-10-16T20:07:20.394 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.389+0000 7fb3678a9640 1 -- start start 2023-10-16T20:07:20.396 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.389+0000 7fb3678a9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb36015cb70 0x7fb3600b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:20.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.389+0000 7fb3678a9640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb3600b4530 0x7fb3600b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:20.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.389+0000 7fb3678a9640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb3600d0bd0 0x7fb3600d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:20.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.389+0000 7fb3678a9640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3600b4ed0 con 0x7fb3600b4530 2023-10-16T20:07:20.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.389+0000 7fb3678a9640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3600b7d20 con 0x7fb3600d0bd0 2023-10-16T20:07:20.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.389+0000 7fb3678a9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3600b7110 con 0x7fb36015cb70 2023-10-16T20:07:20.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.389+0000 7fb364e44640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb3600b4530 0x7fb3600b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:20.399 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.389+0000 7fb364e44640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb3600b4530 0x7fb3600b4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34112/0 (socket says 172.21.15.88:34112) 2023-10-16T20:07:20.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.389+0000 7fb364e44640 1 -- 172.21.15.88:0/144747300 learned_addr learned my addr 172.21.15.88:0/144747300 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:20.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.389+0000 7fb365e46640 1 --2- 172.21.15.88:0/144747300 >> v2:172.21.15.94:3300/0 conn(0x7fb3600d0bd0 0x7fb3600d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:20.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.389+0000 7fb365645640 1 --2- 172.21.15.88:0/144747300 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb36015cb70 0x7fb3600b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:20.401 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.389+0000 7fb364e44640 1 -- 172.21.15.88:0/144747300 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb36015cb70 msgr2=0x7fb3600b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:20.401 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.389+0000 7fb364e44640 1 --2- 172.21.15.88:0/144747300 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb36015cb70 0x7fb3600b83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:20.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.389+0000 7fb364e44640 1 -- 172.21.15.88:0/144747300 >> v2:172.21.15.94:3300/0 conn(0x7fb3600d0bd0 msgr2=0x7fb3600d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:20.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.389+0000 7fb364e44640 1 --2- 172.21.15.88:0/144747300 >> v2:172.21.15.94:3300/0 conn(0x7fb3600d0bd0 0x7fb3600d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:20.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.389+0000 7fb364e44640 1 -- 172.21.15.88:0/144747300 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3600b73f0 con 0x7fb3600b4530 2023-10-16T20:07:20.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.393+0000 7fb364e44640 1 --2- 172.21.15.88:0/144747300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3600b4530 0x7fb3600b4990 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb3600c9b10 tx=0x7fb35001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510a73af463998f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:20.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.393+0000 7fb357fff640 1 -- 172.21.15.88:0/144747300 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3500a89b0 con 0x7fb3600b4530 2023-10-16T20:07:20.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.393+0000 7fb357fff640 1 -- 172.21.15.88:0/144747300 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3500a8b10 con 0x7fb3600b4530 2023-10-16T20:07:20.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.393+0000 7fb357fff640 1 -- 172.21.15.88:0/144747300 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3500a8ea0 con 0x7fb3600b4530 2023-10-16T20:07:20.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.393+0000 7fb3678a9640 1 -- 172.21.15.88:0/144747300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3600b4530 msgr2=0x7fb3600b4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:20.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.393+0000 7fb3678a9640 1 --2- 172.21.15.88:0/144747300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3600b4530 0x7fb3600b4990 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb3600c9b10 tx=0x7fb35001ecf0 comp rx=0 tx=0).stop 2023-10-16T20:07:20.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.393+0000 7fb3678a9640 1 -- 172.21.15.88:0/144747300 shutdown_connections 2023-10-16T20:07:20.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.393+0000 7fb3678a9640 1 --2- 172.21.15.88:0/144747300 >> v2:172.21.15.94:3300/0 conn(0x7fb3600d0bd0 0x7fb3600d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:20.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.393+0000 7fb3678a9640 1 --2- 172.21.15.88:0/144747300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3600b4530 0x7fb3600b4990 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:20.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.393+0000 7fb3678a9640 1 --2- 172.21.15.88:0/144747300 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb36015cb70 0x7fb3600b83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:20.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.393+0000 7fb3678a9640 1 -- 172.21.15.88:0/144747300 >> 172.21.15.88:0/144747300 conn(0x7fb36006b6e0 msgr2=0x7fb36006baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:20.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.393+0000 7fb3678a9640 1 -- 172.21.15.88:0/144747300 shutdown_connections 2023-10-16T20:07:20.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.393+0000 7fb3678a9640 1 -- 172.21.15.88:0/144747300 wait complete. 2023-10-16T20:07:20.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.397+0000 7fb3678a9640 1 Processor -- start 2023-10-16T20:07:20.414 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.397+0000 7fb3678a9640 1 -- start start 2023-10-16T20:07:20.414 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.397+0000 7fb3678a9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3600b4530 0x7fb3600c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:20.414 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.397+0000 7fb3678a9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3600d0bd0 0x7fb3600d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:20.414 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.397+0000 7fb3678a9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb36015cb70 0x7fb3600d2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:20.415 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.397+0000 7fb3678a9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3600c9320 con 0x7fb3600d0bd0 2023-10-16T20:07:20.415 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.397+0000 7fb3678a9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36015e550 con 0x7fb3600b4530 2023-10-16T20:07:20.415 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.397+0000 7fb3678a9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb36015e830 con 0x7fb36015cb70 2023-10-16T20:07:20.416 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.397+0000 7fb364e44640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3600d0bd0 0x7fb3600d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:20.416 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.397+0000 7fb364e44640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3600d0bd0 0x7fb3600d26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34116/0 (socket says 172.21.15.88:34116) 2023-10-16T20:07:20.417 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.397+0000 7fb364e44640 1 -- 172.21.15.88:0/3644940994 learned_addr learned my addr 172.21.15.88:0/3644940994 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:20.417 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.397+0000 7fb364e44640 1 -- 172.21.15.88:0/3644940994 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb36015cb70 msgr2=0x7fb3600d2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:20.417 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.397+0000 7fb364e44640 1 --2- 172.21.15.88:0/3644940994 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb36015cb70 0x7fb3600d2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:20.418 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.397+0000 7fb365645640 1 --2- 172.21.15.88:0/3644940994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3600b4530 0x7fb3600c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:20.418 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.397+0000 7fb364e44640 1 -- 172.21.15.88:0/3644940994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3600b4530 msgr2=0x7fb3600c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:20.418 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.397+0000 7fb364e44640 1 --2- 172.21.15.88:0/3644940994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3600b4530 0x7fb3600c7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:20.418 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.397+0000 7fb364e44640 1 -- 172.21.15.88:0/3644940994 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3500094d0 con 0x7fb3600d0bd0 2023-10-16T20:07:20.419 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.397+0000 7fb365645640 1 --2- 172.21.15.88:0/3644940994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3600b4530 0x7fb3600c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:07:20.419 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.397+0000 7fb364e44640 1 --2- 172.21.15.88:0/3644940994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3600d0bd0 0x7fb3600d26a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fb35001d3a0 tx=0x7fb35001ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:20.419 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.397+0000 7fb355ffb640 1 -- 172.21.15.88:0/3644940994 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3500ba600 con 0x7fb3600d0bd0 2023-10-16T20:07:20.419 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.397+0000 7fb355ffb640 1 -- 172.21.15.88:0/3644940994 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3500ba760 con 0x7fb3600d0bd0 2023-10-16T20:07:20.420 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.397+0000 7fb355ffb640 1 -- 172.21.15.88:0/3644940994 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3500bab10 con 0x7fb3600d0bd0 2023-10-16T20:07:20.420 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.397+0000 7fb3678a9640 1 -- 172.21.15.88:0/3644940994 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb36015eb10 con 0x7fb3600d0bd0 2023-10-16T20:07:20.420 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.397+0000 7fb3678a9640 1 -- 172.21.15.88:0/3644940994 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3600c8260 con 0x7fb3600d0bd0 2023-10-16T20:07:20.420 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.401+0000 7fb3678a9640 1 -- 172.21.15.88:0/3644940994 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb36004e9c0 con 0x7fb3600d0bd0 2023-10-16T20:07:20.420 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.401+0000 7fb355ffb640 1 -- 172.21.15.88:0/3644940994 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb3500bac70 con 0x7fb3600d0bd0 2023-10-16T20:07:20.421 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.401+0000 7fb355ffb640 1 --2- 172.21.15.88:0/3644940994 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb3440664a0 0x7fb344068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:20.421 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.401+0000 7fb355ffb640 1 -- 172.21.15.88:0/3644940994 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fb350122360 con 0x7fb3600d0bd0 2023-10-16T20:07:20.421 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.401+0000 7fb365645640 1 --2- 172.21.15.88:0/3644940994 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb3440664a0 0x7fb344068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:20.421 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.401+0000 7fb365645640 1 --2- 172.21.15.88:0/3644940994 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb3440664a0 0x7fb344068960 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb35c0097a0 tx=0x7fb35c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:20.422 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.405+0000 7fb355ffb640 1 -- 172.21.15.88:0/3644940994 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb3500ef900 con 0x7fb3600d0bd0 2023-10-16T20:07:20.560 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.557+0000 7fb3678a9640 1 -- 172.21.15.88:0/3644940994 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb36015efc0 con 0x7fb3600d0bd0 2023-10-16T20:07:20.564 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.561+0000 7fb355ffb640 1 -- 172.21.15.88:0/3644940994 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb3500f2f20 con 0x7fb3600d0bd0 2023-10-16T20:07:20.567 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.561+0000 7fb3678a9640 1 -- 172.21.15.88:0/3644940994 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb3440664a0 msgr2=0x7fb344068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:20.567 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.561+0000 7fb3678a9640 1 --2- 172.21.15.88:0/3644940994 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb3440664a0 0x7fb344068960 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb35c0097a0 tx=0x7fb35c008040 comp rx=0 tx=0).stop 2023-10-16T20:07:20.567 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.561+0000 7fb3678a9640 1 -- 172.21.15.88:0/3644940994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3600d0bd0 msgr2=0x7fb3600d26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:20.567 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.561+0000 7fb3678a9640 1 --2- 172.21.15.88:0/3644940994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3600d0bd0 0x7fb3600d26a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fb35001d3a0 tx=0x7fb35001ba90 comp rx=0 tx=0).stop 2023-10-16T20:07:20.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.561+0000 7fb3678a9640 1 -- 172.21.15.88:0/3644940994 shutdown_connections 2023-10-16T20:07:20.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.561+0000 7fb3678a9640 1 --2- 172.21.15.88:0/3644940994 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb3440664a0 0x7fb344068960 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:20.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.561+0000 7fb3678a9640 1 --2- 172.21.15.88:0/3644940994 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb36015cb70 0x7fb3600d2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:20.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.561+0000 7fb3678a9640 1 --2- 172.21.15.88:0/3644940994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3600d0bd0 0x7fb3600d26a0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:20.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.561+0000 7fb3678a9640 1 --2- 172.21.15.88:0/3644940994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3600b4530 0x7fb3600c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:20.569 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.561+0000 7fb3678a9640 1 -- 172.21.15.88:0/3644940994 >> 172.21.15.88:0/3644940994 conn(0x7fb36006b6e0 msgr2=0x7fb36015b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:20.569 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.561+0000 7fb3678a9640 1 -- 172.21.15.88:0/3644940994 shutdown_connections 2023-10-16T20:07:20.569 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:20.561+0000 7fb3678a9640 1 -- 172.21.15.88:0/3644940994 wait complete. 2023-10-16T20:07:20.585 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T20:07:20.586 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T20:07:20.678 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T20:07:20.758 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T20:07:20.759 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:07:20.760 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:07:20.769 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354668k,nr_inodes=4088667,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9372 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T20:07:20.770 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:07:20.770 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:07:20.779 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354624k,nr_inodes=4088656,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5605 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T20:07:20.780 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:07:20.780 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:07:20.819 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354668k,nr_inodes=4088667,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9372 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T20:07:20.820 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:07:20.820 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:07:20.831 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354624k,nr_inodes=4088656,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5605 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T20:07:20.832 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T20:07:20.832 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T20:07:21.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.001+0000 7fcb4953e640 1 Processor -- start 2023-10-16T20:07:21.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.001+0000 7fcb4953e640 1 -- start start 2023-10-16T20:07:21.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.001+0000 7fcb4953e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcb440cef00 0x7fcb440b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:21.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.001+0000 7fcb4953e640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb4406b5f0 con 0x7fcb4406ba80 2023-10-16T20:07:21.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.001+0000 7fcb4953e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb440ce240 con 0x7fcb4415cb40 2023-10-16T20:07:21.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.001+0000 7fcb4953e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb440ce800 con 0x7fcb440cef00 2023-10-16T20:07:21.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.001+0000 7fcb437fe640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fcb4406ba80 0x7fcb4406beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:39570/0 (socket says 172.21.15.88:39570) 2023-10-16T20:07:21.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.001+0000 7fcb437fe640 1 -- 172.21.15.88:0/2918335085 learned_addr learned my addr 172.21.15.88:0/2918335085 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:21.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.001+0000 7fcb427fc640 1 --2- 172.21.15.88:0/2918335085 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcb440cef00 0x7fcb440b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:21.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.001+0000 7fcb41ffb640 1 -- 172.21.15.88:0/2918335085 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1082884403 0 0) 0x7fcb4406b5f0 con 0x7fcb4406ba80 2023-10-16T20:07:21.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.001+0000 7fcb41ffb640 1 -- 172.21.15.88:0/2918335085 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb2c003530 con 0x7fcb4406ba80 2023-10-16T20:07:21.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.001+0000 7fcb41ffb640 1 -- 172.21.15.88:0/2918335085 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2301830761 0 0) 0x7fcb440ce240 con 0x7fcb4415cb40 2023-10-16T20:07:21.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.001+0000 7fcb41ffb640 1 -- 172.21.15.88:0/2918335085 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb4406b5f0 con 0x7fcb4415cb40 2023-10-16T20:07:21.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.001+0000 7fcb427fc640 1 -- 172.21.15.88:0/2918335085 >> v1:172.21.15.94:6789/0 conn(0x7fcb4415cb40 legacy=0x7fcb440b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:21.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.001+0000 7fcb427fc640 1 -- 172.21.15.88:0/2918335085 >> v1:172.21.15.88:6789/0 conn(0x7fcb4406ba80 legacy=0x7fcb4406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:21.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.001+0000 7fcb427fc640 1 -- 172.21.15.88:0/2918335085 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb440ce520 con 0x7fcb440cef00 2023-10-16T20:07:21.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.005+0000 7fcb427fc640 1 --2- 172.21.15.88:0/2918335085 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcb440cef00 0x7fcb440b4a60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcb380097f0 tx=0x7fcb3801c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8380ffd906385229 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:21.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.005+0000 7fcb41ffb640 1 -- 172.21.15.88:0/2918335085 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb380ab600 con 0x7fcb440cef00 2023-10-16T20:07:21.010 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.005+0000 7fcb41ffb640 1 -- 172.21.15.88:0/2918335085 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb380a7050 con 0x7fcb440cef00 2023-10-16T20:07:21.010 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.005+0000 7fcb41ffb640 1 -- 172.21.15.88:0/2918335085 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb380ab980 con 0x7fcb440cef00 2023-10-16T20:07:21.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.005+0000 7fcb4953e640 1 -- 172.21.15.88:0/2918335085 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcb440cef00 msgr2=0x7fcb440b4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:21.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.005+0000 7fcb4953e640 1 --2- 172.21.15.88:0/2918335085 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcb440cef00 0x7fcb440b4a60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcb380097f0 tx=0x7fcb3801c370 comp rx=0 tx=0).stop 2023-10-16T20:07:21.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.005+0000 7fcb4953e640 1 -- 172.21.15.88:0/2918335085 shutdown_connections 2023-10-16T20:07:21.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.005+0000 7fcb4953e640 1 --2- 172.21.15.88:0/2918335085 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcb440cef00 0x7fcb440b4a60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:21.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.005+0000 7fcb4953e640 1 -- 172.21.15.88:0/2918335085 >> 172.21.15.88:0/2918335085 conn(0x7fcb440d28d0 msgr2=0x7fcb440d2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:21.012 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.005+0000 7fcb4953e640 1 -- 172.21.15.88:0/2918335085 shutdown_connections 2023-10-16T20:07:21.012 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.005+0000 7fcb4953e640 1 -- 172.21.15.88:0/2918335085 wait complete. 2023-10-16T20:07:21.014 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.009+0000 7fcb4953e640 1 Processor -- start 2023-10-16T20:07:21.014 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.009+0000 7fcb4953e640 1 -- start start 2023-10-16T20:07:21.014 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.009+0000 7fcb4953e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb4406ba80 0x7fcb44156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:21.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.009+0000 7fcb4953e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcb440cef00 0x7fcb440cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:21.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.009+0000 7fcb4953e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb4415cb40 0x7fcb440b69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:21.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.009+0000 7fcb4953e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb44157470 con 0x7fcb4406ba80 2023-10-16T20:07:21.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.009+0000 7fcb4953e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb440b7530 con 0x7fcb4415cb40 2023-10-16T20:07:21.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.009+0000 7fcb4953e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb440cab30 con 0x7fcb440cef00 2023-10-16T20:07:21.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.009+0000 7fcb42ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb4406ba80 0x7fcb44156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:21.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.009+0000 7fcb42ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb4406ba80 0x7fcb44156f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34128/0 (socket says 172.21.15.88:34128) 2023-10-16T20:07:21.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.009+0000 7fcb42ffd640 1 -- 172.21.15.88:0/4096708852 learned_addr learned my addr 172.21.15.88:0/4096708852 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:21.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.009+0000 7fcb42ffd640 1 -- 172.21.15.88:0/4096708852 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcb440cef00 msgr2=0x7fcb440cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:21.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.009+0000 7fcb437fe640 1 --2- 172.21.15.88:0/4096708852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb4415cb40 0x7fcb440b69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:21.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.009+0000 7fcb427fc640 1 --2- 172.21.15.88:0/4096708852 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcb440cef00 0x7fcb440cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:21.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.009+0000 7fcb42ffd640 1 --2- 172.21.15.88:0/4096708852 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcb440cef00 0x7fcb440cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:21.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.009+0000 7fcb42ffd640 1 -- 172.21.15.88:0/4096708852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb4415cb40 msgr2=0x7fcb440b69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:21.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.009+0000 7fcb42ffd640 1 --2- 172.21.15.88:0/4096708852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb4415cb40 0x7fcb440b69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:21.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.009+0000 7fcb42ffd640 1 -- 172.21.15.88:0/4096708852 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb38009420 con 0x7fcb4406ba80 2023-10-16T20:07:21.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.009+0000 7fcb437fe640 1 --2- 172.21.15.88:0/4096708852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb4415cb40 0x7fcb440b69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:07:21.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.009+0000 7fcb427fc640 1 --2- 172.21.15.88:0/4096708852 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcb440cef00 0x7fcb440cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:07:21.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.009+0000 7fcb42ffd640 1 --2- 172.21.15.88:0/4096708852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb4406ba80 0x7fcb44156f30 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fcb3400c420 tx=0x7fcb3400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:21.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.009+0000 7fcb2b7fe640 1 -- 172.21.15.88:0/4096708852 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb340052d0 con 0x7fcb4406ba80 2023-10-16T20:07:21.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.009+0000 7fcb2b7fe640 1 -- 172.21.15.88:0/4096708852 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb34005430 con 0x7fcb4406ba80 2023-10-16T20:07:21.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.009+0000 7fcb2b7fe640 1 -- 172.21.15.88:0/4096708852 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb34003870 con 0x7fcb4406ba80 2023-10-16T20:07:21.019 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.009+0000 7fcb4953e640 1 -- 172.21.15.88:0/4096708852 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb440cae10 con 0x7fcb4406ba80 2023-10-16T20:07:21.019 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.009+0000 7fcb4953e640 1 -- 172.21.15.88:0/4096708852 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb440cc370 con 0x7fcb4406ba80 2023-10-16T20:07:21.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.009+0000 7fcb4953e640 1 -- 172.21.15.88:0/4096708852 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb440674f0 con 0x7fcb4406ba80 2023-10-16T20:07:21.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.013+0000 7fcb2b7fe640 1 -- 172.21.15.88:0/4096708852 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcb340039d0 con 0x7fcb4406ba80 2023-10-16T20:07:21.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.013+0000 7fcb2b7fe640 1 --2- 172.21.15.88:0/4096708852 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcb300664a0 0x7fcb30068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:21.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.013+0000 7fcb2b7fe640 1 -- 172.21.15.88:0/4096708852 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fcb34083230 con 0x7fcb4406ba80 2023-10-16T20:07:21.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.013+0000 7fcb427fc640 1 --2- 172.21.15.88:0/4096708852 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcb300664a0 0x7fcb30068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:21.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.013+0000 7fcb427fc640 1 --2- 172.21.15.88:0/4096708852 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcb300664a0 0x7fcb30068960 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcb380ad000 tx=0x7fcb38008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:21.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.017+0000 7fcb2b7fe640 1 -- 172.21.15.88:0/4096708852 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcb34050870 con 0x7fcb4406ba80 2023-10-16T20:07:21.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.197+0000 7fcb4953e640 1 -- 172.21.15.88:0/4096708852 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb441584c0 con 0x7fcb4406ba80 2023-10-16T20:07:21.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.197+0000 7fcb2b7fe640 1 -- 172.21.15.88:0/4096708852 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+3696 (secure 0 0 0) 0x7fcb34017e80 con 0x7fcb4406ba80 2023-10-16T20:07:21.202 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:07:21.202 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":21,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/696292235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":696292235},{"type":"v1","addr":"172.21.15.94:6834","nonce":696292235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5480,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2473209430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":2473209430},{"type":"v1","addr":"172.21.15.94:6835","nonce":2473209430}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/2152672724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2152672724},{"type":"v1","addr":"172.21.15.88:6835","nonce":2152672724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3107974207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":3107974207},{"type":"v1","addr":"172.21.15.88:6837","nonce":3107974207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2023-10-16T20:07:21.202 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 21 2023-10-16T20:07:21.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.197+0000 7fcb4953e640 1 -- 172.21.15.88:0/4096708852 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcb300664a0 msgr2=0x7fcb30068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:21.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.197+0000 7fcb4953e640 1 --2- 172.21.15.88:0/4096708852 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcb300664a0 0x7fcb30068960 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcb380ad000 tx=0x7fcb38008040 comp rx=0 tx=0).stop 2023-10-16T20:07:21.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.197+0000 7fcb4953e640 1 -- 172.21.15.88:0/4096708852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb4406ba80 msgr2=0x7fcb44156f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:21.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.197+0000 7fcb4953e640 1 --2- 172.21.15.88:0/4096708852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb4406ba80 0x7fcb44156f30 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fcb3400c420 tx=0x7fcb3400c8e0 comp rx=0 tx=0).stop 2023-10-16T20:07:21.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.197+0000 7fcb4953e640 1 -- 172.21.15.88:0/4096708852 shutdown_connections 2023-10-16T20:07:21.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.197+0000 7fcb4953e640 1 --2- 172.21.15.88:0/4096708852 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcb300664a0 0x7fcb30068960 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:21.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.197+0000 7fcb4953e640 1 --2- 172.21.15.88:0/4096708852 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcb440cef00 0x7fcb440cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:21.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.197+0000 7fcb4953e640 1 --2- 172.21.15.88:0/4096708852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb4415cb40 0x7fcb440b69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:21.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.197+0000 7fcb4953e640 1 --2- 172.21.15.88:0/4096708852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb4406ba80 0x7fcb44156f30 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:21.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.197+0000 7fcb4953e640 1 -- 172.21.15.88:0/4096708852 >> 172.21.15.88:0/4096708852 conn(0x7fcb440d28d0 msgr2=0x7fcb440b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:21.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.201+0000 7fcb4953e640 1 -- 172.21.15.88:0/4096708852 shutdown_connections 2023-10-16T20:07:21.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.201+0000 7fcb4953e640 1 -- 172.21.15.88:0/4096708852 wait complete. 2023-10-16T20:07:21.224 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T20:07:21.224 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T20:07:21.224 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T20:07:21.225 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:07:21.225 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T20:07:21.225 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T20:07:21.225 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T20:07:21.226 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:07:21.226 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T20:07:21.226 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T20:07:21.227 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T20:07:21.227 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:07:21.227 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T20:07:21.227 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T20:07:21.228 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T20:07:21.228 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:07:21.238 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T20:07:21.238 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T20:07:21.239 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T20:07:21.240 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T20:07:21.242 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T20:07:21.243 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T20:07:21.243 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T20:07:21.244 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T20:07:21.286 INFO:tasks.ceph.mds.b:Started 2023-10-16T20:07:21.287 DEBUG:teuthology.parallel:result is None 2023-10-16T20:07:21.287 INFO:tasks.ceph.mds.a:Started 2023-10-16T20:07:21.288 DEBUG:teuthology.parallel:result is None 2023-10-16T20:07:21.289 INFO:tasks.ceph.mds.d:Started 2023-10-16T20:07:21.289 INFO:tasks.ceph.mds.c:Started 2023-10-16T20:07:21.289 DEBUG:teuthology.parallel:result is None 2023-10-16T20:07:21.290 DEBUG:teuthology.parallel:result is None 2023-10-16T20:07:21.290 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T20:07:21.373 INFO:tasks.ceph.mds.a.smithi088.stdout:starting mds.a at 2023-10-16T20:07:21.374 INFO:tasks.ceph.mds.c.smithi088.stdout:starting mds.c at 2023-10-16T20:07:21.379 INFO:tasks.ceph.mds.b.smithi094.stdout:starting mds.b at 2023-10-16T20:07:21.381 INFO:tasks.ceph.mds.d.smithi094.stdout:starting mds.d at 2023-10-16T20:07:21.436 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.433+0000 7f2b5ce91640 1 Processor -- start 2023-10-16T20:07:21.437 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.433+0000 7f2b5ce91640 1 -- start start 2023-10-16T20:07:21.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.433+0000 7f2b5ce91640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2b5815cb80 0x7f2b580b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:21.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.433+0000 7f2b5ce91640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b580b2cf0 con 0x7f2b580cebd0 2023-10-16T20:07:21.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.433+0000 7f2b5ce91640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b5806b490 con 0x7f2b580b28b0 2023-10-16T20:07:21.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.433+0000 7f2b5ce91640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b580cf180 con 0x7f2b5815cb80 2023-10-16T20:07:21.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.433+0000 7f2b56d9d640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f2b580cebd0 0x7f2b5806bf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:39588/0 (socket says 172.21.15.88:39588) 2023-10-16T20:07:21.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.433+0000 7f2b56d9d640 1 -- 172.21.15.88:0/4082571781 learned_addr learned my addr 172.21.15.88:0/4082571781 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:21.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.433+0000 7f2b5659c640 1 --2- 172.21.15.88:0/4082571781 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2b5815cb80 0x7f2b580b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:21.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.433+0000 7f2b55d9b640 1 -- 172.21.15.88:0/4082571781 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1311771123 0 0) 0x7f2b580b2cf0 con 0x7f2b580cebd0 2023-10-16T20:07:21.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.433+0000 7f2b55d9b640 1 -- 172.21.15.88:0/4082571781 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b44003530 con 0x7f2b580cebd0 2023-10-16T20:07:21.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.433+0000 7f2b55d9b640 1 -- 172.21.15.88:0/4082571781 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2181642546 0 0) 0x7f2b5806b490 con 0x7f2b580b28b0 2023-10-16T20:07:21.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.433+0000 7f2b55d9b640 1 -- 172.21.15.88:0/4082571781 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b580b2cf0 con 0x7f2b580b28b0 2023-10-16T20:07:21.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.433+0000 7f2b5659c640 1 -- 172.21.15.88:0/4082571781 >> v1:172.21.15.94:6789/0 conn(0x7f2b580b28b0 legacy=0x7f2b580ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:21.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.433+0000 7f2b5659c640 1 -- 172.21.15.88:0/4082571781 >> v1:172.21.15.88:6789/0 conn(0x7f2b580cebd0 legacy=0x7f2b5806bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:21.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.433+0000 7f2b5659c640 1 -- 172.21.15.88:0/4082571781 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5806b700 con 0x7f2b5815cb80 2023-10-16T20:07:21.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.437+0000 7f2b5659c640 1 --2- 172.21.15.88:0/4082571781 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2b5815cb80 0x7f2b580b2370 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2b480097f0 tx=0x7f2b4801c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6246d102a5a74d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:21.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.437+0000 7f2b55d9b640 1 -- 172.21.15.88:0/4082571781 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b480ac620 con 0x7f2b5815cb80 2023-10-16T20:07:21.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.437+0000 7f2b55d9b640 1 -- 172.21.15.88:0/4082571781 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b480ac780 con 0x7f2b5815cb80 2023-10-16T20:07:21.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.437+0000 7f2b55d9b640 1 -- 172.21.15.88:0/4082571781 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b480acb10 con 0x7f2b5815cb80 2023-10-16T20:07:21.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.437+0000 7f2b5ce91640 1 -- 172.21.15.88:0/4082571781 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2b5815cb80 msgr2=0x7f2b580b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:21.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.437+0000 7f2b5ce91640 1 --2- 172.21.15.88:0/4082571781 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2b5815cb80 0x7f2b580b2370 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2b480097f0 tx=0x7f2b4801c490 comp rx=0 tx=0).stop 2023-10-16T20:07:21.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.437+0000 7f2b5ce91640 1 -- 172.21.15.88:0/4082571781 shutdown_connections 2023-10-16T20:07:21.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.437+0000 7f2b5ce91640 1 --2- 172.21.15.88:0/4082571781 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2b5815cb80 0x7f2b580b2370 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:21.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.437+0000 7f2b5ce91640 1 -- 172.21.15.88:0/4082571781 >> 172.21.15.88:0/4082571781 conn(0x7f2b580cd370 msgr2=0x7f2b580cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:21.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.437+0000 7f2b5ce91640 1 -- 172.21.15.88:0/4082571781 shutdown_connections 2023-10-16T20:07:21.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.437+0000 7f2b5ce91640 1 -- 172.21.15.88:0/4082571781 wait complete. 2023-10-16T20:07:21.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.441+0000 7f2b5ce91640 1 Processor -- start 2023-10-16T20:07:21.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.441+0000 7f2b5ce91640 1 -- start start 2023-10-16T20:07:21.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.441+0000 7f2b5ce91640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b580b28b0 0x7f2b580c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:21.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.441+0000 7f2b5ce91640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b580cebd0 0x7f2b581540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:21.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.441+0000 7f2b5ce91640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2b5815cb80 0x7f2b580cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:21.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.441+0000 7f2b5ce91640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b58154610 con 0x7f2b580b28b0 2023-10-16T20:07:21.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.441+0000 7f2b5ce91640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b581f6b60 con 0x7f2b580cebd0 2023-10-16T20:07:21.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.441+0000 7f2b5ce91640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b580c6e50 con 0x7f2b5815cb80 2023-10-16T20:07:21.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.441+0000 7f2b5659c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b580b28b0 0x7f2b580c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:21.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.441+0000 7f2b5659c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b580b28b0 0x7f2b580c6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34150/0 (socket says 172.21.15.88:34150) 2023-10-16T20:07:21.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.441+0000 7f2b5659c640 1 -- 172.21.15.88:0/1931858611 learned_addr learned my addr 172.21.15.88:0/1931858611 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:21.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.441+0000 7f2b5659c640 1 -- 172.21.15.88:0/1931858611 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2b5815cb80 msgr2=0x7f2b580cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:21.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.441+0000 7f2b56d9d640 1 --2- 172.21.15.88:0/1931858611 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2b5815cb80 0x7f2b580cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:21.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.441+0000 7f2b5659c640 1 --2- 172.21.15.88:0/1931858611 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2b5815cb80 0x7f2b580cbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:21.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.441+0000 7f2b5659c640 1 -- 172.21.15.88:0/1931858611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b580cebd0 msgr2=0x7f2b581540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:21.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.441+0000 7f2b5659c640 1 --2- 172.21.15.88:0/1931858611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b580cebd0 0x7f2b581540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:21.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.441+0000 7f2b5659c640 1 -- 172.21.15.88:0/1931858611 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b48009420 con 0x7f2b580b28b0 2023-10-16T20:07:21.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.441+0000 7f2b5659c640 1 --2- 172.21.15.88:0/1931858611 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b580b28b0 0x7f2b580c6870 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2b480097f0 tx=0x7f2b48019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:21.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.441+0000 7f2b4f7fe640 1 -- 172.21.15.88:0/1931858611 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b480b7480 con 0x7f2b580b28b0 2023-10-16T20:07:21.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.441+0000 7f2b4f7fe640 1 -- 172.21.15.88:0/1931858611 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b480b7600 con 0x7f2b580b28b0 2023-10-16T20:07:21.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.441+0000 7f2b4f7fe640 1 -- 172.21.15.88:0/1931858611 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b480b79b0 con 0x7f2b580b28b0 2023-10-16T20:07:21.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.441+0000 7f2b5ce91640 1 -- 172.21.15.88:0/1931858611 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b580c7130 con 0x7f2b580b28b0 2023-10-16T20:07:21.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.441+0000 7f2b5ce91640 1 -- 172.21.15.88:0/1931858611 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b581f70e0 con 0x7f2b580b28b0 2023-10-16T20:07:21.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.445+0000 7f2b5ce91640 1 -- 172.21.15.88:0/1931858611 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b5804e9c0 con 0x7f2b580b28b0 2023-10-16T20:07:21.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.445+0000 7f2b4f7fe640 1 -- 172.21.15.88:0/1931858611 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2b480b3040 con 0x7f2b580b28b0 2023-10-16T20:07:21.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.445+0000 7f2b4f7fe640 1 --2- 172.21.15.88:0/1931858611 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2b380664d0 0x7f2b38068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:21.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.445+0000 7f2b4f7fe640 1 -- 172.21.15.88:0/1931858611 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f2b4811e580 con 0x7f2b580b28b0 2023-10-16T20:07:21.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.445+0000 7f2b4dd9b640 1 --2- 172.21.15.88:0/1931858611 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2b380664d0 0x7f2b38068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:21.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.445+0000 7f2b4dd9b640 1 --2- 172.21.15.88:0/1931858611 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2b380664d0 0x7f2b38068990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2b40007fd0 tx=0x7f2b40007960 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:21.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.449+0000 7f2b4f7fe640 1 -- 172.21.15.88:0/1931858611 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2b480ebbb0 con 0x7f2b580b28b0 2023-10-16T20:07:21.614 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:21.609+0000 7f2b5ce91640 1 -- 172.21.15.88:0/1931858611 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f2b580b6870 con 0x7f2b580b28b0 2023-10-16T20:07:22.238 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.233+0000 7f2b4f7fe640 1 -- 172.21.15.88:0/1931858611 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v52) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2b480b1080 con 0x7f2b580b28b0 2023-10-16T20:07:22.239 INFO:teuthology.orchestra.run.smithi088.stderr: removed all blocklist entries 2023-10-16T20:07:22.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.237+0000 7f2b5ce91640 1 -- 172.21.15.88:0/1931858611 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2b380664d0 msgr2=0x7f2b38068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:22.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.237+0000 7f2b5ce91640 1 --2- 172.21.15.88:0/1931858611 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2b380664d0 0x7f2b38068990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2b40007fd0 tx=0x7f2b40007960 comp rx=0 tx=0).stop 2023-10-16T20:07:22.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.237+0000 7f2b5ce91640 1 -- 172.21.15.88:0/1931858611 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b580b28b0 msgr2=0x7f2b580c6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:22.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.237+0000 7f2b5ce91640 1 --2- 172.21.15.88:0/1931858611 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b580b28b0 0x7f2b580c6870 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2b480097f0 tx=0x7f2b48019230 comp rx=0 tx=0).stop 2023-10-16T20:07:22.242 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.237+0000 7f2b5ce91640 1 -- 172.21.15.88:0/1931858611 shutdown_connections 2023-10-16T20:07:22.242 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.237+0000 7f2b5ce91640 1 --2- 172.21.15.88:0/1931858611 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2b380664d0 0x7f2b38068990 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:22.242 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.237+0000 7f2b5ce91640 1 --2- 172.21.15.88:0/1931858611 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2b5815cb80 0x7f2b580cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:22.242 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.237+0000 7f2b5ce91640 1 --2- 172.21.15.88:0/1931858611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b580cebd0 0x7f2b581540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:22.242 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.237+0000 7f2b5ce91640 1 --2- 172.21.15.88:0/1931858611 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b580b28b0 0x7f2b580c6870 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:22.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.237+0000 7f2b5ce91640 1 -- 172.21.15.88:0/1931858611 >> 172.21.15.88:0/1931858611 conn(0x7f2b580cd370 msgr2=0x7f2b58160bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:22.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.237+0000 7f2b5ce91640 1 -- 172.21.15.88:0/1931858611 shutdown_connections 2023-10-16T20:07:22.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.237+0000 7f2b5ce91640 1 -- 172.21.15.88:0/1931858611 wait complete. 2023-10-16T20:07:22.265 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T20:07:22.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.397+0000 7f298fa9e640 1 Processor -- start 2023-10-16T20:07:22.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.397+0000 7f298fa9e640 1 -- start start 2023-10-16T20:07:22.404 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.401+0000 7f298fa9e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f29880b0c10 0x7f29880b1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:22.404 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.401+0000 7f298fa9e640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29880b2570 con 0x7f29880ade20 2023-10-16T20:07:22.404 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.401+0000 7f298fa9e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29880cca50 con 0x7f29880ada40 2023-10-16T20:07:22.405 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.401+0000 7f298fa9e640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29880b1fb0 con 0x7f29880b0c10 2023-10-16T20:07:22.405 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.401+0000 7f298d039640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f29880ade20 0x7f29880b0500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:46878/0 (socket says 172.21.15.88:46878) 2023-10-16T20:07:22.405 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.401+0000 7f298d039640 1 -- 172.21.15.88:0/1412672268 learned_addr learned my addr 172.21.15.88:0/1412672268 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:22.405 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.401+0000 7f298e03b640 1 --2- 172.21.15.88:0/1412672268 >> v2:172.21.15.94:3300/0 conn(0x7f29880b0c10 0x7f29880b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:22.405 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.401+0000 7f298c838640 1 -- 172.21.15.88:0/1412672268 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 580889773 0 0) 0x7f29880b2570 con 0x7f29880ade20 2023-10-16T20:07:22.405 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.401+0000 7f298c838640 1 -- 172.21.15.88:0/1412672268 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2974003530 con 0x7f29880ade20 2023-10-16T20:07:22.406 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.401+0000 7f298c838640 1 -- 172.21.15.88:0/1412672268 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2862203129 0 0) 0x7f29880cca50 con 0x7f29880ada40 2023-10-16T20:07:22.406 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.401+0000 7f298c838640 1 -- 172.21.15.88:0/1412672268 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29880b2570 con 0x7f29880ada40 2023-10-16T20:07:22.406 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.401+0000 7f298e03b640 1 -- 172.21.15.88:0/1412672268 >> v1:172.21.15.94:6789/0 conn(0x7f29880ada40 legacy=0x7f298815adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:22.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.401+0000 7f298c838640 1 -- 172.21.15.88:0/1412672268 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 21599370 0 0) 0x7f2974003530 con 0x7f29880ade20 2023-10-16T20:07:22.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.401+0000 7f298e03b640 1 -- 172.21.15.88:0/1412672268 >> v1:172.21.15.88:6789/0 conn(0x7f29880ade20 legacy=0x7f29880b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:22.407 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.401+0000 7f298e03b640 1 -- 172.21.15.88:0/1412672268 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29880b2290 con 0x7f29880b0c10 2023-10-16T20:07:22.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.405+0000 7f298e03b640 1 --2- 172.21.15.88:0/1412672268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29880b0c10 0x7f29880b1920 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2980009dc0 tx=0x7f298001c790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4ddd2b34af48fb2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:22.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.405+0000 7f298c838640 1 -- 172.21.15.88:0/1412672268 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2978001680 con 0x7f29880b0c10 2023-10-16T20:07:22.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.405+0000 7f298c838640 1 -- 172.21.15.88:0/1412672268 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29800a5630 con 0x7f29880b0c10 2023-10-16T20:07:22.408 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.405+0000 7f298c838640 1 -- 172.21.15.88:0/1412672268 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29800a59c0 con 0x7f29880b0c10 2023-10-16T20:07:22.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.405+0000 7f298fa9e640 1 -- 172.21.15.88:0/1412672268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29880b0c10 msgr2=0x7f29880b1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:22.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.405+0000 7f298fa9e640 1 --2- 172.21.15.88:0/1412672268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29880b0c10 0x7f29880b1920 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2980009dc0 tx=0x7f298001c790 comp rx=0 tx=0).stop 2023-10-16T20:07:22.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.405+0000 7f298fa9e640 1 -- 172.21.15.88:0/1412672268 shutdown_connections 2023-10-16T20:07:22.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.405+0000 7f298fa9e640 1 --2- 172.21.15.88:0/1412672268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29880b0c10 0x7f29880b1920 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:22.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.405+0000 7f298fa9e640 1 -- 172.21.15.88:0/1412672268 >> 172.21.15.88:0/1412672268 conn(0x7f29880ce5e0 msgr2=0x7f29880ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:22.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.405+0000 7f298fa9e640 1 -- 172.21.15.88:0/1412672268 shutdown_connections 2023-10-16T20:07:22.410 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.405+0000 7f298fa9e640 1 -- 172.21.15.88:0/1412672268 wait complete. 2023-10-16T20:07:22.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.405+0000 7f298fa9e640 1 Processor -- start 2023-10-16T20:07:22.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.405+0000 7f298fa9e640 1 -- start start 2023-10-16T20:07:22.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.405+0000 7f298fa9e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29880ada40 0x7f2988133770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:22.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.405+0000 7f298fa9e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29880ade20 0x7f29880ca800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:22.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.405+0000 7f298fa9e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29880b0c10 0x7f298813db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:22.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.405+0000 7f298fa9e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29880af010 con 0x7f29880b0c10 2023-10-16T20:07:22.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.405+0000 7f298fa9e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29880cad40 con 0x7f29880ade20 2023-10-16T20:07:22.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.405+0000 7f298fa9e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29880ae390 con 0x7f29880ada40 2023-10-16T20:07:22.414 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.409+0000 7f298e03b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29880b0c10 0x7f298813db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:22.414 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.409+0000 7f298e03b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29880b0c10 0x7f298813db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38408/0 (socket says 172.21.15.88:38408) 2023-10-16T20:07:22.414 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.409+0000 7f298d039640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29880ade20 0x7f29880ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:22.414 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.409+0000 7f298e03b640 1 -- 172.21.15.88:0/284331608 learned_addr learned my addr 172.21.15.88:0/284331608 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:22.414 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.409+0000 7f298d83a640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29880ada40 0x7f2988133770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:22.415 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.409+0000 7f298e03b640 1 -- 172.21.15.88:0/284331608 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29880ada40 msgr2=0x7f2988133770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:22.415 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.409+0000 7f298e03b640 1 --2- 172.21.15.88:0/284331608 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29880ada40 0x7f2988133770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:22.415 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.409+0000 7f298e03b640 1 -- 172.21.15.88:0/284331608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29880ade20 msgr2=0x7f29880ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:22.415 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.409+0000 7f298e03b640 1 --2- 172.21.15.88:0/284331608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29880ade20 0x7f29880ca800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:22.415 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.409+0000 7f298e03b640 1 -- 172.21.15.88:0/284331608 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2980009a70 con 0x7f29880b0c10 2023-10-16T20:07:22.416 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.409+0000 7f298e03b640 1 --2- 172.21.15.88:0/284331608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29880b0c10 0x7f298813db50 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f298001cc70 tx=0x7f2980019530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:22.417 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.409+0000 7f297e7fc640 1 -- 172.21.15.88:0/284331608 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29800ad640 con 0x7f29880b0c10 2023-10-16T20:07:22.417 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.409+0000 7f297e7fc640 1 -- 172.21.15.88:0/284331608 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29800ad7a0 con 0x7f29880b0c10 2023-10-16T20:07:22.417 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.409+0000 7f297e7fc640 1 -- 172.21.15.88:0/284331608 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29800adb10 con 0x7f29880b0c10 2023-10-16T20:07:22.417 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.409+0000 7f298fa9e640 1 -- 172.21.15.88:0/284331608 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29880ae670 con 0x7f29880b0c10 2023-10-16T20:07:22.418 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.409+0000 7f298fa9e640 1 -- 172.21.15.88:0/284331608 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29880aec30 con 0x7f29880b0c10 2023-10-16T20:07:22.418 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.409+0000 7f298fa9e640 1 -- 172.21.15.88:0/284331608 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29880af150 con 0x7f29880b0c10 2023-10-16T20:07:22.418 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.409+0000 7f297e7fc640 1 -- 172.21.15.88:0/284331608 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f29800b4050 con 0x7f29880b0c10 2023-10-16T20:07:22.418 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.413+0000 7f297e7fc640 1 --2- 172.21.15.88:0/284331608 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f296c0664a0 0x7f296c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:22.418 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.413+0000 7f297e7fc640 1 -- 172.21.15.88:0/284331608 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f298011e940 con 0x7f29880b0c10 2023-10-16T20:07:22.419 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.413+0000 7f298d83a640 1 --2- 172.21.15.88:0/284331608 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f296c0664a0 0x7f296c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:22.419 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.413+0000 7f298d83a640 1 --2- 172.21.15.88:0/284331608 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f296c0664a0 0x7f296c068960 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f29840011f0 tx=0x7f2984009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:22.419 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.413+0000 7f297e7fc640 1 -- 172.21.15.88:0/284331608 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f29880af150 con 0x7f29880b0c10 2023-10-16T20:07:22.622 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.617+0000 7f298fa9e640 1 -- 172.21.15.88:0/284331608 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f29880af150 con 0x7f29880b0c10 2023-10-16T20:07:22.623 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.617+0000 7f297e7fc640 1 -- 172.21.15.88:0/284331608 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5303 (secure 0 0 0) 0x7f29880af150 con 0x7f29880b0c10 2023-10-16T20:07:22.623 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:07:22.623 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2023-10-16T20:07:22.623 INFO:teuthology.orchestra.run.smithi088.stdout: "auth_dump": [ 2023-10-16T20:07:22.624 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:07:22.624 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.a", 2023-10-16T20:07:22.624 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQAuly1lCHa1JhAAhNsz0rC3KpOhywCHwmjWyg==", 2023-10-16T20:07:22.624 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:07:22.624 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2023-10-16T20:07:22.625 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2023-10-16T20:07:22.625 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2023-10-16T20:07:22.625 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2023-10-16T20:07:22.625 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:07:22.625 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:07:22.625 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:07:22.626 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.b", 2023-10-16T20:07:22.626 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQAuly1lxz58MhAA69WwY8VLqgyX4KQ9pGS5QQ==", 2023-10-16T20:07:22.626 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:07:22.626 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2023-10-16T20:07:22.627 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2023-10-16T20:07:22.627 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2023-10-16T20:07:22.627 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2023-10-16T20:07:22.627 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:07:22.627 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:07:22.627 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:07:22.628 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.c", 2023-10-16T20:07:22.628 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQAuly1ltUzfLRAAJWxUKQWdDYVtMTHHYipcZQ==", 2023-10-16T20:07:22.628 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:07:22.628 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2023-10-16T20:07:22.628 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2023-10-16T20:07:22.629 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2023-10-16T20:07:22.629 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2023-10-16T20:07:22.629 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:07:22.629 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:07:22.629 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:07:22.630 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.d", 2023-10-16T20:07:22.630 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQAuly1l+TXWORAAJv6W5uoM9XLv1keyH1iJGg==", 2023-10-16T20:07:22.630 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:07:22.630 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2023-10-16T20:07:22.630 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2023-10-16T20:07:22.631 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2023-10-16T20:07:22.631 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2023-10-16T20:07:22.631 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:07:22.631 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:07:22.631 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:07:22.632 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.0", 2023-10-16T20:07:22.632 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQAyly1l1tzTGhAANOg63+nIoOxhLwHYO2iqQA==", 2023-10-16T20:07:22.632 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:07:22.632 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2023-10-16T20:07:22.632 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2023-10-16T20:07:22.632 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2023-10-16T20:07:22.633 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:07:22.633 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:07:22.633 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:07:22.633 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.1", 2023-10-16T20:07:22.633 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQA1ly1lgigrIRAA0Mzb88vvzSzGqEzZVsRUtQ==", 2023-10-16T20:07:22.634 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:07:22.634 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2023-10-16T20:07:22.634 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2023-10-16T20:07:22.634 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2023-10-16T20:07:22.634 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:07:22.635 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:07:22.635 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:07:22.635 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.2", 2023-10-16T20:07:22.635 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQA4ly1lN0lgKBAAALodK7N7sDf5GdZrccritg==", 2023-10-16T20:07:22.635 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:07:22.635 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2023-10-16T20:07:22.636 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2023-10-16T20:07:22.636 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2023-10-16T20:07:22.636 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:07:22.636 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:07:22.636 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:07:22.637 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.3", 2023-10-16T20:07:22.637 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQA7ly1lrVjALhAAOc9pmNFdvOZ1gT6VMXOgzA==", 2023-10-16T20:07:22.637 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:07:22.637 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2023-10-16T20:07:22.637 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2023-10-16T20:07:22.637 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2023-10-16T20:07:22.638 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:07:22.638 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:07:22.638 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:07:22.638 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.4", 2023-10-16T20:07:22.639 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQBAly1lVlqzABAAIg5Q4X0rytK7dTFXzhulLw==", 2023-10-16T20:07:22.639 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:07:22.639 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2023-10-16T20:07:22.639 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2023-10-16T20:07:22.639 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2023-10-16T20:07:22.640 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:07:22.640 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:07:22.640 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:07:22.640 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.5", 2023-10-16T20:07:22.640 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQBDly1l9gS2BxAA+VSkv60PdAkqsolXnZSLUg==", 2023-10-16T20:07:22.640 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:07:22.641 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2023-10-16T20:07:22.641 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2023-10-16T20:07:22.641 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2023-10-16T20:07:22.641 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:07:22.641 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:07:22.642 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:07:22.642 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.6", 2023-10-16T20:07:22.642 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQBGly1lADNDDhAA/1Wahdkwdt708rxp8XnTRg==", 2023-10-16T20:07:22.642 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:07:22.642 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2023-10-16T20:07:22.643 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2023-10-16T20:07:22.643 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2023-10-16T20:07:22.643 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:07:22.643 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:07:22.643 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:07:22.643 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.7", 2023-10-16T20:07:22.644 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQBJly1lwLN1FBAABUh5iYOJQHAcceNeGuxQKA==", 2023-10-16T20:07:22.644 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:07:22.644 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2023-10-16T20:07:22.644 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2023-10-16T20:07:22.644 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2023-10-16T20:07:22.644 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:07:22.645 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:07:22.645 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:07:22.645 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.0", 2023-10-16T20:07:22.645 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQAvly1looeRAhAAeLshHOVIhQqoTtlPehorIg==", 2023-10-16T20:07:22.645 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:07:22.645 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2023-10-16T20:07:22.646 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow r", 2023-10-16T20:07:22.646 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rw pool=testcephfs1_data" 2023-10-16T20:07:22.646 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:07:22.646 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:07:22.646 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:07:22.647 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.1", 2023-10-16T20:07:22.647 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQAvly1lvy8qDBAAc789x2xSuuM7SHbL5XbRwg==", 2023-10-16T20:07:22.647 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:07:22.647 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2023-10-16T20:07:22.647 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2023-10-16T20:07:22.647 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2023-10-16T20:07:22.648 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2023-10-16T20:07:22.648 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:07:22.648 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:07:22.648 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:07:22.648 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.2", 2023-10-16T20:07:22.649 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQAvly1lhdfcCBAAJU/W9X3cX1ukGJOXsW3U7A==", 2023-10-16T20:07:22.649 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:07:22.649 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2023-10-16T20:07:22.649 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2023-10-16T20:07:22.649 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2023-10-16T20:07:22.649 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2023-10-16T20:07:22.650 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:07:22.650 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:07:22.650 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:07:22.650 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.3", 2023-10-16T20:07:22.650 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQAvly1lqPsnEhAAzgEkt/iWPI1jIPCKw0rCUg==", 2023-10-16T20:07:22.651 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:07:22.651 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2023-10-16T20:07:22.651 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2023-10-16T20:07:22.651 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2023-10-16T20:07:22.651 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2023-10-16T20:07:22.652 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:07:22.652 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:07:22.652 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:07:22.652 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.admin", 2023-10-16T20:07:22.652 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQAuly1l56PKCRAAdwHp3ILkhpj9GWIccpDmSg==", 2023-10-16T20:07:22.652 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:07:22.653 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2023-10-16T20:07:22.653 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2023-10-16T20:07:22.653 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2023-10-16T20:07:22.653 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2023-10-16T20:07:22.653 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:07:22.653 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:07:22.654 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:07:22.654 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mgr.x", 2023-10-16T20:07:22.654 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQAuly1l9hztIRAAGJWqLd1Aa6sZAzEHc+mXZQ==", 2023-10-16T20:07:22.654 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:07:22.654 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2023-10-16T20:07:22.655 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile mgr", 2023-10-16T20:07:22.655 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2023-10-16T20:07:22.655 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:07:22.655 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:07:22.655 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:07:22.656 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mgr.y", 2023-10-16T20:07:22.656 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQAuly1lDn1VHhAApRTbdVW1sInFHR4ftXIgLA==", 2023-10-16T20:07:22.656 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2023-10-16T20:07:22.656 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2023-10-16T20:07:22.656 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile mgr", 2023-10-16T20:07:22.656 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2023-10-16T20:07:22.657 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:07:22.657 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:07:22.657 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2023-10-16T20:07:22.657 INFO:teuthology.orchestra.run.smithi088.stdout:} 2023-10-16T20:07:22.659 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.621+0000 7f298fa9e640 1 -- 172.21.15.88:0/284331608 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f296c0664a0 msgr2=0x7f296c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:22.659 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.621+0000 7f298fa9e640 1 --2- 172.21.15.88:0/284331608 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f296c0664a0 0x7f296c068960 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f29840011f0 tx=0x7f2984009040 comp rx=0 tx=0).stop 2023-10-16T20:07:22.659 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.621+0000 7f298fa9e640 1 -- 172.21.15.88:0/284331608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29880b0c10 msgr2=0x7f298813db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:22.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.621+0000 7f298fa9e640 1 --2- 172.21.15.88:0/284331608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29880b0c10 0x7f298813db50 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f298001cc70 tx=0x7f2980019530 comp rx=0 tx=0).stop 2023-10-16T20:07:22.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.621+0000 7f298fa9e640 1 -- 172.21.15.88:0/284331608 shutdown_connections 2023-10-16T20:07:22.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.621+0000 7f298fa9e640 1 --2- 172.21.15.88:0/284331608 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f296c0664a0 0x7f296c068960 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:22.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.621+0000 7f298fa9e640 1 --2- 172.21.15.88:0/284331608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29880b0c10 0x7f298813db50 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:22.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.621+0000 7f298fa9e640 1 --2- 172.21.15.88:0/284331608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29880ade20 0x7f29880ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:22.661 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.621+0000 7f298fa9e640 1 --2- 172.21.15.88:0/284331608 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f29880ada40 0x7f2988133770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:22.661 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.621+0000 7f298fa9e640 1 -- 172.21.15.88:0/284331608 >> 172.21.15.88:0/284331608 conn(0x7f29880ce5e0 msgr2=0x7f2988069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:22.661 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.621+0000 7f298fa9e640 1 -- 172.21.15.88:0/284331608 shutdown_connections 2023-10-16T20:07:22.661 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.621+0000 7f298fa9e640 1 -- 172.21.15.88:0/284331608 wait complete. 2023-10-16T20:07:22.662 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:07:22.799 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.793+0000 7f3a81691640 1 Processor -- start 2023-10-16T20:07:22.800 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.793+0000 7f3a81691640 1 -- start start 2023-10-16T20:07:22.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.797+0000 7f3a81691640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3a7c0d26c0 0x7f3a7c161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:22.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.797+0000 7f3a81691640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a7c0b87e0 con 0x7f3a7c15d9f0 2023-10-16T20:07:22.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.797+0000 7f3a81691640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a7c15df10 con 0x7f3a7c0d22e0 2023-10-16T20:07:22.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.797+0000 7f3a81691640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a7c15e1f0 con 0x7f3a7c0d26c0 2023-10-16T20:07:22.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.797+0000 7f3a73fff640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f3a7c15d9f0 0x7f3a7c0cf2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:46892/0 (socket says 172.21.15.88:46892) 2023-10-16T20:07:22.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.797+0000 7f3a73fff640 1 -- 172.21.15.88:0/567388835 learned_addr learned my addr 172.21.15.88:0/567388835 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:22.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.797+0000 7f3a7b7fe640 1 --2- 172.21.15.88:0/567388835 >> v2:172.21.15.94:3300/0 conn(0x7f3a7c0d26c0 0x7f3a7c161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:22.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.797+0000 7f3a7a7fc640 1 -- 172.21.15.88:0/567388835 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1288205205 0 0) 0x7f3a7c0b87e0 con 0x7f3a7c15d9f0 2023-10-16T20:07:22.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.797+0000 7f3a7a7fc640 1 -- 172.21.15.88:0/567388835 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a68003530 con 0x7f3a7c15d9f0 2023-10-16T20:07:22.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.797+0000 7f3a7a7fc640 1 -- 172.21.15.88:0/567388835 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2029539913 0 0) 0x7f3a68003530 con 0x7f3a7c15d9f0 2023-10-16T20:07:22.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.797+0000 7f3a7a7fc640 1 -- 172.21.15.88:0/567388835 >> v2:172.21.15.94:3300/0 conn(0x7f3a7c0d26c0 msgr2=0x7f3a7c161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:22.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.797+0000 7f3a7a7fc640 1 --2- 172.21.15.88:0/567388835 >> v2:172.21.15.94:3300/0 conn(0x7f3a7c0d26c0 0x7f3a7c161450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:22.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.797+0000 7f3a7a7fc640 1 -- 172.21.15.88:0/567388835 >> v1:172.21.15.94:6789/0 conn(0x7f3a7c0d22e0 legacy=0x7f3a7c15d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:22.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.797+0000 7f3a7a7fc640 1 -- 172.21.15.88:0/567388835 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a7c160970 con 0x7f3a7c15d9f0 2023-10-16T20:07:22.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.797+0000 7f3a7a7fc640 1 -- 172.21.15.88:0/567388835 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f3a64001680 con 0x7f3a7c15d9f0 2023-10-16T20:07:22.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.797+0000 7f3a7a7fc640 1 -- 172.21.15.88:0/567388835 >> v1:172.21.15.88:6789/0 conn(0x7f3a7c15d9f0 legacy=0x7f3a7c0cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:22.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.797+0000 7f3a7a7fc640 1 --2- 172.21.15.88:0/567388835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a68003980 0x7f3a68003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:22.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.797+0000 7f3a7a7fc640 1 --2- 172.21.15.88:0/567388835 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a68004380 0x7f3a68004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:22.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.797+0000 7f3a7a7fc640 1 --2- 172.21.15.88:0/567388835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a68005050 0x7f3a6800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:22.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.797+0000 7f3a7a7fc640 1 -- 172.21.15.88:0/567388835 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c15e1f0 con 0x7f3a68003980 2023-10-16T20:07:22.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.797+0000 7f3a7a7fc640 1 -- 172.21.15.88:0/567388835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6800bfb0 con 0x7f3a68005050 2023-10-16T20:07:22.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.797+0000 7f3a7a7fc640 1 -- 172.21.15.88:0/567388835 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6800c170 con 0x7f3a68004380 2023-10-16T20:07:22.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.797+0000 7f3a7affd640 1 --2- 172.21.15.88:0/567388835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a68003980 0x7f3a68003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:22.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.797+0000 7f3a7b7fe640 1 --2- 172.21.15.88:0/567388835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a68005050 0x7f3a6800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:22.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.797+0000 7f3a73fff640 1 --2- 172.21.15.88:0/567388835 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a68004380 0x7f3a68004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:22.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.797+0000 7f3a7affd640 1 -- 172.21.15.88:0/567388835 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a68004380 msgr2=0x7f3a68004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:22.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.797+0000 7f3a7affd640 1 --2- 172.21.15.88:0/567388835 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a68004380 0x7f3a68004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:22.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.797+0000 7f3a7affd640 1 -- 172.21.15.88:0/567388835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a68005050 msgr2=0x7f3a6800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:22.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.797+0000 7f3a7affd640 1 --2- 172.21.15.88:0/567388835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a68005050 0x7f3a6800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:22.808 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.797+0000 7f3a7affd640 1 -- 172.21.15.88:0/567388835 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a68003530 con 0x7f3a68003980 2023-10-16T20:07:22.808 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.797+0000 7f3a7b7fe640 1 -- 172.21.15.88:0/567388835 reap_dead start 2023-10-16T20:07:22.808 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.797+0000 7f3a73fff640 1 --2- 172.21.15.88:0/567388835 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a68004380 0x7f3a68004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:07:22.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.801+0000 7f3a7affd640 1 --2- 172.21.15.88:0/567388835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a68003980 0x7f3a68003d60 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c001860 tx=0x7f3a6c01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c156e8b75ed2ff server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:22.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.801+0000 7f3a7a7fc640 1 -- 172.21.15.88:0/567388835 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c0ab600 con 0x7f3a68003980 2023-10-16T20:07:22.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.801+0000 7f3a7a7fc640 1 -- 172.21.15.88:0/567388835 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a6c0a7050 con 0x7f3a68003980 2023-10-16T20:07:22.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.801+0000 7f3a7a7fc640 1 -- 172.21.15.88:0/567388835 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c0ab9e0 con 0x7f3a68003980 2023-10-16T20:07:22.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.801+0000 7f3a81691640 1 -- 172.21.15.88:0/567388835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a68003980 msgr2=0x7f3a68003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:22.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.801+0000 7f3a81691640 1 --2- 172.21.15.88:0/567388835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a68003980 0x7f3a68003d60 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c001860 tx=0x7f3a6c01c4b0 comp rx=0 tx=0).stop 2023-10-16T20:07:22.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.801+0000 7f3a81691640 1 -- 172.21.15.88:0/567388835 shutdown_connections 2023-10-16T20:07:22.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.801+0000 7f3a81691640 1 --2- 172.21.15.88:0/567388835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a68003980 0x7f3a68003d60 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:22.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.801+0000 7f3a81691640 1 -- 172.21.15.88:0/567388835 >> 172.21.15.88:0/567388835 conn(0x7f3a7c0715c0 msgr2=0x7f3a7c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:22.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.801+0000 7f3a81691640 1 -- 172.21.15.88:0/567388835 shutdown_connections 2023-10-16T20:07:22.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.801+0000 7f3a81691640 1 -- 172.21.15.88:0/567388835 wait complete. 2023-10-16T20:07:22.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.805+0000 7f3a81691640 1 Processor -- start 2023-10-16T20:07:22.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.805+0000 7f3a81691640 1 -- start start 2023-10-16T20:07:22.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.805+0000 7f3a81691640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a68003980 0x7f3a7c14d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:22.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.805+0000 7f3a81691640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a7c0cbd80 0x7f3a7c14e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:22.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.805+0000 7f3a81691640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a7c0cc1e0 0x7f3a7c0c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:22.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.805+0000 7f3a81691640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c14d580 con 0x7f3a7c0cc1e0 2023-10-16T20:07:22.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.805+0000 7f3a81691640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c0c9c70 con 0x7f3a68003980 2023-10-16T20:07:22.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.805+0000 7f3a81691640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c0c9f50 con 0x7f3a7c0cbd80 2023-10-16T20:07:22.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.805+0000 7f3a7b7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a7c0cc1e0 0x7f3a7c0c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:22.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.805+0000 7f3a7b7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a7c0cc1e0 0x7f3a7c0c81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38422/0 (socket says 172.21.15.88:38422) 2023-10-16T20:07:22.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.805+0000 7f3a7b7fe640 1 -- 172.21.15.88:0/2032394886 learned_addr learned my addr 172.21.15.88:0/2032394886 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:22.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.805+0000 7f3a73fff640 1 --2- 172.21.15.88:0/2032394886 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a7c0cbd80 0x7f3a7c14e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:22.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.805+0000 7f3a7affd640 1 --2- 172.21.15.88:0/2032394886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a68003980 0x7f3a7c14d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:22.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.805+0000 7f3a7b7fe640 1 -- 172.21.15.88:0/2032394886 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a7c0cbd80 msgr2=0x7f3a7c14e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:22.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.805+0000 7f3a7b7fe640 1 --2- 172.21.15.88:0/2032394886 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a7c0cbd80 0x7f3a7c14e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:22.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.805+0000 7f3a7b7fe640 1 -- 172.21.15.88:0/2032394886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a68003980 msgr2=0x7f3a7c14d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:22.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.805+0000 7f3a7b7fe640 1 --2- 172.21.15.88:0/2032394886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a68003980 0x7f3a7c14d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:22.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.805+0000 7f3a7b7fe640 1 -- 172.21.15.88:0/2032394886 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a7c0ca230 con 0x7f3a7c0cc1e0 2023-10-16T20:07:22.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.805+0000 7f3a7affd640 1 --2- 172.21.15.88:0/2032394886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a68003980 0x7f3a7c14d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:07:22.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.805+0000 7f3a7b7fe640 1 --2- 172.21.15.88:0/2032394886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a7c0cc1e0 0x7f3a7c0c81d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c161990 tx=0x7f3a7400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:22.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.805+0000 7f3a737fe640 1 -- 172.21.15.88:0/2032394886 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7400e040 con 0x7f3a7c0cc1e0 2023-10-16T20:07:22.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.805+0000 7f3a737fe640 1 -- 172.21.15.88:0/2032394886 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7400f040 con 0x7f3a7c0cc1e0 2023-10-16T20:07:22.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.805+0000 7f3a737fe640 1 -- 172.21.15.88:0/2032394886 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a74015430 con 0x7f3a7c0cc1e0 2023-10-16T20:07:22.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.805+0000 7f3a81691640 1 -- 172.21.15.88:0/2032394886 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a7c0ca510 con 0x7f3a7c0cc1e0 2023-10-16T20:07:22.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.805+0000 7f3a81691640 1 -- 172.21.15.88:0/2032394886 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7c0c7ca0 con 0x7f3a7c0cc1e0 2023-10-16T20:07:22.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.805+0000 7f3a81691640 1 -- 172.21.15.88:0/2032394886 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a7c04e9c0 con 0x7f3a7c0cc1e0 2023-10-16T20:07:22.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.809+0000 7f3a737fe640 1 -- 172.21.15.88:0/2032394886 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3a74015590 con 0x7f3a7c0cc1e0 2023-10-16T20:07:22.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.809+0000 7f3a737fe640 1 --2- 172.21.15.88:0/2032394886 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3a5c0664d0 0x7f3a5c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:22.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.809+0000 7f3a737fe640 1 -- 172.21.15.88:0/2032394886 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f3a74084380 con 0x7f3a7c0cc1e0 2023-10-16T20:07:22.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.809+0000 7f3a7affd640 1 --2- 172.21.15.88:0/2032394886 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3a5c0664d0 0x7f3a5c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:22.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.809+0000 7f3a7affd640 1 --2- 172.21.15.88:0/2032394886 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3a5c0664d0 0x7f3a5c068990 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c0ad000 tx=0x7f3a6c0a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:22.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.813+0000 7f3a737fe640 1 -- 172.21.15.88:0/2032394886 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3a74051a00 con 0x7f3a7c0cc1e0 2023-10-16T20:07:22.966 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.961+0000 7f3a81691640 1 -- 172.21.15.88:0/2032394886 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a7c156410 con 0x7f3a7c0cc1e0 2023-10-16T20:07:22.967 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.961+0000 7f3a737fe640 1 -- 172.21.15.88:0/2032394886 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+12990 (secure 0 0 0) 0x7f3a74055020 con 0x7f3a7c0cc1e0 2023-10-16T20:07:22.967 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:07:22.968 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":54,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:07:22.246647+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:07:22.969 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.965+0000 7f3a81691640 1 -- 172.21.15.88:0/2032394886 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3a5c0664d0 msgr2=0x7f3a5c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:22.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.965+0000 7f3a81691640 1 --2- 172.21.15.88:0/2032394886 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3a5c0664d0 0x7f3a5c068990 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c0ad000 tx=0x7f3a6c0a6000 comp rx=0 tx=0).stop 2023-10-16T20:07:22.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.965+0000 7f3a81691640 1 -- 172.21.15.88:0/2032394886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a7c0cc1e0 msgr2=0x7f3a7c0c81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:22.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.965+0000 7f3a81691640 1 --2- 172.21.15.88:0/2032394886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a7c0cc1e0 0x7f3a7c0c81d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c161990 tx=0x7f3a7400f580 comp rx=0 tx=0).stop 2023-10-16T20:07:22.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.965+0000 7f3a81691640 1 -- 172.21.15.88:0/2032394886 shutdown_connections 2023-10-16T20:07:22.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.965+0000 7f3a81691640 1 --2- 172.21.15.88:0/2032394886 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f3a5c0664d0 0x7f3a5c068990 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:22.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.965+0000 7f3a81691640 1 --2- 172.21.15.88:0/2032394886 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3a7c0cbd80 0x7f3a7c14e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:22.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.965+0000 7f3a81691640 1 --2- 172.21.15.88:0/2032394886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a7c0cc1e0 0x7f3a7c0c81d0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:22.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.965+0000 7f3a81691640 1 --2- 172.21.15.88:0/2032394886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a68003980 0x7f3a7c14d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:22.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.965+0000 7f3a81691640 1 -- 172.21.15.88:0/2032394886 >> 172.21.15.88:0/2032394886 conn(0x7f3a7c0715c0 msgr2=0x7f3a7c0719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:22.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.965+0000 7f3a81691640 1 -- 172.21.15.88:0/2032394886 shutdown_connections 2023-10-16T20:07:22.972 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:22.965+0000 7f3a81691640 1 -- 172.21.15.88:0/2032394886 wait complete. 2023-10-16T20:07:22.989 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T20:07:22.989 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:07:23.129 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.125+0000 7ff8f4af1640 1 Processor -- start 2023-10-16T20:07:23.129 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.125+0000 7ff8f4af1640 1 -- start start 2023-10-16T20:07:23.130 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.125+0000 7ff8f4af1640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff8f015cb70 0x7ff8f00b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:23.131 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.125+0000 7ff8f4af1640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f00b7d20 con 0x7ff8f00b6a80 2023-10-16T20:07:23.131 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.125+0000 7ff8f4af1640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f00b6eb0 con 0x7ff8f00b4530 2023-10-16T20:07:23.131 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.125+0000 7ff8f4af1640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8f00b7110 con 0x7ff8f015cb70 2023-10-16T20:07:23.131 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.125+0000 7ff8ee59c640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff8f015cb70 0x7ff8f00b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:23.131 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.125+0000 7ff8eed9d640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7ff8f00b6a80 0x7ff8f00d12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:46898/0 (socket says 172.21.15.88:46898) 2023-10-16T20:07:23.132 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.125+0000 7ff8eed9d640 1 -- 172.21.15.88:0/1419180538 learned_addr learned my addr 172.21.15.88:0/1419180538 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:23.132 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.125+0000 7ff8ee59c640 1 -- 172.21.15.88:0/1419180538 >> v1:172.21.15.94:6789/0 conn(0x7ff8f00b4530 legacy=0x7ff8f00d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:23.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.125+0000 7ff8ee59c640 1 -- 172.21.15.88:0/1419180538 >> v1:172.21.15.88:6789/0 conn(0x7ff8f00b6a80 legacy=0x7ff8f00d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:23.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.125+0000 7ff8ee59c640 1 -- 172.21.15.88:0/1419180538 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f00b73f0 con 0x7ff8f015cb70 2023-10-16T20:07:23.133 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.125+0000 7ff8edd9b640 1 -- 172.21.15.88:0/1419180538 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3768313832 0 0) 0x7ff8f00b7d20 con 0x7ff8f00b6a80 2023-10-16T20:07:23.134 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.129+0000 7ff8ee59c640 1 --2- 172.21.15.88:0/1419180538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8f015cb70 0x7ff8f00b83d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff8e00097c0 tx=0x7ff8e001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ddb3e851286e56 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:23.134 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.129+0000 7ff8edd9b640 1 -- 172.21.15.88:0/1419180538 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e00a8970 con 0x7ff8f015cb70 2023-10-16T20:07:23.134 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.129+0000 7ff8edd9b640 1 -- 172.21.15.88:0/1419180538 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e00a8ad0 con 0x7ff8f015cb70 2023-10-16T20:07:23.134 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.129+0000 7ff8edd9b640 1 -- 172.21.15.88:0/1419180538 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e00a8e60 con 0x7ff8f015cb70 2023-10-16T20:07:23.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.129+0000 7ff8f4af1640 1 -- 172.21.15.88:0/1419180538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8f015cb70 msgr2=0x7ff8f00b83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:23.135 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.129+0000 7ff8f4af1640 1 --2- 172.21.15.88:0/1419180538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8f015cb70 0x7ff8f00b83d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff8e00097c0 tx=0x7ff8e001ec70 comp rx=0 tx=0).stop 2023-10-16T20:07:23.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.129+0000 7ff8f4af1640 1 -- 172.21.15.88:0/1419180538 shutdown_connections 2023-10-16T20:07:23.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.129+0000 7ff8f4af1640 1 --2- 172.21.15.88:0/1419180538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8f015cb70 0x7ff8f00b83d0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:23.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.129+0000 7ff8f4af1640 1 -- 172.21.15.88:0/1419180538 >> 172.21.15.88:0/1419180538 conn(0x7ff8f006b6e0 msgr2=0x7ff8f006baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:23.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.129+0000 7ff8f4af1640 1 -- 172.21.15.88:0/1419180538 shutdown_connections 2023-10-16T20:07:23.136 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.129+0000 7ff8f4af1640 1 -- 172.21.15.88:0/1419180538 wait complete. 2023-10-16T20:07:23.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.133+0000 7ff8f4af1640 1 Processor -- start 2023-10-16T20:07:23.137 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.133+0000 7ff8f4af1640 1 -- start start 2023-10-16T20:07:23.139 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.133+0000 7ff8f4af1640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff8f00b4530 0x7ff8f00c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:23.139 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.133+0000 7ff8f4af1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8f00b6a80 0x7ff8f00b3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:23.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.133+0000 7ff8f4af1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8f015cb70 0x7ff8f00b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:23.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.133+0000 7ff8f4af1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f01f5550 con 0x7ff8f00b6a80 2023-10-16T20:07:23.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.133+0000 7ff8f4af1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f00c8780 con 0x7ff8f015cb70 2023-10-16T20:07:23.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.133+0000 7ff8f4af1640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f01f57a0 con 0x7ff8f00b4530 2023-10-16T20:07:23.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.133+0000 7ff8e7fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8f00b6a80 0x7ff8f00b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:23.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.133+0000 7ff8e7fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8f00b6a80 0x7ff8f00b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38436/0 (socket says 172.21.15.88:38436) 2023-10-16T20:07:23.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.133+0000 7ff8e7fff640 1 -- 172.21.15.88:0/2485067980 learned_addr learned my addr 172.21.15.88:0/2485067980 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:23.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.133+0000 7ff8ee59c640 1 --2- 172.21.15.88:0/2485067980 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff8f00b4530 0x7ff8f00c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:23.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.133+0000 7ff8eed9d640 1 --2- 172.21.15.88:0/2485067980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8f015cb70 0x7ff8f00b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:23.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.133+0000 7ff8e7fff640 1 -- 172.21.15.88:0/2485067980 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff8f00b4530 msgr2=0x7ff8f00c8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:23.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.133+0000 7ff8e7fff640 1 --2- 172.21.15.88:0/2485067980 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff8f00b4530 0x7ff8f00c8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:23.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.133+0000 7ff8e7fff640 1 -- 172.21.15.88:0/2485067980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8f015cb70 msgr2=0x7ff8f00b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:23.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.133+0000 7ff8e7fff640 1 --2- 172.21.15.88:0/2485067980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8f015cb70 0x7ff8f00b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:23.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.133+0000 7ff8e7fff640 1 -- 172.21.15.88:0/2485067980 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e0009420 con 0x7ff8f00b6a80 2023-10-16T20:07:23.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.133+0000 7ff8e7fff640 1 --2- 172.21.15.88:0/2485067980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8f00b6a80 0x7ff8f00b3560 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800c500 tx=0x7ff8d800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:23.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.133+0000 7ff8e6ffd640 1 -- 172.21.15.88:0/2485067980 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8005220 con 0x7ff8f00b6a80 2023-10-16T20:07:23.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.133+0000 7ff8e6ffd640 1 -- 172.21.15.88:0/2485067980 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d8005380 con 0x7ff8f00b6a80 2023-10-16T20:07:23.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.133+0000 7ff8f4af1640 1 -- 172.21.15.88:0/2485067980 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f01f5a80 con 0x7ff8f00b6a80 2023-10-16T20:07:23.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.133+0000 7ff8e6ffd640 1 -- 172.21.15.88:0/2485067980 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8007270 con 0x7ff8f00b6a80 2023-10-16T20:07:23.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.133+0000 7ff8f4af1640 1 -- 172.21.15.88:0/2485067980 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f01f6040 con 0x7ff8f00b6a80 2023-10-16T20:07:23.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.137+0000 7ff8f4af1640 1 -- 172.21.15.88:0/2485067980 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8f004e9c0 con 0x7ff8f00b6a80 2023-10-16T20:07:23.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.137+0000 7ff8e6ffd640 1 -- 172.21.15.88:0/2485067980 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff8d80073d0 con 0x7ff8f00b6a80 2023-10-16T20:07:23.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.137+0000 7ff8e6ffd640 1 --2- 172.21.15.88:0/2485067980 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff8d00664a0 0x7ff8d0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:23.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.137+0000 7ff8e6ffd640 1 -- 172.21.15.88:0/2485067980 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5177+0+0 (secure 0 0 0) 0x7ff8d8083480 con 0x7ff8f00b6a80 2023-10-16T20:07:23.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.137+0000 7ff8ee59c640 1 --2- 172.21.15.88:0/2485067980 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff8d00664a0 0x7ff8d0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:23.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.137+0000 7ff8ee59c640 1 --2- 172.21.15.88:0/2485067980 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff8d00664a0 0x7ff8d0068960 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff8e00b2000 tx=0x7ff8e0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:23.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.141+0000 7ff8e6ffd640 1 -- 172.21.15.88:0/2485067980 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff8d8050b30 con 0x7ff8f00b6a80 2023-10-16T20:07:23.299 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.293+0000 7ff8f4af1640 1 -- 172.21.15.88:0/2485067980 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff8f00b7980 con 0x7ff8f00b6a80 2023-10-16T20:07:23.300 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.297+0000 7ff8e6ffd640 1 -- 172.21.15.88:0/2485067980 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff8d8054150 con 0x7ff8f00b6a80 2023-10-16T20:07:23.300 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:07:23.302 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.297+0000 7ff8f4af1640 1 -- 172.21.15.88:0/2485067980 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff8d00664a0 msgr2=0x7ff8d0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:23.302 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.297+0000 7ff8f4af1640 1 --2- 172.21.15.88:0/2485067980 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff8d00664a0 0x7ff8d0068960 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff8e00b2000 tx=0x7ff8e0008040 comp rx=0 tx=0).stop 2023-10-16T20:07:23.302 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.297+0000 7ff8f4af1640 1 -- 172.21.15.88:0/2485067980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8f00b6a80 msgr2=0x7ff8f00b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:23.302 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.297+0000 7ff8f4af1640 1 --2- 172.21.15.88:0/2485067980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8f00b6a80 0x7ff8f00b3560 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800c500 tx=0x7ff8d800c9c0 comp rx=0 tx=0).stop 2023-10-16T20:07:23.302 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.297+0000 7ff8f4af1640 1 -- 172.21.15.88:0/2485067980 shutdown_connections 2023-10-16T20:07:23.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.297+0000 7ff8f4af1640 1 --2- 172.21.15.88:0/2485067980 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff8d00664a0 0x7ff8d0068960 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:23.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.297+0000 7ff8f4af1640 1 --2- 172.21.15.88:0/2485067980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8f015cb70 0x7ff8f00b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:23.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.297+0000 7ff8f4af1640 1 --2- 172.21.15.88:0/2485067980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8f00b6a80 0x7ff8f00b3560 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:23.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.297+0000 7ff8f4af1640 1 --2- 172.21.15.88:0/2485067980 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff8f00b4530 0x7ff8f00c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:23.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.297+0000 7ff8f4af1640 1 -- 172.21.15.88:0/2485067980 >> 172.21.15.88:0/2485067980 conn(0x7ff8f006b6e0 msgr2=0x7ff8f00d2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:23.304 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.297+0000 7ff8f4af1640 1 -- 172.21.15.88:0/2485067980 shutdown_connections 2023-10-16T20:07:23.304 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.297+0000 7ff8f4af1640 1 -- 172.21.15.88:0/2485067980 wait complete. 2023-10-16T20:07:23.320 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T20:07:23.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.453+0000 7fcf0949e640 1 Processor -- start 2023-10-16T20:07:23.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.453+0000 7fcf0949e640 1 -- start start 2023-10-16T20:07:23.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.453+0000 7fcf0949e640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fcf041599f0 0x7fcf040b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:23.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.453+0000 7fcf0949e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fcf04069e90 0x7fcf0406a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:23.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.453+0000 7fcf0949e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf040b05a0 0x7fcf040b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:23.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.453+0000 7fcf0949e640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf0406a830 con 0x7fcf041599f0 2023-10-16T20:07:23.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.453+0000 7fcf0949e640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf040b1260 con 0x7fcf04069e90 2023-10-16T20:07:23.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.453+0000 7fcf0949e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf040ae960 con 0x7fcf040b05a0 2023-10-16T20:07:23.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.453+0000 7fcf02ffd640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fcf041599f0 0x7fcf040b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:23.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.457+0000 7fcf02ffd640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fcf041599f0 0x7fcf040b2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38448/0 (socket says 172.21.15.88:38448) 2023-10-16T20:07:23.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.457+0000 7fcf02ffd640 1 -- 172.21.15.88:0/1120505205 learned_addr learned my addr 172.21.15.88:0/1120505205 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:23.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.457+0000 7fcf027fc640 1 --2- 172.21.15.88:0/1120505205 >> v2:172.21.15.94:3300/0 conn(0x7fcf04069e90 0x7fcf0406a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:23.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.457+0000 7fcf037fe640 1 --2- 172.21.15.88:0/1120505205 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf040b05a0 0x7fcf040b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:23.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.457+0000 7fcf02ffd640 1 -- 172.21.15.88:0/1120505205 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf040b05a0 msgr2=0x7fcf040b0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:23.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.457+0000 7fcf02ffd640 1 --2- 172.21.15.88:0/1120505205 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf040b05a0 0x7fcf040b0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:23.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.457+0000 7fcf02ffd640 1 -- 172.21.15.88:0/1120505205 >> v2:172.21.15.94:3300/0 conn(0x7fcf04069e90 msgr2=0x7fcf0406a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:23.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.457+0000 7fcf02ffd640 1 --2- 172.21.15.88:0/1120505205 >> v2:172.21.15.94:3300/0 conn(0x7fcf04069e90 0x7fcf0406a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:23.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.457+0000 7fcf02ffd640 1 -- 172.21.15.88:0/1120505205 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf040aec40 con 0x7fcf041599f0 2023-10-16T20:07:23.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.457+0000 7fcf02ffd640 1 --2- 172.21.15.88:0/1120505205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf041599f0 0x7fcf040b2a30 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fcf040ae500 tx=0x7fcef401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e152b1e075c535 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:23.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.457+0000 7fcf01ffb640 1 -- 172.21.15.88:0/1120505205 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef40aa5f0 con 0x7fcf041599f0 2023-10-16T20:07:23.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.457+0000 7fcf01ffb640 1 -- 172.21.15.88:0/1120505205 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef40aa750 con 0x7fcf041599f0 2023-10-16T20:07:23.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.457+0000 7fcf01ffb640 1 -- 172.21.15.88:0/1120505205 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef40aaae0 con 0x7fcf041599f0 2023-10-16T20:07:23.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.461+0000 7fcf0949e640 1 -- 172.21.15.88:0/1120505205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf041599f0 msgr2=0x7fcf040b2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:23.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.461+0000 7fcf0949e640 1 --2- 172.21.15.88:0/1120505205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf041599f0 0x7fcf040b2a30 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fcf040ae500 tx=0x7fcef401c4d0 comp rx=0 tx=0).stop 2023-10-16T20:07:23.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.461+0000 7fcf0949e640 1 -- 172.21.15.88:0/1120505205 shutdown_connections 2023-10-16T20:07:23.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.461+0000 7fcf0949e640 1 --2- 172.21.15.88:0/1120505205 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf040b05a0 0x7fcf040b0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:23.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.461+0000 7fcf0949e640 1 --2- 172.21.15.88:0/1120505205 >> v2:172.21.15.94:3300/0 conn(0x7fcf04069e90 0x7fcf0406a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:23.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.461+0000 7fcf0949e640 1 --2- 172.21.15.88:0/1120505205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf041599f0 0x7fcf040b2a30 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:23.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.461+0000 7fcf0949e640 1 -- 172.21.15.88:0/1120505205 >> 172.21.15.88:0/1120505205 conn(0x7fcf040ce5e0 msgr2=0x7fcf040ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:23.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.461+0000 7fcf0949e640 1 -- 172.21.15.88:0/1120505205 shutdown_connections 2023-10-16T20:07:23.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.461+0000 7fcf0949e640 1 -- 172.21.15.88:0/1120505205 wait complete. 2023-10-16T20:07:23.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.461+0000 7fcf0949e640 1 Processor -- start 2023-10-16T20:07:23.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.461+0000 7fcf0949e640 1 -- start start 2023-10-16T20:07:23.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.461+0000 7fcf0949e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf04069e90 0x7fcf04139fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:23.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.461+0000 7fcf0949e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf040b05a0 0x7fcf040aa070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:23.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.461+0000 7fcf0949e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf041599f0 0x7fcf040aa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:23.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.461+0000 7fcf0949e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf040ad3a0 con 0x7fcf041599f0 2023-10-16T20:07:23.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.461+0000 7fcf0949e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf040ad0c0 con 0x7fcf04069e90 2023-10-16T20:07:23.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.461+0000 7fcf0949e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf04148830 con 0x7fcf040b05a0 2023-10-16T20:07:23.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.461+0000 7fcf037fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf041599f0 0x7fcf040aa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:23.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.461+0000 7fcf037fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf041599f0 0x7fcf040aa5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38464/0 (socket says 172.21.15.88:38464) 2023-10-16T20:07:23.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.461+0000 7fcf037fe640 1 -- 172.21.15.88:0/4008841398 learned_addr learned my addr 172.21.15.88:0/4008841398 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:23.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.461+0000 7fcf02ffd640 1 --2- 172.21.15.88:0/4008841398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf04069e90 0x7fcf04139fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:23.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.461+0000 7fcf027fc640 1 --2- 172.21.15.88:0/4008841398 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf040b05a0 0x7fcf040aa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:23.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.465+0000 7fcf037fe640 1 -- 172.21.15.88:0/4008841398 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf040b05a0 msgr2=0x7fcf040aa070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:23.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.465+0000 7fcf037fe640 1 --2- 172.21.15.88:0/4008841398 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf040b05a0 0x7fcf040aa070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:23.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.465+0000 7fcf037fe640 1 -- 172.21.15.88:0/4008841398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf04069e90 msgr2=0x7fcf04139fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:23.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.465+0000 7fcf037fe640 1 --2- 172.21.15.88:0/4008841398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf04069e90 0x7fcf04139fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:23.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.465+0000 7fcf037fe640 1 -- 172.21.15.88:0/4008841398 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef40094d0 con 0x7fcf041599f0 2023-10-16T20:07:23.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.465+0000 7fcf037fe640 1 --2- 172.21.15.88:0/4008841398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf041599f0 0x7fcf040aa5b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00efc0 tx=0x7fcefc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:23.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.465+0000 7fceeb7fe640 1 -- 172.21.15.88:0/4008841398 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcefc014480 con 0x7fcf041599f0 2023-10-16T20:07:23.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.465+0000 7fceeb7fe640 1 -- 172.21.15.88:0/4008841398 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcefc00f040 con 0x7fcf041599f0 2023-10-16T20:07:23.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.465+0000 7fceeb7fe640 1 -- 172.21.15.88:0/4008841398 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcefc014820 con 0x7fcf041599f0 2023-10-16T20:07:23.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.465+0000 7fcf0949e640 1 -- 172.21.15.88:0/4008841398 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf04148b10 con 0x7fcf041599f0 2023-10-16T20:07:23.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.465+0000 7fcf0949e640 1 -- 172.21.15.88:0/4008841398 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf040ac390 con 0x7fcf041599f0 2023-10-16T20:07:23.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.465+0000 7fcf0949e640 1 -- 172.21.15.88:0/4008841398 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf04148690 con 0x7fcf041599f0 2023-10-16T20:07:23.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.465+0000 7fceeb7fe640 1 -- 172.21.15.88:0/4008841398 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcefc014980 con 0x7fcf041599f0 2023-10-16T20:07:23.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.465+0000 7fceeb7fe640 1 --2- 172.21.15.88:0/4008841398 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcef00664a0 0x7fcef0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:23.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.465+0000 7fceeb7fe640 1 -- 172.21.15.88:0/4008841398 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fcefc084a50 con 0x7fcf041599f0 2023-10-16T20:07:23.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.465+0000 7fcf02ffd640 1 --2- 172.21.15.88:0/4008841398 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcef00664a0 0x7fcef0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:23.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.469+0000 7fcf02ffd640 1 --2- 172.21.15.88:0/4008841398 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcef00664a0 0x7fcef0068960 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcef40ae000 tx=0x7fcef40b5040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:23.475 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.469+0000 7fceeb7fe640 1 -- 172.21.15.88:0/4008841398 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcf04148690 con 0x7fcf041599f0 2023-10-16T20:07:23.626 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.621+0000 7fcf0949e640 1 -- 172.21.15.88:0/4008841398 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fcf04148690 con 0x7fcf041599f0 2023-10-16T20:07:23.627 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.621+0000 7fceeb7fe640 1 -- 172.21.15.88:0/4008841398 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v54) v1 ==== 84+0+65 (secure 0 0 0) 0x7fcf04148690 con 0x7fcf041599f0 2023-10-16T20:07:23.627 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:07:23.628 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2023-10-16T20:07:23.628 INFO:teuthology.orchestra.run.smithi088.stdout: { 2023-10-16T20:07:23.628 INFO:teuthology.orchestra.run.smithi088.stdout: "poolnum": 1, 2023-10-16T20:07:23.628 INFO:teuthology.orchestra.run.smithi088.stdout: "poolname": ".mgr" 2023-10-16T20:07:23.628 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:07:23.628 INFO:teuthology.orchestra.run.smithi088.stdout:] 2023-10-16T20:07:23.630 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.625+0000 7fcf0949e640 1 -- 172.21.15.88:0/4008841398 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcef00664a0 msgr2=0x7fcef0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:23.630 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.625+0000 7fcf0949e640 1 --2- 172.21.15.88:0/4008841398 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcef00664a0 0x7fcef0068960 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcef40ae000 tx=0x7fcef40b5040 comp rx=0 tx=0).stop 2023-10-16T20:07:23.630 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.625+0000 7fcf0949e640 1 -- 172.21.15.88:0/4008841398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf041599f0 msgr2=0x7fcf040aa5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:23.630 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.625+0000 7fcf0949e640 1 --2- 172.21.15.88:0/4008841398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf041599f0 0x7fcf040aa5b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00efc0 tx=0x7fcefc00c490 comp rx=0 tx=0).stop 2023-10-16T20:07:23.630 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.625+0000 7fcf0949e640 1 -- 172.21.15.88:0/4008841398 shutdown_connections 2023-10-16T20:07:23.631 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.625+0000 7fcf0949e640 1 --2- 172.21.15.88:0/4008841398 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcef00664a0 0x7fcef0068960 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:23.631 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.625+0000 7fcf0949e640 1 --2- 172.21.15.88:0/4008841398 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf040b05a0 0x7fcf040aa070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:23.631 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.625+0000 7fcf0949e640 1 --2- 172.21.15.88:0/4008841398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf041599f0 0x7fcf040aa5b0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:23.631 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.625+0000 7fcf0949e640 1 --2- 172.21.15.88:0/4008841398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf04069e90 0x7fcf04139fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:23.631 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.625+0000 7fcf0949e640 1 -- 172.21.15.88:0/4008841398 >> 172.21.15.88:0/4008841398 conn(0x7fcf040ce5e0 msgr2=0x7fcf040716a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:23.631 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.625+0000 7fcf0949e640 1 -- 172.21.15.88:0/4008841398 shutdown_connections 2023-10-16T20:07:23.632 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.625+0000 7fcf0949e640 1 -- 172.21.15.88:0/4008841398 wait complete. 2023-10-16T20:07:23.648 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'testcephfs1' 2023-10-16T20:07:23.648 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create testcephfs1_metadata --pg_num_min 64 2023-10-16T20:07:23.790 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.785+0000 7fa074c17640 1 Processor -- start 2023-10-16T20:07:23.790 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.785+0000 7fa074c17640 1 -- start start 2023-10-16T20:07:23.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.785+0000 7fa074c17640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa07015cb20 0x7fa0700cca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:23.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.785+0000 7fa074c17640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fa07006e720 0x7fa07006d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:23.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.785+0000 7fa074c17640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa070069f30 con 0x7fa0700abd60 2023-10-16T20:07:23.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.785+0000 7fa074c17640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa07006dc50 con 0x7fa07006e720 2023-10-16T20:07:23.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.785+0000 7fa074c17640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0700cd160 con 0x7fa07015cb20 2023-10-16T20:07:23.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.785+0000 7fa067fff640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fa0700abd60 0x7fa07006e010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:46900/0 (socket says 172.21.15.88:46900) 2023-10-16T20:07:23.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.785+0000 7fa067fff640 1 -- 172.21.15.88:0/2161633372 learned_addr learned my addr 172.21.15.88:0/2161633372 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:23.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.785+0000 7fa06e59c640 1 --2- 172.21.15.88:0/2161633372 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa07015cb20 0x7fa0700cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:23.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.785+0000 7fa06ed9d640 1 --2- 172.21.15.88:0/2161633372 >> v2:172.21.15.94:3300/0 conn(0x7fa07006e720 0x7fa07006d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:23.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.789+0000 7fa06dd9b640 1 -- 172.21.15.88:0/2161633372 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3027522397 0 0) 0x7fa070069f30 con 0x7fa0700abd60 2023-10-16T20:07:23.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.789+0000 7fa06dd9b640 1 -- 172.21.15.88:0/2161633372 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa05c003530 con 0x7fa0700abd60 2023-10-16T20:07:23.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.789+0000 7fa06e59c640 1 -- 172.21.15.88:0/2161633372 >> v2:172.21.15.94:3300/0 conn(0x7fa07006e720 msgr2=0x7fa07006d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:23.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.789+0000 7fa06e59c640 1 --2- 172.21.15.88:0/2161633372 >> v2:172.21.15.94:3300/0 conn(0x7fa07006e720 0x7fa07006d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:23.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.789+0000 7fa06e59c640 1 -- 172.21.15.88:0/2161633372 >> v1:172.21.15.88:6789/0 conn(0x7fa0700abd60 legacy=0x7fa07006e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:23.795 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.789+0000 7fa06e59c640 1 -- 172.21.15.88:0/2161633372 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa07006a1d0 con 0x7fa07015cb20 2023-10-16T20:07:23.796 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.789+0000 7fa06e59c640 1 --2- 172.21.15.88:0/2161633372 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa07015cb20 0x7fa0700cca60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa0600097c0 tx=0x7fa06001c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=934c3127112f8ce8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:23.796 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.789+0000 7fa06dd9b640 1 -- 172.21.15.88:0/2161633372 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0600ac410 con 0x7fa07015cb20 2023-10-16T20:07:23.796 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.793+0000 7fa06dd9b640 1 -- 172.21.15.88:0/2161633372 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0600ac570 con 0x7fa07015cb20 2023-10-16T20:07:23.797 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.793+0000 7fa06dd9b640 1 -- 172.21.15.88:0/2161633372 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0600ac900 con 0x7fa07015cb20 2023-10-16T20:07:23.797 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.793+0000 7fa074c17640 1 -- 172.21.15.88:0/2161633372 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa07015cb20 msgr2=0x7fa0700cca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:23.797 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.793+0000 7fa074c17640 1 --2- 172.21.15.88:0/2161633372 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa07015cb20 0x7fa0700cca60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa0600097c0 tx=0x7fa06001c480 comp rx=0 tx=0).stop 2023-10-16T20:07:23.797 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.793+0000 7fa074c17640 1 -- 172.21.15.88:0/2161633372 shutdown_connections 2023-10-16T20:07:23.797 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.793+0000 7fa074c17640 1 --2- 172.21.15.88:0/2161633372 >> v2:172.21.15.94:3300/0 conn(0x7fa07006e720 0x7fa07006d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:23.798 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.793+0000 7fa074c17640 1 --2- 172.21.15.88:0/2161633372 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa07015cb20 0x7fa0700cca60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:23.798 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.793+0000 7fa074c17640 1 -- 172.21.15.88:0/2161633372 >> 172.21.15.88:0/2161633372 conn(0x7fa0700b0980 msgr2=0x7fa0700b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:23.798 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.793+0000 7fa074c17640 1 -- 172.21.15.88:0/2161633372 shutdown_connections 2023-10-16T20:07:23.798 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.793+0000 7fa074c17640 1 -- 172.21.15.88:0/2161633372 wait complete. 2023-10-16T20:07:23.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.793+0000 7fa074c17640 1 Processor -- start 2023-10-16T20:07:23.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.793+0000 7fa074c17640 1 -- start start 2023-10-16T20:07:23.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.793+0000 7fa074c17640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa07006e720 0x7fa07015f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:23.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.793+0000 7fa074c17640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0700abd60 0x7fa070149070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:23.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.793+0000 7fa074c17640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa07015cb20 0x7fa0700c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:23.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.793+0000 7fa074c17640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07015e850 con 0x7fa07015cb20 2023-10-16T20:07:23.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.793+0000 7fa074c17640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07015a4b0 con 0x7fa07006e720 2023-10-16T20:07:23.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.793+0000 7fa074c17640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0701600a0 con 0x7fa0700abd60 2023-10-16T20:07:23.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.793+0000 7fa06ed9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa07015cb20 0x7fa0700c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:23.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.793+0000 7fa06ed9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa07015cb20 0x7fa0700c9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38472/0 (socket says 172.21.15.88:38472) 2023-10-16T20:07:23.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.793+0000 7fa06ed9d640 1 -- 172.21.15.88:0/2541581871 learned_addr learned my addr 172.21.15.88:0/2541581871 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:23.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.797+0000 7fa06e59c640 1 --2- 172.21.15.88:0/2541581871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa07006e720 0x7fa07015f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:23.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.797+0000 7fa067fff640 1 --2- 172.21.15.88:0/2541581871 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0700abd60 0x7fa070149070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:23.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.797+0000 7fa06ed9d640 1 -- 172.21.15.88:0/2541581871 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0700abd60 msgr2=0x7fa070149070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:23.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.797+0000 7fa06ed9d640 1 --2- 172.21.15.88:0/2541581871 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0700abd60 0x7fa070149070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:23.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.797+0000 7fa06ed9d640 1 -- 172.21.15.88:0/2541581871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa07006e720 msgr2=0x7fa07015f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:23.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.797+0000 7fa06ed9d640 1 --2- 172.21.15.88:0/2541581871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa07006e720 0x7fa07015f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:23.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.797+0000 7fa06ed9d640 1 -- 172.21.15.88:0/2541581871 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa060009420 con 0x7fa07015cb20 2023-10-16T20:07:23.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.797+0000 7fa06ed9d640 1 --2- 172.21.15.88:0/2541581871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa07015cb20 0x7fa0700c9ef0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fa06800efc0 tx=0x7fa06800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:23.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.797+0000 7fa066ffd640 1 -- 172.21.15.88:0/2541581871 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa068014480 con 0x7fa07015cb20 2023-10-16T20:07:23.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.797+0000 7fa066ffd640 1 -- 172.21.15.88:0/2541581871 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06800f040 con 0x7fa07015cb20 2023-10-16T20:07:23.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.797+0000 7fa074c17640 1 -- 172.21.15.88:0/2541581871 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa070159df0 con 0x7fa07015cb20 2023-10-16T20:07:23.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.797+0000 7fa066ffd640 1 -- 172.21.15.88:0/2541581871 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa068014820 con 0x7fa07015cb20 2023-10-16T20:07:23.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.797+0000 7fa074c17640 1 -- 172.21.15.88:0/2541581871 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa07015fd30 con 0x7fa07015cb20 2023-10-16T20:07:23.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.797+0000 7fa074c17640 1 -- 172.21.15.88:0/2541581871 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa070066210 con 0x7fa07015cb20 2023-10-16T20:07:23.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.797+0000 7fa066ffd640 1 -- 172.21.15.88:0/2541581871 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa068014980 con 0x7fa07015cb20 2023-10-16T20:07:23.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.797+0000 7fa066ffd640 1 --2- 172.21.15.88:0/2541581871 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa0500664a0 0x7fa050068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:23.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.797+0000 7fa066ffd640 1 -- 172.21.15.88:0/2541581871 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fa068084a00 con 0x7fa07015cb20 2023-10-16T20:07:23.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.797+0000 7fa06e59c640 1 --2- 172.21.15.88:0/2541581871 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa0500664a0 0x7fa050068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:23.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.801+0000 7fa06e59c640 1 --2- 172.21.15.88:0/2541581871 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa0500664a0 0x7fa050068960 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa0600ae000 tx=0x7fa060008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:23.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.801+0000 7fa066ffd640 1 -- 172.21.15.88:0/2541581871 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa068052130 con 0x7fa07015cb20 2023-10-16T20:07:23.957 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:23.953+0000 7fa074c17640 1 -- 172.21.15.88:0/2541581871 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "testcephfs1_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa07004e9c0 con 0x7fa07015cb20 2023-10-16T20:07:24.253 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.249+0000 7fa066ffd640 1 -- 172.21.15.88:0/2541581871 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "testcephfs1_metadata", "pg_num_min": 64}]=0 pool 'testcephfs1_metadata' created v55) v1 ==== 148+0+0 (secure 0 0 0) 0x7fa0680180b0 con 0x7fa07015cb20 2023-10-16T20:07:24.253 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'testcephfs1_metadata' created 2023-10-16T20:07:24.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.253+0000 7fa074c17640 1 -- 172.21.15.88:0/2541581871 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa0500664a0 msgr2=0x7fa050068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:24.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.253+0000 7fa074c17640 1 --2- 172.21.15.88:0/2541581871 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa0500664a0 0x7fa050068960 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa0600ae000 tx=0x7fa060008040 comp rx=0 tx=0).stop 2023-10-16T20:07:24.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.253+0000 7fa074c17640 1 -- 172.21.15.88:0/2541581871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa07015cb20 msgr2=0x7fa0700c9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:24.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.253+0000 7fa074c17640 1 --2- 172.21.15.88:0/2541581871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa07015cb20 0x7fa0700c9ef0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fa06800efc0 tx=0x7fa06800c490 comp rx=0 tx=0).stop 2023-10-16T20:07:24.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.253+0000 7fa074c17640 1 -- 172.21.15.88:0/2541581871 shutdown_connections 2023-10-16T20:07:24.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.253+0000 7fa074c17640 1 --2- 172.21.15.88:0/2541581871 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa0500664a0 0x7fa050068960 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:24.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.253+0000 7fa074c17640 1 --2- 172.21.15.88:0/2541581871 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa0700abd60 0x7fa070149070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:24.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.253+0000 7fa074c17640 1 --2- 172.21.15.88:0/2541581871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa07015cb20 0x7fa0700c9ef0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:24.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.253+0000 7fa074c17640 1 --2- 172.21.15.88:0/2541581871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa07006e720 0x7fa07015f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:24.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.253+0000 7fa074c17640 1 -- 172.21.15.88:0/2541581871 >> 172.21.15.88:0/2541581871 conn(0x7fa0700b0980 msgr2=0x7fa0700afb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:24.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.253+0000 7fa074c17640 1 -- 172.21.15.88:0/2541581871 shutdown_connections 2023-10-16T20:07:24.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.253+0000 7fa074c17640 1 -- 172.21.15.88:0/2541581871 wait complete. 2023-10-16T20:07:24.281 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create testcephfs1_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T20:07:24.426 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.421+0000 7f6986d65640 1 Processor -- start 2023-10-16T20:07:24.426 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.421+0000 7f6986d65640 1 -- start start 2023-10-16T20:07:24.427 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.421+0000 7f6986d65640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f698015cb70 0x7f69800b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:24.428 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.421+0000 7f6986d65640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f69800b6a80 0x7f69800b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:24.428 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.421+0000 7f6986d65640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69800b4ed0 con 0x7f69800b4530 2023-10-16T20:07:24.428 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.421+0000 7f6986d65640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69800b6f10 con 0x7f69800b6a80 2023-10-16T20:07:24.428 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.421+0000 7f6986d65640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69800b7d20 con 0x7f698015cb70 2023-10-16T20:07:24.428 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.425+0000 7f6984b01640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f698015cb70 0x7f69800b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:24.429 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.425+0000 7f697ffff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f69800b4530 0x7f69800d0bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:35920/0 (socket says 172.21.15.88:35920) 2023-10-16T20:07:24.429 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.425+0000 7f697ffff640 1 -- 172.21.15.88:0/3519013815 learned_addr learned my addr 172.21.15.88:0/3519013815 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:24.429 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.425+0000 7f6985302640 1 --2- 172.21.15.88:0/3519013815 >> v2:172.21.15.94:3300/0 conn(0x7f69800b6a80 0x7f69800b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:24.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.425+0000 7f697f7fe640 1 -- 172.21.15.88:0/3519013815 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2662275811 0 0) 0x7f69800b4ed0 con 0x7f69800b4530 2023-10-16T20:07:24.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.425+0000 7f6984b01640 1 -- 172.21.15.88:0/3519013815 >> v2:172.21.15.94:3300/0 conn(0x7f69800b6a80 msgr2=0x7f69800b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:24.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.425+0000 7f6984b01640 1 --2- 172.21.15.88:0/3519013815 >> v2:172.21.15.94:3300/0 conn(0x7f69800b6a80 0x7f69800b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:24.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.425+0000 7f6984b01640 1 -- 172.21.15.88:0/3519013815 >> v1:172.21.15.94:6789/0 conn(0x7f69800b4530 legacy=0x7f69800d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:24.431 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.425+0000 7f6984b01640 1 -- 172.21.15.88:0/3519013815 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69800b7110 con 0x7f698015cb70 2023-10-16T20:07:24.431 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.425+0000 7f6984b01640 1 --2- 172.21.15.88:0/3519013815 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f698015cb70 0x7f69800b83d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f69700097c0 tx=0x7f697001ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29b613893aa16eaf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:24.432 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.425+0000 7f697f7fe640 1 -- 172.21.15.88:0/3519013815 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69700a8970 con 0x7f698015cb70 2023-10-16T20:07:24.432 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.425+0000 7f697f7fe640 1 -- 172.21.15.88:0/3519013815 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69700a8ad0 con 0x7f698015cb70 2023-10-16T20:07:24.432 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.425+0000 7f697f7fe640 1 -- 172.21.15.88:0/3519013815 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69700a8e60 con 0x7f698015cb70 2023-10-16T20:07:24.433 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.429+0000 7f6986d65640 1 -- 172.21.15.88:0/3519013815 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f698015cb70 msgr2=0x7f69800b83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:24.433 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.429+0000 7f6986d65640 1 --2- 172.21.15.88:0/3519013815 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f698015cb70 0x7f69800b83d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f69700097c0 tx=0x7f697001ec70 comp rx=0 tx=0).stop 2023-10-16T20:07:24.433 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.429+0000 7f6986d65640 1 -- 172.21.15.88:0/3519013815 shutdown_connections 2023-10-16T20:07:24.433 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.429+0000 7f6986d65640 1 --2- 172.21.15.88:0/3519013815 >> v2:172.21.15.94:3300/0 conn(0x7f69800b6a80 0x7f69800b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:24.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.429+0000 7f6986d65640 1 --2- 172.21.15.88:0/3519013815 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f698015cb70 0x7f69800b83d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:24.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.429+0000 7f6986d65640 1 -- 172.21.15.88:0/3519013815 >> 172.21.15.88:0/3519013815 conn(0x7f698006b6e0 msgr2=0x7f698006baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:24.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.429+0000 7f6986d65640 1 -- 172.21.15.88:0/3519013815 shutdown_connections 2023-10-16T20:07:24.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.429+0000 7f6986d65640 1 -- 172.21.15.88:0/3519013815 wait complete. 2023-10-16T20:07:24.437 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.429+0000 7f6986d65640 1 Processor -- start 2023-10-16T20:07:24.437 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.429+0000 7f6986d65640 1 -- start start 2023-10-16T20:07:24.437 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.429+0000 7f6986d65640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69800b4530 0x7f69800cd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:24.437 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.429+0000 7f6986d65640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f69800b6a80 0x7f6980159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:24.437 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.429+0000 7f6986d65640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f698015cb70 0x7f698015a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:24.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.429+0000 7f6986d65640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69800c8c50 con 0x7f69800b4530 2023-10-16T20:07:24.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.429+0000 7f6986d65640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698014c640 con 0x7f698015cb70 2023-10-16T20:07:24.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.429+0000 7f6986d65640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f698014c160 con 0x7f69800b6a80 2023-10-16T20:07:24.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.429+0000 7f6984b01640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69800b4530 0x7f69800cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:24.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.429+0000 7f6984b01640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69800b4530 0x7f69800cd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38476/0 (socket says 172.21.15.88:38476) 2023-10-16T20:07:24.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.429+0000 7f6984b01640 1 -- 172.21.15.88:0/3059271250 learned_addr learned my addr 172.21.15.88:0/3059271250 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:24.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.429+0000 7f697ffff640 1 --2- 172.21.15.88:0/3059271250 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f69800b6a80 0x7f6980159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:24.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.429+0000 7f6985302640 1 --2- 172.21.15.88:0/3059271250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f698015cb70 0x7f698015a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:24.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.433+0000 7f6984b01640 1 -- 172.21.15.88:0/3059271250 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f69800b6a80 msgr2=0x7f6980159eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:24.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.433+0000 7f6984b01640 1 --2- 172.21.15.88:0/3059271250 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f69800b6a80 0x7f6980159eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:24.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.433+0000 7f6984b01640 1 -- 172.21.15.88:0/3059271250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f698015cb70 msgr2=0x7f698015a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:24.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.433+0000 7f6984b01640 1 --2- 172.21.15.88:0/3059271250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f698015cb70 0x7f698015a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:24.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.433+0000 7f6984b01640 1 -- 172.21.15.88:0/3059271250 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6970009420 con 0x7f69800b4530 2023-10-16T20:07:24.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.433+0000 7f6984b01640 1 --2- 172.21.15.88:0/3059271250 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69800b4530 0x7f69800cd480 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f69700b3040 tx=0x7f697001ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:24.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.433+0000 7f697d7fa640 1 -- 172.21.15.88:0/3059271250 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69700b9630 con 0x7f69800b4530 2023-10-16T20:07:24.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.433+0000 7f697d7fa640 1 -- 172.21.15.88:0/3059271250 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69700b97b0 con 0x7f69800b4530 2023-10-16T20:07:24.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.433+0000 7f697d7fa640 1 -- 172.21.15.88:0/3059271250 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69700b9b60 con 0x7f69800b4530 2023-10-16T20:07:24.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.433+0000 7f6986d65640 1 -- 172.21.15.88:0/3059271250 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f698014b5e0 con 0x7f69800b4530 2023-10-16T20:07:24.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.433+0000 7f6986d65640 1 -- 172.21.15.88:0/3059271250 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f698014bba0 con 0x7f69800b4530 2023-10-16T20:07:24.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.433+0000 7f6986d65640 1 -- 172.21.15.88:0/3059271250 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f698004e9c0 con 0x7f69800b4530 2023-10-16T20:07:24.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.433+0000 7f697d7fa640 1 -- 172.21.15.88:0/3059271250 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f69700b7040 con 0x7f69800b4530 2023-10-16T20:07:24.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.437+0000 7f697d7fa640 1 --2- 172.21.15.88:0/3059271250 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f696c0664d0 0x7f696c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:24.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.437+0000 7f697d7fa640 1 -- 172.21.15.88:0/3059271250 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5552+0+0 (secure 0 0 0) 0x7f69701226c0 con 0x7f69800b4530 2023-10-16T20:07:24.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.437+0000 7f697ffff640 1 --2- 172.21.15.88:0/3059271250 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f696c0664d0 0x7f696c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:24.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.437+0000 7f697ffff640 1 --2- 172.21.15.88:0/3059271250 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f696c0664d0 0x7f696c068990 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f698014c9b0 tx=0x7f6974001770 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:24.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.441+0000 7f697d7fa640 1 -- 172.21.15.88:0/3059271250 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f69700efe20 con 0x7f69800b4530 2023-10-16T20:07:24.594 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:24.589+0000 7f6986d65640 1 -- 172.21.15.88:0/3059271250 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "testcephfs1_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f69800ceec0 con 0x7f69800b4530 2023-10-16T20:07:25.261 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.257+0000 7f697d7fa640 1 -- 172.21.15.88:0/3059271250 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "testcephfs1_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'testcephfs1_data' created v56) v1 ==== 180+0+0 (secure 0 0 0) 0x7f69700b5070 con 0x7f69800b4530 2023-10-16T20:07:25.262 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'testcephfs1_data' created 2023-10-16T20:07:25.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.257+0000 7f6986d65640 1 -- 172.21.15.88:0/3059271250 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f696c0664d0 msgr2=0x7f696c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:25.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.257+0000 7f6986d65640 1 --2- 172.21.15.88:0/3059271250 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f696c0664d0 0x7f696c068990 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f698014c9b0 tx=0x7f6974001770 comp rx=0 tx=0).stop 2023-10-16T20:07:25.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.257+0000 7f6986d65640 1 -- 172.21.15.88:0/3059271250 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69800b4530 msgr2=0x7f69800cd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:25.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.257+0000 7f6986d65640 1 --2- 172.21.15.88:0/3059271250 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69800b4530 0x7f69800cd480 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f69700b3040 tx=0x7f697001ba10 comp rx=0 tx=0).stop 2023-10-16T20:07:25.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.261+0000 7f6986d65640 1 -- 172.21.15.88:0/3059271250 shutdown_connections 2023-10-16T20:07:25.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.261+0000 7f6986d65640 1 --2- 172.21.15.88:0/3059271250 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f696c0664d0 0x7f696c068990 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:25.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.261+0000 7f6986d65640 1 --2- 172.21.15.88:0/3059271250 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f69800b6a80 0x7f6980159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:25.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.261+0000 7f6986d65640 1 --2- 172.21.15.88:0/3059271250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f698015cb70 0x7f698015a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:25.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.261+0000 7f6986d65640 1 --2- 172.21.15.88:0/3059271250 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69800b4530 0x7f69800cd480 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:25.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.261+0000 7f6986d65640 1 -- 172.21.15.88:0/3059271250 >> 172.21.15.88:0/3059271250 conn(0x7f698006b6e0 msgr2=0x7f698015b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:25.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.261+0000 7f6986d65640 1 -- 172.21.15.88:0/3059271250 shutdown_connections 2023-10-16T20:07:25.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.261+0000 7f6986d65640 1 -- 172.21.15.88:0/3059271250 wait complete. 2023-10-16T20:07:25.307 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new testcephfs1 testcephfs1_metadata testcephfs1_data 2023-10-16T20:07:25.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.445+0000 7ff4c05a1640 1 Processor -- start 2023-10-16T20:07:25.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.445+0000 7ff4c05a1640 1 -- start start 2023-10-16T20:07:25.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.445+0000 7ff4c05a1640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4b81500a0 0x7ff4b8159f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:25.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.445+0000 7ff4c05a1640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b815aa10 con 0x7ff4b806c2e0 2023-10-16T20:07:25.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.445+0000 7ff4c05a1640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b806ca10 con 0x7ff4b80711f0 2023-10-16T20:07:25.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.445+0000 7ff4c05a1640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b8150480 con 0x7ff4b81500a0 2023-10-16T20:07:25.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.445+0000 7ff4bdb3c640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7ff4b806c2e0 0x7ff4b8070ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:46904/0 (socket says 172.21.15.88:46904) 2023-10-16T20:07:25.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.445+0000 7ff4bdb3c640 1 -- 172.21.15.88:0/1933212206 learned_addr learned my addr 172.21.15.88:0/1933212206 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:25.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.449+0000 7ff4bd33b640 1 -- 172.21.15.88:0/1933212206 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2449317557 0 0) 0x7ff4b815aa10 con 0x7ff4b806c2e0 2023-10-16T20:07:25.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.449+0000 7ff4be33d640 1 --2- 172.21.15.88:0/1933212206 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4b81500a0 0x7ff4b8159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:25.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.449+0000 7ff4bd33b640 1 -- 172.21.15.88:0/1933212206 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4a4003530 con 0x7ff4b806c2e0 2023-10-16T20:07:25.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.449+0000 7ff4bd33b640 1 -- 172.21.15.88:0/1933212206 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2613479040 0 0) 0x7ff4b806ca10 con 0x7ff4b80711f0 2023-10-16T20:07:25.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.449+0000 7ff4bd33b640 1 -- 172.21.15.88:0/1933212206 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4b815aa10 con 0x7ff4b80711f0 2023-10-16T20:07:25.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.449+0000 7ff4bd33b640 1 -- 172.21.15.88:0/1933212206 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3438180648 0 0) 0x7ff4a4003530 con 0x7ff4b806c2e0 2023-10-16T20:07:25.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.449+0000 7ff4bd33b640 1 -- 172.21.15.88:0/1933212206 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4b81500a0 msgr2=0x7ff4b8159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:25.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.449+0000 7ff4bd33b640 1 --2- 172.21.15.88:0/1933212206 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4b81500a0 0x7ff4b8159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:25.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.449+0000 7ff4bd33b640 1 -- 172.21.15.88:0/1933212206 >> v1:172.21.15.94:6789/0 conn(0x7ff4b80711f0 legacy=0x7ff4b815aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:25.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.449+0000 7ff4bd33b640 1 -- 172.21.15.88:0/1933212206 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b806ccf0 con 0x7ff4b806c2e0 2023-10-16T20:07:25.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.449+0000 7ff4bd33b640 1 -- 172.21.15.88:0/1933212206 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7ff4a8001680 con 0x7ff4b806c2e0 2023-10-16T20:07:25.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.449+0000 7ff4bd33b640 1 -- 172.21.15.88:0/1933212206 >> v1:172.21.15.88:6789/0 conn(0x7ff4b806c2e0 legacy=0x7ff4b8070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:25.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.449+0000 7ff4bd33b640 1 --2- 172.21.15.88:0/1933212206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4a4003b40 0x7ff4a4003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:25.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.449+0000 7ff4bd33b640 1 --2- 172.21.15.88:0/1933212206 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4b81500a0 0x7ff4a40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:25.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.449+0000 7ff4bd33b640 1 --2- 172.21.15.88:0/1933212206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4a4004d10 0x7ff4a4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:25.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.449+0000 7ff4bd33b640 1 -- 172.21.15.88:0/1933212206 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b8150480 con 0x7ff4a4003b40 2023-10-16T20:07:25.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.449+0000 7ff4bd33b640 1 -- 172.21.15.88:0/1933212206 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a400dc50 con 0x7ff4a4004d10 2023-10-16T20:07:25.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.449+0000 7ff4bd33b640 1 -- 172.21.15.88:0/1933212206 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a400de10 con 0x7ff4b81500a0 2023-10-16T20:07:25.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.449+0000 7ff4be33d640 1 --2- 172.21.15.88:0/1933212206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4a4003b40 0x7ff4a4003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:25.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.449+0000 7ff4beb3e640 1 --2- 172.21.15.88:0/1933212206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4a4004d10 0x7ff4a4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:25.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.449+0000 7ff4bdb3c640 1 --2- 172.21.15.88:0/1933212206 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4b81500a0 0x7ff4a40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:25.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.449+0000 7ff4be33d640 1 -- 172.21.15.88:0/1933212206 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4b81500a0 msgr2=0x7ff4a40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:25.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.449+0000 7ff4be33d640 1 --2- 172.21.15.88:0/1933212206 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4b81500a0 0x7ff4a40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:25.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.449+0000 7ff4be33d640 1 -- 172.21.15.88:0/1933212206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4a4004d10 msgr2=0x7ff4a4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:25.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.449+0000 7ff4be33d640 1 --2- 172.21.15.88:0/1933212206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4a4004d10 0x7ff4a4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:25.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.449+0000 7ff4be33d640 1 -- 172.21.15.88:0/1933212206 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a4003530 con 0x7ff4a4003b40 2023-10-16T20:07:25.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.453+0000 7ff4be33d640 1 --2- 172.21.15.88:0/1933212206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4a4003b40 0x7ff4a4003f40 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4009ae0 tx=0x7ff4b401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e80f4a60e895a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:25.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.453+0000 7ff4bd33b640 1 -- 172.21.15.88:0/1933212206 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b40a8410 con 0x7ff4a4003b40 2023-10-16T20:07:25.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.453+0000 7ff4bd33b640 1 -- 172.21.15.88:0/1933212206 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b40a8570 con 0x7ff4a4003b40 2023-10-16T20:07:25.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.453+0000 7ff4bd33b640 1 -- 172.21.15.88:0/1933212206 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b40a8920 con 0x7ff4a4003b40 2023-10-16T20:07:25.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.453+0000 7ff4c05a1640 1 -- 172.21.15.88:0/1933212206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4a4003b40 msgr2=0x7ff4a4003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:25.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.453+0000 7ff4c05a1640 1 --2- 172.21.15.88:0/1933212206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4a4003b40 0x7ff4a4003f40 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4009ae0 tx=0x7ff4b401ca50 comp rx=0 tx=0).stop 2023-10-16T20:07:25.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.453+0000 7ff4beb3e640 1 -- 172.21.15.88:0/1933212206 reap_dead start 2023-10-16T20:07:25.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.453+0000 7ff4c05a1640 1 -- 172.21.15.88:0/1933212206 shutdown_connections 2023-10-16T20:07:25.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.453+0000 7ff4c05a1640 1 -- 172.21.15.88:0/1933212206 >> 172.21.15.88:0/1933212206 conn(0x7ff4b80cb080 msgr2=0x7ff4b80cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:25.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.453+0000 7ff4c05a1640 1 -- 172.21.15.88:0/1933212206 shutdown_connections 2023-10-16T20:07:25.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.453+0000 7ff4c05a1640 1 -- 172.21.15.88:0/1933212206 wait complete. 2023-10-16T20:07:25.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.453+0000 7ff4c05a1640 1 Processor -- start 2023-10-16T20:07:25.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.453+0000 7ff4c05a1640 1 -- start start 2023-10-16T20:07:25.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.453+0000 7ff4c05a1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4a4003b40 0x7ff4b814c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:25.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.453+0000 7ff4c05a1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b8151a40 0x7ff4b80b2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:25.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.457+0000 7ff4c05a1640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4b80b2ea0 0x7ff4b8157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:25.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.457+0000 7ff4c05a1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b815a490 con 0x7ff4a4003b40 2023-10-16T20:07:25.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.457+0000 7ff4c05a1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b8151620 con 0x7ff4b8151a40 2023-10-16T20:07:25.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.457+0000 7ff4c05a1640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b81509a0 con 0x7ff4b80b2ea0 2023-10-16T20:07:25.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.457+0000 7ff4be33d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4a4003b40 0x7ff4b814c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:25.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.457+0000 7ff4be33d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4a4003b40 0x7ff4b814c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38488/0 (socket says 172.21.15.88:38488) 2023-10-16T20:07:25.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.457+0000 7ff4be33d640 1 -- 172.21.15.88:0/3214275568 learned_addr learned my addr 172.21.15.88:0/3214275568 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:25.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.457+0000 7ff4beb3e640 1 --2- 172.21.15.88:0/3214275568 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4b80b2ea0 0x7ff4b8157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:25.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.457+0000 7ff4bdb3c640 1 --2- 172.21.15.88:0/3214275568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b8151a40 0x7ff4b80b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:25.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.457+0000 7ff4be33d640 1 -- 172.21.15.88:0/3214275568 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4b80b2ea0 msgr2=0x7ff4b8157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:25.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.457+0000 7ff4be33d640 1 --2- 172.21.15.88:0/3214275568 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4b80b2ea0 0x7ff4b8157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:25.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.457+0000 7ff4be33d640 1 -- 172.21.15.88:0/3214275568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b8151a40 msgr2=0x7ff4b80b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:25.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.457+0000 7ff4be33d640 1 --2- 172.21.15.88:0/3214275568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b8151a40 0x7ff4b80b2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:25.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.457+0000 7ff4be33d640 1 -- 172.21.15.88:0/3214275568 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b8150c80 con 0x7ff4a4003b40 2023-10-16T20:07:25.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.457+0000 7ff4bdb3c640 1 --2- 172.21.15.88:0/3214275568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b8151a40 0x7ff4b80b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:07:25.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.457+0000 7ff4beb3e640 1 --2- 172.21.15.88:0/3214275568 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4b80b2ea0 0x7ff4b8157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:07:25.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.457+0000 7ff4be33d640 1 --2- 172.21.15.88:0/3214275568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4a4003b40 0x7ff4b814c9f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4009ae0 tx=0x7ff4b40197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:25.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.457+0000 7ff4aeffd640 1 -- 172.21.15.88:0/3214275568 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b40a8ce0 con 0x7ff4a4003b40 2023-10-16T20:07:25.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.457+0000 7ff4aeffd640 1 -- 172.21.15.88:0/3214275568 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b40a8e40 con 0x7ff4a4003b40 2023-10-16T20:07:25.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.457+0000 7ff4aeffd640 1 -- 172.21.15.88:0/3214275568 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4004260 con 0x7ff4a4003b40 2023-10-16T20:07:25.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.457+0000 7ff4c05a1640 1 -- 172.21.15.88:0/3214275568 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b8150f60 con 0x7ff4a4003b40 2023-10-16T20:07:25.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.457+0000 7ff4c05a1640 1 -- 172.21.15.88:0/3214275568 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b814cf30 con 0x7ff4a4003b40 2023-10-16T20:07:25.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.457+0000 7ff4c05a1640 1 -- 172.21.15.88:0/3214275568 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b804e9c0 con 0x7ff4a4003b40 2023-10-16T20:07:25.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.457+0000 7ff4aeffd640 1 -- 172.21.15.88:0/3214275568 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff4b40a6040 con 0x7ff4a4003b40 2023-10-16T20:07:25.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.461+0000 7ff4aeffd640 1 --2- 172.21.15.88:0/3214275568 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff49c0664d0 0x7ff49c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:25.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.461+0000 7ff4aeffd640 1 -- 172.21.15.88:0/3214275568 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5939+0+0 (secure 0 0 0) 0x7ff4b411e3b0 con 0x7ff4a4003b40 2023-10-16T20:07:25.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.461+0000 7ff4bdb3c640 1 --2- 172.21.15.88:0/3214275568 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff49c0664d0 0x7ff49c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:25.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.461+0000 7ff4bdb3c640 1 --2- 172.21.15.88:0/3214275568 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff49c0664d0 0x7ff49c068990 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8001210 tx=0x7ff4a8000ee0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:25.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.465+0000 7ff4aeffd640 1 -- 172.21.15.88:0/3214275568 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff4b40eb7b0 con 0x7ff4a4003b40 2023-10-16T20:07:25.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:25.645+0000 7ff4c05a1640 1 -- 172.21.15.88:0/3214275568 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "testcephfs1", "metadata": "testcephfs1_metadata", "data": "testcephfs1_data"} v 0) v1 -- 0x7ff4b806d430 con 0x7ff4a4003b40 2023-10-16T20:07:26.263 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.257+0000 7ff4aeffd640 1 -- 172.21.15.88:0/3214275568 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "testcephfs1", "metadata": "testcephfs1_metadata", "data": "testcephfs1_data"}]=0 new fs with metadata pool 8 and data pool 9 v24) v1 ==== 187+0+0 (secure 0 0 0) 0x7ff4b40c4e60 con 0x7ff4a4003b40 2023-10-16T20:07:26.263 INFO:teuthology.orchestra.run.smithi088.stderr:new fs with metadata pool 8 and data pool 9 2023-10-16T20:07:26.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.261+0000 7ff4c05a1640 1 -- 172.21.15.88:0/3214275568 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff49c0664d0 msgr2=0x7ff49c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:26.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.261+0000 7ff4c05a1640 1 --2- 172.21.15.88:0/3214275568 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff49c0664d0 0x7ff49c068990 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8001210 tx=0x7ff4a8000ee0 comp rx=0 tx=0).stop 2023-10-16T20:07:26.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.261+0000 7ff4c05a1640 1 -- 172.21.15.88:0/3214275568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4a4003b40 msgr2=0x7ff4b814c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:26.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.261+0000 7ff4c05a1640 1 --2- 172.21.15.88:0/3214275568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4a4003b40 0x7ff4b814c9f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4009ae0 tx=0x7ff4b40197f0 comp rx=0 tx=0).stop 2023-10-16T20:07:26.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.261+0000 7ff4c05a1640 1 -- 172.21.15.88:0/3214275568 shutdown_connections 2023-10-16T20:07:26.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.261+0000 7ff4c05a1640 1 --2- 172.21.15.88:0/3214275568 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff49c0664d0 0x7ff49c068990 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:26.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.261+0000 7ff4c05a1640 1 --2- 172.21.15.88:0/3214275568 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4b80b2ea0 0x7ff4b8157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:26.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.261+0000 7ff4c05a1640 1 --2- 172.21.15.88:0/3214275568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b8151a40 0x7ff4b80b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:26.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.261+0000 7ff4c05a1640 1 --2- 172.21.15.88:0/3214275568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4a4003b40 0x7ff4b814c9f0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:26.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.261+0000 7ff4c05a1640 1 -- 172.21.15.88:0/3214275568 >> 172.21.15.88:0/3214275568 conn(0x7ff4b80cb080 msgr2=0x7ff4b80cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:26.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.261+0000 7ff4c05a1640 1 -- 172.21.15.88:0/3214275568 shutdown_connections 2023-10-16T20:07:26.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.261+0000 7ff4c05a1640 1 -- 172.21.15.88:0/3214275568 wait complete. 2023-10-16T20:07:26.295 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:07:26.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.429+0000 7fea519d0640 1 Processor -- start 2023-10-16T20:07:26.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.429+0000 7fea519d0640 1 -- start start 2023-10-16T20:07:26.436 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.429+0000 7fea519d0640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fea4c0d22e0 0x7fea4c161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:26.436 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.429+0000 7fea519d0640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fea4c0d26c0 0x7fea4c15d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:26.436 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.429+0000 7fea519d0640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea4c0b87e0 con 0x7fea4c15d820 2023-10-16T20:07:26.436 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.429+0000 7fea519d0640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea4c15dc40 con 0x7fea4c0d26c0 2023-10-16T20:07:26.437 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.429+0000 7fea519d0640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c15df10 con 0x7fea4c0d22e0 2023-10-16T20:07:26.437 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.429+0000 7fea427fc640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fea4c0d26c0 0x7fea4c15d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:26.437 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.429+0000 7fea4b7fe640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fea4c15d820 0x7fea4c0cf2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:35928/0 (socket says 172.21.15.88:35928) 2023-10-16T20:07:26.437 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.429+0000 7fea4affd640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fea4c0d22e0 0x7fea4c161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:26.437 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.429+0000 7fea4b7fe640 1 -- 172.21.15.88:0/1125909159 learned_addr learned my addr 172.21.15.88:0/1125909159 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:26.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.433+0000 7fea4a7fc640 1 -- 172.21.15.88:0/1125909159 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2477193519 0 0) 0x7fea4c0b87e0 con 0x7fea4c15d820 2023-10-16T20:07:26.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.433+0000 7fea4a7fc640 1 -- 172.21.15.88:0/1125909159 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea38003530 con 0x7fea4c15d820 2023-10-16T20:07:26.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.433+0000 7fea427fc640 1 -- 172.21.15.88:0/1125909159 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fea4c0d22e0 msgr2=0x7fea4c161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:26.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.433+0000 7fea427fc640 1 --2- 172.21.15.88:0/1125909159 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fea4c0d22e0 0x7fea4c161450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:26.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.433+0000 7fea427fc640 1 -- 172.21.15.88:0/1125909159 >> v1:172.21.15.94:6789/0 conn(0x7fea4c15d820 legacy=0x7fea4c0cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:26.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.433+0000 7fea427fc640 1 -- 172.21.15.88:0/1125909159 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea4c15e1f0 con 0x7fea4c0d26c0 2023-10-16T20:07:26.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.433+0000 7fea4affd640 1 --2- 172.21.15.88:0/1125909159 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fea4c0d22e0 0x7fea4c161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:07:26.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.433+0000 7fea427fc640 1 --2- 172.21.15.88:0/1125909159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea4c0d26c0 0x7fea4c15d2e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fea34009d90 tx=0x7fea3401c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc4036794672b9e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:26.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.433+0000 7fea4a7fc640 1 -- 172.21.15.88:0/1125909159 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea340a5860 con 0x7fea4c0d26c0 2023-10-16T20:07:26.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.433+0000 7fea4a7fc640 1 -- 172.21.15.88:0/1125909159 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea340a59c0 con 0x7fea4c0d26c0 2023-10-16T20:07:26.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.433+0000 7fea4a7fc640 1 -- 172.21.15.88:0/1125909159 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea340a5d50 con 0x7fea4c0d26c0 2023-10-16T20:07:26.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.437+0000 7fea519d0640 1 -- 172.21.15.88:0/1125909159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea4c0d26c0 msgr2=0x7fea4c15d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:26.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.437+0000 7fea519d0640 1 --2- 172.21.15.88:0/1125909159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea4c0d26c0 0x7fea4c15d2e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fea34009d90 tx=0x7fea3401c890 comp rx=0 tx=0).stop 2023-10-16T20:07:26.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.437+0000 7fea519d0640 1 -- 172.21.15.88:0/1125909159 shutdown_connections 2023-10-16T20:07:26.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.437+0000 7fea519d0640 1 --2- 172.21.15.88:0/1125909159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea4c0d26c0 0x7fea4c15d2e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:26.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.437+0000 7fea519d0640 1 --2- 172.21.15.88:0/1125909159 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fea4c0d22e0 0x7fea4c161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:26.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.437+0000 7fea519d0640 1 -- 172.21.15.88:0/1125909159 >> 172.21.15.88:0/1125909159 conn(0x7fea4c0715c0 msgr2=0x7fea4c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:26.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.437+0000 7fea519d0640 1 -- 172.21.15.88:0/1125909159 shutdown_connections 2023-10-16T20:07:26.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.437+0000 7fea519d0640 1 -- 172.21.15.88:0/1125909159 wait complete. 2023-10-16T20:07:26.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.437+0000 7fea519d0640 1 Processor -- start 2023-10-16T20:07:26.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.437+0000 7fea519d0640 1 -- start start 2023-10-16T20:07:26.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.437+0000 7fea519d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea4c0d22e0 0x7fea4c1f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:26.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.437+0000 7fea519d0640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fea4c0d26c0 0x7fea4c14fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:26.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.437+0000 7fea519d0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea4c15d820 0x7fea4c151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:26.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.437+0000 7fea519d0640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c152a80 con 0x7fea4c15d820 2023-10-16T20:07:26.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.437+0000 7fea519d0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c151e70 con 0x7fea4c0d22e0 2023-10-16T20:07:26.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.437+0000 7fea519d0640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c152150 con 0x7fea4c0d26c0 2023-10-16T20:07:26.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.437+0000 7fea4b7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea4c15d820 0x7fea4c151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:26.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.437+0000 7fea4affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea4c0d22e0 0x7fea4c1f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:26.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.437+0000 7fea4affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea4c0d22e0 0x7fea4c1f4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:58256/0 (socket says 172.21.15.88:58256) 2023-10-16T20:07:26.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.437+0000 7fea4b7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea4c15d820 0x7fea4c151620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38502/0 (socket says 172.21.15.88:38502) 2023-10-16T20:07:26.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.437+0000 7fea4affd640 1 -- 172.21.15.88:0/1152931050 learned_addr learned my addr 172.21.15.88:0/1152931050 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:26.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.437+0000 7fea427fc640 1 --2- 172.21.15.88:0/1152931050 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fea4c0d26c0 0x7fea4c14fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:26.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.437+0000 7fea4affd640 1 -- 172.21.15.88:0/1152931050 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fea4c0d26c0 msgr2=0x7fea4c14fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:26.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.437+0000 7fea4affd640 1 --2- 172.21.15.88:0/1152931050 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fea4c0d26c0 0x7fea4c14fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:26.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.437+0000 7fea4affd640 1 -- 172.21.15.88:0/1152931050 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea4c15d820 msgr2=0x7fea4c151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:26.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.437+0000 7fea4affd640 1 --2- 172.21.15.88:0/1152931050 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea4c15d820 0x7fea4c151620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:26.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.437+0000 7fea4affd640 1 -- 172.21.15.88:0/1152931050 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3c009420 con 0x7fea4c0d22e0 2023-10-16T20:07:26.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.441+0000 7fea4affd640 1 --2- 172.21.15.88:0/1152931050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea4c0d22e0 0x7fea4c1f4e10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00ebe0 tx=0x7fea3c00c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:26.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.441+0000 7fea48ff9640 1 -- 172.21.15.88:0/1152931050 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c00eee0 con 0x7fea4c0d22e0 2023-10-16T20:07:26.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.441+0000 7fea48ff9640 1 -- 172.21.15.88:0/1152931050 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3c00cc30 con 0x7fea4c0d22e0 2023-10-16T20:07:26.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.441+0000 7fea48ff9640 1 -- 172.21.15.88:0/1152931050 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c013470 con 0x7fea4c0d22e0 2023-10-16T20:07:26.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.441+0000 7fea519d0640 1 -- 172.21.15.88:0/1152931050 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea34009a70 con 0x7fea4c0d22e0 2023-10-16T20:07:26.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.441+0000 7fea519d0640 1 -- 172.21.15.88:0/1152931050 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4c152710 con 0x7fea4c0d22e0 2023-10-16T20:07:26.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.441+0000 7fea519d0640 1 -- 172.21.15.88:0/1152931050 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea4c04e9c0 con 0x7fea4c0d22e0 2023-10-16T20:07:26.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.441+0000 7fea48ff9640 1 -- 172.21.15.88:0/1152931050 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fea3c0135d0 con 0x7fea4c0d22e0 2023-10-16T20:07:26.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.441+0000 7fea48ff9640 1 --2- 172.21.15.88:0/1152931050 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fea2c0664a0 0x7fea2c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:26.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.441+0000 7fea48ff9640 1 -- 172.21.15.88:0/1152931050 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fea3c084240 con 0x7fea4c0d22e0 2023-10-16T20:07:26.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.441+0000 7fea427fc640 1 --2- 172.21.15.88:0/1152931050 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fea2c0664a0 0x7fea2c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:26.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.445+0000 7fea427fc640 1 --2- 172.21.15.88:0/1152931050 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fea2c0664a0 0x7fea2c068960 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fea340af000 tx=0x7fea340a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:26.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.449+0000 7fea48ff9640 1 -- 172.21.15.88:0/1152931050 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fea3c051600 con 0x7fea4c0d22e0 2023-10-16T20:07:26.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.601+0000 7fea519d0640 1 -- 172.21.15.88:0/1152931050 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea4c0d0840 con 0x7fea4c0d22e0 2023-10-16T20:07:26.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.601+0000 7fea48ff9640 1 -- 172.21.15.88:0/1152931050 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16731 (secure 0 0 0) 0x7fea3c054c20 con 0x7fea4c0d22e0 2023-10-16T20:07:26.606 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:07:26.606 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":57,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:07:25.650296+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T20:07:24.001121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"testcephfs1_data","create_time":"2023-10-16T20:07:24.636209+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:07:26.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.601+0000 7fea519d0640 1 -- 172.21.15.88:0/1152931050 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fea2c0664a0 msgr2=0x7fea2c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:26.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.601+0000 7fea519d0640 1 --2- 172.21.15.88:0/1152931050 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fea2c0664a0 0x7fea2c068960 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fea340af000 tx=0x7fea340a7070 comp rx=0 tx=0).stop 2023-10-16T20:07:26.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.601+0000 7fea519d0640 1 -- 172.21.15.88:0/1152931050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea4c0d22e0 msgr2=0x7fea4c1f4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:26.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.601+0000 7fea519d0640 1 --2- 172.21.15.88:0/1152931050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea4c0d22e0 0x7fea4c1f4e10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00ebe0 tx=0x7fea3c00c520 comp rx=0 tx=0).stop 2023-10-16T20:07:26.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.605+0000 7fea519d0640 1 -- 172.21.15.88:0/1152931050 shutdown_connections 2023-10-16T20:07:26.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.605+0000 7fea519d0640 1 --2- 172.21.15.88:0/1152931050 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fea2c0664a0 0x7fea2c068960 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:26.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.605+0000 7fea519d0640 1 --2- 172.21.15.88:0/1152931050 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fea4c0d26c0 0x7fea4c14fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:26.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.605+0000 7fea519d0640 1 --2- 172.21.15.88:0/1152931050 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea4c15d820 0x7fea4c151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:26.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.605+0000 7fea519d0640 1 --2- 172.21.15.88:0/1152931050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea4c0d22e0 0x7fea4c1f4e10 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:26.610 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.605+0000 7fea519d0640 1 -- 172.21.15.88:0/1152931050 >> 172.21.15.88:0/1152931050 conn(0x7fea4c0715c0 msgr2=0x7fea4c15af00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:26.610 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.605+0000 7fea519d0640 1 -- 172.21.15.88:0/1152931050 shutdown_connections 2023-10-16T20:07:26.610 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.605+0000 7fea519d0640 1 -- 172.21.15.88:0/1152931050 wait complete. 2023-10-16T20:07:26.627 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:07:26.657 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T20:07:26.680 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T20:07:26.769 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.765+0000 7f481aa6b640 1 Processor -- start 2023-10-16T20:07:26.770 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.765+0000 7f481aa6b640 1 -- start start 2023-10-16T20:07:26.771 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.765+0000 7f481aa6b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f48140ada40 0x7f48140b1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:26.771 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.765+0000 7f481aa6b640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48140b2570 con 0x7f4814143eb0 2023-10-16T20:07:26.771 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.765+0000 7f481aa6b640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48140cca50 con 0x7f48140ade20 2023-10-16T20:07:26.771 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.765+0000 7f481aa6b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4814144340 con 0x7f48140ada40 2023-10-16T20:07:26.771 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.765+0000 7f4819008640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f4814143eb0 0x7f48140b0500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:46916/0 (socket says 172.21.15.88:46916) 2023-10-16T20:07:26.772 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.765+0000 7f4819008640 1 -- 172.21.15.88:0/936513518 learned_addr learned my addr 172.21.15.88:0/936513518 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:26.772 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.765+0000 7f4818807640 1 --2- 172.21.15.88:0/936513518 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f48140ada40 0x7f48140b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:26.773 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.765+0000 7f480b7fe640 1 -- 172.21.15.88:0/936513518 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3478945599 0 0) 0x7f48140b2570 con 0x7f4814143eb0 2023-10-16T20:07:26.773 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.765+0000 7f480b7fe640 1 -- 172.21.15.88:0/936513518 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47fc003530 con 0x7f4814143eb0 2023-10-16T20:07:26.773 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.769+0000 7f480b7fe640 1 -- 172.21.15.88:0/936513518 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2763235063 0 0) 0x7f48140cca50 con 0x7f48140ade20 2023-10-16T20:07:26.773 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.769+0000 7f480b7fe640 1 -- 172.21.15.88:0/936513518 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48140b2570 con 0x7f48140ade20 2023-10-16T20:07:26.774 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.769+0000 7f480b7fe640 1 -- 172.21.15.88:0/936513518 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2062940115 0 0) 0x7f47fc003530 con 0x7f4814143eb0 2023-10-16T20:07:26.774 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.769+0000 7f4818807640 1 -- 172.21.15.88:0/936513518 >> v1:172.21.15.94:6789/0 conn(0x7f48140ade20 legacy=0x7f481415adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:26.774 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.769+0000 7f4818807640 1 -- 172.21.15.88:0/936513518 >> v1:172.21.15.88:6789/0 conn(0x7f4814143eb0 legacy=0x7f48140b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:26.774 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.769+0000 7f4818807640 1 -- 172.21.15.88:0/936513518 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48140b1fb0 con 0x7f48140ada40 2023-10-16T20:07:26.775 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.769+0000 7f4818807640 1 --2- 172.21.15.88:0/936513518 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f48140ada40 0x7f48140b1920 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f48100097f0 tx=0x7f481001c320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80b08a5109d1bdf6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:26.776 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.769+0000 7f480b7fe640 1 -- 172.21.15.88:0/936513518 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c001a00 con 0x7f48140ada40 2023-10-16T20:07:26.776 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.769+0000 7f480b7fe640 1 -- 172.21.15.88:0/936513518 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48100a7050 con 0x7f48140ada40 2023-10-16T20:07:26.776 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.769+0000 7f480b7fe640 1 -- 172.21.15.88:0/936513518 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48100ab620 con 0x7f48140ada40 2023-10-16T20:07:26.776 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.769+0000 7f481aa6b640 1 -- 172.21.15.88:0/936513518 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f48140ada40 msgr2=0x7f48140b1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:26.776 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.769+0000 7f481aa6b640 1 --2- 172.21.15.88:0/936513518 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f48140ada40 0x7f48140b1920 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f48100097f0 tx=0x7f481001c320 comp rx=0 tx=0).stop 2023-10-16T20:07:26.777 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.769+0000 7f481aa6b640 1 -- 172.21.15.88:0/936513518 shutdown_connections 2023-10-16T20:07:26.777 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.769+0000 7f481aa6b640 1 --2- 172.21.15.88:0/936513518 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f48140ada40 0x7f48140b1920 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:26.777 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.769+0000 7f481aa6b640 1 -- 172.21.15.88:0/936513518 >> 172.21.15.88:0/936513518 conn(0x7f48140ce5e0 msgr2=0x7f48140ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:26.777 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.773+0000 7f481aa6b640 1 -- 172.21.15.88:0/936513518 shutdown_connections 2023-10-16T20:07:26.777 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.773+0000 7f481aa6b640 1 -- 172.21.15.88:0/936513518 wait complete. 2023-10-16T20:07:26.779 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.773+0000 7f481aa6b640 1 Processor -- start 2023-10-16T20:07:26.779 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.773+0000 7f481aa6b640 1 -- start start 2023-10-16T20:07:26.779 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.773+0000 7f481aa6b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48140ada40 0x7f4814155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:26.780 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.773+0000 7f481aa6b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48140ade20 0x7f481415a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:26.780 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.773+0000 7f481aa6b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4814143eb0 0x7f481413adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:26.780 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.773+0000 7f481aa6b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48140ae520 con 0x7f48140ada40 2023-10-16T20:07:26.780 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.773+0000 7f481aa6b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48141395e0 con 0x7f48140ade20 2023-10-16T20:07:26.780 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.773+0000 7f481aa6b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48141398c0 con 0x7f4814143eb0 2023-10-16T20:07:26.781 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.773+0000 7f4818807640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48140ada40 0x7f4814155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:26.781 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.773+0000 7f4818807640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48140ada40 0x7f4814155cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38518/0 (socket says 172.21.15.88:38518) 2023-10-16T20:07:26.781 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.773+0000 7f4818807640 1 -- 172.21.15.88:0/1261449918 learned_addr learned my addr 172.21.15.88:0/1261449918 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:26.781 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.773+0000 7f480bfff640 1 --2- 172.21.15.88:0/1261449918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48140ade20 0x7f481415a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:26.781 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.773+0000 7f4819008640 1 --2- 172.21.15.88:0/1261449918 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4814143eb0 0x7f481413adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:26.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.773+0000 7f4818807640 1 -- 172.21.15.88:0/1261449918 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4814143eb0 msgr2=0x7f481413adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:26.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.773+0000 7f4818807640 1 --2- 172.21.15.88:0/1261449918 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4814143eb0 0x7f481413adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:26.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.773+0000 7f4818807640 1 -- 172.21.15.88:0/1261449918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48140ade20 msgr2=0x7f481415a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:26.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.773+0000 7f4818807640 1 --2- 172.21.15.88:0/1261449918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48140ade20 0x7f481415a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:26.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.773+0000 7f4818807640 1 -- 172.21.15.88:0/1261449918 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4810009420 con 0x7f48140ada40 2023-10-16T20:07:26.783 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.773+0000 7f4818807640 1 --2- 172.21.15.88:0/1261449918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48140ada40 0x7f4814155cf0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f48100ae040 tx=0x7f481001ab60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:26.784 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.777+0000 7f48097fa640 1 -- 172.21.15.88:0/1261449918 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48100abcd0 con 0x7f48140ada40 2023-10-16T20:07:26.784 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.777+0000 7f48097fa640 1 -- 172.21.15.88:0/1261449918 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48100abe50 con 0x7f48140ada40 2023-10-16T20:07:26.784 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.777+0000 7f48097fa640 1 -- 172.21.15.88:0/1261449918 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4810004260 con 0x7f48140ada40 2023-10-16T20:07:26.784 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.777+0000 7f481aa6b640 1 -- 172.21.15.88:0/1261449918 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48140ae7c0 con 0x7f48140ada40 2023-10-16T20:07:26.785 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.777+0000 7f481aa6b640 1 -- 172.21.15.88:0/1261449918 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4814139ba0 con 0x7f48140ada40 2023-10-16T20:07:26.785 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.777+0000 7f481aa6b640 1 -- 172.21.15.88:0/1261449918 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f481404e9c0 con 0x7f48140ada40 2023-10-16T20:07:26.785 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.777+0000 7f48097fa640 1 -- 172.21.15.88:0/1261449918 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f48100043c0 con 0x7f48140ada40 2023-10-16T20:07:26.785 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.777+0000 7f48097fa640 1 --2- 172.21.15.88:0/1261449918 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f48000664d0 0x7f4800068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:26.785 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.777+0000 7f48097fa640 1 -- 172.21.15.88:0/1261449918 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f481011e570 con 0x7f48140ada40 2023-10-16T20:07:26.786 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.777+0000 7f480bfff640 1 --2- 172.21.15.88:0/1261449918 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f48000664d0 0x7f4800068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:26.786 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.781+0000 7f480bfff640 1 --2- 172.21.15.88:0/1261449918 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f48000664d0 0x7f4800068990 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f481413bb10 tx=0x7f4804009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:26.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.781+0000 7f48097fa640 1 -- 172.21.15.88:0/1261449918 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f48100eb880 con 0x7f48140ada40 2023-10-16T20:07:26.938 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.933+0000 7f481aa6b640 1 -- 172.21.15.88:0/1261449918 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f481415b530 con 0x7f48140ada40 2023-10-16T20:07:26.940 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.937+0000 7f48097fa640 1 -- 172.21.15.88:0/1261449918 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16731 (secure 0 0 0) 0x7f48100eeea0 con 0x7f48140ada40 2023-10-16T20:07:26.940 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:07:26.940 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":57,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:07:25.650296+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T20:07:24.001121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"testcephfs1_data","create_time":"2023-10-16T20:07:24.636209+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:07:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.937+0000 7f481aa6b640 1 -- 172.21.15.88:0/1261449918 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f48000664d0 msgr2=0x7f4800068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.937+0000 7f481aa6b640 1 --2- 172.21.15.88:0/1261449918 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f48000664d0 0x7f4800068990 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f481413bb10 tx=0x7f4804009040 comp rx=0 tx=0).stop 2023-10-16T20:07:26.943 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.937+0000 7f481aa6b640 1 -- 172.21.15.88:0/1261449918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48140ada40 msgr2=0x7f4814155cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:26.943 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.937+0000 7f481aa6b640 1 --2- 172.21.15.88:0/1261449918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48140ada40 0x7f4814155cf0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f48100ae040 tx=0x7f481001ab60 comp rx=0 tx=0).stop 2023-10-16T20:07:26.943 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.937+0000 7f481aa6b640 1 -- 172.21.15.88:0/1261449918 shutdown_connections 2023-10-16T20:07:26.943 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.937+0000 7f481aa6b640 1 --2- 172.21.15.88:0/1261449918 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f48000664d0 0x7f4800068990 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:26.943 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.937+0000 7f481aa6b640 1 --2- 172.21.15.88:0/1261449918 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4814143eb0 0x7f481413adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:26.944 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.937+0000 7f481aa6b640 1 --2- 172.21.15.88:0/1261449918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48140ade20 0x7f481415a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:26.944 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.937+0000 7f481aa6b640 1 --2- 172.21.15.88:0/1261449918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48140ada40 0x7f4814155cf0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:26.944 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.937+0000 7f481aa6b640 1 -- 172.21.15.88:0/1261449918 >> 172.21.15.88:0/1261449918 conn(0x7f48140ce5e0 msgr2=0x7f481413df70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:26.944 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.937+0000 7f481aa6b640 1 -- 172.21.15.88:0/1261449918 shutdown_connections 2023-10-16T20:07:26.944 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:26.937+0000 7f481aa6b640 1 -- 172.21.15.88:0/1261449918 wait complete. 2023-10-16T20:07:26.961 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set testcephfs1 standby_count_wanted 0 2023-10-16T20:07:27.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.137+0000 7fa6f4fe4640 1 Processor -- start 2023-10-16T20:07:27.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.137+0000 7fa6f4fe4640 1 -- start start 2023-10-16T20:07:27.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.137+0000 7fa6f4fe4640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fa6f015cb50 0x7fa6f00cd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:27.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.137+0000 7fa6f4fe4640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa6f00cd9c0 0x7fa6f00b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:27.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.137+0000 7fa6f4fe4640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f006b0a0 con 0x7fa6f015dde0 2023-10-16T20:07:27.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.137+0000 7fa6f4fe4640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6f00d0390 con 0x7fa6f015cb50 2023-10-16T20:07:27.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.137+0000 7fa6f4fe4640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f015cf30 con 0x7fa6f00cd9c0 2023-10-16T20:07:27.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.137+0000 7fa6ef7fe640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fa6f015cb50 0x7fa6f00cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:27.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.137+0000 7fa6effff640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fa6f015dde0 0x7fa6f006c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:46930/0 (socket says 172.21.15.88:46930) 2023-10-16T20:07:27.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.137+0000 7fa6effff640 1 -- 172.21.15.88:0/1170421804 learned_addr learned my addr 172.21.15.88:0/1170421804 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:27.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.141+0000 7fa6eeffd640 1 --2- 172.21.15.88:0/1170421804 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa6f00cd9c0 0x7fa6f00b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:27.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.141+0000 7fa6ef7fe640 1 -- 172.21.15.88:0/1170421804 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa6f00cd9c0 msgr2=0x7fa6f00b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:27.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.141+0000 7fa6ef7fe640 1 --2- 172.21.15.88:0/1170421804 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa6f00cd9c0 0x7fa6f00b42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:27.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.141+0000 7fa6ef7fe640 1 -- 172.21.15.88:0/1170421804 >> v1:172.21.15.88:6789/0 conn(0x7fa6f015dde0 legacy=0x7fa6f006c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:27.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.141+0000 7fa6ef7fe640 1 -- 172.21.15.88:0/1170421804 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f00cde80 con 0x7fa6f015cb50 2023-10-16T20:07:27.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.141+0000 7fa6ee7fc640 1 -- 172.21.15.88:0/1170421804 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3112414650 0 0) 0x7fa6f006b0a0 con 0x7fa6f015dde0 2023-10-16T20:07:27.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.141+0000 7fa6ef7fe640 1 --2- 172.21.15.88:0/1170421804 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6f015cb50 0x7fa6f00cd480 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fa6e00097a0 tx=0x7fa6e001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57bc23ffa2cfc898 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:27.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.141+0000 7fa6ee7fc640 1 -- 172.21.15.88:0/1170421804 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e00ab600 con 0x7fa6f015cb50 2023-10-16T20:07:27.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.141+0000 7fa6ee7fc640 1 -- 172.21.15.88:0/1170421804 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e00a7050 con 0x7fa6f015cb50 2023-10-16T20:07:27.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.141+0000 7fa6ee7fc640 1 -- 172.21.15.88:0/1170421804 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e00ab980 con 0x7fa6f015cb50 2023-10-16T20:07:27.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.145+0000 7fa6f4fe4640 1 -- 172.21.15.88:0/1170421804 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6f015cb50 msgr2=0x7fa6f00cd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:27.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.145+0000 7fa6f4fe4640 1 --2- 172.21.15.88:0/1170421804 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6f015cb50 0x7fa6f00cd480 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fa6e00097a0 tx=0x7fa6e001c430 comp rx=0 tx=0).stop 2023-10-16T20:07:27.149 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.145+0000 7fa6f4fe4640 1 -- 172.21.15.88:0/1170421804 shutdown_connections 2023-10-16T20:07:27.149 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.145+0000 7fa6f4fe4640 1 --2- 172.21.15.88:0/1170421804 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa6f00cd9c0 0x7fa6f00b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:27.149 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.145+0000 7fa6f4fe4640 1 --2- 172.21.15.88:0/1170421804 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6f015cb50 0x7fa6f00cd480 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:27.149 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.145+0000 7fa6f4fe4640 1 -- 172.21.15.88:0/1170421804 >> 172.21.15.88:0/1170421804 conn(0x7fa6f00d2850 msgr2=0x7fa6f00d2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:27.149 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.145+0000 7fa6f4fe4640 1 -- 172.21.15.88:0/1170421804 shutdown_connections 2023-10-16T20:07:27.150 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.145+0000 7fa6f4fe4640 1 -- 172.21.15.88:0/1170421804 wait complete. 2023-10-16T20:07:27.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.145+0000 7fa6f4fe4640 1 Processor -- start 2023-10-16T20:07:27.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.145+0000 7fa6f4fe4640 1 -- start start 2023-10-16T20:07:27.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.145+0000 7fa6f4fe4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6f00cd9c0 0x7fa6f00b8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:27.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.145+0000 7fa6f4fe4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6f015cb50 0x7fa6f0157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:27.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.145+0000 7fa6f4fe4640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa6f015dde0 0x7fa6f015f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:27.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.145+0000 7fa6f4fe4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f0160eb0 con 0x7fa6f00cd9c0 2023-10-16T20:07:27.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.145+0000 7fa6f4fe4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f015fe10 con 0x7fa6f015cb50 2023-10-16T20:07:27.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.145+0000 7fa6f4fe4640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f01600f0 con 0x7fa6f015dde0 2023-10-16T20:07:27.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.145+0000 7fa6ef7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6f00cd9c0 0x7fa6f00b8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:27.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.145+0000 7fa6ef7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6f00cd9c0 0x7fa6f00b8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38536/0 (socket says 172.21.15.88:38536) 2023-10-16T20:07:27.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.145+0000 7fa6ef7fe640 1 -- 172.21.15.88:0/1727224086 learned_addr learned my addr 172.21.15.88:0/1727224086 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:27.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.145+0000 7fa6ef7fe640 1 -- 172.21.15.88:0/1727224086 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa6f015dde0 msgr2=0x7fa6f015f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:27.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.145+0000 7fa6ef7fe640 1 --2- 172.21.15.88:0/1727224086 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa6f015dde0 0x7fa6f015f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:27.155 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.145+0000 7fa6eeffd640 1 --2- 172.21.15.88:0/1727224086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6f015cb50 0x7fa6f0157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:27.155 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.145+0000 7fa6ef7fe640 1 -- 172.21.15.88:0/1727224086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6f015cb50 msgr2=0x7fa6f0157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:27.155 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.145+0000 7fa6ef7fe640 1 --2- 172.21.15.88:0/1727224086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6f015cb50 0x7fa6f0157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:27.155 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.145+0000 7fa6ef7fe640 1 -- 172.21.15.88:0/1727224086 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e00093b0 con 0x7fa6f00cd9c0 2023-10-16T20:07:27.155 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.149+0000 7fa6eeffd640 1 --2- 172.21.15.88:0/1727224086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6f015cb50 0x7fa6f0157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:07:27.156 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.149+0000 7fa6ef7fe640 1 --2- 172.21.15.88:0/1727224086 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6f00cd9c0 0x7fa6f00b8270 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fa6e001a060 tx=0x7fa6e00191d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:27.156 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.149+0000 7fa6d7fff640 1 -- 172.21.15.88:0/1727224086 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0005ba0 con 0x7fa6f00cd9c0 2023-10-16T20:07:27.156 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.149+0000 7fa6d7fff640 1 -- 172.21.15.88:0/1727224086 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e0005d00 con 0x7fa6f00cd9c0 2023-10-16T20:07:27.156 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.149+0000 7fa6d7fff640 1 -- 172.21.15.88:0/1727224086 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0006260 con 0x7fa6f00cd9c0 2023-10-16T20:07:27.156 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.149+0000 7fa6f4fe4640 1 -- 172.21.15.88:0/1727224086 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f01603d0 con 0x7fa6f00cd9c0 2023-10-16T20:07:27.156 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.149+0000 7fa6f4fe4640 1 -- 172.21.15.88:0/1727224086 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f0160990 con 0x7fa6f00cd9c0 2023-10-16T20:07:27.157 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.149+0000 7fa6f4fe4640 1 -- 172.21.15.88:0/1727224086 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f0160b80 con 0x7fa6f00cd9c0 2023-10-16T20:07:27.157 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.149+0000 7fa6d7fff640 1 -- 172.21.15.88:0/1727224086 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa6e00b2040 con 0x7fa6f00cd9c0 2023-10-16T20:07:27.157 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.149+0000 7fa6d7fff640 1 --2- 172.21.15.88:0/1727224086 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa6dc0664d0 0x7fa6dc068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:27.158 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.149+0000 7fa6d7fff640 1 -- 172.21.15.88:0/1727224086 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fa6e011d640 con 0x7fa6f00cd9c0 2023-10-16T20:07:27.158 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.149+0000 7fa6eeffd640 1 --2- 172.21.15.88:0/1727224086 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa6dc0664d0 0x7fa6dc068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:27.158 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.153+0000 7fa6eeffd640 1 --2- 172.21.15.88:0/1727224086 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa6dc0664d0 0x7fa6dc068990 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa6f00b2620 tx=0x7fa6e4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:27.159 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.153+0000 7fa6d7fff640 1 -- 172.21.15.88:0/1727224086 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa6e00ea8c0 con 0x7fa6f00cd9c0 2023-10-16T20:07:27.342 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:27.337+0000 7fa6f4fe4640 1 -- 172.21.15.88:0/1727224086 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "testcephfs1", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa6f00674f0 con 0x7fa6f00cd9c0 2023-10-16T20:07:28.283 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.277+0000 7fa6d7fff640 1 -- 172.21.15.88:0/1727224086 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "testcephfs1", "var": "standby_count_wanted", "val": "0"}]=0 v27) v1 ==== 123+0+0 (secure 0 0 0) 0x7fa6e00edee0 con 0x7fa6f00cd9c0 2023-10-16T20:07:28.286 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.281+0000 7fa6f4fe4640 1 -- 172.21.15.88:0/1727224086 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa6dc0664d0 msgr2=0x7fa6dc068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:28.286 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.281+0000 7fa6f4fe4640 1 --2- 172.21.15.88:0/1727224086 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa6dc0664d0 0x7fa6dc068990 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa6f00b2620 tx=0x7fa6e4008040 comp rx=0 tx=0).stop 2023-10-16T20:07:28.286 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.281+0000 7fa6f4fe4640 1 -- 172.21.15.88:0/1727224086 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6f00cd9c0 msgr2=0x7fa6f00b8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:28.287 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.281+0000 7fa6f4fe4640 1 --2- 172.21.15.88:0/1727224086 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6f00cd9c0 0x7fa6f00b8270 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fa6e001a060 tx=0x7fa6e00191d0 comp rx=0 tx=0).stop 2023-10-16T20:07:28.287 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.281+0000 7fa6f4fe4640 1 -- 172.21.15.88:0/1727224086 shutdown_connections 2023-10-16T20:07:28.287 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.281+0000 7fa6f4fe4640 1 --2- 172.21.15.88:0/1727224086 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa6dc0664d0 0x7fa6dc068990 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:28.287 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.281+0000 7fa6f4fe4640 1 --2- 172.21.15.88:0/1727224086 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa6f015dde0 0x7fa6f015f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:28.287 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.281+0000 7fa6f4fe4640 1 --2- 172.21.15.88:0/1727224086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6f015cb50 0x7fa6f0157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:28.288 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.281+0000 7fa6f4fe4640 1 --2- 172.21.15.88:0/1727224086 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6f00cd9c0 0x7fa6f00b8270 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:28.288 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.281+0000 7fa6f4fe4640 1 -- 172.21.15.88:0/1727224086 >> 172.21.15.88:0/1727224086 conn(0x7fa6f00d2850 msgr2=0x7fa6f015bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:28.288 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.281+0000 7fa6f4fe4640 1 -- 172.21.15.88:0/1727224086 shutdown_connections 2023-10-16T20:07:28.288 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.281+0000 7fa6f4fe4640 1 -- 172.21.15.88:0/1727224086 wait complete. 2023-10-16T20:07:28.309 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T20:07:28.309 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set testcephfs1 allow_standby_replay false 2023-10-16T20:07:28.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.449+0000 7f071ca05640 1 Processor -- start 2023-10-16T20:07:28.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.449+0000 7f071ca05640 1 -- start start 2023-10-16T20:07:28.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.449+0000 7f071ca05640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f07181595d0 0x7f07180c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:28.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.449+0000 7f071ca05640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f071806dcd0 0x7f07180c9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:28.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.449+0000 7f071ca05640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f071806c7c0 0x7f071815d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:28.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.449+0000 7f071ca05640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07180b1630 con 0x7f071806c7c0 2023-10-16T20:07:28.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.449+0000 7f071ca05640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f071806ccc0 con 0x7f07181595d0 2023-10-16T20:07:28.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.449+0000 7f071ca05640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f071806cfa0 con 0x7f071806dcd0 2023-10-16T20:07:28.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.449+0000 7f0716d9d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f071806c7c0 0x7f071815d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:28.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.449+0000 7f0716d9d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f071806c7c0 0x7f071815d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38540/0 (socket says 172.21.15.88:38540) 2023-10-16T20:07:28.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.449+0000 7f0716d9d640 1 -- 172.21.15.88:0/1131717845 learned_addr learned my addr 172.21.15.88:0/1131717845 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:28.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.449+0000 7f071659c640 1 --2- 172.21.15.88:0/1131717845 >> v2:172.21.15.94:3300/0 conn(0x7f07181595d0 0x7f07180c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:28.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.449+0000 7f070dd9b640 1 --2- 172.21.15.88:0/1131717845 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f071806dcd0 0x7f07180c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:28.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.449+0000 7f0716d9d640 1 -- 172.21.15.88:0/1131717845 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f071806dcd0 msgr2=0x7f07180c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:28.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.449+0000 7f0716d9d640 1 --2- 172.21.15.88:0/1131717845 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f071806dcd0 0x7f07180c9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:28.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.449+0000 7f0716d9d640 1 -- 172.21.15.88:0/1131717845 >> v2:172.21.15.94:3300/0 conn(0x7f07181595d0 msgr2=0x7f07180c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:28.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.449+0000 7f0716d9d640 1 --2- 172.21.15.88:0/1131717845 >> v2:172.21.15.94:3300/0 conn(0x7f07181595d0 0x7f07180c8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:28.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.449+0000 7f0716d9d640 1 -- 172.21.15.88:0/1131717845 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071806d280 con 0x7f071806c7c0 2023-10-16T20:07:28.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.449+0000 7f070dd9b640 1 --2- 172.21.15.88:0/1131717845 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f071806dcd0 0x7f07180c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:07:28.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.453+0000 7f0716d9d640 1 --2- 172.21.15.88:0/1131717845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f071806c7c0 0x7f071815d370 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f071806e660 tx=0x7f071001c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4fd05a06b07dacc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:28.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.453+0000 7f0715d9b640 1 -- 172.21.15.88:0/1131717845 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07100a5870 con 0x7f071806c7c0 2023-10-16T20:07:28.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.453+0000 7f0715d9b640 1 -- 172.21.15.88:0/1131717845 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07100a59d0 con 0x7f071806c7c0 2023-10-16T20:07:28.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.453+0000 7f0715d9b640 1 -- 172.21.15.88:0/1131717845 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07100a5d60 con 0x7f071806c7c0 2023-10-16T20:07:28.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.453+0000 7f071ca05640 1 -- 172.21.15.88:0/1131717845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f071806c7c0 msgr2=0x7f071815d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:28.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.453+0000 7f071ca05640 1 --2- 172.21.15.88:0/1131717845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f071806c7c0 0x7f071815d370 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f071806e660 tx=0x7f071001c9d0 comp rx=0 tx=0).stop 2023-10-16T20:07:28.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.453+0000 7f071ca05640 1 -- 172.21.15.88:0/1131717845 shutdown_connections 2023-10-16T20:07:28.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.453+0000 7f071ca05640 1 --2- 172.21.15.88:0/1131717845 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f071806dcd0 0x7f07180c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:28.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.453+0000 7f071ca05640 1 --2- 172.21.15.88:0/1131717845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f071806c7c0 0x7f071815d370 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:28.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.453+0000 7f071ca05640 1 --2- 172.21.15.88:0/1131717845 >> v2:172.21.15.94:3300/0 conn(0x7f07181595d0 0x7f07180c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:28.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.453+0000 7f071ca05640 1 -- 172.21.15.88:0/1131717845 >> 172.21.15.88:0/1131717845 conn(0x7f07180cec90 msgr2=0x7f07180b0070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:28.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.453+0000 7f071ca05640 1 -- 172.21.15.88:0/1131717845 shutdown_connections 2023-10-16T20:07:28.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.453+0000 7f071ca05640 1 -- 172.21.15.88:0/1131717845 wait complete. 2023-10-16T20:07:28.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.457+0000 7f071ca05640 1 Processor -- start 2023-10-16T20:07:28.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.457+0000 7f071ca05640 1 -- start start 2023-10-16T20:07:28.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.457+0000 7f071ca05640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f071806c7c0 0x7f07180972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:28.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.457+0000 7f071ca05640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f07181595d0 0x7f07180ab0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:28.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.457+0000 7f071ca05640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f071814f990 0x7f07180ab5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:28.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.457+0000 7f071ca05640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0718097820 con 0x7f071806c7c0 2023-10-16T20:07:28.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.457+0000 7f071ca05640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0718150a30 con 0x7f071814f990 2023-10-16T20:07:28.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.457+0000 7f071ca05640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0718150ec0 con 0x7f07181595d0 2023-10-16T20:07:28.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.457+0000 7f071659c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f071806c7c0 0x7f07180972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:28.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.457+0000 7f071659c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f071806c7c0 0x7f07180972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38556/0 (socket says 172.21.15.88:38556) 2023-10-16T20:07:28.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.457+0000 7f071659c640 1 -- 172.21.15.88:0/272874449 learned_addr learned my addr 172.21.15.88:0/272874449 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:28.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.457+0000 7f0716d9d640 1 --2- 172.21.15.88:0/272874449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f071814f990 0x7f07180ab5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:28.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.457+0000 7f070dd9b640 1 --2- 172.21.15.88:0/272874449 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f07181595d0 0x7f07180ab0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:28.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.457+0000 7f071659c640 1 -- 172.21.15.88:0/272874449 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f07181595d0 msgr2=0x7f07180ab0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:28.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.457+0000 7f071659c640 1 --2- 172.21.15.88:0/272874449 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f07181595d0 0x7f07180ab0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:28.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.457+0000 7f071659c640 1 -- 172.21.15.88:0/272874449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f071814f990 msgr2=0x7f07180ab5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:28.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.457+0000 7f071659c640 1 --2- 172.21.15.88:0/272874449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f071814f990 0x7f07180ab5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:28.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.457+0000 7f071659c640 1 -- 172.21.15.88:0/272874449 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0710009b30 con 0x7f071806c7c0 2023-10-16T20:07:28.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.457+0000 7f071659c640 1 --2- 172.21.15.88:0/272874449 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f071806c7c0 0x7f07180972e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f070800d5f0 tx=0x7f070800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:28.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.457+0000 7f070f7fe640 1 -- 172.21.15.88:0/272874449 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07080125f0 con 0x7f071806c7c0 2023-10-16T20:07:28.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.457+0000 7f070f7fe640 1 -- 172.21.15.88:0/272874449 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0708012750 con 0x7f071806c7c0 2023-10-16T20:07:28.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.457+0000 7f070f7fe640 1 -- 172.21.15.88:0/272874449 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0708012ae0 con 0x7f071806c7c0 2023-10-16T20:07:28.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.457+0000 7f071ca05640 1 -- 172.21.15.88:0/272874449 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f071814fe20 con 0x7f071806c7c0 2023-10-16T20:07:28.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.457+0000 7f071ca05640 1 -- 172.21.15.88:0/272874449 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07181503e0 con 0x7f071806c7c0 2023-10-16T20:07:28.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.461+0000 7f071ca05640 1 -- 172.21.15.88:0/272874449 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0718155890 con 0x7f071806c7c0 2023-10-16T20:07:28.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.461+0000 7f070f7fe640 1 -- 172.21.15.88:0/272874449 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0708012c40 con 0x7f071806c7c0 2023-10-16T20:07:28.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.461+0000 7f070f7fe640 1 --2- 172.21.15.88:0/272874449 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f06f80664a0 0x7f06f8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:28.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.461+0000 7f070f7fe640 1 -- 172.21.15.88:0/272874449 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f07080843f0 con 0x7f071806c7c0 2023-10-16T20:07:28.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.461+0000 7f070dd9b640 1 --2- 172.21.15.88:0/272874449 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f06f80664a0 0x7f06f8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:28.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.461+0000 7f070dd9b640 1 --2- 172.21.15.88:0/272874449 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f06f80664a0 0x7f06f8068960 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0718151a70 tx=0x7f0700008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:28.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.465+0000 7f070f7fe640 1 -- 172.21.15.88:0/272874449 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0708051720 con 0x7f071806c7c0 2023-10-16T20:07:28.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:28.649+0000 7f071ca05640 1 -- 172.21.15.88:0/272874449 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "testcephfs1", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f071804e9c0 con 0x7f071806c7c0 2023-10-16T20:07:29.290 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.285+0000 7f070f7fe640 1 -- 172.21.15.88:0/272874449 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "testcephfs1", "var": "allow_standby_replay", "val": "false"}]=0 v28) v1 ==== 127+0+0 (secure 0 0 0) 0x7f0708007380 con 0x7f071806c7c0 2023-10-16T20:07:29.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.289+0000 7f071ca05640 1 -- 172.21.15.88:0/272874449 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f06f80664a0 msgr2=0x7f06f8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:29.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.289+0000 7f071ca05640 1 --2- 172.21.15.88:0/272874449 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f06f80664a0 0x7f06f8068960 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0718151a70 tx=0x7f0700008040 comp rx=0 tx=0).stop 2023-10-16T20:07:29.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.289+0000 7f071ca05640 1 -- 172.21.15.88:0/272874449 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f071806c7c0 msgr2=0x7f07180972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:29.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.289+0000 7f071ca05640 1 --2- 172.21.15.88:0/272874449 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f071806c7c0 0x7f07180972e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f070800d5f0 tx=0x7f070800dab0 comp rx=0 tx=0).stop 2023-10-16T20:07:29.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.289+0000 7f071ca05640 1 -- 172.21.15.88:0/272874449 shutdown_connections 2023-10-16T20:07:29.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.289+0000 7f071ca05640 1 --2- 172.21.15.88:0/272874449 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f06f80664a0 0x7f06f8068960 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:29.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.289+0000 7f071ca05640 1 --2- 172.21.15.88:0/272874449 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f07181595d0 0x7f07180ab0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:29.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.289+0000 7f071ca05640 1 --2- 172.21.15.88:0/272874449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f071814f990 0x7f07180ab5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:29.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.289+0000 7f071ca05640 1 --2- 172.21.15.88:0/272874449 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f071806c7c0 0x7f07180972e0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:29.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.289+0000 7f071ca05640 1 -- 172.21.15.88:0/272874449 >> 172.21.15.88:0/272874449 conn(0x7f07180cec90 msgr2=0x7f0718154950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:29.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.289+0000 7f071ca05640 1 -- 172.21.15.88:0/272874449 shutdown_connections 2023-10-16T20:07:29.295 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.289+0000 7f071ca05640 1 -- 172.21.15.88:0/272874449 wait complete. 2023-10-16T20:07:29.314 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'testcephfs1' 2023-10-16T20:07:29.314 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T20:07:29.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.449+0000 7f46c66c9640 1 Processor -- start 2023-10-16T20:07:29.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.449+0000 7f46c66c9640 1 -- start start 2023-10-16T20:07:29.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.453+0000 7f46c66c9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46c01599f0 0x7f46c00b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:29.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.453+0000 7f46c66c9640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f46c0069e90 0x7f46c006a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:29.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.453+0000 7f46c66c9640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f46c00b05a0 0x7f46c00b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:29.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.453+0000 7f46c66c9640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46c006a830 con 0x7f46c0069e90 2023-10-16T20:07:29.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.453+0000 7f46c66c9640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46c00b1260 con 0x7f46c00b05a0 2023-10-16T20:07:29.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.453+0000 7f46c66c9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c00ae960 con 0x7f46c01599f0 2023-10-16T20:07:29.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.453+0000 7f46bf7fe640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f46c0069e90 0x7f46c006a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:29.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.453+0000 7f46bf7fe640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f46c0069e90 0x7f46c006a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38564/0 (socket says 172.21.15.88:38564) 2023-10-16T20:07:29.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.453+0000 7f46bf7fe640 1 -- 172.21.15.88:0/10208567 learned_addr learned my addr 172.21.15.88:0/10208567 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:29.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.453+0000 7f46c4c66640 1 --2- 172.21.15.88:0/10208567 >> v2:172.21.15.94:3300/0 conn(0x7f46c00b05a0 0x7f46c00b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:29.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.453+0000 7f46bffff640 1 --2- 172.21.15.88:0/10208567 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46c01599f0 0x7f46c00b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:29.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.453+0000 7f46bf7fe640 1 -- 172.21.15.88:0/10208567 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46c01599f0 msgr2=0x7f46c00b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:29.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.453+0000 7f46bf7fe640 1 --2- 172.21.15.88:0/10208567 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46c01599f0 0x7f46c00b2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:29.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.453+0000 7f46bf7fe640 1 -- 172.21.15.88:0/10208567 >> v2:172.21.15.94:3300/0 conn(0x7f46c00b05a0 msgr2=0x7f46c00b0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:29.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.453+0000 7f46bf7fe640 1 --2- 172.21.15.88:0/10208567 >> v2:172.21.15.94:3300/0 conn(0x7f46c00b05a0 0x7f46c00b0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:29.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.453+0000 7f46bf7fe640 1 -- 172.21.15.88:0/10208567 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c00aec40 con 0x7f46c0069e90 2023-10-16T20:07:29.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.453+0000 7f46bffff640 1 --2- 172.21.15.88:0/10208567 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46c01599f0 0x7f46c00b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:07:29.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.457+0000 7f46bf7fe640 1 --2- 172.21.15.88:0/10208567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46c0069e90 0x7f46c006a2f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f46c00701c0 tx=0x7f46b401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5fb93e8d772b0c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:29.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.457+0000 7f46beffd640 1 -- 172.21.15.88:0/10208567 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b40aa5f0 con 0x7f46c0069e90 2023-10-16T20:07:29.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.457+0000 7f46beffd640 1 -- 172.21.15.88:0/10208567 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b40aa750 con 0x7f46c0069e90 2023-10-16T20:07:29.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.457+0000 7f46beffd640 1 -- 172.21.15.88:0/10208567 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b40aaae0 con 0x7f46c0069e90 2023-10-16T20:07:29.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.457+0000 7f46c66c9640 1 -- 172.21.15.88:0/10208567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46c0069e90 msgr2=0x7f46c006a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:29.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.457+0000 7f46c66c9640 1 --2- 172.21.15.88:0/10208567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46c0069e90 0x7f46c006a2f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f46c00701c0 tx=0x7f46b401c4d0 comp rx=0 tx=0).stop 2023-10-16T20:07:29.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.457+0000 7f46c66c9640 1 -- 172.21.15.88:0/10208567 shutdown_connections 2023-10-16T20:07:29.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.457+0000 7f46c66c9640 1 --2- 172.21.15.88:0/10208567 >> v2:172.21.15.94:3300/0 conn(0x7f46c00b05a0 0x7f46c00b0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:29.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.457+0000 7f46c66c9640 1 --2- 172.21.15.88:0/10208567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46c0069e90 0x7f46c006a2f0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:29.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.457+0000 7f46c66c9640 1 --2- 172.21.15.88:0/10208567 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46c01599f0 0x7f46c00b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:29.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.457+0000 7f46c66c9640 1 -- 172.21.15.88:0/10208567 >> 172.21.15.88:0/10208567 conn(0x7f46c00ce5e0 msgr2=0x7f46c00ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:29.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.457+0000 7f46c66c9640 1 -- 172.21.15.88:0/10208567 shutdown_connections 2023-10-16T20:07:29.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.457+0000 7f46c66c9640 1 -- 172.21.15.88:0/10208567 wait complete. 2023-10-16T20:07:29.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.457+0000 7f46c66c9640 1 Processor -- start 2023-10-16T20:07:29.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.457+0000 7f46c66c9640 1 -- start start 2023-10-16T20:07:29.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.461+0000 7f46c66c9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46c0069e90 0x7f46c015b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:29.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.461+0000 7f46c66c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46c00b05a0 0x7f46c015adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:29.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.461+0000 7f46c66c9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46c01599f0 0x7f46c014e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:29.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.461+0000 7f46c66c9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c015ce10 con 0x7f46c01599f0 2023-10-16T20:07:29.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.461+0000 7f46c66c9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c015b330 con 0x7f46c00b05a0 2023-10-16T20:07:29.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.461+0000 7f46c66c9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c015bd70 con 0x7f46c0069e90 2023-10-16T20:07:29.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.461+0000 7f46c4c66640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46c01599f0 0x7f46c014e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:29.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.461+0000 7f46c4c66640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46c01599f0 0x7f46c014e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38572/0 (socket says 172.21.15.88:38572) 2023-10-16T20:07:29.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.461+0000 7f46c4c66640 1 -- 172.21.15.88:0/4249247540 learned_addr learned my addr 172.21.15.88:0/4249247540 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:29.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.461+0000 7f46bffff640 1 --2- 172.21.15.88:0/4249247540 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46c0069e90 0x7f46c015b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:29.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.461+0000 7f46bf7fe640 1 --2- 172.21.15.88:0/4249247540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46c00b05a0 0x7f46c015adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:29.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.461+0000 7f46c4c66640 1 -- 172.21.15.88:0/4249247540 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46c0069e90 msgr2=0x7f46c015b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:29.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.461+0000 7f46c4c66640 1 --2- 172.21.15.88:0/4249247540 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46c0069e90 0x7f46c015b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:29.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.461+0000 7f46c4c66640 1 -- 172.21.15.88:0/4249247540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46c00b05a0 msgr2=0x7f46c015adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:29.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.461+0000 7f46c4c66640 1 --2- 172.21.15.88:0/4249247540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46c00b05a0 0x7f46c015adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:29.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.461+0000 7f46c4c66640 1 -- 172.21.15.88:0/4249247540 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b40094d0 con 0x7f46c01599f0 2023-10-16T20:07:29.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.461+0000 7f46c4c66640 1 --2- 172.21.15.88:0/4249247540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46c01599f0 0x7f46c014e410 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f46b800eed0 tx=0x7f46b800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:29.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.461+0000 7f46bcff9640 1 -- 172.21.15.88:0/4249247540 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b8010040 con 0x7f46c01599f0 2023-10-16T20:07:29.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.461+0000 7f46bcff9640 1 -- 172.21.15.88:0/4249247540 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b8014400 con 0x7f46c01599f0 2023-10-16T20:07:29.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.461+0000 7f46c66c9640 1 -- 172.21.15.88:0/4249247540 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46c015c050 con 0x7f46c01599f0 2023-10-16T20:07:29.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.461+0000 7f46bcff9640 1 -- 172.21.15.88:0/4249247540 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b8009ea0 con 0x7f46c01599f0 2023-10-16T20:07:29.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.461+0000 7f46c66c9640 1 -- 172.21.15.88:0/4249247540 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c015c610 con 0x7f46c01599f0 2023-10-16T20:07:29.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.461+0000 7f46c66c9640 1 -- 172.21.15.88:0/4249247540 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46c004e9c0 con 0x7f46c01599f0 2023-10-16T20:07:29.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.461+0000 7f46bcff9640 1 -- 172.21.15.88:0/4249247540 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f46b80147b0 con 0x7f46c01599f0 2023-10-16T20:07:29.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.465+0000 7f46bcff9640 1 --2- 172.21.15.88:0/4249247540 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f46ac0664a0 0x7f46ac068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:29.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.465+0000 7f46bcff9640 1 -- 172.21.15.88:0/4249247540 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f46b8084900 con 0x7f46c01599f0 2023-10-16T20:07:29.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.465+0000 7f46bffff640 1 --2- 172.21.15.88:0/4249247540 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f46ac0664a0 0x7f46ac068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:29.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.465+0000 7f46bffff640 1 --2- 172.21.15.88:0/4249247540 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f46ac0664a0 0x7f46ac068960 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f46b0009950 tx=0x7f46b0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:29.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.469+0000 7f46bcff9640 1 -- 172.21.15.88:0/4249247540 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f46b8051cc0 con 0x7f46c01599f0 2023-10-16T20:07:29.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.649+0000 7f46c66c9640 1 -- 172.21.15.88:0/4249247540 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46c006aa90 con 0x7f46c01599f0 2023-10-16T20:07:29.652 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.649+0000 7f46bcff9640 1 -- 172.21.15.88:0/4249247540 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f46b8018020 con 0x7f46c01599f0 2023-10-16T20:07:29.653 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:07:29.653 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5531,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/2485638574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":2485638574},{"type":"v1","addr":"172.21.15.88:6837","nonce":2485638574}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5534,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3821390040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3821390040},{"type":"v1","addr":"172.21.15.94:6834","nonce":3821390040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5968,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2544673478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":2544673478},{"type":"v1","addr":"172.21.15.94:6835","nonce":2544673478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T20:07:25.650705+0000","modified":"2023-10-16T20:07:28.651868+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6210":{"gid":6210,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6836/1018125900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":1018125900},{"type":"v1","addr":"172.21.15.88:6836","nonce":1018125900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"testcephfs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-16T20:07:29.653 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 28 2023-10-16T20:07:29.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.649+0000 7f46c66c9640 1 -- 172.21.15.88:0/4249247540 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f46ac0664a0 msgr2=0x7f46ac068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:29.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.649+0000 7f46c66c9640 1 --2- 172.21.15.88:0/4249247540 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f46ac0664a0 0x7f46ac068960 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f46b0009950 tx=0x7f46b0008040 comp rx=0 tx=0).stop 2023-10-16T20:07:29.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.649+0000 7f46c66c9640 1 -- 172.21.15.88:0/4249247540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46c01599f0 msgr2=0x7f46c014e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:29.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.649+0000 7f46c66c9640 1 --2- 172.21.15.88:0/4249247540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46c01599f0 0x7f46c014e410 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f46b800eed0 tx=0x7f46b800b480 comp rx=0 tx=0).stop 2023-10-16T20:07:29.655 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.649+0000 7f46c66c9640 1 -- 172.21.15.88:0/4249247540 shutdown_connections 2023-10-16T20:07:29.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.649+0000 7f46c66c9640 1 --2- 172.21.15.88:0/4249247540 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f46ac0664a0 0x7f46ac068960 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:29.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.649+0000 7f46c66c9640 1 --2- 172.21.15.88:0/4249247540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46c01599f0 0x7f46c014e410 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:29.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.649+0000 7f46c66c9640 1 --2- 172.21.15.88:0/4249247540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46c00b05a0 0x7f46c015adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:29.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.649+0000 7f46c66c9640 1 --2- 172.21.15.88:0/4249247540 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46c0069e90 0x7f46c015b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:29.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.649+0000 7f46c66c9640 1 -- 172.21.15.88:0/4249247540 >> 172.21.15.88:0/4249247540 conn(0x7f46c00ce5e0 msgr2=0x7f46c014fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:29.657 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.649+0000 7f46c66c9640 1 -- 172.21.15.88:0/4249247540 shutdown_connections 2023-10-16T20:07:29.657 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.649+0000 7f46c66c9640 1 -- 172.21.15.88:0/4249247540 wait complete. 2023-10-16T20:07:29.674 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:07:29.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.805+0000 7feeaf454640 1 Processor -- start 2023-10-16T20:07:29.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.805+0000 7feeaf454640 1 -- start start 2023-10-16T20:07:29.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.809+0000 7feeaf454640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feea80715c0 0x7feea806ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:29.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.809+0000 7feeaf454640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feea80ae720 con 0x7feea806b4e0 2023-10-16T20:07:29.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.809+0000 7feeaf454640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feea80c9a10 con 0x7feea806b100 2023-10-16T20:07:29.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.809+0000 7feeaf454640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea806d650 con 0x7feea80715c0 2023-10-16T20:07:29.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.809+0000 7feead9f1640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7feea806b4e0 0x7feea80c9ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:46946/0 (socket says 172.21.15.88:46946) 2023-10-16T20:07:29.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.809+0000 7feead9f1640 1 -- 172.21.15.88:0/660292091 learned_addr learned my addr 172.21.15.88:0/660292091 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:29.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.809+0000 7feeac9ef640 1 --2- 172.21.15.88:0/660292091 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feea80715c0 0x7feea806ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:29.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.809+0000 7fee9ffff640 1 -- 172.21.15.88:0/660292091 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2018421815 0 0) 0x7feea00014a0 con 0x7feea806b4e0 2023-10-16T20:07:29.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.809+0000 7fee9ffff640 1 -- 172.21.15.88:0/660292091 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee94003530 con 0x7feea806b4e0 2023-10-16T20:07:29.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.809+0000 7fee9ffff640 1 -- 172.21.15.88:0/660292091 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 129325470 0 0) 0x7fee94003530 con 0x7feea806b4e0 2023-10-16T20:07:29.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.809+0000 7fee9ffff640 1 -- 172.21.15.88:0/660292091 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feea80715c0 msgr2=0x7feea806ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:29.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.809+0000 7fee9ffff640 1 --2- 172.21.15.88:0/660292091 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feea80715c0 0x7feea806ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:29.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.809+0000 7fee9ffff640 1 -- 172.21.15.88:0/660292091 >> v1:172.21.15.94:6789/0 conn(0x7feea806b100 legacy=0x7feea806c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:29.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.809+0000 7fee9ffff640 1 -- 172.21.15.88:0/660292091 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea80c9420 con 0x7feea806b4e0 2023-10-16T20:07:29.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.809+0000 7fee9ffff640 1 -- 172.21.15.88:0/660292091 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7feea0001b90 con 0x7feea806b4e0 2023-10-16T20:07:29.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.809+0000 7fee9ffff640 1 -- 172.21.15.88:0/660292091 >> v1:172.21.15.88:6789/0 conn(0x7feea806b4e0 legacy=0x7feea80c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:29.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.809+0000 7fee9ffff640 1 --2- 172.21.15.88:0/660292091 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feea80715c0 0x7fee940039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:29.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.809+0000 7fee9ffff640 1 --2- 172.21.15.88:0/660292091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee94003f30 0x7fee940047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:29.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.809+0000 7fee9ffff640 1 --2- 172.21.15.88:0/660292091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee94004d10 0x7fee94005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:29.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.809+0000 7fee9ffff640 1 -- 172.21.15.88:0/660292091 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea806d650 con 0x7fee94003f30 2023-10-16T20:07:29.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.809+0000 7fee9ffff640 1 -- 172.21.15.88:0/660292091 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9400ba70 con 0x7fee94004d10 2023-10-16T20:07:29.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.809+0000 7fee9ffff640 1 -- 172.21.15.88:0/660292091 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9400bc30 con 0x7feea80715c0 2023-10-16T20:07:29.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.809+0000 7feeac9ef640 1 --2- 172.21.15.88:0/660292091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee94003f30 0x7fee940047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:29.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.809+0000 7feead9f1640 1 --2- 172.21.15.88:0/660292091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee94004d10 0x7fee94005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:29.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.809+0000 7feead1f0640 1 --2- 172.21.15.88:0/660292091 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feea80715c0 0x7fee940039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:29.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.809+0000 7feeac9ef640 1 -- 172.21.15.88:0/660292091 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feea80715c0 msgr2=0x7fee940039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:29.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.809+0000 7feeac9ef640 1 --2- 172.21.15.88:0/660292091 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feea80715c0 0x7fee940039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:29.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.809+0000 7feeac9ef640 1 -- 172.21.15.88:0/660292091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee94004d10 msgr2=0x7fee94005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:29.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.809+0000 7feeac9ef640 1 --2- 172.21.15.88:0/660292091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee94004d10 0x7fee94005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:29.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.809+0000 7feeac9ef640 1 -- 172.21.15.88:0/660292091 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee94003530 con 0x7fee94003f30 2023-10-16T20:07:29.823 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.813+0000 7feeac9ef640 1 --2- 172.21.15.88:0/660292091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee94003f30 0x7fee940047d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fee98009ab0 tx=0x7fee9801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43bf10dedade07e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:29.823 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.813+0000 7fee9ffff640 1 -- 172.21.15.88:0/660292091 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee980a5510 con 0x7fee94003f30 2023-10-16T20:07:29.823 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.813+0000 7fee9ffff640 1 -- 172.21.15.88:0/660292091 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee980a5670 con 0x7fee94003f30 2023-10-16T20:07:29.823 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.813+0000 7fee9ffff640 1 -- 172.21.15.88:0/660292091 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee980a5820 con 0x7fee94003f30 2023-10-16T20:07:29.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.813+0000 7feeaf454640 1 -- 172.21.15.88:0/660292091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee94003f30 msgr2=0x7fee940047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:29.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.813+0000 7feeaf454640 1 --2- 172.21.15.88:0/660292091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee94003f30 0x7fee940047d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fee98009ab0 tx=0x7fee9801ca30 comp rx=0 tx=0).stop 2023-10-16T20:07:29.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.813+0000 7feead9f1640 1 -- 172.21.15.88:0/660292091 reap_dead start 2023-10-16T20:07:29.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.813+0000 7feeaf454640 1 -- 172.21.15.88:0/660292091 shutdown_connections 2023-10-16T20:07:29.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.813+0000 7feeaf454640 1 -- 172.21.15.88:0/660292091 >> 172.21.15.88:0/660292091 conn(0x7feea8070aa0 msgr2=0x7feea8070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:29.825 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.813+0000 7feeaf454640 1 -- 172.21.15.88:0/660292091 shutdown_connections 2023-10-16T20:07:29.825 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.813+0000 7feeaf454640 1 -- 172.21.15.88:0/660292091 wait complete. 2023-10-16T20:07:29.825 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.817+0000 7feeaf454640 1 Processor -- start 2023-10-16T20:07:29.825 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.817+0000 7feeaf454640 1 -- start start 2023-10-16T20:07:29.825 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.817+0000 7feeaf454640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fee94003f30 0x7feea81e6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:29.826 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.817+0000 7feeaf454640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feea81e7630 0x7feea8159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:29.826 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.817+0000 7feeaf454640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feea815e5a0 0x7feea815ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:29.826 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.817+0000 7feeaf454640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea815ef60 con 0x7feea815e5a0 2023-10-16T20:07:29.826 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.817+0000 7feeaf454640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea81e7ab0 con 0x7feea81e7630 2023-10-16T20:07:29.826 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.817+0000 7feeaf454640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea81e7d80 con 0x7fee94003f30 2023-10-16T20:07:29.827 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.817+0000 7feead1f0640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fee94003f30 0x7feea81e6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:29.827 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.817+0000 7feead9f1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feea815e5a0 0x7feea815ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:29.827 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.817+0000 7feead1f0640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fee94003f30 0x7feea81e6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:54646/0 (socket says 172.21.15.88:54646) 2023-10-16T20:07:29.827 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.817+0000 7feead1f0640 1 -- 172.21.15.88:0/874631172 learned_addr learned my addr 172.21.15.88:0/874631172 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:29.827 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.817+0000 7feeac9ef640 1 --2- 172.21.15.88:0/874631172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feea81e7630 0x7feea8159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:29.828 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.817+0000 7feead1f0640 1 -- 172.21.15.88:0/874631172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feea81e7630 msgr2=0x7feea8159320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:29.828 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.817+0000 7feead1f0640 1 --2- 172.21.15.88:0/874631172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feea81e7630 0x7feea8159320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:29.828 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.817+0000 7feead1f0640 1 -- 172.21.15.88:0/874631172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feea815e5a0 msgr2=0x7feea815ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:29.828 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.817+0000 7feead1f0640 1 --2- 172.21.15.88:0/874631172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feea815e5a0 0x7feea815ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:29.828 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.817+0000 7feead1f0640 1 -- 172.21.15.88:0/874631172 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea81e8060 con 0x7fee94003f30 2023-10-16T20:07:29.829 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.817+0000 7feead9f1640 1 --2- 172.21.15.88:0/874631172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feea815e5a0 0x7feea815ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:07:29.829 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.817+0000 7feead1f0640 1 --2- 172.21.15.88:0/874631172 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fee94003f30 0x7feea81e6dc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7feea400f7f0 tx=0x7feea400fcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:29.829 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.817+0000 7fee9dffb640 1 -- 172.21.15.88:0/874631172 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea40050d0 con 0x7fee94003f30 2023-10-16T20:07:29.829 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.817+0000 7fee9dffb640 1 -- 172.21.15.88:0/874631172 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea4005230 con 0x7fee94003f30 2023-10-16T20:07:29.829 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.817+0000 7feeaf454640 1 -- 172.21.15.88:0/874631172 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feea81e8340 con 0x7fee94003f30 2023-10-16T20:07:29.830 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.817+0000 7fee9dffb640 1 -- 172.21.15.88:0/874631172 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea400d730 con 0x7fee94003f30 2023-10-16T20:07:29.830 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.817+0000 7feeaf454640 1 -- 172.21.15.88:0/874631172 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feea81e8900 con 0x7fee94003f30 2023-10-16T20:07:29.831 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.821+0000 7feeaf454640 1 -- 172.21.15.88:0/874631172 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feea804e9c0 con 0x7fee94003f30 2023-10-16T20:07:29.831 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.821+0000 7fee9dffb640 1 -- 172.21.15.88:0/874631172 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7feea400d890 con 0x7fee94003f30 2023-10-16T20:07:29.831 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.821+0000 7fee9dffb640 1 --2- 172.21.15.88:0/874631172 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fee8c0664a0 0x7fee8c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:29.832 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.821+0000 7fee9dffb640 1 -- 172.21.15.88:0/874631172 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7feea4083e30 con 0x7fee94003f30 2023-10-16T20:07:29.832 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.821+0000 7feeac9ef640 1 --2- 172.21.15.88:0/874631172 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fee8c0664a0 0x7fee8c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:29.832 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.821+0000 7feeac9ef640 1 --2- 172.21.15.88:0/874631172 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fee8c0664a0 0x7fee8c068960 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fee980af000 tx=0x7fee980046d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:29.832 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.825+0000 7fee9dffb640 1 -- 172.21.15.88:0/874631172 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7feea4051200 con 0x7fee94003f30 2023-10-16T20:07:29.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.973+0000 7feeaf454640 1 -- 172.21.15.88:0/874631172 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feea8065e70 con 0x7fee94003f30 2023-10-16T20:07:29.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.977+0000 7fee9dffb640 1 -- 172.21.15.88:0/874631172 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16718 (secure 0 0 0) 0x7feea4054820 con 0x7fee94003f30 2023-10-16T20:07:29.981 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:07:29.982 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":58,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:07:27.206614+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T20:07:24.001121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"testcephfs1_data","create_time":"2023-10-16T20:07:24.636209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:07:29.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.977+0000 7feeaf454640 1 -- 172.21.15.88:0/874631172 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fee8c0664a0 msgr2=0x7fee8c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:29.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.977+0000 7feeaf454640 1 --2- 172.21.15.88:0/874631172 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fee8c0664a0 0x7fee8c068960 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fee980af000 tx=0x7fee980046d0 comp rx=0 tx=0).stop 2023-10-16T20:07:29.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.977+0000 7feeaf454640 1 -- 172.21.15.88:0/874631172 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fee94003f30 msgr2=0x7feea81e6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:29.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.977+0000 7feeaf454640 1 --2- 172.21.15.88:0/874631172 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fee94003f30 0x7feea81e6dc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7feea400f7f0 tx=0x7feea400fcb0 comp rx=0 tx=0).stop 2023-10-16T20:07:29.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.977+0000 7feeaf454640 1 -- 172.21.15.88:0/874631172 shutdown_connections 2023-10-16T20:07:29.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.977+0000 7feeaf454640 1 --2- 172.21.15.88:0/874631172 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fee8c0664a0 0x7fee8c068960 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:29.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.977+0000 7feeaf454640 1 --2- 172.21.15.88:0/874631172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feea815e5a0 0x7feea815ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:29.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.977+0000 7feeaf454640 1 --2- 172.21.15.88:0/874631172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feea81e7630 0x7feea8159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:29.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.977+0000 7feeaf454640 1 --2- 172.21.15.88:0/874631172 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fee94003f30 0x7feea81e6dc0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:29.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.977+0000 7feeaf454640 1 -- 172.21.15.88:0/874631172 >> 172.21.15.88:0/874631172 conn(0x7feea8070aa0 msgr2=0x7feea80caf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:29.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.977+0000 7feeaf454640 1 -- 172.21.15.88:0/874631172 shutdown_connections 2023-10-16T20:07:29.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:29.977+0000 7feeaf454640 1 -- 172.21.15.88:0/874631172 wait complete. 2023-10-16T20:07:30.002 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T20:07:30.003 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T20:07:30.139 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.133+0000 7f7c03046640 1 Processor -- start 2023-10-16T20:07:30.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.137+0000 7f7c03046640 1 -- start start 2023-10-16T20:07:30.140 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.137+0000 7f7c03046640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7bfc0ade20 0x7f7bfc0b1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:30.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.137+0000 7f7c03046640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7bfc143eb0 0x7f7bfc0b0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:30.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.137+0000 7f7c03046640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bfc0b2570 con 0x7f7bfc0ada40 2023-10-16T20:07:30.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.137+0000 7f7c03046640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bfc0cca50 con 0x7f7bfc143eb0 2023-10-16T20:07:30.141 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.137+0000 7f7c03046640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc144340 con 0x7f7bfc0ade20 2023-10-16T20:07:30.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.137+0000 7f7bf3fff640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7bfc0ade20 0x7f7bfc0b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:30.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.137+0000 7f7bf3fff640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7bfc0ade20 0x7f7bfc0b1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:54650/0 (socket says 172.21.15.88:54650) 2023-10-16T20:07:30.142 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.137+0000 7f7bf3fff640 1 -- 172.21.15.88:0/2061463773 learned_addr learned my addr 172.21.15.88:0/2061463773 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:30.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.137+0000 7f7c015e3640 1 --2- 172.21.15.88:0/2061463773 >> v2:172.21.15.94:3300/0 conn(0x7f7bfc143eb0 0x7f7bfc0b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:30.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.137+0000 7f7bf3fff640 1 -- 172.21.15.88:0/2061463773 >> v2:172.21.15.94:3300/0 conn(0x7f7bfc143eb0 msgr2=0x7f7bfc0b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:30.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.137+0000 7f7bf3fff640 1 --2- 172.21.15.88:0/2061463773 >> v2:172.21.15.94:3300/0 conn(0x7f7bfc143eb0 0x7f7bfc0b0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:30.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.137+0000 7f7bf3fff640 1 -- 172.21.15.88:0/2061463773 >> v1:172.21.15.94:6789/0 conn(0x7f7bfc0ada40 legacy=0x7f7bfc15adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:30.143 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.137+0000 7f7bf3fff640 1 -- 172.21.15.88:0/2061463773 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bfc0b1fb0 con 0x7f7bfc0ade20 2023-10-16T20:07:30.144 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.141+0000 7f7bf3fff640 1 --2- 172.21.15.88:0/2061463773 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7bfc0ade20 0x7f7bfc0b1920 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7bec009870 tx=0x7f7bec01c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bf8aabf754624b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:30.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.141+0000 7f7bf37fe640 1 -- 172.21.15.88:0/2061463773 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bec0ad640 con 0x7f7bfc0ade20 2023-10-16T20:07:30.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.141+0000 7f7bf37fe640 1 -- 172.21.15.88:0/2061463773 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bec0ad7a0 con 0x7f7bfc0ade20 2023-10-16T20:07:30.145 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.141+0000 7f7bf37fe640 1 -- 172.21.15.88:0/2061463773 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bec0adb30 con 0x7f7bfc0ade20 2023-10-16T20:07:30.146 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.141+0000 7f7c03046640 1 -- 172.21.15.88:0/2061463773 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7bfc0ade20 msgr2=0x7f7bfc0b1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:30.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.141+0000 7f7c03046640 1 --2- 172.21.15.88:0/2061463773 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7bfc0ade20 0x7f7bfc0b1920 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7bec009870 tx=0x7f7bec01c480 comp rx=0 tx=0).stop 2023-10-16T20:07:30.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.141+0000 7f7c03046640 1 -- 172.21.15.88:0/2061463773 shutdown_connections 2023-10-16T20:07:30.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.141+0000 7f7c03046640 1 --2- 172.21.15.88:0/2061463773 >> v2:172.21.15.94:3300/0 conn(0x7f7bfc143eb0 0x7f7bfc0b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:30.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.141+0000 7f7c03046640 1 --2- 172.21.15.88:0/2061463773 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7bfc0ade20 0x7f7bfc0b1920 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:30.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.141+0000 7f7c03046640 1 -- 172.21.15.88:0/2061463773 >> 172.21.15.88:0/2061463773 conn(0x7f7bfc0ce5e0 msgr2=0x7f7bfc0ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:30.147 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.141+0000 7f7c03046640 1 -- 172.21.15.88:0/2061463773 shutdown_connections 2023-10-16T20:07:30.148 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.141+0000 7f7c03046640 1 -- 172.21.15.88:0/2061463773 wait complete. 2023-10-16T20:07:30.149 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.145+0000 7f7c03046640 1 Processor -- start 2023-10-16T20:07:30.149 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.145+0000 7f7c03046640 1 -- start start 2023-10-16T20:07:30.149 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.145+0000 7f7c03046640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7bfc0ada40 0x7f7bfc15ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:30.150 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.145+0000 7f7c03046640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bfc0ade20 0x7f7bfc14a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:30.150 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.145+0000 7f7c03046640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7bfc143eb0 0x7f7bfc14ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:30.150 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.145+0000 7f7c03046640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc137da0 con 0x7f7bfc143eb0 2023-10-16T20:07:30.150 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.145+0000 7f7c03046640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc135ae0 con 0x7f7bfc0ade20 2023-10-16T20:07:30.150 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.145+0000 7f7c03046640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc15c300 con 0x7f7bfc0ada40 2023-10-16T20:07:30.150 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.145+0000 7f7c00de2640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7bfc0ada40 0x7f7bfc15ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:30.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.145+0000 7f7c00de2640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7bfc0ada40 0x7f7bfc15ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:54656/0 (socket says 172.21.15.88:54656) 2023-10-16T20:07:30.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.145+0000 7f7c00de2640 1 -- 172.21.15.88:0/2366133096 learned_addr learned my addr 172.21.15.88:0/2366133096 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:30.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.145+0000 7f7bf3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bfc0ade20 0x7f7bfc14a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:30.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.145+0000 7f7c015e3640 1 --2- 172.21.15.88:0/2366133096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7bfc143eb0 0x7f7bfc14ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:30.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.145+0000 7f7bf3fff640 1 -- 172.21.15.88:0/2366133096 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7bfc0ada40 msgr2=0x7f7bfc15ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:30.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.145+0000 7f7bf3fff640 1 --2- 172.21.15.88:0/2366133096 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7bfc0ada40 0x7f7bfc15ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:30.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.145+0000 7f7bf3fff640 1 -- 172.21.15.88:0/2366133096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7bfc143eb0 msgr2=0x7f7bfc14ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:30.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.145+0000 7f7bf3fff640 1 --2- 172.21.15.88:0/2366133096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7bfc143eb0 0x7f7bfc14ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:30.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.145+0000 7f7bf3fff640 1 -- 172.21.15.88:0/2366133096 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bec0094d0 con 0x7f7bfc0ade20 2023-10-16T20:07:30.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.145+0000 7f7c00de2640 1 --2- 172.21.15.88:0/2366133096 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7bfc0ada40 0x7f7bfc15ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:07:30.153 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.145+0000 7f7bf3fff640 1 --2- 172.21.15.88:0/2366133096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bfc0ade20 0x7f7bfc14a620 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7bec0ac000 tx=0x7f7bec019220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:30.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.145+0000 7f7bf17fa640 1 -- 172.21.15.88:0/2366133096 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bec0b6640 con 0x7f7bfc0ade20 2023-10-16T20:07:30.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.145+0000 7f7c03046640 1 -- 172.21.15.88:0/2366133096 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bfc137a00 con 0x7f7bfc0ade20 2023-10-16T20:07:30.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.145+0000 7f7bf17fa640 1 -- 172.21.15.88:0/2366133096 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bec0a6040 con 0x7f7bfc0ade20 2023-10-16T20:07:30.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.145+0000 7f7c03046640 1 -- 172.21.15.88:0/2366133096 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bfc14b480 con 0x7f7bfc0ade20 2023-10-16T20:07:30.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.145+0000 7f7bf17fa640 1 -- 172.21.15.88:0/2366133096 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bec0b69c0 con 0x7f7bfc0ade20 2023-10-16T20:07:30.155 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.145+0000 7f7c03046640 1 -- 172.21.15.88:0/2366133096 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bfc04e9c0 con 0x7f7bfc0ade20 2023-10-16T20:07:30.155 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.149+0000 7f7bf17fa640 1 -- 172.21.15.88:0/2366133096 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7bec0b4070 con 0x7f7bfc0ade20 2023-10-16T20:07:30.155 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.149+0000 7f7bf17fa640 1 --2- 172.21.15.88:0/2366133096 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7be80664a0 0x7f7be8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:30.156 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.149+0000 7f7bf17fa640 1 -- 172.21.15.88:0/2366133096 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f7bec11e930 con 0x7f7bfc0ade20 2023-10-16T20:07:30.156 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.149+0000 7f7c00de2640 1 --2- 172.21.15.88:0/2366133096 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7be80664a0 0x7f7be8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:30.156 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.149+0000 7f7c00de2640 1 --2- 172.21.15.88:0/2366133096 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7be80664a0 0x7f7be8068960 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc0ac390 tx=0x7f7bf8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:30.157 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.153+0000 7f7bf17fa640 1 -- 172.21.15.88:0/2366133096 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7bec0ebc60 con 0x7f7bfc0ade20 2023-10-16T20:07:30.304 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.301+0000 7f7c03046640 1 -- 172.21.15.88:0/2366133096 --> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7bfc13ed60 con 0x7f7be80664a0 2023-10-16T20:07:30.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.309+0000 7f7bf17fa640 1 -- 172.21.15.88:0/2366133096 <== mgr.4110 v2:172.21.15.88:6832/4078483333 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288712 (secure 0 0 0) 0x7f7bfc13ed60 con 0x7f7be80664a0 2023-10-16T20:07:30.315 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:07:30.317 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2023-10-16T20:07:30.318 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.313+0000 7f7c03046640 1 -- 172.21.15.88:0/2366133096 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7be80664a0 msgr2=0x7f7be8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:30.319 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.313+0000 7f7c03046640 1 --2- 172.21.15.88:0/2366133096 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7be80664a0 0x7f7be8068960 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc0ac390 tx=0x7f7bf8009040 comp rx=0 tx=0).stop 2023-10-16T20:07:30.319 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.313+0000 7f7c03046640 1 -- 172.21.15.88:0/2366133096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bfc0ade20 msgr2=0x7f7bfc14a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:30.319 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.313+0000 7f7c03046640 1 --2- 172.21.15.88:0/2366133096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bfc0ade20 0x7f7bfc14a620 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7bec0ac000 tx=0x7f7bec019220 comp rx=0 tx=0).stop 2023-10-16T20:07:30.319 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.313+0000 7f7c03046640 1 -- 172.21.15.88:0/2366133096 shutdown_connections 2023-10-16T20:07:30.320 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.313+0000 7f7c03046640 1 --2- 172.21.15.88:0/2366133096 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7be80664a0 0x7f7be8068960 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:30.320 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.313+0000 7f7c03046640 1 --2- 172.21.15.88:0/2366133096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7bfc143eb0 0x7f7bfc14ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:30.320 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.313+0000 7f7c03046640 1 --2- 172.21.15.88:0/2366133096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bfc0ade20 0x7f7bfc14a620 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:30.320 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.313+0000 7f7c03046640 1 --2- 172.21.15.88:0/2366133096 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7bfc0ada40 0x7f7bfc15ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:30.320 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.313+0000 7f7c03046640 1 -- 172.21.15.88:0/2366133096 >> 172.21.15.88:0/2366133096 conn(0x7f7bfc0ce5e0 msgr2=0x7f7bfc13f560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:30.321 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.313+0000 7f7c03046640 1 -- 172.21.15.88:0/2366133096 shutdown_connections 2023-10-16T20:07:30.321 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.313+0000 7f7c03046640 1 -- 172.21.15.88:0/2366133096 wait complete. 2023-10-16T20:07:30.336 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2023-10-16T20:07:28.478202+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592196,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":193,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239360,"kb_used_data":6536,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735360,"statfs":{"total":773094113280,"available":772849008640,"internally_reserved":0,"allocated":6692864,"data_stored":3621353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238276153},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1828,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.224847"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794955+0000","last_change":"2023-10-16T20:07:26.231631+0000","last_active":"2023-10-16T20:07:27.794955+0000","last_peered":"2023-10-16T20:07:27.794955+0000","last_clean":"2023-10-16T20:07:27.794955+0000","last_became_active":"2023-10-16T20:07:26.231332+0000","last_became_peered":"2023-10-16T20:07:26.231332+0000","last_unstale":"2023-10-16T20:07:27.794955+0000","last_undegraded":"2023-10-16T20:07:27.794955+0000","last_fullsized":"2023-10-16T20:07:27.794955+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:38:52.502602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211803+0000","last_change":"2023-10-16T20:07:26.236566+0000","last_active":"2023-10-16T20:07:27.211803+0000","last_peered":"2023-10-16T20:07:27.211803+0000","last_clean":"2023-10-16T20:07:27.211803+0000","last_became_active":"2023-10-16T20:07:26.236293+0000","last_became_peered":"2023-10-16T20:07:26.236293+0000","last_unstale":"2023-10-16T20:07:27.211803+0000","last_undegraded":"2023-10-16T20:07:27.211803+0000","last_fullsized":"2023-10-16T20:07:27.211803+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:05:37.489356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213581+0000","last_change":"2023-10-16T20:07:26.233160+0000","last_active":"2023-10-16T20:07:27.213581+0000","last_peered":"2023-10-16T20:07:27.213581+0000","last_clean":"2023-10-16T20:07:27.213581+0000","last_became_active":"2023-10-16T20:07:26.232765+0000","last_became_peered":"2023-10-16T20:07:26.232765+0000","last_unstale":"2023-10-16T20:07:27.213581+0000","last_undegraded":"2023-10-16T20:07:27.213581+0000","last_fullsized":"2023-10-16T20:07:27.213581+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:07:42.996110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795167+0000","last_change":"2023-10-16T20:07:26.219553+0000","last_active":"2023-10-16T20:07:27.795167+0000","last_peered":"2023-10-16T20:07:27.795167+0000","last_clean":"2023-10-16T20:07:27.795167+0000","last_became_active":"2023-10-16T20:07:26.219232+0000","last_became_peered":"2023-10-16T20:07:26.219232+0000","last_unstale":"2023-10-16T20:07:27.795167+0000","last_undegraded":"2023-10-16T20:07:27.795167+0000","last_fullsized":"2023-10-16T20:07:27.795167+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:26:50.399490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794028+0000","last_change":"2023-10-16T20:07:26.231576+0000","last_active":"2023-10-16T20:07:27.794028+0000","last_peered":"2023-10-16T20:07:27.794028+0000","last_clean":"2023-10-16T20:07:27.794028+0000","last_became_active":"2023-10-16T20:07:26.227600+0000","last_became_peered":"2023-10-16T20:07:26.227600+0000","last_unstale":"2023-10-16T20:07:27.794028+0000","last_undegraded":"2023-10-16T20:07:27.794028+0000","last_fullsized":"2023-10-16T20:07:27.794028+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:45:52.100295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-10-16T20:07:25.270580+0000","last_change":"2023-10-16T20:07:25.270580+0000","last_active":"2023-10-16T20:07:25.253517+0000","last_peered":"2023-10-16T20:07:25.253517+0000","last_clean":"2023-10-16T20:07:25.253517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T20:07:25.270580+0000","last_undegraded":"2023-10-16T20:07:25.270580+0000","last_fullsized":"2023-10-16T20:07:25.270580+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794540+0000","last_change":"2023-10-16T20:07:26.229775+0000","last_active":"2023-10-16T20:07:27.794540+0000","last_peered":"2023-10-16T20:07:27.794540+0000","last_clean":"2023-10-16T20:07:27.794540+0000","last_became_active":"2023-10-16T20:07:26.229111+0000","last_became_peered":"2023-10-16T20:07:26.229111+0000","last_unstale":"2023-10-16T20:07:27.794540+0000","last_undegraded":"2023-10-16T20:07:27.794540+0000","last_fullsized":"2023-10-16T20:07:27.794540+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:11:58.985907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211291+0000","last_change":"2023-10-16T20:07:26.237894+0000","last_active":"2023-10-16T20:07:27.211291+0000","last_peered":"2023-10-16T20:07:27.211291+0000","last_clean":"2023-10-16T20:07:27.211291+0000","last_became_active":"2023-10-16T20:07:26.237614+0000","last_became_peered":"2023-10-16T20:07:26.237614+0000","last_unstale":"2023-10-16T20:07:27.211291+0000","last_undegraded":"2023-10-16T20:07:27.211291+0000","last_fullsized":"2023-10-16T20:07:27.211291+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:07:42.996110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795633+0000","last_change":"2023-10-16T20:07:26.228360+0000","last_active":"2023-10-16T20:07:27.795633+0000","last_peered":"2023-10-16T20:07:27.795633+0000","last_clean":"2023-10-16T20:07:27.795633+0000","last_became_active":"2023-10-16T20:07:26.228053+0000","last_became_peered":"2023-10-16T20:07:26.228053+0000","last_unstale":"2023-10-16T20:07:27.795633+0000","last_undegraded":"2023-10-16T20:07:27.795633+0000","last_fullsized":"2023-10-16T20:07:27.795633+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:19:45.099223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795613+0000","last_change":"2023-10-16T20:07:26.235541+0000","last_active":"2023-10-16T20:07:27.795613+0000","last_peered":"2023-10-16T20:07:27.795613+0000","last_clean":"2023-10-16T20:07:27.795613+0000","last_became_active":"2023-10-16T20:07:26.235235+0000","last_became_peered":"2023-10-16T20:07:26.235235+0000","last_unstale":"2023-10-16T20:07:27.795613+0000","last_undegraded":"2023-10-16T20:07:27.795613+0000","last_fullsized":"2023-10-16T20:07:27.795613+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:05:02.583660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795487+0000","last_change":"2023-10-16T20:07:26.234105+0000","last_active":"2023-10-16T20:07:27.795487+0000","last_peered":"2023-10-16T20:07:27.795487+0000","last_clean":"2023-10-16T20:07:27.795487+0000","last_became_active":"2023-10-16T20:07:26.233841+0000","last_became_peered":"2023-10-16T20:07:26.233841+0000","last_unstale":"2023-10-16T20:07:27.795487+0000","last_undegraded":"2023-10-16T20:07:27.795487+0000","last_fullsized":"2023-10-16T20:07:27.795487+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:05:50.890403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795308+0000","last_change":"2023-10-16T20:07:26.228168+0000","last_active":"2023-10-16T20:07:27.795308+0000","last_peered":"2023-10-16T20:07:27.795308+0000","last_clean":"2023-10-16T20:07:27.795308+0000","last_became_active":"2023-10-16T20:07:26.227862+0000","last_became_peered":"2023-10-16T20:07:26.227862+0000","last_unstale":"2023-10-16T20:07:27.795308+0000","last_undegraded":"2023-10-16T20:07:27.795308+0000","last_fullsized":"2023-10-16T20:07:27.795308+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:52:43.995476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213961+0000","last_change":"2023-10-16T20:07:26.230062+0000","last_active":"2023-10-16T20:07:27.213961+0000","last_peered":"2023-10-16T20:07:27.213961+0000","last_clean":"2023-10-16T20:07:27.213961+0000","last_became_active":"2023-10-16T20:07:26.229479+0000","last_became_peered":"2023-10-16T20:07:26.229479+0000","last_unstale":"2023-10-16T20:07:27.213961+0000","last_undegraded":"2023-10-16T20:07:27.213961+0000","last_fullsized":"2023-10-16T20:07:27.213961+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:46:44.338547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795091+0000","last_change":"2023-10-16T20:07:26.232039+0000","last_active":"2023-10-16T20:07:27.795091+0000","last_peered":"2023-10-16T20:07:27.795091+0000","last_clean":"2023-10-16T20:07:27.795091+0000","last_became_active":"2023-10-16T20:07:26.228897+0000","last_became_peered":"2023-10-16T20:07:26.228897+0000","last_unstale":"2023-10-16T20:07:27.795091+0000","last_undegraded":"2023-10-16T20:07:27.795091+0000","last_fullsized":"2023-10-16T20:07:27.795091+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:19:55.369594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794622+0000","last_change":"2023-10-16T20:07:26.229967+0000","last_active":"2023-10-16T20:07:27.794622+0000","last_peered":"2023-10-16T20:07:27.794622+0000","last_clean":"2023-10-16T20:07:27.794622+0000","last_became_active":"2023-10-16T20:07:26.229489+0000","last_became_peered":"2023-10-16T20:07:26.229489+0000","last_unstale":"2023-10-16T20:07:27.794622+0000","last_undegraded":"2023-10-16T20:07:27.794622+0000","last_fullsized":"2023-10-16T20:07:27.794622+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:48:17.850138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-10-16T20:07:25.270380+0000","last_change":"2023-10-16T20:07:25.270380+0000","last_active":"2023-10-16T20:07:25.253517+0000","last_peered":"2023-10-16T20:07:25.253517+0000","last_clean":"2023-10-16T20:07:25.253517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T20:07:25.270380+0000","last_undegraded":"2023-10-16T20:07:25.270380+0000","last_fullsized":"2023-10-16T20:07:25.270380+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211982+0000","last_change":"2023-10-16T20:07:26.236536+0000","last_active":"2023-10-16T20:07:27.211982+0000","last_peered":"2023-10-16T20:07:27.211982+0000","last_clean":"2023-10-16T20:07:27.211982+0000","last_became_active":"2023-10-16T20:07:26.236238+0000","last_became_peered":"2023-10-16T20:07:26.236238+0000","last_unstale":"2023-10-16T20:07:27.211982+0000","last_undegraded":"2023-10-16T20:07:27.211982+0000","last_fullsized":"2023-10-16T20:07:27.211982+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:18:14.053601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213163+0000","last_change":"2023-10-16T20:07:26.233070+0000","last_active":"2023-10-16T20:07:27.213163+0000","last_peered":"2023-10-16T20:07:27.213163+0000","last_clean":"2023-10-16T20:07:27.213163+0000","last_became_active":"2023-10-16T20:07:26.232664+0000","last_became_peered":"2023-10-16T20:07:26.232664+0000","last_unstale":"2023-10-16T20:07:27.213163+0000","last_undegraded":"2023-10-16T20:07:27.213163+0000","last_fullsized":"2023-10-16T20:07:27.213163+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:24:24.649666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794300+0000","last_change":"2023-10-16T20:07:26.229604+0000","last_active":"2023-10-16T20:07:27.794300+0000","last_peered":"2023-10-16T20:07:27.794300+0000","last_clean":"2023-10-16T20:07:27.794300+0000","last_became_active":"2023-10-16T20:07:26.229330+0000","last_became_peered":"2023-10-16T20:07:26.229330+0000","last_unstale":"2023-10-16T20:07:27.794300+0000","last_undegraded":"2023-10-16T20:07:27.794300+0000","last_fullsized":"2023-10-16T20:07:27.794300+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:02:07.982732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211532+0000","last_change":"2023-10-16T20:07:26.229245+0000","last_active":"2023-10-16T20:07:27.211532+0000","last_peered":"2023-10-16T20:07:27.211532+0000","last_clean":"2023-10-16T20:07:27.211532+0000","last_became_active":"2023-10-16T20:07:26.228987+0000","last_became_peered":"2023-10-16T20:07:26.228987+0000","last_unstale":"2023-10-16T20:07:27.211532+0000","last_undegraded":"2023-10-16T20:07:27.211532+0000","last_fullsized":"2023-10-16T20:07:27.211532+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:50:26.093947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213084+0000","last_change":"2023-10-16T20:07:26.238069+0000","last_active":"2023-10-16T20:07:27.213084+0000","last_peered":"2023-10-16T20:07:27.213084+0000","last_clean":"2023-10-16T20:07:27.213084+0000","last_became_active":"2023-10-16T20:07:26.237072+0000","last_became_peered":"2023-10-16T20:07:26.237072+0000","last_unstale":"2023-10-16T20:07:27.213084+0000","last_undegraded":"2023-10-16T20:07:27.213084+0000","last_fullsized":"2023-10-16T20:07:27.213084+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:08:51.178056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211487+0000","last_change":"2023-10-16T20:07:26.240455+0000","last_active":"2023-10-16T20:07:27.211487+0000","last_peered":"2023-10-16T20:07:27.211487+0000","last_clean":"2023-10-16T20:07:27.211487+0000","last_became_active":"2023-10-16T20:07:26.239255+0000","last_became_peered":"2023-10-16T20:07:26.239255+0000","last_unstale":"2023-10-16T20:07:27.211487+0000","last_undegraded":"2023-10-16T20:07:27.211487+0000","last_fullsized":"2023-10-16T20:07:27.211487+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:20:02.841816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213052+0000","last_change":"2023-10-16T20:07:26.240749+0000","last_active":"2023-10-16T20:07:27.213052+0000","last_peered":"2023-10-16T20:07:27.213052+0000","last_clean":"2023-10-16T20:07:27.213052+0000","last_became_active":"2023-10-16T20:07:26.240444+0000","last_became_peered":"2023-10-16T20:07:26.240444+0000","last_unstale":"2023-10-16T20:07:27.213052+0000","last_undegraded":"2023-10-16T20:07:27.213052+0000","last_fullsized":"2023-10-16T20:07:27.213052+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:24:40.444553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213314+0000","last_change":"2023-10-16T20:07:26.224440+0000","last_active":"2023-10-16T20:07:27.213314+0000","last_peered":"2023-10-16T20:07:27.213314+0000","last_clean":"2023-10-16T20:07:27.213314+0000","last_became_active":"2023-10-16T20:07:26.223541+0000","last_became_peered":"2023-10-16T20:07:26.223541+0000","last_unstale":"2023-10-16T20:07:27.213314+0000","last_undegraded":"2023-10-16T20:07:27.213314+0000","last_fullsized":"2023-10-16T20:07:27.213314+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:31:52.974483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795284+0000","last_change":"2023-10-16T20:07:26.235912+0000","last_active":"2023-10-16T20:07:27.795284+0000","last_peered":"2023-10-16T20:07:27.795284+0000","last_clean":"2023-10-16T20:07:27.795284+0000","last_became_active":"2023-10-16T20:07:26.235620+0000","last_became_peered":"2023-10-16T20:07:26.235620+0000","last_unstale":"2023-10-16T20:07:27.795284+0000","last_undegraded":"2023-10-16T20:07:27.795284+0000","last_fullsized":"2023-10-16T20:07:27.795284+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:00:25.064493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795011+0000","last_change":"2023-10-16T20:07:26.228748+0000","last_active":"2023-10-16T20:07:27.795011+0000","last_peered":"2023-10-16T20:07:27.795011+0000","last_clean":"2023-10-16T20:07:27.795011+0000","last_became_active":"2023-10-16T20:07:26.228459+0000","last_became_peered":"2023-10-16T20:07:26.228459+0000","last_unstale":"2023-10-16T20:07:27.795011+0000","last_undegraded":"2023-10-16T20:07:27.795011+0000","last_fullsized":"2023-10-16T20:07:27.795011+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:48:10.263086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-10-16T20:07:25.275831+0000","last_change":"2023-10-16T20:07:25.275831+0000","last_active":"2023-10-16T20:07:25.253517+0000","last_peered":"2023-10-16T20:07:25.253517+0000","last_clean":"2023-10-16T20:07:25.253517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T20:07:25.275831+0000","last_undegraded":"2023-10-16T20:07:25.275831+0000","last_fullsized":"2023-10-16T20:07:25.275831+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794121+0000","last_change":"2023-10-16T20:07:26.227073+0000","last_active":"2023-10-16T20:07:27.794121+0000","last_peered":"2023-10-16T20:07:27.794121+0000","last_clean":"2023-10-16T20:07:27.794121+0000","last_became_active":"2023-10-16T20:07:26.226795+0000","last_became_peered":"2023-10-16T20:07:26.226795+0000","last_unstale":"2023-10-16T20:07:27.794121+0000","last_undegraded":"2023-10-16T20:07:27.794121+0000","last_fullsized":"2023-10-16T20:07:27.794121+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:39:31.048341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211891+0000","last_change":"2023-10-16T20:07:26.233109+0000","last_active":"2023-10-16T20:07:27.211891+0000","last_peered":"2023-10-16T20:07:27.211891+0000","last_clean":"2023-10-16T20:07:27.211891+0000","last_became_active":"2023-10-16T20:07:26.232756+0000","last_became_peered":"2023-10-16T20:07:26.232756+0000","last_unstale":"2023-10-16T20:07:27.211891+0000","last_undegraded":"2023-10-16T20:07:27.211891+0000","last_fullsized":"2023-10-16T20:07:27.211891+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:15:17.549212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.212860+0000","last_change":"2023-10-16T20:07:26.231271+0000","last_active":"2023-10-16T20:07:27.212860+0000","last_peered":"2023-10-16T20:07:27.212860+0000","last_clean":"2023-10-16T20:07:27.212860+0000","last_became_active":"2023-10-16T20:07:26.230979+0000","last_became_peered":"2023-10-16T20:07:26.230979+0000","last_unstale":"2023-10-16T20:07:27.212860+0000","last_undegraded":"2023-10-16T20:07:27.212860+0000","last_fullsized":"2023-10-16T20:07:27.212860+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:59:56.928721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.793827+0000","last_change":"2023-10-16T20:07:26.226909+0000","last_active":"2023-10-16T20:07:27.793827+0000","last_peered":"2023-10-16T20:07:27.793827+0000","last_clean":"2023-10-16T20:07:27.793827+0000","last_became_active":"2023-10-16T20:07:26.225946+0000","last_became_peered":"2023-10-16T20:07:26.225946+0000","last_unstale":"2023-10-16T20:07:27.793827+0000","last_undegraded":"2023-10-16T20:07:27.793827+0000","last_fullsized":"2023-10-16T20:07:27.793827+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:32:39.900878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-10-16T20:07:25.269783+0000","last_change":"2023-10-16T20:07:25.269783+0000","last_active":"2023-10-16T20:07:25.253517+0000","last_peered":"2023-10-16T20:07:25.253517+0000","last_clean":"2023-10-16T20:07:25.253517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T20:07:25.269783+0000","last_undegraded":"2023-10-16T20:07:25.269783+0000","last_fullsized":"2023-10-16T20:07:25.269783+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-10-16T20:07:25.272214+0000","last_change":"2023-10-16T20:07:25.272214+0000","last_active":"2023-10-16T20:07:25.253517+0000","last_peered":"2023-10-16T20:07:25.253517+0000","last_clean":"2023-10-16T20:07:25.253517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T20:07:25.272214+0000","last_undegraded":"2023-10-16T20:07:25.272214+0000","last_fullsized":"2023-10-16T20:07:25.272214+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-16T20:07:25.289736+0000","last_change":"2023-10-16T20:07:25.282198+0000","last_active":"2023-10-16T20:07:25.289736+0000","last_peered":"2023-10-16T20:07:25.289736+0000","last_clean":"2023-10-16T20:07:25.289736+0000","last_became_active":"2023-10-16T20:07:25.281929+0000","last_became_peered":"2023-10-16T20:07:25.281929+0000","last_unstale":"2023-10-16T20:07:25.289736+0000","last_undegraded":"2023-10-16T20:07:25.289736+0000","last_fullsized":"2023-10-16T20:07:25.289736+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:53:06.346584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795651+0000","last_change":"2023-10-16T20:07:26.234958+0000","last_active":"2023-10-16T20:07:27.795651+0000","last_peered":"2023-10-16T20:07:27.795651+0000","last_clean":"2023-10-16T20:07:27.795651+0000","last_became_active":"2023-10-16T20:07:26.234607+0000","last_became_peered":"2023-10-16T20:07:26.234607+0000","last_unstale":"2023-10-16T20:07:27.795651+0000","last_undegraded":"2023-10-16T20:07:27.795651+0000","last_fullsized":"2023-10-16T20:07:27.795651+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:32:22.385642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211762+0000","last_change":"2023-10-16T20:07:25.288386+0000","last_active":"2023-10-16T20:07:27.211762+0000","last_peered":"2023-10-16T20:07:27.211762+0000","last_clean":"2023-10-16T20:07:27.211762+0000","last_became_active":"2023-10-16T20:07:25.288083+0000","last_became_peered":"2023-10-16T20:07:25.288083+0000","last_unstale":"2023-10-16T20:07:27.211762+0000","last_undegraded":"2023-10-16T20:07:27.211762+0000","last_fullsized":"2023-10-16T20:07:27.211762+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:07:40.203583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-10-16T20:07:25.271490+0000","last_change":"2023-10-16T20:07:25.271490+0000","last_active":"2023-10-16T20:07:25.253517+0000","last_peered":"2023-10-16T20:07:25.253517+0000","last_clean":"2023-10-16T20:07:25.253517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T20:07:25.271490+0000","last_undegraded":"2023-10-16T20:07:25.271490+0000","last_fullsized":"2023-10-16T20:07:25.271490+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-16T20:07:25.299552+0000","last_change":"2023-10-16T20:07:25.298262+0000","last_active":"2023-10-16T20:07:25.299552+0000","last_peered":"2023-10-16T20:07:25.299552+0000","last_clean":"2023-10-16T20:07:25.299552+0000","last_became_active":"2023-10-16T20:07:25.297972+0000","last_became_peered":"2023-10-16T20:07:25.297972+0000","last_unstale":"2023-10-16T20:07:25.299552+0000","last_undegraded":"2023-10-16T20:07:25.299552+0000","last_fullsized":"2023-10-16T20:07:25.299552+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:10:50.848372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-10-16T20:07:25.271193+0000","last_change":"2023-10-16T20:07:25.271193+0000","last_active":"2023-10-16T20:07:25.253517+0000","last_peered":"2023-10-16T20:07:25.253517+0000","last_clean":"2023-10-16T20:07:25.253517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T20:07:25.271193+0000","last_undegraded":"2023-10-16T20:07:25.271193+0000","last_fullsized":"2023-10-16T20:07:25.271193+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"57'2","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795402+0000","last_change":"2023-10-16T20:07:25.278253+0000","last_active":"2023-10-16T20:07:27.795402+0000","last_peered":"2023-10-16T20:07:27.795402+0000","last_clean":"2023-10-16T20:07:27.795402+0000","last_became_active":"2023-10-16T20:07:25.277952+0000","last_became_peered":"2023-10-16T20:07:25.277952+0000","last_unstale":"2023-10-16T20:07:27.795402+0000","last_undegraded":"2023-10-16T20:07:27.795402+0000","last_fullsized":"2023-10-16T20:07:27.795402+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:21:59.522984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211501+0000","last_change":"2023-10-16T20:07:26.240164+0000","last_active":"2023-10-16T20:07:27.211501+0000","last_peered":"2023-10-16T20:07:27.211501+0000","last_clean":"2023-10-16T20:07:27.211501+0000","last_became_active":"2023-10-16T20:07:26.238843+0000","last_became_peered":"2023-10-16T20:07:26.238843+0000","last_unstale":"2023-10-16T20:07:27.211501+0000","last_undegraded":"2023-10-16T20:07:27.211501+0000","last_fullsized":"2023-10-16T20:07:27.211501+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:06:05.571248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794813+0000","last_change":"2023-10-16T20:07:25.287011+0000","last_active":"2023-10-16T20:07:27.794813+0000","last_peered":"2023-10-16T20:07:27.794813+0000","last_clean":"2023-10-16T20:07:27.794813+0000","last_became_active":"2023-10-16T20:07:25.286461+0000","last_became_peered":"2023-10-16T20:07:25.286461+0000","last_unstale":"2023-10-16T20:07:27.794813+0000","last_undegraded":"2023-10-16T20:07:27.794813+0000","last_fullsized":"2023-10-16T20:07:27.794813+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:46:43.335284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211670+0000","last_change":"2023-10-16T20:07:26.234473+0000","last_active":"2023-10-16T20:07:27.211670+0000","last_peered":"2023-10-16T20:07:27.211670+0000","last_clean":"2023-10-16T20:07:27.211670+0000","last_became_active":"2023-10-16T20:07:26.231718+0000","last_became_peered":"2023-10-16T20:07:26.231718+0000","last_unstale":"2023-10-16T20:07:27.211670+0000","last_undegraded":"2023-10-16T20:07:27.211670+0000","last_fullsized":"2023-10-16T20:07:27.211670+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:20:50.294097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-16T20:07:25.284514+0000","last_change":"2023-10-16T20:07:25.280487+0000","last_active":"2023-10-16T20:07:25.284514+0000","last_peered":"2023-10-16T20:07:25.284514+0000","last_clean":"2023-10-16T20:07:25.284514+0000","last_became_active":"2023-10-16T20:07:25.280197+0000","last_became_peered":"2023-10-16T20:07:25.280197+0000","last_unstale":"2023-10-16T20:07:25.284514+0000","last_undegraded":"2023-10-16T20:07:25.284514+0000","last_fullsized":"2023-10-16T20:07:25.284514+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:04:58.332866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213447+0000","last_change":"2023-10-16T20:07:26.225259+0000","last_active":"2023-10-16T20:07:27.213447+0000","last_peered":"2023-10-16T20:07:27.213447+0000","last_clean":"2023-10-16T20:07:27.213447+0000","last_became_active":"2023-10-16T20:07:26.224788+0000","last_became_peered":"2023-10-16T20:07:26.224788+0000","last_unstale":"2023-10-16T20:07:27.213447+0000","last_undegraded":"2023-10-16T20:07:27.213447+0000","last_fullsized":"2023-10-16T20:07:27.213447+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:50:26.015829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794639+0000","last_change":"2023-10-16T20:07:25.277974+0000","last_active":"2023-10-16T20:07:27.794639+0000","last_peered":"2023-10-16T20:07:27.794639+0000","last_clean":"2023-10-16T20:07:27.794639+0000","last_became_active":"2023-10-16T20:07:25.276748+0000","last_became_peered":"2023-10-16T20:07:25.276748+0000","last_unstale":"2023-10-16T20:07:27.794639+0000","last_undegraded":"2023-10-16T20:07:27.794639+0000","last_fullsized":"2023-10-16T20:07:27.794639+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:21:47.873107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211677+0000","last_change":"2023-10-16T20:07:26.231756+0000","last_active":"2023-10-16T20:07:27.211677+0000","last_peered":"2023-10-16T20:07:27.211677+0000","last_clean":"2023-10-16T20:07:27.211677+0000","last_became_active":"2023-10-16T20:07:26.231500+0000","last_became_peered":"2023-10-16T20:07:26.231500+0000","last_unstale":"2023-10-16T20:07:27.211677+0000","last_undegraded":"2023-10-16T20:07:27.211677+0000","last_fullsized":"2023-10-16T20:07:27.211677+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:11:19.933794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794560+0000","last_change":"2023-10-16T20:07:25.280055+0000","last_active":"2023-10-16T20:07:27.794560+0000","last_peered":"2023-10-16T20:07:27.794560+0000","last_clean":"2023-10-16T20:07:27.794560+0000","last_became_active":"2023-10-16T20:07:25.279711+0000","last_became_peered":"2023-10-16T20:07:25.279711+0000","last_unstale":"2023-10-16T20:07:27.794560+0000","last_undegraded":"2023-10-16T20:07:27.794560+0000","last_fullsized":"2023-10-16T20:07:27.794560+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:47:48.571528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795692+0000","last_change":"2023-10-16T20:07:26.234101+0000","last_active":"2023-10-16T20:07:27.795692+0000","last_peered":"2023-10-16T20:07:27.795692+0000","last_clean":"2023-10-16T20:07:27.795692+0000","last_became_active":"2023-10-16T20:07:26.233286+0000","last_became_peered":"2023-10-16T20:07:26.233286+0000","last_unstale":"2023-10-16T20:07:27.795692+0000","last_undegraded":"2023-10-16T20:07:27.795692+0000","last_fullsized":"2023-10-16T20:07:27.795692+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:12:55.180570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213511+0000","last_change":"2023-10-16T20:07:25.291493+0000","last_active":"2023-10-16T20:07:27.213511+0000","last_peered":"2023-10-16T20:07:27.213511+0000","last_clean":"2023-10-16T20:07:27.213511+0000","last_became_active":"2023-10-16T20:07:25.291189+0000","last_became_peered":"2023-10-16T20:07:25.291189+0000","last_unstale":"2023-10-16T20:07:27.213511+0000","last_undegraded":"2023-10-16T20:07:27.213511+0000","last_fullsized":"2023-10-16T20:07:27.213511+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:21:59.522984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213424+0000","last_change":"2023-10-16T20:07:26.237797+0000","last_active":"2023-10-16T20:07:27.213424+0000","last_peered":"2023-10-16T20:07:27.213424+0000","last_clean":"2023-10-16T20:07:27.213424+0000","last_became_active":"2023-10-16T20:07:26.237483+0000","last_became_peered":"2023-10-16T20:07:26.237483+0000","last_unstale":"2023-10-16T20:07:27.213424+0000","last_undegraded":"2023-10-16T20:07:27.213424+0000","last_fullsized":"2023-10-16T20:07:27.213424+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:52:43.995476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211842+0000","last_change":"2023-10-16T20:07:25.286755+0000","last_active":"2023-10-16T20:07:27.211842+0000","last_peered":"2023-10-16T20:07:27.211842+0000","last_clean":"2023-10-16T20:07:27.211842+0000","last_became_active":"2023-10-16T20:07:25.286230+0000","last_became_peered":"2023-10-16T20:07:25.286230+0000","last_unstale":"2023-10-16T20:07:27.211842+0000","last_undegraded":"2023-10-16T20:07:27.211842+0000","last_fullsized":"2023-10-16T20:07:27.211842+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:48:09.259823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795180+0000","last_change":"2023-10-16T20:07:26.231320+0000","last_active":"2023-10-16T20:07:27.795180+0000","last_peered":"2023-10-16T20:07:27.795180+0000","last_clean":"2023-10-16T20:07:27.795180+0000","last_became_active":"2023-10-16T20:07:26.230814+0000","last_became_peered":"2023-10-16T20:07:26.230814+0000","last_unstale":"2023-10-16T20:07:27.795180+0000","last_undegraded":"2023-10-16T20:07:27.795180+0000","last_fullsized":"2023-10-16T20:07:27.795180+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:19:45.099223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213843+0000","last_change":"2023-10-16T20:07:25.298302+0000","last_active":"2023-10-16T20:07:27.213843+0000","last_peered":"2023-10-16T20:07:27.213843+0000","last_clean":"2023-10-16T20:07:27.213843+0000","last_became_active":"2023-10-16T20:07:25.298040+0000","last_became_peered":"2023-10-16T20:07:25.298040+0000","last_unstale":"2023-10-16T20:07:27.213843+0000","last_undegraded":"2023-10-16T20:07:27.213843+0000","last_fullsized":"2023-10-16T20:07:27.213843+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:01:51.549639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213679+0000","last_change":"2023-10-16T20:07:26.235759+0000","last_active":"2023-10-16T20:07:27.213679+0000","last_peered":"2023-10-16T20:07:27.213679+0000","last_clean":"2023-10-16T20:07:27.213679+0000","last_became_active":"2023-10-16T20:07:26.235472+0000","last_became_peered":"2023-10-16T20:07:26.235472+0000","last_unstale":"2023-10-16T20:07:27.213679+0000","last_undegraded":"2023-10-16T20:07:27.213679+0000","last_fullsized":"2023-10-16T20:07:27.213679+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:00:35.644195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795164+0000","last_change":"2023-10-16T20:07:25.282309+0000","last_active":"2023-10-16T20:07:27.795164+0000","last_peered":"2023-10-16T20:07:27.795164+0000","last_clean":"2023-10-16T20:07:27.795164+0000","last_became_active":"2023-10-16T20:07:25.281997+0000","last_became_peered":"2023-10-16T20:07:25.281997+0000","last_unstale":"2023-10-16T20:07:27.795164+0000","last_undegraded":"2023-10-16T20:07:27.795164+0000","last_fullsized":"2023-10-16T20:07:27.795164+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:31:51.971221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794630+0000","last_change":"2023-10-16T20:07:26.236069+0000","last_active":"2023-10-16T20:07:27.794630+0000","last_peered":"2023-10-16T20:07:27.794630+0000","last_clean":"2023-10-16T20:07:27.794630+0000","last_became_active":"2023-10-16T20:07:26.235798+0000","last_became_peered":"2023-10-16T20:07:26.235798+0000","last_unstale":"2023-10-16T20:07:27.794630+0000","last_undegraded":"2023-10-16T20:07:27.794630+0000","last_fullsized":"2023-10-16T20:07:27.794630+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:24:24.649666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211759+0000","last_change":"2023-10-16T20:07:25.288129+0000","last_active":"2023-10-16T20:07:27.211759+0000","last_peered":"2023-10-16T20:07:27.211759+0000","last_clean":"2023-10-16T20:07:27.211759+0000","last_became_active":"2023-10-16T20:07:25.287112+0000","last_became_peered":"2023-10-16T20:07:25.287112+0000","last_unstale":"2023-10-16T20:07:27.211759+0000","last_undegraded":"2023-10-16T20:07:27.211759+0000","last_fullsized":"2023-10-16T20:07:27.211759+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:24:23.646404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213241+0000","last_change":"2023-10-16T20:07:26.227260+0000","last_active":"2023-10-16T20:07:27.213241+0000","last_peered":"2023-10-16T20:07:27.213241+0000","last_clean":"2023-10-16T20:07:27.213241+0000","last_became_active":"2023-10-16T20:07:26.226926+0000","last_became_peered":"2023-10-16T20:07:26.226926+0000","last_unstale":"2023-10-16T20:07:27.213241+0000","last_undegraded":"2023-10-16T20:07:27.213241+0000","last_fullsized":"2023-10-16T20:07:27.213241+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:26:50.399490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213734+0000","last_change":"2023-10-16T20:07:25.287377+0000","last_active":"2023-10-16T20:07:27.213734+0000","last_peered":"2023-10-16T20:07:27.213734+0000","last_clean":"2023-10-16T20:07:27.213734+0000","last_became_active":"2023-10-16T20:07:25.286994+0000","last_became_peered":"2023-10-16T20:07:25.286994+0000","last_unstale":"2023-10-16T20:07:27.213734+0000","last_undegraded":"2023-10-16T20:07:27.213734+0000","last_fullsized":"2023-10-16T20:07:27.213734+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:16:43.918657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795364+0000","last_change":"2023-10-16T20:07:26.229963+0000","last_active":"2023-10-16T20:07:27.795364+0000","last_peered":"2023-10-16T20:07:27.795364+0000","last_clean":"2023-10-16T20:07:27.795364+0000","last_became_active":"2023-10-16T20:07:26.229654+0000","last_became_peered":"2023-10-16T20:07:26.229654+0000","last_unstale":"2023-10-16T20:07:27.795364+0000","last_undegraded":"2023-10-16T20:07:27.795364+0000","last_fullsized":"2023-10-16T20:07:27.795364+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:06:41.715102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794719+0000","last_change":"2023-10-16T20:07:25.285312+0000","last_active":"2023-10-16T20:07:27.794719+0000","last_peered":"2023-10-16T20:07:27.794719+0000","last_clean":"2023-10-16T20:07:27.794719+0000","last_became_active":"2023-10-16T20:07:25.285025+0000","last_became_peered":"2023-10-16T20:07:25.285025+0000","last_unstale":"2023-10-16T20:07:27.794719+0000","last_undegraded":"2023-10-16T20:07:27.794719+0000","last_fullsized":"2023-10-16T20:07:27.794719+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:59:56.818441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213171+0000","last_change":"2023-10-16T20:07:26.241281+0000","last_active":"2023-10-16T20:07:27.213171+0000","last_peered":"2023-10-16T20:07:27.213171+0000","last_clean":"2023-10-16T20:07:27.213171+0000","last_became_active":"2023-10-16T20:07:26.240963+0000","last_became_peered":"2023-10-16T20:07:26.240963+0000","last_unstale":"2023-10-16T20:07:27.213171+0000","last_undegraded":"2023-10-16T20:07:27.213171+0000","last_fullsized":"2023-10-16T20:07:27.213171+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:07:41.206845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":20,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213458+0000","last_change":"2023-10-16T20:07:25.289444+0000","last_active":"2023-10-16T20:07:27.213458+0000","last_peered":"2023-10-16T20:07:27.213458+0000","last_clean":"2023-10-16T20:07:27.213458+0000","last_became_active":"2023-10-16T20:07:25.289138+0000","last_became_peered":"2023-10-16T20:07:25.289138+0000","last_unstale":"2023-10-16T20:07:27.213458+0000","last_undegraded":"2023-10-16T20:07:27.213458+0000","last_fullsized":"2023-10-16T20:07:27.213458+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:41:59.036278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213178+0000","last_change":"2023-10-16T20:07:26.238893+0000","last_active":"2023-10-16T20:07:27.213178+0000","last_peered":"2023-10-16T20:07:27.213178+0000","last_clean":"2023-10-16T20:07:27.213178+0000","last_became_active":"2023-10-16T20:07:26.238253+0000","last_became_peered":"2023-10-16T20:07:26.238253+0000","last_unstale":"2023-10-16T20:07:27.213178+0000","last_undegraded":"2023-10-16T20:07:27.213178+0000","last_fullsized":"2023-10-16T20:07:27.213178+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:48:10.263086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211987+0000","last_change":"2023-10-16T20:07:25.286326+0000","last_active":"2023-10-16T20:07:27.211987+0000","last_peered":"2023-10-16T20:07:27.211987+0000","last_clean":"2023-10-16T20:07:27.211987+0000","last_became_active":"2023-10-16T20:07:25.286056+0000","last_became_peered":"2023-10-16T20:07:25.286056+0000","last_unstale":"2023-10-16T20:07:27.211987+0000","last_undegraded":"2023-10-16T20:07:27.211987+0000","last_fullsized":"2023-10-16T20:07:27.211987+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:48:04.524856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795523+0000","last_change":"2023-10-16T20:07:26.234861+0000","last_active":"2023-10-16T20:07:27.795523+0000","last_peered":"2023-10-16T20:07:27.795523+0000","last_clean":"2023-10-16T20:07:27.795523+0000","last_became_active":"2023-10-16T20:07:26.234532+0000","last_became_peered":"2023-10-16T20:07:26.234532+0000","last_unstale":"2023-10-16T20:07:27.795523+0000","last_undegraded":"2023-10-16T20:07:27.795523+0000","last_fullsized":"2023-10-16T20:07:27.795523+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:00:35.644195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-16T20:07:25.290803+0000","last_change":"2023-10-16T20:07:25.288050+0000","last_active":"2023-10-16T20:07:25.290803+0000","last_peered":"2023-10-16T20:07:25.290803+0000","last_clean":"2023-10-16T20:07:25.290803+0000","last_became_active":"2023-10-16T20:07:25.287781+0000","last_became_peered":"2023-10-16T20:07:25.287781+0000","last_unstale":"2023-10-16T20:07:25.290803+0000","last_undegraded":"2023-10-16T20:07:25.290803+0000","last_fullsized":"2023-10-16T20:07:25.290803+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:08:56.710610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211579+0000","last_change":"2023-10-16T20:07:26.237328+0000","last_active":"2023-10-16T20:07:27.211579+0000","last_peered":"2023-10-16T20:07:27.211579+0000","last_clean":"2023-10-16T20:07:27.211579+0000","last_became_active":"2023-10-16T20:07:26.237047+0000","last_became_peered":"2023-10-16T20:07:26.237047+0000","last_unstale":"2023-10-16T20:07:27.211579+0000","last_undegraded":"2023-10-16T20:07:27.211579+0000","last_fullsized":"2023-10-16T20:07:27.211579+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:17:40.171938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"57'1","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795196+0000","last_change":"2023-10-16T20:07:25.288443+0000","last_active":"2023-10-16T20:07:27.795196+0000","last_peered":"2023-10-16T20:07:27.795196+0000","last_clean":"2023-10-16T20:07:27.795196+0000","last_became_active":"2023-10-16T20:07:25.288154+0000","last_became_peered":"2023-10-16T20:07:25.288154+0000","last_unstale":"2023-10-16T20:07:27.795196+0000","last_undegraded":"2023-10-16T20:07:27.795196+0000","last_fullsized":"2023-10-16T20:07:27.795196+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:00:13.174520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.214028+0000","last_change":"2023-10-16T20:07:26.235840+0000","last_active":"2023-10-16T20:07:27.214028+0000","last_peered":"2023-10-16T20:07:27.214028+0000","last_clean":"2023-10-16T20:07:27.214028+0000","last_became_active":"2023-10-16T20:07:26.235489+0000","last_became_peered":"2023-10-16T20:07:26.235489+0000","last_unstale":"2023-10-16T20:07:27.214028+0000","last_undegraded":"2023-10-16T20:07:27.214028+0000","last_fullsized":"2023-10-16T20:07:27.214028+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:52:00.457041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-16T20:07:25.290853+0000","last_change":"2023-10-16T20:07:25.286338+0000","last_active":"2023-10-16T20:07:25.290853+0000","last_peered":"2023-10-16T20:07:25.290853+0000","last_clean":"2023-10-16T20:07:25.290853+0000","last_became_active":"2023-10-16T20:07:25.286056+0000","last_became_peered":"2023-10-16T20:07:25.286056+0000","last_unstale":"2023-10-16T20:07:25.290853+0000","last_undegraded":"2023-10-16T20:07:25.290853+0000","last_fullsized":"2023-10-16T20:07:25.290853+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:15:07.982791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794861+0000","last_change":"2023-10-16T20:07:26.228461+0000","last_active":"2023-10-16T20:07:27.794861+0000","last_peered":"2023-10-16T20:07:27.794861+0000","last_clean":"2023-10-16T20:07:27.794861+0000","last_became_active":"2023-10-16T20:07:26.228148+0000","last_became_peered":"2023-10-16T20:07:26.228148+0000","last_unstale":"2023-10-16T20:07:27.794861+0000","last_undegraded":"2023-10-16T20:07:27.794861+0000","last_fullsized":"2023-10-16T20:07:27.794861+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:07:41.206845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211975+0000","last_change":"2023-10-16T20:07:25.287428+0000","last_active":"2023-10-16T20:07:27.211975+0000","last_peered":"2023-10-16T20:07:27.211975+0000","last_clean":"2023-10-16T20:07:27.211975+0000","last_became_active":"2023-10-16T20:07:25.287146+0000","last_became_peered":"2023-10-16T20:07:25.287146+0000","last_unstale":"2023-10-16T20:07:27.211975+0000","last_undegraded":"2023-10-16T20:07:27.211975+0000","last_fullsized":"2023-10-16T20:07:27.211975+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:26:49.396227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-10-16T20:07:25.274784+0000","last_change":"2023-10-16T20:07:25.274784+0000","last_active":"2023-10-16T20:07:25.253517+0000","last_peered":"2023-10-16T20:07:25.253517+0000","last_clean":"2023-10-16T20:07:25.253517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T20:07:25.274784+0000","last_undegraded":"2023-10-16T20:07:25.274784+0000","last_fullsized":"2023-10-16T20:07:25.274784+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794887+0000","last_change":"2023-10-16T20:07:25.293106+0000","last_active":"2023-10-16T20:07:27.794887+0000","last_peered":"2023-10-16T20:07:27.794887+0000","last_clean":"2023-10-16T20:07:27.794887+0000","last_became_active":"2023-10-16T20:07:25.292583+0000","last_became_peered":"2023-10-16T20:07:25.292583+0000","last_unstale":"2023-10-16T20:07:27.794887+0000","last_undegraded":"2023-10-16T20:07:27.794887+0000","last_fullsized":"2023-10-16T20:07:27.794887+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:11:57.982645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-10-16T20:07:25.269695+0000","last_change":"2023-10-16T20:07:25.269695+0000","last_active":"2023-10-16T20:07:25.253517+0000","last_peered":"2023-10-16T20:07:25.253517+0000","last_clean":"2023-10-16T20:07:25.253517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T20:07:25.269695+0000","last_undegraded":"2023-10-16T20:07:25.269695+0000","last_fullsized":"2023-10-16T20:07:25.269695+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794652+0000","last_change":"2023-10-16T20:07:25.286566+0000","last_active":"2023-10-16T20:07:27.794652+0000","last_peered":"2023-10-16T20:07:27.794652+0000","last_clean":"2023-10-16T20:07:27.794652+0000","last_became_active":"2023-10-16T20:07:25.285812+0000","last_became_peered":"2023-10-16T20:07:25.285812+0000","last_unstale":"2023-10-16T20:07:27.794652+0000","last_undegraded":"2023-10-16T20:07:27.794652+0000","last_fullsized":"2023-10-16T20:07:27.794652+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:33:14.446674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795249+0000","last_change":"2023-10-16T20:07:26.219929+0000","last_active":"2023-10-16T20:07:27.795249+0000","last_peered":"2023-10-16T20:07:27.795249+0000","last_clean":"2023-10-16T20:07:27.795249+0000","last_became_active":"2023-10-16T20:07:26.219429+0000","last_became_peered":"2023-10-16T20:07:26.219429+0000","last_unstale":"2023-10-16T20:07:27.795249+0000","last_undegraded":"2023-10-16T20:07:27.795249+0000","last_fullsized":"2023-10-16T20:07:27.795249+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:48:05.528118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"57'2","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795653+0000","last_change":"2023-10-16T20:07:25.294532+0000","last_active":"2023-10-16T20:07:27.795653+0000","last_peered":"2023-10-16T20:07:27.795653+0000","last_clean":"2023-10-16T20:07:27.795653+0000","last_became_active":"2023-10-16T20:07:25.294069+0000","last_became_peered":"2023-10-16T20:07:25.294069+0000","last_unstale":"2023-10-16T20:07:27.795653+0000","last_undegraded":"2023-10-16T20:07:27.795653+0000","last_fullsized":"2023-10-16T20:07:27.795653+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:19:54.366332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795505+0000","last_change":"2023-10-16T20:07:26.227126+0000","last_active":"2023-10-16T20:07:27.795505+0000","last_peered":"2023-10-16T20:07:27.795505+0000","last_clean":"2023-10-16T20:07:27.795505+0000","last_became_active":"2023-10-16T20:07:26.226825+0000","last_became_peered":"2023-10-16T20:07:26.226825+0000","last_unstale":"2023-10-16T20:07:27.795505+0000","last_undegraded":"2023-10-16T20:07:27.795505+0000","last_fullsized":"2023-10-16T20:07:27.795505+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:07:42.996110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-16T20:07:25.290452+0000","last_change":"2023-10-16T20:07:25.283798+0000","last_active":"2023-10-16T20:07:25.290452+0000","last_peered":"2023-10-16T20:07:25.290452+0000","last_clean":"2023-10-16T20:07:25.290452+0000","last_became_active":"2023-10-16T20:07:25.283541+0000","last_became_peered":"2023-10-16T20:07:25.283541+0000","last_unstale":"2023-10-16T20:07:25.290452+0000","last_undegraded":"2023-10-16T20:07:25.290452+0000","last_fullsized":"2023-10-16T20:07:25.290452+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:58:24.840735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795328+0000","last_change":"2023-10-16T20:07:26.231142+0000","last_active":"2023-10-16T20:07:27.795328+0000","last_peered":"2023-10-16T20:07:27.795328+0000","last_clean":"2023-10-16T20:07:27.795328+0000","last_became_active":"2023-10-16T20:07:26.230516+0000","last_became_peered":"2023-10-16T20:07:26.230516+0000","last_unstale":"2023-10-16T20:07:27.795328+0000","last_undegraded":"2023-10-16T20:07:27.795328+0000","last_fullsized":"2023-10-16T20:07:27.795328+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:12:55.180570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213956+0000","last_change":"2023-10-16T20:07:25.279560+0000","last_active":"2023-10-16T20:07:27.213956+0000","last_peered":"2023-10-16T20:07:27.213956+0000","last_clean":"2023-10-16T20:07:27.213956+0000","last_became_active":"2023-10-16T20:07:25.278610+0000","last_became_peered":"2023-10-16T20:07:25.278610+0000","last_unstale":"2023-10-16T20:07:27.213956+0000","last_undegraded":"2023-10-16T20:07:27.213956+0000","last_fullsized":"2023-10-16T20:07:27.213956+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:57:47.332831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-10-16T20:07:25.270337+0000","last_change":"2023-10-16T20:07:25.270337+0000","last_active":"2023-10-16T20:07:25.253517+0000","last_peered":"2023-10-16T20:07:25.253517+0000","last_clean":"2023-10-16T20:07:25.253517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T20:07:25.270337+0000","last_undegraded":"2023-10-16T20:07:25.270337+0000","last_fullsized":"2023-10-16T20:07:25.270337+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795499+0000","last_change":"2023-10-16T20:07:25.277203+0000","last_active":"2023-10-16T20:07:27.795499+0000","last_peered":"2023-10-16T20:07:27.795499+0000","last_clean":"2023-10-16T20:07:27.795499+0000","last_became_active":"2023-10-16T20:07:25.276442+0000","last_became_peered":"2023-10-16T20:07:25.276442+0000","last_unstale":"2023-10-16T20:07:27.795499+0000","last_undegraded":"2023-10-16T20:07:27.795499+0000","last_fullsized":"2023-10-16T20:07:27.795499+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:59:55.925458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211630+0000","last_change":"2023-10-16T20:07:26.232496+0000","last_active":"2023-10-16T20:07:27.211630+0000","last_peered":"2023-10-16T20:07:27.211630+0000","last_clean":"2023-10-16T20:07:27.211630+0000","last_became_active":"2023-10-16T20:07:26.232250+0000","last_became_peered":"2023-10-16T20:07:26.232250+0000","last_unstale":"2023-10-16T20:07:27.211630+0000","last_undegraded":"2023-10-16T20:07:27.211630+0000","last_fullsized":"2023-10-16T20:07:27.211630+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:31:02.703353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795042+0000","last_change":"2023-10-16T20:07:25.292231+0000","last_active":"2023-10-16T20:07:27.795042+0000","last_peered":"2023-10-16T20:07:27.795042+0000","last_clean":"2023-10-16T20:07:27.795042+0000","last_became_active":"2023-10-16T20:07:25.291896+0000","last_became_peered":"2023-10-16T20:07:25.291896+0000","last_unstale":"2023-10-16T20:07:27.795042+0000","last_undegraded":"2023-10-16T20:07:27.795042+0000","last_fullsized":"2023-10-16T20:07:27.795042+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:48:16.846875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213397+0000","last_change":"2023-10-16T20:07:26.240662+0000","last_active":"2023-10-16T20:07:27.213397+0000","last_peered":"2023-10-16T20:07:27.213397+0000","last_clean":"2023-10-16T20:07:27.213397+0000","last_became_active":"2023-10-16T20:07:26.240340+0000","last_became_peered":"2023-10-16T20:07:26.240340+0000","last_unstale":"2023-10-16T20:07:27.213397+0000","last_undegraded":"2023-10-16T20:07:27.213397+0000","last_fullsized":"2023-10-16T20:07:27.213397+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:48:05.528118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795183+0000","last_change":"2023-10-16T20:07:25.284134+0000","last_active":"2023-10-16T20:07:27.795183+0000","last_peered":"2023-10-16T20:07:27.795183+0000","last_clean":"2023-10-16T20:07:27.795183+0000","last_became_active":"2023-10-16T20:07:25.281555+0000","last_became_peered":"2023-10-16T20:07:25.281555+0000","last_unstale":"2023-10-16T20:07:27.795183+0000","last_undegraded":"2023-10-16T20:07:27.795183+0000","last_fullsized":"2023-10-16T20:07:27.795183+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:50:25.012567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794226+0000","last_change":"2023-10-16T20:07:26.230788+0000","last_active":"2023-10-16T20:07:27.794226+0000","last_peered":"2023-10-16T20:07:27.794226+0000","last_clean":"2023-10-16T20:07:27.794226+0000","last_became_active":"2023-10-16T20:07:26.230419+0000","last_became_peered":"2023-10-16T20:07:26.230419+0000","last_unstale":"2023-10-16T20:07:27.794226+0000","last_undegraded":"2023-10-16T20:07:27.794226+0000","last_fullsized":"2023-10-16T20:07:27.794226+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:06:41.715102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794377+0000","last_change":"2023-10-16T20:07:25.291904+0000","last_active":"2023-10-16T20:07:27.794377+0000","last_peered":"2023-10-16T20:07:27.794377+0000","last_clean":"2023-10-16T20:07:27.794377+0000","last_became_active":"2023-10-16T20:07:25.291633+0000","last_became_peered":"2023-10-16T20:07:25.291633+0000","last_unstale":"2023-10-16T20:07:27.794377+0000","last_undegraded":"2023-10-16T20:07:27.794377+0000","last_fullsized":"2023-10-16T20:07:27.794377+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:50.000078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211440+0000","last_change":"2023-10-16T20:07:26.232488+0000","last_active":"2023-10-16T20:07:27.211440+0000","last_peered":"2023-10-16T20:07:27.211440+0000","last_clean":"2023-10-16T20:07:27.211440+0000","last_became_active":"2023-10-16T20:07:26.232162+0000","last_became_peered":"2023-10-16T20:07:26.232162+0000","last_unstale":"2023-10-16T20:07:27.211440+0000","last_undegraded":"2023-10-16T20:07:27.211440+0000","last_fullsized":"2023-10-16T20:07:27.211440+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:52:00.457041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.212059+0000","last_change":"2023-10-16T20:07:25.288746+0000","last_active":"2023-10-16T20:07:27.212059+0000","last_peered":"2023-10-16T20:07:27.212059+0000","last_clean":"2023-10-16T20:07:27.212059+0000","last_became_active":"2023-10-16T20:07:25.288469+0000","last_became_peered":"2023-10-16T20:07:25.288469+0000","last_unstale":"2023-10-16T20:07:27.212059+0000","last_undegraded":"2023-10-16T20:07:27.212059+0000","last_fullsized":"2023-10-16T20:07:27.212059+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:08:50.174794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211357+0000","last_change":"2023-10-16T20:07:26.228623+0000","last_active":"2023-10-16T20:07:27.211357+0000","last_peered":"2023-10-16T20:07:27.211357+0000","last_clean":"2023-10-16T20:07:27.211357+0000","last_became_active":"2023-10-16T20:07:26.226488+0000","last_became_peered":"2023-10-16T20:07:26.226488+0000","last_unstale":"2023-10-16T20:07:27.211357+0000","last_undegraded":"2023-10-16T20:07:27.211357+0000","last_fullsized":"2023-10-16T20:07:27.211357+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:00:35.644195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213885+0000","last_change":"2023-10-16T20:07:25.279504+0000","last_active":"2023-10-16T20:07:27.213885+0000","last_peered":"2023-10-16T20:07:27.213885+0000","last_clean":"2023-10-16T20:07:27.213885+0000","last_became_active":"2023-10-16T20:07:25.279177+0000","last_became_peered":"2023-10-16T20:07:25.279177+0000","last_unstale":"2023-10-16T20:07:27.213885+0000","last_undegraded":"2023-10-16T20:07:27.213885+0000","last_fullsized":"2023-10-16T20:07:27.213885+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:18:10.920675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":197,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-16T20:07:26.219256+0000","last_change":"2023-10-16T20:04:46.335822+0000","last_active":"2023-10-16T20:07:26.219256+0000","last_peered":"2023-10-16T20:07:26.219256+0000","last_clean":"2023-10-16T20:07:26.219256+0000","last_became_active":"2023-10-16T20:04:46.334617+0000","last_became_peered":"2023-10-16T20:04:46.334617+0000","last_unstale":"2023-10-16T20:07:26.219256+0000","last_undegraded":"2023-10-16T20:07:26.219256+0000","last_fullsized":"2023-10-16T20:07:26.219256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:45.306240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:45.306240+0000","last_clean_scrub_stamp":"2023-10-16T20:04:45.306240+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:42:11.153721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1828,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":2626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":29,"ondisk_log_size":29,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509476,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1290240,"data_stored":898654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.70299999999999996,"15min":0.70899999999999996},"min":{"1min":0.372,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93400000000000005,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70999999999999996,"15min":0.70799999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.95299999999999996,"5min":1.077,"15min":1.077},"last":0.73299999999999998}]},{"osd":1,"last update":"Mon Oct 16 20:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.70099999999999996,"15min":0.70299999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98499999999999999,"5min":1.78,"15min":1.78},"last":0.745},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.70799999999999996,"15min":0.71599999999999997},"min":{"1min":0.45100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95899999999999996,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":0.65700000000000003}]},{"osd":2,"last update":"Mon Oct 16 20:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.73199999999999998,"15min":0.74399999999999999},"min":{"1min":0.38200000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84599999999999997,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.78700000000000003,"15min":0.79600000000000004},"min":{"1min":0.47399999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.026,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.86599999999999999}]},{"osd":3,"last update":"Mon Oct 16 20:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.70399999999999996,"15min":0.70699999999999996},"min":{"1min":0.52400000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98799999999999999,"5min":1.994,"15min":1.994},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.77200000000000002,"15min":0.79000000000000004},"min":{"1min":0.46700000000000003,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.92000000000000004,"5min":2.629,"15min":2.629},"last":0.91300000000000003}]},{"osd":4,"last update":"Mon Oct 16 20:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.64600000000000002,"15min":0.66000000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91300000000000003,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.65700000000000003,"15min":0.66200000000000003},"min":{"1min":0.42699999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84999999999999998,"5min":1.944,"15min":1.944},"last":0.58599999999999997}]},{"osd":5,"last update":"Mon Oct 16 20:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.73099999999999998,"15min":0.749},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.86699999999999999,"5min":2.8809999999999998,"15min":2.8809999999999998},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.71499999999999997,"15min":0.71699999999999997},"min":{"1min":0.55300000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.86499999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.54600000000000004}]},{"osd":6,"last update":"Mon Oct 16 20:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.754,"15min":0.75700000000000001},"min":{"1min":0.58099999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.96699999999999997,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.73599999999999999,"15min":0.749},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.85199999999999998,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.77300000000000002}]}]},{"osd":6,"up_from":15,"seq":64424509477,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31856,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":688128,"data_stored":308126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.85299999999999998,"15min":0.879},"min":{"1min":0.50800000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94899999999999995,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.83599999999999997,"15min":0.87},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90700000000000003,"5min":4.6139999999999999,"15min":4.6139999999999999},"last":0.51600000000000001}]},{"osd":1,"last update":"Mon Oct 16 20:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.879,"15min":0.90400000000000003},"min":{"1min":0.495,"5min":0.438,"15min":0.438},"max":{"1min":0.995,"5min":4.6639999999999997,"15min":4.6639999999999997},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":1.048,"15min":1.1080000000000001},"min":{"1min":0.44400000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90800000000000003,"5min":4.8140000000000001,"15min":4.8140000000000001},"last":0.61499999999999999}]},{"osd":2,"last update":"Mon Oct 16 20:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.93500000000000005,"15min":0.97399999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88400000000000001,"5min":5.3769999999999998,"15min":5.3769999999999998},"last":0.502},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.67700000000000005,"15min":0.67100000000000004},"min":{"1min":0.48299999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.54600000000000004}]},{"osd":3,"last update":"Mon Oct 16 20:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.91000000000000003,"15min":0.93799999999999994},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.97399999999999998,"5min":4.5460000000000003,"15min":4.5460000000000003},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.76100000000000001,"15min":0.77100000000000002},"min":{"1min":0.51200000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.92200000000000004,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.53300000000000003}]},{"osd":4,"last update":"Mon Oct 16 20:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.74199999999999999,"15min":0.754},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88300000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.69199999999999995,"15min":0.69899999999999995},"min":{"1min":0.45000000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.94599999999999995,"5min":1.355,"15min":1.355},"last":0.42399999999999999}]},{"osd":5,"last update":"Mon Oct 16 20:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.71299999999999997,"15min":0.72299999999999998},"min":{"1min":0.441,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88800000000000001,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.441},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80400000000000005,"15min":0.80700000000000005},"min":{"1min":0.57599999999999996,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.98699999999999999,"5min":1.218,"15min":1.218},"last":0.59399999999999997}]},{"osd":7,"last update":"Mon Oct 16 20:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.748,"15min":0.748},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.97799999999999998,"5min":1.052,"15min":1.052},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.79700000000000004,"15min":0.80300000000000005},"min":{"1min":0.52900000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.91300000000000003,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.48899999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509475,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":696320,"data_stored":308778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.63300000000000001,"15min":0.629},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.872,"5min":1.264,"15min":1.264},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.64500000000000002,"15min":0.64300000000000002},"min":{"1min":0.436,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.85499999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.64000000000000001}]},{"osd":1,"last update":"Mon Oct 16 20:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.61599999999999999,"15min":0.60799999999999998},"min":{"1min":0.46999999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.64600000000000002,"15min":0.64000000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.51100000000000001}]},{"osd":2,"last update":"Mon Oct 16 20:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67200000000000004,"15min":0.67000000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.85699999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.66000000000000003,"15min":0.65300000000000002},"min":{"1min":0.52700000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.66400000000000003}]},{"osd":3,"last update":"Mon Oct 16 20:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.77600000000000002,"15min":0.78200000000000003},"min":{"1min":0.498,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.92000000000000004,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.746,"5min":0.70799999999999996,"15min":0.70199999999999996},"min":{"1min":0.56200000000000006,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90400000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.70699999999999996}]},{"osd":4,"last update":"Mon Oct 16 20:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.42799999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91100000000000003,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.746},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70299999999999996,"15min":0.70199999999999996},"min":{"1min":0.44500000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.442}]},{"osd":6,"last update":"Mon Oct 16 20:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67000000000000004,"15min":0.66900000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73199999999999998,"15min":0.73099999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.61699999999999999}]},{"osd":7,"last update":"Mon Oct 16 20:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.75,"15min":0.75},"min":{"1min":0.54500000000000004,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.96999999999999997,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.74299999999999999,"15min":0.746},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.60199999999999998}]}]},{"osd":4,"up_from":15,"seq":64424509475,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":700416,"data_stored":309738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.77900000000000003,"15min":0.79900000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.81799999999999995,"5min":3.835,"15min":3.835},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.79400000000000004,"15min":0.81799999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.94799999999999995,"5min":3.7610000000000001,"15min":3.7610000000000001},"last":0.91700000000000004}]},{"osd":1,"last update":"Mon Oct 16 20:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67400000000000004,"15min":0.67600000000000005},"min":{"1min":0.48099999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89100000000000001,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.875},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.50600000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.89100000000000001}]},{"osd":2,"last update":"Mon Oct 16 20:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.73399999999999999,"15min":0.73499999999999999},"min":{"1min":0.55700000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.92900000000000005,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.57099999999999995,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.91000000000000003,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.77800000000000002}]},{"osd":3,"last update":"Mon Oct 16 20:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.79900000000000004,"15min":0.80400000000000005},"min":{"1min":0.627,"5min":0.44,"15min":0.44},"max":{"1min":0.95399999999999996,"5min":1.163,"15min":1.163},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.79100000000000004,"15min":0.79500000000000004},"min":{"1min":0.64000000000000001,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.125,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.93999999999999995}]},{"osd":5,"last update":"Mon Oct 16 20:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.66300000000000003,"15min":0.67100000000000004},"min":{"1min":0.439,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86799999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.69099999999999995,"15min":0.69899999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85599999999999998,"5min":1.089,"15min":1.089},"last":0.85799999999999998}]},{"osd":6,"last update":"Mon Oct 16 20:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.72599999999999998,"15min":0.73499999999999999},"min":{"1min":0.51900000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94299999999999995,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.76700000000000002,"15min":0.78500000000000003},"min":{"1min":0.48299999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.96099999999999997,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.60899999999999999}]},{"osd":7,"last update":"Mon Oct 16 20:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.77300000000000002,"15min":0.78700000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.84199999999999997,"15min":0.85399999999999998},"min":{"1min":0.59199999999999997,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.97299999999999998,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.74399999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509476,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":696320,"data_stored":308182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.45600000000000002,"15min":0.46100000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.61099999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.46700000000000003,"15min":0.46600000000000003},"min":{"1min":0.38900000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.64100000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.49199999999999999}]},{"osd":1,"last update":"Mon Oct 16 20:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.45000000000000001,"15min":0.45700000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64600000000000002,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.51100000000000001,"15min":0.51700000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66400000000000003,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.55000000000000004}]},{"osd":2,"last update":"Mon Oct 16 20:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.499,"15min":0.501},"min":{"1min":0.39400000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.74099999999999999,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.69399999999999995,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.59999999999999998}]},{"osd":4,"last update":"Mon Oct 16 20:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.77200000000000002,"15min":0.78000000000000003},"min":{"1min":0.44500000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.94799999999999995,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.014},{"interface":"front","average":{"1min":0.753,"5min":0.80900000000000005,"15min":0.81899999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.97099999999999997,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.96099999999999997}]},{"osd":5,"last update":"Mon Oct 16 20:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.75800000000000001,"15min":0.76800000000000002},"min":{"1min":0.51200000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.93100000000000005,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.88},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.78200000000000003,"15min":0.78900000000000003},"min":{"1min":0.56200000000000006,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0169999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.81499999999999995}]},{"osd":6,"last update":"Mon Oct 16 20:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.77600000000000002,"15min":0.78800000000000003},"min":{"1min":0.52800000000000002,"5min":0.433,"15min":0.433},"max":{"1min":1.034,"5min":1.774,"15min":1.774},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.80300000000000005,"15min":0.81699999999999995},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.004,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.92800000000000005}]},{"osd":7,"last update":"Mon Oct 16 20:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.79400000000000004,"15min":0.80600000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.98799999999999999,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.79200000000000004,"15min":0.80600000000000005},"min":{"1min":0.503,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.89700000000000002,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.79300000000000004}]}]},{"osd":2,"up_from":15,"seq":64424509475,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":684032,"data_stored":301537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.40699999999999997,"15min":0.41299999999999998},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.50800000000000001,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.47899999999999998,"15min":0.48499999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.70899999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.41499999999999998}]},{"osd":1,"last update":"Mon Oct 16 20:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.46700000000000003,"15min":0.46600000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.61299999999999999,"5min":0.623,"15min":0.623},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.36899999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.45200000000000001}]},{"osd":3,"last update":"Mon Oct 16 20:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.47499999999999998,"15min":0.47699999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.40100000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.34499999999999997}]},{"osd":4,"last update":"Mon Oct 16 20:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.76300000000000001,"15min":0.77100000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.91900000000000004,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.749,"15min":0.75800000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0660000000000001,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.55800000000000005}]},{"osd":5,"last update":"Mon Oct 16 20:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.72599999999999998,"15min":0.73199999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.93200000000000005,"5min":1.145,"15min":1.145},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.747,"15min":0.754},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.91900000000000004,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.54300000000000004}]},{"osd":6,"last update":"Mon Oct 16 20:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.752,"15min":0.753},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97099999999999997,"5min":1.165,"15min":1.165},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.76200000000000001,"15min":0.76800000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.95399999999999996,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.64000000000000001}]},{"osd":7,"last update":"Mon Oct 16 20:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76900000000000002,"15min":0.77000000000000002},"min":{"1min":0.58099999999999996,"5min":0.502,"15min":0.502},"max":{"1min":1.0369999999999999,"5min":1.167,"15min":1.167},"last":0.624},{"interface":"front","average":{"1min":0.745,"5min":0.77700000000000002,"15min":0.78300000000000003},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.95099999999999996,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.69999999999999996}]}]},{"osd":1,"up_from":15,"seq":64424509476,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":692224,"data_stored":308216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.53000000000000003,"15min":0.55100000000000005},"min":{"1min":0.30399999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.58899999999999997,"5min":3.867,"15min":3.867},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.442,"5min":0.48699999999999999,"15min":0.495},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.63500000000000001,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.59399999999999997}]},{"osd":2,"last update":"Mon Oct 16 20:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.69499999999999995,"15min":0.73699999999999999},"min":{"1min":0.32900000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.72299999999999998,"5min":5.6040000000000001,"15min":5.6040000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.76000000000000001,"15min":0.81200000000000006},"min":{"1min":0.34699999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.58299999999999996,"5min":7.8380000000000001,"15min":7.8380000000000001},"last":0.627}]},{"osd":3,"last update":"Mon Oct 16 20:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.68799999999999994,"15min":0.71999999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.63500000000000001,"5min":5.2039999999999997,"15min":5.2039999999999997},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.63700000000000001,"15min":0.66600000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.626,"5min":3.9470000000000001,"15min":3.9470000000000001},"last":0.67000000000000004}]},{"osd":4,"last update":"Mon Oct 16 20:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.84399999999999997,"15min":0.86399999999999999},"min":{"1min":0.48299999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.91800000000000004,"5min":4.1500000000000004,"15min":4.1500000000000004},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.85299999999999998,"15min":0.875},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.94099999999999995,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":0.93300000000000005}]},{"osd":5,"last update":"Mon Oct 16 20:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.753,"15min":0.75800000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93100000000000005,"5min":1.496,"15min":1.496},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.86499999999999999,"15min":0.88900000000000001},"min":{"1min":0.58399999999999996,"5min":0.501,"15min":0.501},"max":{"1min":0.874,"5min":4.0910000000000002,"15min":4.0910000000000002},"last":0.96999999999999997}]},{"osd":6,"last update":"Mon Oct 16 20:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.86099999999999999,"15min":0.88200000000000001},"min":{"1min":0.57699999999999996,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.93300000000000005,"5min":4.21,"15min":4.21},"last":1.01},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.76200000000000001,"15min":0.76500000000000001},"min":{"1min":0.58699999999999997,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.89300000000000002,"5min":1.401,"15min":1.401},"last":0.95299999999999996}]},{"osd":7,"last update":"Mon Oct 16 20:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79800000000000004,"15min":0.79900000000000004},"min":{"1min":0.64800000000000002,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.96199999999999997,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.86299999999999999,"15min":0.88300000000000001},"min":{"1min":0.54100000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.90500000000000003,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":0.88200000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28240,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343600,"statfs":{"total":96636764160,"available":96607846400,"internally_reserved":0,"allocated":1245184,"data_stored":878122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 20:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.56200000000000006,"15min":0.59099999999999997},"min":{"1min":0.27800000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.56699999999999995,"5min":3.9289999999999998,"15min":3.9289999999999998},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.57699999999999996,"15min":0.60899999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.55700000000000005,"5min":4.0910000000000002,"15min":4.0910000000000002},"last":0.49399999999999999}]},{"osd":2,"last update":"Mon Oct 16 20:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.64500000000000002,"15min":0.67500000000000004},"min":{"1min":0.33400000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.59599999999999997,"5min":4.1520000000000001,"15min":4.1520000000000001},"last":0.438},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.67400000000000004,"15min":0.70799999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.60499999999999998,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.45400000000000001}]},{"osd":3,"last update":"Mon Oct 16 20:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.621,"15min":0.64700000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63100000000000001,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.39500000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.73499999999999999,"5min":1.006,"15min":1.006},"last":0.47499999999999998}]},{"osd":4,"last update":"Mon Oct 16 20:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.83499999999999996,"15min":0.86799999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.85599999999999998,"5min":4.5250000000000004,"15min":4.5250000000000004},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.746,"15min":0.75},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0189999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.68899999999999995}]},{"osd":5,"last update":"Mon Oct 16 20:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.874,"15min":0.90600000000000003},"min":{"1min":0.505,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.88200000000000001,"5min":3.9990000000000001,"15min":3.9990000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.92900000000000005,"15min":0.96299999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.006,"5min":4.4779999999999998,"15min":4.4779999999999998},"last":0.63700000000000001}]},{"osd":6,"last update":"Mon Oct 16 20:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.76300000000000001,"15min":0.76900000000000002},"min":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0129999999999999,"5min":1.363,"15min":1.363},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.90900000000000003,"15min":0.94699999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.98299999999999998,"5min":3.956,"15min":3.956},"last":0.55300000000000005}]},{"osd":7,"last update":"Mon Oct 16 20:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.91600000000000004,"15min":0.94899999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0349999999999999,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.90800000000000003,"15min":0.94199999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.96699999999999997,"5min":4.5640000000000001,"15min":4.5640000000000001},"last":0.56699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T20:07:30.342 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T20:07:30.517 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.513+0000 7fcf0710f640 1 Processor -- start 2023-10-16T20:07:30.517 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.513+0000 7fcf0710f640 1 -- start start 2023-10-16T20:07:30.519 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.513+0000 7fcf0710f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf0015d9f0 0x7fcf00161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:30.519 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.513+0000 7fcf0710f640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fcf000d26c0 0x7fcf000cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:30.519 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.513+0000 7fcf0710f640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf000b87e0 con 0x7fcf000d22e0 2023-10-16T20:07:30.520 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.513+0000 7fcf0710f640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf0015df10 con 0x7fcf000d26c0 2023-10-16T20:07:30.520 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.513+0000 7fcf0710f640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0015e1f0 con 0x7fcf0015d9f0 2023-10-16T20:07:30.520 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.513+0000 7fcf056ac640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fcf000d26c0 0x7fcf000cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:30.520 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.513+0000 7fcf04eab640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fcf000d22e0 0x7fcf0015d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:46956/0 (socket says 172.21.15.88:46956) 2023-10-16T20:07:30.520 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.513+0000 7fcf04eab640 1 -- 172.21.15.88:0/981534857 learned_addr learned my addr 172.21.15.88:0/981534857 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:30.520 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.513+0000 7fcef7fff640 1 --2- 172.21.15.88:0/981534857 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf0015d9f0 0x7fcf00161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:30.521 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.513+0000 7fcf056ac640 1 -- 172.21.15.88:0/981534857 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf0015d9f0 msgr2=0x7fcf00161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:30.521 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.513+0000 7fcf056ac640 1 --2- 172.21.15.88:0/981534857 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf0015d9f0 0x7fcf00161450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:30.521 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.513+0000 7fcf056ac640 1 -- 172.21.15.88:0/981534857 >> v1:172.21.15.88:6789/0 conn(0x7fcf000d22e0 legacy=0x7fcf0015d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:30.521 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.513+0000 7fcf056ac640 1 -- 172.21.15.88:0/981534857 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf00160970 con 0x7fcf000d26c0 2023-10-16T20:07:30.522 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.517+0000 7fcf056ac640 1 --2- 172.21.15.88:0/981534857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf000d26c0 0x7fcf000cf2e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009d90 tx=0x7fcef801c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db77048b57c92b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:30.522 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.517+0000 7fcef77fe640 1 -- 172.21.15.88:0/981534857 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef80a5950 con 0x7fcf000d26c0 2023-10-16T20:07:30.522 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.517+0000 7fcef77fe640 1 -- 172.21.15.88:0/981534857 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef80a5ab0 con 0x7fcf000d26c0 2023-10-16T20:07:30.523 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.517+0000 7fcef77fe640 1 -- 172.21.15.88:0/981534857 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef80a5e40 con 0x7fcf000d26c0 2023-10-16T20:07:30.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.517+0000 7fcf0710f640 1 -- 172.21.15.88:0/981534857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf000d26c0 msgr2=0x7fcf000cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:30.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.517+0000 7fcf0710f640 1 --2- 172.21.15.88:0/981534857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf000d26c0 0x7fcf000cf2e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009d90 tx=0x7fcef801c890 comp rx=0 tx=0).stop 2023-10-16T20:07:30.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.517+0000 7fcf0710f640 1 -- 172.21.15.88:0/981534857 shutdown_connections 2023-10-16T20:07:30.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.517+0000 7fcf0710f640 1 --2- 172.21.15.88:0/981534857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf000d26c0 0x7fcf000cf2e0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:30.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.517+0000 7fcf0710f640 1 --2- 172.21.15.88:0/981534857 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf0015d9f0 0x7fcf00161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:30.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.517+0000 7fcf0710f640 1 -- 172.21.15.88:0/981534857 >> 172.21.15.88:0/981534857 conn(0x7fcf000715c0 msgr2=0x7fcf000719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:30.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.517+0000 7fcf0710f640 1 -- 172.21.15.88:0/981534857 shutdown_connections 2023-10-16T20:07:30.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.517+0000 7fcf0710f640 1 -- 172.21.15.88:0/981534857 wait complete. 2023-10-16T20:07:30.528 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.521+0000 7fcf0710f640 1 Processor -- start 2023-10-16T20:07:30.528 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.521+0000 7fcf0710f640 1 -- start start 2023-10-16T20:07:30.528 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.521+0000 7fcf0710f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf000d22e0 0x7fcf0015bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:30.528 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.521+0000 7fcf0710f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf000d26c0 0x7fcf0015c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:30.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.521+0000 7fcf0710f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf0015d9f0 0x7fcf0015ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:30.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.521+0000 7fcf0710f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf00160250 con 0x7fcf0015d9f0 2023-10-16T20:07:30.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.521+0000 7fcf0710f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0015f9b0 con 0x7fcf000d26c0 2023-10-16T20:07:30.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.521+0000 7fcf0710f640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0015fc90 con 0x7fcf000d22e0 2023-10-16T20:07:30.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.521+0000 7fcf056ac640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf0015d9f0 0x7fcf0015ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:30.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.521+0000 7fcf056ac640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf0015d9f0 0x7fcf0015ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38626/0 (socket says 172.21.15.88:38626) 2023-10-16T20:07:30.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.521+0000 7fcf056ac640 1 -- 172.21.15.88:0/672350606 learned_addr learned my addr 172.21.15.88:0/672350606 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:30.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.521+0000 7fcf04eab640 1 --2- 172.21.15.88:0/672350606 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf000d22e0 0x7fcf0015bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:30.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.521+0000 7fcef7fff640 1 --2- 172.21.15.88:0/672350606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf000d26c0 0x7fcf0015c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:30.531 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.521+0000 7fcf056ac640 1 -- 172.21.15.88:0/672350606 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf000d22e0 msgr2=0x7fcf0015bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:30.531 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.521+0000 7fcf056ac640 1 --2- 172.21.15.88:0/672350606 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf000d22e0 0x7fcf0015bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:30.531 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.521+0000 7fcf056ac640 1 -- 172.21.15.88:0/672350606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf000d26c0 msgr2=0x7fcf0015c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:30.531 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.521+0000 7fcf056ac640 1 --2- 172.21.15.88:0/672350606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf000d26c0 0x7fcf0015c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:30.531 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.521+0000 7fcf056ac640 1 -- 172.21.15.88:0/672350606 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef8009a70 con 0x7fcf0015d9f0 2023-10-16T20:07:30.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.521+0000 7fcf056ac640 1 --2- 172.21.15.88:0/672350606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf0015d9f0 0x7fcf0015ec40 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fcef801af40 tx=0x7fcef8019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:30.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.521+0000 7fcef57fa640 1 -- 172.21.15.88:0/672350606 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef80ade40 con 0x7fcf0015d9f0 2023-10-16T20:07:30.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.521+0000 7fcef57fa640 1 -- 172.21.15.88:0/672350606 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef8009450 con 0x7fcf0015d9f0 2023-10-16T20:07:30.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.521+0000 7fcf0710f640 1 -- 172.21.15.88:0/672350606 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf0015ff70 con 0x7fcf0015d9f0 2023-10-16T20:07:30.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.521+0000 7fcef57fa640 1 -- 172.21.15.88:0/672350606 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef80b9670 con 0x7fcf0015d9f0 2023-10-16T20:07:30.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.521+0000 7fcf0710f640 1 -- 172.21.15.88:0/672350606 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf0014a330 con 0x7fcf0015d9f0 2023-10-16T20:07:30.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.525+0000 7fcf0710f640 1 -- 172.21.15.88:0/672350606 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf001f6430 con 0x7fcf0015d9f0 2023-10-16T20:07:30.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.525+0000 7fcef57fa640 1 -- 172.21.15.88:0/672350606 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcef80039a0 con 0x7fcf0015d9f0 2023-10-16T20:07:30.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.525+0000 7fcef57fa640 1 --2- 172.21.15.88:0/672350606 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fceec0664d0 0x7fceec068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:30.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.525+0000 7fcef57fa640 1 -- 172.21.15.88:0/672350606 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fcef811f400 con 0x7fcf0015d9f0 2023-10-16T20:07:30.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.525+0000 7fcf04eab640 1 --2- 172.21.15.88:0/672350606 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fceec0664d0 0x7fceec068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:30.535 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.525+0000 7fcf04eab640 1 --2- 172.21.15.88:0/672350606 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fceec0664d0 0x7fceec068990 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcefc000f70 tx=0x7fcefc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:30.535 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.529+0000 7fcef57fa640 1 -- 172.21.15.88:0/672350606 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcf001f6430 con 0x7fcf0015d9f0 2023-10-16T20:07:30.683 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.677+0000 7fcf0710f640 1 -- 172.21.15.88:0/672350606 --> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcf0015fe00 con 0x7fceec0664d0 2023-10-16T20:07:30.690 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.685+0000 7fcef57fa640 1 -- 172.21.15.88:0/672350606 <== mgr.4110 v2:172.21.15.88:6832/4078483333 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289528 (secure 0 0 0) 0x7fcf0015fe00 con 0x7fceec0664d0 2023-10-16T20:07:30.693 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:07:30.695 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2023-10-16T20:07:30.696 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.689+0000 7fcf0710f640 1 -- 172.21.15.88:0/672350606 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fceec0664d0 msgr2=0x7fceec068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:30.697 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.689+0000 7fcf0710f640 1 --2- 172.21.15.88:0/672350606 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fceec0664d0 0x7fceec068990 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcefc000f70 tx=0x7fcefc009040 comp rx=0 tx=0).stop 2023-10-16T20:07:30.697 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.689+0000 7fcf0710f640 1 -- 172.21.15.88:0/672350606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf0015d9f0 msgr2=0x7fcf0015ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:30.697 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.689+0000 7fcf0710f640 1 --2- 172.21.15.88:0/672350606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf0015d9f0 0x7fcf0015ec40 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fcef801af40 tx=0x7fcef8019630 comp rx=0 tx=0).stop 2023-10-16T20:07:30.697 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.689+0000 7fcf0710f640 1 -- 172.21.15.88:0/672350606 shutdown_connections 2023-10-16T20:07:30.698 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.689+0000 7fcf0710f640 1 --2- 172.21.15.88:0/672350606 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fceec0664d0 0x7fceec068990 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:30.698 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.689+0000 7fcf0710f640 1 --2- 172.21.15.88:0/672350606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf0015d9f0 0x7fcf0015ec40 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:30.698 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.689+0000 7fcf0710f640 1 --2- 172.21.15.88:0/672350606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf000d26c0 0x7fcf0015c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:30.698 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.689+0000 7fcf0710f640 1 --2- 172.21.15.88:0/672350606 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf000d22e0 0x7fcf0015bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:30.698 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.689+0000 7fcf0710f640 1 -- 172.21.15.88:0/672350606 >> 172.21.15.88:0/672350606 conn(0x7fcf000715c0 msgr2=0x7fcf000b66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:30.699 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.689+0000 7fcf0710f640 1 -- 172.21.15.88:0/672350606 shutdown_connections 2023-10-16T20:07:30.699 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.689+0000 7fcf0710f640 1 -- 172.21.15.88:0/672350606 wait complete. 2023-10-16T20:07:30.714 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2023-10-16T20:07:30.478547+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":116,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239408,"kb_used_data":6588,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735312,"statfs":{"total":773094113280,"available":772848959488,"internally_reserved":0,"allocated":6746112,"data_stored":3643919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238276153},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000902"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794955+0000","last_change":"2023-10-16T20:07:26.231631+0000","last_active":"2023-10-16T20:07:27.794955+0000","last_peered":"2023-10-16T20:07:27.794955+0000","last_clean":"2023-10-16T20:07:27.794955+0000","last_became_active":"2023-10-16T20:07:26.231332+0000","last_became_peered":"2023-10-16T20:07:26.231332+0000","last_unstale":"2023-10-16T20:07:27.794955+0000","last_undegraded":"2023-10-16T20:07:27.794955+0000","last_fullsized":"2023-10-16T20:07:27.794955+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:38:52.502602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211803+0000","last_change":"2023-10-16T20:07:26.236566+0000","last_active":"2023-10-16T20:07:27.211803+0000","last_peered":"2023-10-16T20:07:27.211803+0000","last_clean":"2023-10-16T20:07:27.211803+0000","last_became_active":"2023-10-16T20:07:26.236293+0000","last_became_peered":"2023-10-16T20:07:26.236293+0000","last_unstale":"2023-10-16T20:07:27.211803+0000","last_undegraded":"2023-10-16T20:07:27.211803+0000","last_fullsized":"2023-10-16T20:07:27.211803+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:05:37.489356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213581+0000","last_change":"2023-10-16T20:07:26.233160+0000","last_active":"2023-10-16T20:07:27.213581+0000","last_peered":"2023-10-16T20:07:27.213581+0000","last_clean":"2023-10-16T20:07:27.213581+0000","last_became_active":"2023-10-16T20:07:26.232765+0000","last_became_peered":"2023-10-16T20:07:26.232765+0000","last_unstale":"2023-10-16T20:07:27.213581+0000","last_undegraded":"2023-10-16T20:07:27.213581+0000","last_fullsized":"2023-10-16T20:07:27.213581+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:07:42.996110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795167+0000","last_change":"2023-10-16T20:07:26.219553+0000","last_active":"2023-10-16T20:07:27.795167+0000","last_peered":"2023-10-16T20:07:27.795167+0000","last_clean":"2023-10-16T20:07:27.795167+0000","last_became_active":"2023-10-16T20:07:26.219232+0000","last_became_peered":"2023-10-16T20:07:26.219232+0000","last_unstale":"2023-10-16T20:07:27.795167+0000","last_undegraded":"2023-10-16T20:07:27.795167+0000","last_fullsized":"2023-10-16T20:07:27.795167+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:26:50.399490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794028+0000","last_change":"2023-10-16T20:07:26.231576+0000","last_active":"2023-10-16T20:07:27.794028+0000","last_peered":"2023-10-16T20:07:27.794028+0000","last_clean":"2023-10-16T20:07:27.794028+0000","last_became_active":"2023-10-16T20:07:26.227600+0000","last_became_peered":"2023-10-16T20:07:26.227600+0000","last_unstale":"2023-10-16T20:07:27.794028+0000","last_undegraded":"2023-10-16T20:07:27.794028+0000","last_fullsized":"2023-10-16T20:07:27.794028+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:45:52.100295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794229+0000","last_change":"2023-10-16T20:07:26.224614+0000","last_active":"2023-10-16T20:07:27.794229+0000","last_peered":"2023-10-16T20:07:27.794229+0000","last_clean":"2023-10-16T20:07:27.794229+0000","last_became_active":"2023-10-16T20:07:26.224307+0000","last_became_peered":"2023-10-16T20:07:26.224307+0000","last_unstale":"2023-10-16T20:07:27.794229+0000","last_undegraded":"2023-10-16T20:07:27.794229+0000","last_fullsized":"2023-10-16T20:07:27.794229+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:38:02.890451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794540+0000","last_change":"2023-10-16T20:07:26.229775+0000","last_active":"2023-10-16T20:07:27.794540+0000","last_peered":"2023-10-16T20:07:27.794540+0000","last_clean":"2023-10-16T20:07:27.794540+0000","last_became_active":"2023-10-16T20:07:26.229111+0000","last_became_peered":"2023-10-16T20:07:26.229111+0000","last_unstale":"2023-10-16T20:07:27.794540+0000","last_undegraded":"2023-10-16T20:07:27.794540+0000","last_fullsized":"2023-10-16T20:07:27.794540+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:11:58.985907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211291+0000","last_change":"2023-10-16T20:07:26.237894+0000","last_active":"2023-10-16T20:07:27.211291+0000","last_peered":"2023-10-16T20:07:27.211291+0000","last_clean":"2023-10-16T20:07:27.211291+0000","last_became_active":"2023-10-16T20:07:26.237614+0000","last_became_peered":"2023-10-16T20:07:26.237614+0000","last_unstale":"2023-10-16T20:07:27.211291+0000","last_undegraded":"2023-10-16T20:07:27.211291+0000","last_fullsized":"2023-10-16T20:07:27.211291+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:07:42.996110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795633+0000","last_change":"2023-10-16T20:07:26.228360+0000","last_active":"2023-10-16T20:07:27.795633+0000","last_peered":"2023-10-16T20:07:27.795633+0000","last_clean":"2023-10-16T20:07:27.795633+0000","last_became_active":"2023-10-16T20:07:26.228053+0000","last_became_peered":"2023-10-16T20:07:26.228053+0000","last_unstale":"2023-10-16T20:07:27.795633+0000","last_undegraded":"2023-10-16T20:07:27.795633+0000","last_fullsized":"2023-10-16T20:07:27.795633+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:19:45.099223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795613+0000","last_change":"2023-10-16T20:07:26.235541+0000","last_active":"2023-10-16T20:07:27.795613+0000","last_peered":"2023-10-16T20:07:27.795613+0000","last_clean":"2023-10-16T20:07:27.795613+0000","last_became_active":"2023-10-16T20:07:26.235235+0000","last_became_peered":"2023-10-16T20:07:26.235235+0000","last_unstale":"2023-10-16T20:07:27.795613+0000","last_undegraded":"2023-10-16T20:07:27.795613+0000","last_fullsized":"2023-10-16T20:07:27.795613+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:05:02.583660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795487+0000","last_change":"2023-10-16T20:07:26.234105+0000","last_active":"2023-10-16T20:07:27.795487+0000","last_peered":"2023-10-16T20:07:27.795487+0000","last_clean":"2023-10-16T20:07:27.795487+0000","last_became_active":"2023-10-16T20:07:26.233841+0000","last_became_peered":"2023-10-16T20:07:26.233841+0000","last_unstale":"2023-10-16T20:07:27.795487+0000","last_undegraded":"2023-10-16T20:07:27.795487+0000","last_fullsized":"2023-10-16T20:07:27.795487+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:05:50.890403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795308+0000","last_change":"2023-10-16T20:07:26.228168+0000","last_active":"2023-10-16T20:07:27.795308+0000","last_peered":"2023-10-16T20:07:27.795308+0000","last_clean":"2023-10-16T20:07:27.795308+0000","last_became_active":"2023-10-16T20:07:26.227862+0000","last_became_peered":"2023-10-16T20:07:26.227862+0000","last_unstale":"2023-10-16T20:07:27.795308+0000","last_undegraded":"2023-10-16T20:07:27.795308+0000","last_fullsized":"2023-10-16T20:07:27.795308+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:52:43.995476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213961+0000","last_change":"2023-10-16T20:07:26.230062+0000","last_active":"2023-10-16T20:07:27.213961+0000","last_peered":"2023-10-16T20:07:27.213961+0000","last_clean":"2023-10-16T20:07:27.213961+0000","last_became_active":"2023-10-16T20:07:26.229479+0000","last_became_peered":"2023-10-16T20:07:26.229479+0000","last_unstale":"2023-10-16T20:07:27.213961+0000","last_undegraded":"2023-10-16T20:07:27.213961+0000","last_fullsized":"2023-10-16T20:07:27.213961+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:46:44.338547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795091+0000","last_change":"2023-10-16T20:07:26.232039+0000","last_active":"2023-10-16T20:07:27.795091+0000","last_peered":"2023-10-16T20:07:27.795091+0000","last_clean":"2023-10-16T20:07:27.795091+0000","last_became_active":"2023-10-16T20:07:26.228897+0000","last_became_peered":"2023-10-16T20:07:26.228897+0000","last_unstale":"2023-10-16T20:07:27.795091+0000","last_undegraded":"2023-10-16T20:07:27.795091+0000","last_fullsized":"2023-10-16T20:07:27.795091+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:19:55.369594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794622+0000","last_change":"2023-10-16T20:07:26.229967+0000","last_active":"2023-10-16T20:07:27.794622+0000","last_peered":"2023-10-16T20:07:27.794622+0000","last_clean":"2023-10-16T20:07:27.794622+0000","last_became_active":"2023-10-16T20:07:26.229489+0000","last_became_peered":"2023-10-16T20:07:26.229489+0000","last_unstale":"2023-10-16T20:07:27.794622+0000","last_undegraded":"2023-10-16T20:07:27.794622+0000","last_fullsized":"2023-10-16T20:07:27.794622+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:48:17.850138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.793790+0000","last_change":"2023-10-16T20:07:26.224916+0000","last_active":"2023-10-16T20:07:27.793790+0000","last_peered":"2023-10-16T20:07:27.793790+0000","last_clean":"2023-10-16T20:07:27.793790+0000","last_became_active":"2023-10-16T20:07:26.224524+0000","last_became_peered":"2023-10-16T20:07:26.224524+0000","last_unstale":"2023-10-16T20:07:27.793790+0000","last_undegraded":"2023-10-16T20:07:27.793790+0000","last_fullsized":"2023-10-16T20:07:27.793790+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:43:39.455752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211982+0000","last_change":"2023-10-16T20:07:26.236536+0000","last_active":"2023-10-16T20:07:27.211982+0000","last_peered":"2023-10-16T20:07:27.211982+0000","last_clean":"2023-10-16T20:07:27.211982+0000","last_became_active":"2023-10-16T20:07:26.236238+0000","last_became_peered":"2023-10-16T20:07:26.236238+0000","last_unstale":"2023-10-16T20:07:27.211982+0000","last_undegraded":"2023-10-16T20:07:27.211982+0000","last_fullsized":"2023-10-16T20:07:27.211982+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:18:14.053601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213163+0000","last_change":"2023-10-16T20:07:26.233070+0000","last_active":"2023-10-16T20:07:27.213163+0000","last_peered":"2023-10-16T20:07:27.213163+0000","last_clean":"2023-10-16T20:07:27.213163+0000","last_became_active":"2023-10-16T20:07:26.232664+0000","last_became_peered":"2023-10-16T20:07:26.232664+0000","last_unstale":"2023-10-16T20:07:27.213163+0000","last_undegraded":"2023-10-16T20:07:27.213163+0000","last_fullsized":"2023-10-16T20:07:27.213163+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:24:24.649666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794300+0000","last_change":"2023-10-16T20:07:26.229604+0000","last_active":"2023-10-16T20:07:27.794300+0000","last_peered":"2023-10-16T20:07:27.794300+0000","last_clean":"2023-10-16T20:07:27.794300+0000","last_became_active":"2023-10-16T20:07:26.229330+0000","last_became_peered":"2023-10-16T20:07:26.229330+0000","last_unstale":"2023-10-16T20:07:27.794300+0000","last_undegraded":"2023-10-16T20:07:27.794300+0000","last_fullsized":"2023-10-16T20:07:27.794300+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:02:07.982732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211532+0000","last_change":"2023-10-16T20:07:26.229245+0000","last_active":"2023-10-16T20:07:27.211532+0000","last_peered":"2023-10-16T20:07:27.211532+0000","last_clean":"2023-10-16T20:07:27.211532+0000","last_became_active":"2023-10-16T20:07:26.228987+0000","last_became_peered":"2023-10-16T20:07:26.228987+0000","last_unstale":"2023-10-16T20:07:27.211532+0000","last_undegraded":"2023-10-16T20:07:27.211532+0000","last_fullsized":"2023-10-16T20:07:27.211532+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:50:26.093947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213084+0000","last_change":"2023-10-16T20:07:26.238069+0000","last_active":"2023-10-16T20:07:27.213084+0000","last_peered":"2023-10-16T20:07:27.213084+0000","last_clean":"2023-10-16T20:07:27.213084+0000","last_became_active":"2023-10-16T20:07:26.237072+0000","last_became_peered":"2023-10-16T20:07:26.237072+0000","last_unstale":"2023-10-16T20:07:27.213084+0000","last_undegraded":"2023-10-16T20:07:27.213084+0000","last_fullsized":"2023-10-16T20:07:27.213084+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:08:51.178056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211487+0000","last_change":"2023-10-16T20:07:26.240455+0000","last_active":"2023-10-16T20:07:27.211487+0000","last_peered":"2023-10-16T20:07:27.211487+0000","last_clean":"2023-10-16T20:07:27.211487+0000","last_became_active":"2023-10-16T20:07:26.239255+0000","last_became_peered":"2023-10-16T20:07:26.239255+0000","last_unstale":"2023-10-16T20:07:27.211487+0000","last_undegraded":"2023-10-16T20:07:27.211487+0000","last_fullsized":"2023-10-16T20:07:27.211487+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:20:02.841816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213052+0000","last_change":"2023-10-16T20:07:26.240749+0000","last_active":"2023-10-16T20:07:27.213052+0000","last_peered":"2023-10-16T20:07:27.213052+0000","last_clean":"2023-10-16T20:07:27.213052+0000","last_became_active":"2023-10-16T20:07:26.240444+0000","last_became_peered":"2023-10-16T20:07:26.240444+0000","last_unstale":"2023-10-16T20:07:27.213052+0000","last_undegraded":"2023-10-16T20:07:27.213052+0000","last_fullsized":"2023-10-16T20:07:27.213052+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:24:40.444553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213314+0000","last_change":"2023-10-16T20:07:26.224440+0000","last_active":"2023-10-16T20:07:27.213314+0000","last_peered":"2023-10-16T20:07:27.213314+0000","last_clean":"2023-10-16T20:07:27.213314+0000","last_became_active":"2023-10-16T20:07:26.223541+0000","last_became_peered":"2023-10-16T20:07:26.223541+0000","last_unstale":"2023-10-16T20:07:27.213314+0000","last_undegraded":"2023-10-16T20:07:27.213314+0000","last_fullsized":"2023-10-16T20:07:27.213314+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:31:52.974483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795284+0000","last_change":"2023-10-16T20:07:26.235912+0000","last_active":"2023-10-16T20:07:27.795284+0000","last_peered":"2023-10-16T20:07:27.795284+0000","last_clean":"2023-10-16T20:07:27.795284+0000","last_became_active":"2023-10-16T20:07:26.235620+0000","last_became_peered":"2023-10-16T20:07:26.235620+0000","last_unstale":"2023-10-16T20:07:27.795284+0000","last_undegraded":"2023-10-16T20:07:27.795284+0000","last_fullsized":"2023-10-16T20:07:27.795284+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:00:25.064493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795011+0000","last_change":"2023-10-16T20:07:26.228748+0000","last_active":"2023-10-16T20:07:27.795011+0000","last_peered":"2023-10-16T20:07:27.795011+0000","last_clean":"2023-10-16T20:07:27.795011+0000","last_became_active":"2023-10-16T20:07:26.228459+0000","last_became_peered":"2023-10-16T20:07:26.228459+0000","last_unstale":"2023-10-16T20:07:27.795011+0000","last_undegraded":"2023-10-16T20:07:27.795011+0000","last_fullsized":"2023-10-16T20:07:27.795011+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:48:10.263086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794677+0000","last_change":"2023-10-16T20:07:26.226658+0000","last_active":"2023-10-16T20:07:27.794677+0000","last_peered":"2023-10-16T20:07:27.794677+0000","last_clean":"2023-10-16T20:07:27.794677+0000","last_became_active":"2023-10-16T20:07:26.226335+0000","last_became_peered":"2023-10-16T20:07:26.226335+0000","last_unstale":"2023-10-16T20:07:27.794677+0000","last_undegraded":"2023-10-16T20:07:27.794677+0000","last_fullsized":"2023-10-16T20:07:27.794677+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:57:26.911719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794121+0000","last_change":"2023-10-16T20:07:26.227073+0000","last_active":"2023-10-16T20:07:27.794121+0000","last_peered":"2023-10-16T20:07:27.794121+0000","last_clean":"2023-10-16T20:07:27.794121+0000","last_became_active":"2023-10-16T20:07:26.226795+0000","last_became_peered":"2023-10-16T20:07:26.226795+0000","last_unstale":"2023-10-16T20:07:27.794121+0000","last_undegraded":"2023-10-16T20:07:27.794121+0000","last_fullsized":"2023-10-16T20:07:27.794121+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:39:31.048341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211891+0000","last_change":"2023-10-16T20:07:26.233109+0000","last_active":"2023-10-16T20:07:27.211891+0000","last_peered":"2023-10-16T20:07:27.211891+0000","last_clean":"2023-10-16T20:07:27.211891+0000","last_became_active":"2023-10-16T20:07:26.232756+0000","last_became_peered":"2023-10-16T20:07:26.232756+0000","last_unstale":"2023-10-16T20:07:27.211891+0000","last_undegraded":"2023-10-16T20:07:27.211891+0000","last_fullsized":"2023-10-16T20:07:27.211891+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:15:17.549212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.212860+0000","last_change":"2023-10-16T20:07:26.231271+0000","last_active":"2023-10-16T20:07:27.212860+0000","last_peered":"2023-10-16T20:07:27.212860+0000","last_clean":"2023-10-16T20:07:27.212860+0000","last_became_active":"2023-10-16T20:07:26.230979+0000","last_became_peered":"2023-10-16T20:07:26.230979+0000","last_unstale":"2023-10-16T20:07:27.212860+0000","last_undegraded":"2023-10-16T20:07:27.212860+0000","last_fullsized":"2023-10-16T20:07:27.212860+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:59:56.928721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.793827+0000","last_change":"2023-10-16T20:07:26.226909+0000","last_active":"2023-10-16T20:07:27.793827+0000","last_peered":"2023-10-16T20:07:27.793827+0000","last_clean":"2023-10-16T20:07:27.793827+0000","last_became_active":"2023-10-16T20:07:26.225946+0000","last_became_peered":"2023-10-16T20:07:26.225946+0000","last_unstale":"2023-10-16T20:07:27.793827+0000","last_undegraded":"2023-10-16T20:07:27.793827+0000","last_fullsized":"2023-10-16T20:07:27.793827+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:32:39.900878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.793825+0000","last_change":"2023-10-16T20:07:26.234841+0000","last_active":"2023-10-16T20:07:27.793825+0000","last_peered":"2023-10-16T20:07:27.793825+0000","last_clean":"2023-10-16T20:07:27.793825+0000","last_became_active":"2023-10-16T20:07:26.234553+0000","last_became_peered":"2023-10-16T20:07:26.234553+0000","last_unstale":"2023-10-16T20:07:27.793825+0000","last_undegraded":"2023-10-16T20:07:27.793825+0000","last_fullsized":"2023-10-16T20:07:27.793825+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:18:05.186862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794594+0000","last_change":"2023-10-16T20:07:26.229115+0000","last_active":"2023-10-16T20:07:27.794594+0000","last_peered":"2023-10-16T20:07:27.794594+0000","last_clean":"2023-10-16T20:07:27.794594+0000","last_became_active":"2023-10-16T20:07:26.228885+0000","last_became_peered":"2023-10-16T20:07:26.228885+0000","last_unstale":"2023-10-16T20:07:27.794594+0000","last_undegraded":"2023-10-16T20:07:27.794594+0000","last_fullsized":"2023-10-16T20:07:27.794594+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:46:07.888226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-16T20:07:26.277894+0000","last_change":"2023-10-16T20:07:25.282198+0000","last_active":"2023-10-16T20:07:26.277894+0000","last_peered":"2023-10-16T20:07:26.277894+0000","last_clean":"2023-10-16T20:07:26.277894+0000","last_became_active":"2023-10-16T20:07:25.281929+0000","last_became_peered":"2023-10-16T20:07:25.281929+0000","last_unstale":"2023-10-16T20:07:26.277894+0000","last_undegraded":"2023-10-16T20:07:26.277894+0000","last_fullsized":"2023-10-16T20:07:26.277894+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:32:56.773665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795651+0000","last_change":"2023-10-16T20:07:26.234958+0000","last_active":"2023-10-16T20:07:27.795651+0000","last_peered":"2023-10-16T20:07:27.795651+0000","last_clean":"2023-10-16T20:07:27.795651+0000","last_became_active":"2023-10-16T20:07:26.234607+0000","last_became_peered":"2023-10-16T20:07:26.234607+0000","last_unstale":"2023-10-16T20:07:27.795651+0000","last_undegraded":"2023-10-16T20:07:27.795651+0000","last_fullsized":"2023-10-16T20:07:27.795651+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:32:22.385642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211762+0000","last_change":"2023-10-16T20:07:25.288386+0000","last_active":"2023-10-16T20:07:27.211762+0000","last_peered":"2023-10-16T20:07:27.211762+0000","last_clean":"2023-10-16T20:07:27.211762+0000","last_became_active":"2023-10-16T20:07:25.288083+0000","last_became_peered":"2023-10-16T20:07:25.288083+0000","last_unstale":"2023-10-16T20:07:27.211762+0000","last_undegraded":"2023-10-16T20:07:27.211762+0000","last_fullsized":"2023-10-16T20:07:27.211762+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:07:40.203583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794628+0000","last_change":"2023-10-16T20:07:26.233550+0000","last_active":"2023-10-16T20:07:27.794628+0000","last_peered":"2023-10-16T20:07:27.794628+0000","last_clean":"2023-10-16T20:07:27.794628+0000","last_became_active":"2023-10-16T20:07:26.233264+0000","last_became_peered":"2023-10-16T20:07:26.233264+0000","last_unstale":"2023-10-16T20:07:27.794628+0000","last_undegraded":"2023-10-16T20:07:27.794628+0000","last_fullsized":"2023-10-16T20:07:27.794628+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:35:09.295599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-16T20:07:26.279433+0000","last_change":"2023-10-16T20:07:25.298262+0000","last_active":"2023-10-16T20:07:26.279433+0000","last_peered":"2023-10-16T20:07:26.279433+0000","last_clean":"2023-10-16T20:07:26.279433+0000","last_became_active":"2023-10-16T20:07:25.297972+0000","last_became_peered":"2023-10-16T20:07:25.297972+0000","last_unstale":"2023-10-16T20:07:26.279433+0000","last_undegraded":"2023-10-16T20:07:26.279433+0000","last_fullsized":"2023-10-16T20:07:26.279433+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:59:35.695277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794799+0000","last_change":"2023-10-16T20:07:26.234199+0000","last_active":"2023-10-16T20:07:27.794799+0000","last_peered":"2023-10-16T20:07:27.794799+0000","last_clean":"2023-10-16T20:07:27.794799+0000","last_became_active":"2023-10-16T20:07:26.233751+0000","last_became_peered":"2023-10-16T20:07:26.233751+0000","last_unstale":"2023-10-16T20:07:27.794799+0000","last_undegraded":"2023-10-16T20:07:27.794799+0000","last_fullsized":"2023-10-16T20:07:27.794799+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:43:41.970837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"57'2","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795402+0000","last_change":"2023-10-16T20:07:25.278253+0000","last_active":"2023-10-16T20:07:27.795402+0000","last_peered":"2023-10-16T20:07:27.795402+0000","last_clean":"2023-10-16T20:07:27.795402+0000","last_became_active":"2023-10-16T20:07:25.277952+0000","last_became_peered":"2023-10-16T20:07:25.277952+0000","last_unstale":"2023-10-16T20:07:27.795402+0000","last_undegraded":"2023-10-16T20:07:27.795402+0000","last_fullsized":"2023-10-16T20:07:27.795402+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:21:59.522984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211501+0000","last_change":"2023-10-16T20:07:26.240164+0000","last_active":"2023-10-16T20:07:27.211501+0000","last_peered":"2023-10-16T20:07:27.211501+0000","last_clean":"2023-10-16T20:07:27.211501+0000","last_became_active":"2023-10-16T20:07:26.238843+0000","last_became_peered":"2023-10-16T20:07:26.238843+0000","last_unstale":"2023-10-16T20:07:27.211501+0000","last_undegraded":"2023-10-16T20:07:27.211501+0000","last_fullsized":"2023-10-16T20:07:27.211501+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:06:05.571248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794813+0000","last_change":"2023-10-16T20:07:25.287011+0000","last_active":"2023-10-16T20:07:27.794813+0000","last_peered":"2023-10-16T20:07:27.794813+0000","last_clean":"2023-10-16T20:07:27.794813+0000","last_became_active":"2023-10-16T20:07:25.286461+0000","last_became_peered":"2023-10-16T20:07:25.286461+0000","last_unstale":"2023-10-16T20:07:27.794813+0000","last_undegraded":"2023-10-16T20:07:27.794813+0000","last_fullsized":"2023-10-16T20:07:27.794813+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:46:43.335284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211670+0000","last_change":"2023-10-16T20:07:26.234473+0000","last_active":"2023-10-16T20:07:27.211670+0000","last_peered":"2023-10-16T20:07:27.211670+0000","last_clean":"2023-10-16T20:07:27.211670+0000","last_became_active":"2023-10-16T20:07:26.231718+0000","last_became_peered":"2023-10-16T20:07:26.231718+0000","last_unstale":"2023-10-16T20:07:27.211670+0000","last_undegraded":"2023-10-16T20:07:27.211670+0000","last_fullsized":"2023-10-16T20:07:27.211670+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:20:50.294097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-16T20:07:26.277221+0000","last_change":"2023-10-16T20:07:25.280487+0000","last_active":"2023-10-16T20:07:26.277221+0000","last_peered":"2023-10-16T20:07:26.277221+0000","last_clean":"2023-10-16T20:07:26.277221+0000","last_became_active":"2023-10-16T20:07:25.280197+0000","last_became_peered":"2023-10-16T20:07:25.280197+0000","last_unstale":"2023-10-16T20:07:26.277221+0000","last_undegraded":"2023-10-16T20:07:26.277221+0000","last_fullsized":"2023-10-16T20:07:26.277221+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:52:22.909728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213447+0000","last_change":"2023-10-16T20:07:26.225259+0000","last_active":"2023-10-16T20:07:27.213447+0000","last_peered":"2023-10-16T20:07:27.213447+0000","last_clean":"2023-10-16T20:07:27.213447+0000","last_became_active":"2023-10-16T20:07:26.224788+0000","last_became_peered":"2023-10-16T20:07:26.224788+0000","last_unstale":"2023-10-16T20:07:27.213447+0000","last_undegraded":"2023-10-16T20:07:27.213447+0000","last_fullsized":"2023-10-16T20:07:27.213447+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:50:26.015829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794639+0000","last_change":"2023-10-16T20:07:25.277974+0000","last_active":"2023-10-16T20:07:27.794639+0000","last_peered":"2023-10-16T20:07:27.794639+0000","last_clean":"2023-10-16T20:07:27.794639+0000","last_became_active":"2023-10-16T20:07:25.276748+0000","last_became_peered":"2023-10-16T20:07:25.276748+0000","last_unstale":"2023-10-16T20:07:27.794639+0000","last_undegraded":"2023-10-16T20:07:27.794639+0000","last_fullsized":"2023-10-16T20:07:27.794639+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:21:47.873107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211677+0000","last_change":"2023-10-16T20:07:26.231756+0000","last_active":"2023-10-16T20:07:27.211677+0000","last_peered":"2023-10-16T20:07:27.211677+0000","last_clean":"2023-10-16T20:07:27.211677+0000","last_became_active":"2023-10-16T20:07:26.231500+0000","last_became_peered":"2023-10-16T20:07:26.231500+0000","last_unstale":"2023-10-16T20:07:27.211677+0000","last_undegraded":"2023-10-16T20:07:27.211677+0000","last_fullsized":"2023-10-16T20:07:27.211677+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:11:19.933794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794560+0000","last_change":"2023-10-16T20:07:25.280055+0000","last_active":"2023-10-16T20:07:27.794560+0000","last_peered":"2023-10-16T20:07:27.794560+0000","last_clean":"2023-10-16T20:07:27.794560+0000","last_became_active":"2023-10-16T20:07:25.279711+0000","last_became_peered":"2023-10-16T20:07:25.279711+0000","last_unstale":"2023-10-16T20:07:27.794560+0000","last_undegraded":"2023-10-16T20:07:27.794560+0000","last_fullsized":"2023-10-16T20:07:27.794560+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:47:48.571528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795692+0000","last_change":"2023-10-16T20:07:26.234101+0000","last_active":"2023-10-16T20:07:27.795692+0000","last_peered":"2023-10-16T20:07:27.795692+0000","last_clean":"2023-10-16T20:07:27.795692+0000","last_became_active":"2023-10-16T20:07:26.233286+0000","last_became_peered":"2023-10-16T20:07:26.233286+0000","last_unstale":"2023-10-16T20:07:27.795692+0000","last_undegraded":"2023-10-16T20:07:27.795692+0000","last_fullsized":"2023-10-16T20:07:27.795692+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:12:55.180570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213511+0000","last_change":"2023-10-16T20:07:25.291493+0000","last_active":"2023-10-16T20:07:27.213511+0000","last_peered":"2023-10-16T20:07:27.213511+0000","last_clean":"2023-10-16T20:07:27.213511+0000","last_became_active":"2023-10-16T20:07:25.291189+0000","last_became_peered":"2023-10-16T20:07:25.291189+0000","last_unstale":"2023-10-16T20:07:27.213511+0000","last_undegraded":"2023-10-16T20:07:27.213511+0000","last_fullsized":"2023-10-16T20:07:27.213511+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:21:59.522984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213424+0000","last_change":"2023-10-16T20:07:26.237797+0000","last_active":"2023-10-16T20:07:27.213424+0000","last_peered":"2023-10-16T20:07:27.213424+0000","last_clean":"2023-10-16T20:07:27.213424+0000","last_became_active":"2023-10-16T20:07:26.237483+0000","last_became_peered":"2023-10-16T20:07:26.237483+0000","last_unstale":"2023-10-16T20:07:27.213424+0000","last_undegraded":"2023-10-16T20:07:27.213424+0000","last_fullsized":"2023-10-16T20:07:27.213424+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:52:43.995476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211842+0000","last_change":"2023-10-16T20:07:25.286755+0000","last_active":"2023-10-16T20:07:27.211842+0000","last_peered":"2023-10-16T20:07:27.211842+0000","last_clean":"2023-10-16T20:07:27.211842+0000","last_became_active":"2023-10-16T20:07:25.286230+0000","last_became_peered":"2023-10-16T20:07:25.286230+0000","last_unstale":"2023-10-16T20:07:27.211842+0000","last_undegraded":"2023-10-16T20:07:27.211842+0000","last_fullsized":"2023-10-16T20:07:27.211842+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:48:09.259823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795180+0000","last_change":"2023-10-16T20:07:26.231320+0000","last_active":"2023-10-16T20:07:27.795180+0000","last_peered":"2023-10-16T20:07:27.795180+0000","last_clean":"2023-10-16T20:07:27.795180+0000","last_became_active":"2023-10-16T20:07:26.230814+0000","last_became_peered":"2023-10-16T20:07:26.230814+0000","last_unstale":"2023-10-16T20:07:27.795180+0000","last_undegraded":"2023-10-16T20:07:27.795180+0000","last_fullsized":"2023-10-16T20:07:27.795180+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:19:45.099223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213843+0000","last_change":"2023-10-16T20:07:25.298302+0000","last_active":"2023-10-16T20:07:27.213843+0000","last_peered":"2023-10-16T20:07:27.213843+0000","last_clean":"2023-10-16T20:07:27.213843+0000","last_became_active":"2023-10-16T20:07:25.298040+0000","last_became_peered":"2023-10-16T20:07:25.298040+0000","last_unstale":"2023-10-16T20:07:27.213843+0000","last_undegraded":"2023-10-16T20:07:27.213843+0000","last_fullsized":"2023-10-16T20:07:27.213843+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:01:51.549639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213679+0000","last_change":"2023-10-16T20:07:26.235759+0000","last_active":"2023-10-16T20:07:27.213679+0000","last_peered":"2023-10-16T20:07:27.213679+0000","last_clean":"2023-10-16T20:07:27.213679+0000","last_became_active":"2023-10-16T20:07:26.235472+0000","last_became_peered":"2023-10-16T20:07:26.235472+0000","last_unstale":"2023-10-16T20:07:27.213679+0000","last_undegraded":"2023-10-16T20:07:27.213679+0000","last_fullsized":"2023-10-16T20:07:27.213679+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:00:35.644195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795164+0000","last_change":"2023-10-16T20:07:25.282309+0000","last_active":"2023-10-16T20:07:27.795164+0000","last_peered":"2023-10-16T20:07:27.795164+0000","last_clean":"2023-10-16T20:07:27.795164+0000","last_became_active":"2023-10-16T20:07:25.281997+0000","last_became_peered":"2023-10-16T20:07:25.281997+0000","last_unstale":"2023-10-16T20:07:27.795164+0000","last_undegraded":"2023-10-16T20:07:27.795164+0000","last_fullsized":"2023-10-16T20:07:27.795164+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:31:51.971221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794630+0000","last_change":"2023-10-16T20:07:26.236069+0000","last_active":"2023-10-16T20:07:27.794630+0000","last_peered":"2023-10-16T20:07:27.794630+0000","last_clean":"2023-10-16T20:07:27.794630+0000","last_became_active":"2023-10-16T20:07:26.235798+0000","last_became_peered":"2023-10-16T20:07:26.235798+0000","last_unstale":"2023-10-16T20:07:27.794630+0000","last_undegraded":"2023-10-16T20:07:27.794630+0000","last_fullsized":"2023-10-16T20:07:27.794630+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:24:24.649666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211759+0000","last_change":"2023-10-16T20:07:25.288129+0000","last_active":"2023-10-16T20:07:27.211759+0000","last_peered":"2023-10-16T20:07:27.211759+0000","last_clean":"2023-10-16T20:07:27.211759+0000","last_became_active":"2023-10-16T20:07:25.287112+0000","last_became_peered":"2023-10-16T20:07:25.287112+0000","last_unstale":"2023-10-16T20:07:27.211759+0000","last_undegraded":"2023-10-16T20:07:27.211759+0000","last_fullsized":"2023-10-16T20:07:27.211759+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:24:23.646404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213241+0000","last_change":"2023-10-16T20:07:26.227260+0000","last_active":"2023-10-16T20:07:27.213241+0000","last_peered":"2023-10-16T20:07:27.213241+0000","last_clean":"2023-10-16T20:07:27.213241+0000","last_became_active":"2023-10-16T20:07:26.226926+0000","last_became_peered":"2023-10-16T20:07:26.226926+0000","last_unstale":"2023-10-16T20:07:27.213241+0000","last_undegraded":"2023-10-16T20:07:27.213241+0000","last_fullsized":"2023-10-16T20:07:27.213241+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:26:50.399490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213734+0000","last_change":"2023-10-16T20:07:25.287377+0000","last_active":"2023-10-16T20:07:27.213734+0000","last_peered":"2023-10-16T20:07:27.213734+0000","last_clean":"2023-10-16T20:07:27.213734+0000","last_became_active":"2023-10-16T20:07:25.286994+0000","last_became_peered":"2023-10-16T20:07:25.286994+0000","last_unstale":"2023-10-16T20:07:27.213734+0000","last_undegraded":"2023-10-16T20:07:27.213734+0000","last_fullsized":"2023-10-16T20:07:27.213734+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:16:43.918657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795364+0000","last_change":"2023-10-16T20:07:26.229963+0000","last_active":"2023-10-16T20:07:27.795364+0000","last_peered":"2023-10-16T20:07:27.795364+0000","last_clean":"2023-10-16T20:07:27.795364+0000","last_became_active":"2023-10-16T20:07:26.229654+0000","last_became_peered":"2023-10-16T20:07:26.229654+0000","last_unstale":"2023-10-16T20:07:27.795364+0000","last_undegraded":"2023-10-16T20:07:27.795364+0000","last_fullsized":"2023-10-16T20:07:27.795364+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:06:41.715102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794719+0000","last_change":"2023-10-16T20:07:25.285312+0000","last_active":"2023-10-16T20:07:27.794719+0000","last_peered":"2023-10-16T20:07:27.794719+0000","last_clean":"2023-10-16T20:07:27.794719+0000","last_became_active":"2023-10-16T20:07:25.285025+0000","last_became_peered":"2023-10-16T20:07:25.285025+0000","last_unstale":"2023-10-16T20:07:27.794719+0000","last_undegraded":"2023-10-16T20:07:27.794719+0000","last_fullsized":"2023-10-16T20:07:27.794719+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:59:56.818441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213171+0000","last_change":"2023-10-16T20:07:26.241281+0000","last_active":"2023-10-16T20:07:27.213171+0000","last_peered":"2023-10-16T20:07:27.213171+0000","last_clean":"2023-10-16T20:07:27.213171+0000","last_became_active":"2023-10-16T20:07:26.240963+0000","last_became_peered":"2023-10-16T20:07:26.240963+0000","last_unstale":"2023-10-16T20:07:27.213171+0000","last_undegraded":"2023-10-16T20:07:27.213171+0000","last_fullsized":"2023-10-16T20:07:27.213171+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:07:41.206845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":20,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213458+0000","last_change":"2023-10-16T20:07:25.289444+0000","last_active":"2023-10-16T20:07:27.213458+0000","last_peered":"2023-10-16T20:07:27.213458+0000","last_clean":"2023-10-16T20:07:27.213458+0000","last_became_active":"2023-10-16T20:07:25.289138+0000","last_became_peered":"2023-10-16T20:07:25.289138+0000","last_unstale":"2023-10-16T20:07:27.213458+0000","last_undegraded":"2023-10-16T20:07:27.213458+0000","last_fullsized":"2023-10-16T20:07:27.213458+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:41:59.036278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213178+0000","last_change":"2023-10-16T20:07:26.238893+0000","last_active":"2023-10-16T20:07:27.213178+0000","last_peered":"2023-10-16T20:07:27.213178+0000","last_clean":"2023-10-16T20:07:27.213178+0000","last_became_active":"2023-10-16T20:07:26.238253+0000","last_became_peered":"2023-10-16T20:07:26.238253+0000","last_unstale":"2023-10-16T20:07:27.213178+0000","last_undegraded":"2023-10-16T20:07:27.213178+0000","last_fullsized":"2023-10-16T20:07:27.213178+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:48:10.263086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211987+0000","last_change":"2023-10-16T20:07:25.286326+0000","last_active":"2023-10-16T20:07:27.211987+0000","last_peered":"2023-10-16T20:07:27.211987+0000","last_clean":"2023-10-16T20:07:27.211987+0000","last_became_active":"2023-10-16T20:07:25.286056+0000","last_became_peered":"2023-10-16T20:07:25.286056+0000","last_unstale":"2023-10-16T20:07:27.211987+0000","last_undegraded":"2023-10-16T20:07:27.211987+0000","last_fullsized":"2023-10-16T20:07:27.211987+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:48:04.524856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795523+0000","last_change":"2023-10-16T20:07:26.234861+0000","last_active":"2023-10-16T20:07:27.795523+0000","last_peered":"2023-10-16T20:07:27.795523+0000","last_clean":"2023-10-16T20:07:27.795523+0000","last_became_active":"2023-10-16T20:07:26.234532+0000","last_became_peered":"2023-10-16T20:07:26.234532+0000","last_unstale":"2023-10-16T20:07:27.795523+0000","last_undegraded":"2023-10-16T20:07:27.795523+0000","last_fullsized":"2023-10-16T20:07:27.795523+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:00:35.644195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-16T20:07:26.277396+0000","last_change":"2023-10-16T20:07:25.288050+0000","last_active":"2023-10-16T20:07:26.277396+0000","last_peered":"2023-10-16T20:07:26.277396+0000","last_clean":"2023-10-16T20:07:26.277396+0000","last_became_active":"2023-10-16T20:07:25.287781+0000","last_became_peered":"2023-10-16T20:07:25.287781+0000","last_unstale":"2023-10-16T20:07:26.277396+0000","last_undegraded":"2023-10-16T20:07:26.277396+0000","last_fullsized":"2023-10-16T20:07:26.277396+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:07:42.421793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211579+0000","last_change":"2023-10-16T20:07:26.237328+0000","last_active":"2023-10-16T20:07:27.211579+0000","last_peered":"2023-10-16T20:07:27.211579+0000","last_clean":"2023-10-16T20:07:27.211579+0000","last_became_active":"2023-10-16T20:07:26.237047+0000","last_became_peered":"2023-10-16T20:07:26.237047+0000","last_unstale":"2023-10-16T20:07:27.211579+0000","last_undegraded":"2023-10-16T20:07:27.211579+0000","last_fullsized":"2023-10-16T20:07:27.211579+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:17:40.171938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"57'1","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795196+0000","last_change":"2023-10-16T20:07:25.288443+0000","last_active":"2023-10-16T20:07:27.795196+0000","last_peered":"2023-10-16T20:07:27.795196+0000","last_clean":"2023-10-16T20:07:27.795196+0000","last_became_active":"2023-10-16T20:07:25.288154+0000","last_became_peered":"2023-10-16T20:07:25.288154+0000","last_unstale":"2023-10-16T20:07:27.795196+0000","last_undegraded":"2023-10-16T20:07:27.795196+0000","last_fullsized":"2023-10-16T20:07:27.795196+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:00:13.174520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.214028+0000","last_change":"2023-10-16T20:07:26.235840+0000","last_active":"2023-10-16T20:07:27.214028+0000","last_peered":"2023-10-16T20:07:27.214028+0000","last_clean":"2023-10-16T20:07:27.214028+0000","last_became_active":"2023-10-16T20:07:26.235489+0000","last_became_peered":"2023-10-16T20:07:26.235489+0000","last_unstale":"2023-10-16T20:07:27.214028+0000","last_undegraded":"2023-10-16T20:07:27.214028+0000","last_fullsized":"2023-10-16T20:07:27.214028+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:52:00.457041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-16T20:07:26.221154+0000","last_change":"2023-10-16T20:07:25.286338+0000","last_active":"2023-10-16T20:07:26.221154+0000","last_peered":"2023-10-16T20:07:26.221154+0000","last_clean":"2023-10-16T20:07:26.221154+0000","last_became_active":"2023-10-16T20:07:25.286056+0000","last_became_peered":"2023-10-16T20:07:25.286056+0000","last_unstale":"2023-10-16T20:07:26.221154+0000","last_undegraded":"2023-10-16T20:07:26.221154+0000","last_fullsized":"2023-10-16T20:07:26.221154+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:58:41.853643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794861+0000","last_change":"2023-10-16T20:07:26.228461+0000","last_active":"2023-10-16T20:07:27.794861+0000","last_peered":"2023-10-16T20:07:27.794861+0000","last_clean":"2023-10-16T20:07:27.794861+0000","last_became_active":"2023-10-16T20:07:26.228148+0000","last_became_peered":"2023-10-16T20:07:26.228148+0000","last_unstale":"2023-10-16T20:07:27.794861+0000","last_undegraded":"2023-10-16T20:07:27.794861+0000","last_fullsized":"2023-10-16T20:07:27.794861+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:07:41.206845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211975+0000","last_change":"2023-10-16T20:07:25.287428+0000","last_active":"2023-10-16T20:07:27.211975+0000","last_peered":"2023-10-16T20:07:27.211975+0000","last_clean":"2023-10-16T20:07:27.211975+0000","last_became_active":"2023-10-16T20:07:25.287146+0000","last_became_peered":"2023-10-16T20:07:25.287146+0000","last_unstale":"2023-10-16T20:07:27.211975+0000","last_undegraded":"2023-10-16T20:07:27.211975+0000","last_fullsized":"2023-10-16T20:07:27.211975+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:26:49.396227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794363+0000","last_change":"2023-10-16T20:07:26.227646+0000","last_active":"2023-10-16T20:07:27.794363+0000","last_peered":"2023-10-16T20:07:27.794363+0000","last_clean":"2023-10-16T20:07:27.794363+0000","last_became_active":"2023-10-16T20:07:26.227380+0000","last_became_peered":"2023-10-16T20:07:26.227380+0000","last_unstale":"2023-10-16T20:07:27.794363+0000","last_undegraded":"2023-10-16T20:07:27.794363+0000","last_fullsized":"2023-10-16T20:07:27.794363+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:46:20.681993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794887+0000","last_change":"2023-10-16T20:07:25.293106+0000","last_active":"2023-10-16T20:07:27.794887+0000","last_peered":"2023-10-16T20:07:27.794887+0000","last_clean":"2023-10-16T20:07:27.794887+0000","last_became_active":"2023-10-16T20:07:25.292583+0000","last_became_peered":"2023-10-16T20:07:25.292583+0000","last_unstale":"2023-10-16T20:07:27.794887+0000","last_undegraded":"2023-10-16T20:07:27.794887+0000","last_fullsized":"2023-10-16T20:07:27.794887+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:11:57.982645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794074+0000","last_change":"2023-10-16T20:07:26.228514+0000","last_active":"2023-10-16T20:07:27.794074+0000","last_peered":"2023-10-16T20:07:27.794074+0000","last_clean":"2023-10-16T20:07:27.794074+0000","last_became_active":"2023-10-16T20:07:26.228225+0000","last_became_peered":"2023-10-16T20:07:26.228225+0000","last_unstale":"2023-10-16T20:07:27.794074+0000","last_undegraded":"2023-10-16T20:07:27.794074+0000","last_fullsized":"2023-10-16T20:07:27.794074+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:01:23.910122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794652+0000","last_change":"2023-10-16T20:07:25.286566+0000","last_active":"2023-10-16T20:07:27.794652+0000","last_peered":"2023-10-16T20:07:27.794652+0000","last_clean":"2023-10-16T20:07:27.794652+0000","last_became_active":"2023-10-16T20:07:25.285812+0000","last_became_peered":"2023-10-16T20:07:25.285812+0000","last_unstale":"2023-10-16T20:07:27.794652+0000","last_undegraded":"2023-10-16T20:07:27.794652+0000","last_fullsized":"2023-10-16T20:07:27.794652+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:33:14.446674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795249+0000","last_change":"2023-10-16T20:07:26.219929+0000","last_active":"2023-10-16T20:07:27.795249+0000","last_peered":"2023-10-16T20:07:27.795249+0000","last_clean":"2023-10-16T20:07:27.795249+0000","last_became_active":"2023-10-16T20:07:26.219429+0000","last_became_peered":"2023-10-16T20:07:26.219429+0000","last_unstale":"2023-10-16T20:07:27.795249+0000","last_undegraded":"2023-10-16T20:07:27.795249+0000","last_fullsized":"2023-10-16T20:07:27.795249+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:48:05.528118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"57'2","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795653+0000","last_change":"2023-10-16T20:07:25.294532+0000","last_active":"2023-10-16T20:07:27.795653+0000","last_peered":"2023-10-16T20:07:27.795653+0000","last_clean":"2023-10-16T20:07:27.795653+0000","last_became_active":"2023-10-16T20:07:25.294069+0000","last_became_peered":"2023-10-16T20:07:25.294069+0000","last_unstale":"2023-10-16T20:07:27.795653+0000","last_undegraded":"2023-10-16T20:07:27.795653+0000","last_fullsized":"2023-10-16T20:07:27.795653+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:19:54.366332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795505+0000","last_change":"2023-10-16T20:07:26.227126+0000","last_active":"2023-10-16T20:07:27.795505+0000","last_peered":"2023-10-16T20:07:27.795505+0000","last_clean":"2023-10-16T20:07:27.795505+0000","last_became_active":"2023-10-16T20:07:26.226825+0000","last_became_peered":"2023-10-16T20:07:26.226825+0000","last_unstale":"2023-10-16T20:07:27.795505+0000","last_undegraded":"2023-10-16T20:07:27.795505+0000","last_fullsized":"2023-10-16T20:07:27.795505+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:07:42.996110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-16T20:07:26.221609+0000","last_change":"2023-10-16T20:07:25.283798+0000","last_active":"2023-10-16T20:07:26.221609+0000","last_peered":"2023-10-16T20:07:26.221609+0000","last_clean":"2023-10-16T20:07:26.221609+0000","last_became_active":"2023-10-16T20:07:25.283541+0000","last_became_peered":"2023-10-16T20:07:25.283541+0000","last_unstale":"2023-10-16T20:07:26.221609+0000","last_undegraded":"2023-10-16T20:07:26.221609+0000","last_fullsized":"2023-10-16T20:07:26.221609+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:16:06.823192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795328+0000","last_change":"2023-10-16T20:07:26.231142+0000","last_active":"2023-10-16T20:07:27.795328+0000","last_peered":"2023-10-16T20:07:27.795328+0000","last_clean":"2023-10-16T20:07:27.795328+0000","last_became_active":"2023-10-16T20:07:26.230516+0000","last_became_peered":"2023-10-16T20:07:26.230516+0000","last_unstale":"2023-10-16T20:07:27.795328+0000","last_undegraded":"2023-10-16T20:07:27.795328+0000","last_fullsized":"2023-10-16T20:07:27.795328+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:12:55.180570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213956+0000","last_change":"2023-10-16T20:07:25.279560+0000","last_active":"2023-10-16T20:07:27.213956+0000","last_peered":"2023-10-16T20:07:27.213956+0000","last_clean":"2023-10-16T20:07:27.213956+0000","last_became_active":"2023-10-16T20:07:25.278610+0000","last_became_peered":"2023-10-16T20:07:25.278610+0000","last_unstale":"2023-10-16T20:07:27.213956+0000","last_undegraded":"2023-10-16T20:07:27.213956+0000","last_fullsized":"2023-10-16T20:07:27.213956+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:57:47.332831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794832+0000","last_change":"2023-10-16T20:07:26.233885+0000","last_active":"2023-10-16T20:07:27.794832+0000","last_peered":"2023-10-16T20:07:27.794832+0000","last_clean":"2023-10-16T20:07:27.794832+0000","last_became_active":"2023-10-16T20:07:26.233332+0000","last_became_peered":"2023-10-16T20:07:26.233332+0000","last_unstale":"2023-10-16T20:07:27.794832+0000","last_undegraded":"2023-10-16T20:07:27.794832+0000","last_fullsized":"2023-10-16T20:07:27.794832+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:26:09.886058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795499+0000","last_change":"2023-10-16T20:07:25.277203+0000","last_active":"2023-10-16T20:07:27.795499+0000","last_peered":"2023-10-16T20:07:27.795499+0000","last_clean":"2023-10-16T20:07:27.795499+0000","last_became_active":"2023-10-16T20:07:25.276442+0000","last_became_peered":"2023-10-16T20:07:25.276442+0000","last_unstale":"2023-10-16T20:07:27.795499+0000","last_undegraded":"2023-10-16T20:07:27.795499+0000","last_fullsized":"2023-10-16T20:07:27.795499+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:59:55.925458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211630+0000","last_change":"2023-10-16T20:07:26.232496+0000","last_active":"2023-10-16T20:07:27.211630+0000","last_peered":"2023-10-16T20:07:27.211630+0000","last_clean":"2023-10-16T20:07:27.211630+0000","last_became_active":"2023-10-16T20:07:26.232250+0000","last_became_peered":"2023-10-16T20:07:26.232250+0000","last_unstale":"2023-10-16T20:07:27.211630+0000","last_undegraded":"2023-10-16T20:07:27.211630+0000","last_fullsized":"2023-10-16T20:07:27.211630+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:31:02.703353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795042+0000","last_change":"2023-10-16T20:07:25.292231+0000","last_active":"2023-10-16T20:07:27.795042+0000","last_peered":"2023-10-16T20:07:27.795042+0000","last_clean":"2023-10-16T20:07:27.795042+0000","last_became_active":"2023-10-16T20:07:25.291896+0000","last_became_peered":"2023-10-16T20:07:25.291896+0000","last_unstale":"2023-10-16T20:07:27.795042+0000","last_undegraded":"2023-10-16T20:07:27.795042+0000","last_fullsized":"2023-10-16T20:07:27.795042+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:48:16.846875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213397+0000","last_change":"2023-10-16T20:07:26.240662+0000","last_active":"2023-10-16T20:07:27.213397+0000","last_peered":"2023-10-16T20:07:27.213397+0000","last_clean":"2023-10-16T20:07:27.213397+0000","last_became_active":"2023-10-16T20:07:26.240340+0000","last_became_peered":"2023-10-16T20:07:26.240340+0000","last_unstale":"2023-10-16T20:07:27.213397+0000","last_undegraded":"2023-10-16T20:07:27.213397+0000","last_fullsized":"2023-10-16T20:07:27.213397+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:48:05.528118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.795183+0000","last_change":"2023-10-16T20:07:25.284134+0000","last_active":"2023-10-16T20:07:27.795183+0000","last_peered":"2023-10-16T20:07:27.795183+0000","last_clean":"2023-10-16T20:07:27.795183+0000","last_became_active":"2023-10-16T20:07:25.281555+0000","last_became_peered":"2023-10-16T20:07:25.281555+0000","last_unstale":"2023-10-16T20:07:27.795183+0000","last_undegraded":"2023-10-16T20:07:27.795183+0000","last_fullsized":"2023-10-16T20:07:27.795183+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:50:25.012567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794226+0000","last_change":"2023-10-16T20:07:26.230788+0000","last_active":"2023-10-16T20:07:27.794226+0000","last_peered":"2023-10-16T20:07:27.794226+0000","last_clean":"2023-10-16T20:07:27.794226+0000","last_became_active":"2023-10-16T20:07:26.230419+0000","last_became_peered":"2023-10-16T20:07:26.230419+0000","last_unstale":"2023-10-16T20:07:27.794226+0000","last_undegraded":"2023-10-16T20:07:27.794226+0000","last_fullsized":"2023-10-16T20:07:27.794226+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:06:41.715102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.794377+0000","last_change":"2023-10-16T20:07:25.291904+0000","last_active":"2023-10-16T20:07:27.794377+0000","last_peered":"2023-10-16T20:07:27.794377+0000","last_clean":"2023-10-16T20:07:27.794377+0000","last_became_active":"2023-10-16T20:07:25.291633+0000","last_became_peered":"2023-10-16T20:07:25.291633+0000","last_unstale":"2023-10-16T20:07:27.794377+0000","last_undegraded":"2023-10-16T20:07:27.794377+0000","last_fullsized":"2023-10-16T20:07:27.794377+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:50.000078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211440+0000","last_change":"2023-10-16T20:07:26.232488+0000","last_active":"2023-10-16T20:07:27.211440+0000","last_peered":"2023-10-16T20:07:27.211440+0000","last_clean":"2023-10-16T20:07:27.211440+0000","last_became_active":"2023-10-16T20:07:26.232162+0000","last_became_peered":"2023-10-16T20:07:26.232162+0000","last_unstale":"2023-10-16T20:07:27.211440+0000","last_undegraded":"2023-10-16T20:07:27.211440+0000","last_fullsized":"2023-10-16T20:07:27.211440+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:52:00.457041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.212059+0000","last_change":"2023-10-16T20:07:25.288746+0000","last_active":"2023-10-16T20:07:27.212059+0000","last_peered":"2023-10-16T20:07:27.212059+0000","last_clean":"2023-10-16T20:07:27.212059+0000","last_became_active":"2023-10-16T20:07:25.288469+0000","last_became_peered":"2023-10-16T20:07:25.288469+0000","last_unstale":"2023-10-16T20:07:27.212059+0000","last_undegraded":"2023-10-16T20:07:27.212059+0000","last_fullsized":"2023-10-16T20:07:27.212059+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:08:50.174794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.211357+0000","last_change":"2023-10-16T20:07:26.228623+0000","last_active":"2023-10-16T20:07:27.211357+0000","last_peered":"2023-10-16T20:07:27.211357+0000","last_clean":"2023-10-16T20:07:27.211357+0000","last_became_active":"2023-10-16T20:07:26.226488+0000","last_became_peered":"2023-10-16T20:07:26.226488+0000","last_unstale":"2023-10-16T20:07:27.211357+0000","last_undegraded":"2023-10-16T20:07:27.211357+0000","last_fullsized":"2023-10-16T20:07:27.211357+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:25.253517+0000","last_clean_scrub_stamp":"2023-10-16T20:07:25.253517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:00:35.644195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213885+0000","last_change":"2023-10-16T20:07:25.279504+0000","last_active":"2023-10-16T20:07:27.213885+0000","last_peered":"2023-10-16T20:07:27.213885+0000","last_clean":"2023-10-16T20:07:27.213885+0000","last_became_active":"2023-10-16T20:07:25.279177+0000","last_became_peered":"2023-10-16T20:07:25.279177+0000","last_unstale":"2023-10-16T20:07:27.213885+0000","last_undegraded":"2023-10-16T20:07:27.213885+0000","last_fullsized":"2023-10-16T20:07:27.213885+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:07:24.250255+0000","last_clean_scrub_stamp":"2023-10-16T20:07:24.250255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:18:10.920675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":197,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-16T20:07:26.219256+0000","last_change":"2023-10-16T20:04:46.335822+0000","last_active":"2023-10-16T20:07:26.219256+0000","last_peered":"2023-10-16T20:07:26.219256+0000","last_clean":"2023-10-16T20:07:26.219256+0000","last_became_active":"2023-10-16T20:04:46.334617+0000","last_became_peered":"2023-10-16T20:04:46.334617+0000","last_unstale":"2023-10-16T20:07:26.219256+0000","last_undegraded":"2023-10-16T20:07:26.219256+0000","last_fullsized":"2023-10-16T20:07:26.219256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:45.306240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:45.306240+0000","last_clean_scrub_stamp":"2023-10-16T20:04:45.306240+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:42:11.153721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509476,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1290240,"data_stored":898654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.70299999999999996,"15min":0.70899999999999996},"min":{"1min":0.372,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93400000000000005,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70999999999999996,"15min":0.70799999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.95299999999999996,"5min":1.077,"15min":1.077},"last":0.73299999999999998}]},{"osd":1,"last update":"Mon Oct 16 20:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.70099999999999996,"15min":0.70299999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98499999999999999,"5min":1.78,"15min":1.78},"last":0.745},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.70799999999999996,"15min":0.71599999999999997},"min":{"1min":0.45100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95899999999999996,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":0.65700000000000003}]},{"osd":2,"last update":"Mon Oct 16 20:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.73199999999999998,"15min":0.74399999999999999},"min":{"1min":0.38200000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84599999999999997,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.78700000000000003,"15min":0.79600000000000004},"min":{"1min":0.47399999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.026,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.86599999999999999}]},{"osd":3,"last update":"Mon Oct 16 20:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.70399999999999996,"15min":0.70699999999999996},"min":{"1min":0.52400000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98799999999999999,"5min":1.994,"15min":1.994},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.77200000000000002,"15min":0.79000000000000004},"min":{"1min":0.46700000000000003,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.92000000000000004,"5min":2.629,"15min":2.629},"last":0.91300000000000003}]},{"osd":4,"last update":"Mon Oct 16 20:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.64600000000000002,"15min":0.66000000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91300000000000003,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.65700000000000003,"15min":0.66200000000000003},"min":{"1min":0.42699999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84999999999999998,"5min":1.944,"15min":1.944},"last":0.58599999999999997}]},{"osd":5,"last update":"Mon Oct 16 20:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.73099999999999998,"15min":0.749},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.86699999999999999,"5min":2.8809999999999998,"15min":2.8809999999999998},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.71499999999999997,"15min":0.71699999999999997},"min":{"1min":0.55300000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.86499999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.54600000000000004}]},{"osd":6,"last update":"Mon Oct 16 20:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.754,"15min":0.75700000000000001},"min":{"1min":0.58099999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.96699999999999997,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.73599999999999999,"15min":0.749},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.85199999999999998,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.77300000000000002}]}]},{"osd":6,"up_from":15,"seq":64424509477,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31856,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":688128,"data_stored":308126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.85299999999999998,"15min":0.879},"min":{"1min":0.50800000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94899999999999995,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.83599999999999997,"15min":0.87},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90700000000000003,"5min":4.6139999999999999,"15min":4.6139999999999999},"last":0.51600000000000001}]},{"osd":1,"last update":"Mon Oct 16 20:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.879,"15min":0.90400000000000003},"min":{"1min":0.495,"5min":0.438,"15min":0.438},"max":{"1min":0.995,"5min":4.6639999999999997,"15min":4.6639999999999997},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":1.048,"15min":1.1080000000000001},"min":{"1min":0.44400000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90800000000000003,"5min":4.8140000000000001,"15min":4.8140000000000001},"last":0.61499999999999999}]},{"osd":2,"last update":"Mon Oct 16 20:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.93500000000000005,"15min":0.97399999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88400000000000001,"5min":5.3769999999999998,"15min":5.3769999999999998},"last":0.502},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.67700000000000005,"15min":0.67100000000000004},"min":{"1min":0.48299999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.54600000000000004}]},{"osd":3,"last update":"Mon Oct 16 20:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.91000000000000003,"15min":0.93799999999999994},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.97399999999999998,"5min":4.5460000000000003,"15min":4.5460000000000003},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.76100000000000001,"15min":0.77100000000000002},"min":{"1min":0.51200000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.92200000000000004,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.53300000000000003}]},{"osd":4,"last update":"Mon Oct 16 20:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.74199999999999999,"15min":0.754},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88300000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.69199999999999995,"15min":0.69899999999999995},"min":{"1min":0.45000000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.94599999999999995,"5min":1.355,"15min":1.355},"last":0.42399999999999999}]},{"osd":5,"last update":"Mon Oct 16 20:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.71299999999999997,"15min":0.72299999999999998},"min":{"1min":0.441,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88800000000000001,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.441},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80400000000000005,"15min":0.80700000000000005},"min":{"1min":0.57599999999999996,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.98699999999999999,"5min":1.218,"15min":1.218},"last":0.59399999999999997}]},{"osd":7,"last update":"Mon Oct 16 20:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.748,"15min":0.748},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.97799999999999998,"5min":1.052,"15min":1.052},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.79700000000000004,"15min":0.80300000000000005},"min":{"1min":0.52900000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.91300000000000003,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.48899999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509475,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":696320,"data_stored":308778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.63300000000000001,"15min":0.629},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.872,"5min":1.264,"15min":1.264},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.64500000000000002,"15min":0.64300000000000002},"min":{"1min":0.436,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.85499999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.64000000000000001}]},{"osd":1,"last update":"Mon Oct 16 20:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.61599999999999999,"15min":0.60799999999999998},"min":{"1min":0.46999999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.64600000000000002,"15min":0.64000000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.51100000000000001}]},{"osd":2,"last update":"Mon Oct 16 20:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67200000000000004,"15min":0.67000000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.85699999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.66000000000000003,"15min":0.65300000000000002},"min":{"1min":0.52700000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.66400000000000003}]},{"osd":3,"last update":"Mon Oct 16 20:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.77600000000000002,"15min":0.78200000000000003},"min":{"1min":0.498,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.92000000000000004,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.746,"5min":0.70799999999999996,"15min":0.70199999999999996},"min":{"1min":0.56200000000000006,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90400000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.70699999999999996}]},{"osd":4,"last update":"Mon Oct 16 20:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.42799999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91100000000000003,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.746},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70299999999999996,"15min":0.70199999999999996},"min":{"1min":0.44500000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.442}]},{"osd":6,"last update":"Mon Oct 16 20:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67000000000000004,"15min":0.66900000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73199999999999998,"15min":0.73099999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.61699999999999999}]},{"osd":7,"last update":"Mon Oct 16 20:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.75,"15min":0.75},"min":{"1min":0.54500000000000004,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.96999999999999997,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.74299999999999999,"15min":0.746},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.60199999999999998}]}]},{"osd":4,"up_from":15,"seq":64424509475,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":700416,"data_stored":309738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.77900000000000003,"15min":0.79900000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.81799999999999995,"5min":3.835,"15min":3.835},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.79400000000000004,"15min":0.81799999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.94799999999999995,"5min":3.7610000000000001,"15min":3.7610000000000001},"last":0.91700000000000004}]},{"osd":1,"last update":"Mon Oct 16 20:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67400000000000004,"15min":0.67600000000000005},"min":{"1min":0.48099999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89100000000000001,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.875},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.50600000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.89100000000000001}]},{"osd":2,"last update":"Mon Oct 16 20:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.73399999999999999,"15min":0.73499999999999999},"min":{"1min":0.55700000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.92900000000000005,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.57099999999999995,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.91000000000000003,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.77800000000000002}]},{"osd":3,"last update":"Mon Oct 16 20:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.79900000000000004,"15min":0.80400000000000005},"min":{"1min":0.627,"5min":0.44,"15min":0.44},"max":{"1min":0.95399999999999996,"5min":1.163,"15min":1.163},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.79100000000000004,"15min":0.79500000000000004},"min":{"1min":0.64000000000000001,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.125,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.93999999999999995}]},{"osd":5,"last update":"Mon Oct 16 20:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.66300000000000003,"15min":0.67100000000000004},"min":{"1min":0.439,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86799999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.69099999999999995,"15min":0.69899999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85599999999999998,"5min":1.089,"15min":1.089},"last":0.85799999999999998}]},{"osd":6,"last update":"Mon Oct 16 20:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.72599999999999998,"15min":0.73499999999999999},"min":{"1min":0.51900000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94299999999999995,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.76700000000000002,"15min":0.78500000000000003},"min":{"1min":0.48299999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.96099999999999997,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.60899999999999999}]},{"osd":7,"last update":"Mon Oct 16 20:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.77300000000000002,"15min":0.78700000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.84199999999999997,"15min":0.85399999999999998},"min":{"1min":0.59199999999999997,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.97299999999999998,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.74399999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509476,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":696320,"data_stored":308182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.45600000000000002,"15min":0.46100000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.61099999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.46700000000000003,"15min":0.46600000000000003},"min":{"1min":0.38900000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.64100000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.49199999999999999}]},{"osd":1,"last update":"Mon Oct 16 20:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.45000000000000001,"15min":0.45700000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64600000000000002,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.51100000000000001,"15min":0.51700000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66400000000000003,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.55000000000000004}]},{"osd":2,"last update":"Mon Oct 16 20:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.499,"15min":0.501},"min":{"1min":0.39400000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.74099999999999999,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.69399999999999995,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.59999999999999998}]},{"osd":4,"last update":"Mon Oct 16 20:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.77200000000000002,"15min":0.78000000000000003},"min":{"1min":0.44500000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.94799999999999995,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.014},{"interface":"front","average":{"1min":0.753,"5min":0.80900000000000005,"15min":0.81899999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.97099999999999997,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.96099999999999997}]},{"osd":5,"last update":"Mon Oct 16 20:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.75800000000000001,"15min":0.76800000000000002},"min":{"1min":0.51200000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.93100000000000005,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.88},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.78200000000000003,"15min":0.78900000000000003},"min":{"1min":0.56200000000000006,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0169999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.81499999999999995}]},{"osd":6,"last update":"Mon Oct 16 20:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.77600000000000002,"15min":0.78800000000000003},"min":{"1min":0.52800000000000002,"5min":0.433,"15min":0.433},"max":{"1min":1.034,"5min":1.774,"15min":1.774},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.80300000000000005,"15min":0.81699999999999995},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.004,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.92800000000000005}]},{"osd":7,"last update":"Mon Oct 16 20:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.79400000000000004,"15min":0.80600000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.98799999999999999,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.79200000000000004,"15min":0.80600000000000005},"min":{"1min":0.503,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.89700000000000002,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.79300000000000004}]}]},{"osd":2,"up_from":15,"seq":64424509475,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":684032,"data_stored":301537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.40699999999999997,"15min":0.41299999999999998},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.50800000000000001,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.47899999999999998,"15min":0.48499999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.70899999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.41499999999999998}]},{"osd":1,"last update":"Mon Oct 16 20:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.46700000000000003,"15min":0.46600000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.61299999999999999,"5min":0.623,"15min":0.623},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.36899999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.45200000000000001}]},{"osd":3,"last update":"Mon Oct 16 20:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.47499999999999998,"15min":0.47699999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.40100000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.34499999999999997}]},{"osd":4,"last update":"Mon Oct 16 20:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.76300000000000001,"15min":0.77100000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.91900000000000004,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.749,"15min":0.75800000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0660000000000001,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.55800000000000005}]},{"osd":5,"last update":"Mon Oct 16 20:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.72599999999999998,"15min":0.73199999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.93200000000000005,"5min":1.145,"15min":1.145},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.747,"15min":0.754},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.91900000000000004,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.54300000000000004}]},{"osd":6,"last update":"Mon Oct 16 20:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.752,"15min":0.753},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97099999999999997,"5min":1.165,"15min":1.165},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.76200000000000001,"15min":0.76800000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.95399999999999996,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.64000000000000001}]},{"osd":7,"last update":"Mon Oct 16 20:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76900000000000002,"15min":0.77000000000000002},"min":{"1min":0.58099999999999996,"5min":0.502,"15min":0.502},"max":{"1min":1.0369999999999999,"5min":1.167,"15min":1.167},"last":0.624},{"interface":"front","average":{"1min":0.745,"5min":0.77700000000000002,"15min":0.78300000000000003},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.95099999999999996,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.69999999999999996}]}]},{"osd":1,"up_from":15,"seq":64424509476,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":692224,"data_stored":308216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.53000000000000003,"15min":0.55100000000000005},"min":{"1min":0.30399999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.58899999999999997,"5min":3.867,"15min":3.867},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.442,"5min":0.48699999999999999,"15min":0.495},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.63500000000000001,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.59399999999999997}]},{"osd":2,"last update":"Mon Oct 16 20:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.69499999999999995,"15min":0.73699999999999999},"min":{"1min":0.32900000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.72299999999999998,"5min":5.6040000000000001,"15min":5.6040000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.76000000000000001,"15min":0.81200000000000006},"min":{"1min":0.34699999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.58299999999999996,"5min":7.8380000000000001,"15min":7.8380000000000001},"last":0.627}]},{"osd":3,"last update":"Mon Oct 16 20:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.68799999999999994,"15min":0.71999999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.63500000000000001,"5min":5.2039999999999997,"15min":5.2039999999999997},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.63700000000000001,"15min":0.66600000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.626,"5min":3.9470000000000001,"15min":3.9470000000000001},"last":0.67000000000000004}]},{"osd":4,"last update":"Mon Oct 16 20:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.84399999999999997,"15min":0.86399999999999999},"min":{"1min":0.48299999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.91800000000000004,"5min":4.1500000000000004,"15min":4.1500000000000004},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.85299999999999998,"15min":0.875},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.94099999999999995,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":0.93300000000000005}]},{"osd":5,"last update":"Mon Oct 16 20:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.753,"15min":0.75800000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93100000000000005,"5min":1.496,"15min":1.496},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.86499999999999999,"15min":0.88900000000000001},"min":{"1min":0.58399999999999996,"5min":0.501,"15min":0.501},"max":{"1min":0.874,"5min":4.0910000000000002,"15min":4.0910000000000002},"last":0.96999999999999997}]},{"osd":6,"last update":"Mon Oct 16 20:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.86099999999999999,"15min":0.88200000000000001},"min":{"1min":0.57699999999999996,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.93300000000000005,"5min":4.21,"15min":4.21},"last":1.01},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.76200000000000001,"15min":0.76500000000000001},"min":{"1min":0.58699999999999997,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.89300000000000002,"5min":1.401,"15min":1.401},"last":0.95299999999999996}]},{"osd":7,"last update":"Mon Oct 16 20:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79800000000000004,"15min":0.79900000000000004},"min":{"1min":0.64800000000000002,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.96199999999999997,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.86299999999999999,"15min":0.88300000000000001},"min":{"1min":0.54100000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.90500000000000003,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":0.88200000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574885,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28288,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343552,"statfs":{"total":96636764160,"available":96607797248,"internally_reserved":0,"allocated":1298432,"data_stored":900688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 20:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.56200000000000006,"15min":0.59099999999999997},"min":{"1min":0.27800000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.56699999999999995,"5min":3.9289999999999998,"15min":3.9289999999999998},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.57699999999999996,"15min":0.60899999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.55700000000000005,"5min":4.0910000000000002,"15min":4.0910000000000002},"last":0.499}]},{"osd":2,"last update":"Mon Oct 16 20:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.64500000000000002,"15min":0.67500000000000004},"min":{"1min":0.33400000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.59599999999999997,"5min":4.1520000000000001,"15min":4.1520000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.67400000000000004,"15min":0.70799999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.60499999999999998,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.68700000000000006}]},{"osd":3,"last update":"Mon Oct 16 20:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.621,"15min":0.64700000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63100000000000001,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.39500000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.73499999999999999,"5min":1.006,"15min":1.006},"last":0.66000000000000003}]},{"osd":4,"last update":"Mon Oct 16 20:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.83499999999999996,"15min":0.86799999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.85599999999999998,"5min":4.5250000000000004,"15min":4.5250000000000004},"last":0.87},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.746,"15min":0.75},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0189999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.79000000000000004}]},{"osd":5,"last update":"Mon Oct 16 20:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.874,"15min":0.90600000000000003},"min":{"1min":0.505,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.88200000000000001,"5min":3.9990000000000001,"15min":3.9990000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.92900000000000005,"15min":0.96299999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.006,"5min":4.4779999999999998,"15min":4.4779999999999998},"last":0.73299999999999998}]},{"osd":6,"last update":"Mon Oct 16 20:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.76300000000000001,"15min":0.76900000000000002},"min":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0129999999999999,"5min":1.363,"15min":1.363},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.90900000000000003,"15min":0.94699999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.98299999999999998,"5min":3.956,"15min":3.956},"last":0.77000000000000002}]},{"osd":7,"last update":"Mon Oct 16 20:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.91600000000000004,"15min":0.94899999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0349999999999999,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.90800000000000003,"15min":0.94199999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.96699999999999997,"5min":4.5640000000000001,"15min":4.5640000000000001},"last":0.80400000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T20:07:30.720 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T20:07:30.721 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=testcephfs1' mds allow 2023-10-16T20:07:30.898 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.893+0000 7f7ce7417640 1 Processor -- start 2023-10-16T20:07:30.899 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.893+0000 7f7ce7417640 1 -- start start 2023-10-16T20:07:30.900 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.897+0000 7f7ce7417640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7ce015cb70 0x7f7ce00b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:30.900 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.897+0000 7f7ce7417640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f7ce00b4530 0x7f7ce00b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:30.900 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.897+0000 7f7ce7417640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ce00b4ed0 con 0x7f7ce00d0bd0 2023-10-16T20:07:30.900 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.897+0000 7f7ce7417640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ce00b7d20 con 0x7f7ce00b4530 2023-10-16T20:07:30.901 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.897+0000 7f7ce7417640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce00b7110 con 0x7f7ce015cb70 2023-10-16T20:07:30.901 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.897+0000 7f7ce49b2640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f7ce00b4530 0x7f7ce00b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:30.901 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.897+0000 7f7ce49b2640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f7ce00b4530 0x7f7ce00b4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38642/0 (socket says 172.21.15.88:38642) 2023-10-16T20:07:30.901 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.897+0000 7f7ce49b2640 1 -- 172.21.15.88:0/2083593632 learned_addr learned my addr 172.21.15.88:0/2083593632 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:30.901 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.897+0000 7f7ce51b3640 1 --2- 172.21.15.88:0/2083593632 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7ce015cb70 0x7f7ce00b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:30.902 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.897+0000 7f7ce49b2640 1 -- 172.21.15.88:0/2083593632 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7ce015cb70 msgr2=0x7f7ce00b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:30.902 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.897+0000 7f7ce49b2640 1 --2- 172.21.15.88:0/2083593632 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7ce015cb70 0x7f7ce00b83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:30.902 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.897+0000 7f7ce49b2640 1 -- 172.21.15.88:0/2083593632 >> v1:172.21.15.94:6789/0 conn(0x7f7ce00d0bd0 legacy=0x7f7ce00d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:30.902 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.897+0000 7f7ce49b2640 1 -- 172.21.15.88:0/2083593632 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce00b73f0 con 0x7f7ce00b4530 2023-10-16T20:07:30.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.897+0000 7f7ce49b2640 1 --2- 172.21.15.88:0/2083593632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ce00b4530 0x7f7ce00b4990 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0009870 tx=0x7f7cd001ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c912219a667324e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:30.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.897+0000 7f7cd7fff640 1 -- 172.21.15.88:0/2083593632 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd00a8a30 con 0x7f7ce00b4530 2023-10-16T20:07:30.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.897+0000 7f7cd7fff640 1 -- 172.21.15.88:0/2083593632 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd00a8b90 con 0x7f7ce00b4530 2023-10-16T20:07:30.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.897+0000 7f7cd7fff640 1 -- 172.21.15.88:0/2083593632 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd00b0440 con 0x7f7ce00b4530 2023-10-16T20:07:30.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.901+0000 7f7ce7417640 1 -- 172.21.15.88:0/2083593632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ce00b4530 msgr2=0x7f7ce00b4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:30.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.901+0000 7f7ce7417640 1 --2- 172.21.15.88:0/2083593632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ce00b4530 0x7f7ce00b4990 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0009870 tx=0x7f7cd001ed30 comp rx=0 tx=0).stop 2023-10-16T20:07:30.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.901+0000 7f7ce7417640 1 -- 172.21.15.88:0/2083593632 shutdown_connections 2023-10-16T20:07:30.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.901+0000 7f7ce7417640 1 --2- 172.21.15.88:0/2083593632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ce00b4530 0x7f7ce00b4990 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:30.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.901+0000 7f7ce7417640 1 --2- 172.21.15.88:0/2083593632 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7ce015cb70 0x7f7ce00b83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:30.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.901+0000 7f7ce7417640 1 -- 172.21.15.88:0/2083593632 >> 172.21.15.88:0/2083593632 conn(0x7f7ce006b6e0 msgr2=0x7f7ce006baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:30.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.901+0000 7f7ce7417640 1 -- 172.21.15.88:0/2083593632 shutdown_connections 2023-10-16T20:07:30.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.901+0000 7f7ce7417640 1 -- 172.21.15.88:0/2083593632 wait complete. 2023-10-16T20:07:30.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.901+0000 7f7ce7417640 1 Processor -- start 2023-10-16T20:07:30.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.901+0000 7f7ce7417640 1 -- start start 2023-10-16T20:07:30.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.901+0000 7f7ce7417640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ce00b4530 0x7f7ce014d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:30.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.901+0000 7f7ce7417640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7ce00d0bd0 0x7f7ce00c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:30.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.901+0000 7f7ce7417640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ce015cb70 0x7f7ce01503a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:30.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.901+0000 7f7ce7417640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce00c6140 con 0x7f7ce00b4530 2023-10-16T20:07:30.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.901+0000 7f7ce7417640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce00c6420 con 0x7f7ce015cb70 2023-10-16T20:07:30.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.901+0000 7f7ce7417640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce00c6700 con 0x7f7ce00d0bd0 2023-10-16T20:07:30.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.901+0000 7f7ce51b3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ce00b4530 0x7f7ce014d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:30.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.901+0000 7f7ce51b3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ce00b4530 0x7f7ce014d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38648/0 (socket says 172.21.15.88:38648) 2023-10-16T20:07:30.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.901+0000 7f7ce51b3640 1 -- 172.21.15.88:0/2308171851 learned_addr learned my addr 172.21.15.88:0/2308171851 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:30.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.901+0000 7f7ce59b4640 1 --2- 172.21.15.88:0/2308171851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ce015cb70 0x7f7ce01503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:30.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.901+0000 7f7ce49b2640 1 --2- 172.21.15.88:0/2308171851 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7ce00d0bd0 0x7f7ce00c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:30.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.905+0000 7f7ce51b3640 1 -- 172.21.15.88:0/2308171851 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7ce00d0bd0 msgr2=0x7f7ce00c71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:30.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.905+0000 7f7ce51b3640 1 --2- 172.21.15.88:0/2308171851 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7ce00d0bd0 0x7f7ce00c71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:30.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.905+0000 7f7ce51b3640 1 -- 172.21.15.88:0/2308171851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ce015cb70 msgr2=0x7f7ce01503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:30.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.905+0000 7f7ce51b3640 1 --2- 172.21.15.88:0/2308171851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ce015cb70 0x7f7ce01503a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:30.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.905+0000 7f7ce51b3640 1 -- 172.21.15.88:0/2308171851 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd00094d0 con 0x7f7ce00b4530 2023-10-16T20:07:30.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.905+0000 7f7ce51b3640 1 --2- 172.21.15.88:0/2308171851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ce00b4530 0x7f7ce014d530 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00d5f0 tx=0x7f7cdc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:30.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.905+0000 7f7cd5ffb640 1 -- 172.21.15.88:0/2308171851 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc0125f0 con 0x7f7ce00b4530 2023-10-16T20:07:30.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.905+0000 7f7cd5ffb640 1 -- 172.21.15.88:0/2308171851 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cdc012750 con 0x7f7ce00b4530 2023-10-16T20:07:30.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.905+0000 7f7cd5ffb640 1 -- 172.21.15.88:0/2308171851 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc012ae0 con 0x7f7ce00b4530 2023-10-16T20:07:30.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.905+0000 7f7ce7417640 1 -- 172.21.15.88:0/2308171851 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce00c69e0 con 0x7f7ce00b4530 2023-10-16T20:07:30.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.905+0000 7f7ce7417640 1 -- 172.21.15.88:0/2308171851 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce014da70 con 0x7f7ce00b4530 2023-10-16T20:07:30.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.905+0000 7f7ce7417640 1 -- 172.21.15.88:0/2308171851 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce004e9c0 con 0x7f7ce00b4530 2023-10-16T20:07:30.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.905+0000 7f7cd5ffb640 1 -- 172.21.15.88:0/2308171851 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7cdc012c40 con 0x7f7ce00b4530 2023-10-16T20:07:30.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.905+0000 7f7cd5ffb640 1 --2- 172.21.15.88:0/2308171851 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7cc40664d0 0x7f7cc4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:30.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.905+0000 7f7cd5ffb640 1 -- 172.21.15.88:0/2308171851 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f7cdc084330 con 0x7f7ce00b4530 2023-10-16T20:07:30.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.905+0000 7f7ce49b2640 1 --2- 172.21.15.88:0/2308171851 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7cc40664d0 0x7f7cc4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:30.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.909+0000 7f7ce49b2640 1 --2- 172.21.15.88:0/2308171851 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7cc40664d0 0x7f7cc4068990 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00b2000 tx=0x7f7cd0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:30.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:30.909+0000 7f7cd5ffb640 1 -- 172.21.15.88:0/2308171851 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7cdc0516f0 con 0x7f7ce00b4530 2023-10-16T20:07:31.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.109+0000 7f7ce7417640 1 -- 172.21.15.88:0/2308171851 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=testcephfs1", "mds", "allow"]} v 0) v1 -- 0x7f7ce00ce340 con 0x7f7ce00b4530 2023-10-16T20:07:31.117 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.113+0000 7f7cd5ffb640 1 -- 172.21.15.88:0/2308171851 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=testcephfs1", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 195+0+152 (secure 0 0 0) 0x7f7cdc007380 con 0x7f7ce00b4530 2023-10-16T20:07:31.117 INFO:teuthology.orchestra.run.smithi088.stdout:[client.0] 2023-10-16T20:07:31.118 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQAvly1looeRAhAAeLshHOVIhQqoTtlPehorIg== 2023-10-16T20:07:31.118 INFO:teuthology.orchestra.run.smithi088.stdout: caps mds = "allow" 2023-10-16T20:07:31.118 INFO:teuthology.orchestra.run.smithi088.stdout: caps mon = "allow r" 2023-10-16T20:07:31.118 INFO:teuthology.orchestra.run.smithi088.stdout: caps osd = "allow rw tag cephfs data=testcephfs1" 2023-10-16T20:07:31.119 INFO:teuthology.orchestra.run.smithi088.stderr:updated caps for client.0 2023-10-16T20:07:31.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.113+0000 7f7ce7417640 1 -- 172.21.15.88:0/2308171851 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7cc40664d0 msgr2=0x7f7cc4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:31.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.113+0000 7f7ce7417640 1 --2- 172.21.15.88:0/2308171851 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7cc40664d0 0x7f7cc4068990 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00b2000 tx=0x7f7cd0009040 comp rx=0 tx=0).stop 2023-10-16T20:07:31.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.113+0000 7f7ce7417640 1 -- 172.21.15.88:0/2308171851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ce00b4530 msgr2=0x7f7ce014d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:31.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.113+0000 7f7ce7417640 1 --2- 172.21.15.88:0/2308171851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ce00b4530 0x7f7ce014d530 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00d5f0 tx=0x7f7cdc00dab0 comp rx=0 tx=0).stop 2023-10-16T20:07:31.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.113+0000 7f7ce7417640 1 -- 172.21.15.88:0/2308171851 shutdown_connections 2023-10-16T20:07:31.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.113+0000 7f7ce7417640 1 --2- 172.21.15.88:0/2308171851 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7cc40664d0 0x7f7cc4068990 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:31.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.113+0000 7f7ce7417640 1 --2- 172.21.15.88:0/2308171851 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7ce00d0bd0 0x7f7ce00c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:31.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.113+0000 7f7ce7417640 1 --2- 172.21.15.88:0/2308171851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ce015cb70 0x7f7ce01503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:31.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.113+0000 7f7ce7417640 1 --2- 172.21.15.88:0/2308171851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ce00b4530 0x7f7ce014d530 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:31.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.113+0000 7f7ce7417640 1 -- 172.21.15.88:0/2308171851 >> 172.21.15.88:0/2308171851 conn(0x7f7ce006b6e0 msgr2=0x7f7ce00d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:31.122 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.113+0000 7f7ce7417640 1 -- 172.21.15.88:0/2308171851 shutdown_connections 2023-10-16T20:07:31.122 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.117+0000 7f7ce7417640 1 -- 172.21.15.88:0/2308171851 wait complete. 2023-10-16T20:07:31.138 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T20:07:31.277 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.273+0000 7f1c3946b640 1 Processor -- start 2023-10-16T20:07:31.277 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.273+0000 7f1c3946b640 1 -- start start 2023-10-16T20:07:31.278 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.273+0000 7f1c3946b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1c340b0c10 0x7f1c340b1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:31.278 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.273+0000 7f1c3946b640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c340b2570 con 0x7f1c340ade20 2023-10-16T20:07:31.278 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.273+0000 7f1c3946b640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c340cca50 con 0x7f1c340ada40 2023-10-16T20:07:31.279 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.273+0000 7f1c3946b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c340b1fb0 con 0x7f1c340b0c10 2023-10-16T20:07:31.279 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.273+0000 7f1c327fc640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f1c340ade20 0x7f1c340b0500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:46962/0 (socket says 172.21.15.88:46962) 2023-10-16T20:07:31.279 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.273+0000 7f1c327fc640 1 -- 172.21.15.88:0/3420650201 learned_addr learned my addr 172.21.15.88:0/3420650201 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:31.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.273+0000 7f1c337fe640 1 --2- 172.21.15.88:0/3420650201 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1c340b0c10 0x7f1c340b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:31.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.273+0000 7f1c31ffb640 1 -- 172.21.15.88:0/3420650201 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1864379384 0 0) 0x7f1c340b2570 con 0x7f1c340ade20 2023-10-16T20:07:31.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.273+0000 7f1c31ffb640 1 -- 172.21.15.88:0/3420650201 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c1c003530 con 0x7f1c340ade20 2023-10-16T20:07:31.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.273+0000 7f1c31ffb640 1 -- 172.21.15.88:0/3420650201 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2891847509 0 0) 0x7f1c340cca50 con 0x7f1c340ada40 2023-10-16T20:07:31.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.273+0000 7f1c31ffb640 1 -- 172.21.15.88:0/3420650201 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c340b2570 con 0x7f1c340ada40 2023-10-16T20:07:31.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.273+0000 7f1c31ffb640 1 -- 172.21.15.88:0/3420650201 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3328937889 0 0) 0x7f1c1c003530 con 0x7f1c340ade20 2023-10-16T20:07:31.283 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.273+0000 7f1c31ffb640 1 -- 172.21.15.88:0/3420650201 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1c340b0c10 msgr2=0x7f1c340b1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:31.283 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.273+0000 7f1c31ffb640 1 --2- 172.21.15.88:0/3420650201 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1c340b0c10 0x7f1c340b1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:31.283 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.273+0000 7f1c31ffb640 1 -- 172.21.15.88:0/3420650201 >> v1:172.21.15.94:6789/0 conn(0x7f1c340ada40 legacy=0x7f1c3415adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:31.283 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.273+0000 7f1c31ffb640 1 -- 172.21.15.88:0/3420650201 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c340b2290 con 0x7f1c340ade20 2023-10-16T20:07:31.284 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.273+0000 7f1c31ffb640 1 -- 172.21.15.88:0/3420650201 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f1c28001680 con 0x7f1c340ade20 2023-10-16T20:07:31.284 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.273+0000 7f1c31ffb640 1 -- 172.21.15.88:0/3420650201 >> v1:172.21.15.88:6789/0 conn(0x7f1c340ade20 legacy=0x7f1c340b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:31.284 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.273+0000 7f1c31ffb640 1 --2- 172.21.15.88:0/3420650201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c1c003b40 0x7f1c1c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:31.284 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.273+0000 7f1c31ffb640 1 --2- 172.21.15.88:0/3420650201 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c1c0047d0 0x7f1c1c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:31.284 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.273+0000 7f1c31ffb640 1 --2- 172.21.15.88:0/3420650201 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1c340b0c10 0x7f1c1c0051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:31.284 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.273+0000 7f1c31ffb640 1 -- 172.21.15.88:0/3420650201 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c340b1fb0 con 0x7f1c1c0047d0 2023-10-16T20:07:31.285 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.273+0000 7f1c31ffb640 1 -- 172.21.15.88:0/3420650201 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c00bad0 con 0x7f1c1c003b40 2023-10-16T20:07:31.285 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.273+0000 7f1c31ffb640 1 -- 172.21.15.88:0/3420650201 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c00bc90 con 0x7f1c340b0c10 2023-10-16T20:07:31.285 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.273+0000 7f1c327fc640 1 --2- 172.21.15.88:0/3420650201 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c1c0047d0 0x7f1c1c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:31.285 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.277+0000 7f1c32ffd640 1 --2- 172.21.15.88:0/3420650201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c1c003b40 0x7f1c1c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:31.285 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.277+0000 7f1c337fe640 1 --2- 172.21.15.88:0/3420650201 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1c340b0c10 0x7f1c1c0051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:31.286 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.277+0000 7f1c327fc640 1 -- 172.21.15.88:0/3420650201 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1c340b0c10 msgr2=0x7f1c1c0051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:31.286 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.277+0000 7f1c327fc640 1 --2- 172.21.15.88:0/3420650201 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1c340b0c10 0x7f1c1c0051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:31.286 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.277+0000 7f1c327fc640 1 -- 172.21.15.88:0/3420650201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c1c003b40 msgr2=0x7f1c1c003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:31.286 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.277+0000 7f1c327fc640 1 --2- 172.21.15.88:0/3420650201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c1c003b40 0x7f1c1c003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:31.286 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.277+0000 7f1c327fc640 1 -- 172.21.15.88:0/3420650201 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c1c003530 con 0x7f1c1c0047d0 2023-10-16T20:07:31.290 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.277+0000 7f1c327fc640 1 --2- 172.21.15.88:0/3420650201 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c1c0047d0 0x7f1c1c004c50 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1c2800c160 tx=0x7f1c2801cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804ff96583bf5700 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:31.290 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.277+0000 7f1c31ffb640 1 -- 172.21.15.88:0/3420650201 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c280058e0 con 0x7f1c1c0047d0 2023-10-16T20:07:31.290 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.277+0000 7f1c31ffb640 1 -- 172.21.15.88:0/3420650201 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c28005a40 con 0x7f1c1c0047d0 2023-10-16T20:07:31.290 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.277+0000 7f1c31ffb640 1 -- 172.21.15.88:0/3420650201 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c28005dd0 con 0x7f1c1c0047d0 2023-10-16T20:07:31.291 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.277+0000 7f1c3946b640 1 -- 172.21.15.88:0/3420650201 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c1c0047d0 msgr2=0x7f1c1c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:31.291 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.277+0000 7f1c3946b640 1 --2- 172.21.15.88:0/3420650201 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c1c0047d0 0x7f1c1c004c50 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1c2800c160 tx=0x7f1c2801cb70 comp rx=0 tx=0).stop 2023-10-16T20:07:31.291 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.277+0000 7f1c337fe640 1 -- 172.21.15.88:0/3420650201 reap_dead start 2023-10-16T20:07:31.291 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.281+0000 7f1c3946b640 1 -- 172.21.15.88:0/3420650201 shutdown_connections 2023-10-16T20:07:31.291 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.281+0000 7f1c3946b640 1 -- 172.21.15.88:0/3420650201 >> 172.21.15.88:0/3420650201 conn(0x7f1c340ce5e0 msgr2=0x7f1c340ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:31.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.281+0000 7f1c3946b640 1 -- 172.21.15.88:0/3420650201 shutdown_connections 2023-10-16T20:07:31.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.281+0000 7f1c3946b640 1 -- 172.21.15.88:0/3420650201 wait complete. 2023-10-16T20:07:31.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.281+0000 7f1c3946b640 1 Processor -- start 2023-10-16T20:07:31.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.281+0000 7f1c3946b640 1 -- start start 2023-10-16T20:07:31.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.281+0000 7f1c3946b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c1c0047d0 0x7f1c3415ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:31.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.281+0000 7f1c3946b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1c340afd10 0x7f1c3413b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:31.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.281+0000 7f1c3946b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c3413bd90 0x7f1c34162720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:31.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.281+0000 7f1c3946b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c340aec70 con 0x7f1c3413bd90 2023-10-16T20:07:31.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.281+0000 7f1c3946b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c340aef50 con 0x7f1c1c0047d0 2023-10-16T20:07:31.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.281+0000 7f1c3946b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c340af230 con 0x7f1c340afd10 2023-10-16T20:07:31.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.281+0000 7f1c337fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c3413bd90 0x7f1c34162720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:31.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.281+0000 7f1c32ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c1c0047d0 0x7f1c3415ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:31.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.281+0000 7f1c337fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c3413bd90 0x7f1c34162720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38672/0 (socket says 172.21.15.88:38672) 2023-10-16T20:07:31.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.281+0000 7f1c337fe640 1 -- 172.21.15.88:0/1571742264 learned_addr learned my addr 172.21.15.88:0/1571742264 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:31.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.281+0000 7f1c32ffd640 1 --2- 172.21.15.88:0/1571742264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c1c0047d0 0x7f1c3415ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:58368/0 (socket says 172.21.15.88:58368) 2023-10-16T20:07:31.295 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.281+0000 7f1c327fc640 1 --2- 172.21.15.88:0/1571742264 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1c340afd10 0x7f1c3413b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:31.295 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.281+0000 7f1c337fe640 1 -- 172.21.15.88:0/1571742264 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1c340afd10 msgr2=0x7f1c3413b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:31.295 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.281+0000 7f1c337fe640 1 --2- 172.21.15.88:0/1571742264 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1c340afd10 0x7f1c3413b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:31.295 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.281+0000 7f1c337fe640 1 -- 172.21.15.88:0/1571742264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c1c0047d0 msgr2=0x7f1c3415ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:31.295 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.281+0000 7f1c337fe640 1 --2- 172.21.15.88:0/1571742264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c1c0047d0 0x7f1c3415ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:31.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.281+0000 7f1c337fe640 1 -- 172.21.15.88:0/1571742264 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c340af510 con 0x7f1c3413bd90 2023-10-16T20:07:31.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.281+0000 7f1c32ffd640 1 --2- 172.21.15.88:0/1571742264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c1c0047d0 0x7f1c3415ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:07:31.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.281+0000 7f1c337fe640 1 --2- 172.21.15.88:0/1571742264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c3413bd90 0x7f1c34162720 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c004750 tx=0x7f1c2c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:31.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.285+0000 7f1c1b7fe640 1 -- 172.21.15.88:0/1571742264 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c00e040 con 0x7f1c3413bd90 2023-10-16T20:07:31.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.285+0000 7f1c1b7fe640 1 -- 172.21.15.88:0/1571742264 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c2c00f040 con 0x7f1c3413bd90 2023-10-16T20:07:31.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.285+0000 7f1c1b7fe640 1 -- 172.21.15.88:0/1571742264 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c015670 con 0x7f1c3413bd90 2023-10-16T20:07:31.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.285+0000 7f1c3946b640 1 -- 172.21.15.88:0/1571742264 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c340af7f0 con 0x7f1c3413bd90 2023-10-16T20:07:31.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.285+0000 7f1c3946b640 1 -- 172.21.15.88:0/1571742264 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c340b0190 con 0x7f1c3413bd90 2023-10-16T20:07:31.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.285+0000 7f1c3946b640 1 -- 172.21.15.88:0/1571742264 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c34142010 con 0x7f1c3413bd90 2023-10-16T20:07:31.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.285+0000 7f1c1b7fe640 1 -- 172.21.15.88:0/1571742264 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1c2c0157d0 con 0x7f1c3413bd90 2023-10-16T20:07:31.298 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.285+0000 7f1c1b7fe640 1 --2- 172.21.15.88:0/1571742264 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1c200664d0 0x7f1c20068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:31.298 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.285+0000 7f1c1b7fe640 1 -- 172.21.15.88:0/1571742264 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f1c2c084600 con 0x7f1c3413bd90 2023-10-16T20:07:31.299 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.285+0000 7f1c32ffd640 1 --2- 172.21.15.88:0/1571742264 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1c200664d0 0x7f1c20068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:31.299 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.285+0000 7f1c32ffd640 1 --2- 172.21.15.88:0/1571742264 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1c200664d0 0x7f1c20068990 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1c340ada10 tx=0x7f1c24009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:31.299 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.289+0000 7f1c1b7fe640 1 -- 172.21.15.88:0/1571742264 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1c2c051880 con 0x7f1c3413bd90 2023-10-16T20:07:31.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.473+0000 7f1c3946b640 1 -- 172.21.15.88:0/1571742264 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c3404e9c0 con 0x7f1c3413bd90 2023-10-16T20:07:31.480 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.477+0000 7f1c1b7fe640 1 -- 172.21.15.88:0/1571742264 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f1c2c054ea0 con 0x7f1c3413bd90 2023-10-16T20:07:31.480 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:07:31.481 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5531,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/2485638574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":2485638574},{"type":"v1","addr":"172.21.15.88:6837","nonce":2485638574}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5534,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3821390040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3821390040},{"type":"v1","addr":"172.21.15.94:6834","nonce":3821390040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5968,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2544673478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":2544673478},{"type":"v1","addr":"172.21.15.94:6835","nonce":2544673478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T20:07:25.650705+0000","modified":"2023-10-16T20:07:28.651868+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6210":{"gid":6210,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6836/1018125900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":1018125900},{"type":"v1","addr":"172.21.15.88:6836","nonce":1018125900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"testcephfs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-16T20:07:31.481 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 28 2023-10-16T20:07:31.482 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.477+0000 7f1c3946b640 1 -- 172.21.15.88:0/1571742264 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1c200664d0 msgr2=0x7f1c20068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:31.482 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.477+0000 7f1c3946b640 1 --2- 172.21.15.88:0/1571742264 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1c200664d0 0x7f1c20068990 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1c340ada10 tx=0x7f1c24009040 comp rx=0 tx=0).stop 2023-10-16T20:07:31.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.477+0000 7f1c3946b640 1 -- 172.21.15.88:0/1571742264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c3413bd90 msgr2=0x7f1c34162720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:31.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.477+0000 7f1c3946b640 1 --2- 172.21.15.88:0/1571742264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c3413bd90 0x7f1c34162720 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c004750 tx=0x7f1c2c00f580 comp rx=0 tx=0).stop 2023-10-16T20:07:31.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.477+0000 7f1c3946b640 1 -- 172.21.15.88:0/1571742264 shutdown_connections 2023-10-16T20:07:31.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.477+0000 7f1c3946b640 1 --2- 172.21.15.88:0/1571742264 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f1c200664d0 0x7f1c20068990 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:31.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.477+0000 7f1c3946b640 1 --2- 172.21.15.88:0/1571742264 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1c340afd10 0x7f1c3413b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:31.484 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.477+0000 7f1c3946b640 1 --2- 172.21.15.88:0/1571742264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c3413bd90 0x7f1c34162720 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:31.484 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.477+0000 7f1c3946b640 1 --2- 172.21.15.88:0/1571742264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c1c0047d0 0x7f1c3415ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:31.484 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.477+0000 7f1c3946b640 1 -- 172.21.15.88:0/1571742264 >> 172.21.15.88:0/1571742264 conn(0x7f1c340ce5e0 msgr2=0x7f1c340b0500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:31.484 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.477+0000 7f1c3946b640 1 -- 172.21.15.88:0/1571742264 shutdown_connections 2023-10-16T20:07:31.485 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:31.477+0000 7f1c3946b640 1 -- 172.21.15.88:0/1571742264 wait complete. 2023-10-16T20:07:31.904 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T20:07:25.650705+0000', 'modified': '2023-10-16T20:07:28.651868+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6210}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6210': {'gid': 6210, 'name': 'a', 'rank': 0, 'incarnation': 24, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.88:6836/1018125900', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6834', 'nonce': 1018125900}, {'type': 'v1', 'addr': '172.21.15.88:6836', 'nonce': 1018125900}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'testcephfs1', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T20:07:31.904 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T20:07:31.905 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-16T20:07:32.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.041+0000 7f61bd69c640 1 Processor -- start 2023-10-16T20:07:32.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.041+0000 7f61bd69c640 1 -- start start 2023-10-16T20:07:32.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.041+0000 7f61bd69c640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f61b815ca00 0x7f61b8161330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:32.048 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.041+0000 7f61bd69c640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f61b815cde0 0x7f61b815dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:32.048 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.041+0000 7f61bd69c640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61b815e210 con 0x7f61b80cf390 2023-10-16T20:07:32.048 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.041+0000 7f61bd69c640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61b81608c0 con 0x7f61b815ca00 2023-10-16T20:07:32.048 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.041+0000 7f61bd69c640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61b80cdfa0 con 0x7f61b815cde0 2023-10-16T20:07:32.048 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.045+0000 7f61b6ffd640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f61b815ca00 0x7f61b8161330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:32.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.045+0000 7f61b77fe640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f61b80cf390 0x7f61b80cf7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:37330/0 (socket says 172.21.15.88:37330) 2023-10-16T20:07:32.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.045+0000 7f61b77fe640 1 -- 172.21.15.88:0/303772403 learned_addr learned my addr 172.21.15.88:0/303772403 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:32.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.045+0000 7f61b67fc640 1 --2- 172.21.15.88:0/303772403 >> v2:172.21.15.94:3300/0 conn(0x7f61b815cde0 0x7f61b815dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:32.050 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.045+0000 7f61b6ffd640 1 -- 172.21.15.88:0/303772403 >> v2:172.21.15.94:3300/0 conn(0x7f61b815cde0 msgr2=0x7f61b815dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:32.050 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.045+0000 7f61b6ffd640 1 --2- 172.21.15.88:0/303772403 >> v2:172.21.15.94:3300/0 conn(0x7f61b815cde0 0x7f61b815dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.050 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.045+0000 7f61b6ffd640 1 -- 172.21.15.88:0/303772403 >> v1:172.21.15.88:6789/0 conn(0x7f61b80cf390 legacy=0x7f61b80cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:32.050 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.045+0000 7f61b6ffd640 1 -- 172.21.15.88:0/303772403 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b8160b30 con 0x7f61b815ca00 2023-10-16T20:07:32.051 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.045+0000 7f61b5ffb640 1 -- 172.21.15.88:0/303772403 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 314604191 0 0) 0x7f61b815e210 con 0x7f61b80cf390 2023-10-16T20:07:32.052 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.045+0000 7f61b6ffd640 1 --2- 172.21.15.88:0/303772403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61b815ca00 0x7f61b8161330 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f61b80d0660 tx=0x7f61a801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=689ed6a405c1aaf7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:32.052 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.045+0000 7f61b5ffb640 1 -- 172.21.15.88:0/303772403 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a80ac620 con 0x7f61b815ca00 2023-10-16T20:07:32.052 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.045+0000 7f61b5ffb640 1 -- 172.21.15.88:0/303772403 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a80ac780 con 0x7f61b815ca00 2023-10-16T20:07:32.053 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.045+0000 7f61b5ffb640 1 -- 172.21.15.88:0/303772403 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a80acb10 con 0x7f61b815ca00 2023-10-16T20:07:32.053 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.049+0000 7f61bd69c640 1 -- 172.21.15.88:0/303772403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61b815ca00 msgr2=0x7f61b8161330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:32.053 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.049+0000 7f61bd69c640 1 --2- 172.21.15.88:0/303772403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61b815ca00 0x7f61b8161330 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f61b80d0660 tx=0x7f61a801c3a0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.053 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.049+0000 7f61bd69c640 1 -- 172.21.15.88:0/303772403 shutdown_connections 2023-10-16T20:07:32.053 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.049+0000 7f61bd69c640 1 --2- 172.21.15.88:0/303772403 >> v2:172.21.15.94:3300/0 conn(0x7f61b815cde0 0x7f61b815dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.054 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.049+0000 7f61bd69c640 1 --2- 172.21.15.88:0/303772403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61b815ca00 0x7f61b8161330 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.054 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.049+0000 7f61bd69c640 1 -- 172.21.15.88:0/303772403 >> 172.21.15.88:0/303772403 conn(0x7f61b80715c0 msgr2=0x7f61b80719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:32.054 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.049+0000 7f61bd69c640 1 -- 172.21.15.88:0/303772403 shutdown_connections 2023-10-16T20:07:32.054 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.049+0000 7f61bd69c640 1 -- 172.21.15.88:0/303772403 wait complete. 2023-10-16T20:07:32.056 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.049+0000 7f61bd69c640 1 Processor -- start 2023-10-16T20:07:32.057 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.049+0000 7f61bd69c640 1 -- start start 2023-10-16T20:07:32.057 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.049+0000 7f61bd69c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61b80cf390 0x7f61b814cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:32.057 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.049+0000 7f61bd69c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61b815ca00 0x7f61b814d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:32.057 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.049+0000 7f61bd69c640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f61b815cde0 0x7f61b814d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:32.057 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.049+0000 7f61bd69c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b815f260 con 0x7f61b815ca00 2023-10-16T20:07:32.058 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.049+0000 7f61bd69c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b81600c0 con 0x7f61b80cf390 2023-10-16T20:07:32.058 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.049+0000 7f61bd69c640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b815f4b0 con 0x7f61b815cde0 2023-10-16T20:07:32.058 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.049+0000 7f61b67fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61b815ca00 0x7f61b814d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:32.058 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.049+0000 7f61b67fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61b815ca00 0x7f61b814d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47500/0 (socket says 172.21.15.88:47500) 2023-10-16T20:07:32.058 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.049+0000 7f61b67fc640 1 -- 172.21.15.88:0/1789754794 learned_addr learned my addr 172.21.15.88:0/1789754794 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:32.059 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.049+0000 7f61b6ffd640 1 --2- 172.21.15.88:0/1789754794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61b80cf390 0x7f61b814cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:32.059 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.049+0000 7f61b67fc640 1 -- 172.21.15.88:0/1789754794 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f61b815cde0 msgr2=0x7f61b814d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:32.059 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.049+0000 7f61b67fc640 1 --2- 172.21.15.88:0/1789754794 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f61b815cde0 0x7f61b814d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.059 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.049+0000 7f61b67fc640 1 -- 172.21.15.88:0/1789754794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61b80cf390 msgr2=0x7f61b814cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:32.059 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.049+0000 7f61b67fc640 1 --2- 172.21.15.88:0/1789754794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61b80cf390 0x7f61b814cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.060 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.049+0000 7f61b67fc640 1 -- 172.21.15.88:0/1789754794 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a8009420 con 0x7f61b815ca00 2023-10-16T20:07:32.060 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.049+0000 7f61b6ffd640 1 --2- 172.21.15.88:0/1789754794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61b80cf390 0x7f61b814cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:07:32.060 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.053+0000 7f61b67fc640 1 --2- 172.21.15.88:0/1789754794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61b815ca00 0x7f61b814d160 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f61ac00e960 tx=0x7f61ac00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:32.060 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.053+0000 7f619f7fe640 1 -- 172.21.15.88:0/1789754794 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac009690 con 0x7f61b815ca00 2023-10-16T20:07:32.060 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.053+0000 7f619f7fe640 1 -- 172.21.15.88:0/1789754794 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ac00ca00 con 0x7f61b815ca00 2023-10-16T20:07:32.060 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.053+0000 7f61bd69c640 1 -- 172.21.15.88:0/1789754794 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61b815f790 con 0x7f61b815ca00 2023-10-16T20:07:32.061 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.053+0000 7f619f7fe640 1 -- 172.21.15.88:0/1789754794 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac00cd70 con 0x7f61b815ca00 2023-10-16T20:07:32.061 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.053+0000 7f61bd69c640 1 -- 172.21.15.88:0/1789754794 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61b815fd50 con 0x7f61b815ca00 2023-10-16T20:07:32.064 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.053+0000 7f619f7fe640 1 -- 172.21.15.88:0/1789754794 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f61ac013470 con 0x7f61b815ca00 2023-10-16T20:07:32.064 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.053+0000 7f619f7fe640 1 --2- 172.21.15.88:0/1789754794 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f61a40664a0 0x7f61a4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:32.064 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.053+0000 7f619f7fe640 1 -- 172.21.15.88:0/1789754794 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f61ac084210 con 0x7f61b815ca00 2023-10-16T20:07:32.064 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.053+0000 7f61b6ffd640 1 --2- 172.21.15.88:0/1789754794 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f61a40664a0 0x7f61a4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:32.065 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.053+0000 7f61c0a34000 1 Processor -- start 2023-10-16T20:07:32.065 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.057+0000 7f61c0a34000 1 -- start start 2023-10-16T20:07:32.065 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.057+0000 7f61b6ffd640 1 --2- 172.21.15.88:0/1789754794 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f61a40664a0 0x7f61a4068960 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f61a80ae000 tx=0x7f61a8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:32.065 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.057+0000 7f61c0a34000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x562b782438a0 0x562b78243c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:32.066 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.057+0000 7f61c0a34000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b782441c0 0x562b781f9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:32.066 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.057+0000 7f61c0a34000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b781f9c50 0x562b781fa030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:32.066 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.057+0000 7f61c0a34000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b77feecc0 con 0x562b782441c0 2023-10-16T20:07:32.066 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.057+0000 7f61c0a34000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b78051920 con 0x562b781f9c50 2023-10-16T20:07:32.066 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.057+0000 7f61c0a34000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b7804ee50 con 0x562b782438a0 2023-10-16T20:07:32.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.057+0000 7f61b67fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b782441c0 0x562b781f9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:32.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.057+0000 7f61b67fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b782441c0 0x562b781f9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47502/0 (socket says 172.21.15.88:47502) 2023-10-16T20:07:32.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.057+0000 7f61b67fc640 1 -- 172.21.15.88:0/2757689219 learned_addr learned my addr 172.21.15.88:0/2757689219 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:32.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.057+0000 7f61b67fc640 1 -- 172.21.15.88:0/2757689219 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x562b782438a0 msgr2=0x562b78243c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:32.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.057+0000 7f61b67fc640 1 --2- 172.21.15.88:0/2757689219 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x562b782438a0 0x562b78243c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.068 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.057+0000 7f61b67fc640 1 -- 172.21.15.88:0/2757689219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b781f9c50 msgr2=0x562b781fa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:32.068 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.057+0000 7f61b67fc640 1 --2- 172.21.15.88:0/2757689219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b781f9c50 0x562b781fa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.068 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.057+0000 7f61b67fc640 1 -- 172.21.15.88:0/2757689219 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b77d43890 con 0x562b782441c0 2023-10-16T20:07:32.068 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.057+0000 7f61b67fc640 1 --2- 172.21.15.88:0/2757689219 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b782441c0 0x562b781f9710 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f61ac05cfb0 tx=0x7f61ac0559a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6806b0517e2a5f02 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:32.068 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.057+0000 7f619cff9640 1 -- 172.21.15.88:0/2757689219 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac05d4e0 con 0x562b782441c0 2023-10-16T20:07:32.069 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.057+0000 7f619cff9640 1 -- 172.21.15.88:0/2757689219 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ac053580 con 0x562b782441c0 2023-10-16T20:07:32.069 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.057+0000 7f619cff9640 1 -- 172.21.15.88:0/2757689219 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac0518f0 con 0x562b782441c0 2023-10-16T20:07:32.069 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.057+0000 7f61c0a34000 1 -- 172.21.15.88:0/2757689219 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b782441c0 msgr2=0x562b781f9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:32.069 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.057+0000 7f61c0a34000 1 --2- 172.21.15.88:0/2757689219 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b782441c0 0x562b781f9710 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f61ac05cfb0 tx=0x7f61ac0559a0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.069 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.057+0000 7f61c0a34000 1 -- 172.21.15.88:0/2757689219 shutdown_connections 2023-10-16T20:07:32.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.057+0000 7f61c0a34000 1 --2- 172.21.15.88:0/2757689219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b781f9c50 0x562b781fa030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.057+0000 7f61c0a34000 1 --2- 172.21.15.88:0/2757689219 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b782441c0 0x562b781f9710 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.057+0000 7f61c0a34000 1 --2- 172.21.15.88:0/2757689219 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x562b782438a0 0x562b78243c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.057+0000 7f61c0a34000 1 -- 172.21.15.88:0/2757689219 >> 172.21.15.88:0/2757689219 conn(0x562b781f0fc0 msgr2=0x562b781f7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:32.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.057+0000 7f61c0a34000 1 -- 172.21.15.88:0/2757689219 shutdown_connections 2023-10-16T20:07:32.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.057+0000 7f61c0a34000 1 -- 172.21.15.88:0/2757689219 wait complete. 2023-10-16T20:07:32.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.057+0000 7f61c0a34000 1 -- start start 2023-10-16T20:07:32.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.057+0000 7f61c0a34000 1 Processor -- start 2023-10-16T20:07:32.074 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f61c0a34000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b781f9c50 0x562b781fefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:32.075 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f61c0a34000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x562b782438a0 0x562b781ff500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:32.075 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f61c0a34000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b782441c0 0x562b78231d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:32.075 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f61c0a34000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b7804a740 con 0x562b782441c0 2023-10-16T20:07:32.075 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f61c0a34000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b78009d10 con 0x562b781f9c50 2023-10-16T20:07:32.076 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f61c0a34000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b780099b0 con 0x562b782438a0 2023-10-16T20:07:32.076 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f61c0a34000 10 client.0 _ll_register_callbacks cb 0x562b781e5390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T20:07:32.076 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f61b77fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b782441c0 0x562b78231d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:32.076 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f61b77fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b782441c0 0x562b78231d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47516/0 (socket says 172.21.15.88:47516) 2023-10-16T20:07:32.076 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f61b77fe640 1 -- 172.21.15.88:0/1174732175 learned_addr learned my addr 172.21.15.88:0/1174732175 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:32.077 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f61b67fc640 1 --2- 172.21.15.88:0/1174732175 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x562b782438a0 0x562b781ff500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:32.077 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f61b6ffd640 1 --2- 172.21.15.88:0/1174732175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b781f9c50 0x562b781fefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:32.077 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f61b77fe640 1 -- 172.21.15.88:0/1174732175 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x562b782438a0 msgr2=0x562b781ff500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:32.077 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f61b77fe640 1 --2- 172.21.15.88:0/1174732175 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x562b782438a0 0x562b781ff500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.077 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f61b77fe640 1 -- 172.21.15.88:0/1174732175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b781f9c50 msgr2=0x562b781fefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:32.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f61b77fe640 1 --2- 172.21.15.88:0/1174732175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b781f9c50 0x562b781fefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f61b77fe640 1 -- 172.21.15.88:0/1174732175 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61b815fd50 con 0x562b782441c0 2023-10-16T20:07:32.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f61b6ffd640 1 --2- 172.21.15.88:0/1174732175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b781f9c50 0x562b781fefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:07:32.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f61b67fc640 1 --2- 172.21.15.88:0/1174732175 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x562b782438a0 0x562b781ff500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:07:32.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f61b77fe640 1 --2- 172.21.15.88:0/1174732175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b782441c0 0x562b78231d80 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f61b000c3e0 tx=0x7f61b000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35372d831e3deac0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:32.079 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f6195ffb640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2023-10-16T20:07:32.079 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f6195ffb640 1 -- 172.21.15.88:0/1174732175 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b00070d0 con 0x562b782441c0 2023-10-16T20:07:32.079 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f61c0a34000 1 -- 172.21.15.88:0/1174732175 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b77ffe370 con 0x562b782441c0 2023-10-16T20:07:32.079 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f6195ffb640 1 -- 172.21.15.88:0/1174732175 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f61b0015c30 con 0x562b782441c0 2023-10-16T20:07:32.079 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f6195ffb640 1 -- 172.21.15.88:0/1174732175 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61b0015e70 con 0x562b782441c0 2023-10-16T20:07:32.080 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f61c0a34000 10 client.6342 fetch_fsmap learned FSMap version 28 2023-10-16T20:07:32.080 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f61c0a34000 1 -- 172.21.15.88:0/1174732175 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x562b7800c460 con 0x562b782441c0 2023-10-16T20:07:32.080 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f6195ffb640 1 -- 172.21.15.88:0/1174732175 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562b77ffe370 con 0x562b782441c0 2023-10-16T20:07:32.080 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f6195ffb640 1 -- 172.21.15.88:0/1174732175 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 28) v1 ==== 3299+0+0 (secure 0 0 0) 0x7f61b0007230 con 0x562b782441c0 2023-10-16T20:07:32.080 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f61c0a34000 10 client.6342 fetch_fsmap finished waiting for FSMap version 28 2023-10-16T20:07:32.081 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f61c0a34000 10 client.6342 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-16T20:07:32.081 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f61c0a34000 20 client.6342 populate_metadata read hostname 'smithi088' 2023-10-16T20:07:32.081 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f61c0a34000 1 --2- 172.21.15.88:0/1174732175 >> [v2:172.21.15.88:6834/1018125900,v1:172.21.15.88:6836/1018125900] conn(0x562b78253940 0x562b78255d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:32.081 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f61c0a34000 4 client.6342 mds_command: new command op to 6210 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T20:07:32.081 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f61c0a34000 1 -- 172.21.15.88:0/1174732175 --> [v2:172.21.15.88:6834/1018125900,v1:172.21.15.88:6836/1018125900] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b77fe42d0 con 0x562b78253940 2023-10-16T20:07:32.082 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f61b6ffd640 1 --2- 172.21.15.88:0/1174732175 >> [v2:172.21.15.88:6834/1018125900,v1:172.21.15.88:6836/1018125900] conn(0x562b78253940 0x562b78255d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:32.082 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f61b6ffd640 1 --2- 172.21.15.88:0/1174732175 >> [v2:172.21.15.88:6834/1018125900,v1:172.21.15.88:6836/1018125900] conn(0x562b78253940 0x562b78255d30 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96ee9278b5eca857 server_cookie=2180f3d12c0b1314 in_seq=0 out_seq=0 2023-10-16T20:07:32.082 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.061+0000 7f6195ffb640 10 client.6342 ms_handle_connect on v2:172.21.15.88:6834/1018125900 2023-10-16T20:07:32.082 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.065+0000 7f6195ffb640 1 -- 172.21.15.88:0/1174732175 <== mds.0 v2:172.21.15.88:6834/1018125900 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562b77fe42d0 con 0x562b78253940 2023-10-16T20:07:32.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.065+0000 7f6195ffb640 10 client.6342 handle_command_reply: tid=0 2023-10-16T20:07:32.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.065+0000 7f61c0a34000 1 client.6342 shutdown 2023-10-16T20:07:32.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.065+0000 7f61c0a34000 1 -- 172.21.15.88:0/1174732175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b782441c0 msgr2=0x562b78231d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:32.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.065+0000 7f61c0a34000 1 --2- 172.21.15.88:0/1174732175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b782441c0 0x562b78231d80 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f61b000c3e0 tx=0x7f61b000c8a0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.065+0000 7f61c0a34000 1 -- 172.21.15.88:0/1174732175 shutdown_connections 2023-10-16T20:07:32.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.065+0000 7f61c0a34000 1 --2- 172.21.15.88:0/1174732175 >> [v2:172.21.15.88:6834/1018125900,v1:172.21.15.88:6836/1018125900] conn(0x562b78253940 0x562b78255d30 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.065+0000 7f61c0a34000 1 --2- 172.21.15.88:0/1174732175 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x562b782438a0 0x562b781ff500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.065+0000 7f61c0a34000 1 --2- 172.21.15.88:0/1174732175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b782441c0 0x562b78231d80 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.065+0000 7f6195ffb640 0 client.6342 ms_handle_reset on v2:172.21.15.88:6834/1018125900 2023-10-16T20:07:32.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.065+0000 7f61c0a34000 1 --2- 172.21.15.88:0/1174732175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b781f9c50 0x562b781fefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.065+0000 7f61c0a34000 1 -- 172.21.15.88:0/1174732175 >> 172.21.15.88:0/1174732175 conn(0x562b781f0fc0 msgr2=0x562b781f86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:32.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.065+0000 7f61c0a34000 1 -- 172.21.15.88:0/1174732175 shutdown_connections 2023-10-16T20:07:32.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.065+0000 7f61c0a34000 1 -- 172.21.15.88:0/1174732175 wait complete. 2023-10-16T20:07:32.086 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.065+0000 7f61c0a34000 20 client.6342 trim_cache size 0 max 16384 2023-10-16T20:07:32.091 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.077+0000 7f61c0a34000 1 Processor -- start 2023-10-16T20:07:32.092 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.077+0000 7f61c0a34000 1 -- start start 2023-10-16T20:07:32.092 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.081+0000 7f61c0a34000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b781f0fc0 0x562b7829ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:32.092 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.081+0000 7f61c0a34000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b781f9c50 0x562b7829f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:32.092 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.081+0000 7f61c0a34000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x562b782438a0 0x562b7829f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:32.093 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.081+0000 7f61c0a34000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b77d04940 con 0x562b781f0fc0 2023-10-16T20:07:32.093 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.081+0000 7f61c0a34000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b780094d0 con 0x562b781f9c50 2023-10-16T20:07:32.093 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.081+0000 7f61c0a34000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b782319f0 con 0x562b782438a0 2023-10-16T20:07:32.093 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.081+0000 7f61b6ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b781f0fc0 0x562b7829ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:32.093 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.081+0000 7f61b6ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b781f0fc0 0x562b7829ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47526/0 (socket says 172.21.15.88:47526) 2023-10-16T20:07:32.094 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.081+0000 7f61b6ffd640 1 -- 172.21.15.88:0/2158806576 learned_addr learned my addr 172.21.15.88:0/2158806576 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:32.094 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.081+0000 7f61b67fc640 1 --2- 172.21.15.88:0/2158806576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b781f9c50 0x562b7829f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:32.094 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.081+0000 7f61b77fe640 1 --2- 172.21.15.88:0/2158806576 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x562b782438a0 0x562b7829f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:32.094 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.081+0000 7f61b6ffd640 1 -- 172.21.15.88:0/2158806576 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x562b782438a0 msgr2=0x562b7829f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:32.094 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.081+0000 7f61b6ffd640 1 --2- 172.21.15.88:0/2158806576 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x562b782438a0 0x562b7829f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.081+0000 7f61b6ffd640 1 -- 172.21.15.88:0/2158806576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b781f9c50 msgr2=0x562b7829f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:32.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.081+0000 7f61b6ffd640 1 --2- 172.21.15.88:0/2158806576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b781f9c50 0x562b7829f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.081+0000 7f61b6ffd640 1 -- 172.21.15.88:0/2158806576 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b77ff2760 con 0x562b781f0fc0 2023-10-16T20:07:32.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.081+0000 7f61b6ffd640 1 --2- 172.21.15.88:0/2158806576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b781f0fc0 0x562b7829ed20 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f61a80af040 tx=0x7f61a8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6461345e41e6ec61 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:32.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.081+0000 7f619d7fa640 1 -- 172.21.15.88:0/2158806576 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a80bd040 con 0x562b781f0fc0 2023-10-16T20:07:32.096 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.081+0000 7f619d7fa640 1 -- 172.21.15.88:0/2158806576 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a80c5c40 con 0x562b781f0fc0 2023-10-16T20:07:32.096 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.081+0000 7f619d7fa640 1 -- 172.21.15.88:0/2158806576 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a80c5050 con 0x562b781f0fc0 2023-10-16T20:07:32.096 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.081+0000 7f61c0a34000 1 -- 172.21.15.88:0/2158806576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b781f0fc0 msgr2=0x562b7829ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:32.096 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.081+0000 7f61c0a34000 1 --2- 172.21.15.88:0/2158806576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b781f0fc0 0x562b7829ed20 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f61a80af040 tx=0x7f61a8004740 comp rx=0 tx=0).stop 2023-10-16T20:07:32.096 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.081+0000 7f61c0a34000 1 -- 172.21.15.88:0/2158806576 shutdown_connections 2023-10-16T20:07:32.097 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.081+0000 7f61c0a34000 1 --2- 172.21.15.88:0/2158806576 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x562b782438a0 0x562b7829f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.097 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.081+0000 7f61c0a34000 1 --2- 172.21.15.88:0/2158806576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b781f9c50 0x562b7829f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.097 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.081+0000 7f61c0a34000 1 --2- 172.21.15.88:0/2158806576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b781f0fc0 0x562b7829ed20 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.097 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.081+0000 7f61c0a34000 1 -- 172.21.15.88:0/2158806576 >> 172.21.15.88:0/2158806576 conn(0x562b78231b30 msgr2=0x562b782565c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:32.097 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.081+0000 7f61c0a34000 1 -- 172.21.15.88:0/2158806576 shutdown_connections 2023-10-16T20:07:32.097 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.081+0000 7f61c0a34000 1 -- 172.21.15.88:0/2158806576 wait complete. 2023-10-16T20:07:32.098 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.081+0000 7f61c0a34000 1 -- start start 2023-10-16T20:07:32.098 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.081+0000 7f61c0a34000 1 Processor -- start 2023-10-16T20:07:32.098 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.081+0000 7f61c0a34000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b781f0fc0 0x562b78230560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:32.098 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61c0a34000 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b781f9c50 0x562b7825f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:32.099 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61c0a34000 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x562b782438a0 0x562b7825f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:32.099 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61c0a34000 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b781fd4f0 con 0x562b781f9c50 2023-10-16T20:07:32.099 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61c0a34000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b78044a40 con 0x562b781f0fc0 2023-10-16T20:07:32.099 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61c0a34000 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b78047f50 con 0x562b782438a0 2023-10-16T20:07:32.099 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61c0a34000 10 client.0 _ll_register_callbacks cb 0x7f61ac0515a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T20:07:32.099 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61b67fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b781f9c50 0x562b7825f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:32.100 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61b67fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b781f9c50 0x562b7825f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47538/0 (socket says 172.21.15.88:47538) 2023-10-16T20:07:32.100 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61b67fc640 1 -- 172.21.15.88:0/2418064935 learned_addr learned my addr 172.21.15.88:0/2418064935 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:32.100 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61b6ffd640 1 --2- 172.21.15.88:0/2418064935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b781f0fc0 0x562b78230560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:32.100 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61b77fe640 1 --2- 172.21.15.88:0/2418064935 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x562b782438a0 0x562b7825f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:32.100 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61b67fc640 1 -- 172.21.15.88:0/2418064935 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x562b782438a0 msgr2=0x562b7825f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:32.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61b67fc640 1 --2- 172.21.15.88:0/2418064935 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x562b782438a0 0x562b7825f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61b67fc640 1 -- 172.21.15.88:0/2418064935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b781f0fc0 msgr2=0x562b78230560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:32.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61b67fc640 1 --2- 172.21.15.88:0/2418064935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b781f0fc0 0x562b78230560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61b67fc640 1 -- 172.21.15.88:0/2418064935 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61b8160b30 con 0x562b781f9c50 2023-10-16T20:07:32.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61b67fc640 1 --2- 172.21.15.88:0/2418064935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b781f9c50 0x562b7825f0f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f61ac0516e0 tx=0x7f61ac00efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a020524bd3771dc3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:32.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f6195ffb640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2023-10-16T20:07:32.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f6195ffb640 1 -- 172.21.15.88:0/2418064935 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac0822f0 con 0x562b781f9c50 2023-10-16T20:07:32.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f6195ffb640 1 -- 172.21.15.88:0/2418064935 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f61ac05d4e0 con 0x562b781f9c50 2023-10-16T20:07:32.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61c0a34000 1 -- 172.21.15.88:0/2418064935 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b781fc3b0 con 0x562b781f9c50 2023-10-16T20:07:32.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f6195ffb640 1 -- 172.21.15.88:0/2418064935 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61ac073120 con 0x562b781f9c50 2023-10-16T20:07:32.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61c0a34000 10 client.6348 fetch_fsmap learned FSMap version 28 2023-10-16T20:07:32.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61c0a34000 1 -- 172.21.15.88:0/2418064935 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x562b77d43cc0 con 0x562b781f9c50 2023-10-16T20:07:32.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f6195ffb640 1 -- 172.21.15.88:0/2418064935 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562b781fc3b0 con 0x562b781f9c50 2023-10-16T20:07:32.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f6195ffb640 1 -- 172.21.15.88:0/2418064935 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 28) v1 ==== 3299+0+0 (secure 0 0 0) 0x7f61ac011070 con 0x562b781f9c50 2023-10-16T20:07:32.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61c0a34000 10 client.6348 fetch_fsmap finished waiting for FSMap version 28 2023-10-16T20:07:32.104 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61c0a34000 10 client.6348 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-16T20:07:32.104 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61c0a34000 20 client.6348 populate_metadata read hostname 'smithi088' 2023-10-16T20:07:32.104 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61c0a34000 1 --2- 172.21.15.88:0/2418064935 >> [v2:172.21.15.88:6834/1018125900,v1:172.21.15.88:6836/1018125900] conn(0x562b782441c0 0x562b782aafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:32.104 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61c0a34000 4 client.6348 mds_command: new command op to 6210 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T20:07:32.104 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61c0a34000 1 -- 172.21.15.88:0/2418064935 --> [v2:172.21.15.88:6834/1018125900,v1:172.21.15.88:6836/1018125900] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562b77fe26f0 con 0x562b782441c0 2023-10-16T20:07:32.104 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61b6ffd640 1 --2- 172.21.15.88:0/2418064935 >> [v2:172.21.15.88:6834/1018125900,v1:172.21.15.88:6836/1018125900] conn(0x562b782441c0 0x562b782aafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:32.105 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61b6ffd640 1 --2- 172.21.15.88:0/2418064935 >> [v2:172.21.15.88:6834/1018125900,v1:172.21.15.88:6836/1018125900] conn(0x562b782441c0 0x562b782aafb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=926aad9669ac92bc server_cookie=76d2738e09f8d32c in_seq=0 out_seq=0 2023-10-16T20:07:32.105 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f6195ffb640 10 client.6348 ms_handle_connect on v2:172.21.15.88:6834/1018125900 2023-10-16T20:07:32.105 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f6195ffb640 1 -- 172.21.15.88:0/2418064935 <== mds.0 v2:172.21.15.88:6834/1018125900 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x562b77fe26f0 con 0x562b782441c0 2023-10-16T20:07:32.105 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f6195ffb640 10 client.6348 handle_command_reply: tid=0 2023-10-16T20:07:32.105 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61c0a34000 1 client.6348 shutdown 2023-10-16T20:07:32.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61c0a34000 1 -- 172.21.15.88:0/2418064935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b781f9c50 msgr2=0x562b7825f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:32.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61c0a34000 1 --2- 172.21.15.88:0/2418064935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b781f9c50 0x562b7825f0f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f61ac0516e0 tx=0x7f61ac00efd0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61c0a34000 1 -- 172.21.15.88:0/2418064935 shutdown_connections 2023-10-16T20:07:32.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61c0a34000 1 --2- 172.21.15.88:0/2418064935 >> [v2:172.21.15.88:6834/1018125900,v1:172.21.15.88:6836/1018125900] conn(0x562b782441c0 0x562b782aafb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61c0a34000 1 --2- 172.21.15.88:0/2418064935 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x562b782438a0 0x562b7825f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61c0a34000 1 --2- 172.21.15.88:0/2418064935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b781f9c50 0x562b7825f0f0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f6195ffb640 0 client.6348 ms_handle_reset on v2:172.21.15.88:6834/1018125900 2023-10-16T20:07:32.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61c0a34000 1 --2- 172.21.15.88:0/2418064935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b781f0fc0 0x562b78230560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61c0a34000 1 -- 172.21.15.88:0/2418064935 >> 172.21.15.88:0/2418064935 conn(0x562b78231b30 msgr2=0x562b78256cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:32.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61c0a34000 1 -- 172.21.15.88:0/2418064935 shutdown_connections 2023-10-16T20:07:32.108 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61c0a34000 1 -- 172.21.15.88:0/2418064935 wait complete. 2023-10-16T20:07:32.108 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61c0a34000 20 client.6348 trim_cache size 0 max 16384 2023-10-16T20:07:32.108 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2023-10-16T20:07:32.109 INFO:teuthology.orchestra.run.smithi088.stdout: "cluster_fsid": "d11187c0-3dd8-4c92-8d13-6ec536230748", 2023-10-16T20:07:32.109 INFO:teuthology.orchestra.run.smithi088.stdout: "whoami": 0, 2023-10-16T20:07:32.109 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 6210, 2023-10-16T20:07:32.109 INFO:teuthology.orchestra.run.smithi088.stdout: "want_state": "up:active", 2023-10-16T20:07:32.109 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "up:active", 2023-10-16T20:07:32.110 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "testcephfs1", 2023-10-16T20:07:32.110 INFO:teuthology.orchestra.run.smithi088.stdout: "rank_uptime": 5.8263152299999996, 2023-10-16T20:07:32.110 INFO:teuthology.orchestra.run.smithi088.stdout: "mdsmap_epoch": 28, 2023-10-16T20:07:32.110 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch": 58, 2023-10-16T20:07:32.110 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch_barrier": 58, 2023-10-16T20:07:32.110 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 10.712063555 2023-10-16T20:07:32.111 INFO:teuthology.orchestra.run.smithi088.stdout:} 2023-10-16T20:07:32.112 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61bd69c640 1 -- 172.21.15.88:0/1789754794 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f61a40664a0 msgr2=0x7f61a4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:32.112 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61bd69c640 1 --2- 172.21.15.88:0/1789754794 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f61a40664a0 0x7f61a4068960 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f61a80ae000 tx=0x7f61a8008040 comp rx=0 tx=0).stop 2023-10-16T20:07:32.112 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61bd69c640 1 -- 172.21.15.88:0/1789754794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61b815ca00 msgr2=0x7f61b814d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:32.113 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61bd69c640 1 --2- 172.21.15.88:0/1789754794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61b815ca00 0x7f61b814d160 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f61ac00e960 tx=0x7f61ac00ee20 comp rx=0 tx=0).stop 2023-10-16T20:07:32.113 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61bd69c640 1 -- 172.21.15.88:0/1789754794 shutdown_connections 2023-10-16T20:07:32.113 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61bd69c640 1 --2- 172.21.15.88:0/1789754794 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f61a40664a0 0x7f61a4068960 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.113 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61bd69c640 1 --2- 172.21.15.88:0/1789754794 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f61b815cde0 0x7f61b814d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.113 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61bd69c640 1 --2- 172.21.15.88:0/1789754794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61b815ca00 0x7f61b814d160 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61bd69c640 1 --2- 172.21.15.88:0/1789754794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61b80cf390 0x7f61b814cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61bd69c640 1 -- 172.21.15.88:0/1789754794 >> 172.21.15.88:0/1789754794 conn(0x7f61b80715c0 msgr2=0x7f61b80c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:32.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61bd69c640 1 -- 172.21.15.88:0/1789754794 shutdown_connections 2023-10-16T20:07:32.114 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.085+0000 7f61bd69c640 1 -- 172.21.15.88:0/1789754794 wait complete. 2023-10-16T20:07:32.115 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T20:07:32.242 INFO:tasks.ceph.mds.a.smithi088.stderr:2023-10-16T20:07:32.237+0000 7f239fea7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T20:07:32.252 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.249+0000 7fb69253e640 1 Processor -- start 2023-10-16T20:07:32.253 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.249+0000 7fb69253e640 1 -- start start 2023-10-16T20:07:32.254 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.249+0000 7fb69253e640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb68c0d26c0 0x7fb68c161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:32.254 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.249+0000 7fb69253e640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb68c0b87e0 con 0x7fb68c15d9f0 2023-10-16T20:07:32.254 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.249+0000 7fb69253e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb68c15df10 con 0x7fb68c0d22e0 2023-10-16T20:07:32.254 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.249+0000 7fb69253e640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb68c15e1f0 con 0x7fb68c0d26c0 2023-10-16T20:07:32.255 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.249+0000 7fb68b7fe640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fb68c15d9f0 0x7fb68c0cf2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:37336/0 (socket says 172.21.15.88:37336) 2023-10-16T20:07:32.255 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.249+0000 7fb68b7fe640 1 -- 172.21.15.88:0/4088459549 learned_addr learned my addr 172.21.15.88:0/4088459549 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:32.255 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.249+0000 7fb690adb640 1 --2- 172.21.15.88:0/4088459549 >> v2:172.21.15.88:3300/0 conn(0x7fb68c0d26c0 0x7fb68c161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:32.256 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.249+0000 7fb690adb640 1 -- 172.21.15.88:0/4088459549 >> v1:172.21.15.94:6789/0 conn(0x7fb68c0d22e0 legacy=0x7fb68c15d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:32.256 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.249+0000 7fb690adb640 1 -- 172.21.15.88:0/4088459549 >> v1:172.21.15.88:6789/0 conn(0x7fb68c15d9f0 legacy=0x7fb68c0cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:32.256 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.249+0000 7fb690adb640 1 -- 172.21.15.88:0/4088459549 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb68c160970 con 0x7fb68c0d26c0 2023-10-16T20:07:32.257 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.253+0000 7fb690adb640 1 --2- 172.21.15.88:0/4088459549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb68c0d26c0 0x7fb68c161450 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fb684009d90 tx=0x7fb68401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe9615bfe4baaa8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:32.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.253+0000 7fb68affd640 1 -- 172.21.15.88:0/4088459549 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6840a58f0 con 0x7fb68c0d26c0 2023-10-16T20:07:32.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.253+0000 7fb68affd640 1 -- 172.21.15.88:0/4088459549 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6840a5a50 con 0x7fb68c0d26c0 2023-10-16T20:07:32.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.253+0000 7fb68affd640 1 -- 172.21.15.88:0/4088459549 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6840a5de0 con 0x7fb68c0d26c0 2023-10-16T20:07:32.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.253+0000 7fb69253e640 1 -- 172.21.15.88:0/4088459549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb68c0d26c0 msgr2=0x7fb68c161450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:32.258 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.253+0000 7fb69253e640 1 --2- 172.21.15.88:0/4088459549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb68c0d26c0 0x7fb68c161450 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fb684009d90 tx=0x7fb68401c920 comp rx=0 tx=0).stop 2023-10-16T20:07:32.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.253+0000 7fb69253e640 1 -- 172.21.15.88:0/4088459549 shutdown_connections 2023-10-16T20:07:32.259 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.253+0000 7fb69253e640 1 --2- 172.21.15.88:0/4088459549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb68c0d26c0 0x7fb68c161450 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.253+0000 7fb69253e640 1 -- 172.21.15.88:0/4088459549 >> 172.21.15.88:0/4088459549 conn(0x7fb68c0715c0 msgr2=0x7fb68c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:32.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.253+0000 7fb69253e640 1 -- 172.21.15.88:0/4088459549 shutdown_connections 2023-10-16T20:07:32.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.253+0000 7fb69253e640 1 -- 172.21.15.88:0/4088459549 wait complete. 2023-10-16T20:07:32.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.257+0000 7fb69253e640 1 Processor -- start 2023-10-16T20:07:32.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.257+0000 7fb69253e640 1 -- start start 2023-10-16T20:07:32.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.257+0000 7fb69253e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb68c0d22e0 0x7fb68c145880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:32.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.257+0000 7fb69253e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb68c0d26c0 0x7fb68c151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:32.263 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.257+0000 7fb69253e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb68c15d9f0 0x7fb68c151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:32.263 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.257+0000 7fb69253e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c0ccc40 con 0x7fb68c15d9f0 2023-10-16T20:07:32.263 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.257+0000 7fb69253e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c152300 con 0x7fb68c0d22e0 2023-10-16T20:07:32.263 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.257+0000 7fb69253e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c0c5e70 con 0x7fb68c0d26c0 2023-10-16T20:07:32.263 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.257+0000 7fb690adb640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb68c15d9f0 0x7fb68c151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:32.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.257+0000 7fb690adb640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb68c15d9f0 0x7fb68c151dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47560/0 (socket says 172.21.15.88:47560) 2023-10-16T20:07:32.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.257+0000 7fb690adb640 1 -- 172.21.15.88:0/1715681495 learned_addr learned my addr 172.21.15.88:0/1715681495 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:32.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.257+0000 7fb68b7fe640 1 --2- 172.21.15.88:0/1715681495 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb68c0d26c0 0x7fb68c151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:32.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.257+0000 7fb68bfff640 1 --2- 172.21.15.88:0/1715681495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb68c0d22e0 0x7fb68c145880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:32.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.257+0000 7fb690adb640 1 -- 172.21.15.88:0/1715681495 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb68c0d26c0 msgr2=0x7fb68c151880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:32.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.257+0000 7fb690adb640 1 --2- 172.21.15.88:0/1715681495 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb68c0d26c0 0x7fb68c151880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.257+0000 7fb690adb640 1 -- 172.21.15.88:0/1715681495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb68c0d22e0 msgr2=0x7fb68c145880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:32.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.257+0000 7fb690adb640 1 --2- 172.21.15.88:0/1715681495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb68c0d22e0 0x7fb68c145880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.257+0000 7fb690adb640 1 -- 172.21.15.88:0/1715681495 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb684009a70 con 0x7fb68c15d9f0 2023-10-16T20:07:32.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.257+0000 7fb690adb640 1 --2- 172.21.15.88:0/1715681495 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb68c15d9f0 0x7fb68c151dc0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fb6840b0040 tx=0x7fb6840196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:32.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.257+0000 7fb688ff9640 1 -- 172.21.15.88:0/1715681495 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6840b9860 con 0x7fb68c15d9f0 2023-10-16T20:07:32.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.257+0000 7fb688ff9640 1 -- 172.21.15.88:0/1715681495 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6840b99c0 con 0x7fb68c15d9f0 2023-10-16T20:07:32.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.257+0000 7fb69253e640 1 -- 172.21.15.88:0/1715681495 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb68c0cc960 con 0x7fb68c15d9f0 2023-10-16T20:07:32.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.257+0000 7fb688ff9640 1 -- 172.21.15.88:0/1715681495 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6840b9d30 con 0x7fb68c15d9f0 2023-10-16T20:07:32.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.257+0000 7fb69253e640 1 -- 172.21.15.88:0/1715681495 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb68c152810 con 0x7fb68c15d9f0 2023-10-16T20:07:32.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.261+0000 7fb69253e640 1 -- 172.21.15.88:0/1715681495 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb68c04e9c0 con 0x7fb68c15d9f0 2023-10-16T20:07:32.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.261+0000 7fb688ff9640 1 -- 172.21.15.88:0/1715681495 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb6840ad440 con 0x7fb68c15d9f0 2023-10-16T20:07:32.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.261+0000 7fb688ff9640 1 --2- 172.21.15.88:0/1715681495 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb6780664a0 0x7fb678068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:32.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.261+0000 7fb688ff9640 1 -- 172.21.15.88:0/1715681495 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fb68411f530 con 0x7fb68c15d9f0 2023-10-16T20:07:32.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.261+0000 7fb68bfff640 1 --2- 172.21.15.88:0/1715681495 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb6780664a0 0x7fb678068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:32.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.261+0000 7fb68bfff640 1 --2- 172.21.15.88:0/1715681495 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb6780664a0 0x7fb678068960 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb67c000f70 tx=0x7fb67c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:32.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.265+0000 7fb688ff9640 1 -- 172.21.15.88:0/1715681495 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb6840ec840 con 0x7fb68c15d9f0 2023-10-16T20:07:32.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.449+0000 7fb69253e640 1 -- 172.21.15.88:0/1715681495 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb68c146570 con 0x7fb68c15d9f0 2023-10-16T20:07:32.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.449+0000 7fb688ff9640 1 -- 172.21.15.88:0/1715681495 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) v1 ==== 53+0+88 (secure 0 0 0) 0x7fb6840efe60 con 0x7fb68c15d9f0 2023-10-16T20:07:32.454 INFO:teuthology.orchestra.run.smithi088.stdout:name: testcephfs1, metadata pool: testcephfs1_metadata, data pools: [testcephfs1_data ] 2023-10-16T20:07:32.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.453+0000 7fb69253e640 1 -- 172.21.15.88:0/1715681495 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb6780664a0 msgr2=0x7fb678068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:32.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.453+0000 7fb69253e640 1 --2- 172.21.15.88:0/1715681495 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb6780664a0 0x7fb678068960 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb67c000f70 tx=0x7fb67c009040 comp rx=0 tx=0).stop 2023-10-16T20:07:32.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.453+0000 7fb69253e640 1 -- 172.21.15.88:0/1715681495 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb68c15d9f0 msgr2=0x7fb68c151dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:32.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.453+0000 7fb69253e640 1 --2- 172.21.15.88:0/1715681495 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb68c15d9f0 0x7fb68c151dc0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fb6840b0040 tx=0x7fb6840196c0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.453+0000 7fb69253e640 1 -- 172.21.15.88:0/1715681495 shutdown_connections 2023-10-16T20:07:32.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.453+0000 7fb69253e640 1 --2- 172.21.15.88:0/1715681495 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb6780664a0 0x7fb678068960 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.453+0000 7fb69253e640 1 --2- 172.21.15.88:0/1715681495 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb68c0d26c0 0x7fb68c151880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.453+0000 7fb69253e640 1 --2- 172.21.15.88:0/1715681495 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb68c15d9f0 0x7fb68c151dc0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.453+0000 7fb69253e640 1 --2- 172.21.15.88:0/1715681495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb68c0d22e0 0x7fb68c145880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:32.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.453+0000 7fb69253e640 1 -- 172.21.15.88:0/1715681495 >> 172.21.15.88:0/1715681495 conn(0x7fb68c0715c0 msgr2=0x7fb68c0b5c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:32.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.453+0000 7fb69253e640 1 -- 172.21.15.88:0/1715681495 shutdown_connections 2023-10-16T20:07:32.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:32.453+0000 7fb69253e640 1 -- 172.21.15.88:0/1715681495 wait complete. 2023-10-16T20:07:32.475 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T20:07:32.475 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T20:07:32.475 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi088.front.sepia.ceph.com 2023-10-16T20:07:32.476 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T20:07:32.476 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T20:07:32.476 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T20:07:32.477 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T20:07:32.477 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T20:07:32.477 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T20:07:32.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:32.478 DEBUG:teuthology.orchestra.run.smithi088:> ip addr 2023-10-16T20:07:32.488 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T20:07:32.488 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T20:07:32.488 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T20:07:32.489 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:07:32.489 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2023-10-16T20:07:32.489 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:07:32.489 INFO:teuthology.orchestra.run.smithi088.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:07:32.489 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f0 brd ff:ff:ff:ff:ff:ff 2023-10-16T20:07:32.490 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f0 2023-10-16T20:07:32.490 INFO:teuthology.orchestra.run.smithi088.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:07:32.490 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f1 brd ff:ff:ff:ff:ff:ff 2023-10-16T20:07:32.490 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f1 2023-10-16T20:07:32.490 INFO:teuthology.orchestra.run.smithi088.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:07:32.491 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2c brd ff:ff:ff:ff:ff:ff 2023-10-16T20:07:32.491 INFO:teuthology.orchestra.run.smithi088.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T20:07:32.491 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2d brd ff:ff:ff:ff:ff:ff 2023-10-16T20:07:32.491 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.21.15.88/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-16T20:07:32.491 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft 42279sec preferred_lft 42279sec 2023-10-16T20:07:32.492 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link 2023-10-16T20:07:32.492 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:07:32.492 INFO:teuthology.orchestra.run.smithi088.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-16T20:07:32.492 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 02:42:d9:3b:fb:39 brd ff:ff:ff:ff:ff:ff 2023-10-16T20:07:32.492 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-16T20:07:32.493 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:07:32.493 INFO:teuthology.orchestra.run.smithi088.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T20:07:32.493 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-16T20:07:32.493 INFO:teuthology.orchestra.run.smithi088.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T20:07:32.493 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:07:32.494 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-16T20:07:32.494 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:07:32.494 INFO:teuthology.orchestra.run.smithi088.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T20:07:32.494 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:07:32.494 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-16T20:07:32.495 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:07:32.495 INFO:teuthology.orchestra.run.smithi088.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T20:07:32.495 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:07:32.495 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-16T20:07:32.495 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:07:32.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:32.496 DEBUG:teuthology.orchestra.run.smithi088:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T20:07:32.549 INFO:teuthology.orchestra.run.smithi088.stdout:1 2023-10-16T20:07:32.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:32.552 DEBUG:teuthology.orchestra.run.smithi088:> route 2023-10-16T20:07:32.610 INFO:teuthology.orchestra.run.smithi088.stdout:Kernel IP routing table 2023-10-16T20:07:32.611 INFO:teuthology.orchestra.run.smithi088.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T20:07:32.611 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-16T20:07:32.611 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T20:07:32.611 INFO:teuthology.orchestra.run.smithi088.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-16T20:07:32.612 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T20:07:32.612 INFO:teuthology.orchestra.run.smithi088.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:07:32.612 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:07:32.612 INFO:teuthology.orchestra.run.smithi088.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:07:32.612 INFO:teuthology.orchestra.run.smithi088.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T20:07:32.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:32.613 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec bash -c ' 2023-10-16T20:07:32.614 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2023-10-16T20:07:32.614 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T20:07:32.615 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T20:07:32.615 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T20:07:32.615 DEBUG:teuthology.orchestra.run.smithi088:> ') 2023-10-16T20:07:32.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:32.702 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2023-10-16T20:07:32.712 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T20:07:32.712 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T20:07:32.712 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T20:07:32.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:32.713 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set brx.0 up 2023-10-16T20:07:32.779 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-16T20:07:32.780 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T20:07:32.780 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T20:07:32.831 INFO:teuthology.orchestra.run.smithi088.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T20:07:32.832 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T20:07:32.832 DEBUG:teuthology.orchestra.run.smithi088:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T20:07:32.883 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2023-10-16T20:07:32.943 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T20:07:32.997 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2023-10-16T20:07:32.997 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2023-10-16T20:07:32.997 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2023-10-16T20:07:32.998 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2023-10-16T20:07:32.998 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2023-10-16T20:07:32.998 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2023-10-16T20:07:32.998 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2023-10-16T20:07:32.998 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2023-10-16T20:07:32.999 INFO:teuthology.orchestra.run.smithi088.stdout:/run/lock 2023-10-16T20:07:32.999 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2023-10-16T20:07:32.999 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2023-10-16T20:07:32.999 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2023-10-16T20:07:32.999 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T20:07:33.000 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2023-10-16T20:07:33.000 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2023-10-16T20:07:33.000 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2023-10-16T20:07:33.000 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2023-10-16T20:07:33.000 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/fs/nfsd 2023-10-16T20:07:33.001 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2023-10-16T20:07:33.001 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2023-10-16T20:07:33.001 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T20:07:33.001 INFO:teuthology.orchestra.run.smithi088.stdout:/run/rpc_pipefs 2023-10-16T20:07:33.001 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2023-10-16T20:07:33.001 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2023-10-16T20:07:33.002 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T20:07:33.002 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T20:07:33.002 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T20:07:33.002 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T20:07:33.002 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2023-10-16T20:07:33.003 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:07:33.003 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:07:33.003 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:07:33.003 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:07:33.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:33.004 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2023-10-16T20:07:33.055 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-16T20:07:33.056 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-16T20:07:33.106 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2023-10-16T20:07:33.127 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T20:07:33.185 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2023-10-16T20:07:33.185 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2023-10-16T20:07:33.185 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2023-10-16T20:07:33.186 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2023-10-16T20:07:33.186 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2023-10-16T20:07:33.186 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2023-10-16T20:07:33.186 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2023-10-16T20:07:33.186 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2023-10-16T20:07:33.187 INFO:teuthology.orchestra.run.smithi088.stdout:/run/lock 2023-10-16T20:07:33.187 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2023-10-16T20:07:33.187 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2023-10-16T20:07:33.187 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2023-10-16T20:07:33.187 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T20:07:33.188 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2023-10-16T20:07:33.188 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2023-10-16T20:07:33.188 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2023-10-16T20:07:33.188 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2023-10-16T20:07:33.188 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/fs/nfsd 2023-10-16T20:07:33.189 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2023-10-16T20:07:33.189 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2023-10-16T20:07:33.189 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T20:07:33.189 INFO:teuthology.orchestra.run.smithi088.stdout:/run/rpc_pipefs 2023-10-16T20:07:33.189 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2023-10-16T20:07:33.190 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2023-10-16T20:07:33.190 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T20:07:33.190 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T20:07:33.190 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T20:07:33.190 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T20:07:33.190 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2023-10-16T20:07:33.191 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:07:33.191 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:07:33.191 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:07:33.191 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:07:33.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:33.192 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2023-10-16T20:07:33.201 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:2023-10-16T20:07:33.197+0000 7fbf0ddc6580 -1 init, newargv = 0x561726f35cf0 newargc=15 2023-10-16T20:07:33.201 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[52084]: starting ceph client 2023-10-16T20:07:33.208 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[52084]: starting fuse 2023-10-16T20:07:33.211 INFO:teuthology.orchestra.run.smithi088.stdout:43 2023-10-16T20:07:33.211 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-10-16T20:07:33.211 DEBUG:teuthology.orchestra.run.smithi088:> sudo stdin-killer -- python3 -c ' 2023-10-16T20:07:33.212 DEBUG:teuthology.orchestra.run.smithi088:> import glob 2023-10-16T20:07:33.212 DEBUG:teuthology.orchestra.run.smithi088:> import re 2023-10-16T20:07:33.213 DEBUG:teuthology.orchestra.run.smithi088:> import os 2023-10-16T20:07:33.213 DEBUG:teuthology.orchestra.run.smithi088:> import subprocess 2023-10-16T20:07:33.213 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:07:33.214 DEBUG:teuthology.orchestra.run.smithi088:> def _find_admin_socket(client_name): 2023-10-16T20:07:33.214 DEBUG:teuthology.orchestra.run.smithi088:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-16T20:07:33.214 DEBUG:teuthology.orchestra.run.smithi088:> files = glob.glob(asok_path) 2023-10-16T20:07:33.215 DEBUG:teuthology.orchestra.run.smithi088:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-16T20:07:33.215 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:07:33.215 DEBUG:teuthology.orchestra.run.smithi088:> # Given a non-glob path, it better be there 2023-10-16T20:07:33.216 DEBUG:teuthology.orchestra.run.smithi088:> if "*" not in asok_path: 2023-10-16T20:07:33.216 DEBUG:teuthology.orchestra.run.smithi088:> assert(len(files) == 1) 2023-10-16T20:07:33.216 DEBUG:teuthology.orchestra.run.smithi088:> return files[0] 2023-10-16T20:07:33.217 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:07:33.217 DEBUG:teuthology.orchestra.run.smithi088:> for f in files: 2023-10-16T20:07:33.217 DEBUG:teuthology.orchestra.run.smithi088:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-16T20:07:33.218 DEBUG:teuthology.orchestra.run.smithi088:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-16T20:07:33.218 DEBUG:teuthology.orchestra.run.smithi088:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-16T20:07:33.218 DEBUG:teuthology.orchestra.run.smithi088:> contents = proc_f.read() 2023-10-16T20:07:33.219 DEBUG:teuthology.orchestra.run.smithi088:> if mountpoint in contents: 2023-10-16T20:07:33.219 DEBUG:teuthology.orchestra.run.smithi088:> return f 2023-10-16T20:07:33.220 DEBUG:teuthology.orchestra.run.smithi088:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-16T20:07:33.220 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:07:33.220 DEBUG:teuthology.orchestra.run.smithi088:> print(_find_admin_socket("client.0")) 2023-10-16T20:07:33.221 DEBUG:teuthology.orchestra.run.smithi088:> ' 2023-10-16T20:07:33.335 INFO:teuthology.orchestra.run.smithi088.stdout:/var/run/ceph/ceph-client.0.52084.asok 2023-10-16T20:07:33.338 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-16T20:07:33.346 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52084.asok 2023-10-16T20:07:33.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:33.347 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52084.asok status 2023-10-16T20:07:33.515 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2023-10-16T20:07:33.515 INFO:teuthology.orchestra.run.smithi088.stdout: "metadata": { 2023-10-16T20:07:33.516 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_sha1": "4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0", 2023-10-16T20:07:33.516 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_version": "ceph version 18.0.0-6730-g4ba8a190 (4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0) reef (dev)", 2023-10-16T20:07:33.516 INFO:teuthology.orchestra.run.smithi088.stdout: "entity_id": "0", 2023-10-16T20:07:33.516 INFO:teuthology.orchestra.run.smithi088.stdout: "hostname": "smithi088", 2023-10-16T20:07:33.516 INFO:teuthology.orchestra.run.smithi088.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-16T20:07:33.517 INFO:teuthology.orchestra.run.smithi088.stdout: "pid": "52084", 2023-10-16T20:07:33.517 INFO:teuthology.orchestra.run.smithi088.stdout: "root": "/" 2023-10-16T20:07:33.517 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:07:33.517 INFO:teuthology.orchestra.run.smithi088.stdout: "dentry_count": 0, 2023-10-16T20:07:33.517 INFO:teuthology.orchestra.run.smithi088.stdout: "dentry_pinned_count": 0, 2023-10-16T20:07:33.518 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 6366, 2023-10-16T20:07:33.518 INFO:teuthology.orchestra.run.smithi088.stdout: "inst": { 2023-10-16T20:07:33.518 INFO:teuthology.orchestra.run.smithi088.stdout: "name": { 2023-10-16T20:07:33.518 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "client", 2023-10-16T20:07:33.518 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 6366 2023-10-16T20:07:33.519 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:07:33.519 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": { 2023-10-16T20:07:33.519 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2023-10-16T20:07:33.519 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "192.168.0.1:0", 2023-10-16T20:07:33.519 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 1329823057 2023-10-16T20:07:33.520 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:07:33.520 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:07:33.520 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": { 2023-10-16T20:07:33.520 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2023-10-16T20:07:33.520 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "192.168.0.1:0", 2023-10-16T20:07:33.521 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 1329823057 2023-10-16T20:07:33.521 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:07:33.521 INFO:teuthology.orchestra.run.smithi088.stdout: "inst_str": "client.6366 192.168.0.1:0/1329823057", 2023-10-16T20:07:33.521 INFO:teuthology.orchestra.run.smithi088.stdout: "addr_str": "192.168.0.1:0/1329823057", 2023-10-16T20:07:33.521 INFO:teuthology.orchestra.run.smithi088.stdout: "inode_count": 1, 2023-10-16T20:07:33.521 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_epoch": 28, 2023-10-16T20:07:33.522 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_epoch": 58, 2023-10-16T20:07:33.522 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_epoch_barrier": 0, 2023-10-16T20:07:33.522 INFO:teuthology.orchestra.run.smithi088.stdout: "blocklisted": false, 2023-10-16T20:07:33.522 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "testcephfs1" 2023-10-16T20:07:33.522 INFO:teuthology.orchestra.run.smithi088.stdout:} 2023-10-16T20:07:33.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:33.531 DEBUG:teuthology.orchestra.run.smithi088:> stat --file-system '--printf=%T 2023-10-16T20:07:33.532 DEBUG:teuthology.orchestra.run.smithi088:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-16T20:07:33.541 INFO:teuthology.orchestra.run.smithi088.stdout:fuseblk 2023-10-16T20:07:33.541 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-16T20:07:33.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:33.542 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T20:07:33.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:33.604 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch test_file) 2023-10-16T20:07:33.662 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow r pool=testcephfs1_data' 2023-10-16T20:07:33.849 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.845+0000 7f62a25aa640 1 Processor -- start 2023-10-16T20:07:33.850 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.845+0000 7f62a25aa640 1 -- start start 2023-10-16T20:07:33.851 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.845+0000 7f62a25aa640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f629c06d3f0 0x7f629c06d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:33.851 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.845+0000 7f62a25aa640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f629c15ad30 con 0x7f629c0b4260 2023-10-16T20:07:33.851 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.845+0000 7f62a25aa640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f629c0b4640 con 0x7f629c0b0da0 2023-10-16T20:07:33.852 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.845+0000 7f62a25aa640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f629c06dda0 con 0x7f629c06d3f0 2023-10-16T20:07:33.852 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.845+0000 7f629bfff640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f629c0b4260 0x7f629c15dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:37344/0 (socket says 172.21.15.88:37344) 2023-10-16T20:07:33.852 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.845+0000 7f629bfff640 1 -- 172.21.15.88:0/2376857585 learned_addr learned my addr 172.21.15.88:0/2376857585 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:33.853 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.845+0000 7f62a0b47640 1 --2- 172.21.15.88:0/2376857585 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f629c06d3f0 0x7f629c06d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:33.853 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.849+0000 7f629affd640 1 -- 172.21.15.88:0/2376857585 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 286963293 0 0) 0x7f629c15ad30 con 0x7f629c0b4260 2023-10-16T20:07:33.854 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.849+0000 7f629affd640 1 -- 172.21.15.88:0/2376857585 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6284005740 con 0x7f629c0b4260 2023-10-16T20:07:33.854 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.849+0000 7f629affd640 1 -- 172.21.15.88:0/2376857585 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2256835550 0 0) 0x7f629c0b4640 con 0x7f629c0b0da0 2023-10-16T20:07:33.854 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.849+0000 7f629affd640 1 -- 172.21.15.88:0/2376857585 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f629c15ad30 con 0x7f629c0b0da0 2023-10-16T20:07:33.854 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.849+0000 7f62a0b47640 1 -- 172.21.15.88:0/2376857585 >> v1:172.21.15.94:6789/0 conn(0x7f629c0b0da0 legacy=0x7f629c159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:33.854 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.849+0000 7f629affd640 1 -- 172.21.15.88:0/2376857585 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4048158462 0 0) 0x7f6284005740 con 0x7f629c0b4260 2023-10-16T20:07:33.855 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.849+0000 7f62a0b47640 1 -- 172.21.15.88:0/2376857585 >> v1:172.21.15.88:6789/0 conn(0x7f629c0b4260 legacy=0x7f629c15dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:33.855 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.849+0000 7f62a0b47640 1 -- 172.21.15.88:0/2376857585 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629c15a770 con 0x7f629c06d3f0 2023-10-16T20:07:33.856 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.849+0000 7f62a0b47640 1 --2- 172.21.15.88:0/2376857585 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f629c06d3f0 0x7f629c06d860 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6294009dc0 tx=0x7f629401c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fae0505acdf99ec7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:33.856 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.849+0000 7f629affd640 1 -- 172.21.15.88:0/2376857585 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628c001680 con 0x7f629c06d3f0 2023-10-16T20:07:33.856 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.849+0000 7f629affd640 1 -- 172.21.15.88:0/2376857585 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62940a5530 con 0x7f629c06d3f0 2023-10-16T20:07:33.857 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.849+0000 7f629affd640 1 -- 172.21.15.88:0/2376857585 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62940a58c0 con 0x7f629c06d3f0 2023-10-16T20:07:33.857 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.853+0000 7f62a25aa640 1 -- 172.21.15.88:0/2376857585 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f629c06d3f0 msgr2=0x7f629c06d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:33.857 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.853+0000 7f62a25aa640 1 --2- 172.21.15.88:0/2376857585 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f629c06d3f0 0x7f629c06d860 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6294009dc0 tx=0x7f629401c800 comp rx=0 tx=0).stop 2023-10-16T20:07:33.857 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.853+0000 7f62a25aa640 1 -- 172.21.15.88:0/2376857585 shutdown_connections 2023-10-16T20:07:33.858 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.853+0000 7f62a25aa640 1 --2- 172.21.15.88:0/2376857585 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f629c06d3f0 0x7f629c06d860 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:33.858 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.853+0000 7f62a25aa640 1 -- 172.21.15.88:0/2376857585 >> 172.21.15.88:0/2376857585 conn(0x7f629c0cb080 msgr2=0x7f629c0cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:33.858 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.853+0000 7f62a25aa640 1 -- 172.21.15.88:0/2376857585 shutdown_connections 2023-10-16T20:07:33.858 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.853+0000 7f62a25aa640 1 -- 172.21.15.88:0/2376857585 wait complete. 2023-10-16T20:07:33.860 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.853+0000 7f62a25aa640 1 Processor -- start 2023-10-16T20:07:33.860 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.853+0000 7f62a25aa640 1 -- start start 2023-10-16T20:07:33.861 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.853+0000 7f62a25aa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f629c06d3f0 0x7f629c156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:33.861 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.853+0000 7f62a25aa640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f629c0b0da0 0x7f629c152530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:33.861 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.853+0000 7f62a25aa640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f629c0b4260 0x7f629c137b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:33.861 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.853+0000 7f62a25aa640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629c152fd0 con 0x7f629c0b4260 2023-10-16T20:07:33.861 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.853+0000 7f62a25aa640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629c0ab890 con 0x7f629c06d3f0 2023-10-16T20:07:33.862 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.853+0000 7f62a25aa640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f629c0abaa0 con 0x7f629c0b0da0 2023-10-16T20:07:33.862 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.853+0000 7f62a0b47640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f629c0b4260 0x7f629c137b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:33.862 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.853+0000 7f62a0b47640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f629c0b4260 0x7f629c137b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47570/0 (socket says 172.21.15.88:47570) 2023-10-16T20:07:33.862 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.853+0000 7f62a0b47640 1 -- 172.21.15.88:0/184698117 learned_addr learned my addr 172.21.15.88:0/184698117 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:33.862 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.853+0000 7f629bfff640 1 --2- 172.21.15.88:0/184698117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f629c06d3f0 0x7f629c156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:33.863 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.853+0000 7f629b7fe640 1 --2- 172.21.15.88:0/184698117 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f629c0b0da0 0x7f629c152530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:33.863 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.857+0000 7f62a0b47640 1 -- 172.21.15.88:0/184698117 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f629c0b0da0 msgr2=0x7f629c152530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:33.863 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.857+0000 7f62a0b47640 1 --2- 172.21.15.88:0/184698117 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f629c0b0da0 0x7f629c152530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:33.863 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.857+0000 7f62a0b47640 1 -- 172.21.15.88:0/184698117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f629c06d3f0 msgr2=0x7f629c156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:33.863 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.857+0000 7f62a0b47640 1 --2- 172.21.15.88:0/184698117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f629c06d3f0 0x7f629c156e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:33.864 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.857+0000 7f62a0b47640 1 -- 172.21.15.88:0/184698117 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6294009a70 con 0x7f629c0b4260 2023-10-16T20:07:33.864 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.857+0000 7f62a0b47640 1 --2- 172.21.15.88:0/184698117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f629c0b4260 0x7f629c137b20 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f62940b0040 tx=0x7f62940195a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:33.864 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.857+0000 7f6298ff9640 1 -- 172.21.15.88:0/184698117 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6294003ba0 con 0x7f629c0b4260 2023-10-16T20:07:33.864 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.857+0000 7f6298ff9640 1 -- 172.21.15.88:0/184698117 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6294003d20 con 0x7f629c0b4260 2023-10-16T20:07:33.864 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.857+0000 7f6298ff9640 1 -- 172.21.15.88:0/184698117 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6294004270 con 0x7f629c0b4260 2023-10-16T20:07:33.864 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.857+0000 7f62a25aa640 1 -- 172.21.15.88:0/184698117 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f629c15a540 con 0x7f629c0b4260 2023-10-16T20:07:33.865 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.857+0000 7f62a25aa640 1 -- 172.21.15.88:0/184698117 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f629c0ac060 con 0x7f629c0b4260 2023-10-16T20:07:33.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.857+0000 7f62a25aa640 1 -- 172.21.15.88:0/184698117 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f629c0adbe0 con 0x7f629c0b4260 2023-10-16T20:07:33.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.857+0000 7f6298ff9640 1 -- 172.21.15.88:0/184698117 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f62940b4050 con 0x7f629c0b4260 2023-10-16T20:07:33.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.857+0000 7f6298ff9640 1 --2- 172.21.15.88:0/184698117 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f62880664a0 0x7f6288068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:33.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.857+0000 7f6298ff9640 1 -- 172.21.15.88:0/184698117 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f629411e9b0 con 0x7f629c0b4260 2023-10-16T20:07:33.867 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.857+0000 7f629bfff640 1 --2- 172.21.15.88:0/184698117 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f62880664a0 0x7f6288068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:33.867 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.861+0000 7f629bfff640 1 --2- 172.21.15.88:0/184698117 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f62880664a0 0x7f6288068960 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f628c000fe0 tx=0x7f628c00f1d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:33.867 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:33.861+0000 7f6298ff9640 1 -- 172.21.15.88:0/184698117 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f62940ebd70 con 0x7f629c0b4260 2023-10-16T20:07:34.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.061+0000 7f62a25aa640 1 -- 172.21.15.88:0/184698117 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow r pool=testcephfs1_data"]} v 0) v1 -- 0x7f629c04e9c0 con 0x7f629c0b4260 2023-10-16T20:07:34.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.065+0000 7f6298ff9640 1 -- 172.21.15.88:0/184698117 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow r pool=testcephfs1_data"]}]=0 updated caps for client.0 v18) v1 ==== 188+0+145 (secure 0 0 0) 0x7f62940b20b0 con 0x7f629c0b4260 2023-10-16T20:07:34.070 INFO:teuthology.orchestra.run.smithi088.stdout:[client.0] 2023-10-16T20:07:34.071 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQAvly1looeRAhAAeLshHOVIhQqoTtlPehorIg== 2023-10-16T20:07:34.071 INFO:teuthology.orchestra.run.smithi088.stdout: caps mds = "allow" 2023-10-16T20:07:34.071 INFO:teuthology.orchestra.run.smithi088.stdout: caps mon = "allow r" 2023-10-16T20:07:34.071 INFO:teuthology.orchestra.run.smithi088.stdout: caps osd = "allow r pool=testcephfs1_data" 2023-10-16T20:07:34.072 INFO:teuthology.orchestra.run.smithi088.stderr:updated caps for client.0 2023-10-16T20:07:34.072 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.069+0000 7f62a25aa640 1 -- 172.21.15.88:0/184698117 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f62880664a0 msgr2=0x7f6288068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:34.073 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.069+0000 7f62a25aa640 1 --2- 172.21.15.88:0/184698117 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f62880664a0 0x7f6288068960 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f628c000fe0 tx=0x7f628c00f1d0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.073 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.069+0000 7f62a25aa640 1 -- 172.21.15.88:0/184698117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f629c0b4260 msgr2=0x7f629c137b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:34.073 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.069+0000 7f62a25aa640 1 --2- 172.21.15.88:0/184698117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f629c0b4260 0x7f629c137b20 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f62940b0040 tx=0x7f62940195a0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.073 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.069+0000 7f62a25aa640 1 -- 172.21.15.88:0/184698117 shutdown_connections 2023-10-16T20:07:34.073 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.069+0000 7f62a25aa640 1 --2- 172.21.15.88:0/184698117 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f62880664a0 0x7f6288068960 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.073 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.069+0000 7f62a25aa640 1 --2- 172.21.15.88:0/184698117 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f629c0b0da0 0x7f629c152530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.074 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.069+0000 7f62a25aa640 1 --2- 172.21.15.88:0/184698117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f629c0b4260 0x7f629c137b20 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.074 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.069+0000 7f62a25aa640 1 --2- 172.21.15.88:0/184698117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f629c06d3f0 0x7f629c156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.074 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.069+0000 7f62a25aa640 1 -- 172.21.15.88:0/184698117 >> 172.21.15.88:0/184698117 conn(0x7f629c0cb080 msgr2=0x7f629c159a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:34.074 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.069+0000 7f62a25aa640 1 -- 172.21.15.88:0/184698117 shutdown_connections 2023-10-16T20:07:34.074 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.069+0000 7f62a25aa640 1 -- 172.21.15.88:0/184698117 wait complete. 2023-10-16T20:07:34.091 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:07:34.091 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:07:34.101 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:07:34.287 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.281+0000 7f679db2a640 1 Processor -- start 2023-10-16T20:07:34.288 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f679db2a640 1 -- start start 2023-10-16T20:07:34.289 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f679db2a640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f67980d22e0 0x7f6798161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:34.289 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f679db2a640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f679815d9f0 0x7f67980cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:34.289 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f679db2a640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67980b87e0 con 0x7f67980d26c0 2023-10-16T20:07:34.290 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f679db2a640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f679815df10 con 0x7f67980d22e0 2023-10-16T20:07:34.290 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f679db2a640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f679815e1f0 con 0x7f679815d9f0 2023-10-16T20:07:34.290 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f6796ffd640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f67980d26c0 0x7f679815d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:37358/0 (socket says 172.21.15.88:37358) 2023-10-16T20:07:34.290 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f6796ffd640 1 -- 172.21.15.88:0/1609840703 learned_addr learned my addr 172.21.15.88:0/1609840703 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:34.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f6797fff640 1 --2- 172.21.15.88:0/1609840703 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f679815d9f0 0x7f67980cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:34.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f67967fc640 1 -- 172.21.15.88:0/1609840703 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1203225210 0 0) 0x7f67980b87e0 con 0x7f67980d26c0 2023-10-16T20:07:34.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f67977fe640 1 --2- 172.21.15.88:0/1609840703 >> v2:172.21.15.94:3300/0 conn(0x7f67980d22e0 0x7f6798161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:34.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f67967fc640 1 -- 172.21.15.88:0/1609840703 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6780003530 con 0x7f67980d26c0 2023-10-16T20:07:34.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f67967fc640 1 -- 172.21.15.88:0/1609840703 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1488511374 0 0) 0x7f6780003530 con 0x7f67980d26c0 2023-10-16T20:07:34.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f67967fc640 1 -- 172.21.15.88:0/1609840703 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f679815d9f0 msgr2=0x7f67980cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:34.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f67967fc640 1 --2- 172.21.15.88:0/1609840703 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f679815d9f0 0x7f67980cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f67967fc640 1 -- 172.21.15.88:0/1609840703 >> v2:172.21.15.94:3300/0 conn(0x7f67980d22e0 msgr2=0x7f6798161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:34.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f67967fc640 1 --2- 172.21.15.88:0/1609840703 >> v2:172.21.15.94:3300/0 conn(0x7f67980d22e0 0x7f6798161450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f67967fc640 1 -- 172.21.15.88:0/1609840703 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6798160970 con 0x7f67980d26c0 2023-10-16T20:07:34.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f67977fe640 1 --2- 172.21.15.88:0/1609840703 >> v2:172.21.15.94:3300/0 conn(0x7f67980d22e0 0x7f6798161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:07:34.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f67967fc640 1 -- 172.21.15.88:0/1609840703 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f678c0016a0 con 0x7f67980d26c0 2023-10-16T20:07:34.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f6797fff640 1 --2- 172.21.15.88:0/1609840703 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f679815d9f0 0x7f67980cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:07:34.295 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f67967fc640 1 -- 172.21.15.88:0/1609840703 >> v1:172.21.15.88:6789/0 conn(0x7f67980d26c0 legacy=0x7f679815d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:34.295 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f67967fc640 1 --2- 172.21.15.88:0/1609840703 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f679815d9f0 0x7f6780003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:34.295 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f67967fc640 1 --2- 172.21.15.88:0/1609840703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6780003f70 0x7f67800047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:34.295 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f67967fc640 1 --2- 172.21.15.88:0/1609840703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6780004d10 0x7f6780005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:34.295 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f67967fc640 1 -- 172.21.15.88:0/1609840703 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679815df10 con 0x7f6780004d10 2023-10-16T20:07:34.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f67967fc640 1 -- 172.21.15.88:0/1609840703 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679815e1f0 con 0x7f6780003f70 2023-10-16T20:07:34.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f67967fc640 1 -- 172.21.15.88:0/1609840703 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678000baf0 con 0x7f679815d9f0 2023-10-16T20:07:34.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f6797fff640 1 --2- 172.21.15.88:0/1609840703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6780004d10 0x7f6780005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:34.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f67977fe640 1 --2- 172.21.15.88:0/1609840703 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f679815d9f0 0x7f6780003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:34.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f6796ffd640 1 --2- 172.21.15.88:0/1609840703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6780003f70 0x7f67800047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:34.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f6797fff640 1 -- 172.21.15.88:0/1609840703 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f679815d9f0 msgr2=0x7f6780003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:34.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f6797fff640 1 --2- 172.21.15.88:0/1609840703 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f679815d9f0 0x7f6780003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f6797fff640 1 -- 172.21.15.88:0/1609840703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6780003f70 msgr2=0x7f67800047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:34.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f6797fff640 1 --2- 172.21.15.88:0/1609840703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6780003f70 0x7f67800047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.285+0000 7f6797fff640 1 -- 172.21.15.88:0/1609840703 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6780003530 con 0x7f6780004d10 2023-10-16T20:07:34.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.289+0000 7f6797fff640 1 --2- 172.21.15.88:0/1609840703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6780004d10 0x7f6780005190 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f67900073e0 tx=0x7f679001cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a16543c3796772 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:34.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.289+0000 7f67967fc640 1 -- 172.21.15.88:0/1609840703 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67900a5da0 con 0x7f6780004d10 2023-10-16T20:07:34.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.289+0000 7f67967fc640 1 -- 172.21.15.88:0/1609840703 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67900ad440 con 0x7f6780004d10 2023-10-16T20:07:34.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.289+0000 7f67967fc640 1 -- 172.21.15.88:0/1609840703 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67900ad7b0 con 0x7f6780004d10 2023-10-16T20:07:34.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.289+0000 7f679db2a640 1 -- 172.21.15.88:0/1609840703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6780004d10 msgr2=0x7f6780005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:34.302 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.289+0000 7f679db2a640 1 --2- 172.21.15.88:0/1609840703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6780004d10 0x7f6780005190 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f67900073e0 tx=0x7f679001cea0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.302 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.289+0000 7f6797fff640 1 -- 172.21.15.88:0/1609840703 reap_dead start 2023-10-16T20:07:34.302 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.289+0000 7f679db2a640 1 -- 172.21.15.88:0/1609840703 shutdown_connections 2023-10-16T20:07:34.302 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.289+0000 7f679db2a640 1 -- 172.21.15.88:0/1609840703 >> 172.21.15.88:0/1609840703 conn(0x7f67980715c0 msgr2=0x7f67980719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:34.302 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.289+0000 7f679db2a640 1 -- 172.21.15.88:0/1609840703 shutdown_connections 2023-10-16T20:07:34.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.289+0000 7f679db2a640 1 -- 172.21.15.88:0/1609840703 wait complete. 2023-10-16T20:07:34.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.293+0000 7f679db2a640 1 Processor -- start 2023-10-16T20:07:34.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.293+0000 7f679db2a640 1 -- start start 2023-10-16T20:07:34.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.293+0000 7f679db2a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6780004d10 0x7f6798151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:34.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.293+0000 7f679db2a640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6798156c20 0x7f679815f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:34.304 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.293+0000 7f679db2a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6798157080 0x7f679815fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:34.304 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.293+0000 7f679db2a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67981522c0 con 0x7f6798157080 2023-10-16T20:07:34.304 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.293+0000 7f679db2a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67980cadd0 con 0x7f6780004d10 2023-10-16T20:07:34.304 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.293+0000 7f679db2a640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67980cb0b0 con 0x7f6798156c20 2023-10-16T20:07:34.304 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.293+0000 7f6797fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6798157080 0x7f679815fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:34.305 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.293+0000 7f6797fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6798157080 0x7f679815fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47586/0 (socket says 172.21.15.88:47586) 2023-10-16T20:07:34.305 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.293+0000 7f6797fff640 1 -- 172.21.15.88:0/1110121425 learned_addr learned my addr 172.21.15.88:0/1110121425 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:34.305 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.293+0000 7f6796ffd640 1 --2- 172.21.15.88:0/1110121425 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6798156c20 0x7f679815f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:34.305 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.293+0000 7f67977fe640 1 --2- 172.21.15.88:0/1110121425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6780004d10 0x7f6798151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:34.305 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.293+0000 7f6797fff640 1 -- 172.21.15.88:0/1110121425 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6798156c20 msgr2=0x7f679815f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:34.306 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.293+0000 7f6797fff640 1 --2- 172.21.15.88:0/1110121425 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6798156c20 0x7f679815f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.306 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.293+0000 7f6797fff640 1 -- 172.21.15.88:0/1110121425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6780004d10 msgr2=0x7f6798151d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:34.306 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.293+0000 7f6797fff640 1 --2- 172.21.15.88:0/1110121425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6780004d10 0x7f6798151d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.306 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.293+0000 7f6797fff640 1 -- 172.21.15.88:0/1110121425 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67980cb390 con 0x7f6798157080 2023-10-16T20:07:34.306 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.293+0000 7f6797fff640 1 --2- 172.21.15.88:0/1110121425 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6798157080 0x7f679815fe60 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f67900073e0 tx=0x7f6780005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:34.307 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.293+0000 7f677ffff640 1 -- 172.21.15.88:0/1110121425 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67900ad9c0 con 0x7f6798157080 2023-10-16T20:07:34.307 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.293+0000 7f677ffff640 1 -- 172.21.15.88:0/1110121425 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67900adb20 con 0x7f6798157080 2023-10-16T20:07:34.307 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.293+0000 7f679db2a640 1 -- 172.21.15.88:0/1110121425 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67980cb670 con 0x7f6798157080 2023-10-16T20:07:34.307 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.293+0000 7f677ffff640 1 -- 172.21.15.88:0/1110121425 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67900ade90 con 0x7f6798157080 2023-10-16T20:07:34.307 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.293+0000 7f679db2a640 1 -- 172.21.15.88:0/1110121425 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67980cc930 con 0x7f6798157080 2023-10-16T20:07:34.308 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.297+0000 7f679db2a640 1 -- 172.21.15.88:0/1110121425 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f679804e9c0 con 0x7f6798157080 2023-10-16T20:07:34.308 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.297+0000 7f677ffff640 1 -- 172.21.15.88:0/1110121425 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f67900b4040 con 0x7f6798157080 2023-10-16T20:07:34.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.297+0000 7f677ffff640 1 --2- 172.21.15.88:0/1110121425 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f67840664d0 0x7f6784068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:34.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.297+0000 7f677ffff640 1 -- 172.21.15.88:0/1110121425 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f679011e1c0 con 0x7f6798157080 2023-10-16T20:07:34.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.297+0000 7f67977fe640 1 --2- 172.21.15.88:0/1110121425 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f67840664d0 0x7f6784068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:34.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.297+0000 7f67977fe640 1 --2- 172.21.15.88:0/1110121425 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f67840664d0 0x7f6784068990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6788009a00 tx=0x7f6788008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:34.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.301+0000 7f677ffff640 1 -- 172.21.15.88:0/1110121425 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f67900eb460 con 0x7f6798157080 2023-10-16T20:07:34.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.457+0000 7f679db2a640 1 -- 172.21.15.88:0/1110121425 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67980cbb20 con 0x7f6798157080 2023-10-16T20:07:34.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.457+0000 7f677ffff640 1 -- 172.21.15.88:0/1110121425 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16718 (secure 0 0 0) 0x7f67900be090 con 0x7f6798157080 2023-10-16T20:07:34.462 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:07:34.462 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":58,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:07:27.206614+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T20:07:24.001121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"testcephfs1_data","create_time":"2023-10-16T20:07:24.636209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:07:34.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.457+0000 7f679db2a640 1 -- 172.21.15.88:0/1110121425 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f67840664d0 msgr2=0x7f6784068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:34.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.457+0000 7f679db2a640 1 --2- 172.21.15.88:0/1110121425 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f67840664d0 0x7f6784068990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6788009a00 tx=0x7f6788008040 comp rx=0 tx=0).stop 2023-10-16T20:07:34.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.457+0000 7f679db2a640 1 -- 172.21.15.88:0/1110121425 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6798157080 msgr2=0x7f679815fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:34.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.457+0000 7f679db2a640 1 --2- 172.21.15.88:0/1110121425 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6798157080 0x7f679815fe60 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f67900073e0 tx=0x7f6780005800 comp rx=0 tx=0).stop 2023-10-16T20:07:34.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.457+0000 7f679db2a640 1 -- 172.21.15.88:0/1110121425 shutdown_connections 2023-10-16T20:07:34.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.457+0000 7f679db2a640 1 --2- 172.21.15.88:0/1110121425 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f67840664d0 0x7f6784068990 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.457+0000 7f679db2a640 1 --2- 172.21.15.88:0/1110121425 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6798156c20 0x7f679815f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.457+0000 7f679db2a640 1 --2- 172.21.15.88:0/1110121425 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6798157080 0x7f679815fe60 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.457+0000 7f679db2a640 1 --2- 172.21.15.88:0/1110121425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6780004d10 0x7f6798151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.457+0000 7f679db2a640 1 -- 172.21.15.88:0/1110121425 >> 172.21.15.88:0/1110121425 conn(0x7f67980715c0 msgr2=0x7f67980b66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:34.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.461+0000 7f679db2a640 1 -- 172.21.15.88:0/1110121425 shutdown_connections 2023-10-16T20:07:34.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.461+0000 7f679db2a640 1 -- 172.21.15.88:0/1110121425 wait complete. 2023-10-16T20:07:34.483 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'testcephfs1_metadata', 'create_time': '2023-10-16T20:07:24.001121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'testcephfs1_data', 'create_time': '2023-10-16T20:07:24.636209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T20:07:34.484 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:07:34.624 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.621+0000 7fec86f2a640 1 Processor -- start 2023-10-16T20:07:34.624 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.621+0000 7fec86f2a640 1 -- start start 2023-10-16T20:07:34.625 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.621+0000 7fec86f2a640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fec8015cea0 0x7fec8006af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:34.625 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.621+0000 7fec86f2a640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec8006a3a0 con 0x7fec8015cac0 2023-10-16T20:07:34.625 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.621+0000 7fec86f2a640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec800ca650 con 0x7fec800cd370 2023-10-16T20:07:34.626 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.621+0000 7fec86f2a640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec800cac10 con 0x7fec8015cea0 2023-10-16T20:07:34.626 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.621+0000 7fec84cc6640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fec8015cac0 0x7fec800ccc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:37370/0 (socket says 172.21.15.88:37370) 2023-10-16T20:07:34.626 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.621+0000 7fec84cc6640 1 -- 172.21.15.88:0/4253747297 learned_addr learned my addr 172.21.15.88:0/4253747297 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:34.628 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.621+0000 7fec854c7640 1 --2- 172.21.15.88:0/4253747297 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fec8015cea0 0x7fec8006af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:34.628 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.621+0000 7fec7f7fe640 1 -- 172.21.15.88:0/4253747297 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 446331867 0 0) 0x7fec8006a3a0 con 0x7fec8015cac0 2023-10-16T20:07:34.628 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.621+0000 7fec7f7fe640 1 -- 172.21.15.88:0/4253747297 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec70003530 con 0x7fec8015cac0 2023-10-16T20:07:34.629 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.621+0000 7fec7f7fe640 1 -- 172.21.15.88:0/4253747297 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2497515750 0 0) 0x7fec70003530 con 0x7fec8015cac0 2023-10-16T20:07:34.629 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.621+0000 7fec7f7fe640 1 -- 172.21.15.88:0/4253747297 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fec8015cea0 msgr2=0x7fec8006af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:34.629 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.621+0000 7fec7f7fe640 1 --2- 172.21.15.88:0/4253747297 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fec8015cea0 0x7fec8006af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.629 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.621+0000 7fec7f7fe640 1 -- 172.21.15.88:0/4253747297 >> v1:172.21.15.94:6789/0 conn(0x7fec800cd370 legacy=0x7fec8006a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:34.629 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.621+0000 7fec7f7fe640 1 -- 172.21.15.88:0/4253747297 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec800ca930 con 0x7fec8015cac0 2023-10-16T20:07:34.629 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.621+0000 7fec7f7fe640 1 -- 172.21.15.88:0/4253747297 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7fec74001680 con 0x7fec8015cac0 2023-10-16T20:07:34.630 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.621+0000 7fec7f7fe640 1 -- 172.21.15.88:0/4253747297 >> v1:172.21.15.88:6789/0 conn(0x7fec8015cac0 legacy=0x7fec800ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:34.630 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.621+0000 7fec7f7fe640 1 --2- 172.21.15.88:0/4253747297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec700039b0 0x7fec70003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:34.630 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.621+0000 7fec7f7fe640 1 --2- 172.21.15.88:0/4253747297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec70004360 0x7fec70004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:34.630 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.621+0000 7fec7f7fe640 1 --2- 172.21.15.88:0/4253747297 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fec8015cea0 0x7fec70005090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:34.630 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.621+0000 7fec7f7fe640 1 -- 172.21.15.88:0/4253747297 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec800cac10 con 0x7fec70004360 2023-10-16T20:07:34.631 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.621+0000 7fec7f7fe640 1 -- 172.21.15.88:0/4253747297 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7000b9c0 con 0x7fec700039b0 2023-10-16T20:07:34.631 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.621+0000 7fec7f7fe640 1 -- 172.21.15.88:0/4253747297 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec7000bb80 con 0x7fec8015cea0 2023-10-16T20:07:34.631 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.621+0000 7fec7ffff640 1 --2- 172.21.15.88:0/4253747297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec70004360 0x7fec70004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:34.631 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.621+0000 7fec84cc6640 1 --2- 172.21.15.88:0/4253747297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec700039b0 0x7fec70003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:34.631 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.621+0000 7fec854c7640 1 --2- 172.21.15.88:0/4253747297 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fec8015cea0 0x7fec70005090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:34.632 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.621+0000 7fec7ffff640 1 -- 172.21.15.88:0/4253747297 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fec8015cea0 msgr2=0x7fec70005090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:34.632 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.621+0000 7fec7ffff640 1 --2- 172.21.15.88:0/4253747297 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fec8015cea0 0x7fec70005090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.632 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.621+0000 7fec7ffff640 1 -- 172.21.15.88:0/4253747297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec700039b0 msgr2=0x7fec70003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:34.632 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.621+0000 7fec7ffff640 1 --2- 172.21.15.88:0/4253747297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec700039b0 0x7fec70003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.632 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.621+0000 7fec7ffff640 1 -- 172.21.15.88:0/4253747297 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec70003530 con 0x7fec70004360 2023-10-16T20:07:34.635 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.625+0000 7fec7ffff640 1 --2- 172.21.15.88:0/4253747297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec70004360 0x7fec70004830 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fec6c001860 tx=0x7fec6c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e118805a9adcad3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:34.635 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.625+0000 7fec7f7fe640 1 -- 172.21.15.88:0/4253747297 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c0ab600 con 0x7fec70004360 2023-10-16T20:07:34.636 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.625+0000 7fec7f7fe640 1 -- 172.21.15.88:0/4253747297 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec6c0a7050 con 0x7fec70004360 2023-10-16T20:07:34.636 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.625+0000 7fec7f7fe640 1 -- 172.21.15.88:0/4253747297 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c0ab9a0 con 0x7fec70004360 2023-10-16T20:07:34.636 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.625+0000 7fec86f2a640 1 -- 172.21.15.88:0/4253747297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec70004360 msgr2=0x7fec70004830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:34.636 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.625+0000 7fec86f2a640 1 --2- 172.21.15.88:0/4253747297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec70004360 0x7fec70004830 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fec6c001860 tx=0x7fec6c01c4e0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.636 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.625+0000 7fec854c7640 1 -- 172.21.15.88:0/4253747297 reap_dead start 2023-10-16T20:07:34.637 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.625+0000 7fec86f2a640 1 -- 172.21.15.88:0/4253747297 shutdown_connections 2023-10-16T20:07:34.637 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.625+0000 7fec86f2a640 1 -- 172.21.15.88:0/4253747297 >> 172.21.15.88:0/4253747297 conn(0x7fec800b0970 msgr2=0x7fec800b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:34.637 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.625+0000 7fec86f2a640 1 -- 172.21.15.88:0/4253747297 shutdown_connections 2023-10-16T20:07:34.637 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.625+0000 7fec86f2a640 1 -- 172.21.15.88:0/4253747297 wait complete. 2023-10-16T20:07:34.637 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.629+0000 7fec86f2a640 1 Processor -- start 2023-10-16T20:07:34.638 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.629+0000 7fec86f2a640 1 -- start start 2023-10-16T20:07:34.638 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.629+0000 7fec86f2a640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fec70004360 0x7fec800c5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:34.638 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.629+0000 7fec86f2a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec800c6490 0x7fec801f21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:34.638 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.629+0000 7fec86f2a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec801f2730 0x7fec801f2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:34.638 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.629+0000 7fec86f2a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec80157a50 con 0x7fec801f2730 2023-10-16T20:07:34.639 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.629+0000 7fec86f2a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec800c7640 con 0x7fec800c6490 2023-10-16T20:07:34.639 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.629+0000 7fec86f2a640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec801f3120 con 0x7fec70004360 2023-10-16T20:07:34.639 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.629+0000 7fec854c7640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec801f2730 0x7fec801f2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:34.639 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.629+0000 7fec854c7640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec801f2730 0x7fec801f2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47604/0 (socket says 172.21.15.88:47604) 2023-10-16T20:07:34.639 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.629+0000 7fec854c7640 1 -- 172.21.15.88:0/1758086518 learned_addr learned my addr 172.21.15.88:0/1758086518 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:34.640 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.629+0000 7fec84cc6640 1 --2- 172.21.15.88:0/1758086518 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fec70004360 0x7fec800c5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:34.640 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.629+0000 7fec7ffff640 1 --2- 172.21.15.88:0/1758086518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec800c6490 0x7fec801f21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:34.640 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.629+0000 7fec854c7640 1 -- 172.21.15.88:0/1758086518 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fec70004360 msgr2=0x7fec800c5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:34.640 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.629+0000 7fec854c7640 1 --2- 172.21.15.88:0/1758086518 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fec70004360 0x7fec800c5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.640 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.629+0000 7fec854c7640 1 -- 172.21.15.88:0/1758086518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec800c6490 msgr2=0x7fec801f21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:34.640 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.629+0000 7fec854c7640 1 --2- 172.21.15.88:0/1758086518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec800c6490 0x7fec801f21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.641 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.629+0000 7fec854c7640 1 -- 172.21.15.88:0/1758086518 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec800b4780 con 0x7fec801f2730 2023-10-16T20:07:34.641 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.629+0000 7fec854c7640 1 --2- 172.21.15.88:0/1758086518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec801f2730 0x7fec801f2be0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fec80071f60 tx=0x7fec7800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:34.641 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.629+0000 7fec7d7fa640 1 -- 172.21.15.88:0/1758086518 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7800e040 con 0x7fec801f2730 2023-10-16T20:07:34.641 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.629+0000 7fec7d7fa640 1 -- 172.21.15.88:0/1758086518 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7800f040 con 0x7fec801f2730 2023-10-16T20:07:34.641 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.629+0000 7fec7d7fa640 1 -- 172.21.15.88:0/1758086518 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec78015670 con 0x7fec801f2730 2023-10-16T20:07:34.642 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.629+0000 7fec86f2a640 1 -- 172.21.15.88:0/1758086518 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec800c6ac0 con 0x7fec801f2730 2023-10-16T20:07:34.642 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.629+0000 7fec86f2a640 1 -- 172.21.15.88:0/1758086518 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec800c7080 con 0x7fec801f2730 2023-10-16T20:07:34.642 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.633+0000 7fec86f2a640 1 -- 172.21.15.88:0/1758086518 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec800c6870 con 0x7fec801f2730 2023-10-16T20:07:34.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.633+0000 7fec7d7fa640 1 -- 172.21.15.88:0/1758086518 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fec780157d0 con 0x7fec801f2730 2023-10-16T20:07:34.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.633+0000 7fec7d7fa640 1 --2- 172.21.15.88:0/1758086518 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fec640664a0 0x7fec64068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:34.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.633+0000 7fec7d7fa640 1 -- 172.21.15.88:0/1758086518 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fec78084670 con 0x7fec801f2730 2023-10-16T20:07:34.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.633+0000 7fec84cc6640 1 --2- 172.21.15.88:0/1758086518 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fec640664a0 0x7fec64068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:34.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.633+0000 7fec84cc6640 1 --2- 172.21.15.88:0/1758086518 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fec640664a0 0x7fec64068960 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fec74000c10 tx=0x7fec740047f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:34.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.637+0000 7fec7d7fa640 1 -- 172.21.15.88:0/1758086518 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fec78051980 con 0x7fec801f2730 2023-10-16T20:07:34.789 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.785+0000 7fec86f2a640 1 -- 172.21.15.88:0/1758086518 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fec8004e9c0 con 0x7fec801f2730 2023-10-16T20:07:34.789 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.785+0000 7fec7d7fa640 1 -- 172.21.15.88:0/1758086518 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7fec78054fa0 con 0x7fec801f2730 2023-10-16T20:07:34.790 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:07:34.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.785+0000 7fec86f2a640 1 -- 172.21.15.88:0/1758086518 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fec640664a0 msgr2=0x7fec64068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:34.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.785+0000 7fec86f2a640 1 --2- 172.21.15.88:0/1758086518 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fec640664a0 0x7fec64068960 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fec74000c10 tx=0x7fec740047f0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.785+0000 7fec86f2a640 1 -- 172.21.15.88:0/1758086518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec801f2730 msgr2=0x7fec801f2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:34.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.785+0000 7fec86f2a640 1 --2- 172.21.15.88:0/1758086518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec801f2730 0x7fec801f2be0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fec80071f60 tx=0x7fec7800f580 comp rx=0 tx=0).stop 2023-10-16T20:07:34.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.785+0000 7fec86f2a640 1 -- 172.21.15.88:0/1758086518 shutdown_connections 2023-10-16T20:07:34.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.785+0000 7fec86f2a640 1 --2- 172.21.15.88:0/1758086518 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fec640664a0 0x7fec64068960 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.785+0000 7fec86f2a640 1 --2- 172.21.15.88:0/1758086518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec801f2730 0x7fec801f2be0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.785+0000 7fec86f2a640 1 --2- 172.21.15.88:0/1758086518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec800c6490 0x7fec801f21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.785+0000 7fec86f2a640 1 --2- 172.21.15.88:0/1758086518 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fec70004360 0x7fec800c5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.785+0000 7fec86f2a640 1 -- 172.21.15.88:0/1758086518 >> 172.21.15.88:0/1758086518 conn(0x7fec800b0970 msgr2=0x7fec8006c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:34.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.785+0000 7fec86f2a640 1 -- 172.21.15.88:0/1758086518 shutdown_connections 2023-10-16T20:07:34.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.785+0000 7fec86f2a640 1 -- 172.21.15.88:0/1758086518 wait complete. 2023-10-16T20:07:34.810 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_metadata pg_num 2023-10-16T20:07:34.950 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.945+0000 7f97b1cc9640 1 Processor -- start 2023-10-16T20:07:34.951 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.945+0000 7f97b1cc9640 1 -- start start 2023-10-16T20:07:34.952 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.949+0000 7f97b1cc9640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f97ac0b4260 0x7f97ac15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:34.953 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.949+0000 7f97b1cc9640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f97ac0b0da0 0x7f97ac159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:34.953 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.949+0000 7f97b1cc9640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97ac15ad30 con 0x7f97ac15a1f0 2023-10-16T20:07:34.953 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.949+0000 7f97b1cc9640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97ac0b4640 con 0x7f97ac0b4260 2023-10-16T20:07:34.953 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.949+0000 7f97b1cc9640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97ac15a770 con 0x7f97ac0b0da0 2023-10-16T20:07:34.953 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.949+0000 7f97ab7fe640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f97ac0b4260 0x7f97ac15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:34.954 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.949+0000 7f97ab7fe640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f97ac0b4260 0x7f97ac15dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47612/0 (socket says 172.21.15.88:47612) 2023-10-16T20:07:34.954 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.949+0000 7f97abfff640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f97ac15a1f0 0x7f97ac06d3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:37380/0 (socket says 172.21.15.88:37380) 2023-10-16T20:07:34.954 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.949+0000 7f97ab7fe640 1 -- 172.21.15.88:0/3174408036 learned_addr learned my addr 172.21.15.88:0/3174408036 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:34.954 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.949+0000 7f97aaffd640 1 --2- 172.21.15.88:0/3174408036 >> v2:172.21.15.94:3300/0 conn(0x7f97ac0b0da0 0x7f97ac159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:34.954 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.949+0000 7f97ab7fe640 1 -- 172.21.15.88:0/3174408036 >> v2:172.21.15.94:3300/0 conn(0x7f97ac0b0da0 msgr2=0x7f97ac159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:34.955 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.949+0000 7f97ab7fe640 1 --2- 172.21.15.88:0/3174408036 >> v2:172.21.15.94:3300/0 conn(0x7f97ac0b0da0 0x7f97ac159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.955 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.949+0000 7f97ab7fe640 1 -- 172.21.15.88:0/3174408036 >> v1:172.21.15.88:6789/0 conn(0x7f97ac15a1f0 legacy=0x7f97ac06d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:34.955 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.949+0000 7f97ab7fe640 1 -- 172.21.15.88:0/3174408036 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ac15aa50 con 0x7f97ac0b4260 2023-10-16T20:07:34.955 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.949+0000 7f97aa7fc640 1 -- 172.21.15.88:0/3174408036 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 121765186 0 0) 0x7f97ac15ad30 con 0x7f97ac15a1f0 2023-10-16T20:07:34.956 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.953+0000 7f97ab7fe640 1 --2- 172.21.15.88:0/3174408036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97ac0b4260 0x7f97ac15dc90 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f97ac0ae330 tx=0x7f979c01e650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d3ef73e0cb987c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:34.957 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.953+0000 7f97aa7fc640 1 -- 172.21.15.88:0/3174408036 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c0a8620 con 0x7f97ac0b4260 2023-10-16T20:07:34.957 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.953+0000 7f97aa7fc640 1 -- 172.21.15.88:0/3174408036 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979c0a8780 con 0x7f97ac0b4260 2023-10-16T20:07:34.957 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.953+0000 7f97aa7fc640 1 -- 172.21.15.88:0/3174408036 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c0a8b10 con 0x7f97ac0b4260 2023-10-16T20:07:34.957 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.953+0000 7f97b1cc9640 1 -- 172.21.15.88:0/3174408036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97ac0b4260 msgr2=0x7f97ac15dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:34.957 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.953+0000 7f97b1cc9640 1 --2- 172.21.15.88:0/3174408036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97ac0b4260 0x7f97ac15dc90 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f97ac0ae330 tx=0x7f979c01e650 comp rx=0 tx=0).stop 2023-10-16T20:07:34.958 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.953+0000 7f97b1cc9640 1 -- 172.21.15.88:0/3174408036 shutdown_connections 2023-10-16T20:07:34.958 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.953+0000 7f97b1cc9640 1 --2- 172.21.15.88:0/3174408036 >> v2:172.21.15.94:3300/0 conn(0x7f97ac0b0da0 0x7f97ac159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.958 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.953+0000 7f97b1cc9640 1 --2- 172.21.15.88:0/3174408036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97ac0b4260 0x7f97ac15dc90 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.958 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.953+0000 7f97b1cc9640 1 -- 172.21.15.88:0/3174408036 >> 172.21.15.88:0/3174408036 conn(0x7f97ac0cb080 msgr2=0x7f97ac0cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:34.958 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.953+0000 7f97b1cc9640 1 -- 172.21.15.88:0/3174408036 shutdown_connections 2023-10-16T20:07:34.959 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.953+0000 7f97b1cc9640 1 -- 172.21.15.88:0/3174408036 wait complete. 2023-10-16T20:07:34.960 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.953+0000 7f97b1cc9640 1 Processor -- start 2023-10-16T20:07:34.961 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.953+0000 7f97b1cc9640 1 -- start start 2023-10-16T20:07:34.961 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.953+0000 7f97b1cc9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f97ac0b0da0 0x7f97ac1586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:34.961 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.953+0000 7f97b1cc9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97ac0b4260 0x7f97ac14c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:34.961 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.953+0000 7f97b1cc9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97ac15a1f0 0x7f97ac153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:34.961 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.953+0000 7f97b1cc9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac157620 con 0x7f97ac0b4260 2023-10-16T20:07:34.962 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.953+0000 7f97b1cc9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac157900 con 0x7f97ac15a1f0 2023-10-16T20:07:34.962 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.953+0000 7f97b1cc9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac157be0 con 0x7f97ac0b0da0 2023-10-16T20:07:34.962 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.957+0000 7f97aaffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97ac0b4260 0x7f97ac14c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:34.962 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.957+0000 7f97aaffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97ac0b4260 0x7f97ac14c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47614/0 (socket says 172.21.15.88:47614) 2023-10-16T20:07:34.962 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.957+0000 7f97aaffd640 1 -- 172.21.15.88:0/3191596222 learned_addr learned my addr 172.21.15.88:0/3191596222 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:34.963 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.957+0000 7f97aaffd640 1 -- 172.21.15.88:0/3191596222 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f97ac0b0da0 msgr2=0x7f97ac1586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:34.963 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.957+0000 7f97aaffd640 1 --2- 172.21.15.88:0/3191596222 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f97ac0b0da0 0x7f97ac1586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.963 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.957+0000 7f97aaffd640 1 -- 172.21.15.88:0/3191596222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97ac15a1f0 msgr2=0x7f97ac153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:34.963 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.957+0000 7f97aaffd640 1 --2- 172.21.15.88:0/3191596222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97ac15a1f0 0x7f97ac153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:34.963 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.957+0000 7f97aaffd640 1 -- 172.21.15.88:0/3191596222 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979c0094d0 con 0x7f97ac0b4260 2023-10-16T20:07:34.964 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.957+0000 7f97aaffd640 1 --2- 172.21.15.88:0/3191596222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97ac0b4260 0x7f97ac14c080 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f97a000e930 tx=0x7f97a000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:34.964 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.957+0000 7f9793fff640 1 -- 172.21.15.88:0/3191596222 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a0009690 con 0x7f97ac0b4260 2023-10-16T20:07:34.964 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.957+0000 7f9793fff640 1 -- 172.21.15.88:0/3191596222 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a000ca00 con 0x7f97ac0b4260 2023-10-16T20:07:34.964 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.957+0000 7f9793fff640 1 -- 172.21.15.88:0/3191596222 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a000cd70 con 0x7f97ac0b4260 2023-10-16T20:07:34.964 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.957+0000 7f97b1cc9640 1 -- 172.21.15.88:0/3191596222 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ac157ec0 con 0x7f97ac0b4260 2023-10-16T20:07:34.965 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.957+0000 7f97b1cc9640 1 -- 172.21.15.88:0/3191596222 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ac157310 con 0x7f97ac0b4260 2023-10-16T20:07:34.965 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.957+0000 7f97b1cc9640 1 -- 172.21.15.88:0/3191596222 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ac04e9c0 con 0x7f97ac0b4260 2023-10-16T20:07:34.966 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.957+0000 7f9793fff640 1 -- 172.21.15.88:0/3191596222 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f97a0013470 con 0x7f97ac0b4260 2023-10-16T20:07:34.966 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.957+0000 7f9793fff640 1 --2- 172.21.15.88:0/3191596222 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f97980664a0 0x7f9798068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:34.966 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.957+0000 7f9793fff640 1 -- 172.21.15.88:0/3191596222 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f97a0084350 con 0x7f97ac0b4260 2023-10-16T20:07:34.966 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.961+0000 7f97ab7fe640 1 --2- 172.21.15.88:0/3191596222 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f97980664a0 0x7f9798068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:34.967 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.961+0000 7f97ab7fe640 1 --2- 172.21.15.88:0/3191596222 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f97980664a0 0x7f9798068960 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f979c0b1000 tx=0x7f979c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:34.967 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:34.961+0000 7f9793fff640 1 -- 172.21.15.88:0/3191596222 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f97a0051690 con 0x7f97ac0b4260 2023-10-16T20:07:35.116 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.113+0000 7f97b1cc9640 1 -- 172.21.15.88:0/3191596222 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"} v 0) v1 -- 0x7f97ac071500 con 0x7f97ac0b4260 2023-10-16T20:07:35.117 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.113+0000 7f9793fff640 1 -- 172.21.15.88:0/3191596222 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"}]=0 v58) v1 ==== 109+0+11 (secure 0 0 0) 0x7f97a0054cb0 con 0x7f97ac0b4260 2023-10-16T20:07:35.117 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:07:35.118 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.113+0000 7f97b1cc9640 1 -- 172.21.15.88:0/3191596222 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f97980664a0 msgr2=0x7f9798068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:35.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.113+0000 7f97b1cc9640 1 --2- 172.21.15.88:0/3191596222 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f97980664a0 0x7f9798068960 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f979c0b1000 tx=0x7f979c008040 comp rx=0 tx=0).stop 2023-10-16T20:07:35.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.113+0000 7f97b1cc9640 1 -- 172.21.15.88:0/3191596222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97ac0b4260 msgr2=0x7f97ac14c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:35.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.113+0000 7f97b1cc9640 1 --2- 172.21.15.88:0/3191596222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97ac0b4260 0x7f97ac14c080 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f97a000e930 tx=0x7f97a000edf0 comp rx=0 tx=0).stop 2023-10-16T20:07:35.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.113+0000 7f97b1cc9640 1 -- 172.21.15.88:0/3191596222 shutdown_connections 2023-10-16T20:07:35.119 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.113+0000 7f97b1cc9640 1 --2- 172.21.15.88:0/3191596222 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f97980664a0 0x7f9798068960 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:35.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.113+0000 7f97b1cc9640 1 --2- 172.21.15.88:0/3191596222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97ac15a1f0 0x7f97ac153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:35.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.113+0000 7f97b1cc9640 1 --2- 172.21.15.88:0/3191596222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97ac0b4260 0x7f97ac14c080 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:35.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.113+0000 7f97b1cc9640 1 --2- 172.21.15.88:0/3191596222 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f97ac0b0da0 0x7f97ac1586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:35.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.113+0000 7f97b1cc9640 1 -- 172.21.15.88:0/3191596222 >> 172.21.15.88:0/3191596222 conn(0x7f97ac0cb080 msgr2=0x7f97ac0b33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:35.120 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.113+0000 7f97b1cc9640 1 -- 172.21.15.88:0/3191596222 shutdown_connections 2023-10-16T20:07:35.121 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.113+0000 7f97b1cc9640 1 -- 172.21.15.88:0/3191596222 wait complete. 2023-10-16T20:07:35.137 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_data pg_num 2023-10-16T20:07:35.278 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.273+0000 7f491ecea640 1 Processor -- start 2023-10-16T20:07:35.279 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.273+0000 7f491ecea640 1 -- start start 2023-10-16T20:07:35.279 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.273+0000 7f491ecea640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f49180b4260 0x7f491815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:35.280 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.273+0000 7f491ecea640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49180b0da0 0x7f4918159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:35.280 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.273+0000 7f491ecea640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f491815a1f0 0x7f491806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:35.280 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.273+0000 7f491ecea640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f491815ad30 con 0x7f49180b4260 2023-10-16T20:07:35.280 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.273+0000 7f491ecea640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49180b4640 con 0x7f491815a1f0 2023-10-16T20:07:35.280 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.273+0000 7f491ecea640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f491815a770 con 0x7f49180b0da0 2023-10-16T20:07:35.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.277+0000 7f491ca86640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f49180b4260 0x7f491815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:35.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.277+0000 7f491ca86640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f49180b4260 0x7f491815dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47628/0 (socket says 172.21.15.88:47628) 2023-10-16T20:07:35.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.277+0000 7f491ca86640 1 -- 172.21.15.88:0/1563217129 learned_addr learned my addr 172.21.15.88:0/1563217129 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:35.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.277+0000 7f491d287640 1 --2- 172.21.15.88:0/1563217129 >> v2:172.21.15.94:3300/0 conn(0x7f491815a1f0 0x7f491806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:35.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.277+0000 7f4917fff640 1 --2- 172.21.15.88:0/1563217129 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49180b0da0 0x7f4918159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:35.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.277+0000 7f491ca86640 1 -- 172.21.15.88:0/1563217129 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49180b0da0 msgr2=0x7f4918159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:35.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.277+0000 7f491ca86640 1 --2- 172.21.15.88:0/1563217129 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49180b0da0 0x7f4918159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:35.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.277+0000 7f491ca86640 1 -- 172.21.15.88:0/1563217129 >> v2:172.21.15.94:3300/0 conn(0x7f491815a1f0 msgr2=0x7f491806d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:35.283 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.277+0000 7f491ca86640 1 --2- 172.21.15.88:0/1563217129 >> v2:172.21.15.94:3300/0 conn(0x7f491815a1f0 0x7f491806d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:35.283 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.277+0000 7f491ca86640 1 -- 172.21.15.88:0/1563217129 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f491815aa50 con 0x7f49180b4260 2023-10-16T20:07:35.286 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.277+0000 7f491ca86640 1 --2- 172.21.15.88:0/1563217129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49180b4260 0x7f491815dc90 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f49180b1200 tx=0x7f490c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387529dac649e784 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:35.286 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.277+0000 7f49177fe640 1 -- 172.21.15.88:0/1563217129 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c0ac640 con 0x7f49180b4260 2023-10-16T20:07:35.286 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.277+0000 7f49177fe640 1 -- 172.21.15.88:0/1563217129 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f490c0ac7a0 con 0x7f49180b4260 2023-10-16T20:07:35.286 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.277+0000 7f49177fe640 1 -- 172.21.15.88:0/1563217129 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c0acb30 con 0x7f49180b4260 2023-10-16T20:07:35.286 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.281+0000 7f491ecea640 1 -- 172.21.15.88:0/1563217129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49180b4260 msgr2=0x7f491815dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:35.287 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.281+0000 7f491ecea640 1 --2- 172.21.15.88:0/1563217129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49180b4260 0x7f491815dc90 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f49180b1200 tx=0x7f490c01c440 comp rx=0 tx=0).stop 2023-10-16T20:07:35.287 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.281+0000 7f491ecea640 1 -- 172.21.15.88:0/1563217129 shutdown_connections 2023-10-16T20:07:35.287 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.281+0000 7f491ecea640 1 --2- 172.21.15.88:0/1563217129 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49180b0da0 0x7f4918159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:35.287 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.281+0000 7f491ecea640 1 --2- 172.21.15.88:0/1563217129 >> v2:172.21.15.94:3300/0 conn(0x7f491815a1f0 0x7f491806d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:35.287 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.281+0000 7f491ecea640 1 --2- 172.21.15.88:0/1563217129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49180b4260 0x7f491815dc90 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:35.288 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.281+0000 7f491ecea640 1 -- 172.21.15.88:0/1563217129 >> 172.21.15.88:0/1563217129 conn(0x7f49180cb080 msgr2=0x7f49180cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:35.288 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.281+0000 7f491ecea640 1 -- 172.21.15.88:0/1563217129 shutdown_connections 2023-10-16T20:07:35.288 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.281+0000 7f491ecea640 1 -- 172.21.15.88:0/1563217129 wait complete. 2023-10-16T20:07:35.288 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.281+0000 7f491ecea640 1 Processor -- start 2023-10-16T20:07:35.288 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.281+0000 7f491ecea640 1 -- start start 2023-10-16T20:07:35.290 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.281+0000 7f491ecea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49180b0da0 0x7f491815c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:35.291 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.281+0000 7f491ecea640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49180b4260 0x7f491815c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:35.291 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.281+0000 7f491ecea640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f491815a1f0 0x7f4918157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:35.291 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.281+0000 7f491ecea640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4918159250 con 0x7f491815a1f0 2023-10-16T20:07:35.291 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.281+0000 7f491ecea640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49181589b0 con 0x7f49180b0da0 2023-10-16T20:07:35.291 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.281+0000 7f491ecea640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4918158c90 con 0x7f49180b4260 2023-10-16T20:07:35.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.281+0000 7f491d287640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f491815a1f0 0x7f4918157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:35.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.281+0000 7f491d287640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f491815a1f0 0x7f4918157c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47634/0 (socket says 172.21.15.88:47634) 2023-10-16T20:07:35.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.281+0000 7f491d287640 1 -- 172.21.15.88:0/1949118813 learned_addr learned my addr 172.21.15.88:0/1949118813 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:35.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.281+0000 7f4917fff640 1 --2- 172.21.15.88:0/1949118813 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49180b4260 0x7f491815c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:35.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.281+0000 7f491ca86640 1 --2- 172.21.15.88:0/1949118813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49180b0da0 0x7f491815c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:35.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.285+0000 7f491d287640 1 -- 172.21.15.88:0/1949118813 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49180b4260 msgr2=0x7f491815c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:35.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.285+0000 7f491d287640 1 --2- 172.21.15.88:0/1949118813 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49180b4260 0x7f491815c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:35.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.285+0000 7f491d287640 1 -- 172.21.15.88:0/1949118813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49180b0da0 msgr2=0x7f491815c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:35.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.285+0000 7f491d287640 1 --2- 172.21.15.88:0/1949118813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49180b0da0 0x7f491815c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:35.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.285+0000 7f491d287640 1 -- 172.21.15.88:0/1949118813 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c0094d0 con 0x7f491815a1f0 2023-10-16T20:07:35.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.285+0000 7f491d287640 1 --2- 172.21.15.88:0/1949118813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f491815a1f0 0x7f4918157c50 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f4910007e40 tx=0x7f491000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:35.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.285+0000 7f49157fa640 1 -- 172.21.15.88:0/1949118813 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4910014620 con 0x7f491815a1f0 2023-10-16T20:07:35.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.285+0000 7f49157fa640 1 -- 172.21.15.88:0/1949118813 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491000ee90 con 0x7f491815a1f0 2023-10-16T20:07:35.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.285+0000 7f49157fa640 1 -- 172.21.15.88:0/1949118813 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49100149e0 con 0x7f491815a1f0 2023-10-16T20:07:35.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.285+0000 7f491ecea640 1 -- 172.21.15.88:0/1949118813 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4918158f70 con 0x7f491815a1f0 2023-10-16T20:07:35.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.285+0000 7f491ecea640 1 -- 172.21.15.88:0/1949118813 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4918149710 con 0x7f491815a1f0 2023-10-16T20:07:35.295 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.285+0000 7f491ecea640 1 -- 172.21.15.88:0/1949118813 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49180cbec0 con 0x7f491815a1f0 2023-10-16T20:07:35.295 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.285+0000 7f49157fa640 1 -- 172.21.15.88:0/1949118813 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4910014b40 con 0x7f491815a1f0 2023-10-16T20:07:35.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.285+0000 7f49157fa640 1 --2- 172.21.15.88:0/1949118813 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f48fc0664a0 0x7f48fc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:35.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.285+0000 7f49157fa640 1 -- 172.21.15.88:0/1949118813 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f4910084850 con 0x7f491815a1f0 2023-10-16T20:07:35.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.285+0000 7f491ca86640 1 --2- 172.21.15.88:0/1949118813 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f48fc0664a0 0x7f48fc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:35.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.289+0000 7f491ca86640 1 --2- 172.21.15.88:0/1949118813 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f48fc0664a0 0x7f48fc068960 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f490c0ae000 tx=0x7f490c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:35.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.289+0000 7f49157fa640 1 -- 172.21.15.88:0/1949118813 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f49180cbec0 con 0x7f491815a1f0 2023-10-16T20:07:35.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.441+0000 7f491ecea640 1 -- 172.21.15.88:0/1949118813 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"} v 0) v1 -- 0x7f49180cbec0 con 0x7f491815a1f0 2023-10-16T20:07:35.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.441+0000 7f49157fa640 1 -- 172.21.15.88:0/1949118813 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"}]=0 v58) v1 ==== 105+0+11 (secure 0 0 0) 0x7f49180cbec0 con 0x7f491815a1f0 2023-10-16T20:07:35.445 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2023-10-16T20:07:35.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.441+0000 7f491ecea640 1 -- 172.21.15.88:0/1949118813 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f48fc0664a0 msgr2=0x7f48fc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:35.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.441+0000 7f491ecea640 1 --2- 172.21.15.88:0/1949118813 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f48fc0664a0 0x7f48fc068960 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f490c0ae000 tx=0x7f490c008040 comp rx=0 tx=0).stop 2023-10-16T20:07:35.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.441+0000 7f491ecea640 1 -- 172.21.15.88:0/1949118813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f491815a1f0 msgr2=0x7f4918157c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:35.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.441+0000 7f491ecea640 1 --2- 172.21.15.88:0/1949118813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f491815a1f0 0x7f4918157c50 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f4910007e40 tx=0x7f491000e4b0 comp rx=0 tx=0).stop 2023-10-16T20:07:35.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.441+0000 7f491ecea640 1 -- 172.21.15.88:0/1949118813 shutdown_connections 2023-10-16T20:07:35.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.441+0000 7f491ecea640 1 --2- 172.21.15.88:0/1949118813 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f48fc0664a0 0x7f48fc068960 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:35.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.441+0000 7f491ecea640 1 --2- 172.21.15.88:0/1949118813 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49180b4260 0x7f491815c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:35.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.441+0000 7f491ecea640 1 --2- 172.21.15.88:0/1949118813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f491815a1f0 0x7f4918157c50 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:35.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.441+0000 7f491ecea640 1 --2- 172.21.15.88:0/1949118813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49180b0da0 0x7f491815c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:35.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.441+0000 7f491ecea640 1 -- 172.21.15.88:0/1949118813 >> 172.21.15.88:0/1949118813 conn(0x7f49180cb080 msgr2=0x7f49180729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:35.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.441+0000 7f491ecea640 1 -- 172.21.15.88:0/1949118813 shutdown_connections 2023-10-16T20:07:35.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.441+0000 7f491ecea640 1 -- 172.21.15.88:0/1949118813 wait complete. 2023-10-16T20:07:35.466 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T20:07:35.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.601+0000 7f5c19580640 1 Processor -- start 2023-10-16T20:07:35.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.601+0000 7f5c19580640 1 -- start start 2023-10-16T20:07:35.606 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.601+0000 7f5c19580640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5c141584a0 0x7f5c1415e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:35.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.601+0000 7f5c19580640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c140b21e0 0x7f5c1415b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:35.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.601+0000 7f5c19580640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5c1415bc20 0x7f5c140cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:35.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.601+0000 7f5c19580640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c1415f4d0 con 0x7f5c141584a0 2023-10-16T20:07:35.607 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.601+0000 7f5c19580640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c1415afb0 con 0x7f5c1415bc20 2023-10-16T20:07:35.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.601+0000 7f5c19580640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1415b250 con 0x7f5c140b21e0 2023-10-16T20:07:35.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.601+0000 7f5c12ffd640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5c141584a0 0x7f5c1415e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:35.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.601+0000 7f5c12ffd640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5c141584a0 0x7f5c1415e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47648/0 (socket says 172.21.15.88:47648) 2023-10-16T20:07:35.608 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.601+0000 7f5c12ffd640 1 -- 172.21.15.88:0/196542578 learned_addr learned my addr 172.21.15.88:0/196542578 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:35.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.601+0000 7f5c0a7fc640 1 --2- 172.21.15.88:0/196542578 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c140b21e0 0x7f5c1415b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:35.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.601+0000 7f5c137fe640 1 --2- 172.21.15.88:0/196542578 >> v2:172.21.15.94:3300/0 conn(0x7f5c1415bc20 0x7f5c140cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:35.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.601+0000 7f5c12ffd640 1 -- 172.21.15.88:0/196542578 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c140b21e0 msgr2=0x7f5c1415b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:35.609 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.601+0000 7f5c12ffd640 1 --2- 172.21.15.88:0/196542578 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c140b21e0 0x7f5c1415b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:35.610 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.601+0000 7f5c12ffd640 1 -- 172.21.15.88:0/196542578 >> v2:172.21.15.94:3300/0 conn(0x7f5c1415bc20 msgr2=0x7f5c140cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:35.610 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.601+0000 7f5c12ffd640 1 --2- 172.21.15.88:0/196542578 >> v2:172.21.15.94:3300/0 conn(0x7f5c1415bc20 0x7f5c140cc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:35.610 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.601+0000 7f5c12ffd640 1 -- 172.21.15.88:0/196542578 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1415ec30 con 0x7f5c141584a0 2023-10-16T20:07:35.610 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.601+0000 7f5c137fe640 1 --2- 172.21.15.88:0/196542578 >> v2:172.21.15.94:3300/0 conn(0x7f5c1415bc20 0x7f5c140cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:07:35.611 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.605+0000 7f5c12ffd640 1 --2- 172.21.15.88:0/196542578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c141584a0 0x7f5c1415e5a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5c140cf630 tx=0x7f5bfc01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db297a6f13a7e407 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:35.612 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.605+0000 7f5c127fc640 1 -- 172.21.15.88:0/196542578 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc0ac640 con 0x7f5c141584a0 2023-10-16T20:07:35.612 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.605+0000 7f5c127fc640 1 -- 172.21.15.88:0/196542578 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bfc0ac7a0 con 0x7f5c141584a0 2023-10-16T20:07:35.612 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.605+0000 7f5c127fc640 1 -- 172.21.15.88:0/196542578 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc0acb30 con 0x7f5c141584a0 2023-10-16T20:07:35.612 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.605+0000 7f5c19580640 1 -- 172.21.15.88:0/196542578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c141584a0 msgr2=0x7f5c1415e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:35.612 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.605+0000 7f5c19580640 1 --2- 172.21.15.88:0/196542578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c141584a0 0x7f5c1415e5a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5c140cf630 tx=0x7f5bfc01c440 comp rx=0 tx=0).stop 2023-10-16T20:07:35.613 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.605+0000 7f5c19580640 1 -- 172.21.15.88:0/196542578 shutdown_connections 2023-10-16T20:07:35.613 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.605+0000 7f5c19580640 1 --2- 172.21.15.88:0/196542578 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c140b21e0 0x7f5c1415b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:35.613 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.605+0000 7f5c19580640 1 --2- 172.21.15.88:0/196542578 >> v2:172.21.15.94:3300/0 conn(0x7f5c1415bc20 0x7f5c140cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:35.613 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.605+0000 7f5c19580640 1 --2- 172.21.15.88:0/196542578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c141584a0 0x7f5c1415e5a0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:35.613 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.605+0000 7f5c19580640 1 -- 172.21.15.88:0/196542578 >> 172.21.15.88:0/196542578 conn(0x7f5c140ce5e0 msgr2=0x7f5c140ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:35.614 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.605+0000 7f5c19580640 1 -- 172.21.15.88:0/196542578 shutdown_connections 2023-10-16T20:07:35.614 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.605+0000 7f5c19580640 1 -- 172.21.15.88:0/196542578 wait complete. 2023-10-16T20:07:35.616 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.609+0000 7f5c19580640 1 Processor -- start 2023-10-16T20:07:35.616 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.609+0000 7f5c19580640 1 -- start start 2023-10-16T20:07:35.616 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.609+0000 7f5c19580640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c140b21e0 0x7f5c14153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:35.616 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.609+0000 7f5c19580640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c141584a0 0x7f5c14070a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:35.617 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.609+0000 7f5c19580640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c14070fa0 0x7f5c141f5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:35.617 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.609+0000 7f5c19580640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c14154320 con 0x7f5c141584a0 2023-10-16T20:07:35.617 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.609+0000 7f5c19580640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c140705d0 con 0x7f5c140b21e0 2023-10-16T20:07:35.617 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.609+0000 7f5c19580640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c141f6e30 con 0x7f5c14070fa0 2023-10-16T20:07:35.617 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.609+0000 7f5c0a7fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c141584a0 0x7f5c14070a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:35.618 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.609+0000 7f5c0a7fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c141584a0 0x7f5c14070a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47656/0 (socket says 172.21.15.88:47656) 2023-10-16T20:07:35.618 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.609+0000 7f5c0a7fc640 1 -- 172.21.15.88:0/2120110571 learned_addr learned my addr 172.21.15.88:0/2120110571 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:35.618 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.609+0000 7f5c12ffd640 1 --2- 172.21.15.88:0/2120110571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c140b21e0 0x7f5c14153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:35.618 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.609+0000 7f5c137fe640 1 --2- 172.21.15.88:0/2120110571 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c14070fa0 0x7f5c141f5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:35.618 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.609+0000 7f5c0a7fc640 1 -- 172.21.15.88:0/2120110571 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c14070fa0 msgr2=0x7f5c141f5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:35.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.609+0000 7f5c0a7fc640 1 --2- 172.21.15.88:0/2120110571 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c14070fa0 0x7f5c141f5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:35.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.609+0000 7f5c0a7fc640 1 -- 172.21.15.88:0/2120110571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c140b21e0 msgr2=0x7f5c14153de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:35.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.609+0000 7f5c0a7fc640 1 --2- 172.21.15.88:0/2120110571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c140b21e0 0x7f5c14153de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:35.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.609+0000 7f5c0a7fc640 1 -- 172.21.15.88:0/2120110571 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bfc0094d0 con 0x7f5c141584a0 2023-10-16T20:07:35.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.609+0000 7f5c137fe640 1 --2- 172.21.15.88:0/2120110571 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c14070fa0 0x7f5c141f5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:07:35.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.609+0000 7f5c0a7fc640 1 --2- 172.21.15.88:0/2120110571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c141584a0 0x7f5c14070a60 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f5c0400c7f0 tx=0x7f5c0400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:35.620 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.609+0000 7f5c0bfff640 1 -- 172.21.15.88:0/2120110571 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c04004220 con 0x7f5c141584a0 2023-10-16T20:07:35.620 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.609+0000 7f5c0bfff640 1 -- 172.21.15.88:0/2120110571 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c04004380 con 0x7f5c141584a0 2023-10-16T20:07:35.620 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.609+0000 7f5c19580640 1 -- 172.21.15.88:0/2120110571 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c141f6870 con 0x7f5c141584a0 2023-10-16T20:07:35.620 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.609+0000 7f5c0bfff640 1 -- 172.21.15.88:0/2120110571 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c04003860 con 0x7f5c141584a0 2023-10-16T20:07:35.620 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.609+0000 7f5c19580640 1 -- 172.21.15.88:0/2120110571 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c14070260 con 0x7f5c141584a0 2023-10-16T20:07:35.621 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.613+0000 7f5c19580640 1 -- 172.21.15.88:0/2120110571 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c1406d180 con 0x7f5c141584a0 2023-10-16T20:07:35.621 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.613+0000 7f5c0bfff640 1 -- 172.21.15.88:0/2120110571 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5c04003bd0 con 0x7f5c141584a0 2023-10-16T20:07:35.622 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.613+0000 7f5c0bfff640 1 --2- 172.21.15.88:0/2120110571 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5bf40664a0 0x7f5bf4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:35.622 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.613+0000 7f5c0bfff640 1 -- 172.21.15.88:0/2120110571 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f5c04083f80 con 0x7f5c141584a0 2023-10-16T20:07:35.622 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.613+0000 7f5c12ffd640 1 --2- 172.21.15.88:0/2120110571 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5bf40664a0 0x7f5bf4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:35.622 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.613+0000 7f5c12ffd640 1 --2- 172.21.15.88:0/2120110571 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5bf40664a0 0x7f5bf4068960 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc0ae000 tx=0x7f5bfc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:35.623 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.617+0000 7f5c0bfff640 1 -- 172.21.15.88:0/2120110571 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5c04051230 con 0x7f5c141584a0 2023-10-16T20:07:35.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.777+0000 7f5c19580640 1 -- 172.21.15.88:0/2120110571 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5c1404e9c0 con 0x7f5c141584a0 2023-10-16T20:07:35.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.777+0000 7f5c0bfff640 1 -- 172.21.15.88:0/2120110571 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v58) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5c04054850 con 0x7f5c141584a0 2023-10-16T20:07:35.783 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2023-10-16T20:07:35.784 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.781+0000 7f5c19580640 1 -- 172.21.15.88:0/2120110571 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5bf40664a0 msgr2=0x7f5bf4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:35.785 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.781+0000 7f5c19580640 1 --2- 172.21.15.88:0/2120110571 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5bf40664a0 0x7f5bf4068960 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc0ae000 tx=0x7f5bfc008040 comp rx=0 tx=0).stop 2023-10-16T20:07:35.785 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.781+0000 7f5c19580640 1 -- 172.21.15.88:0/2120110571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c141584a0 msgr2=0x7f5c14070a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:35.785 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.781+0000 7f5c19580640 1 --2- 172.21.15.88:0/2120110571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c141584a0 0x7f5c14070a60 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f5c0400c7f0 tx=0x7f5c0400ccb0 comp rx=0 tx=0).stop 2023-10-16T20:07:35.785 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.781+0000 7f5c19580640 1 -- 172.21.15.88:0/2120110571 shutdown_connections 2023-10-16T20:07:35.785 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.781+0000 7f5c19580640 1 --2- 172.21.15.88:0/2120110571 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5bf40664a0 0x7f5bf4068960 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:35.786 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.781+0000 7f5c19580640 1 --2- 172.21.15.88:0/2120110571 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c14070fa0 0x7f5c141f5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:35.786 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.781+0000 7f5c19580640 1 --2- 172.21.15.88:0/2120110571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c141584a0 0x7f5c14070a60 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:35.786 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.781+0000 7f5c19580640 1 --2- 172.21.15.88:0/2120110571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c140b21e0 0x7f5c14153de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:35.786 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.781+0000 7f5c19580640 1 -- 172.21.15.88:0/2120110571 >> 172.21.15.88:0/2120110571 conn(0x7f5c140ce5e0 msgr2=0x7f5c141577e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:35.786 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.781+0000 7f5c19580640 1 -- 172.21.15.88:0/2120110571 shutdown_connections 2023-10-16T20:07:35.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.781+0000 7f5c19580640 1 -- 172.21.15.88:0/2120110571 wait complete. 2023-10-16T20:07:35.803 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:07:35.804 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:07:35.813 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:07:35.990 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.985+0000 7fb609165640 1 Processor -- start 2023-10-16T20:07:35.991 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.985+0000 7fb609165640 1 -- start start 2023-10-16T20:07:35.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.989+0000 7fb609165640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb6040b0cb0 0x7fb60406a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:35.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.989+0000 7fb609165640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb60406a830 con 0x7fb6041599f0 2023-10-16T20:07:35.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.989+0000 7fb609165640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6040b1260 con 0x7fb604069e90 2023-10-16T20:07:35.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.989+0000 7fb609165640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6040b3140 con 0x7fb6040b0cb0 2023-10-16T20:07:35.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.989+0000 7fb602d9d640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fb6041599f0 0x7fb6040b2a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:37382/0 (socket says 172.21.15.88:37382) 2023-10-16T20:07:35.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.989+0000 7fb602d9d640 1 -- 172.21.15.88:0/3465649493 learned_addr learned my addr 172.21.15.88:0/3465649493 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:35.995 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.989+0000 7fb60359e640 1 --2- 172.21.15.88:0/3465649493 >> v2:172.21.15.94:3300/0 conn(0x7fb6040b0cb0 0x7fb60406a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:35.995 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.989+0000 7fb60259c640 1 -- 172.21.15.88:0/3465649493 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2620171206 0 0) 0x7fb60406a830 con 0x7fb6041599f0 2023-10-16T20:07:35.995 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.989+0000 7fb60259c640 1 -- 172.21.15.88:0/3465649493 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5f0003530 con 0x7fb6041599f0 2023-10-16T20:07:35.995 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.989+0000 7fb60259c640 1 -- 172.21.15.88:0/3465649493 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2587377588 0 0) 0x7fb5f0003530 con 0x7fb6041599f0 2023-10-16T20:07:35.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.989+0000 7fb60259c640 1 -- 172.21.15.88:0/3465649493 >> v2:172.21.15.94:3300/0 conn(0x7fb6040b0cb0 msgr2=0x7fb60406a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:35.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.989+0000 7fb60259c640 1 --2- 172.21.15.88:0/3465649493 >> v2:172.21.15.94:3300/0 conn(0x7fb6040b0cb0 0x7fb60406a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:35.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.989+0000 7fb60259c640 1 -- 172.21.15.88:0/3465649493 >> v1:172.21.15.94:6789/0 conn(0x7fb604069e90 legacy=0x7fb6040b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:35.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.989+0000 7fb60259c640 1 -- 172.21.15.88:0/3465649493 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6040ae960 con 0x7fb6041599f0 2023-10-16T20:07:35.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.989+0000 7fb60259c640 1 -- 172.21.15.88:0/3465649493 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7fb5f4001680 con 0x7fb6041599f0 2023-10-16T20:07:35.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.989+0000 7fb60359e640 1 --2- 172.21.15.88:0/3465649493 >> v2:172.21.15.94:3300/0 conn(0x7fb6040b0cb0 0x7fb60406a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:07:35.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.989+0000 7fb60259c640 1 -- 172.21.15.88:0/3465649493 >> v1:172.21.15.88:6789/0 conn(0x7fb6041599f0 legacy=0x7fb6040b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:35.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.989+0000 7fb60259c640 1 --2- 172.21.15.88:0/3465649493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5f0003980 0x7fb5f0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:35.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.989+0000 7fb60259c640 1 --2- 172.21.15.88:0/3465649493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5f00047d0 0x7fb5f0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:35.998 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.989+0000 7fb60259c640 1 --2- 172.21.15.88:0/3465649493 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb5f0005190 0x7fb5f000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:35.998 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.989+0000 7fb60259c640 1 -- 172.21.15.88:0/3465649493 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6040b3140 con 0x7fb5f0003980 2023-10-16T20:07:35.998 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.989+0000 7fb60259c640 1 -- 172.21.15.88:0/3465649493 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f000c030 con 0x7fb5f00047d0 2023-10-16T20:07:35.998 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.989+0000 7fb60259c640 1 -- 172.21.15.88:0/3465649493 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f000c1f0 con 0x7fb5f0005190 2023-10-16T20:07:35.998 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.989+0000 7fb602d9d640 1 --2- 172.21.15.88:0/3465649493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5f0003980 0x7fb5f0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:35.999 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.989+0000 7fb5fa59c640 1 --2- 172.21.15.88:0/3465649493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5f00047d0 0x7fb5f0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:35.999 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.989+0000 7fb60359e640 1 --2- 172.21.15.88:0/3465649493 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb5f0005190 0x7fb5f000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:35.999 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.989+0000 7fb602d9d640 1 -- 172.21.15.88:0/3465649493 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb5f0005190 msgr2=0x7fb5f000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:35.999 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.989+0000 7fb602d9d640 1 --2- 172.21.15.88:0/3465649493 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb5f0005190 0x7fb5f000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:35.999 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.989+0000 7fb602d9d640 1 -- 172.21.15.88:0/3465649493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5f00047d0 msgr2=0x7fb5f0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:36.000 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.989+0000 7fb602d9d640 1 --2- 172.21.15.88:0/3465649493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5f00047d0 0x7fb5f0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.000 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.989+0000 7fb602d9d640 1 -- 172.21.15.88:0/3465649493 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f0003530 con 0x7fb5f0003980 2023-10-16T20:07:36.000 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.989+0000 7fb60359e640 1 -- 172.21.15.88:0/3465649493 reap_dead start 2023-10-16T20:07:36.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.993+0000 7fb602d9d640 1 --2- 172.21.15.88:0/3465649493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5f0003980 0x7fb5f0003da0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400c240 tx=0x7fb5f401cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5146f781cf2c6b8c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:36.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.993+0000 7fb60259c640 1 -- 172.21.15.88:0/3465649493 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4005ae0 con 0x7fb5f0003980 2023-10-16T20:07:36.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.993+0000 7fb60259c640 1 -- 172.21.15.88:0/3465649493 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f4005c40 con 0x7fb5f0003980 2023-10-16T20:07:36.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.993+0000 7fb60259c640 1 -- 172.21.15.88:0/3465649493 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f40072b0 con 0x7fb5f0003980 2023-10-16T20:07:36.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.993+0000 7fb609165640 1 -- 172.21.15.88:0/3465649493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5f0003980 msgr2=0x7fb5f0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:36.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.993+0000 7fb609165640 1 --2- 172.21.15.88:0/3465649493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5f0003980 0x7fb5f0003da0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400c240 tx=0x7fb5f401cb90 comp rx=0 tx=0).stop 2023-10-16T20:07:36.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.993+0000 7fb609165640 1 -- 172.21.15.88:0/3465649493 shutdown_connections 2023-10-16T20:07:36.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.993+0000 7fb609165640 1 --2- 172.21.15.88:0/3465649493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5f0003980 0x7fb5f0003da0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.993+0000 7fb609165640 1 -- 172.21.15.88:0/3465649493 >> 172.21.15.88:0/3465649493 conn(0x7fb6040ce5e0 msgr2=0x7fb6040ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:36.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.993+0000 7fb609165640 1 -- 172.21.15.88:0/3465649493 shutdown_connections 2023-10-16T20:07:36.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.993+0000 7fb609165640 1 -- 172.21.15.88:0/3465649493 wait complete. 2023-10-16T20:07:36.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.993+0000 7fb609165640 1 Processor -- start 2023-10-16T20:07:36.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.993+0000 7fb609165640 1 -- start start 2023-10-16T20:07:36.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.997+0000 7fb609165640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5f0003980 0x7fb604148940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:36.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.997+0000 7fb609165640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb604142ef0 0x7fb60415be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:36.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.997+0000 7fb609165640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb604143330 0x7fb60415c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:36.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.997+0000 7fb609165640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb604142a60 con 0x7fb604143330 2023-10-16T20:07:36.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.997+0000 7fb609165640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60415b260 con 0x7fb5f0003980 2023-10-16T20:07:36.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.997+0000 7fb609165640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb604141e50 con 0x7fb604142ef0 2023-10-16T20:07:36.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.997+0000 7fb60359e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb604143330 0x7fb60415c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:36.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.997+0000 7fb60359e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb604143330 0x7fb60415c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47668/0 (socket says 172.21.15.88:47668) 2023-10-16T20:07:36.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.997+0000 7fb602d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5f0003980 0x7fb604148940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:36.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.997+0000 7fb60359e640 1 -- 172.21.15.88:0/3090741618 learned_addr learned my addr 172.21.15.88:0/3090741618 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:36.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.997+0000 7fb5fa59c640 1 --2- 172.21.15.88:0/3090741618 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb604142ef0 0x7fb60415be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:36.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.997+0000 7fb60359e640 1 -- 172.21.15.88:0/3090741618 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb604142ef0 msgr2=0x7fb60415be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:36.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.997+0000 7fb60359e640 1 --2- 172.21.15.88:0/3090741618 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb604142ef0 0x7fb60415be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.997+0000 7fb60359e640 1 -- 172.21.15.88:0/3090741618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5f0003980 msgr2=0x7fb604148940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:36.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.997+0000 7fb60359e640 1 --2- 172.21.15.88:0/3090741618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5f0003980 0x7fb604148940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.997+0000 7fb60359e640 1 -- 172.21.15.88:0/3090741618 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb604142130 con 0x7fb604143330 2023-10-16T20:07:36.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.997+0000 7fb60359e640 1 --2- 172.21.15.88:0/3090741618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb604143330 0x7fb60415c360 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fb60406e5c0 tx=0x7fb5fc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:36.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.997+0000 7fb5fbfff640 1 -- 172.21.15.88:0/3090741618 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc00e070 con 0x7fb604143330 2023-10-16T20:07:36.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.997+0000 7fb5fbfff640 1 -- 172.21.15.88:0/3090741618 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5fc011e60 con 0x7fb604143330 2023-10-16T20:07:36.010 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.997+0000 7fb5fbfff640 1 -- 172.21.15.88:0/3090741618 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc0039a0 con 0x7fb604143330 2023-10-16T20:07:36.010 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.997+0000 7fb609165640 1 -- 172.21.15.88:0/3090741618 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb604142410 con 0x7fb604143330 2023-10-16T20:07:36.010 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.997+0000 7fb609165640 1 -- 172.21.15.88:0/3090741618 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb604148e80 con 0x7fb604143330 2023-10-16T20:07:36.010 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:35.997+0000 7fb609165640 1 -- 172.21.15.88:0/3090741618 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb604142bd0 con 0x7fb604143330 2023-10-16T20:07:36.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.001+0000 7fb5fbfff640 1 -- 172.21.15.88:0/3090741618 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb5fc003b00 con 0x7fb604143330 2023-10-16T20:07:36.012 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.001+0000 7fb5fbfff640 1 --2- 172.21.15.88:0/3090741618 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb5e40664a0 0x7fb5e4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:36.012 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.001+0000 7fb5fbfff640 1 -- 172.21.15.88:0/3090741618 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fb5fc084710 con 0x7fb604143330 2023-10-16T20:07:36.012 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.001+0000 7fb602d9d640 1 --2- 172.21.15.88:0/3090741618 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb5e40664a0 0x7fb5e4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:36.012 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.001+0000 7fb602d9d640 1 --2- 172.21.15.88:0/3090741618 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb5e40664a0 0x7fb5e4068960 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40ab000 tx=0x7fb5f40a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:36.013 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.005+0000 7fb5fbfff640 1 -- 172.21.15.88:0/3090741618 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb5fc0519b0 con 0x7fb604143330 2023-10-16T20:07:36.157 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.153+0000 7fb609165640 1 -- 172.21.15.88:0/3090741618 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb60404e9c0 con 0x7fb604143330 2023-10-16T20:07:36.158 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.153+0000 7fb5fbfff640 1 -- 172.21.15.88:0/3090741618 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16718 (secure 0 0 0) 0x7fb5fc082e60 con 0x7fb604143330 2023-10-16T20:07:36.159 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:07:36.159 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":58,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:07:27.206614+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T20:07:24.001121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"testcephfs1_data","create_time":"2023-10-16T20:07:24.636209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:07:36.161 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.157+0000 7fb609165640 1 -- 172.21.15.88:0/3090741618 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb5e40664a0 msgr2=0x7fb5e4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:36.161 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.157+0000 7fb609165640 1 --2- 172.21.15.88:0/3090741618 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb5e40664a0 0x7fb5e4068960 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40ab000 tx=0x7fb5f40a7000 comp rx=0 tx=0).stop 2023-10-16T20:07:36.161 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.157+0000 7fb609165640 1 -- 172.21.15.88:0/3090741618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb604143330 msgr2=0x7fb60415c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:36.161 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.157+0000 7fb609165640 1 --2- 172.21.15.88:0/3090741618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb604143330 0x7fb60415c360 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fb60406e5c0 tx=0x7fb5fc009f90 comp rx=0 tx=0).stop 2023-10-16T20:07:36.161 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.157+0000 7fb609165640 1 -- 172.21.15.88:0/3090741618 shutdown_connections 2023-10-16T20:07:36.162 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.157+0000 7fb609165640 1 --2- 172.21.15.88:0/3090741618 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb5e40664a0 0x7fb5e4068960 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.162 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.157+0000 7fb609165640 1 --2- 172.21.15.88:0/3090741618 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb604142ef0 0x7fb60415be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.162 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.157+0000 7fb609165640 1 --2- 172.21.15.88:0/3090741618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb604143330 0x7fb60415c360 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.162 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.157+0000 7fb609165640 1 --2- 172.21.15.88:0/3090741618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5f0003980 0x7fb604148940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.163 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.157+0000 7fb609165640 1 -- 172.21.15.88:0/3090741618 >> 172.21.15.88:0/3090741618 conn(0x7fb6040ce5e0 msgr2=0x7fb604071860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:36.163 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.157+0000 7fb609165640 1 -- 172.21.15.88:0/3090741618 shutdown_connections 2023-10-16T20:07:36.163 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.157+0000 7fb609165640 1 -- 172.21.15.88:0/3090741618 wait complete. 2023-10-16T20:07:36.180 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'testcephfs1_metadata', 'create_time': '2023-10-16T20:07:24.001121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'testcephfs1_data', 'create_time': '2023-10-16T20:07:24.636209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T20:07:36.181 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:07:36.324 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.321+0000 7fda61ed6640 1 Processor -- start 2023-10-16T20:07:36.325 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.321+0000 7fda61ed6640 1 -- start start 2023-10-16T20:07:36.326 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.321+0000 7fda61ed6640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fda5c15a820 0x7fda5c0cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:36.326 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.321+0000 7fda61ed6640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda5c15ac00 0x7fda5c0b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:36.326 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.321+0000 7fda61ed6640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fda5c0b2930 0x7fda5c06c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:36.327 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.321+0000 7fda61ed6640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda5c0b2da0 con 0x7fda5c0b2930 2023-10-16T20:07:36.327 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.321+0000 7fda61ed6640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda5c0ce4b0 con 0x7fda5c15a820 2023-10-16T20:07:36.327 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.321+0000 7fda61ed6640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c15e270 con 0x7fda5c15ac00 2023-10-16T20:07:36.327 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.321+0000 7fda5bfff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fda5c0b2930 0x7fda5c06c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:36.327 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.321+0000 7fda5bfff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fda5c0b2930 0x7fda5c06c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47672/0 (socket says 172.21.15.88:47672) 2023-10-16T20:07:36.327 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.321+0000 7fda5bfff640 1 -- 172.21.15.88:0/1909722592 learned_addr learned my addr 172.21.15.88:0/1909722592 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:36.328 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.321+0000 7fda5b7fe640 1 --2- 172.21.15.88:0/1909722592 >> v2:172.21.15.94:3300/0 conn(0x7fda5c15a820 0x7fda5c0cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:36.328 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.321+0000 7fda53fff640 1 --2- 172.21.15.88:0/1909722592 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda5c15ac00 0x7fda5c0b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:36.328 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.321+0000 7fda5bfff640 1 -- 172.21.15.88:0/1909722592 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda5c15ac00 msgr2=0x7fda5c0b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:36.329 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.321+0000 7fda5bfff640 1 --2- 172.21.15.88:0/1909722592 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda5c15ac00 0x7fda5c0b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.329 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.321+0000 7fda5bfff640 1 -- 172.21.15.88:0/1909722592 >> v2:172.21.15.94:3300/0 conn(0x7fda5c15a820 msgr2=0x7fda5c0cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:36.329 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.321+0000 7fda5bfff640 1 --2- 172.21.15.88:0/1909722592 >> v2:172.21.15.94:3300/0 conn(0x7fda5c15a820 0x7fda5c0cf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.329 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.321+0000 7fda5bfff640 1 -- 172.21.15.88:0/1909722592 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5c0ce700 con 0x7fda5c0b2930 2023-10-16T20:07:36.331 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.325+0000 7fda5bfff640 1 --2- 172.21.15.88:0/1909722592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda5c0b2930 0x7fda5c06c340 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0ce460 tx=0x7fda5401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6fd6b93336c602 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:36.331 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.325+0000 7fda5affd640 1 -- 172.21.15.88:0/1909722592 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda540a5820 con 0x7fda5c0b2930 2023-10-16T20:07:36.331 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.325+0000 7fda5affd640 1 -- 172.21.15.88:0/1909722592 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda540a5980 con 0x7fda5c0b2930 2023-10-16T20:07:36.331 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.325+0000 7fda5affd640 1 -- 172.21.15.88:0/1909722592 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda540a5d10 con 0x7fda5c0b2930 2023-10-16T20:07:36.331 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.325+0000 7fda61ed6640 1 -- 172.21.15.88:0/1909722592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda5c0b2930 msgr2=0x7fda5c06c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:36.332 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.325+0000 7fda61ed6640 1 --2- 172.21.15.88:0/1909722592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda5c0b2930 0x7fda5c06c340 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0ce460 tx=0x7fda5401c9a0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.332 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.325+0000 7fda61ed6640 1 -- 172.21.15.88:0/1909722592 shutdown_connections 2023-10-16T20:07:36.332 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.325+0000 7fda61ed6640 1 --2- 172.21.15.88:0/1909722592 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda5c15ac00 0x7fda5c0b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.332 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.325+0000 7fda61ed6640 1 --2- 172.21.15.88:0/1909722592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda5c0b2930 0x7fda5c06c340 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.332 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.325+0000 7fda61ed6640 1 --2- 172.21.15.88:0/1909722592 >> v2:172.21.15.94:3300/0 conn(0x7fda5c15a820 0x7fda5c0cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.332 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.325+0000 7fda61ed6640 1 -- 172.21.15.88:0/1909722592 >> 172.21.15.88:0/1909722592 conn(0x7fda5c0cd3f0 msgr2=0x7fda5c0cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:36.333 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.325+0000 7fda61ed6640 1 -- 172.21.15.88:0/1909722592 shutdown_connections 2023-10-16T20:07:36.333 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.325+0000 7fda61ed6640 1 -- 172.21.15.88:0/1909722592 wait complete. 2023-10-16T20:07:36.335 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.329+0000 7fda61ed6640 1 Processor -- start 2023-10-16T20:07:36.335 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.329+0000 7fda61ed6640 1 -- start start 2023-10-16T20:07:36.335 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.329+0000 7fda61ed6640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda5c0b2930 0x7fda5c1530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:36.336 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.329+0000 7fda61ed6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda5c15a820 0x7fda5c14f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:36.336 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.329+0000 7fda61ed6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda5c15ac00 0x7fda5c15edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:36.336 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.329+0000 7fda61ed6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c15f620 con 0x7fda5c15a820 2023-10-16T20:07:36.336 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.329+0000 7fda61ed6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c15f900 con 0x7fda5c15ac00 2023-10-16T20:07:36.336 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.329+0000 7fda61ed6640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c15fbe0 con 0x7fda5c0b2930 2023-10-16T20:07:36.337 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.329+0000 7fda53fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda5c15a820 0x7fda5c14f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:36.337 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.329+0000 7fda53fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda5c15a820 0x7fda5c14f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47684/0 (socket says 172.21.15.88:47684) 2023-10-16T20:07:36.337 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.329+0000 7fda53fff640 1 -- 172.21.15.88:0/1402881003 learned_addr learned my addr 172.21.15.88:0/1402881003 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:36.337 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.329+0000 7fda5b7fe640 1 --2- 172.21.15.88:0/1402881003 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda5c0b2930 0x7fda5c1530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:36.337 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.329+0000 7fda5bfff640 1 --2- 172.21.15.88:0/1402881003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda5c15ac00 0x7fda5c15edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:36.338 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.329+0000 7fda53fff640 1 -- 172.21.15.88:0/1402881003 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda5c0b2930 msgr2=0x7fda5c1530f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:36.338 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.329+0000 7fda53fff640 1 --2- 172.21.15.88:0/1402881003 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda5c0b2930 0x7fda5c1530f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.338 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.329+0000 7fda53fff640 1 -- 172.21.15.88:0/1402881003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda5c15ac00 msgr2=0x7fda5c15edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:36.338 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.329+0000 7fda53fff640 1 --2- 172.21.15.88:0/1402881003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda5c15ac00 0x7fda5c15edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.338 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.329+0000 7fda53fff640 1 -- 172.21.15.88:0/1402881003 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda54009b30 con 0x7fda5c15a820 2023-10-16T20:07:36.339 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.329+0000 7fda5bfff640 1 --2- 172.21.15.88:0/1402881003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda5c15ac00 0x7fda5c15edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:07:36.339 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.329+0000 7fda5b7fe640 1 --2- 172.21.15.88:0/1402881003 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda5c0b2930 0x7fda5c1530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:07:36.339 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.329+0000 7fda53fff640 1 --2- 172.21.15.88:0/1402881003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda5c15a820 0x7fda5c14f3c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fda4400e930 tx=0x7fda4400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:36.339 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.329+0000 7fda58ff9640 1 -- 172.21.15.88:0/1402881003 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda44009690 con 0x7fda5c15a820 2023-10-16T20:07:36.339 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.329+0000 7fda58ff9640 1 -- 172.21.15.88:0/1402881003 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4400ca00 con 0x7fda5c15a820 2023-10-16T20:07:36.340 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.329+0000 7fda61ed6640 1 -- 172.21.15.88:0/1402881003 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda5c15fec0 con 0x7fda5c15a820 2023-10-16T20:07:36.340 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.329+0000 7fda58ff9640 1 -- 172.21.15.88:0/1402881003 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4400cd70 con 0x7fda5c15a820 2023-10-16T20:07:36.340 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.329+0000 7fda61ed6640 1 -- 172.21.15.88:0/1402881003 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda5c15f310 con 0x7fda5c15a820 2023-10-16T20:07:36.340 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.329+0000 7fda61ed6640 1 -- 172.21.15.88:0/1402881003 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda5c04e9c0 con 0x7fda5c15a820 2023-10-16T20:07:36.341 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.333+0000 7fda58ff9640 1 -- 172.21.15.88:0/1402881003 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fda44013470 con 0x7fda5c15a820 2023-10-16T20:07:36.341 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.333+0000 7fda58ff9640 1 --2- 172.21.15.88:0/1402881003 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fda3c0664a0 0x7fda3c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:36.341 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.333+0000 7fda58ff9640 1 -- 172.21.15.88:0/1402881003 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fda44084350 con 0x7fda5c15a820 2023-10-16T20:07:36.342 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.333+0000 7fda5b7fe640 1 --2- 172.21.15.88:0/1402881003 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fda3c0664a0 0x7fda3c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:36.342 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.333+0000 7fda5b7fe640 1 --2- 172.21.15.88:0/1402881003 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fda3c0664a0 0x7fda3c068960 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fda5c1f9220 tx=0x7fda4c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:36.342 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.337+0000 7fda58ff9640 1 -- 172.21.15.88:0/1402881003 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fda44051690 con 0x7fda5c15a820 2023-10-16T20:07:36.497 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.493+0000 7fda61ed6640 1 -- 172.21.15.88:0/1402881003 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fda5c0afcc0 con 0x7fda5c15a820 2023-10-16T20:07:36.498 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.493+0000 7fda58ff9640 1 -- 172.21.15.88:0/1402881003 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7fda44054cb0 con 0x7fda5c15a820 2023-10-16T20:07:36.498 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:07:36.500 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.497+0000 7fda61ed6640 1 -- 172.21.15.88:0/1402881003 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fda3c0664a0 msgr2=0x7fda3c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:36.500 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.497+0000 7fda61ed6640 1 --2- 172.21.15.88:0/1402881003 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fda3c0664a0 0x7fda3c068960 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fda5c1f9220 tx=0x7fda4c008040 comp rx=0 tx=0).stop 2023-10-16T20:07:36.501 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.497+0000 7fda61ed6640 1 -- 172.21.15.88:0/1402881003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda5c15a820 msgr2=0x7fda5c14f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:36.501 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.497+0000 7fda61ed6640 1 --2- 172.21.15.88:0/1402881003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda5c15a820 0x7fda5c14f3c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fda4400e930 tx=0x7fda4400edf0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.501 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.497+0000 7fda61ed6640 1 -- 172.21.15.88:0/1402881003 shutdown_connections 2023-10-16T20:07:36.501 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.497+0000 7fda61ed6640 1 --2- 172.21.15.88:0/1402881003 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fda3c0664a0 0x7fda3c068960 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.501 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.497+0000 7fda61ed6640 1 --2- 172.21.15.88:0/1402881003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda5c15ac00 0x7fda5c15edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.502 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.497+0000 7fda61ed6640 1 --2- 172.21.15.88:0/1402881003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda5c15a820 0x7fda5c14f3c0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.502 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.497+0000 7fda61ed6640 1 --2- 172.21.15.88:0/1402881003 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda5c0b2930 0x7fda5c1530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.502 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.497+0000 7fda61ed6640 1 -- 172.21.15.88:0/1402881003 >> 172.21.15.88:0/1402881003 conn(0x7fda5c0cd3f0 msgr2=0x7fda5c06bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:36.502 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.497+0000 7fda61ed6640 1 -- 172.21.15.88:0/1402881003 shutdown_connections 2023-10-16T20:07:36.502 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.497+0000 7fda61ed6640 1 -- 172.21.15.88:0/1402881003 wait complete. 2023-10-16T20:07:36.519 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_metadata pg_num 2023-10-16T20:07:36.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.657+0000 7fb91b380640 1 Processor -- start 2023-10-16T20:07:36.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.657+0000 7fb91b380640 1 -- start start 2023-10-16T20:07:36.661 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.657+0000 7fb91b380640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb91415cf90 0x7fb9140ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:36.661 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.657+0000 7fb91b380640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb914161540 0x7fb9140cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:36.662 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.657+0000 7fb91b380640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb914071360 0x7fb9140717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:36.662 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.657+0000 7fb91b380640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb914071d30 con 0x7fb914071360 2023-10-16T20:07:36.662 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.657+0000 7fb91b380640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9140ca5f0 con 0x7fb914161540 2023-10-16T20:07:36.662 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.657+0000 7fb91b380640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9140ca840 con 0x7fb91415cf90 2023-10-16T20:07:36.663 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.657+0000 7fb91991d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb914071360 0x7fb9140717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:36.664 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.657+0000 7fb91991d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb914071360 0x7fb9140717f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47700/0 (socket says 172.21.15.88:47700) 2023-10-16T20:07:36.664 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.657+0000 7fb91991d640 1 -- 172.21.15.88:0/3876264927 learned_addr learned my addr 172.21.15.88:0/3876264927 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:36.664 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.657+0000 7fb91891b640 1 --2- 172.21.15.88:0/3876264927 >> v2:172.21.15.94:3300/0 conn(0x7fb914161540 0x7fb9140cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:36.664 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.657+0000 7fb91911c640 1 --2- 172.21.15.88:0/3876264927 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb91415cf90 0x7fb9140ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:36.664 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.657+0000 7fb91991d640 1 -- 172.21.15.88:0/3876264927 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb91415cf90 msgr2=0x7fb9140ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:36.665 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.657+0000 7fb91991d640 1 --2- 172.21.15.88:0/3876264927 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb91415cf90 0x7fb9140ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.665 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.657+0000 7fb91991d640 1 -- 172.21.15.88:0/3876264927 >> v2:172.21.15.94:3300/0 conn(0x7fb914161540 msgr2=0x7fb9140cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:36.665 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.657+0000 7fb91991d640 1 --2- 172.21.15.88:0/3876264927 >> v2:172.21.15.94:3300/0 conn(0x7fb914161540 0x7fb9140cb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.665 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.657+0000 7fb91991d640 1 -- 172.21.15.88:0/3876264927 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9140cab20 con 0x7fb914071360 2023-10-16T20:07:36.666 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.661+0000 7fb91991d640 1 --2- 172.21.15.88:0/3876264927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb914071360 0x7fb9140717f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb914070a90 tx=0x7fb90c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6643b96db778534b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:36.667 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.661+0000 7fb90bfff640 1 -- 172.21.15.88:0/3876264927 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90c0a5820 con 0x7fb914071360 2023-10-16T20:07:36.667 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.661+0000 7fb90bfff640 1 -- 172.21.15.88:0/3876264927 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb90c0a5980 con 0x7fb914071360 2023-10-16T20:07:36.667 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.661+0000 7fb90bfff640 1 -- 172.21.15.88:0/3876264927 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90c0a5d10 con 0x7fb914071360 2023-10-16T20:07:36.667 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.661+0000 7fb91b380640 1 -- 172.21.15.88:0/3876264927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb914071360 msgr2=0x7fb9140717f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:36.667 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.661+0000 7fb91b380640 1 --2- 172.21.15.88:0/3876264927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb914071360 0x7fb9140717f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb914070a90 tx=0x7fb90c01c9a0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.668 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.661+0000 7fb91b380640 1 -- 172.21.15.88:0/3876264927 shutdown_connections 2023-10-16T20:07:36.668 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.661+0000 7fb91b380640 1 --2- 172.21.15.88:0/3876264927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb914071360 0x7fb9140717f0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.668 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.661+0000 7fb91b380640 1 --2- 172.21.15.88:0/3876264927 >> v2:172.21.15.94:3300/0 conn(0x7fb914161540 0x7fb9140cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.668 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.661+0000 7fb91b380640 1 --2- 172.21.15.88:0/3876264927 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb91415cf90 0x7fb9140ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.668 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.661+0000 7fb91b380640 1 -- 172.21.15.88:0/3876264927 >> 172.21.15.88:0/3876264927 conn(0x7fb9140cf710 msgr2=0x7fb9140cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:36.668 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.661+0000 7fb91b380640 1 -- 172.21.15.88:0/3876264927 shutdown_connections 2023-10-16T20:07:36.669 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.661+0000 7fb91b380640 1 -- 172.21.15.88:0/3876264927 wait complete. 2023-10-16T20:07:36.671 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.665+0000 7fb91b380640 1 Processor -- start 2023-10-16T20:07:36.671 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.665+0000 7fb91b380640 1 -- start start 2023-10-16T20:07:36.671 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.665+0000 7fb91b380640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb914071360 0x7fb91415d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:36.671 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.665+0000 7fb91b380640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb91415cf90 0x7fb9140ae430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:36.671 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.665+0000 7fb91b380640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb914161540 0x7fb9140c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:36.672 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.665+0000 7fb91b380640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9141f5880 con 0x7fb914071360 2023-10-16T20:07:36.672 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.665+0000 7fb91b380640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9140ae970 con 0x7fb91415cf90 2023-10-16T20:07:36.672 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.665+0000 7fb91b380640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb91415db60 con 0x7fb914161540 2023-10-16T20:07:36.672 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.665+0000 7fb91911c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb914071360 0x7fb91415d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:36.672 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.665+0000 7fb91911c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb914071360 0x7fb91415d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47706/0 (socket says 172.21.15.88:47706) 2023-10-16T20:07:36.673 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.665+0000 7fb91911c640 1 -- 172.21.15.88:0/3747492378 learned_addr learned my addr 172.21.15.88:0/3747492378 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:36.673 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.665+0000 7fb91891b640 1 --2- 172.21.15.88:0/3747492378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb91415cf90 0x7fb9140ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:36.673 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.665+0000 7fb91991d640 1 --2- 172.21.15.88:0/3747492378 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb914161540 0x7fb9140c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:36.673 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.665+0000 7fb91911c640 1 -- 172.21.15.88:0/3747492378 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb914161540 msgr2=0x7fb9140c7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:36.673 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.665+0000 7fb91911c640 1 --2- 172.21.15.88:0/3747492378 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb914161540 0x7fb9140c7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.674 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.665+0000 7fb91911c640 1 -- 172.21.15.88:0/3747492378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb91415cf90 msgr2=0x7fb9140ae430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:36.674 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.665+0000 7fb91911c640 1 --2- 172.21.15.88:0/3747492378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb91415cf90 0x7fb9140ae430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.674 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.665+0000 7fb91911c640 1 -- 172.21.15.88:0/3747492378 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb90c009b30 con 0x7fb914071360 2023-10-16T20:07:36.674 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.665+0000 7fb91911c640 1 --2- 172.21.15.88:0/3747492378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb914071360 0x7fb91415d4f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb91000e870 tx=0x7fb91000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:36.674 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.665+0000 7fb909ffb640 1 -- 172.21.15.88:0/3747492378 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91000ee60 con 0x7fb914071360 2023-10-16T20:07:36.675 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.665+0000 7fb909ffb640 1 -- 172.21.15.88:0/3747492378 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb91000c940 con 0x7fb914071360 2023-10-16T20:07:36.675 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.665+0000 7fb909ffb640 1 -- 172.21.15.88:0/3747492378 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91000ccf0 con 0x7fb914071360 2023-10-16T20:07:36.675 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.665+0000 7fb91b380640 1 -- 172.21.15.88:0/3747492378 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb91415de40 con 0x7fb914071360 2023-10-16T20:07:36.675 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.665+0000 7fb91b380640 1 -- 172.21.15.88:0/3747492378 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb91415e120 con 0x7fb914071360 2023-10-16T20:07:36.676 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.669+0000 7fb91b380640 1 -- 172.21.15.88:0/3747492378 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb91404e9c0 con 0x7fb914071360 2023-10-16T20:07:36.676 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.669+0000 7fb909ffb640 1 -- 172.21.15.88:0/3747492378 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb910013470 con 0x7fb914071360 2023-10-16T20:07:36.677 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.669+0000 7fb909ffb640 1 --2- 172.21.15.88:0/3747492378 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb8f80664d0 0x7fb8f8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:36.677 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.669+0000 7fb909ffb640 1 -- 172.21.15.88:0/3747492378 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fb910084060 con 0x7fb914071360 2023-10-16T20:07:36.677 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.669+0000 7fb91891b640 1 --2- 172.21.15.88:0/3747492378 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb8f80664d0 0x7fb8f8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:36.677 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.669+0000 7fb91891b640 1 --2- 172.21.15.88:0/3747492378 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb8f80664d0 0x7fb8f8068990 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb9140c6360 tx=0x7fb904009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:36.678 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.673+0000 7fb909ffb640 1 -- 172.21.15.88:0/3747492378 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb9100b5090 con 0x7fb914071360 2023-10-16T20:07:36.825 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.821+0000 7fb91b380640 1 -- 172.21.15.88:0/3747492378 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb9140c6520 con 0x7fb914071360 2023-10-16T20:07:36.826 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.821+0000 7fb909ffb640 1 -- 172.21.15.88:0/3747492378 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"}]=0 v58) v1 ==== 109+0+11 (secure 0 0 0) 0x7fb910051390 con 0x7fb914071360 2023-10-16T20:07:36.826 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:07:36.828 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.825+0000 7fb91b380640 1 -- 172.21.15.88:0/3747492378 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb8f80664d0 msgr2=0x7fb8f8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:36.828 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.825+0000 7fb91b380640 1 --2- 172.21.15.88:0/3747492378 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb8f80664d0 0x7fb8f8068990 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb9140c6360 tx=0x7fb904009190 comp rx=0 tx=0).stop 2023-10-16T20:07:36.828 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.825+0000 7fb91b380640 1 -- 172.21.15.88:0/3747492378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb914071360 msgr2=0x7fb91415d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:36.828 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.825+0000 7fb91b380640 1 --2- 172.21.15.88:0/3747492378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb914071360 0x7fb91415d4f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb91000e870 tx=0x7fb91000ed30 comp rx=0 tx=0).stop 2023-10-16T20:07:36.829 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.825+0000 7fb91b380640 1 -- 172.21.15.88:0/3747492378 shutdown_connections 2023-10-16T20:07:36.829 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.825+0000 7fb91b380640 1 --2- 172.21.15.88:0/3747492378 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb8f80664d0 0x7fb8f8068990 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.829 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.825+0000 7fb91b380640 1 --2- 172.21.15.88:0/3747492378 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb914161540 0x7fb9140c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.829 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.825+0000 7fb91b380640 1 --2- 172.21.15.88:0/3747492378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb91415cf90 0x7fb9140ae430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.829 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.825+0000 7fb91b380640 1 --2- 172.21.15.88:0/3747492378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb914071360 0x7fb91415d4f0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.830 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.825+0000 7fb91b380640 1 -- 172.21.15.88:0/3747492378 >> 172.21.15.88:0/3747492378 conn(0x7fb9140cf710 msgr2=0x7fb9140cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:36.830 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.825+0000 7fb91b380640 1 -- 172.21.15.88:0/3747492378 shutdown_connections 2023-10-16T20:07:36.830 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.825+0000 7fb91b380640 1 -- 172.21.15.88:0/3747492378 wait complete. 2023-10-16T20:07:36.847 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_data pg_num 2023-10-16T20:07:36.988 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.985+0000 7f58c2254640 1 Processor -- start 2023-10-16T20:07:36.988 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.985+0000 7f58c2254640 1 -- start start 2023-10-16T20:07:36.989 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.985+0000 7f58c2254640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f58bc15a820 0x7f58bc0cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:36.990 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.985+0000 7f58c2254640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f58bc0b2b00 0x7f58bc06c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:36.990 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.985+0000 7f58c2254640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58bc0ce4b0 con 0x7f58bc15ac00 2023-10-16T20:07:36.990 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.985+0000 7f58c2254640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58bc15e270 con 0x7f58bc0b2b00 2023-10-16T20:07:36.990 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.985+0000 7f58c2254640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58bc0ce700 con 0x7f58bc15a820 2023-10-16T20:07:36.991 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.985+0000 7f58bbfff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f58bc0b2b00 0x7f58bc06c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:36.991 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.985+0000 7f58bbfff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f58bc0b2b00 0x7f58bc06c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47710/0 (socket says 172.21.15.88:47710) 2023-10-16T20:07:36.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.985+0000 7f58bbfff640 1 -- 172.21.15.88:0/3142133858 learned_addr learned my addr 172.21.15.88:0/3142133858 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:36.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.985+0000 7f58bb7fe640 1 --2- 172.21.15.88:0/3142133858 >> v2:172.21.15.94:3300/0 conn(0x7f58bc15a820 0x7f58bc0cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:36.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.985+0000 7f58bbfff640 1 -- 172.21.15.88:0/3142133858 >> v2:172.21.15.94:3300/0 conn(0x7f58bc15a820 msgr2=0x7f58bc0cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:36.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.985+0000 7f58bbfff640 1 --2- 172.21.15.88:0/3142133858 >> v2:172.21.15.94:3300/0 conn(0x7f58bc15a820 0x7f58bc0cf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.994 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.985+0000 7f58bbfff640 1 -- 172.21.15.88:0/3142133858 >> v1:172.21.15.94:6789/0 conn(0x7f58bc15ac00 legacy=0x7f58bc0b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:36.994 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.985+0000 7f58bbfff640 1 -- 172.21.15.88:0/3142133858 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58bc0ce9e0 con 0x7f58bc0b2b00 2023-10-16T20:07:36.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.989+0000 7f58bbfff640 1 --2- 172.21.15.88:0/3142133858 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58bc0b2b00 0x7f58bc06c340 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0ce460 tx=0x7f58b401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e7996c42785c34 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:36.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.989+0000 7f58baffd640 1 -- 172.21.15.88:0/3142133858 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc06cea0 con 0x7f58bc0b2b00 2023-10-16T20:07:36.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.989+0000 7f58baffd640 1 -- 172.21.15.88:0/3142133858 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b40a5880 con 0x7f58bc0b2b00 2023-10-16T20:07:36.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.989+0000 7f58baffd640 1 -- 172.21.15.88:0/3142133858 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b40a5c10 con 0x7f58bc0b2b00 2023-10-16T20:07:36.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.989+0000 7f58c2254640 1 -- 172.21.15.88:0/3142133858 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58bc0b2b00 msgr2=0x7f58bc06c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:36.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.989+0000 7f58c2254640 1 --2- 172.21.15.88:0/3142133858 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58bc0b2b00 0x7f58bc06c340 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0ce460 tx=0x7f58b401c9a0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.989+0000 7f58c2254640 1 -- 172.21.15.88:0/3142133858 shutdown_connections 2023-10-16T20:07:36.998 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.989+0000 7f58c2254640 1 --2- 172.21.15.88:0/3142133858 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58bc0b2b00 0x7f58bc06c340 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.998 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.989+0000 7f58c2254640 1 --2- 172.21.15.88:0/3142133858 >> v2:172.21.15.94:3300/0 conn(0x7f58bc15a820 0x7f58bc0cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:36.998 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.989+0000 7f58c2254640 1 -- 172.21.15.88:0/3142133858 >> 172.21.15.88:0/3142133858 conn(0x7f58bc0cd3f0 msgr2=0x7f58bc0cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:36.998 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.989+0000 7f58c2254640 1 -- 172.21.15.88:0/3142133858 shutdown_connections 2023-10-16T20:07:36.998 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.989+0000 7f58c2254640 1 -- 172.21.15.88:0/3142133858 wait complete. 2023-10-16T20:07:36.999 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.989+0000 7f58c2254640 1 Processor -- start 2023-10-16T20:07:36.999 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.989+0000 7f58c2254640 1 -- start start 2023-10-16T20:07:37.001 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.993+0000 7f58c2254640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58bc0b2b00 0x7f58bc15a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:37.001 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.993+0000 7f58c2254640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f58bc15a820 0x7f58bc153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:37.002 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.993+0000 7f58c2254640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58bc15ac00 0x7f58bc1f8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:37.002 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.993+0000 7f58c2254640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc0b1dd0 con 0x7f58bc0b2b00 2023-10-16T20:07:37.002 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.993+0000 7f58c2254640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc159570 con 0x7f58bc15ac00 2023-10-16T20:07:37.002 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.993+0000 7f58c2254640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc0b1fb0 con 0x7f58bc15a820 2023-10-16T20:07:37.002 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.993+0000 7f58bb7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58bc0b2b00 0x7f58bc15a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:37.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.993+0000 7f58bb7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58bc0b2b00 0x7f58bc15a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47724/0 (socket says 172.21.15.88:47724) 2023-10-16T20:07:37.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.993+0000 7f58bb7fe640 1 -- 172.21.15.88:0/1989991219 learned_addr learned my addr 172.21.15.88:0/1989991219 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:37.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.993+0000 7f58bbfff640 1 --2- 172.21.15.88:0/1989991219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58bc15ac00 0x7f58bc1f8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:37.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.993+0000 7f58b3fff640 1 --2- 172.21.15.88:0/1989991219 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f58bc15a820 0x7f58bc153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:37.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.993+0000 7f58bb7fe640 1 -- 172.21.15.88:0/1989991219 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f58bc15a820 msgr2=0x7f58bc153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:37.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.993+0000 7f58bb7fe640 1 --2- 172.21.15.88:0/1989991219 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f58bc15a820 0x7f58bc153d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:37.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.993+0000 7f58bb7fe640 1 -- 172.21.15.88:0/1989991219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58bc15ac00 msgr2=0x7f58bc1f8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:37.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.993+0000 7f58bb7fe640 1 --2- 172.21.15.88:0/1989991219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58bc15ac00 0x7f58bc1f8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:37.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.993+0000 7f58bb7fe640 1 -- 172.21.15.88:0/1989991219 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b4009b30 con 0x7f58bc0b2b00 2023-10-16T20:07:37.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.993+0000 7f58bb7fe640 1 --2- 172.21.15.88:0/1989991219 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58bc0b2b00 0x7f58bc15a030 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00e930 tx=0x7f58ac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:37.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.993+0000 7f58b8ff9640 1 -- 172.21.15.88:0/1989991219 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac009690 con 0x7f58bc0b2b00 2023-10-16T20:07:37.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.993+0000 7f58b8ff9640 1 -- 172.21.15.88:0/1989991219 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac00ca00 con 0x7f58bc0b2b00 2023-10-16T20:07:37.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.993+0000 7f58c2254640 1 -- 172.21.15.88:0/1989991219 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58bc0b2290 con 0x7f58bc0b2b00 2023-10-16T20:07:37.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.993+0000 7f58b8ff9640 1 -- 172.21.15.88:0/1989991219 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac00cd70 con 0x7f58bc0b2b00 2023-10-16T20:07:37.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.993+0000 7f58c2254640 1 -- 172.21.15.88:0/1989991219 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58bc154250 con 0x7f58bc0b2b00 2023-10-16T20:07:37.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.993+0000 7f58c2254640 1 -- 172.21.15.88:0/1989991219 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58bc04e9c0 con 0x7f58bc0b2b00 2023-10-16T20:07:37.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.993+0000 7f58b8ff9640 1 -- 172.21.15.88:0/1989991219 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f58ac013470 con 0x7f58bc0b2b00 2023-10-16T20:07:37.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.997+0000 7f58b8ff9640 1 --2- 172.21.15.88:0/1989991219 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f589c0664a0 0x7f589c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:37.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.997+0000 7f58b8ff9640 1 -- 172.21.15.88:0/1989991219 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f58ac084220 con 0x7f58bc0b2b00 2023-10-16T20:07:37.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.997+0000 7f58b3fff640 1 --2- 172.21.15.88:0/1989991219 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f589c0664a0 0x7f589c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:37.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:36.997+0000 7f58b3fff640 1 --2- 172.21.15.88:0/1989991219 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f589c0664a0 0x7f589c068960 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f58bc15ba60 tx=0x7f58a4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:37.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.001+0000 7f58b8ff9640 1 -- 172.21.15.88:0/1989991219 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f58ac051690 con 0x7f58bc0b2b00 2023-10-16T20:07:37.151 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.149+0000 7f58c2254640 1 -- 172.21.15.88:0/1989991219 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"} v 0) v1 -- 0x7f58bc06cf90 con 0x7f58bc0b2b00 2023-10-16T20:07:37.152 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.149+0000 7f58b8ff9640 1 -- 172.21.15.88:0/1989991219 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"}]=0 v58) v1 ==== 105+0+11 (secure 0 0 0) 0x7f58ac054cb0 con 0x7f58bc0b2b00 2023-10-16T20:07:37.152 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2023-10-16T20:07:37.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.149+0000 7f58c2254640 1 -- 172.21.15.88:0/1989991219 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f589c0664a0 msgr2=0x7f589c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:37.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.149+0000 7f58c2254640 1 --2- 172.21.15.88:0/1989991219 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f589c0664a0 0x7f589c068960 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f58bc15ba60 tx=0x7f58a4009040 comp rx=0 tx=0).stop 2023-10-16T20:07:37.154 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.149+0000 7f58c2254640 1 -- 172.21.15.88:0/1989991219 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58bc0b2b00 msgr2=0x7f58bc15a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:37.155 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.149+0000 7f58c2254640 1 --2- 172.21.15.88:0/1989991219 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58bc0b2b00 0x7f58bc15a030 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00e930 tx=0x7f58ac00edf0 comp rx=0 tx=0).stop 2023-10-16T20:07:37.155 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.149+0000 7f58c2254640 1 -- 172.21.15.88:0/1989991219 shutdown_connections 2023-10-16T20:07:37.155 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.149+0000 7f58c2254640 1 --2- 172.21.15.88:0/1989991219 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f589c0664a0 0x7f589c068960 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:37.155 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.149+0000 7f58c2254640 1 --2- 172.21.15.88:0/1989991219 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f58bc15a820 0x7f58bc153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:37.155 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.149+0000 7f58c2254640 1 --2- 172.21.15.88:0/1989991219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58bc15ac00 0x7f58bc1f8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:37.156 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.149+0000 7f58c2254640 1 --2- 172.21.15.88:0/1989991219 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58bc0b2b00 0x7f58bc15a030 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:37.156 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.149+0000 7f58c2254640 1 -- 172.21.15.88:0/1989991219 >> 172.21.15.88:0/1989991219 conn(0x7f58bc0cd3f0 msgr2=0x7f58bc1600a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:37.156 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.149+0000 7f58c2254640 1 -- 172.21.15.88:0/1989991219 shutdown_connections 2023-10-16T20:07:37.156 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.149+0000 7f58c2254640 1 -- 172.21.15.88:0/1989991219 wait complete. 2023-10-16T20:07:37.173 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T20:07:37.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.309+0000 7f12080a9640 1 Processor -- start 2023-10-16T20:07:37.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.309+0000 7f12080a9640 1 -- start start 2023-10-16T20:07:37.314 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.309+0000 7f12080a9640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f12000adf80 0x7f120015a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:37.314 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.309+0000 7f12080a9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f120015d480 0x7f12000afa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:37.314 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.309+0000 7f12080a9640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f120015e400 con 0x7f1200069e90 2023-10-16T20:07:37.314 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.309+0000 7f12080a9640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1200159e40 con 0x7f12000adf80 2023-10-16T20:07:37.315 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.309+0000 7f12080a9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f120015de40 con 0x7f120015d480 2023-10-16T20:07:37.315 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.309+0000 7f1205644640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f12000adf80 0x7f120015a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:37.315 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.309+0000 7f1205644640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f12000adf80 0x7f120015a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47728/0 (socket says 172.21.15.88:47728) 2023-10-16T20:07:37.315 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.309+0000 7f1205644640 1 -- 172.21.15.88:0/1938075862 learned_addr learned my addr 172.21.15.88:0/1938075862 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:37.315 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.309+0000 7f1206646640 1 --2- 172.21.15.88:0/1938075862 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f120015d480 0x7f12000afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:37.316 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.309+0000 7f1205644640 1 -- 172.21.15.88:0/1938075862 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f120015d480 msgr2=0x7f12000afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:37.316 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.309+0000 7f1205644640 1 --2- 172.21.15.88:0/1938075862 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f120015d480 0x7f12000afa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:37.316 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.309+0000 7f1205644640 1 -- 172.21.15.88:0/1938075862 >> v1:172.21.15.94:6789/0 conn(0x7f1200069e90 legacy=0x7f120006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:37.316 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.309+0000 7f1205644640 1 -- 172.21.15.88:0/1938075862 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f120015e120 con 0x7f12000adf80 2023-10-16T20:07:37.317 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.313+0000 7f1205644640 1 --2- 172.21.15.88:0/1938075862 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12000adf80 0x7f120015a830 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009870 tx=0x7f11f001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf11beb5efbba780 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:37.317 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.313+0000 7f1204e43640 1 -- 172.21.15.88:0/1938075862 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f00aa670 con 0x7f12000adf80 2023-10-16T20:07:37.317 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.313+0000 7f1204e43640 1 -- 172.21.15.88:0/1938075862 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f00aa7d0 con 0x7f12000adf80 2023-10-16T20:07:37.318 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.313+0000 7f1204e43640 1 -- 172.21.15.88:0/1938075862 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f00aab60 con 0x7f12000adf80 2023-10-16T20:07:37.318 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.313+0000 7f12080a9640 1 -- 172.21.15.88:0/1938075862 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12000adf80 msgr2=0x7f120015a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:37.319 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.313+0000 7f12080a9640 1 --2- 172.21.15.88:0/1938075862 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12000adf80 0x7f120015a830 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009870 tx=0x7f11f001c510 comp rx=0 tx=0).stop 2023-10-16T20:07:37.319 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.313+0000 7f12080a9640 1 -- 172.21.15.88:0/1938075862 shutdown_connections 2023-10-16T20:07:37.319 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.313+0000 7f12080a9640 1 --2- 172.21.15.88:0/1938075862 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f120015d480 0x7f12000afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:37.319 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.313+0000 7f12080a9640 1 --2- 172.21.15.88:0/1938075862 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12000adf80 0x7f120015a830 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:37.319 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.313+0000 7f12080a9640 1 -- 172.21.15.88:0/1938075862 >> 172.21.15.88:0/1938075862 conn(0x7f12000ce5e0 msgr2=0x7f12000ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:37.320 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.313+0000 7f12080a9640 1 -- 172.21.15.88:0/1938075862 shutdown_connections 2023-10-16T20:07:37.320 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.313+0000 7f12080a9640 1 -- 172.21.15.88:0/1938075862 wait complete. 2023-10-16T20:07:37.321 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.317+0000 7f12080a9640 1 Processor -- start 2023-10-16T20:07:37.321 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.317+0000 7f12080a9640 1 -- start start 2023-10-16T20:07:37.322 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.317+0000 7f12080a9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1200069e90 0x7f1200148440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:37.322 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.317+0000 7f12080a9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12000adf80 0x7f12001520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:37.322 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.317+0000 7f12080a9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f120015d480 0x7f1200152600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:37.322 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.317+0000 7f12080a9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1200152f40 con 0x7f12000adf80 2023-10-16T20:07:37.323 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.317+0000 7f12080a9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1200153f90 con 0x7f1200069e90 2023-10-16T20:07:37.323 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.317+0000 7f12080a9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1200070b90 con 0x7f120015d480 2023-10-16T20:07:37.323 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.317+0000 7f1205644640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12000adf80 0x7f12001520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:37.323 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.317+0000 7f1205e45640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1200069e90 0x7f1200148440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:37.323 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.317+0000 7f1205644640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12000adf80 0x7f12001520c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47734/0 (socket says 172.21.15.88:47734) 2023-10-16T20:07:37.324 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.317+0000 7f1205644640 1 -- 172.21.15.88:0/3985062860 learned_addr learned my addr 172.21.15.88:0/3985062860 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:37.324 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.317+0000 7f1206646640 1 --2- 172.21.15.88:0/3985062860 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f120015d480 0x7f1200152600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:37.324 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.317+0000 7f1205644640 1 -- 172.21.15.88:0/3985062860 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f120015d480 msgr2=0x7f1200152600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:37.324 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.317+0000 7f1205644640 1 --2- 172.21.15.88:0/3985062860 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f120015d480 0x7f1200152600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:37.324 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.317+0000 7f1205644640 1 -- 172.21.15.88:0/3985062860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1200069e90 msgr2=0x7f1200148440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:37.325 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.317+0000 7f1205644640 1 --2- 172.21.15.88:0/3985062860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1200069e90 0x7f1200148440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:37.325 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.317+0000 7f1205644640 1 -- 172.21.15.88:0/3985062860 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f00094d0 con 0x7f12000adf80 2023-10-16T20:07:37.326 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.317+0000 7f1205644640 1 --2- 172.21.15.88:0/3985062860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12000adf80 0x7f12001520c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f11f00ac040 tx=0x7f11f001ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:37.326 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.321+0000 7f11f67fc640 1 -- 172.21.15.88:0/3985062860 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f00b3800 con 0x7f12000adf80 2023-10-16T20:07:37.326 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.321+0000 7f11f67fc640 1 -- 172.21.15.88:0/3985062860 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f00b3960 con 0x7f12000adf80 2023-10-16T20:07:37.326 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.321+0000 7f12080a9640 1 -- 172.21.15.88:0/3985062860 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1200070260 con 0x7f12000adf80 2023-10-16T20:07:37.327 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.321+0000 7f11f67fc640 1 -- 172.21.15.88:0/3985062860 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f00b3d10 con 0x7f12000adf80 2023-10-16T20:07:37.327 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.321+0000 7f12080a9640 1 -- 172.21.15.88:0/3985062860 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1200070820 con 0x7f12000adf80 2023-10-16T20:07:37.327 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.321+0000 7f12080a9640 1 -- 172.21.15.88:0/3985062860 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f120006d180 con 0x7f12000adf80 2023-10-16T20:07:37.328 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.321+0000 7f11f67fc640 1 -- 172.21.15.88:0/3985062860 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f11f00c6390 con 0x7f12000adf80 2023-10-16T20:07:37.328 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.321+0000 7f11f67fc640 1 --2- 172.21.15.88:0/3985062860 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f11e40664a0 0x7f11e4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:37.328 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.325+0000 7f11f67fc640 1 -- 172.21.15.88:0/3985062860 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f11f011eba0 con 0x7f12000adf80 2023-10-16T20:07:37.329 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.325+0000 7f1205e45640 1 --2- 172.21.15.88:0/3985062860 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f11e40664a0 0x7f11e4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:37.329 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.325+0000 7f1205e45640 1 --2- 172.21.15.88:0/3985062860 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f11e40664a0 0x7f11e4068960 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f11fc007fd0 tx=0x7f11fc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:37.332 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.329+0000 7f11f67fc640 1 -- 172.21.15.88:0/3985062860 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f11f00ebf70 con 0x7f12000adf80 2023-10-16T20:07:37.496 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.493+0000 7f12080a9640 1 -- 172.21.15.88:0/3985062860 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f120004e9c0 con 0x7f12000adf80 2023-10-16T20:07:37.497 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.493+0000 7f11f67fc640 1 -- 172.21.15.88:0/3985062860 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v58) v1 ==== 80+0+0 (secure 0 0 0) 0x7f11f00ef590 con 0x7f12000adf80 2023-10-16T20:07:37.497 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2023-10-16T20:07:37.498 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.493+0000 7f12080a9640 1 -- 172.21.15.88:0/3985062860 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f11e40664a0 msgr2=0x7f11e4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:37.499 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.493+0000 7f12080a9640 1 --2- 172.21.15.88:0/3985062860 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f11e40664a0 0x7f11e4068960 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f11fc007fd0 tx=0x7f11fc009040 comp rx=0 tx=0).stop 2023-10-16T20:07:37.499 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.493+0000 7f12080a9640 1 -- 172.21.15.88:0/3985062860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12000adf80 msgr2=0x7f12001520c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:37.499 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.493+0000 7f12080a9640 1 --2- 172.21.15.88:0/3985062860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12000adf80 0x7f12001520c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f11f00ac040 tx=0x7f11f001ad50 comp rx=0 tx=0).stop 2023-10-16T20:07:37.499 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.493+0000 7f12080a9640 1 -- 172.21.15.88:0/3985062860 shutdown_connections 2023-10-16T20:07:37.499 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.493+0000 7f12080a9640 1 --2- 172.21.15.88:0/3985062860 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f11e40664a0 0x7f11e4068960 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:37.500 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.493+0000 7f12080a9640 1 --2- 172.21.15.88:0/3985062860 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f120015d480 0x7f1200152600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:37.500 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.493+0000 7f12080a9640 1 --2- 172.21.15.88:0/3985062860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12000adf80 0x7f12001520c0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:37.500 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.493+0000 7f12080a9640 1 --2- 172.21.15.88:0/3985062860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1200069e90 0x7f1200148440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:37.500 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.493+0000 7f12080a9640 1 -- 172.21.15.88:0/3985062860 >> 172.21.15.88:0/3985062860 conn(0x7f12000ce5e0 msgr2=0x7f12000b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:37.500 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.493+0000 7f12080a9640 1 -- 172.21.15.88:0/3985062860 shutdown_connections 2023-10-16T20:07:37.501 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.493+0000 7f12080a9640 1 -- 172.21.15.88:0/3985062860 wait complete. 2023-10-16T20:07:37.517 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi088.front.sepia.ceph.com... 2023-10-16T20:07:37.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:37.518 DEBUG:teuthology.orchestra.run.smithi088:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-16T20:07:37.537 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[52084]: fuse finished with error 0 and tester_r 0 2023-10-16T20:07:37.539 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:07:37.588 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2023-10-16T20:07:37.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:37.589 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T20:07:37.599 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T20:07:37.774 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.769+0000 7f316cbaa640 1 Processor -- start 2023-10-16T20:07:37.775 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.769+0000 7f316cbaa640 1 -- start start 2023-10-16T20:07:37.775 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.773+0000 7f316cbaa640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f31680715c0 0x7f316806ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:37.776 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.773+0000 7f316cbaa640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31680ae720 con 0x7f316806b100 2023-10-16T20:07:37.776 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.773+0000 7f316cbaa640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31680c9a10 con 0x7f316806b4e0 2023-10-16T20:07:37.776 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.773+0000 7f316cbaa640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f316806d650 con 0x7f31680715c0 2023-10-16T20:07:37.776 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.773+0000 7f316659c640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f316806b100 0x7f316806c370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:37398/0 (socket says 172.21.15.88:37398) 2023-10-16T20:07:37.776 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.773+0000 7f316659c640 1 -- 172.21.15.88:0/1818248326 learned_addr learned my addr 172.21.15.88:0/1818248326 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:37.778 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.773+0000 7f315dd9b640 1 --2- 172.21.15.88:0/1818248326 >> v2:172.21.15.94:3300/0 conn(0x7f31680715c0 0x7f316806ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:37.778 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.773+0000 7f3165d9b640 1 -- 172.21.15.88:0/1818248326 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2338273818 0 0) 0x7f3158001120 con 0x7f316806b100 2023-10-16T20:07:37.779 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.773+0000 7f3165d9b640 1 -- 172.21.15.88:0/1818248326 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3154003530 con 0x7f316806b100 2023-10-16T20:07:37.779 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.773+0000 7f3165d9b640 1 -- 172.21.15.88:0/1818248326 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3289306207 0 0) 0x7f3154003530 con 0x7f316806b100 2023-10-16T20:07:37.779 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.773+0000 7f3165d9b640 1 -- 172.21.15.88:0/1818248326 >> v2:172.21.15.94:3300/0 conn(0x7f31680715c0 msgr2=0x7f316806ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:37.779 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.773+0000 7f3165d9b640 1 --2- 172.21.15.88:0/1818248326 >> v2:172.21.15.94:3300/0 conn(0x7f31680715c0 0x7f316806ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:37.779 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.773+0000 7f3165d9b640 1 -- 172.21.15.88:0/1818248326 >> v1:172.21.15.94:6789/0 conn(0x7f316806b4e0 legacy=0x7f31680c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:37.780 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.773+0000 7f3165d9b640 1 -- 172.21.15.88:0/1818248326 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31680c9420 con 0x7f316806b100 2023-10-16T20:07:37.780 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.773+0000 7f3165d9b640 1 -- 172.21.15.88:0/1818248326 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f3158001810 con 0x7f316806b100 2023-10-16T20:07:37.780 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.773+0000 7f3165d9b640 1 -- 172.21.15.88:0/1818248326 >> v1:172.21.15.88:6789/0 conn(0x7f316806b100 legacy=0x7f316806c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:37.780 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.773+0000 7f3165d9b640 1 --2- 172.21.15.88:0/1818248326 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3154003980 0x7f3154003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:37.780 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.773+0000 7f3165d9b640 1 --2- 172.21.15.88:0/1818248326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31540047d0 0x7f3154004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:37.781 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.773+0000 7f3165d9b640 1 --2- 172.21.15.88:0/1818248326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3154005190 0x7f315400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:37.781 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.773+0000 7f3165d9b640 1 -- 172.21.15.88:0/1818248326 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316806d650 con 0x7f31540047d0 2023-10-16T20:07:37.781 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.773+0000 7f3165d9b640 1 -- 172.21.15.88:0/1818248326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315400c030 con 0x7f3154005190 2023-10-16T20:07:37.781 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.773+0000 7f3165d9b640 1 -- 172.21.15.88:0/1818248326 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f315400c1f0 con 0x7f3154003980 2023-10-16T20:07:37.781 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.773+0000 7f315dd9b640 1 --2- 172.21.15.88:0/1818248326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31540047d0 0x7f3154004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:37.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.773+0000 7f3166d9d640 1 --2- 172.21.15.88:0/1818248326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3154005190 0x7f315400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:37.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.773+0000 7f316659c640 1 --2- 172.21.15.88:0/1818248326 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3154003980 0x7f3154003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:37.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.773+0000 7f315dd9b640 1 -- 172.21.15.88:0/1818248326 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3154003980 msgr2=0x7f3154003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:37.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.773+0000 7f315dd9b640 1 --2- 172.21.15.88:0/1818248326 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3154003980 0x7f3154003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:37.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.773+0000 7f315dd9b640 1 -- 172.21.15.88:0/1818248326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3154005190 msgr2=0x7f315400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:37.783 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.773+0000 7f315dd9b640 1 --2- 172.21.15.88:0/1818248326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3154005190 0x7f315400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:37.783 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.773+0000 7f315dd9b640 1 -- 172.21.15.88:0/1818248326 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3154003530 con 0x7f31540047d0 2023-10-16T20:07:37.783 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.773+0000 7f3166d9d640 1 -- 172.21.15.88:0/1818248326 reap_dead start 2023-10-16T20:07:37.783 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.773+0000 7f316659c640 1 --2- 172.21.15.88:0/1818248326 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3154003980 0x7f3154003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:07:37.786 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.777+0000 7f315dd9b640 1 --2- 172.21.15.88:0/1818248326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31540047d0 0x7f3154004c50 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3150009ae0 tx=0x7f315001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=960864ce2a0534a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:37.786 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.777+0000 7f3165d9b640 1 -- 172.21.15.88:0/1818248326 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31500a5640 con 0x7f31540047d0 2023-10-16T20:07:37.786 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.777+0000 7f3165d9b640 1 -- 172.21.15.88:0/1818248326 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31500a57a0 con 0x7f31540047d0 2023-10-16T20:07:37.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.777+0000 7f3165d9b640 1 -- 172.21.15.88:0/1818248326 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31500a5b50 con 0x7f31540047d0 2023-10-16T20:07:37.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.777+0000 7f316cbaa640 1 -- 172.21.15.88:0/1818248326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31540047d0 msgr2=0x7f3154004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:37.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.777+0000 7f316cbaa640 1 --2- 172.21.15.88:0/1818248326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31540047d0 0x7f3154004c50 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3150009ae0 tx=0x7f315001ca50 comp rx=0 tx=0).stop 2023-10-16T20:07:37.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.777+0000 7f316cbaa640 1 -- 172.21.15.88:0/1818248326 shutdown_connections 2023-10-16T20:07:37.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.777+0000 7f316cbaa640 1 --2- 172.21.15.88:0/1818248326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31540047d0 0x7f3154004c50 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:37.788 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.777+0000 7f316cbaa640 1 -- 172.21.15.88:0/1818248326 >> 172.21.15.88:0/1818248326 conn(0x7f3168070aa0 msgr2=0x7f3168070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:37.788 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.777+0000 7f316cbaa640 1 -- 172.21.15.88:0/1818248326 shutdown_connections 2023-10-16T20:07:37.788 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.777+0000 7f316cbaa640 1 -- 172.21.15.88:0/1818248326 wait complete. 2023-10-16T20:07:37.788 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.777+0000 7f316cbaa640 1 Processor -- start 2023-10-16T20:07:37.788 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.777+0000 7f316cbaa640 1 -- start start 2023-10-16T20:07:37.788 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.781+0000 7f316cbaa640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f31540047d0 0x7f31680c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:37.789 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.781+0000 7f316cbaa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31681e77b0 0x7f31680b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:37.789 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.781+0000 7f316cbaa640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31680c43a0 0x7f31680c4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:37.789 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.781+0000 7f316cbaa640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31680c2f40 con 0x7f31680c43a0 2023-10-16T20:07:37.789 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.781+0000 7f316cbaa640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31680c3220 con 0x7f31681e77b0 2023-10-16T20:07:37.789 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.781+0000 7f316cbaa640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31680c3500 con 0x7f31540047d0 2023-10-16T20:07:37.790 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.781+0000 7f3166d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31680c43a0 0x7f31680c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:37.790 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.781+0000 7f3166d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31680c43a0 0x7f31680c4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47750/0 (socket says 172.21.15.88:47750) 2023-10-16T20:07:37.790 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.781+0000 7f3166d9d640 1 -- 172.21.15.88:0/1951144712 learned_addr learned my addr 172.21.15.88:0/1951144712 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:37.790 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.781+0000 7f316659c640 1 --2- 172.21.15.88:0/1951144712 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f31540047d0 0x7f31680c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:37.790 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.781+0000 7f315dd9b640 1 --2- 172.21.15.88:0/1951144712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31681e77b0 0x7f31680b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:37.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.781+0000 7f3166d9d640 1 -- 172.21.15.88:0/1951144712 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f31540047d0 msgr2=0x7f31680c28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:37.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.781+0000 7f3166d9d640 1 --2- 172.21.15.88:0/1951144712 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f31540047d0 0x7f31680c28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:37.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.781+0000 7f3166d9d640 1 -- 172.21.15.88:0/1951144712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31681e77b0 msgr2=0x7f31680b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:37.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.781+0000 7f3166d9d640 1 --2- 172.21.15.88:0/1951144712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31681e77b0 0x7f31680b4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:37.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.781+0000 7f3166d9d640 1 -- 172.21.15.88:0/1951144712 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31680c37e0 con 0x7f31680c43a0 2023-10-16T20:07:37.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.781+0000 7f3166d9d640 1 --2- 172.21.15.88:0/1951144712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31680c43a0 0x7f31680c4850 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f31680702c0 tx=0x7f316000fb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:37.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.781+0000 7f315f7fe640 1 -- 172.21.15.88:0/1951144712 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316000fe50 con 0x7f31680c43a0 2023-10-16T20:07:37.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.781+0000 7f315f7fe640 1 -- 172.21.15.88:0/1951144712 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3160006d70 con 0x7f31680c43a0 2023-10-16T20:07:37.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.781+0000 7f316cbaa640 1 -- 172.21.15.88:0/1951144712 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31680c3ac0 con 0x7f31680c43a0 2023-10-16T20:07:37.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.781+0000 7f315f7fe640 1 -- 172.21.15.88:0/1951144712 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3160003bc0 con 0x7f31680c43a0 2023-10-16T20:07:37.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.781+0000 7f316cbaa640 1 -- 172.21.15.88:0/1951144712 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31680b4780 con 0x7f31680c43a0 2023-10-16T20:07:37.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.781+0000 7f316cbaa640 1 -- 172.21.15.88:0/1951144712 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f316804e9c0 con 0x7f31680c43a0 2023-10-16T20:07:37.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.781+0000 7f315f7fe640 1 -- 172.21.15.88:0/1951144712 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3160003d20 con 0x7f31680c43a0 2023-10-16T20:07:37.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.785+0000 7f315f7fe640 1 --2- 172.21.15.88:0/1951144712 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f31480664a0 0x7f3148068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:37.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.785+0000 7f315f7fe640 1 -- 172.21.15.88:0/1951144712 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f3160083800 con 0x7f31680c43a0 2023-10-16T20:07:37.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.785+0000 7f316659c640 1 --2- 172.21.15.88:0/1951144712 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f31480664a0 0x7f3148068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:37.795 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.785+0000 7f316659c640 1 --2- 172.21.15.88:0/1951144712 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f31480664a0 0x7f3148068960 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3158001280 tx=0x7f3158001150 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:37.795 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.789+0000 7f315f7fe640 1 -- 172.21.15.88:0/1951144712 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3160050bc0 con 0x7f31680c43a0 2023-10-16T20:07:37.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.969+0000 7f316cbaa640 1 -- 172.21.15.88:0/1951144712 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f31680ae320 con 0x7f31680c43a0 2023-10-16T20:07:37.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.969+0000 7f315f7fe640 1 -- 172.21.15.88:0/1951144712 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) v1 ==== 53+0+88 (secure 0 0 0) 0x7f31680ae320 con 0x7f31680c43a0 2023-10-16T20:07:37.974 INFO:teuthology.orchestra.run.smithi088.stdout:name: testcephfs1, metadata pool: testcephfs1_metadata, data pools: [testcephfs1_data ] 2023-10-16T20:07:37.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.969+0000 7f316cbaa640 1 -- 172.21.15.88:0/1951144712 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f31480664a0 msgr2=0x7f3148068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:37.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.969+0000 7f316cbaa640 1 --2- 172.21.15.88:0/1951144712 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f31480664a0 0x7f3148068960 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3158001280 tx=0x7f3158001150 comp rx=0 tx=0).stop 2023-10-16T20:07:37.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.969+0000 7f316cbaa640 1 -- 172.21.15.88:0/1951144712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31680c43a0 msgr2=0x7f31680c4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:37.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.969+0000 7f316cbaa640 1 --2- 172.21.15.88:0/1951144712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31680c43a0 0x7f31680c4850 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f31680702c0 tx=0x7f316000fb40 comp rx=0 tx=0).stop 2023-10-16T20:07:37.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.969+0000 7f316cbaa640 1 -- 172.21.15.88:0/1951144712 shutdown_connections 2023-10-16T20:07:37.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.969+0000 7f316cbaa640 1 --2- 172.21.15.88:0/1951144712 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f31480664a0 0x7f3148068960 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:37.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.969+0000 7f316cbaa640 1 --2- 172.21.15.88:0/1951144712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31680c43a0 0x7f31680c4850 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:37.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.969+0000 7f316cbaa640 1 --2- 172.21.15.88:0/1951144712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31681e77b0 0x7f31680b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:37.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.969+0000 7f316cbaa640 1 --2- 172.21.15.88:0/1951144712 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f31540047d0 0x7f31680c28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:37.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.969+0000 7f316cbaa640 1 -- 172.21.15.88:0/1951144712 >> 172.21.15.88:0/1951144712 conn(0x7f3168070aa0 msgr2=0x7f31680afd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:37.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.969+0000 7f316cbaa640 1 -- 172.21.15.88:0/1951144712 shutdown_connections 2023-10-16T20:07:37.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:37.969+0000 7f316cbaa640 1 -- 172.21.15.88:0/1951144712 wait complete. 2023-10-16T20:07:37.995 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T20:07:37.995 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T20:07:37.995 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi088.front.sepia.ceph.com 2023-10-16T20:07:37.996 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T20:07:37.996 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T20:07:37.996 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T20:07:37.997 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T20:07:37.997 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T20:07:37.997 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T20:07:37.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:37.998 DEBUG:teuthology.orchestra.run.smithi088:> ip addr 2023-10-16T20:07:38.008 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T20:07:38.008 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T20:07:38.009 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T20:07:38.009 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:07:38.009 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2023-10-16T20:07:38.009 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:07:38.009 INFO:teuthology.orchestra.run.smithi088.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:07:38.009 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f0 brd ff:ff:ff:ff:ff:ff 2023-10-16T20:07:38.010 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f0 2023-10-16T20:07:38.010 INFO:teuthology.orchestra.run.smithi088.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:07:38.010 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f1 brd ff:ff:ff:ff:ff:ff 2023-10-16T20:07:38.010 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f1 2023-10-16T20:07:38.010 INFO:teuthology.orchestra.run.smithi088.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:07:38.011 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2c brd ff:ff:ff:ff:ff:ff 2023-10-16T20:07:38.011 INFO:teuthology.orchestra.run.smithi088.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T20:07:38.011 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2d brd ff:ff:ff:ff:ff:ff 2023-10-16T20:07:38.011 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.21.15.88/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-16T20:07:38.011 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft 42274sec preferred_lft 42274sec 2023-10-16T20:07:38.012 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link 2023-10-16T20:07:38.012 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:07:38.012 INFO:teuthology.orchestra.run.smithi088.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-16T20:07:38.012 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 02:42:d9:3b:fb:39 brd ff:ff:ff:ff:ff:ff 2023-10-16T20:07:38.012 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-16T20:07:38.013 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:07:38.013 INFO:teuthology.orchestra.run.smithi088.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T20:07:38.013 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-16T20:07:38.013 INFO:teuthology.orchestra.run.smithi088.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T20:07:38.013 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:07:38.014 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-16T20:07:38.014 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:07:38.014 INFO:teuthology.orchestra.run.smithi088.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T20:07:38.014 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:07:38.014 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-16T20:07:38.015 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:07:38.015 INFO:teuthology.orchestra.run.smithi088.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T20:07:38.015 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:07:38.015 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-16T20:07:38.015 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:07:38.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:38.016 DEBUG:teuthology.orchestra.run.smithi088:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T20:07:38.069 INFO:teuthology.orchestra.run.smithi088.stdout:1 2023-10-16T20:07:38.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:38.072 DEBUG:teuthology.orchestra.run.smithi088:> route 2023-10-16T20:07:38.126 INFO:teuthology.orchestra.run.smithi088.stdout:Kernel IP routing table 2023-10-16T20:07:38.126 INFO:teuthology.orchestra.run.smithi088.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T20:07:38.126 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-16T20:07:38.127 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T20:07:38.127 INFO:teuthology.orchestra.run.smithi088.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-16T20:07:38.127 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T20:07:38.127 INFO:teuthology.orchestra.run.smithi088.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:07:38.127 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:07:38.128 INFO:teuthology.orchestra.run.smithi088.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:07:38.128 INFO:teuthology.orchestra.run.smithi088.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T20:07:38.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:38.129 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec bash -c ' 2023-10-16T20:07:38.129 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2023-10-16T20:07:38.130 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T20:07:38.130 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T20:07:38.130 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T20:07:38.131 DEBUG:teuthology.orchestra.run.smithi088:> ') 2023-10-16T20:07:38.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:38.218 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2023-10-16T20:07:38.228 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T20:07:38.228 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T20:07:38.228 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T20:07:38.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:38.229 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set brx.0 up 2023-10-16T20:07:38.292 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-16T20:07:38.292 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T20:07:38.293 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T20:07:38.349 INFO:teuthology.orchestra.run.smithi088.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T20:07:38.349 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T20:07:38.350 DEBUG:teuthology.orchestra.run.smithi088:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T20:07:38.399 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2023-10-16T20:07:38.464 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T20:07:38.517 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2023-10-16T20:07:38.518 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2023-10-16T20:07:38.518 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2023-10-16T20:07:38.518 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2023-10-16T20:07:38.518 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2023-10-16T20:07:38.518 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2023-10-16T20:07:38.519 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2023-10-16T20:07:38.519 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2023-10-16T20:07:38.519 INFO:teuthology.orchestra.run.smithi088.stdout:/run/lock 2023-10-16T20:07:38.519 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2023-10-16T20:07:38.519 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2023-10-16T20:07:38.520 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2023-10-16T20:07:38.520 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T20:07:38.520 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2023-10-16T20:07:38.520 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2023-10-16T20:07:38.520 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2023-10-16T20:07:38.521 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2023-10-16T20:07:38.521 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/fs/nfsd 2023-10-16T20:07:38.521 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2023-10-16T20:07:38.521 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2023-10-16T20:07:38.521 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T20:07:38.521 INFO:teuthology.orchestra.run.smithi088.stdout:/run/rpc_pipefs 2023-10-16T20:07:38.522 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2023-10-16T20:07:38.522 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2023-10-16T20:07:38.522 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T20:07:38.522 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T20:07:38.522 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T20:07:38.523 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T20:07:38.523 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2023-10-16T20:07:38.523 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:07:38.523 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:07:38.523 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:07:38.523 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:07:38.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:38.525 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2023-10-16T20:07:38.579 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-16T20:07:38.580 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-16T20:07:38.626 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2023-10-16T20:07:38.648 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T20:07:38.701 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2023-10-16T20:07:38.701 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2023-10-16T20:07:38.701 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2023-10-16T20:07:38.702 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2023-10-16T20:07:38.702 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2023-10-16T20:07:38.702 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2023-10-16T20:07:38.702 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2023-10-16T20:07:38.702 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2023-10-16T20:07:38.703 INFO:teuthology.orchestra.run.smithi088.stdout:/run/lock 2023-10-16T20:07:38.703 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2023-10-16T20:07:38.703 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2023-10-16T20:07:38.703 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2023-10-16T20:07:38.703 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T20:07:38.704 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2023-10-16T20:07:38.704 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2023-10-16T20:07:38.704 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2023-10-16T20:07:38.704 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2023-10-16T20:07:38.704 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/fs/nfsd 2023-10-16T20:07:38.704 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2023-10-16T20:07:38.705 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2023-10-16T20:07:38.705 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T20:07:38.705 INFO:teuthology.orchestra.run.smithi088.stdout:/run/rpc_pipefs 2023-10-16T20:07:38.705 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2023-10-16T20:07:38.705 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2023-10-16T20:07:38.706 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T20:07:38.706 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T20:07:38.706 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T20:07:38.706 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T20:07:38.706 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2023-10-16T20:07:38.707 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:07:38.707 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:07:38.707 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:07:38.707 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:07:38.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:38.708 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2023-10-16T20:07:38.721 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:2023-10-16T20:07:38.717+0000 7f41d02bf580 -1 init, newargv = 0x5562b39b7ee0 newargc=15 2023-10-16T20:07:38.721 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[52482]: starting ceph client 2023-10-16T20:07:38.728 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[52482]: starting fuse 2023-10-16T20:07:38.730 INFO:teuthology.orchestra.run.smithi088.stdout:43 2023-10-16T20:07:38.730 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-10-16T20:07:38.730 DEBUG:teuthology.orchestra.run.smithi088:> sudo stdin-killer -- python3 -c ' 2023-10-16T20:07:38.731 DEBUG:teuthology.orchestra.run.smithi088:> import glob 2023-10-16T20:07:38.731 DEBUG:teuthology.orchestra.run.smithi088:> import re 2023-10-16T20:07:38.732 DEBUG:teuthology.orchestra.run.smithi088:> import os 2023-10-16T20:07:38.732 DEBUG:teuthology.orchestra.run.smithi088:> import subprocess 2023-10-16T20:07:38.732 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:07:38.733 DEBUG:teuthology.orchestra.run.smithi088:> def _find_admin_socket(client_name): 2023-10-16T20:07:38.733 DEBUG:teuthology.orchestra.run.smithi088:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-16T20:07:38.733 DEBUG:teuthology.orchestra.run.smithi088:> files = glob.glob(asok_path) 2023-10-16T20:07:38.734 DEBUG:teuthology.orchestra.run.smithi088:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-16T20:07:38.734 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:07:38.734 DEBUG:teuthology.orchestra.run.smithi088:> # Given a non-glob path, it better be there 2023-10-16T20:07:38.735 DEBUG:teuthology.orchestra.run.smithi088:> if "*" not in asok_path: 2023-10-16T20:07:38.735 DEBUG:teuthology.orchestra.run.smithi088:> assert(len(files) == 1) 2023-10-16T20:07:38.735 DEBUG:teuthology.orchestra.run.smithi088:> return files[0] 2023-10-16T20:07:38.736 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:07:38.736 DEBUG:teuthology.orchestra.run.smithi088:> for f in files: 2023-10-16T20:07:38.736 DEBUG:teuthology.orchestra.run.smithi088:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-16T20:07:38.737 DEBUG:teuthology.orchestra.run.smithi088:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-16T20:07:38.737 DEBUG:teuthology.orchestra.run.smithi088:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-16T20:07:38.737 DEBUG:teuthology.orchestra.run.smithi088:> contents = proc_f.read() 2023-10-16T20:07:38.738 DEBUG:teuthology.orchestra.run.smithi088:> if mountpoint in contents: 2023-10-16T20:07:38.738 DEBUG:teuthology.orchestra.run.smithi088:> return f 2023-10-16T20:07:38.738 DEBUG:teuthology.orchestra.run.smithi088:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-16T20:07:38.739 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:07:38.739 DEBUG:teuthology.orchestra.run.smithi088:> print(_find_admin_socket("client.0")) 2023-10-16T20:07:38.739 DEBUG:teuthology.orchestra.run.smithi088:> ' 2023-10-16T20:07:38.855 INFO:teuthology.orchestra.run.smithi088.stdout:/var/run/ceph/ceph-client.0.52482.asok 2023-10-16T20:07:38.859 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-16T20:07:38.866 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52482.asok 2023-10-16T20:07:38.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:38.867 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52482.asok status 2023-10-16T20:07:39.035 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2023-10-16T20:07:39.036 INFO:teuthology.orchestra.run.smithi088.stdout: "metadata": { 2023-10-16T20:07:39.036 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_sha1": "4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0", 2023-10-16T20:07:39.036 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_version": "ceph version 18.0.0-6730-g4ba8a190 (4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0) reef (dev)", 2023-10-16T20:07:39.036 INFO:teuthology.orchestra.run.smithi088.stdout: "entity_id": "0", 2023-10-16T20:07:39.037 INFO:teuthology.orchestra.run.smithi088.stdout: "hostname": "smithi088", 2023-10-16T20:07:39.037 INFO:teuthology.orchestra.run.smithi088.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-16T20:07:39.037 INFO:teuthology.orchestra.run.smithi088.stdout: "pid": "52482", 2023-10-16T20:07:39.037 INFO:teuthology.orchestra.run.smithi088.stdout: "root": "/" 2023-10-16T20:07:39.037 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:07:39.038 INFO:teuthology.orchestra.run.smithi088.stdout: "dentry_count": 0, 2023-10-16T20:07:39.038 INFO:teuthology.orchestra.run.smithi088.stdout: "dentry_pinned_count": 0, 2023-10-16T20:07:39.038 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 6447, 2023-10-16T20:07:39.038 INFO:teuthology.orchestra.run.smithi088.stdout: "inst": { 2023-10-16T20:07:39.038 INFO:teuthology.orchestra.run.smithi088.stdout: "name": { 2023-10-16T20:07:39.039 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "client", 2023-10-16T20:07:39.039 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 6447 2023-10-16T20:07:39.039 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:07:39.039 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": { 2023-10-16T20:07:39.039 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2023-10-16T20:07:39.040 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "192.168.0.1:0", 2023-10-16T20:07:39.040 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 1972060934 2023-10-16T20:07:39.040 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:07:39.040 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:07:39.040 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": { 2023-10-16T20:07:39.040 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2023-10-16T20:07:39.041 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "192.168.0.1:0", 2023-10-16T20:07:39.041 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 1972060934 2023-10-16T20:07:39.041 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:07:39.041 INFO:teuthology.orchestra.run.smithi088.stdout: "inst_str": "client.6447 192.168.0.1:0/1972060934", 2023-10-16T20:07:39.041 INFO:teuthology.orchestra.run.smithi088.stdout: "addr_str": "192.168.0.1:0/1972060934", 2023-10-16T20:07:39.042 INFO:teuthology.orchestra.run.smithi088.stdout: "inode_count": 1, 2023-10-16T20:07:39.042 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_epoch": 28, 2023-10-16T20:07:39.042 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_epoch": 58, 2023-10-16T20:07:39.042 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_epoch_barrier": 0, 2023-10-16T20:07:39.042 INFO:teuthology.orchestra.run.smithi088.stdout: "blocklisted": false, 2023-10-16T20:07:39.043 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "testcephfs1" 2023-10-16T20:07:39.043 INFO:teuthology.orchestra.run.smithi088.stdout:} 2023-10-16T20:07:39.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:39.051 DEBUG:teuthology.orchestra.run.smithi088:> stat --file-system '--printf=%T 2023-10-16T20:07:39.051 DEBUG:teuthology.orchestra.run.smithi088:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-16T20:07:39.061 INFO:teuthology.orchestra.run.smithi088.stdout:fuseblk 2023-10-16T20:07:39.061 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-16T20:07:39.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:39.062 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T20:07:39.123 DEBUG:teuthology.orchestra.run.smithi088:> stdin-killer -- python3 -c ' 2023-10-16T20:07:39.124 DEBUG:teuthology.orchestra.run.smithi088:> import os 2023-10-16T20:07:39.124 DEBUG:teuthology.orchestra.run.smithi088:> import errno 2023-10-16T20:07:39.124 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:07:39.125 DEBUG:teuthology.orchestra.run.smithi088:> fd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR) 2023-10-16T20:07:39.125 DEBUG:teuthology.orchestra.run.smithi088:> try: 2023-10-16T20:07:39.125 DEBUG:teuthology.orchestra.run.smithi088:> if False: 2023-10-16T20:07:39.126 DEBUG:teuthology.orchestra.run.smithi088:> ret = os.read(fd, 1024) 2023-10-16T20:07:39.126 DEBUG:teuthology.orchestra.run.smithi088:> else: 2023-10-16T20:07:39.126 DEBUG:teuthology.orchestra.run.smithi088:> os.write(fd, b'"'"'content'"'"') 2023-10-16T20:07:39.127 DEBUG:teuthology.orchestra.run.smithi088:> except OSError as e: 2023-10-16T20:07:39.127 DEBUG:teuthology.orchestra.run.smithi088:> if e.errno != errno.EPERM: 2023-10-16T20:07:39.128 DEBUG:teuthology.orchestra.run.smithi088:> raise 2023-10-16T20:07:39.128 DEBUG:teuthology.orchestra.run.smithi088:> else: 2023-10-16T20:07:39.128 DEBUG:teuthology.orchestra.run.smithi088:> raise RuntimeError("client does not check permission of data pool") 2023-10-16T20:07:39.128 DEBUG:teuthology.orchestra.run.smithi088:> ' 2023-10-16T20:07:39.238 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-16T20:07:39.244 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow w pool=testcephfs1_data' 2023-10-16T20:07:39.432 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.429+0000 7efc3690f640 1 Processor -- start 2023-10-16T20:07:39.432 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.429+0000 7efc3690f640 1 -- start start 2023-10-16T20:07:39.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.429+0000 7efc3690f640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7efc30164570 0x7efc300af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:39.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.429+0000 7efc3690f640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7efc30070250 0x7efc300706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:39.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.429+0000 7efc3690f640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc300ae840 con 0x7efc300afe20 2023-10-16T20:07:39.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.429+0000 7efc3690f640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc30070c00 con 0x7efc30164570 2023-10-16T20:07:39.435 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.429+0000 7efc3690f640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc300aeb20 con 0x7efc30070250 2023-10-16T20:07:39.435 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.429+0000 7efc2ffff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7efc30164570 0x7efc300af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:39.435 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.429+0000 7efc2ffff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7efc30164570 0x7efc300af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47762/0 (socket says 172.21.15.88:47762) 2023-10-16T20:07:39.435 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.429+0000 7efc2f7fe640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7efc300afe20 0x7efc300addf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:37408/0 (socket says 172.21.15.88:37408) 2023-10-16T20:07:39.435 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.429+0000 7efc2ffff640 1 -- 172.21.15.88:0/1442600397 learned_addr learned my addr 172.21.15.88:0/1442600397 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:39.436 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.429+0000 7efc34eac640 1 --2- 172.21.15.88:0/1442600397 >> v2:172.21.15.94:3300/0 conn(0x7efc30070250 0x7efc300706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:39.436 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.429+0000 7efc2ffff640 1 -- 172.21.15.88:0/1442600397 >> v2:172.21.15.94:3300/0 conn(0x7efc30070250 msgr2=0x7efc300706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:39.436 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.429+0000 7efc2ffff640 1 --2- 172.21.15.88:0/1442600397 >> v2:172.21.15.94:3300/0 conn(0x7efc30070250 0x7efc300706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:39.436 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.429+0000 7efc2ffff640 1 -- 172.21.15.88:0/1442600397 >> v1:172.21.15.88:6789/0 conn(0x7efc300afe20 legacy=0x7efc300addf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:39.436 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.429+0000 7efc2ffff640 1 -- 172.21.15.88:0/1442600397 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc300aee00 con 0x7efc30164570 2023-10-16T20:07:39.437 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.433+0000 7efc2ffff640 1 --2- 172.21.15.88:0/1442600397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc30164570 0x7efc300af8e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7efc300701e0 tx=0x7efc2001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3721c57837d07a2f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:39.437 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.433+0000 7efc2effd640 1 -- 172.21.15.88:0/1442600397 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc200a89f0 con 0x7efc30164570 2023-10-16T20:07:39.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.433+0000 7efc2effd640 1 -- 172.21.15.88:0/1442600397 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc200a8b50 con 0x7efc30164570 2023-10-16T20:07:39.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.433+0000 7efc2effd640 1 -- 172.21.15.88:0/1442600397 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc200b0440 con 0x7efc30164570 2023-10-16T20:07:39.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.433+0000 7efc3690f640 1 -- 172.21.15.88:0/1442600397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc30164570 msgr2=0x7efc300af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:39.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.433+0000 7efc3690f640 1 --2- 172.21.15.88:0/1442600397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc30164570 0x7efc300af8e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7efc300701e0 tx=0x7efc2001ecf0 comp rx=0 tx=0).stop 2023-10-16T20:07:39.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.433+0000 7efc3690f640 1 -- 172.21.15.88:0/1442600397 shutdown_connections 2023-10-16T20:07:39.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.433+0000 7efc3690f640 1 --2- 172.21.15.88:0/1442600397 >> v2:172.21.15.94:3300/0 conn(0x7efc30070250 0x7efc300706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:39.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.433+0000 7efc3690f640 1 --2- 172.21.15.88:0/1442600397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc30164570 0x7efc300af8e0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:39.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.433+0000 7efc3690f640 1 -- 172.21.15.88:0/1442600397 >> 172.21.15.88:0/1442600397 conn(0x7efc300cb080 msgr2=0x7efc300cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:39.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.433+0000 7efc3690f640 1 -- 172.21.15.88:0/1442600397 shutdown_connections 2023-10-16T20:07:39.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.433+0000 7efc3690f640 1 -- 172.21.15.88:0/1442600397 wait complete. 2023-10-16T20:07:39.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.437+0000 7efc3690f640 1 Processor -- start 2023-10-16T20:07:39.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.437+0000 7efc3690f640 1 -- start start 2023-10-16T20:07:39.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.437+0000 7efc3690f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efc30070250 0x7efc301589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:39.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.437+0000 7efc3690f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc300afe20 0x7efc301f66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:39.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.437+0000 7efc3690f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc30164570 0x7efc3006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:39.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.437+0000 7efc3690f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc300a94e0 con 0x7efc300afe20 2023-10-16T20:07:39.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.437+0000 7efc3690f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc300a9a10 con 0x7efc30164570 2023-10-16T20:07:39.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.437+0000 7efc3690f640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc300a9c40 con 0x7efc30070250 2023-10-16T20:07:39.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.437+0000 7efc2f7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc300afe20 0x7efc301f66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:39.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.437+0000 7efc2f7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc300afe20 0x7efc301f66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47764/0 (socket says 172.21.15.88:47764) 2023-10-16T20:07:39.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.437+0000 7efc2f7fe640 1 -- 172.21.15.88:0/2711535705 learned_addr learned my addr 172.21.15.88:0/2711535705 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:39.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.437+0000 7efc2ffff640 1 --2- 172.21.15.88:0/2711535705 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efc30070250 0x7efc301589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:39.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.437+0000 7efc34eac640 1 --2- 172.21.15.88:0/2711535705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc30164570 0x7efc3006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:39.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.437+0000 7efc2f7fe640 1 -- 172.21.15.88:0/2711535705 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efc30070250 msgr2=0x7efc301589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:39.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.437+0000 7efc2f7fe640 1 --2- 172.21.15.88:0/2711535705 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efc30070250 0x7efc301589b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:39.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.437+0000 7efc2f7fe640 1 -- 172.21.15.88:0/2711535705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc30164570 msgr2=0x7efc3006d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:39.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.437+0000 7efc2f7fe640 1 --2- 172.21.15.88:0/2711535705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc30164570 0x7efc3006d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:39.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.437+0000 7efc2f7fe640 1 -- 172.21.15.88:0/2711535705 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc200094d0 con 0x7efc300afe20 2023-10-16T20:07:39.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.437+0000 7efc2f7fe640 1 --2- 172.21.15.88:0/2711535705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc300afe20 0x7efc301f66b0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7efc2400c490 tx=0x7efc2400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:39.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.437+0000 7efc2cff9640 1 -- 172.21.15.88:0/2711535705 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc24007480 con 0x7efc300afe20 2023-10-16T20:07:39.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.437+0000 7efc2cff9640 1 -- 172.21.15.88:0/2711535705 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc240075e0 con 0x7efc300afe20 2023-10-16T20:07:39.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.437+0000 7efc3690f640 1 -- 172.21.15.88:0/2711535705 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc300a9f20 con 0x7efc300afe20 2023-10-16T20:07:39.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.437+0000 7efc2cff9640 1 -- 172.21.15.88:0/2711535705 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc24004a10 con 0x7efc300afe20 2023-10-16T20:07:39.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.437+0000 7efc3690f640 1 -- 172.21.15.88:0/2711535705 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc300aa4e0 con 0x7efc300afe20 2023-10-16T20:07:39.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.441+0000 7efc3690f640 1 -- 172.21.15.88:0/2711535705 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc30070fb0 con 0x7efc300afe20 2023-10-16T20:07:39.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.441+0000 7efc2cff9640 1 -- 172.21.15.88:0/2711535705 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7efc24004b70 con 0x7efc300afe20 2023-10-16T20:07:39.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.441+0000 7efc2cff9640 1 --2- 172.21.15.88:0/2711535705 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7efc1c0664a0 0x7efc1c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:39.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.441+0000 7efc2cff9640 1 -- 172.21.15.88:0/2711535705 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7efc24083410 con 0x7efc300afe20 2023-10-16T20:07:39.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.441+0000 7efc2ffff640 1 --2- 172.21.15.88:0/2711535705 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7efc1c0664a0 0x7efc1c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:39.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.441+0000 7efc2ffff640 1 --2- 172.21.15.88:0/2711535705 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7efc1c0664a0 0x7efc1c068960 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7efc200b2000 tx=0x7efc20009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:39.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.445+0000 7efc2cff9640 1 -- 172.21.15.88:0/2711535705 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7efc24050740 con 0x7efc300afe20 2023-10-16T20:07:39.653 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.649+0000 7efc3690f640 1 -- 172.21.15.88:0/2711535705 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow w pool=testcephfs1_data"]} v 0) v1 -- 0x7efc300aa6d0 con 0x7efc300afe20 2023-10-16T20:07:39.656 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.653+0000 7efc2cff9640 1 -- 172.21.15.88:0/2711535705 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow w pool=testcephfs1_data"]}]=0 updated caps for client.0 v19) v1 ==== 188+0+145 (secure 0 0 0) 0x7efc24053d60 con 0x7efc300afe20 2023-10-16T20:07:39.656 INFO:teuthology.orchestra.run.smithi088.stdout:[client.0] 2023-10-16T20:07:39.656 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQAvly1looeRAhAAeLshHOVIhQqoTtlPehorIg== 2023-10-16T20:07:39.656 INFO:teuthology.orchestra.run.smithi088.stdout: caps mds = "allow" 2023-10-16T20:07:39.657 INFO:teuthology.orchestra.run.smithi088.stdout: caps mon = "allow r" 2023-10-16T20:07:39.657 INFO:teuthology.orchestra.run.smithi088.stdout: caps osd = "allow w pool=testcephfs1_data" 2023-10-16T20:07:39.657 INFO:teuthology.orchestra.run.smithi088.stderr:updated caps for client.0 2023-10-16T20:07:39.658 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.653+0000 7efc3690f640 1 -- 172.21.15.88:0/2711535705 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7efc1c0664a0 msgr2=0x7efc1c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:39.658 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.653+0000 7efc3690f640 1 --2- 172.21.15.88:0/2711535705 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7efc1c0664a0 0x7efc1c068960 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7efc200b2000 tx=0x7efc20009040 comp rx=0 tx=0).stop 2023-10-16T20:07:39.659 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.653+0000 7efc3690f640 1 -- 172.21.15.88:0/2711535705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc300afe20 msgr2=0x7efc301f66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:39.659 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.653+0000 7efc3690f640 1 --2- 172.21.15.88:0/2711535705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc300afe20 0x7efc301f66b0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7efc2400c490 tx=0x7efc2400c950 comp rx=0 tx=0).stop 2023-10-16T20:07:39.659 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.653+0000 7efc3690f640 1 -- 172.21.15.88:0/2711535705 shutdown_connections 2023-10-16T20:07:39.659 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.653+0000 7efc3690f640 1 --2- 172.21.15.88:0/2711535705 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7efc1c0664a0 0x7efc1c068960 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:39.659 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.653+0000 7efc3690f640 1 --2- 172.21.15.88:0/2711535705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc30164570 0x7efc3006d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:39.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.653+0000 7efc3690f640 1 --2- 172.21.15.88:0/2711535705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc300afe20 0x7efc301f66b0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:39.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.653+0000 7efc3690f640 1 --2- 172.21.15.88:0/2711535705 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efc30070250 0x7efc301589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:39.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.653+0000 7efc3690f640 1 -- 172.21.15.88:0/2711535705 >> 172.21.15.88:0/2711535705 conn(0x7efc300cb080 msgr2=0x7efc300cca40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:39.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.653+0000 7efc3690f640 1 -- 172.21.15.88:0/2711535705 shutdown_connections 2023-10-16T20:07:39.660 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.653+0000 7efc3690f640 1 -- 172.21.15.88:0/2711535705 wait complete. 2023-10-16T20:07:39.677 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:07:39.677 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:07:39.687 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:07:39.868 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.865+0000 7f0a3f365640 1 Processor -- start 2023-10-16T20:07:39.868 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.865+0000 7f0a3f365640 1 -- start start 2023-10-16T20:07:39.869 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.865+0000 7f0a3f365640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0a3806b5b0 0x7f0a380b1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:39.869 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.865+0000 7f0a3f365640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a38071b80 con 0x7f0a3806df00 2023-10-16T20:07:39.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.865+0000 7f0a3f365640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a3806c510 con 0x7f0a3815ab10 2023-10-16T20:07:39.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.865+0000 7f0a3f365640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a3806c7f0 con 0x7f0a3806b5b0 2023-10-16T20:07:39.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.865+0000 7f0a3d902640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f0a3806df00 0x7f0a3806cff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:37414/0 (socket says 172.21.15.88:37414) 2023-10-16T20:07:39.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.865+0000 7f0a3d902640 1 -- 172.21.15.88:0/284691175 learned_addr learned my addr 172.21.15.88:0/284691175 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:39.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.865+0000 7f0a3c900640 1 --2- 172.21.15.88:0/284691175 >> v2:172.21.15.94:3300/0 conn(0x7f0a3806b5b0 0x7f0a380b1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:39.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.865+0000 7f0a2ffff640 1 -- 172.21.15.88:0/284691175 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 349873419 0 0) 0x7f0a38071b80 con 0x7f0a3806df00 2023-10-16T20:07:39.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.865+0000 7f0a2ffff640 1 -- 172.21.15.88:0/284691175 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a24003530 con 0x7f0a3806df00 2023-10-16T20:07:39.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.865+0000 7f0a3c900640 1 -- 172.21.15.88:0/284691175 >> v1:172.21.15.94:6789/0 conn(0x7f0a3815ab10 legacy=0x7f0a3806aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:39.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.865+0000 7f0a2ffff640 1 -- 172.21.15.88:0/284691175 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1787866451 0 0) 0x7f0a3806c510 con 0x7f0a3815ab10 2023-10-16T20:07:39.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.865+0000 7f0a3c900640 1 -- 172.21.15.88:0/284691175 >> v1:172.21.15.88:6789/0 conn(0x7f0a3806df00 legacy=0x7f0a3806cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:39.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.865+0000 7f0a3c900640 1 -- 172.21.15.88:0/284691175 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3806cad0 con 0x7f0a3806b5b0 2023-10-16T20:07:39.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.869+0000 7f0a3c900640 1 --2- 172.21.15.88:0/284691175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a3806b5b0 0x7f0a380b1cb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0a280097c0 tx=0x7f0a2801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=196fed0a20a74048 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:39.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.869+0000 7f0a2ffff640 1 -- 172.21.15.88:0/284691175 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a280ac640 con 0x7f0a3806b5b0 2023-10-16T20:07:39.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.869+0000 7f0a2ffff640 1 -- 172.21.15.88:0/284691175 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a280ac7a0 con 0x7f0a3806b5b0 2023-10-16T20:07:39.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.869+0000 7f0a2ffff640 1 -- 172.21.15.88:0/284691175 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a280acb30 con 0x7f0a3806b5b0 2023-10-16T20:07:39.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.869+0000 7f0a3f365640 1 -- 172.21.15.88:0/284691175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a3806b5b0 msgr2=0x7f0a380b1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:39.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.869+0000 7f0a3f365640 1 --2- 172.21.15.88:0/284691175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a3806b5b0 0x7f0a380b1cb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0a280097c0 tx=0x7f0a2801c480 comp rx=0 tx=0).stop 2023-10-16T20:07:39.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.869+0000 7f0a3f365640 1 -- 172.21.15.88:0/284691175 shutdown_connections 2023-10-16T20:07:39.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.869+0000 7f0a3f365640 1 --2- 172.21.15.88:0/284691175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a3806b5b0 0x7f0a380b1cb0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:39.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.869+0000 7f0a3f365640 1 -- 172.21.15.88:0/284691175 >> 172.21.15.88:0/284691175 conn(0x7f0a38070aa0 msgr2=0x7f0a38070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:39.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.869+0000 7f0a3f365640 1 -- 172.21.15.88:0/284691175 shutdown_connections 2023-10-16T20:07:39.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.869+0000 7f0a3f365640 1 -- 172.21.15.88:0/284691175 wait complete. 2023-10-16T20:07:39.878 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.873+0000 7f0a3f365640 1 Processor -- start 2023-10-16T20:07:39.878 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.873+0000 7f0a3f365640 1 -- start start 2023-10-16T20:07:39.879 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.873+0000 7f0a3f365640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a3806b5b0 0x7f0a3815b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:39.879 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.873+0000 7f0a3f365640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a3806df00 0x7f0a38158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:39.879 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.873+0000 7f0a3f365640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a3815ab10 0x7f0a38159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:39.879 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.873+0000 7f0a3f365640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3815ba40 con 0x7f0a3815ab10 2023-10-16T20:07:39.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.873+0000 7f0a3f365640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a380c28e0 con 0x7f0a3806b5b0 2023-10-16T20:07:39.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.873+0000 7f0a3f365640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a380c3420 con 0x7f0a3806df00 2023-10-16T20:07:39.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.873+0000 7f0a3d902640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a3815ab10 0x7f0a38159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:39.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.873+0000 7f0a3d902640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a3815ab10 0x7f0a38159160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47776/0 (socket says 172.21.15.88:47776) 2023-10-16T20:07:39.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.873+0000 7f0a3d902640 1 -- 172.21.15.88:0/1027196536 learned_addr learned my addr 172.21.15.88:0/1027196536 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:39.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.873+0000 7f0a3d101640 1 --2- 172.21.15.88:0/1027196536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a3806b5b0 0x7f0a3815b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:39.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.873+0000 7f0a3c900640 1 --2- 172.21.15.88:0/1027196536 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a3806df00 0x7f0a38158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:39.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.873+0000 7f0a3d902640 1 -- 172.21.15.88:0/1027196536 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a3806df00 msgr2=0x7f0a38158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:39.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.873+0000 7f0a3d902640 1 --2- 172.21.15.88:0/1027196536 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a3806df00 0x7f0a38158c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:39.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.873+0000 7f0a3d902640 1 -- 172.21.15.88:0/1027196536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a3806b5b0 msgr2=0x7f0a3815b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:39.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.873+0000 7f0a3d902640 1 --2- 172.21.15.88:0/1027196536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a3806b5b0 0x7f0a3815b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:39.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.873+0000 7f0a3d902640 1 -- 172.21.15.88:0/1027196536 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a28009420 con 0x7f0a3815ab10 2023-10-16T20:07:39.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.873+0000 7f0a3d902640 1 --2- 172.21.15.88:0/1027196536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a3815ab10 0x7f0a38159160 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000cb40 tx=0x7f0a3000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:39.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.873+0000 7f0a2dffb640 1 -- 172.21.15.88:0/1027196536 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3000aca0 con 0x7f0a3815ab10 2023-10-16T20:07:39.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.873+0000 7f0a2dffb640 1 -- 172.21.15.88:0/1027196536 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a3000ae00 con 0x7f0a3815ab10 2023-10-16T20:07:39.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.873+0000 7f0a3f365640 1 -- 172.21.15.88:0/1027196536 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a380c2af0 con 0x7f0a3815ab10 2023-10-16T20:07:39.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.873+0000 7f0a2dffb640 1 -- 172.21.15.88:0/1027196536 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a30012620 con 0x7f0a3815ab10 2023-10-16T20:07:39.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.873+0000 7f0a3f365640 1 -- 172.21.15.88:0/1027196536 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a380c30b0 con 0x7f0a3815ab10 2023-10-16T20:07:39.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.877+0000 7f0a3f365640 1 -- 172.21.15.88:0/1027196536 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a3804e9c0 con 0x7f0a3815ab10 2023-10-16T20:07:39.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.877+0000 7f0a2dffb640 1 -- 172.21.15.88:0/1027196536 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0a30007070 con 0x7f0a3815ab10 2023-10-16T20:07:39.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.877+0000 7f0a2dffb640 1 --2- 172.21.15.88:0/1027196536 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0a1c0664a0 0x7f0a1c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:39.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.877+0000 7f0a2dffb640 1 -- 172.21.15.88:0/1027196536 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f0a30083ff0 con 0x7f0a3815ab10 2023-10-16T20:07:39.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.877+0000 7f0a3d101640 1 --2- 172.21.15.88:0/1027196536 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0a1c0664a0 0x7f0a1c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:39.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.877+0000 7f0a3d101640 1 --2- 172.21.15.88:0/1027196536 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0a1c0664a0 0x7f0a1c068960 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0a34007fd0 tx=0x7f0a34009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:39.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:39.881+0000 7f0a2dffb640 1 -- 172.21.15.88:0/1027196536 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0a300b5090 con 0x7f0a3815ab10 2023-10-16T20:07:40.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.033+0000 7f0a3f365640 1 -- 172.21.15.88:0/1027196536 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a380c3560 con 0x7f0a3815ab10 2023-10-16T20:07:40.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.033+0000 7f0a2dffb640 1 -- 172.21.15.88:0/1027196536 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16718 (secure 0 0 0) 0x7f0a30017090 con 0x7f0a3815ab10 2023-10-16T20:07:40.037 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:07:40.038 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":58,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:07:27.206614+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T20:07:24.001121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"testcephfs1_data","create_time":"2023-10-16T20:07:24.636209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:07:40.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.033+0000 7f0a3f365640 1 -- 172.21.15.88:0/1027196536 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0a1c0664a0 msgr2=0x7f0a1c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:40.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.033+0000 7f0a3f365640 1 --2- 172.21.15.88:0/1027196536 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0a1c0664a0 0x7f0a1c068960 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0a34007fd0 tx=0x7f0a34009040 comp rx=0 tx=0).stop 2023-10-16T20:07:40.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.033+0000 7f0a3f365640 1 -- 172.21.15.88:0/1027196536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a3815ab10 msgr2=0x7f0a38159160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:40.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.033+0000 7f0a3f365640 1 --2- 172.21.15.88:0/1027196536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a3815ab10 0x7f0a38159160 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000cb40 tx=0x7f0a3000a060 comp rx=0 tx=0).stop 2023-10-16T20:07:40.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.033+0000 7f0a3f365640 1 -- 172.21.15.88:0/1027196536 shutdown_connections 2023-10-16T20:07:40.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.033+0000 7f0a3f365640 1 --2- 172.21.15.88:0/1027196536 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0a1c0664a0 0x7f0a1c068960 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:40.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.033+0000 7f0a3f365640 1 --2- 172.21.15.88:0/1027196536 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a3806df00 0x7f0a38158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:40.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.033+0000 7f0a3f365640 1 --2- 172.21.15.88:0/1027196536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a3815ab10 0x7f0a38159160 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:40.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.033+0000 7f0a3f365640 1 --2- 172.21.15.88:0/1027196536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a3806b5b0 0x7f0a3815b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:40.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.033+0000 7f0a3f365640 1 -- 172.21.15.88:0/1027196536 >> 172.21.15.88:0/1027196536 conn(0x7f0a38070aa0 msgr2=0x7f0a38070e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:40.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.033+0000 7f0a3f365640 1 -- 172.21.15.88:0/1027196536 shutdown_connections 2023-10-16T20:07:40.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.033+0000 7f0a3f365640 1 -- 172.21.15.88:0/1027196536 wait complete. 2023-10-16T20:07:40.058 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'testcephfs1_metadata', 'create_time': '2023-10-16T20:07:24.001121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'testcephfs1_data', 'create_time': '2023-10-16T20:07:24.636209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T20:07:40.059 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:07:40.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.193+0000 7f019e980640 1 Processor -- start 2023-10-16T20:07:40.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.193+0000 7f019e980640 1 -- start start 2023-10-16T20:07:40.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.197+0000 7f019e980640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f01980b0da0 0x7f0198159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:40.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.197+0000 7f019e980640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f019815ad30 con 0x7f01980b4260 2023-10-16T20:07:40.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.197+0000 7f019e980640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01980b4640 con 0x7f019815a1f0 2023-10-16T20:07:40.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.197+0000 7f019e980640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f019815a770 con 0x7f01980b0da0 2023-10-16T20:07:40.202 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.197+0000 7f0197fff640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f01980b4260 0x7f019815dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:37418/0 (socket says 172.21.15.88:37418) 2023-10-16T20:07:40.202 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.197+0000 7f0197fff640 1 -- 172.21.15.88:0/1675919831 learned_addr learned my addr 172.21.15.88:0/1675919831 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:40.202 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.197+0000 7f01977fe640 1 --2- 172.21.15.88:0/1675919831 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f01980b0da0 0x7f0198159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:40.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.197+0000 7f0196ffd640 1 -- 172.21.15.88:0/1675919831 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 681074789 0 0) 0x7f019815ad30 con 0x7f01980b4260 2023-10-16T20:07:40.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.197+0000 7f0196ffd640 1 -- 172.21.15.88:0/1675919831 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0180005740 con 0x7f01980b4260 2023-10-16T20:07:40.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.197+0000 7f0196ffd640 1 -- 172.21.15.88:0/1675919831 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 579179513 0 0) 0x7f01980b4640 con 0x7f019815a1f0 2023-10-16T20:07:40.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.197+0000 7f0196ffd640 1 -- 172.21.15.88:0/1675919831 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f019815ad30 con 0x7f019815a1f0 2023-10-16T20:07:40.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.197+0000 7f0196ffd640 1 -- 172.21.15.88:0/1675919831 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1211431742 0 0) 0x7f0180005740 con 0x7f01980b4260 2023-10-16T20:07:40.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.197+0000 7f01977fe640 1 -- 172.21.15.88:0/1675919831 >> v1:172.21.15.94:6789/0 conn(0x7f019815a1f0 legacy=0x7f019806d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:40.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.197+0000 7f01977fe640 1 -- 172.21.15.88:0/1675919831 >> v1:172.21.15.88:6789/0 conn(0x7f01980b4260 legacy=0x7f019815dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:40.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.197+0000 7f01977fe640 1 -- 172.21.15.88:0/1675919831 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f019815aa50 con 0x7f01980b0da0 2023-10-16T20:07:40.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.201+0000 7f01977fe640 1 --2- 172.21.15.88:0/1675919831 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f01980b0da0 0x7f0198159cb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f018c0097f0 tx=0x7f018c01c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=527ac27428feecbb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:40.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.201+0000 7f0196ffd640 1 -- 172.21.15.88:0/1675919831 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0188001680 con 0x7f01980b0da0 2023-10-16T20:07:40.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.201+0000 7f0196ffd640 1 -- 172.21.15.88:0/1675919831 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f018c0a7050 con 0x7f01980b0da0 2023-10-16T20:07:40.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.201+0000 7f0196ffd640 1 -- 172.21.15.88:0/1675919831 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018c0ab620 con 0x7f01980b0da0 2023-10-16T20:07:40.206 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.201+0000 7f019e980640 1 -- 172.21.15.88:0/1675919831 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f01980b0da0 msgr2=0x7f0198159cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:40.206 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.201+0000 7f019e980640 1 --2- 172.21.15.88:0/1675919831 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f01980b0da0 0x7f0198159cb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f018c0097f0 tx=0x7f018c01c350 comp rx=0 tx=0).stop 2023-10-16T20:07:40.206 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.201+0000 7f019e980640 1 -- 172.21.15.88:0/1675919831 shutdown_connections 2023-10-16T20:07:40.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.201+0000 7f019e980640 1 --2- 172.21.15.88:0/1675919831 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f01980b0da0 0x7f0198159cb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:40.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.201+0000 7f019e980640 1 -- 172.21.15.88:0/1675919831 >> 172.21.15.88:0/1675919831 conn(0x7f01980cb080 msgr2=0x7f01980cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:40.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.201+0000 7f019e980640 1 -- 172.21.15.88:0/1675919831 shutdown_connections 2023-10-16T20:07:40.207 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.201+0000 7f019e980640 1 -- 172.21.15.88:0/1675919831 wait complete. 2023-10-16T20:07:40.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.201+0000 7f019e980640 1 Processor -- start 2023-10-16T20:07:40.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.201+0000 7f019e980640 1 -- start start 2023-10-16T20:07:40.211 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.205+0000 7f019e980640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01980b0da0 0x7f01981f5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:40.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.205+0000 7f019e980640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01980b4260 0x7f01980ad180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:40.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.205+0000 7f019e980640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f019815a1f0 0x7f0198139ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:40.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.205+0000 7f019e980640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019813aa50 con 0x7f01980b0da0 2023-10-16T20:07:40.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.205+0000 7f019e980640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019813a490 con 0x7f01980b4260 2023-10-16T20:07:40.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.205+0000 7f019e980640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f019813a770 con 0x7f019815a1f0 2023-10-16T20:07:40.212 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.205+0000 7f0197fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01980b0da0 0x7f01981f5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:40.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.205+0000 7f0197fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01980b0da0 0x7f01981f5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47782/0 (socket says 172.21.15.88:47782) 2023-10-16T20:07:40.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.205+0000 7f0197fff640 1 -- 172.21.15.88:0/3358969573 learned_addr learned my addr 172.21.15.88:0/3358969573 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:40.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.205+0000 7f01977fe640 1 --2- 172.21.15.88:0/3358969573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01980b4260 0x7f01980ad180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:40.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.205+0000 7f019cf1d640 1 --2- 172.21.15.88:0/3358969573 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f019815a1f0 0x7f0198139ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:40.213 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.205+0000 7f0197fff640 1 -- 172.21.15.88:0/3358969573 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f019815a1f0 msgr2=0x7f0198139ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:40.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.205+0000 7f0197fff640 1 --2- 172.21.15.88:0/3358969573 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f019815a1f0 0x7f0198139ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:40.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.205+0000 7f0197fff640 1 -- 172.21.15.88:0/3358969573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01980b4260 msgr2=0x7f01980ad180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:40.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.205+0000 7f0197fff640 1 --2- 172.21.15.88:0/3358969573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01980b4260 0x7f01980ad180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:40.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.205+0000 7f0197fff640 1 -- 172.21.15.88:0/3358969573 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f018c009420 con 0x7f01980b0da0 2023-10-16T20:07:40.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.205+0000 7f0197fff640 1 --2- 172.21.15.88:0/3358969573 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01980b0da0 0x7f01981f5be0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f018800c750 tx=0x7f018800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:40.214 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.205+0000 7f0194ff9640 1 -- 172.21.15.88:0/3358969573 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0188005af0 con 0x7f01980b0da0 2023-10-16T20:07:40.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.205+0000 7f0194ff9640 1 -- 172.21.15.88:0/3358969573 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0188005c50 con 0x7f01980b0da0 2023-10-16T20:07:40.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.205+0000 7f019e980640 1 -- 172.21.15.88:0/3358969573 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01981f53e0 con 0x7f01980b0da0 2023-10-16T20:07:40.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.205+0000 7f0194ff9640 1 -- 172.21.15.88:0/3358969573 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0188007070 con 0x7f01980b0da0 2023-10-16T20:07:40.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.205+0000 7f019e980640 1 -- 172.21.15.88:0/3358969573 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01981f6120 con 0x7f01980b0da0 2023-10-16T20:07:40.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.205+0000 7f019e980640 1 -- 172.21.15.88:0/3358969573 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01981f5240 con 0x7f01980b0da0 2023-10-16T20:07:40.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.209+0000 7f0194ff9640 1 -- 172.21.15.88:0/3358969573 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f01880071d0 con 0x7f01980b0da0 2023-10-16T20:07:40.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.209+0000 7f0194ff9640 1 --2- 172.21.15.88:0/3358969573 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f01840664a0 0x7f0184068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:40.217 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.209+0000 7f0194ff9640 1 -- 172.21.15.88:0/3358969573 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f0188083c90 con 0x7f01980b0da0 2023-10-16T20:07:40.217 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.209+0000 7f01977fe640 1 --2- 172.21.15.88:0/3358969573 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f01840664a0 0x7f0184068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:40.217 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.209+0000 7f01977fe640 1 --2- 172.21.15.88:0/3358969573 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f01840664a0 0x7f0184068960 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f018c0ad000 tx=0x7f018c0a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:40.217 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.213+0000 7f0194ff9640 1 -- 172.21.15.88:0/3358969573 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f01981f5240 con 0x7f01980b0da0 2023-10-16T20:07:40.367 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.361+0000 7f019e980640 1 -- 172.21.15.88:0/3358969573 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f01981f5240 con 0x7f01980b0da0 2023-10-16T20:07:40.368 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.365+0000 7f0194ff9640 1 -- 172.21.15.88:0/3358969573 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7f01981f5240 con 0x7f01980b0da0 2023-10-16T20:07:40.368 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:07:40.370 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.365+0000 7f019e980640 1 -- 172.21.15.88:0/3358969573 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f01840664a0 msgr2=0x7f0184068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:40.370 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.365+0000 7f019e980640 1 --2- 172.21.15.88:0/3358969573 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f01840664a0 0x7f0184068960 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f018c0ad000 tx=0x7f018c0a6000 comp rx=0 tx=0).stop 2023-10-16T20:07:40.370 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.365+0000 7f019e980640 1 -- 172.21.15.88:0/3358969573 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01980b0da0 msgr2=0x7f01981f5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:40.370 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.365+0000 7f019e980640 1 --2- 172.21.15.88:0/3358969573 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01980b0da0 0x7f01981f5be0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f018800c750 tx=0x7f018800cc10 comp rx=0 tx=0).stop 2023-10-16T20:07:40.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.365+0000 7f019e980640 1 -- 172.21.15.88:0/3358969573 shutdown_connections 2023-10-16T20:07:40.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.365+0000 7f019e980640 1 --2- 172.21.15.88:0/3358969573 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f01840664a0 0x7f0184068960 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:40.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.365+0000 7f019e980640 1 --2- 172.21.15.88:0/3358969573 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f019815a1f0 0x7f0198139ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:40.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.365+0000 7f019e980640 1 --2- 172.21.15.88:0/3358969573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01980b4260 0x7f01980ad180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:40.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.365+0000 7f019e980640 1 --2- 172.21.15.88:0/3358969573 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01980b0da0 0x7f01981f5be0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:40.372 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.365+0000 7f019e980640 1 -- 172.21.15.88:0/3358969573 >> 172.21.15.88:0/3358969573 conn(0x7f01980cb080 msgr2=0x7f01981552b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:40.372 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.365+0000 7f019e980640 1 -- 172.21.15.88:0/3358969573 shutdown_connections 2023-10-16T20:07:40.372 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.365+0000 7f019e980640 1 -- 172.21.15.88:0/3358969573 wait complete. 2023-10-16T20:07:40.388 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_metadata pg_num 2023-10-16T20:07:40.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.525+0000 7f646e33e640 1 Processor -- start 2023-10-16T20:07:40.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.525+0000 7f646e33e640 1 -- start start 2023-10-16T20:07:40.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.525+0000 7f646e33e640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f64680b4260 0x7f646815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:40.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.525+0000 7f646e33e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f64680b0da0 0x7f6468159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:40.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.525+0000 7f646e33e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f646815ad30 con 0x7f646815a1f0 2023-10-16T20:07:40.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.525+0000 7f646e33e640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64680b4640 con 0x7f64680b4260 2023-10-16T20:07:40.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.525+0000 7f646e33e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646815a770 con 0x7f64680b0da0 2023-10-16T20:07:40.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.529+0000 7f6467fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f64680b4260 0x7f646815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:40.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.529+0000 7f6467fff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f64680b4260 0x7f646815dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47788/0 (socket says 172.21.15.88:47788) 2023-10-16T20:07:40.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.529+0000 7f6467fff640 1 -- 172.21.15.88:0/2185075235 learned_addr learned my addr 172.21.15.88:0/2185075235 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:40.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.529+0000 7f64677fe640 1 --2- 172.21.15.88:0/2185075235 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f64680b0da0 0x7f6468159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:40.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.529+0000 7f6467fff640 1 -- 172.21.15.88:0/2185075235 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f64680b0da0 msgr2=0x7f6468159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:40.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.529+0000 7f6467fff640 1 --2- 172.21.15.88:0/2185075235 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f64680b0da0 0x7f6468159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:40.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.529+0000 7f6467fff640 1 -- 172.21.15.88:0/2185075235 >> v1:172.21.15.94:6789/0 conn(0x7f646815a1f0 legacy=0x7f646806d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:40.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.529+0000 7f6467fff640 1 -- 172.21.15.88:0/2185075235 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646815aa50 con 0x7f64680b4260 2023-10-16T20:07:40.535 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.529+0000 7f6467fff640 1 --2- 172.21.15.88:0/2185075235 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64680b4260 0x7f646815dc90 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f6458009870 tx=0x7f645801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bca7428a1f8fa33 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:40.535 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.529+0000 7f6466ffd640 1 -- 172.21.15.88:0/2185075235 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64580ab680 con 0x7f64680b4260 2023-10-16T20:07:40.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.533+0000 7f6466ffd640 1 -- 172.21.15.88:0/2185075235 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64580ab7e0 con 0x7f64680b4260 2023-10-16T20:07:40.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.533+0000 7f6466ffd640 1 -- 172.21.15.88:0/2185075235 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64580abb70 con 0x7f64680b4260 2023-10-16T20:07:40.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.533+0000 7f646e33e640 1 -- 172.21.15.88:0/2185075235 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64680b4260 msgr2=0x7f646815dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:40.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.533+0000 7f646e33e640 1 --2- 172.21.15.88:0/2185075235 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64680b4260 0x7f646815dc90 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f6458009870 tx=0x7f645801c480 comp rx=0 tx=0).stop 2023-10-16T20:07:40.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.533+0000 7f646e33e640 1 -- 172.21.15.88:0/2185075235 shutdown_connections 2023-10-16T20:07:40.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.533+0000 7f646e33e640 1 --2- 172.21.15.88:0/2185075235 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f64680b0da0 0x7f6468159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:40.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.533+0000 7f646e33e640 1 --2- 172.21.15.88:0/2185075235 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64680b4260 0x7f646815dc90 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:40.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.533+0000 7f646e33e640 1 -- 172.21.15.88:0/2185075235 >> 172.21.15.88:0/2185075235 conn(0x7f64680cb080 msgr2=0x7f64680cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:40.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.533+0000 7f646e33e640 1 -- 172.21.15.88:0/2185075235 shutdown_connections 2023-10-16T20:07:40.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.533+0000 7f646e33e640 1 -- 172.21.15.88:0/2185075235 wait complete. 2023-10-16T20:07:40.540 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.533+0000 7f646e33e640 1 Processor -- start 2023-10-16T20:07:40.540 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.533+0000 7f646e33e640 1 -- start start 2023-10-16T20:07:40.540 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.533+0000 7f646e33e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64680b0da0 0x7f646814a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:40.540 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.533+0000 7f646e33e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f64680b4260 0x7f64680ab890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:40.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.533+0000 7f646e33e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f646815a1f0 0x7f64680abdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:40.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.533+0000 7f646e33e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646814a680 con 0x7f646815a1f0 2023-10-16T20:07:40.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.533+0000 7f646e33e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6468162870 con 0x7f64680b0da0 2023-10-16T20:07:40.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.533+0000 7f646e33e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646815fe20 con 0x7f64680b4260 2023-10-16T20:07:40.541 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.537+0000 7f646c8db640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f646815a1f0 0x7f64680abdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:40.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.537+0000 7f6467fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64680b0da0 0x7f646814a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:40.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.537+0000 7f6467fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64680b0da0 0x7f646814a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:54916/0 (socket says 172.21.15.88:54916) 2023-10-16T20:07:40.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.537+0000 7f6467fff640 1 -- 172.21.15.88:0/1445955922 learned_addr learned my addr 172.21.15.88:0/1445955922 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:40.542 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.537+0000 7f646c8db640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f646815a1f0 0x7f64680abdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47794/0 (socket says 172.21.15.88:47794) 2023-10-16T20:07:40.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.537+0000 7f64677fe640 1 --2- 172.21.15.88:0/1445955922 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f64680b4260 0x7f64680ab890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:40.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.537+0000 7f646c8db640 1 -- 172.21.15.88:0/1445955922 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f64680b4260 msgr2=0x7f64680ab890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:40.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.537+0000 7f646c8db640 1 --2- 172.21.15.88:0/1445955922 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f64680b4260 0x7f64680ab890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:40.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.537+0000 7f646c8db640 1 -- 172.21.15.88:0/1445955922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64680b0da0 msgr2=0x7f646814a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:40.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.537+0000 7f646c8db640 1 --2- 172.21.15.88:0/1445955922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64680b0da0 0x7f646814a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:40.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.537+0000 7f646c8db640 1 -- 172.21.15.88:0/1445955922 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64580094d0 con 0x7f646815a1f0 2023-10-16T20:07:40.544 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.537+0000 7f646c8db640 1 --2- 172.21.15.88:0/1445955922 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f646815a1f0 0x7f64680abdd0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f646000c700 tx=0x7f646000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:40.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.537+0000 7f6464ff9640 1 -- 172.21.15.88:0/1445955922 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6460003b80 con 0x7f646815a1f0 2023-10-16T20:07:40.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.537+0000 7f6464ff9640 1 -- 172.21.15.88:0/1445955922 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6460003ce0 con 0x7f646815a1f0 2023-10-16T20:07:40.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.537+0000 7f6464ff9640 1 -- 172.21.15.88:0/1445955922 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646000aa30 con 0x7f646815a1f0 2023-10-16T20:07:40.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.537+0000 7f646e33e640 1 -- 172.21.15.88:0/1445955922 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f646815f2a0 con 0x7f646815a1f0 2023-10-16T20:07:40.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.537+0000 7f646e33e640 1 -- 172.21.15.88:0/1445955922 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f646815f860 con 0x7f646815a1f0 2023-10-16T20:07:40.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.537+0000 7f646e33e640 1 -- 172.21.15.88:0/1445955922 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6468162a00 con 0x7f646815a1f0 2023-10-16T20:07:40.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.537+0000 7f6464ff9640 1 -- 172.21.15.88:0/1445955922 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f646000ab90 con 0x7f646815a1f0 2023-10-16T20:07:40.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.537+0000 7f6464ff9640 1 --2- 172.21.15.88:0/1445955922 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f64540664a0 0x7f6454068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:40.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.537+0000 7f6464ff9640 1 -- 172.21.15.88:0/1445955922 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f6460083d20 con 0x7f646815a1f0 2023-10-16T20:07:40.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.541+0000 7f6467fff640 1 --2- 172.21.15.88:0/1445955922 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f64540664a0 0x7f6454068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:40.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.541+0000 7f6467fff640 1 --2- 172.21.15.88:0/1445955922 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f64540664a0 0x7f6454068960 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f645801a0b0 tx=0x7f64580b6040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:40.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.541+0000 7f6464ff9640 1 -- 172.21.15.88:0/1445955922 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6468162a00 con 0x7f646815a1f0 2023-10-16T20:07:40.695 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.689+0000 7f646e33e640 1 -- 172.21.15.88:0/1445955922 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"} v 0) v1 -- 0x7f646815ce80 con 0x7f646815a1f0 2023-10-16T20:07:40.696 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.693+0000 7f6464ff9640 1 -- 172.21.15.88:0/1445955922 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"}]=0 v58) v1 ==== 109+0+11 (secure 0 0 0) 0x7f6460050fc0 con 0x7f646815a1f0 2023-10-16T20:07:40.696 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:07:40.697 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.693+0000 7f646e33e640 1 -- 172.21.15.88:0/1445955922 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f64540664a0 msgr2=0x7f6454068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:40.698 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.693+0000 7f646e33e640 1 --2- 172.21.15.88:0/1445955922 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f64540664a0 0x7f6454068960 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f645801a0b0 tx=0x7f64580b6040 comp rx=0 tx=0).stop 2023-10-16T20:07:40.698 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.693+0000 7f646e33e640 1 -- 172.21.15.88:0/1445955922 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f646815a1f0 msgr2=0x7f64680abdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:40.698 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.693+0000 7f646e33e640 1 --2- 172.21.15.88:0/1445955922 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f646815a1f0 0x7f64680abdd0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f646000c700 tx=0x7f646000cbc0 comp rx=0 tx=0).stop 2023-10-16T20:07:40.698 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.693+0000 7f646e33e640 1 -- 172.21.15.88:0/1445955922 shutdown_connections 2023-10-16T20:07:40.698 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.693+0000 7f646e33e640 1 --2- 172.21.15.88:0/1445955922 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f64540664a0 0x7f6454068960 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:40.699 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.693+0000 7f646e33e640 1 --2- 172.21.15.88:0/1445955922 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f64680b4260 0x7f64680ab890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:40.699 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.693+0000 7f646e33e640 1 --2- 172.21.15.88:0/1445955922 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f646815a1f0 0x7f64680abdd0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:40.699 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.693+0000 7f646e33e640 1 --2- 172.21.15.88:0/1445955922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64680b0da0 0x7f646814a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:40.699 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.693+0000 7f646e33e640 1 -- 172.21.15.88:0/1445955922 >> 172.21.15.88:0/1445955922 conn(0x7f64680cb080 msgr2=0x7f64680aeda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:40.699 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.693+0000 7f646e33e640 1 -- 172.21.15.88:0/1445955922 shutdown_connections 2023-10-16T20:07:40.699 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.693+0000 7f646e33e640 1 -- 172.21.15.88:0/1445955922 wait complete. 2023-10-16T20:07:40.717 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_data pg_num 2023-10-16T20:07:40.858 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.853+0000 7f033c380640 1 Processor -- start 2023-10-16T20:07:40.858 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.853+0000 7f033c380640 1 -- start start 2023-10-16T20:07:40.860 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.853+0000 7f033c380640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f0334154f30 0x7f03340b0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:40.860 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.853+0000 7f033c380640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f03340b0a40 0x7f033415aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:40.860 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.853+0000 7f033c380640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f033415b530 0x7f03340b1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:40.861 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.853+0000 7f033c380640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03340b0ef0 con 0x7f0334154f30 2023-10-16T20:07:40.861 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.853+0000 7f033c380640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0334155310 con 0x7f03340b0a40 2023-10-16T20:07:40.861 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.853+0000 7f033c380640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033415bd20 con 0x7f033415b530 2023-10-16T20:07:40.861 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.857+0000 7f033a11c640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f0334154f30 0x7f03340b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:40.861 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.857+0000 7f033a11c640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f0334154f30 0x7f03340b0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47810/0 (socket says 172.21.15.88:47810) 2023-10-16T20:07:40.862 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.857+0000 7f033a11c640 1 -- 172.21.15.88:0/3593830655 learned_addr learned my addr 172.21.15.88:0/3593830655 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:40.862 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.857+0000 7f033991b640 1 --2- 172.21.15.88:0/3593830655 >> v2:172.21.15.94:3300/0 conn(0x7f03340b0a40 0x7f033415aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:40.862 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.857+0000 7f033a91d640 1 --2- 172.21.15.88:0/3593830655 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f033415b530 0x7f03340b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:40.862 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.857+0000 7f033a11c640 1 -- 172.21.15.88:0/3593830655 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f033415b530 msgr2=0x7f03340b1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:40.862 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.857+0000 7f033a11c640 1 --2- 172.21.15.88:0/3593830655 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f033415b530 0x7f03340b1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:40.862 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.857+0000 7f033a11c640 1 -- 172.21.15.88:0/3593830655 >> v2:172.21.15.94:3300/0 conn(0x7f03340b0a40 msgr2=0x7f033415aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:40.863 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.857+0000 7f033a11c640 1 --2- 172.21.15.88:0/3593830655 >> v2:172.21.15.94:3300/0 conn(0x7f03340b0a40 0x7f033415aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:40.863 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.857+0000 7f033a11c640 1 -- 172.21.15.88:0/3593830655 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03340702f0 con 0x7f0334154f30 2023-10-16T20:07:40.864 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.857+0000 7f033a11c640 1 --2- 172.21.15.88:0/3593830655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0334154f30 0x7f03340b0500 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f03340cb020 tx=0x7f033001c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5baec430ef7318 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:40.864 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.857+0000 7f033911a640 1 -- 172.21.15.88:0/3593830655 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03300ab600 con 0x7f0334154f30 2023-10-16T20:07:40.864 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.857+0000 7f033911a640 1 -- 172.21.15.88:0/3593830655 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03300a7050 con 0x7f0334154f30 2023-10-16T20:07:40.865 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.857+0000 7f033911a640 1 -- 172.21.15.88:0/3593830655 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03300ab980 con 0x7f0334154f30 2023-10-16T20:07:40.865 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.861+0000 7f033c380640 1 -- 172.21.15.88:0/3593830655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0334154f30 msgr2=0x7f03340b0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:40.865 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.861+0000 7f033c380640 1 --2- 172.21.15.88:0/3593830655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0334154f30 0x7f03340b0500 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f03340cb020 tx=0x7f033001c210 comp rx=0 tx=0).stop 2023-10-16T20:07:40.865 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.861+0000 7f033c380640 1 -- 172.21.15.88:0/3593830655 shutdown_connections 2023-10-16T20:07:40.865 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.861+0000 7f033c380640 1 --2- 172.21.15.88:0/3593830655 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f033415b530 0x7f03340b1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:40.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.861+0000 7f033c380640 1 --2- 172.21.15.88:0/3593830655 >> v2:172.21.15.94:3300/0 conn(0x7f03340b0a40 0x7f033415aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:40.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.861+0000 7f033c380640 1 --2- 172.21.15.88:0/3593830655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0334154f30 0x7f03340b0500 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:40.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.861+0000 7f033c380640 1 -- 172.21.15.88:0/3593830655 >> 172.21.15.88:0/3593830655 conn(0x7f03340cb080 msgr2=0x7f03340cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:40.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.861+0000 7f033c380640 1 -- 172.21.15.88:0/3593830655 shutdown_connections 2023-10-16T20:07:40.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.861+0000 7f033c380640 1 -- 172.21.15.88:0/3593830655 wait complete. 2023-10-16T20:07:40.868 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.861+0000 7f033c380640 1 Processor -- start 2023-10-16T20:07:40.868 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.861+0000 7f033c380640 1 -- start start 2023-10-16T20:07:40.869 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.861+0000 7f033c380640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f03340b0a40 0x7f033414f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:40.869 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.861+0000 7f033c380640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0334154f30 0x7f033414e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:40.869 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.861+0000 7f033c380640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f033415b530 0x7f03341516c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:40.869 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.861+0000 7f033c380640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03340aa220 con 0x7f033415b530 2023-10-16T20:07:40.869 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.861+0000 7f033c380640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03341506b0 con 0x7f0334154f30 2023-10-16T20:07:40.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.861+0000 7f033c380640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0334151230 con 0x7f03340b0a40 2023-10-16T20:07:40.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.861+0000 7f033a91d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f033415b530 0x7f03341516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:40.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.861+0000 7f033a91d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f033415b530 0x7f03341516c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47826/0 (socket says 172.21.15.88:47826) 2023-10-16T20:07:40.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.861+0000 7f033a91d640 1 -- 172.21.15.88:0/2020394973 learned_addr learned my addr 172.21.15.88:0/2020394973 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:40.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.861+0000 7f033991b640 1 --2- 172.21.15.88:0/2020394973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0334154f30 0x7f033414e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:40.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.861+0000 7f033a11c640 1 --2- 172.21.15.88:0/2020394973 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f03340b0a40 0x7f033414f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:40.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.865+0000 7f033a91d640 1 -- 172.21.15.88:0/2020394973 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f03340b0a40 msgr2=0x7f033414f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:40.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.865+0000 7f033a91d640 1 --2- 172.21.15.88:0/2020394973 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f03340b0a40 0x7f033414f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:40.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.865+0000 7f033a91d640 1 -- 172.21.15.88:0/2020394973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0334154f30 msgr2=0x7f033414e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:40.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.865+0000 7f033a91d640 1 --2- 172.21.15.88:0/2020394973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0334154f30 0x7f033414e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:40.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.865+0000 7f033a91d640 1 -- 172.21.15.88:0/2020394973 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03300094d0 con 0x7f033415b530 2023-10-16T20:07:40.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.865+0000 7f033a91d640 1 --2- 172.21.15.88:0/2020394973 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f033415b530 0x7f03341516c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f032c007e90 tx=0x7f032c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:40.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.865+0000 7f032affd640 1 -- 172.21.15.88:0/2020394973 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c014420 con 0x7f033415b530 2023-10-16T20:07:40.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.865+0000 7f032affd640 1 -- 172.21.15.88:0/2020394973 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032c014580 con 0x7f033415b530 2023-10-16T20:07:40.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.865+0000 7f033c380640 1 -- 172.21.15.88:0/2020394973 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0334150900 con 0x7f033415b530 2023-10-16T20:07:40.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.865+0000 7f032affd640 1 -- 172.21.15.88:0/2020394973 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c014930 con 0x7f033415b530 2023-10-16T20:07:40.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.865+0000 7f033c380640 1 -- 172.21.15.88:0/2020394973 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0334150ec0 con 0x7f033415b530 2023-10-16T20:07:40.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.865+0000 7f033c380640 1 -- 172.21.15.88:0/2020394973 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f033404e9c0 con 0x7f033415b530 2023-10-16T20:07:40.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.865+0000 7f032affd640 1 -- 172.21.15.88:0/2020394973 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f032c014a90 con 0x7f033415b530 2023-10-16T20:07:40.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.865+0000 7f032affd640 1 --2- 172.21.15.88:0/2020394973 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f03180664a0 0x7f0318068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:40.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.865+0000 7f032affd640 1 -- 172.21.15.88:0/2020394973 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f032c084700 con 0x7f033415b530 2023-10-16T20:07:40.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.865+0000 7f033a11c640 1 --2- 172.21.15.88:0/2020394973 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f03180664a0 0x7f0318068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:40.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.869+0000 7f033a11c640 1 --2- 172.21.15.88:0/2020394973 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f03180664a0 0x7f0318068960 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f03300ad000 tx=0x7f03300a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:40.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:40.869+0000 7f032affd640 1 -- 172.21.15.88:0/2020394973 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f032c051ac0 con 0x7f033415b530 2023-10-16T20:07:41.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.017+0000 7f033c380640 1 -- 172.21.15.88:0/2020394973 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"} v 0) v1 -- 0x7f0334151370 con 0x7f033415b530 2023-10-16T20:07:41.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.017+0000 7f032affd640 1 -- 172.21.15.88:0/2020394973 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"}]=0 v58) v1 ==== 105+0+11 (secure 0 0 0) 0x7f032c0550e0 con 0x7f033415b530 2023-10-16T20:07:41.025 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2023-10-16T20:07:41.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.021+0000 7f033c380640 1 -- 172.21.15.88:0/2020394973 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f03180664a0 msgr2=0x7f0318068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:41.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.021+0000 7f033c380640 1 --2- 172.21.15.88:0/2020394973 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f03180664a0 0x7f0318068960 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f03300ad000 tx=0x7f03300a6000 comp rx=0 tx=0).stop 2023-10-16T20:07:41.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.021+0000 7f033c380640 1 -- 172.21.15.88:0/2020394973 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f033415b530 msgr2=0x7f03341516c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:41.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.021+0000 7f033c380640 1 --2- 172.21.15.88:0/2020394973 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f033415b530 0x7f03341516c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f032c007e90 tx=0x7f032c00e4b0 comp rx=0 tx=0).stop 2023-10-16T20:07:41.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.021+0000 7f033c380640 1 -- 172.21.15.88:0/2020394973 shutdown_connections 2023-10-16T20:07:41.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.021+0000 7f033c380640 1 --2- 172.21.15.88:0/2020394973 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f03180664a0 0x7f0318068960 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:41.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.021+0000 7f033c380640 1 --2- 172.21.15.88:0/2020394973 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f033415b530 0x7f03341516c0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:41.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.021+0000 7f033c380640 1 --2- 172.21.15.88:0/2020394973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0334154f30 0x7f033414e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:41.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.021+0000 7f033c380640 1 --2- 172.21.15.88:0/2020394973 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f03340b0a40 0x7f033414f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:41.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.021+0000 7f033c380640 1 -- 172.21.15.88:0/2020394973 >> 172.21.15.88:0/2020394973 conn(0x7f03340cb080 msgr2=0x7f0334153220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:41.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.021+0000 7f033c380640 1 -- 172.21.15.88:0/2020394973 shutdown_connections 2023-10-16T20:07:41.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.021+0000 7f033c380640 1 -- 172.21.15.88:0/2020394973 wait complete. 2023-10-16T20:07:41.044 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T20:07:41.182 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.177+0000 7f6e09f65640 1 Processor -- start 2023-10-16T20:07:41.182 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.177+0000 7f6e09f65640 1 -- start start 2023-10-16T20:07:41.184 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.177+0000 7f6e09f65640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f6e04069e90 0x7f6e0406a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:41.184 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.177+0000 7f6e09f65640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6e040adf80 0x7f6e0415a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:41.184 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.177+0000 7f6e09f65640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f6e0415d480 0x7f6e040afa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:41.184 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.177+0000 7f6e09f65640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e04159e40 con 0x7f6e04069e90 2023-10-16T20:07:41.184 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.177+0000 7f6e09f65640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e0415e400 con 0x7f6e0415d480 2023-10-16T20:07:41.185 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.177+0000 7f6e09f65640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0406a7b0 con 0x7f6e040adf80 2023-10-16T20:07:41.185 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.177+0000 7f6e037fe640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f6e04069e90 0x7f6e0406a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:41.185 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.181+0000 7f6e037fe640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f6e04069e90 0x7f6e0406a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47832/0 (socket says 172.21.15.88:47832) 2023-10-16T20:07:41.185 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.181+0000 7f6e037fe640 1 -- 172.21.15.88:0/3951057437 learned_addr learned my addr 172.21.15.88:0/3951057437 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:41.185 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.181+0000 7f6e037fe640 1 -- 172.21.15.88:0/3951057437 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6e040adf80 msgr2=0x7f6e0415a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:41.185 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.181+0000 7f6e037fe640 1 --2- 172.21.15.88:0/3951057437 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6e040adf80 0x7f6e0415a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:41.186 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.181+0000 7f6e03fff640 1 --2- 172.21.15.88:0/3951057437 >> v2:172.21.15.94:3300/0 conn(0x7f6e0415d480 0x7f6e040afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:41.186 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.181+0000 7f6e037fe640 1 -- 172.21.15.88:0/3951057437 >> v2:172.21.15.94:3300/0 conn(0x7f6e0415d480 msgr2=0x7f6e040afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:41.186 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.181+0000 7f6e037fe640 1 --2- 172.21.15.88:0/3951057437 >> v2:172.21.15.94:3300/0 conn(0x7f6e0415d480 0x7f6e040afa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:41.186 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.181+0000 7f6e037fe640 1 -- 172.21.15.88:0/3951057437 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0415de40 con 0x7f6e04069e90 2023-10-16T20:07:41.186 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.181+0000 7f6e03fff640 1 --2- 172.21.15.88:0/3951057437 >> v2:172.21.15.94:3300/0 conn(0x7f6e0415d480 0x7f6e040afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:07:41.187 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.181+0000 7f6e037fe640 1 --2- 172.21.15.88:0/3951057437 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e04069e90 0x7f6e0406a270 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f6e040725c0 tx=0x7f6df401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a665cf3632793f7b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:41.187 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.181+0000 7f6e02ffd640 1 -- 172.21.15.88:0/3951057437 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df40aa5f0 con 0x7f6e04069e90 2023-10-16T20:07:41.188 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.181+0000 7f6e02ffd640 1 -- 172.21.15.88:0/3951057437 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df40aa750 con 0x7f6e04069e90 2023-10-16T20:07:41.188 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.181+0000 7f6e02ffd640 1 -- 172.21.15.88:0/3951057437 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df40aaae0 con 0x7f6e04069e90 2023-10-16T20:07:41.189 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.185+0000 7f6e09f65640 1 -- 172.21.15.88:0/3951057437 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e04069e90 msgr2=0x7f6e0406a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:41.189 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.185+0000 7f6e09f65640 1 --2- 172.21.15.88:0/3951057437 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e04069e90 0x7f6e0406a270 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f6e040725c0 tx=0x7f6df401c4d0 comp rx=0 tx=0).stop 2023-10-16T20:07:41.189 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.185+0000 7f6e09f65640 1 -- 172.21.15.88:0/3951057437 shutdown_connections 2023-10-16T20:07:41.189 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.185+0000 7f6e09f65640 1 --2- 172.21.15.88:0/3951057437 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6e040adf80 0x7f6e0415a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:41.189 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.185+0000 7f6e09f65640 1 --2- 172.21.15.88:0/3951057437 >> v2:172.21.15.94:3300/0 conn(0x7f6e0415d480 0x7f6e040afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:41.190 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.185+0000 7f6e09f65640 1 --2- 172.21.15.88:0/3951057437 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e04069e90 0x7f6e0406a270 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:41.190 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.185+0000 7f6e09f65640 1 -- 172.21.15.88:0/3951057437 >> 172.21.15.88:0/3951057437 conn(0x7f6e040ce5e0 msgr2=0x7f6e040ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:41.190 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.185+0000 7f6e09f65640 1 -- 172.21.15.88:0/3951057437 shutdown_connections 2023-10-16T20:07:41.190 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.185+0000 7f6e09f65640 1 -- 172.21.15.88:0/3951057437 wait complete. 2023-10-16T20:07:41.192 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.185+0000 7f6e09f65640 1 Processor -- start 2023-10-16T20:07:41.193 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.185+0000 7f6e09f65640 1 -- start start 2023-10-16T20:07:41.193 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.185+0000 7f6e09f65640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6e04069e90 0x7f6e0415c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:41.193 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.185+0000 7f6e09f65640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e040adf80 0x7f6e041f9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:41.193 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.185+0000 7f6e09f65640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e0415d480 0x7f6e04071a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:41.193 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.185+0000 7f6e09f65640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e041f99e0 con 0x7f6e040adf80 2023-10-16T20:07:41.194 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.185+0000 7f6e09f65640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0415cc30 con 0x7f6e0415d480 2023-10-16T20:07:41.194 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.185+0000 7f6e09f65640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0415cf10 con 0x7f6e04069e90 2023-10-16T20:07:41.194 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.185+0000 7f6dfbfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e040adf80 0x7f6e041f9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:41.194 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.185+0000 7f6dfbfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e040adf80 0x7f6e041f9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47844/0 (socket says 172.21.15.88:47844) 2023-10-16T20:07:41.194 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.185+0000 7f6dfbfff640 1 -- 172.21.15.88:0/3665445836 learned_addr learned my addr 172.21.15.88:0/3665445836 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:41.195 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.185+0000 7f6e037fe640 1 --2- 172.21.15.88:0/3665445836 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6e04069e90 0x7f6e0415c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:41.195 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.185+0000 7f6e03fff640 1 --2- 172.21.15.88:0/3665445836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e0415d480 0x7f6e04071a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:41.195 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.185+0000 7f6dfbfff640 1 -- 172.21.15.88:0/3665445836 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6e04069e90 msgr2=0x7f6e0415c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:41.195 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.185+0000 7f6dfbfff640 1 --2- 172.21.15.88:0/3665445836 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6e04069e90 0x7f6e0415c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:41.195 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.185+0000 7f6dfbfff640 1 -- 172.21.15.88:0/3665445836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e0415d480 msgr2=0x7f6e04071a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:41.196 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.185+0000 7f6dfbfff640 1 --2- 172.21.15.88:0/3665445836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e0415d480 0x7f6e04071a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:41.196 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.185+0000 7f6dfbfff640 1 -- 172.21.15.88:0/3665445836 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df40094d0 con 0x7f6e040adf80 2023-10-16T20:07:41.196 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.189+0000 7f6dfbfff640 1 --2- 172.21.15.88:0/3665445836 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e040adf80 0x7f6e041f9e70 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6dec00b410 tx=0x7f6dec00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:41.196 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.189+0000 7f6e00ff9640 1 -- 172.21.15.88:0/3665445836 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec011830 con 0x7f6e040adf80 2023-10-16T20:07:41.196 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.189+0000 7f6e00ff9640 1 -- 172.21.15.88:0/3665445836 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dec011990 con 0x7f6e040adf80 2023-10-16T20:07:41.196 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.189+0000 7f6e00ff9640 1 -- 172.21.15.88:0/3665445836 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec011d40 con 0x7f6e040adf80 2023-10-16T20:07:41.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.189+0000 7f6e09f65640 1 -- 172.21.15.88:0/3665445836 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e041f9390 con 0x7f6e040adf80 2023-10-16T20:07:41.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.189+0000 7f6e09f65640 1 -- 172.21.15.88:0/3665445836 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e04071fd0 con 0x7f6e040adf80 2023-10-16T20:07:41.198 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.189+0000 7f6e09f65640 1 -- 172.21.15.88:0/3665445836 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e0406d180 con 0x7f6e040adf80 2023-10-16T20:07:41.198 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.189+0000 7f6e00ff9640 1 -- 172.21.15.88:0/3665445836 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6dec01a400 con 0x7f6e040adf80 2023-10-16T20:07:41.198 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.189+0000 7f6e00ff9640 1 --2- 172.21.15.88:0/3665445836 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f6de40664a0 0x7f6de4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:41.198 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.189+0000 7f6e00ff9640 1 -- 172.21.15.88:0/3665445836 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f6dec084060 con 0x7f6e040adf80 2023-10-16T20:07:41.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.189+0000 7f6e037fe640 1 --2- 172.21.15.88:0/3665445836 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f6de40664a0 0x7f6de4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:41.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.193+0000 7f6e037fe640 1 --2- 172.21.15.88:0/3665445836 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f6de40664a0 0x7f6de4068960 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6df40ae000 tx=0x7f6df40b5040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:41.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.193+0000 7f6e00ff9640 1 -- 172.21.15.88:0/3665445836 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6dec0b5090 con 0x7f6e040adf80 2023-10-16T20:07:41.364 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.361+0000 7f6e09f65640 1 -- 172.21.15.88:0/3665445836 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6e0404e9c0 con 0x7f6e040adf80 2023-10-16T20:07:41.364 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.361+0000 7f6e00ff9640 1 -- 172.21.15.88:0/3665445836 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v58) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6dec015080 con 0x7f6e040adf80 2023-10-16T20:07:41.364 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2023-10-16T20:07:41.366 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.361+0000 7f6e09f65640 1 -- 172.21.15.88:0/3665445836 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f6de40664a0 msgr2=0x7f6de4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:41.366 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.361+0000 7f6e09f65640 1 --2- 172.21.15.88:0/3665445836 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f6de40664a0 0x7f6de4068960 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6df40ae000 tx=0x7f6df40b5040 comp rx=0 tx=0).stop 2023-10-16T20:07:41.366 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.361+0000 7f6e09f65640 1 -- 172.21.15.88:0/3665445836 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e040adf80 msgr2=0x7f6e041f9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:41.366 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.361+0000 7f6e09f65640 1 --2- 172.21.15.88:0/3665445836 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e040adf80 0x7f6e041f9e70 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6dec00b410 tx=0x7f6dec00b8d0 comp rx=0 tx=0).stop 2023-10-16T20:07:41.367 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.361+0000 7f6e09f65640 1 -- 172.21.15.88:0/3665445836 shutdown_connections 2023-10-16T20:07:41.367 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.361+0000 7f6e09f65640 1 --2- 172.21.15.88:0/3665445836 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f6de40664a0 0x7f6de4068960 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:41.367 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.361+0000 7f6e09f65640 1 --2- 172.21.15.88:0/3665445836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e0415d480 0x7f6e04071a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:41.367 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.361+0000 7f6e09f65640 1 --2- 172.21.15.88:0/3665445836 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e040adf80 0x7f6e041f9e70 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:41.367 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.361+0000 7f6e09f65640 1 --2- 172.21.15.88:0/3665445836 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6e04069e90 0x7f6e0415c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:41.368 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.361+0000 7f6e09f65640 1 -- 172.21.15.88:0/3665445836 >> 172.21.15.88:0/3665445836 conn(0x7f6e040ce5e0 msgr2=0x7f6e04072920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:41.368 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.361+0000 7f6e09f65640 1 -- 172.21.15.88:0/3665445836 shutdown_connections 2023-10-16T20:07:41.368 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.361+0000 7f6e09f65640 1 -- 172.21.15.88:0/3665445836 wait complete. 2023-10-16T20:07:41.385 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:07:41.385 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:07:41.395 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:07:41.573 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.569+0000 7fcb41756640 1 Processor -- start 2023-10-16T20:07:41.573 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.569+0000 7fcb41756640 1 -- start start 2023-10-16T20:07:41.574 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.569+0000 7fcb41756640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fcb3c0cf940 0x7fcb3c0d3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:41.574 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.569+0000 7fcb41756640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb3c0d0c40 con 0x7fcb3c0cde80 2023-10-16T20:07:41.575 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.569+0000 7fcb41756640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb3c0ce2d0 con 0x7fcb3c15c9a0 2023-10-16T20:07:41.575 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.569+0000 7fcb41756640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb3c0d37b0 con 0x7fcb3c0cf940 2023-10-16T20:07:41.575 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.569+0000 7fcb3b7fe640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fcb3c0cde80 0x7fcb3c0cf230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:37426/0 (socket says 172.21.15.88:37426) 2023-10-16T20:07:41.575 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.569+0000 7fcb3b7fe640 1 -- 172.21.15.88:0/3817826663 learned_addr learned my addr 172.21.15.88:0/3817826663 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:41.575 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.569+0000 7fcb40f55640 1 --2- 172.21.15.88:0/3817826663 >> v2:172.21.15.94:3300/0 conn(0x7fcb3c0cf940 0x7fcb3c0d3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:41.576 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.569+0000 7fcb3affd640 1 -- 172.21.15.88:0/3817826663 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 495344468 0 0) 0x7fcb3c0d0c40 con 0x7fcb3c0cde80 2023-10-16T20:07:41.576 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.569+0000 7fcb3affd640 1 -- 172.21.15.88:0/3817826663 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb24003530 con 0x7fcb3c0cde80 2023-10-16T20:07:41.577 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.569+0000 7fcb3affd640 1 -- 172.21.15.88:0/3817826663 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3824498637 0 0) 0x7fcb3c0ce2d0 con 0x7fcb3c15c9a0 2023-10-16T20:07:41.577 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.569+0000 7fcb3affd640 1 -- 172.21.15.88:0/3817826663 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb3c0d0c40 con 0x7fcb3c15c9a0 2023-10-16T20:07:41.577 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.569+0000 7fcb40f55640 1 -- 172.21.15.88:0/3817826663 >> v1:172.21.15.94:6789/0 conn(0x7fcb3c15c9a0 legacy=0x7fcb3c0d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:41.577 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.569+0000 7fcb40f55640 1 -- 172.21.15.88:0/3817826663 >> v1:172.21.15.88:6789/0 conn(0x7fcb3c0cde80 legacy=0x7fcb3c0cf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:41.577 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.569+0000 7fcb40f55640 1 -- 172.21.15.88:0/3817826663 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb3c0d0630 con 0x7fcb3c0cf940 2023-10-16T20:07:41.578 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.573+0000 7fcb40f55640 1 --2- 172.21.15.88:0/3817826663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb3c0cf940 0x7fcb3c0d3270 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009d90 tx=0x7fcb3401c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ba5965a79ce8624 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:41.578 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.573+0000 7fcb3affd640 1 -- 172.21.15.88:0/3817826663 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb340a5870 con 0x7fcb3c0cf940 2023-10-16T20:07:41.578 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.573+0000 7fcb3affd640 1 -- 172.21.15.88:0/3817826663 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb340a59d0 con 0x7fcb3c0cf940 2023-10-16T20:07:41.579 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.573+0000 7fcb3affd640 1 -- 172.21.15.88:0/3817826663 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb340a5d60 con 0x7fcb3c0cf940 2023-10-16T20:07:41.579 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.573+0000 7fcb41756640 1 -- 172.21.15.88:0/3817826663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb3c0cf940 msgr2=0x7fcb3c0d3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:41.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.573+0000 7fcb41756640 1 --2- 172.21.15.88:0/3817826663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb3c0cf940 0x7fcb3c0d3270 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009d90 tx=0x7fcb3401c8c0 comp rx=0 tx=0).stop 2023-10-16T20:07:41.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.573+0000 7fcb41756640 1 -- 172.21.15.88:0/3817826663 shutdown_connections 2023-10-16T20:07:41.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.573+0000 7fcb41756640 1 --2- 172.21.15.88:0/3817826663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb3c0cf940 0x7fcb3c0d3270 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:41.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.573+0000 7fcb41756640 1 -- 172.21.15.88:0/3817826663 >> 172.21.15.88:0/3817826663 conn(0x7fcb3c09bc30 msgr2=0x7fcb3c0d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:41.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.573+0000 7fcb41756640 1 -- 172.21.15.88:0/3817826663 shutdown_connections 2023-10-16T20:07:41.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.573+0000 7fcb41756640 1 -- 172.21.15.88:0/3817826663 wait complete. 2023-10-16T20:07:41.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.577+0000 7fcb41756640 1 Processor -- start 2023-10-16T20:07:41.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.577+0000 7fcb41756640 1 -- start start 2023-10-16T20:07:41.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.577+0000 7fcb41756640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb3c0cde80 0x7fcb3c15f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:41.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.577+0000 7fcb41756640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcb3c0cf940 0x7fcb3c15d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:41.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.577+0000 7fcb41756640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb3c15c9a0 0x7fcb3c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:41.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.577+0000 7fcb41756640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c155310 con 0x7fcb3c15c9a0 2023-10-16T20:07:41.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.577+0000 7fcb41756640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c15f7d0 con 0x7fcb3c0cde80 2023-10-16T20:07:41.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.577+0000 7fcb41756640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c15fab0 con 0x7fcb3c0cf940 2023-10-16T20:07:41.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.577+0000 7fcb40f55640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb3c15c9a0 0x7fcb3c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:41.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.577+0000 7fcb40f55640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb3c15c9a0 0x7fcb3c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47854/0 (socket says 172.21.15.88:47854) 2023-10-16T20:07:41.584 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.577+0000 7fcb40f55640 1 -- 172.21.15.88:0/3523494488 learned_addr learned my addr 172.21.15.88:0/3523494488 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:41.584 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.577+0000 7fcb3bfff640 1 --2- 172.21.15.88:0/3523494488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb3c0cde80 0x7fcb3c15f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:41.584 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.577+0000 7fcb3b7fe640 1 --2- 172.21.15.88:0/3523494488 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcb3c0cf940 0x7fcb3c15d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:41.585 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.577+0000 7fcb40f55640 1 -- 172.21.15.88:0/3523494488 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcb3c0cf940 msgr2=0x7fcb3c15d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:41.585 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.577+0000 7fcb40f55640 1 --2- 172.21.15.88:0/3523494488 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcb3c0cf940 0x7fcb3c15d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:41.586 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.577+0000 7fcb40f55640 1 -- 172.21.15.88:0/3523494488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb3c0cde80 msgr2=0x7fcb3c15f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:41.586 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.577+0000 7fcb40f55640 1 --2- 172.21.15.88:0/3523494488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb3c0cde80 0x7fcb3c15f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:41.586 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.577+0000 7fcb40f55640 1 -- 172.21.15.88:0/3523494488 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb34009a70 con 0x7fcb3c15c9a0 2023-10-16T20:07:41.586 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.577+0000 7fcb40f55640 1 --2- 172.21.15.88:0/3523494488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb3c15c9a0 0x7fcb3c15d5d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fcb340b0040 tx=0x7fcb34019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:41.586 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.577+0000 7fcb38ff9640 1 -- 172.21.15.88:0/3523494488 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb340b9860 con 0x7fcb3c15c9a0 2023-10-16T20:07:41.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.577+0000 7fcb38ff9640 1 -- 172.21.15.88:0/3523494488 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb340b99c0 con 0x7fcb3c15c9a0 2023-10-16T20:07:41.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.577+0000 7fcb38ff9640 1 -- 172.21.15.88:0/3523494488 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb340b9d70 con 0x7fcb3c15c9a0 2023-10-16T20:07:41.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.577+0000 7fcb41756640 1 -- 172.21.15.88:0/3523494488 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb3c15cf00 con 0x7fcb3c15c9a0 2023-10-16T20:07:41.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.577+0000 7fcb41756640 1 -- 172.21.15.88:0/3523494488 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb3c0b54b0 con 0x7fcb3c15c9a0 2023-10-16T20:07:41.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.581+0000 7fcb41756640 1 -- 172.21.15.88:0/3523494488 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb3c0674f0 con 0x7fcb3c15c9a0 2023-10-16T20:07:41.587 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.581+0000 7fcb38ff9640 1 -- 172.21.15.88:0/3523494488 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcb340ad440 con 0x7fcb3c15c9a0 2023-10-16T20:07:41.588 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.581+0000 7fcb38ff9640 1 --2- 172.21.15.88:0/3523494488 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcb280664a0 0x7fcb28068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:41.588 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.581+0000 7fcb38ff9640 1 -- 172.21.15.88:0/3523494488 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fcb3411f2b0 con 0x7fcb3c15c9a0 2023-10-16T20:07:41.589 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.581+0000 7fcb3bfff640 1 --2- 172.21.15.88:0/3523494488 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcb280664a0 0x7fcb28068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:41.589 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.581+0000 7fcb3bfff640 1 --2- 172.21.15.88:0/3523494488 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcb280664a0 0x7fcb28068960 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c001240 tx=0x7fcb2c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:41.590 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.585+0000 7fcb38ff9640 1 -- 172.21.15.88:0/3523494488 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcb340ec690 con 0x7fcb3c15c9a0 2023-10-16T20:07:41.741 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.737+0000 7fcb41756640 1 -- 172.21.15.88:0/3523494488 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb3c1600c0 con 0x7fcb3c15c9a0 2023-10-16T20:07:41.742 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.737+0000 7fcb38ff9640 1 -- 172.21.15.88:0/3523494488 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16718 (secure 0 0 0) 0x7fcb340efcb0 con 0x7fcb3c15c9a0 2023-10-16T20:07:41.742 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:07:41.742 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":58,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:07:27.206614+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T20:07:24.001121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"testcephfs1_data","create_time":"2023-10-16T20:07:24.636209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:07:41.744 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.741+0000 7fcb41756640 1 -- 172.21.15.88:0/3523494488 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcb280664a0 msgr2=0x7fcb28068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:41.744 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.741+0000 7fcb41756640 1 --2- 172.21.15.88:0/3523494488 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcb280664a0 0x7fcb28068960 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c001240 tx=0x7fcb2c009040 comp rx=0 tx=0).stop 2023-10-16T20:07:41.744 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.741+0000 7fcb41756640 1 -- 172.21.15.88:0/3523494488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb3c15c9a0 msgr2=0x7fcb3c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:41.745 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.741+0000 7fcb41756640 1 --2- 172.21.15.88:0/3523494488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb3c15c9a0 0x7fcb3c15d5d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fcb340b0040 tx=0x7fcb34019660 comp rx=0 tx=0).stop 2023-10-16T20:07:41.745 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.741+0000 7fcb41756640 1 -- 172.21.15.88:0/3523494488 shutdown_connections 2023-10-16T20:07:41.745 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.741+0000 7fcb41756640 1 --2- 172.21.15.88:0/3523494488 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcb280664a0 0x7fcb28068960 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:41.745 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.741+0000 7fcb41756640 1 --2- 172.21.15.88:0/3523494488 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcb3c0cf940 0x7fcb3c15d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:41.745 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.741+0000 7fcb41756640 1 --2- 172.21.15.88:0/3523494488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb3c15c9a0 0x7fcb3c15d5d0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:41.745 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.741+0000 7fcb41756640 1 --2- 172.21.15.88:0/3523494488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb3c0cde80 0x7fcb3c15f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:41.746 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.741+0000 7fcb41756640 1 -- 172.21.15.88:0/3523494488 >> 172.21.15.88:0/3523494488 conn(0x7fcb3c09bc30 msgr2=0x7fcb3c0b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:41.746 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.741+0000 7fcb41756640 1 -- 172.21.15.88:0/3523494488 shutdown_connections 2023-10-16T20:07:41.746 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.741+0000 7fcb41756640 1 -- 172.21.15.88:0/3523494488 wait complete. 2023-10-16T20:07:41.763 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'testcephfs1_metadata', 'create_time': '2023-10-16T20:07:24.001121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'testcephfs1_data', 'create_time': '2023-10-16T20:07:24.636209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T20:07:41.764 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:07:41.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.897+0000 7fe5bb71d640 1 Processor -- start 2023-10-16T20:07:41.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.901+0000 7fe5bb71d640 1 -- start start 2023-10-16T20:07:41.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.901+0000 7fe5bb71d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe5b40b0c10 0x7fe5b415aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:41.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.901+0000 7fe5bb71d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fe5b415b530 0x7fe5b406c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:41.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.901+0000 7fe5bb71d640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5b415bd20 con 0x7fe5b4154f30 2023-10-16T20:07:41.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.901+0000 7fe5bb71d640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5b4155310 con 0x7fe5b415b530 2023-10-16T20:07:41.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.901+0000 7fe5bb71d640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5b40702f0 con 0x7fe5b40b0c10 2023-10-16T20:07:41.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.901+0000 7fe5b9cba640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fe5b415b530 0x7fe5b406c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:41.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.901+0000 7fe5b9cba640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fe5b415b530 0x7fe5b406c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47870/0 (socket says 172.21.15.88:47870) 2023-10-16T20:07:41.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.901+0000 7fe5b9cba640 1 -- 172.21.15.88:0/2388194520 learned_addr learned my addr 172.21.15.88:0/2388194520 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:41.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.901+0000 7fe5b8cb8640 1 --2- 172.21.15.88:0/2388194520 >> v2:172.21.15.94:3300/0 conn(0x7fe5b40b0c10 0x7fe5b415aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:41.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.901+0000 7fe5b9cba640 1 -- 172.21.15.88:0/2388194520 >> v2:172.21.15.94:3300/0 conn(0x7fe5b40b0c10 msgr2=0x7fe5b415aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:41.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.901+0000 7fe5b9cba640 1 --2- 172.21.15.88:0/2388194520 >> v2:172.21.15.94:3300/0 conn(0x7fe5b40b0c10 0x7fe5b415aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:41.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.901+0000 7fe5b9cba640 1 -- 172.21.15.88:0/2388194520 >> v1:172.21.15.94:6789/0 conn(0x7fe5b4154f30 legacy=0x7fe5b40b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:41.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.901+0000 7fe5b9cba640 1 -- 172.21.15.88:0/2388194520 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b40705d0 con 0x7fe5b415b530 2023-10-16T20:07:41.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.901+0000 7fe5b8cb8640 1 --2- 172.21.15.88:0/2388194520 >> v2:172.21.15.94:3300/0 conn(0x7fe5b40b0c10 0x7fe5b415aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:07:41.909 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.905+0000 7fe5b9cba640 1 --2- 172.21.15.88:0/2388194520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5b415b530 0x7fe5b406c2e0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe5b406e700 tx=0x7fe5ac01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=340e7dce7691c1e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:41.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.905+0000 7fe5abfff640 1 -- 172.21.15.88:0/2388194520 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac0a5860 con 0x7fe5b415b530 2023-10-16T20:07:41.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.905+0000 7fe5abfff640 1 -- 172.21.15.88:0/2388194520 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5ac0a59c0 con 0x7fe5b415b530 2023-10-16T20:07:41.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.905+0000 7fe5abfff640 1 -- 172.21.15.88:0/2388194520 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac0a5d50 con 0x7fe5b415b530 2023-10-16T20:07:41.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.905+0000 7fe5bb71d640 1 -- 172.21.15.88:0/2388194520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5b415b530 msgr2=0x7fe5b406c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:41.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.905+0000 7fe5bb71d640 1 --2- 172.21.15.88:0/2388194520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5b415b530 0x7fe5b406c2e0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe5b406e700 tx=0x7fe5ac01c9a0 comp rx=0 tx=0).stop 2023-10-16T20:07:41.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.905+0000 7fe5bb71d640 1 -- 172.21.15.88:0/2388194520 shutdown_connections 2023-10-16T20:07:41.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.905+0000 7fe5bb71d640 1 --2- 172.21.15.88:0/2388194520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5b415b530 0x7fe5b406c2e0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:41.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.905+0000 7fe5bb71d640 1 --2- 172.21.15.88:0/2388194520 >> v2:172.21.15.94:3300/0 conn(0x7fe5b40b0c10 0x7fe5b415aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:41.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.905+0000 7fe5bb71d640 1 -- 172.21.15.88:0/2388194520 >> 172.21.15.88:0/2388194520 conn(0x7fe5b40cb080 msgr2=0x7fe5b40cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:41.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.905+0000 7fe5bb71d640 1 -- 172.21.15.88:0/2388194520 shutdown_connections 2023-10-16T20:07:41.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.905+0000 7fe5bb71d640 1 -- 172.21.15.88:0/2388194520 wait complete. 2023-10-16T20:07:41.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.905+0000 7fe5bb71d640 1 Processor -- start 2023-10-16T20:07:41.914 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.905+0000 7fe5bb71d640 1 -- start start 2023-10-16T20:07:41.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.909+0000 7fe5bb71d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5b40b0c10 0x7fe5b40a92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:41.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.909+0000 7fe5bb71d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe5b4154f30 0x7fe5b40cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:41.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.909+0000 7fe5bb71d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5b415b530 0x7fe5b413eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:41.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.909+0000 7fe5bb71d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b40a8220 con 0x7fe5b40b0c10 2023-10-16T20:07:41.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.909+0000 7fe5bb71d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b40a8500 con 0x7fe5b415b530 2023-10-16T20:07:41.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.909+0000 7fe5bb71d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b40a87e0 con 0x7fe5b4154f30 2023-10-16T20:07:41.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.909+0000 7fe5b94b9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5b40b0c10 0x7fe5b40a92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:41.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.909+0000 7fe5b94b9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5b40b0c10 0x7fe5b40a92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47886/0 (socket says 172.21.15.88:47886) 2023-10-16T20:07:41.916 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.909+0000 7fe5b94b9640 1 -- 172.21.15.88:0/2561861026 learned_addr learned my addr 172.21.15.88:0/2561861026 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:41.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.909+0000 7fe5b9cba640 1 --2- 172.21.15.88:0/2561861026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5b415b530 0x7fe5b413eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:41.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.909+0000 7fe5b8cb8640 1 --2- 172.21.15.88:0/2561861026 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe5b4154f30 0x7fe5b40cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:41.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.909+0000 7fe5b94b9640 1 -- 172.21.15.88:0/2561861026 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe5b4154f30 msgr2=0x7fe5b40cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:41.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.909+0000 7fe5b94b9640 1 --2- 172.21.15.88:0/2561861026 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe5b4154f30 0x7fe5b40cd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:41.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.909+0000 7fe5b94b9640 1 -- 172.21.15.88:0/2561861026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5b415b530 msgr2=0x7fe5b413eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:41.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.909+0000 7fe5b94b9640 1 --2- 172.21.15.88:0/2561861026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5b415b530 0x7fe5b413eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:41.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.909+0000 7fe5b94b9640 1 -- 172.21.15.88:0/2561861026 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ac009b30 con 0x7fe5b40b0c10 2023-10-16T20:07:41.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.909+0000 7fe5b94b9640 1 --2- 172.21.15.88:0/2561861026 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5b40b0c10 0x7fe5b40a92c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe5b000c500 tx=0x7fe5b000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:41.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.909+0000 7fe5a9ffb640 1 -- 172.21.15.88:0/2561861026 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0005220 con 0x7fe5b40b0c10 2023-10-16T20:07:41.918 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.909+0000 7fe5a9ffb640 1 -- 172.21.15.88:0/2561861026 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b0005380 con 0x7fe5b40b0c10 2023-10-16T20:07:41.919 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.909+0000 7fe5a9ffb640 1 -- 172.21.15.88:0/2561861026 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0007270 con 0x7fe5b40b0c10 2023-10-16T20:07:41.919 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.909+0000 7fe5bb71d640 1 -- 172.21.15.88:0/2561861026 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5b40a8ac0 con 0x7fe5b40b0c10 2023-10-16T20:07:41.919 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.909+0000 7fe5bb71d640 1 -- 172.21.15.88:0/2561861026 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5b40cd5b0 con 0x7fe5b40b0c10 2023-10-16T20:07:41.919 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.909+0000 7fe5bb71d640 1 -- 172.21.15.88:0/2561861026 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5b404e9c0 con 0x7fe5b40b0c10 2023-10-16T20:07:41.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.909+0000 7fe5a9ffb640 1 -- 172.21.15.88:0/2561861026 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe5b00073d0 con 0x7fe5b40b0c10 2023-10-16T20:07:41.920 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.913+0000 7fe5a9ffb640 1 --2- 172.21.15.88:0/2561861026 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe5980664a0 0x7fe598068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:41.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.913+0000 7fe5a9ffb640 1 -- 172.21.15.88:0/2561861026 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fe5b00838e0 con 0x7fe5b40b0c10 2023-10-16T20:07:41.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.913+0000 7fe5b8cb8640 1 --2- 172.21.15.88:0/2561861026 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe5980664a0 0x7fe598068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:41.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.913+0000 7fe5b8cb8640 1 --2- 172.21.15.88:0/2561861026 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe5980664a0 0x7fe598068960 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe5b41f7eb0 tx=0x7fe5a4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:41.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:41.917+0000 7fe5a9ffb640 1 -- 172.21.15.88:0/2561861026 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe5b0050b70 con 0x7fe5b40b0c10 2023-10-16T20:07:42.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.065+0000 7fe5bb71d640 1 -- 172.21.15.88:0/2561861026 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe5b40b0500 con 0x7fe5b40b0c10 2023-10-16T20:07:42.068 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.065+0000 7fe5a9ffb640 1 -- 172.21.15.88:0/2561861026 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe5b0054190 con 0x7fe5b40b0c10 2023-10-16T20:07:42.069 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:07:42.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.065+0000 7fe5bb71d640 1 -- 172.21.15.88:0/2561861026 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe5980664a0 msgr2=0x7fe598068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:42.070 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.065+0000 7fe5bb71d640 1 --2- 172.21.15.88:0/2561861026 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe5980664a0 0x7fe598068960 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe5b41f7eb0 tx=0x7fe5a4008040 comp rx=0 tx=0).stop 2023-10-16T20:07:42.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.065+0000 7fe5bb71d640 1 -- 172.21.15.88:0/2561861026 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5b40b0c10 msgr2=0x7fe5b40a92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:42.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.065+0000 7fe5bb71d640 1 --2- 172.21.15.88:0/2561861026 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5b40b0c10 0x7fe5b40a92c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe5b000c500 tx=0x7fe5b000c9c0 comp rx=0 tx=0).stop 2023-10-16T20:07:42.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.065+0000 7fe5bb71d640 1 -- 172.21.15.88:0/2561861026 shutdown_connections 2023-10-16T20:07:42.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.065+0000 7fe5bb71d640 1 --2- 172.21.15.88:0/2561861026 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe5980664a0 0x7fe598068960 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:42.072 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.065+0000 7fe5bb71d640 1 --2- 172.21.15.88:0/2561861026 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe5b4154f30 0x7fe5b40cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:42.072 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.065+0000 7fe5bb71d640 1 --2- 172.21.15.88:0/2561861026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5b415b530 0x7fe5b413eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:42.072 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.065+0000 7fe5bb71d640 1 --2- 172.21.15.88:0/2561861026 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5b40b0c10 0x7fe5b40a92c0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:42.072 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.065+0000 7fe5bb71d640 1 -- 172.21.15.88:0/2561861026 >> 172.21.15.88:0/2561861026 conn(0x7fe5b40cb080 msgr2=0x7fe5b4150c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:42.072 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.065+0000 7fe5bb71d640 1 -- 172.21.15.88:0/2561861026 shutdown_connections 2023-10-16T20:07:42.072 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.065+0000 7fe5bb71d640 1 -- 172.21.15.88:0/2561861026 wait complete. 2023-10-16T20:07:42.089 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_metadata pg_num 2023-10-16T20:07:42.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.225+0000 7fa317605640 1 Processor -- start 2023-10-16T20:07:42.232 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.229+0000 7fa317605640 1 -- start start 2023-10-16T20:07:42.233 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.229+0000 7fa317605640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fa31015cb70 0x7fa3100b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:42.234 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.229+0000 7fa317605640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fa3100b6a80 0x7fa3100b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:42.234 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.229+0000 7fa317605640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3100b4ed0 con 0x7fa3100b4530 2023-10-16T20:07:42.234 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.229+0000 7fa317605640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3100b7d20 con 0x7fa3100b6a80 2023-10-16T20:07:42.234 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.229+0000 7fa317605640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3100b6eb0 con 0x7fa31015cb70 2023-10-16T20:07:42.234 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.229+0000 7fa315ba2640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fa3100b6a80 0x7fa3100b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:42.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.229+0000 7fa315ba2640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fa3100b6a80 0x7fa3100b4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45050/0 (socket says 172.21.15.88:45050) 2023-10-16T20:07:42.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.229+0000 7fa315ba2640 1 -- 172.21.15.88:0/3705131531 learned_addr learned my addr 172.21.15.88:0/3705131531 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:42.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.229+0000 7fa314ba0640 1 --1- 172.21.15.88:0/3705131531 >> v1:172.21.15.88:6789/0 conn(0x7fa3100b4530 0x7fa3100d0bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:35864/0 (socket says 172.21.15.88:35864) 2023-10-16T20:07:42.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.229+0000 7fa3153a1640 1 --2- 172.21.15.88:0/3705131531 >> v2:172.21.15.94:3300/0 conn(0x7fa31015cb70 0x7fa3100b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:42.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.229+0000 7fa315ba2640 1 -- 172.21.15.88:0/3705131531 >> v2:172.21.15.94:3300/0 conn(0x7fa31015cb70 msgr2=0x7fa3100b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:42.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.229+0000 7fa315ba2640 1 --2- 172.21.15.88:0/3705131531 >> v2:172.21.15.94:3300/0 conn(0x7fa31015cb70 0x7fa3100b83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:42.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.229+0000 7fa315ba2640 1 -- 172.21.15.88:0/3705131531 >> v1:172.21.15.88:6789/0 conn(0x7fa3100b4530 legacy=0x7fa3100d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:42.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.229+0000 7fa315ba2640 1 -- 172.21.15.88:0/3705131531 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3100b7110 con 0x7fa3100b6a80 2023-10-16T20:07:42.237 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.233+0000 7fa315ba2640 1 --2- 172.21.15.88:0/3705131531 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3100b6a80 0x7fa3100b4990 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa31008bbc0 tx=0x7fa30801f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3729a796447a37 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:42.238 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.233+0000 7fa307fff640 1 -- 172.21.15.88:0/3705131531 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3080af400 con 0x7fa3100b6a80 2023-10-16T20:07:42.238 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.233+0000 7fa307fff640 1 -- 172.21.15.88:0/3705131531 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3080ab050 con 0x7fa3100b6a80 2023-10-16T20:07:42.238 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.233+0000 7fa307fff640 1 -- 172.21.15.88:0/3705131531 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3080af780 con 0x7fa3100b6a80 2023-10-16T20:07:42.239 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.233+0000 7fa317605640 1 -- 172.21.15.88:0/3705131531 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3100b6a80 msgr2=0x7fa3100b4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:42.239 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.233+0000 7fa317605640 1 --2- 172.21.15.88:0/3705131531 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3100b6a80 0x7fa3100b4990 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa31008bbc0 tx=0x7fa30801f1c0 comp rx=0 tx=0).stop 2023-10-16T20:07:42.239 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.233+0000 7fa317605640 1 -- 172.21.15.88:0/3705131531 shutdown_connections 2023-10-16T20:07:42.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.233+0000 7fa317605640 1 --2- 172.21.15.88:0/3705131531 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3100b6a80 0x7fa3100b4990 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:42.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.233+0000 7fa317605640 1 --2- 172.21.15.88:0/3705131531 >> v2:172.21.15.94:3300/0 conn(0x7fa31015cb70 0x7fa3100b83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:42.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.233+0000 7fa317605640 1 -- 172.21.15.88:0/3705131531 >> 172.21.15.88:0/3705131531 conn(0x7fa31006b6e0 msgr2=0x7fa31006baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:42.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.233+0000 7fa317605640 1 -- 172.21.15.88:0/3705131531 shutdown_connections 2023-10-16T20:07:42.241 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.233+0000 7fa317605640 1 -- 172.21.15.88:0/3705131531 wait complete. 2023-10-16T20:07:42.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.237+0000 7fa317605640 1 Processor -- start 2023-10-16T20:07:42.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.237+0000 7fa317605640 1 -- start start 2023-10-16T20:07:42.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.237+0000 7fa317605640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3100b4530 0x7fa31014a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:42.243 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.237+0000 7fa317605640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3100b6a80 0x7fa3100cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:42.244 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.237+0000 7fa317605640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa31015cb70 0x7fa3100c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:42.244 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.237+0000 7fa317605640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3100c9f60 con 0x7fa3100b4530 2023-10-16T20:07:42.244 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.237+0000 7fa317605640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3100ca240 con 0x7fa3100b6a80 2023-10-16T20:07:42.244 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.237+0000 7fa317605640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3100ca520 con 0x7fa31015cb70 2023-10-16T20:07:42.245 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.237+0000 7fa3153a1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3100b4530 0x7fa31014a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:42.245 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.237+0000 7fa3153a1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3100b4530 0x7fa31014a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45060/0 (socket says 172.21.15.88:45060) 2023-10-16T20:07:42.245 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.237+0000 7fa3153a1640 1 -- 172.21.15.88:0/3651105583 learned_addr learned my addr 172.21.15.88:0/3651105583 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:42.245 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.237+0000 7fa314ba0640 1 --2- 172.21.15.88:0/3651105583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3100b6a80 0x7fa3100cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:42.245 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.237+0000 7fa315ba2640 1 --2- 172.21.15.88:0/3651105583 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa31015cb70 0x7fa3100c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:42.246 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.237+0000 7fa3153a1640 1 -- 172.21.15.88:0/3651105583 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa31015cb70 msgr2=0x7fa3100c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:42.246 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.237+0000 7fa3153a1640 1 --2- 172.21.15.88:0/3651105583 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa31015cb70 0x7fa3100c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:42.246 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.237+0000 7fa3153a1640 1 -- 172.21.15.88:0/3651105583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3100b6a80 msgr2=0x7fa3100cb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:42.246 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.237+0000 7fa3153a1640 1 --2- 172.21.15.88:0/3651105583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3100b6a80 0x7fa3100cb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:42.246 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.237+0000 7fa3153a1640 1 -- 172.21.15.88:0/3651105583 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa308009b30 con 0x7fa3100b4530 2023-10-16T20:07:42.247 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.237+0000 7fa3153a1640 1 --2- 172.21.15.88:0/3651105583 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3100b4530 0x7fa31014a0b0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa30c00e930 tx=0x7fa30c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:42.247 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.237+0000 7fa305ffb640 1 -- 172.21.15.88:0/3651105583 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c009690 con 0x7fa3100b4530 2023-10-16T20:07:42.247 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.237+0000 7fa305ffb640 1 -- 172.21.15.88:0/3651105583 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30c00ca00 con 0x7fa3100b4530 2023-10-16T20:07:42.247 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.237+0000 7fa317605640 1 -- 172.21.15.88:0/3651105583 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3100ca800 con 0x7fa3100b4530 2023-10-16T20:07:42.247 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.237+0000 7fa305ffb640 1 -- 172.21.15.88:0/3651105583 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c00cd70 con 0x7fa3100b4530 2023-10-16T20:07:42.248 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.237+0000 7fa317605640 1 -- 172.21.15.88:0/3651105583 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3100c9b80 con 0x7fa3100b4530 2023-10-16T20:07:42.249 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.241+0000 7fa317605640 1 -- 172.21.15.88:0/3651105583 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa31004e9c0 con 0x7fa3100b4530 2023-10-16T20:07:42.249 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.241+0000 7fa305ffb640 1 -- 172.21.15.88:0/3651105583 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa30c013470 con 0x7fa3100b4530 2023-10-16T20:07:42.249 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.241+0000 7fa305ffb640 1 --2- 172.21.15.88:0/3651105583 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa2f40664d0 0x7fa2f4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:42.249 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.241+0000 7fa305ffb640 1 -- 172.21.15.88:0/3651105583 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fa30c0841e0 con 0x7fa3100b4530 2023-10-16T20:07:42.250 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.241+0000 7fa314ba0640 1 --2- 172.21.15.88:0/3651105583 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa2f40664d0 0x7fa2f4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:42.250 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.241+0000 7fa314ba0640 1 --2- 172.21.15.88:0/3651105583 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa2f40664d0 0x7fa2f4068990 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa31015ff10 tx=0x7fa300007e50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:42.250 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.245+0000 7fa305ffb640 1 -- 172.21.15.88:0/3651105583 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa30c051660 con 0x7fa3100b4530 2023-10-16T20:07:42.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.397+0000 7fa317605640 1 -- 172.21.15.88:0/3651105583 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa3100b6550 con 0x7fa3100b4530 2023-10-16T20:07:42.401 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.397+0000 7fa305ffb640 1 -- 172.21.15.88:0/3651105583 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"}]=0 v58) v1 ==== 109+0+11 (secure 0 0 0) 0x7fa30c054c80 con 0x7fa3100b4530 2023-10-16T20:07:42.401 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:07:42.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.397+0000 7fa317605640 1 -- 172.21.15.88:0/3651105583 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa2f40664d0 msgr2=0x7fa2f4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:42.403 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.397+0000 7fa317605640 1 --2- 172.21.15.88:0/3651105583 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa2f40664d0 0x7fa2f4068990 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa31015ff10 tx=0x7fa300007e50 comp rx=0 tx=0).stop 2023-10-16T20:07:42.403 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.397+0000 7fa317605640 1 -- 172.21.15.88:0/3651105583 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3100b4530 msgr2=0x7fa31014a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:42.403 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.397+0000 7fa317605640 1 --2- 172.21.15.88:0/3651105583 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3100b4530 0x7fa31014a0b0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa30c00e930 tx=0x7fa30c00edf0 comp rx=0 tx=0).stop 2023-10-16T20:07:42.403 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.397+0000 7fa317605640 1 -- 172.21.15.88:0/3651105583 shutdown_connections 2023-10-16T20:07:42.404 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.397+0000 7fa317605640 1 --2- 172.21.15.88:0/3651105583 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa2f40664d0 0x7fa2f4068990 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:42.404 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.397+0000 7fa317605640 1 --2- 172.21.15.88:0/3651105583 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa31015cb70 0x7fa3100c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:42.404 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.397+0000 7fa317605640 1 --2- 172.21.15.88:0/3651105583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3100b6a80 0x7fa3100cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:42.404 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.397+0000 7fa317605640 1 --2- 172.21.15.88:0/3651105583 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3100b4530 0x7fa31014a0b0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:42.405 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.397+0000 7fa317605640 1 -- 172.21.15.88:0/3651105583 >> 172.21.15.88:0/3651105583 conn(0x7fa31006b6e0 msgr2=0x7fa310158140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:42.405 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.397+0000 7fa317605640 1 -- 172.21.15.88:0/3651105583 shutdown_connections 2023-10-16T20:07:42.405 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.397+0000 7fa317605640 1 -- 172.21.15.88:0/3651105583 wait complete. 2023-10-16T20:07:42.421 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_data pg_num 2023-10-16T20:07:42.562 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.557+0000 7fae8bf94640 1 Processor -- start 2023-10-16T20:07:42.562 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.557+0000 7fae8bf94640 1 -- start start 2023-10-16T20:07:42.564 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.557+0000 7fae8bf94640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fae841457f0 0x7fae84160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:42.564 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.561+0000 7fae8bf94640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fae840aeb90 0x7fae84070250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:42.565 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.561+0000 7fae8bf94640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae8406dab0 con 0x7fae84146290 2023-10-16T20:07:42.565 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.561+0000 7fae8bf94640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae84145bd0 con 0x7fae840aeb90 2023-10-16T20:07:42.565 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.561+0000 7fae8bf94640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae8406c2e0 con 0x7fae841457f0 2023-10-16T20:07:42.565 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.561+0000 7fae8952f640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fae840aeb90 0x7fae84070250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:42.565 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.561+0000 7fae8952f640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fae840aeb90 0x7fae84070250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45068/0 (socket says 172.21.15.88:45068) 2023-10-16T20:07:42.566 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.561+0000 7fae8952f640 1 -- 172.21.15.88:0/292510401 learned_addr learned my addr 172.21.15.88:0/292510401 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:42.566 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.561+0000 7fae89d30640 1 --2- 172.21.15.88:0/292510401 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fae841457f0 0x7fae84160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:42.566 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.561+0000 7fae8952f640 1 -- 172.21.15.88:0/292510401 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fae841457f0 msgr2=0x7fae84160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:42.566 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.561+0000 7fae8952f640 1 --2- 172.21.15.88:0/292510401 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fae841457f0 0x7fae84160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:42.567 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.561+0000 7fae8952f640 1 -- 172.21.15.88:0/292510401 >> v1:172.21.15.94:6789/0 conn(0x7fae84146290 legacy=0x7fae840ab890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:42.567 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.561+0000 7fae8952f640 1 -- 172.21.15.88:0/292510401 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae84160ec0 con 0x7fae840aeb90 2023-10-16T20:07:42.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.561+0000 7fae8952f640 1 --2- 172.21.15.88:0/292510401 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae840aeb90 0x7fae84070250 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fae74009870 tx=0x7fae7401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf1388e03ad6e8c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:42.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.561+0000 7fae88d2e640 1 -- 172.21.15.88:0/292510401 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae740ab620 con 0x7fae840aeb90 2023-10-16T20:07:42.569 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.561+0000 7fae88d2e640 1 -- 172.21.15.88:0/292510401 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae740ab780 con 0x7fae840aeb90 2023-10-16T20:07:42.569 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.561+0000 7fae88d2e640 1 -- 172.21.15.88:0/292510401 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae740abb10 con 0x7fae840aeb90 2023-10-16T20:07:42.569 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.565+0000 7fae8bf94640 1 -- 172.21.15.88:0/292510401 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae840aeb90 msgr2=0x7fae84070250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:42.569 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.565+0000 7fae8bf94640 1 --2- 172.21.15.88:0/292510401 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae840aeb90 0x7fae84070250 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fae74009870 tx=0x7fae7401c510 comp rx=0 tx=0).stop 2023-10-16T20:07:42.570 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.565+0000 7fae8bf94640 1 -- 172.21.15.88:0/292510401 shutdown_connections 2023-10-16T20:07:42.570 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.565+0000 7fae8bf94640 1 --2- 172.21.15.88:0/292510401 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae840aeb90 0x7fae84070250 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:42.570 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.565+0000 7fae8bf94640 1 --2- 172.21.15.88:0/292510401 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fae841457f0 0x7fae84160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:42.570 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.565+0000 7fae8bf94640 1 -- 172.21.15.88:0/292510401 >> 172.21.15.88:0/292510401 conn(0x7fae840cb080 msgr2=0x7fae840cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:42.570 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.565+0000 7fae8bf94640 1 -- 172.21.15.88:0/292510401 shutdown_connections 2023-10-16T20:07:42.571 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.565+0000 7fae8bf94640 1 -- 172.21.15.88:0/292510401 wait complete. 2023-10-16T20:07:42.573 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.565+0000 7fae8bf94640 1 Processor -- start 2023-10-16T20:07:42.573 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.565+0000 7fae8bf94640 1 -- start start 2023-10-16T20:07:42.573 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.565+0000 7fae8bf94640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae840aeb90 0x7fae8415c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:42.573 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.565+0000 7fae8bf94640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae841457f0 0x7fae8413d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:42.574 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.565+0000 7fae8bf94640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fae84146290 0x7fae8413df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:42.574 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.565+0000 7fae8bf94640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8415ca40 con 0x7fae840aeb90 2023-10-16T20:07:42.574 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.565+0000 7fae8bf94640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8415cc70 con 0x7fae841457f0 2023-10-16T20:07:42.574 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.565+0000 7fae8bf94640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae8415cf50 con 0x7fae84146290 2023-10-16T20:07:42.574 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.565+0000 7fae89d30640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae840aeb90 0x7fae8415c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:42.575 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.565+0000 7fae89d30640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae840aeb90 0x7fae8415c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45070/0 (socket says 172.21.15.88:45070) 2023-10-16T20:07:42.575 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.565+0000 7fae89d30640 1 -- 172.21.15.88:0/4278980295 learned_addr learned my addr 172.21.15.88:0/4278980295 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:42.575 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.565+0000 7fae8a531640 1 --2- 172.21.15.88:0/4278980295 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fae84146290 0x7fae8413df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:42.575 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.565+0000 7fae8952f640 1 --2- 172.21.15.88:0/4278980295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae841457f0 0x7fae8413d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:42.576 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.565+0000 7fae89d30640 1 -- 172.21.15.88:0/4278980295 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fae84146290 msgr2=0x7fae8413df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:42.576 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.565+0000 7fae89d30640 1 --2- 172.21.15.88:0/4278980295 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fae84146290 0x7fae8413df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:42.576 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.565+0000 7fae89d30640 1 -- 172.21.15.88:0/4278980295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae841457f0 msgr2=0x7fae8413d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:42.576 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.565+0000 7fae89d30640 1 --2- 172.21.15.88:0/4278980295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae841457f0 0x7fae8413d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:42.576 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.565+0000 7fae89d30640 1 -- 172.21.15.88:0/4278980295 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae740094d0 con 0x7fae840aeb90 2023-10-16T20:07:42.577 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.565+0000 7fae8a531640 1 --2- 172.21.15.88:0/4278980295 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fae84146290 0x7fae8413df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:07:42.577 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.565+0000 7fae8952f640 1 --2- 172.21.15.88:0/4278980295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae841457f0 0x7fae8413d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:07:42.577 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.569+0000 7fae89d30640 1 --2- 172.21.15.88:0/4278980295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae840aeb90 0x7fae8415c500 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fae8000d800 tx=0x7fae8000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:42.577 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.569+0000 7fae7a7fc640 1 -- 172.21.15.88:0/4278980295 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8000b6c0 con 0x7fae840aeb90 2023-10-16T20:07:42.577 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.569+0000 7fae7a7fc640 1 -- 172.21.15.88:0/4278980295 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8000b860 con 0x7fae840aeb90 2023-10-16T20:07:42.578 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.569+0000 7fae8bf94640 1 -- 172.21.15.88:0/4278980295 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae8415d230 con 0x7fae840aeb90 2023-10-16T20:07:42.578 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.569+0000 7fae7a7fc640 1 -- 172.21.15.88:0/4278980295 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8000bc10 con 0x7fae840aeb90 2023-10-16T20:07:42.578 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.569+0000 7fae8bf94640 1 -- 172.21.15.88:0/4278980295 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae8415d7f0 con 0x7fae840aeb90 2023-10-16T20:07:42.578 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.569+0000 7fae8bf94640 1 -- 172.21.15.88:0/4278980295 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae840cae80 con 0x7fae840aeb90 2023-10-16T20:07:42.579 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.569+0000 7fae7a7fc640 1 -- 172.21.15.88:0/4278980295 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fae8001e460 con 0x7fae840aeb90 2023-10-16T20:07:42.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.569+0000 7fae7a7fc640 1 --2- 172.21.15.88:0/4278980295 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fae680664a0 0x7fae68068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:42.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.569+0000 7fae7a7fc640 1 -- 172.21.15.88:0/4278980295 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fae80084560 con 0x7fae840aeb90 2023-10-16T20:07:42.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.569+0000 7fae8952f640 1 --2- 172.21.15.88:0/4278980295 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fae680664a0 0x7fae68068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:42.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.573+0000 7fae8952f640 1 --2- 172.21.15.88:0/4278980295 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fae680664a0 0x7fae68068960 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fae740af000 tx=0x7fae74008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:42.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.573+0000 7fae7a7fc640 1 -- 172.21.15.88:0/4278980295 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fae800518f0 con 0x7fae840aeb90 2023-10-16T20:07:42.728 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.725+0000 7fae8bf94640 1 -- 172.21.15.88:0/4278980295 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"} v 0) v1 -- 0x7fae840cc880 con 0x7fae840aeb90 2023-10-16T20:07:42.728 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.725+0000 7fae7a7fc640 1 -- 172.21.15.88:0/4278980295 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"}]=0 v58) v1 ==== 105+0+11 (secure 0 0 0) 0x7fae80054f10 con 0x7fae840aeb90 2023-10-16T20:07:42.729 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2023-10-16T20:07:42.730 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.725+0000 7fae8bf94640 1 -- 172.21.15.88:0/4278980295 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fae680664a0 msgr2=0x7fae68068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:42.730 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.725+0000 7fae8bf94640 1 --2- 172.21.15.88:0/4278980295 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fae680664a0 0x7fae68068960 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fae740af000 tx=0x7fae74008040 comp rx=0 tx=0).stop 2023-10-16T20:07:42.730 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.725+0000 7fae8bf94640 1 -- 172.21.15.88:0/4278980295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae840aeb90 msgr2=0x7fae8415c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:42.731 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.725+0000 7fae8bf94640 1 --2- 172.21.15.88:0/4278980295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae840aeb90 0x7fae8415c500 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fae8000d800 tx=0x7fae8000dcc0 comp rx=0 tx=0).stop 2023-10-16T20:07:42.731 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.725+0000 7fae8bf94640 1 -- 172.21.15.88:0/4278980295 shutdown_connections 2023-10-16T20:07:42.731 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.725+0000 7fae8bf94640 1 --2- 172.21.15.88:0/4278980295 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fae680664a0 0x7fae68068960 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:42.732 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.725+0000 7fae8bf94640 1 --2- 172.21.15.88:0/4278980295 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fae84146290 0x7fae8413df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:42.732 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.725+0000 7fae8bf94640 1 --2- 172.21.15.88:0/4278980295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae841457f0 0x7fae8413d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:42.732 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.725+0000 7fae8bf94640 1 --2- 172.21.15.88:0/4278980295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae840aeb90 0x7fae8415c500 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:42.732 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.725+0000 7fae8bf94640 1 -- 172.21.15.88:0/4278980295 >> 172.21.15.88:0/4278980295 conn(0x7fae840cb080 msgr2=0x7fae840cd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:42.732 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.725+0000 7fae8bf94640 1 -- 172.21.15.88:0/4278980295 shutdown_connections 2023-10-16T20:07:42.733 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.725+0000 7fae8bf94640 1 -- 172.21.15.88:0/4278980295 wait complete. 2023-10-16T20:07:42.749 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T20:07:42.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.881+0000 7f942f73e640 1 Processor -- start 2023-10-16T20:07:42.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.885+0000 7f942f73e640 1 -- start start 2023-10-16T20:07:42.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.885+0000 7f942f73e640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f94280afdf0 0x7f94280aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:42.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.885+0000 7f942f73e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f94280cc830 0x7f942806a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:42.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.885+0000 7f942f73e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9428162580 con 0x7f9428069e90 2023-10-16T20:07:42.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.885+0000 7f942f73e640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94280ccc50 con 0x7f94280afdf0 2023-10-16T20:07:42.891 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.885+0000 7f942f73e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942806a860 con 0x7f94280cc830 2023-10-16T20:07:42.891 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.885+0000 7f942d4da640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f94280afdf0 0x7f94280aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:42.891 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.885+0000 7f942d4da640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f94280afdf0 0x7f94280aecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45074/0 (socket says 172.21.15.88:45074) 2023-10-16T20:07:42.891 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.885+0000 7f942d4da640 1 -- 172.21.15.88:0/1668237736 learned_addr learned my addr 172.21.15.88:0/1668237736 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:42.892 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.885+0000 7f942dcdb640 1 --2- 172.21.15.88:0/1668237736 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f94280cc830 0x7f942806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:42.892 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.885+0000 7f942d4da640 1 -- 172.21.15.88:0/1668237736 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f94280cc830 msgr2=0x7f942806a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:42.892 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.885+0000 7f942d4da640 1 --2- 172.21.15.88:0/1668237736 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f94280cc830 0x7f942806a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:42.892 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.885+0000 7f942d4da640 1 -- 172.21.15.88:0/1668237736 >> v1:172.21.15.94:6789/0 conn(0x7f9428069e90 legacy=0x7f94280cc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:42.892 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.885+0000 7f942d4da640 1 -- 172.21.15.88:0/1668237736 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94280ae830 con 0x7f94280afdf0 2023-10-16T20:07:42.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.889+0000 7f942d4da640 1 --2- 172.21.15.88:0/1668237736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94280afdf0 0x7f94280aecc0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f9424009870 tx=0x7f942401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630127bbb5ac1c7f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:42.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.889+0000 7f941ffff640 1 -- 172.21.15.88:0/1668237736 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94240ab5f0 con 0x7f94280afdf0 2023-10-16T20:07:42.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.889+0000 7f941ffff640 1 -- 172.21.15.88:0/1668237736 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94240ab750 con 0x7f94280afdf0 2023-10-16T20:07:42.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.889+0000 7f941ffff640 1 -- 172.21.15.88:0/1668237736 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94240abae0 con 0x7f94280afdf0 2023-10-16T20:07:42.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.889+0000 7f942f73e640 1 -- 172.21.15.88:0/1668237736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94280afdf0 msgr2=0x7f94280aecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:42.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.889+0000 7f942f73e640 1 --2- 172.21.15.88:0/1668237736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94280afdf0 0x7f94280aecc0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f9424009870 tx=0x7f942401c3f0 comp rx=0 tx=0).stop 2023-10-16T20:07:42.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.889+0000 7f942f73e640 1 -- 172.21.15.88:0/1668237736 shutdown_connections 2023-10-16T20:07:42.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.889+0000 7f942f73e640 1 --2- 172.21.15.88:0/1668237736 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f94280cc830 0x7f942806a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:42.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.889+0000 7f942f73e640 1 --2- 172.21.15.88:0/1668237736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94280afdf0 0x7f94280aecc0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:42.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.889+0000 7f942f73e640 1 -- 172.21.15.88:0/1668237736 >> 172.21.15.88:0/1668237736 conn(0x7f94280ce5e0 msgr2=0x7f94280ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:42.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.889+0000 7f942f73e640 1 -- 172.21.15.88:0/1668237736 shutdown_connections 2023-10-16T20:07:42.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.889+0000 7f942f73e640 1 -- 172.21.15.88:0/1668237736 wait complete. 2023-10-16T20:07:42.899 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.889+0000 7f942f73e640 1 Processor -- start 2023-10-16T20:07:42.899 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.889+0000 7f942f73e640 1 -- start start 2023-10-16T20:07:42.899 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.893+0000 7f942f73e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9428069e90 0x7f9428156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:42.899 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.893+0000 7f942f73e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f94280afdf0 0x7f9428154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:42.900 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.893+0000 7f942f73e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94280cc830 0x7f9428151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:42.900 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.893+0000 7f942f73e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9428162860 con 0x7f94280cc830 2023-10-16T20:07:42.900 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.893+0000 7f942f73e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94281545a0 con 0x7f9428069e90 2023-10-16T20:07:42.900 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.893+0000 7f942f73e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94281571d0 con 0x7f94280afdf0 2023-10-16T20:07:42.900 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.893+0000 7f942dcdb640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94280cc830 0x7f9428151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:42.901 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.893+0000 7f942dcdb640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94280cc830 0x7f9428151a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45078/0 (socket says 172.21.15.88:45078) 2023-10-16T20:07:42.901 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.893+0000 7f942dcdb640 1 -- 172.21.15.88:0/3739578244 learned_addr learned my addr 172.21.15.88:0/3739578244 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:42.901 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.893+0000 7f942d4da640 1 --2- 172.21.15.88:0/3739578244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9428069e90 0x7f9428156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:42.901 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.893+0000 7f942ccd9640 1 --2- 172.21.15.88:0/3739578244 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f94280afdf0 0x7f9428154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:42.901 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.893+0000 7f942dcdb640 1 -- 172.21.15.88:0/3739578244 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f94280afdf0 msgr2=0x7f9428154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:42.902 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.893+0000 7f942dcdb640 1 --2- 172.21.15.88:0/3739578244 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f94280afdf0 0x7f9428154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:42.902 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.893+0000 7f942dcdb640 1 -- 172.21.15.88:0/3739578244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9428069e90 msgr2=0x7f9428156bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:42.902 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.893+0000 7f942dcdb640 1 --2- 172.21.15.88:0/3739578244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9428069e90 0x7f9428156bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:42.902 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.893+0000 7f942dcdb640 1 -- 172.21.15.88:0/3739578244 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94240094d0 con 0x7f94280cc830 2023-10-16T20:07:42.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.893+0000 7f942ccd9640 1 --2- 172.21.15.88:0/3739578244 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f94280afdf0 0x7f9428154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:07:42.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.893+0000 7f942dcdb640 1 --2- 172.21.15.88:0/3739578244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94280cc830 0x7f9428151a40 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f9420007e70 tx=0x7f942000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:42.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.893+0000 7f941dffb640 1 -- 172.21.15.88:0/3739578244 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9420014620 con 0x7f94280cc830 2023-10-16T20:07:42.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.893+0000 7f941dffb640 1 -- 172.21.15.88:0/3739578244 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94200147a0 con 0x7f94280cc830 2023-10-16T20:07:42.903 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.893+0000 7f942f73e640 1 -- 172.21.15.88:0/3739578244 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94281574b0 con 0x7f94280cc830 2023-10-16T20:07:42.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.893+0000 7f941dffb640 1 -- 172.21.15.88:0/3739578244 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9420014b50 con 0x7f94280cc830 2023-10-16T20:07:42.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.893+0000 7f942f73e640 1 -- 172.21.15.88:0/3739578244 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9428157a70 con 0x7f94280cc830 2023-10-16T20:07:42.904 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.893+0000 7f942f73e640 1 -- 172.21.15.88:0/3739578244 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94281629a0 con 0x7f94280cc830 2023-10-16T20:07:42.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.897+0000 7f941dffb640 1 -- 172.21.15.88:0/3739578244 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9420014cb0 con 0x7f94280cc830 2023-10-16T20:07:42.905 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.897+0000 7f941dffb640 1 --2- 172.21.15.88:0/3739578244 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f940c0664f0 0x7f940c0689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:42.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.897+0000 7f941dffb640 1 -- 172.21.15.88:0/3739578244 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f94200848d0 con 0x7f94280cc830 2023-10-16T20:07:42.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.897+0000 7f942d4da640 1 --2- 172.21.15.88:0/3739578244 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f940c0664f0 0x7f940c0689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:42.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.897+0000 7f942d4da640 1 --2- 172.21.15.88:0/3739578244 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f940c0664f0 0x7f940c0689b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f94240af000 tx=0x7f94240b6040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:42.906 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:42.901+0000 7f941dffb640 1 -- 172.21.15.88:0/3739578244 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f94281629a0 con 0x7f94280cc830 2023-10-16T20:07:43.064 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:43.061+0000 7f942f73e640 1 -- 172.21.15.88:0/3739578244 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f94281629a0 con 0x7f94280cc830 2023-10-16T20:07:43.065 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:43.061+0000 7f941dffb640 1 -- 172.21.15.88:0/3739578244 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v58) v1 ==== 80+0+0 (secure 0 0 0) 0x7f94281629a0 con 0x7f94280cc830 2023-10-16T20:07:43.065 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2023-10-16T20:07:43.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:43.061+0000 7f942f73e640 1 -- 172.21.15.88:0/3739578244 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f940c0664f0 msgr2=0x7f940c0689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:43.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:43.061+0000 7f942f73e640 1 --2- 172.21.15.88:0/3739578244 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f940c0664f0 0x7f940c0689b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f94240af000 tx=0x7f94240b6040 comp rx=0 tx=0).stop 2023-10-16T20:07:43.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:43.061+0000 7f942f73e640 1 -- 172.21.15.88:0/3739578244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94280cc830 msgr2=0x7f9428151a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:43.067 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:43.061+0000 7f942f73e640 1 --2- 172.21.15.88:0/3739578244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94280cc830 0x7f9428151a40 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f9420007e70 tx=0x7f942000e4b0 comp rx=0 tx=0).stop 2023-10-16T20:07:43.068 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:43.061+0000 7f942f73e640 1 -- 172.21.15.88:0/3739578244 shutdown_connections 2023-10-16T20:07:43.068 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:43.061+0000 7f942f73e640 1 --2- 172.21.15.88:0/3739578244 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f940c0664f0 0x7f940c0689b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:43.068 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:43.061+0000 7f942f73e640 1 --2- 172.21.15.88:0/3739578244 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f94280afdf0 0x7f9428154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:43.068 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:43.061+0000 7f942f73e640 1 --2- 172.21.15.88:0/3739578244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94280cc830 0x7f9428151a40 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:43.068 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:43.061+0000 7f942f73e640 1 --2- 172.21.15.88:0/3739578244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9428069e90 0x7f9428156bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:43.069 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:43.061+0000 7f942f73e640 1 -- 172.21.15.88:0/3739578244 >> 172.21.15.88:0/3739578244 conn(0x7f94280ce5e0 msgr2=0x7f94280cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:43.069 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:43.061+0000 7f942f73e640 1 -- 172.21.15.88:0/3739578244 shutdown_connections 2023-10-16T20:07:43.069 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:43.061+0000 7f942f73e640 1 -- 172.21.15.88:0/3739578244 wait complete. 2023-10-16T20:07:43.086 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi088.front.sepia.ceph.com... 2023-10-16T20:07:43.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:43.087 DEBUG:teuthology.orchestra.run.smithi088:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-16T20:07:43.105 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[52482]: fuse finished with error 0 and tester_r 0 2023-10-16T20:07:43.108 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:07:49.210 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2023-10-16T20:07:49.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:49.211 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T20:07:49.223 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T20:07:49.416 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.413+0000 7f0960a9e640 1 Processor -- start 2023-10-16T20:07:49.416 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.413+0000 7f0960a9e640 1 -- start start 2023-10-16T20:07:49.418 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.413+0000 7f0960a9e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f095c0d26c0 0x7f095c161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:49.418 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.413+0000 7f0960a9e640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f095c0b87e0 con 0x7f095c0d22e0 2023-10-16T20:07:49.418 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.413+0000 7f0960a9e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f095c15df10 con 0x7f095c15d9f0 2023-10-16T20:07:49.419 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.413+0000 7f0960a9e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f095c15e1f0 con 0x7f095c0d26c0 2023-10-16T20:07:49.420 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.413+0000 7f095a59c640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f095c0d22e0 0x7f095c15d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:35866/0 (socket says 172.21.15.88:35866) 2023-10-16T20:07:49.420 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.413+0000 7f095a59c640 1 -- 172.21.15.88:0/3746808497 learned_addr learned my addr 172.21.15.88:0/3746808497 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:49.420 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.413+0000 7f095ad9d640 1 --2- 172.21.15.88:0/3746808497 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f095c0d26c0 0x7f095c161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:49.420 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.413+0000 7f095959a640 1 -- 172.21.15.88:0/3746808497 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1901141471 0 0) 0x7f095c0b87e0 con 0x7f095c0d22e0 2023-10-16T20:07:49.421 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.413+0000 7f095959a640 1 -- 172.21.15.88:0/3746808497 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0944003530 con 0x7f095c0d22e0 2023-10-16T20:07:49.421 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.413+0000 7f095959a640 1 -- 172.21.15.88:0/3746808497 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 653396417 0 0) 0x7f095c15df10 con 0x7f095c15d9f0 2023-10-16T20:07:49.421 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.413+0000 7f095959a640 1 -- 172.21.15.88:0/3746808497 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f095c0b87e0 con 0x7f095c15d9f0 2023-10-16T20:07:49.421 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.413+0000 7f095ad9d640 1 -- 172.21.15.88:0/3746808497 >> v1:172.21.15.94:6789/0 conn(0x7f095c15d9f0 legacy=0x7f095c0cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:49.421 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.413+0000 7f095ad9d640 1 -- 172.21.15.88:0/3746808497 >> v1:172.21.15.88:6789/0 conn(0x7f095c0d22e0 legacy=0x7f095c15d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:49.422 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.413+0000 7f095959a640 1 -- 172.21.15.88:0/3746808497 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3668312701 0 0) 0x7f0944003530 con 0x7f095c0d22e0 2023-10-16T20:07:49.422 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.413+0000 7f095ad9d640 1 -- 172.21.15.88:0/3746808497 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095c160970 con 0x7f095c0d26c0 2023-10-16T20:07:49.423 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.417+0000 7f095ad9d640 1 --2- 172.21.15.88:0/3746808497 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f095c0d26c0 0x7f095c161450 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0954009dc0 tx=0x7f095401c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60e65156de347224 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:49.423 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.417+0000 7f095959a640 1 -- 172.21.15.88:0/3746808497 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c001680 con 0x7f095c0d26c0 2023-10-16T20:07:49.424 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.417+0000 7f095959a640 1 -- 172.21.15.88:0/3746808497 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09540a5520 con 0x7f095c0d26c0 2023-10-16T20:07:49.424 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.417+0000 7f095959a640 1 -- 172.21.15.88:0/3746808497 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09540a58b0 con 0x7f095c0d26c0 2023-10-16T20:07:49.424 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.417+0000 7f0960a9e640 1 -- 172.21.15.88:0/3746808497 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f095c0d26c0 msgr2=0x7f095c161450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:49.424 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.417+0000 7f0960a9e640 1 --2- 172.21.15.88:0/3746808497 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f095c0d26c0 0x7f095c161450 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0954009dc0 tx=0x7f095401c7d0 comp rx=0 tx=0).stop 2023-10-16T20:07:49.425 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.417+0000 7f0960a9e640 1 -- 172.21.15.88:0/3746808497 shutdown_connections 2023-10-16T20:07:49.425 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.417+0000 7f0960a9e640 1 --2- 172.21.15.88:0/3746808497 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f095c0d26c0 0x7f095c161450 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:49.426 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.417+0000 7f0960a9e640 1 -- 172.21.15.88:0/3746808497 >> 172.21.15.88:0/3746808497 conn(0x7f095c0715c0 msgr2=0x7f095c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:49.426 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.417+0000 7f0960a9e640 1 -- 172.21.15.88:0/3746808497 shutdown_connections 2023-10-16T20:07:49.426 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.417+0000 7f0960a9e640 1 -- 172.21.15.88:0/3746808497 wait complete. 2023-10-16T20:07:49.428 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.421+0000 7f0960a9e640 1 Processor -- start 2023-10-16T20:07:49.429 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.421+0000 7f0960a9e640 1 -- start start 2023-10-16T20:07:49.429 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.421+0000 7f0960a9e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f095c0d22e0 0x7f095c1562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:49.429 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.421+0000 7f0960a9e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f095c0d26c0 0x7f095c151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:49.429 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.421+0000 7f0960a9e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f095c15d9f0 0x7f095c14eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:49.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.421+0000 7f0960a9e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095c151e90 con 0x7f095c0d22e0 2023-10-16T20:07:49.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.421+0000 7f0960a9e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095c1520f0 con 0x7f095c0d26c0 2023-10-16T20:07:49.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.421+0000 7f0960a9e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f095c1523d0 con 0x7f095c15d9f0 2023-10-16T20:07:49.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.421+0000 7f095a59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f095c0d22e0 0x7f095c1562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:49.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.421+0000 7f0959d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f095c0d26c0 0x7f095c151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:49.431 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.421+0000 7f095a59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f095c0d22e0 0x7f095c1562b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45092/0 (socket says 172.21.15.88:45092) 2023-10-16T20:07:49.431 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.421+0000 7f095a59c640 1 -- 172.21.15.88:0/621773692 learned_addr learned my addr 172.21.15.88:0/621773692 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:49.431 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.421+0000 7f095ad9d640 1 --2- 172.21.15.88:0/621773692 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f095c15d9f0 0x7f095c14eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:49.431 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.421+0000 7f095a59c640 1 -- 172.21.15.88:0/621773692 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f095c15d9f0 msgr2=0x7f095c14eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:49.432 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.421+0000 7f095a59c640 1 --2- 172.21.15.88:0/621773692 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f095c15d9f0 0x7f095c14eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:49.432 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.421+0000 7f095a59c640 1 -- 172.21.15.88:0/621773692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f095c0d26c0 msgr2=0x7f095c151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:49.432 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.421+0000 7f095a59c640 1 --2- 172.21.15.88:0/621773692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f095c0d26c0 0x7f095c151950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:49.432 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.421+0000 7f095a59c640 1 -- 172.21.15.88:0/621773692 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0954009a70 con 0x7f095c0d22e0 2023-10-16T20:07:49.432 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.421+0000 7f0959d9b640 1 --2- 172.21.15.88:0/621773692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f095c0d26c0 0x7f095c151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:07:49.433 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.421+0000 7f095a59c640 1 --2- 172.21.15.88:0/621773692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f095c0d22e0 0x7f095c1562b0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f094c00c780 tx=0x7f094c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:49.433 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.421+0000 7f094affd640 1 -- 172.21.15.88:0/621773692 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c005ae0 con 0x7f095c0d22e0 2023-10-16T20:07:49.433 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.421+0000 7f094affd640 1 -- 172.21.15.88:0/621773692 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094c005c60 con 0x7f095c0d22e0 2023-10-16T20:07:49.433 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.421+0000 7f094affd640 1 -- 172.21.15.88:0/621773692 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c007070 con 0x7f095c0d22e0 2023-10-16T20:07:49.433 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.421+0000 7f0960a9e640 1 -- 172.21.15.88:0/621773692 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f095c0cab80 con 0x7f095c0d22e0 2023-10-16T20:07:49.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.421+0000 7f0960a9e640 1 -- 172.21.15.88:0/621773692 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f095c14f400 con 0x7f095c0d22e0 2023-10-16T20:07:49.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.421+0000 7f0960a9e640 1 -- 172.21.15.88:0/621773692 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f095c1525a0 con 0x7f095c0d22e0 2023-10-16T20:07:49.434 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.425+0000 7f094affd640 1 -- 172.21.15.88:0/621773692 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f094c0071d0 con 0x7f095c0d22e0 2023-10-16T20:07:49.435 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.425+0000 7f094affd640 1 --2- 172.21.15.88:0/621773692 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f093c0664d0 0x7f093c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:49.435 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.425+0000 7f094affd640 1 -- 172.21.15.88:0/621773692 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f094c0836c0 con 0x7f095c0d22e0 2023-10-16T20:07:49.436 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.425+0000 7f0959d9b640 1 --2- 172.21.15.88:0/621773692 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f093c0664d0 0x7f093c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:49.436 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.425+0000 7f0959d9b640 1 --2- 172.21.15.88:0/621773692 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f093c0664d0 0x7f093c068990 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f095c0b86c0 tx=0x7f0950009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:49.436 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.429+0000 7f094affd640 1 -- 172.21.15.88:0/621773692 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f094c050a00 con 0x7f095c0d22e0 2023-10-16T20:07:49.616 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.613+0000 7f0960a9e640 1 -- 172.21.15.88:0/621773692 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f095c04e9c0 con 0x7f095c0d22e0 2023-10-16T20:07:49.616 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.613+0000 7f094affd640 1 -- 172.21.15.88:0/621773692 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) v1 ==== 53+0+88 (secure 0 0 0) 0x7f094c017070 con 0x7f095c0d22e0 2023-10-16T20:07:49.616 INFO:teuthology.orchestra.run.smithi088.stdout:name: testcephfs1, metadata pool: testcephfs1_metadata, data pools: [testcephfs1_data ] 2023-10-16T20:07:49.618 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.613+0000 7f0960a9e640 1 -- 172.21.15.88:0/621773692 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f093c0664d0 msgr2=0x7f093c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:49.618 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.613+0000 7f0960a9e640 1 --2- 172.21.15.88:0/621773692 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f093c0664d0 0x7f093c068990 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f095c0b86c0 tx=0x7f0950009040 comp rx=0 tx=0).stop 2023-10-16T20:07:49.618 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.613+0000 7f0960a9e640 1 -- 172.21.15.88:0/621773692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f095c0d22e0 msgr2=0x7f095c1562b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:49.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.613+0000 7f0960a9e640 1 --2- 172.21.15.88:0/621773692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f095c0d22e0 0x7f095c1562b0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f094c00c780 tx=0x7f094c00cc40 comp rx=0 tx=0).stop 2023-10-16T20:07:49.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.613+0000 7f0960a9e640 1 -- 172.21.15.88:0/621773692 shutdown_connections 2023-10-16T20:07:49.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.613+0000 7f0960a9e640 1 --2- 172.21.15.88:0/621773692 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f093c0664d0 0x7f093c068990 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:49.619 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.613+0000 7f0960a9e640 1 --2- 172.21.15.88:0/621773692 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f095c15d9f0 0x7f095c14eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:49.620 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.613+0000 7f0960a9e640 1 --2- 172.21.15.88:0/621773692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f095c0d26c0 0x7f095c151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:49.620 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.613+0000 7f0960a9e640 1 --2- 172.21.15.88:0/621773692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f095c0d22e0 0x7f095c1562b0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:49.620 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.613+0000 7f0960a9e640 1 -- 172.21.15.88:0/621773692 >> 172.21.15.88:0/621773692 conn(0x7f095c0715c0 msgr2=0x7f095c0b6680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:49.620 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.613+0000 7f0960a9e640 1 -- 172.21.15.88:0/621773692 shutdown_connections 2023-10-16T20:07:49.620 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:49.613+0000 7f0960a9e640 1 -- 172.21.15.88:0/621773692 wait complete. 2023-10-16T20:07:49.637 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T20:07:49.637 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T20:07:49.638 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi088.front.sepia.ceph.com 2023-10-16T20:07:49.638 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T20:07:49.638 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T20:07:49.639 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T20:07:49.639 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T20:07:49.639 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T20:07:49.640 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T20:07:49.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:49.641 DEBUG:teuthology.orchestra.run.smithi088:> ip addr 2023-10-16T20:07:49.651 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T20:07:49.651 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T20:07:49.652 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T20:07:49.652 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:07:49.652 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2023-10-16T20:07:49.652 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:07:49.652 INFO:teuthology.orchestra.run.smithi088.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:07:49.652 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f0 brd ff:ff:ff:ff:ff:ff 2023-10-16T20:07:49.653 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f0 2023-10-16T20:07:49.653 INFO:teuthology.orchestra.run.smithi088.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:07:49.653 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f1 brd ff:ff:ff:ff:ff:ff 2023-10-16T20:07:49.653 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f1 2023-10-16T20:07:49.653 INFO:teuthology.orchestra.run.smithi088.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T20:07:49.654 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2c brd ff:ff:ff:ff:ff:ff 2023-10-16T20:07:49.654 INFO:teuthology.orchestra.run.smithi088.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T20:07:49.654 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2d brd ff:ff:ff:ff:ff:ff 2023-10-16T20:07:49.654 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.21.15.88/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-16T20:07:49.654 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft 42262sec preferred_lft 42262sec 2023-10-16T20:07:49.654 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link 2023-10-16T20:07:49.655 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:07:49.655 INFO:teuthology.orchestra.run.smithi088.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-16T20:07:49.655 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 02:42:d9:3b:fb:39 brd ff:ff:ff:ff:ff:ff 2023-10-16T20:07:49.655 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-16T20:07:49.655 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:07:49.655 INFO:teuthology.orchestra.run.smithi088.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T20:07:49.656 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-16T20:07:49.656 INFO:teuthology.orchestra.run.smithi088.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T20:07:49.656 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:07:49.656 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-16T20:07:49.656 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:07:49.656 INFO:teuthology.orchestra.run.smithi088.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T20:07:49.657 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:07:49.657 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-16T20:07:49.657 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:07:49.657 INFO:teuthology.orchestra.run.smithi088.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T20:07:49.657 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:07:49.658 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-16T20:07:49.658 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2023-10-16T20:07:49.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:49.658 DEBUG:teuthology.orchestra.run.smithi088:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T20:07:49.717 INFO:teuthology.orchestra.run.smithi088.stdout:1 2023-10-16T20:07:49.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:49.719 DEBUG:teuthology.orchestra.run.smithi088:> route 2023-10-16T20:07:49.774 INFO:teuthology.orchestra.run.smithi088.stdout:Kernel IP routing table 2023-10-16T20:07:49.774 INFO:teuthology.orchestra.run.smithi088.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T20:07:49.774 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-16T20:07:49.775 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T20:07:49.775 INFO:teuthology.orchestra.run.smithi088.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-16T20:07:49.775 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T20:07:49.775 INFO:teuthology.orchestra.run.smithi088.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:07:49.775 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:07:49.776 INFO:teuthology.orchestra.run.smithi088.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T20:07:49.776 INFO:teuthology.orchestra.run.smithi088.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T20:07:49.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:49.777 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec bash -c ' 2023-10-16T20:07:49.777 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2023-10-16T20:07:49.778 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T20:07:49.778 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T20:07:49.779 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T20:07:49.779 DEBUG:teuthology.orchestra.run.smithi088:> ') 2023-10-16T20:07:49.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:49.866 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2023-10-16T20:07:49.875 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T20:07:49.876 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T20:07:49.876 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T20:07:49.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:49.877 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set brx.0 up 2023-10-16T20:07:49.939 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-16T20:07:49.939 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T20:07:49.940 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T20:07:49.991 INFO:teuthology.orchestra.run.smithi088.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T20:07:49.992 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T20:07:49.992 DEBUG:teuthology.orchestra.run.smithi088:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T20:07:50.043 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2023-10-16T20:07:50.108 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T20:07:50.161 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2023-10-16T20:07:50.161 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2023-10-16T20:07:50.162 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2023-10-16T20:07:50.162 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2023-10-16T20:07:50.162 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2023-10-16T20:07:50.163 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2023-10-16T20:07:50.163 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2023-10-16T20:07:50.163 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2023-10-16T20:07:50.164 INFO:teuthology.orchestra.run.smithi088.stdout:/run/lock 2023-10-16T20:07:50.164 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2023-10-16T20:07:50.164 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2023-10-16T20:07:50.164 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2023-10-16T20:07:50.164 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T20:07:50.165 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2023-10-16T20:07:50.165 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2023-10-16T20:07:50.165 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2023-10-16T20:07:50.165 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2023-10-16T20:07:50.165 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/fs/nfsd 2023-10-16T20:07:50.166 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2023-10-16T20:07:50.166 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2023-10-16T20:07:50.166 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T20:07:50.166 INFO:teuthology.orchestra.run.smithi088.stdout:/run/rpc_pipefs 2023-10-16T20:07:50.167 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2023-10-16T20:07:50.167 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2023-10-16T20:07:50.167 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T20:07:50.167 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T20:07:50.167 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T20:07:50.167 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T20:07:50.168 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2023-10-16T20:07:50.168 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:07:50.168 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:07:50.168 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:07:50.169 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:07:50.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:50.170 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2023-10-16T20:07:50.219 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-16T20:07:50.220 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-16T20:07:50.267 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2023-10-16T20:07:50.288 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T20:07:50.341 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2023-10-16T20:07:50.341 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2023-10-16T20:07:50.342 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2023-10-16T20:07:50.342 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2023-10-16T20:07:50.342 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2023-10-16T20:07:50.342 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2023-10-16T20:07:50.343 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2023-10-16T20:07:50.343 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2023-10-16T20:07:50.343 INFO:teuthology.orchestra.run.smithi088.stdout:/run/lock 2023-10-16T20:07:50.343 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2023-10-16T20:07:50.343 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2023-10-16T20:07:50.343 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2023-10-16T20:07:50.344 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T20:07:50.344 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2023-10-16T20:07:50.344 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2023-10-16T20:07:50.344 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2023-10-16T20:07:50.344 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2023-10-16T20:07:50.345 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/fs/nfsd 2023-10-16T20:07:50.345 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2023-10-16T20:07:50.345 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2023-10-16T20:07:50.345 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T20:07:50.345 INFO:teuthology.orchestra.run.smithi088.stdout:/run/rpc_pipefs 2023-10-16T20:07:50.346 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2023-10-16T20:07:50.346 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2023-10-16T20:07:50.346 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T20:07:50.346 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T20:07:50.346 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T20:07:50.347 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T20:07:50.347 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2023-10-16T20:07:50.347 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:07:50.347 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:07:50.347 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:07:50.348 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:07:50.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:50.349 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2023-10-16T20:07:50.367 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:2023-10-16T20:07:50.361+0000 7f6236466580 -1 init, newargv = 0x5586d9752ca0 newargc=15 2023-10-16T20:07:50.367 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[52880]: starting ceph client 2023-10-16T20:07:50.374 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[52880]: starting fuse 2023-10-16T20:07:50.377 INFO:teuthology.orchestra.run.smithi088.stdout:43 2023-10-16T20:07:50.377 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-10-16T20:07:50.377 DEBUG:teuthology.orchestra.run.smithi088:> sudo stdin-killer -- python3 -c ' 2023-10-16T20:07:50.378 DEBUG:teuthology.orchestra.run.smithi088:> import glob 2023-10-16T20:07:50.378 DEBUG:teuthology.orchestra.run.smithi088:> import re 2023-10-16T20:07:50.379 DEBUG:teuthology.orchestra.run.smithi088:> import os 2023-10-16T20:07:50.379 DEBUG:teuthology.orchestra.run.smithi088:> import subprocess 2023-10-16T20:07:50.379 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:07:50.380 DEBUG:teuthology.orchestra.run.smithi088:> def _find_admin_socket(client_name): 2023-10-16T20:07:50.380 DEBUG:teuthology.orchestra.run.smithi088:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-16T20:07:50.380 DEBUG:teuthology.orchestra.run.smithi088:> files = glob.glob(asok_path) 2023-10-16T20:07:50.381 DEBUG:teuthology.orchestra.run.smithi088:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-16T20:07:50.381 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:07:50.382 DEBUG:teuthology.orchestra.run.smithi088:> # Given a non-glob path, it better be there 2023-10-16T20:07:50.382 DEBUG:teuthology.orchestra.run.smithi088:> if "*" not in asok_path: 2023-10-16T20:07:50.382 DEBUG:teuthology.orchestra.run.smithi088:> assert(len(files) == 1) 2023-10-16T20:07:50.383 DEBUG:teuthology.orchestra.run.smithi088:> return files[0] 2023-10-16T20:07:50.383 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:07:50.383 DEBUG:teuthology.orchestra.run.smithi088:> for f in files: 2023-10-16T20:07:50.384 DEBUG:teuthology.orchestra.run.smithi088:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-16T20:07:50.384 DEBUG:teuthology.orchestra.run.smithi088:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-16T20:07:50.384 DEBUG:teuthology.orchestra.run.smithi088:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-16T20:07:50.385 DEBUG:teuthology.orchestra.run.smithi088:> contents = proc_f.read() 2023-10-16T20:07:50.385 DEBUG:teuthology.orchestra.run.smithi088:> if mountpoint in contents: 2023-10-16T20:07:50.386 DEBUG:teuthology.orchestra.run.smithi088:> return f 2023-10-16T20:07:50.386 DEBUG:teuthology.orchestra.run.smithi088:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-16T20:07:50.386 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:07:50.387 DEBUG:teuthology.orchestra.run.smithi088:> print(_find_admin_socket("client.0")) 2023-10-16T20:07:50.387 DEBUG:teuthology.orchestra.run.smithi088:> ' 2023-10-16T20:07:50.500 INFO:teuthology.orchestra.run.smithi088.stdout:/var/run/ceph/ceph-client.0.52880.asok 2023-10-16T20:07:50.504 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-16T20:07:50.511 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52880.asok 2023-10-16T20:07:50.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:50.512 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52880.asok status 2023-10-16T20:07:50.683 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2023-10-16T20:07:50.684 INFO:teuthology.orchestra.run.smithi088.stdout: "metadata": { 2023-10-16T20:07:50.684 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_sha1": "4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0", 2023-10-16T20:07:50.684 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_version": "ceph version 18.0.0-6730-g4ba8a190 (4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0) reef (dev)", 2023-10-16T20:07:50.684 INFO:teuthology.orchestra.run.smithi088.stdout: "entity_id": "0", 2023-10-16T20:07:50.685 INFO:teuthology.orchestra.run.smithi088.stdout: "hostname": "smithi088", 2023-10-16T20:07:50.685 INFO:teuthology.orchestra.run.smithi088.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-16T20:07:50.685 INFO:teuthology.orchestra.run.smithi088.stdout: "pid": "52880", 2023-10-16T20:07:50.686 INFO:teuthology.orchestra.run.smithi088.stdout: "root": "/" 2023-10-16T20:07:50.686 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:07:50.686 INFO:teuthology.orchestra.run.smithi088.stdout: "dentry_count": 0, 2023-10-16T20:07:50.686 INFO:teuthology.orchestra.run.smithi088.stdout: "dentry_pinned_count": 0, 2023-10-16T20:07:50.686 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 6534, 2023-10-16T20:07:50.687 INFO:teuthology.orchestra.run.smithi088.stdout: "inst": { 2023-10-16T20:07:50.687 INFO:teuthology.orchestra.run.smithi088.stdout: "name": { 2023-10-16T20:07:50.687 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "client", 2023-10-16T20:07:50.687 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 6534 2023-10-16T20:07:50.687 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:07:50.688 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": { 2023-10-16T20:07:50.688 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2023-10-16T20:07:50.688 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "192.168.0.1:0", 2023-10-16T20:07:50.688 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 1864764234 2023-10-16T20:07:50.688 INFO:teuthology.orchestra.run.smithi088.stdout: } 2023-10-16T20:07:50.688 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:07:50.689 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": { 2023-10-16T20:07:50.689 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2023-10-16T20:07:50.689 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "192.168.0.1:0", 2023-10-16T20:07:50.689 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 1864764234 2023-10-16T20:07:50.689 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2023-10-16T20:07:50.690 INFO:teuthology.orchestra.run.smithi088.stdout: "inst_str": "client.6534 192.168.0.1:0/1864764234", 2023-10-16T20:07:50.690 INFO:teuthology.orchestra.run.smithi088.stdout: "addr_str": "192.168.0.1:0/1864764234", 2023-10-16T20:07:50.690 INFO:teuthology.orchestra.run.smithi088.stdout: "inode_count": 1, 2023-10-16T20:07:50.690 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_epoch": 28, 2023-10-16T20:07:50.691 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_epoch": 58, 2023-10-16T20:07:50.691 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_epoch_barrier": 0, 2023-10-16T20:07:50.691 INFO:teuthology.orchestra.run.smithi088.stdout: "blocklisted": false, 2023-10-16T20:07:50.691 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "testcephfs1" 2023-10-16T20:07:50.691 INFO:teuthology.orchestra.run.smithi088.stdout:} 2023-10-16T20:07:50.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:50.699 DEBUG:teuthology.orchestra.run.smithi088:> stat --file-system '--printf=%T 2023-10-16T20:07:50.699 DEBUG:teuthology.orchestra.run.smithi088:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-16T20:07:50.709 INFO:teuthology.orchestra.run.smithi088.stdout:fuseblk 2023-10-16T20:07:50.710 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-16T20:07:50.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:50.710 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T20:07:50.771 DEBUG:teuthology.orchestra.run.smithi088:> stdin-killer -- python3 -c ' 2023-10-16T20:07:50.772 DEBUG:teuthology.orchestra.run.smithi088:> import os 2023-10-16T20:07:50.772 DEBUG:teuthology.orchestra.run.smithi088:> import errno 2023-10-16T20:07:50.772 DEBUG:teuthology.orchestra.run.smithi088:> 2023-10-16T20:07:50.773 DEBUG:teuthology.orchestra.run.smithi088:> fd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR) 2023-10-16T20:07:50.773 DEBUG:teuthology.orchestra.run.smithi088:> try: 2023-10-16T20:07:50.773 DEBUG:teuthology.orchestra.run.smithi088:> if True: 2023-10-16T20:07:50.774 DEBUG:teuthology.orchestra.run.smithi088:> ret = os.read(fd, 1024) 2023-10-16T20:07:50.774 DEBUG:teuthology.orchestra.run.smithi088:> else: 2023-10-16T20:07:50.774 DEBUG:teuthology.orchestra.run.smithi088:> os.write(fd, b'"'"'content'"'"') 2023-10-16T20:07:50.775 DEBUG:teuthology.orchestra.run.smithi088:> except OSError as e: 2023-10-16T20:07:50.775 DEBUG:teuthology.orchestra.run.smithi088:> if e.errno != errno.EPERM: 2023-10-16T20:07:50.775 DEBUG:teuthology.orchestra.run.smithi088:> raise 2023-10-16T20:07:50.776 DEBUG:teuthology.orchestra.run.smithi088:> else: 2023-10-16T20:07:50.776 DEBUG:teuthology.orchestra.run.smithi088:> raise RuntimeError("client does not check permission of data pool") 2023-10-16T20:07:50.776 DEBUG:teuthology.orchestra.run.smithi088:> ' 2023-10-16T20:07:50.881 INFO:teuthology.orchestra.run.smithi088.stderr:Traceback (most recent call last): 2023-10-16T20:07:50.881 INFO:teuthology.orchestra.run.smithi088.stderr: File "", line 15, in 2023-10-16T20:07:50.881 INFO:teuthology.orchestra.run.smithi088.stderr:RuntimeError: client does not check permission of data pool 2023-10-16T20:07:50.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:50 stdin-killer INFO: command exited with status 1: exiting normally with same code! 2023-10-16T20:07:50.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T20:07:50.890 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=testcephfs1_data' 2023-10-16T20:07:51.076 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.073+0000 7f4d89d01640 1 Processor -- start 2023-10-16T20:07:51.076 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.073+0000 7f4d89d01640 1 -- start start 2023-10-16T20:07:51.077 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.073+0000 7f4d89d01640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4d8415a820 0x7f4d840cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:51.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.073+0000 7f4d89d01640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4d8415ac00 0x7f4d840b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:51.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.073+0000 7f4d89d01640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d840ce4b0 con 0x7f4d840b2930 2023-10-16T20:07:51.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.073+0000 7f4d89d01640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d840b2d40 con 0x7f4d8415ac00 2023-10-16T20:07:51.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.073+0000 7f4d89d01640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d8415e270 con 0x7f4d8415a820 2023-10-16T20:07:51.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.073+0000 7f4d7bfff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4d8415ac00 0x7f4d840b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:51.081 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.073+0000 7f4d7bfff640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4d8415ac00 0x7f4d840b23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45096/0 (socket says 172.21.15.88:45096) 2023-10-16T20:07:51.081 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.073+0000 7f4d7bfff640 1 -- 172.21.15.88:0/2478090118 learned_addr learned my addr 172.21.15.88:0/2478090118 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:51.081 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.073+0000 7f4d837fe640 1 --2- 172.21.15.88:0/2478090118 >> v2:172.21.15.94:3300/0 conn(0x7f4d8415a820 0x7f4d840cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:51.081 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.073+0000 7f4d7bfff640 1 -- 172.21.15.88:0/2478090118 >> v2:172.21.15.94:3300/0 conn(0x7f4d8415a820 msgr2=0x7f4d840cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:51.082 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.073+0000 7f4d7bfff640 1 --2- 172.21.15.88:0/2478090118 >> v2:172.21.15.94:3300/0 conn(0x7f4d8415a820 0x7f4d840cf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:51.082 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.073+0000 7f4d7bfff640 1 -- 172.21.15.88:0/2478090118 >> v1:172.21.15.94:6789/0 conn(0x7f4d840b2930 legacy=0x7f4d8406c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:51.082 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.073+0000 7f4d7bfff640 1 -- 172.21.15.88:0/2478090118 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d840ce700 con 0x7f4d8415ac00 2023-10-16T20:07:51.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.077+0000 7f4d7bfff640 1 --2- 172.21.15.88:0/2478090118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d8415ac00 0x7f4d840b23f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f4d840ce460 tx=0x7f4d6c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5848f74fdba6ce48 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:51.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.077+0000 7f4d82ffd640 1 -- 172.21.15.88:0/2478090118 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c0ac640 con 0x7f4d8415ac00 2023-10-16T20:07:51.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.077+0000 7f4d82ffd640 1 -- 172.21.15.88:0/2478090118 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6c0ac7a0 con 0x7f4d8415ac00 2023-10-16T20:07:51.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.077+0000 7f4d82ffd640 1 -- 172.21.15.88:0/2478090118 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c0acb30 con 0x7f4d8415ac00 2023-10-16T20:07:51.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.077+0000 7f4d89d01640 1 -- 172.21.15.88:0/2478090118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d8415ac00 msgr2=0x7f4d840b23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:51.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.077+0000 7f4d89d01640 1 --2- 172.21.15.88:0/2478090118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d8415ac00 0x7f4d840b23f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f4d840ce460 tx=0x7f4d6c01c4d0 comp rx=0 tx=0).stop 2023-10-16T20:07:51.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.077+0000 7f4d89d01640 1 -- 172.21.15.88:0/2478090118 shutdown_connections 2023-10-16T20:07:51.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.077+0000 7f4d89d01640 1 --2- 172.21.15.88:0/2478090118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d8415ac00 0x7f4d840b23f0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:51.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.077+0000 7f4d89d01640 1 --2- 172.21.15.88:0/2478090118 >> v2:172.21.15.94:3300/0 conn(0x7f4d8415a820 0x7f4d840cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:51.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.077+0000 7f4d89d01640 1 -- 172.21.15.88:0/2478090118 >> 172.21.15.88:0/2478090118 conn(0x7f4d840cd3f0 msgr2=0x7f4d840cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:51.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.077+0000 7f4d89d01640 1 -- 172.21.15.88:0/2478090118 shutdown_connections 2023-10-16T20:07:51.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.077+0000 7f4d89d01640 1 -- 172.21.15.88:0/2478090118 wait complete. 2023-10-16T20:07:51.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.077+0000 7f4d89d01640 1 Processor -- start 2023-10-16T20:07:51.086 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.077+0000 7f4d89d01640 1 -- start start 2023-10-16T20:07:51.087 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.081+0000 7f4d89d01640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d840b2930 0x7f4d840cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:51.088 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.081+0000 7f4d89d01640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4d8415a820 0x7f4d8414ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:51.088 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.081+0000 7f4d89d01640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d8415ac00 0x7f4d84153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:51.088 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.081+0000 7f4d89d01640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d84152550 con 0x7f4d840b2930 2023-10-16T20:07:51.088 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.081+0000 7f4d89d01640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d841516c0 con 0x7f4d8415ac00 2023-10-16T20:07:51.088 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.081+0000 7f4d89d01640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d840cc5a0 con 0x7f4d8415a820 2023-10-16T20:07:51.089 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.081+0000 7f4d837fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d840b2930 0x7f4d840cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:51.089 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.081+0000 7f4d837fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d840b2930 0x7f4d840cc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45106/0 (socket says 172.21.15.88:45106) 2023-10-16T20:07:51.089 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.081+0000 7f4d837fe640 1 -- 172.21.15.88:0/990512662 learned_addr learned my addr 172.21.15.88:0/990512662 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:51.089 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.081+0000 7f4d7bfff640 1 --2- 172.21.15.88:0/990512662 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4d8415a820 0x7f4d8414ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:51.089 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.081+0000 7f4d83fff640 1 --2- 172.21.15.88:0/990512662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d8415ac00 0x7f4d84153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:51.090 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.081+0000 7f4d837fe640 1 -- 172.21.15.88:0/990512662 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4d8415a820 msgr2=0x7f4d8414ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:51.090 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.081+0000 7f4d837fe640 1 --2- 172.21.15.88:0/990512662 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4d8415a820 0x7f4d8414ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:51.090 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.081+0000 7f4d837fe640 1 -- 172.21.15.88:0/990512662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d8415ac00 msgr2=0x7f4d84153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:51.090 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.081+0000 7f4d837fe640 1 --2- 172.21.15.88:0/990512662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d8415ac00 0x7f4d84153aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:51.090 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.081+0000 7f4d837fe640 1 -- 172.21.15.88:0/990512662 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6c0094d0 con 0x7f4d840b2930 2023-10-16T20:07:51.090 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.081+0000 7f4d837fe640 1 --2- 172.21.15.88:0/990512662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d840b2930 0x7f4d840cc060 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400d800 tx=0x7f4d7400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:51.091 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.081+0000 7f4d80ff9640 1 -- 172.21.15.88:0/990512662 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7400b6c0 con 0x7f4d840b2930 2023-10-16T20:07:51.091 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.081+0000 7f4d80ff9640 1 -- 172.21.15.88:0/990512662 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d7400b860 con 0x7f4d840b2930 2023-10-16T20:07:51.091 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.081+0000 7f4d89d01640 1 -- 172.21.15.88:0/990512662 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d841518d0 con 0x7f4d840b2930 2023-10-16T20:07:51.091 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.081+0000 7f4d80ff9640 1 -- 172.21.15.88:0/990512662 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7400bc10 con 0x7f4d840b2930 2023-10-16T20:07:51.091 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.081+0000 7f4d89d01640 1 -- 172.21.15.88:0/990512662 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d84151e90 con 0x7f4d840b2930 2023-10-16T20:07:51.091 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.081+0000 7f4d89d01640 1 -- 172.21.15.88:0/990512662 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d84152690 con 0x7f4d840b2930 2023-10-16T20:07:51.092 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.085+0000 7f4d80ff9640 1 -- 172.21.15.88:0/990512662 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4d7401e460 con 0x7f4d840b2930 2023-10-16T20:07:51.092 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.085+0000 7f4d80ff9640 1 --2- 172.21.15.88:0/990512662 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4d640664a0 0x7f4d64068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:51.093 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.085+0000 7f4d80ff9640 1 -- 172.21.15.88:0/990512662 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f4d74084490 con 0x7f4d840b2930 2023-10-16T20:07:51.093 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.085+0000 7f4d7bfff640 1 --2- 172.21.15.88:0/990512662 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4d640664a0 0x7f4d64068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:51.093 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.085+0000 7f4d7bfff640 1 --2- 172.21.15.88:0/990512662 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4d640664a0 0x7f4d64068960 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c0ae000 tx=0x7f4d6c0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:51.093 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.089+0000 7f4d80ff9640 1 -- 172.21.15.88:0/990512662 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4d84152690 con 0x7f4d840b2930 2023-10-16T20:07:51.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.289+0000 7f4d89d01640 1 -- 172.21.15.88:0/990512662 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=testcephfs1_data"]} v 0) v1 -- 0x7f4d84152690 con 0x7f4d840b2930 2023-10-16T20:07:51.298 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.293+0000 7f4d80ff9640 1 -- 172.21.15.88:0/990512662 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=testcephfs1_data"]}]=0 updated caps for client.0 v20) v1 ==== 189+0+146 (secure 0 0 0) 0x7f4d84152690 con 0x7f4d840b2930 2023-10-16T20:07:51.298 INFO:teuthology.orchestra.run.smithi088.stdout:[client.0] 2023-10-16T20:07:51.298 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQAvly1looeRAhAAeLshHOVIhQqoTtlPehorIg== 2023-10-16T20:07:51.299 INFO:teuthology.orchestra.run.smithi088.stdout: caps mds = "allow" 2023-10-16T20:07:51.299 INFO:teuthology.orchestra.run.smithi088.stdout: caps mon = "allow r" 2023-10-16T20:07:51.299 INFO:teuthology.orchestra.run.smithi088.stdout: caps osd = "allow rw pool=testcephfs1_data" 2023-10-16T20:07:51.300 INFO:teuthology.orchestra.run.smithi088.stderr:updated caps for client.0 2023-10-16T20:07:51.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.297+0000 7f4d89d01640 1 -- 172.21.15.88:0/990512662 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4d640664a0 msgr2=0x7f4d64068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:51.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.297+0000 7f4d89d01640 1 --2- 172.21.15.88:0/990512662 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4d640664a0 0x7f4d64068960 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c0ae000 tx=0x7f4d6c0a7000 comp rx=0 tx=0).stop 2023-10-16T20:07:51.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.297+0000 7f4d89d01640 1 -- 172.21.15.88:0/990512662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d840b2930 msgr2=0x7f4d840cc060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:51.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.297+0000 7f4d89d01640 1 --2- 172.21.15.88:0/990512662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d840b2930 0x7f4d840cc060 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400d800 tx=0x7f4d7400dcc0 comp rx=0 tx=0).stop 2023-10-16T20:07:51.302 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.297+0000 7f4d89d01640 1 -- 172.21.15.88:0/990512662 shutdown_connections 2023-10-16T20:07:51.302 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.297+0000 7f4d89d01640 1 --2- 172.21.15.88:0/990512662 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4d640664a0 0x7f4d64068960 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:51.302 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.297+0000 7f4d89d01640 1 --2- 172.21.15.88:0/990512662 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4d8415a820 0x7f4d8414ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:51.302 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.297+0000 7f4d89d01640 1 --2- 172.21.15.88:0/990512662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d8415ac00 0x7f4d84153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:51.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.297+0000 7f4d89d01640 1 --2- 172.21.15.88:0/990512662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d840b2930 0x7f4d840cc060 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:51.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.297+0000 7f4d89d01640 1 -- 172.21.15.88:0/990512662 >> 172.21.15.88:0/990512662 conn(0x7f4d840cd3f0 msgr2=0x7f4d8415b220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:51.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.297+0000 7f4d89d01640 1 -- 172.21.15.88:0/990512662 shutdown_connections 2023-10-16T20:07:51.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.297+0000 7f4d89d01640 1 -- 172.21.15.88:0/990512662 wait complete. 2023-10-16T20:07:51.319 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T20:07:51.320 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T20:07:51.418 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T20:07:51.482 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T20:07:51.483 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:07:51.483 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:07:51.493 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:07:51.672 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.669+0000 7f0813a79640 1 Processor -- start 2023-10-16T20:07:51.673 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.669+0000 7f0813a79640 1 -- start start 2023-10-16T20:07:51.674 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.669+0000 7f0813a79640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f080c0d26c0 0x7f080c161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:51.674 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.669+0000 7f0813a79640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f080c0b87e0 con 0x7f080c15d9f0 2023-10-16T20:07:51.674 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.669+0000 7f0813a79640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f080c15df10 con 0x7f080c0d22e0 2023-10-16T20:07:51.674 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.669+0000 7f0813a79640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f080c15e1f0 con 0x7f080c0d26c0 2023-10-16T20:07:51.675 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.669+0000 7f0811014640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f080c15d9f0 0x7f080c0cf2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:35876/0 (socket says 172.21.15.88:35876) 2023-10-16T20:07:51.675 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.669+0000 7f0811014640 1 -- 172.21.15.88:0/277538713 learned_addr learned my addr 172.21.15.88:0/277538713 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:51.675 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.669+0000 7f0812016640 1 --2- 172.21.15.88:0/277538713 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f080c0d26c0 0x7f080c161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:51.676 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.669+0000 7f0810813640 1 -- 172.21.15.88:0/277538713 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3214343282 0 0) 0x7f080c0b87e0 con 0x7f080c15d9f0 2023-10-16T20:07:51.676 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.669+0000 7f0810813640 1 -- 172.21.15.88:0/277538713 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07f8003530 con 0x7f080c15d9f0 2023-10-16T20:07:51.676 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.669+0000 7f0810813640 1 -- 172.21.15.88:0/277538713 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3894911655 0 0) 0x7f080c15df10 con 0x7f080c0d22e0 2023-10-16T20:07:51.677 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.669+0000 7f0810813640 1 -- 172.21.15.88:0/277538713 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f080c0b87e0 con 0x7f080c0d22e0 2023-10-16T20:07:51.677 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.669+0000 7f0812016640 1 -- 172.21.15.88:0/277538713 >> v1:172.21.15.94:6789/0 conn(0x7f080c0d22e0 legacy=0x7f080c15d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:51.677 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.669+0000 7f0812016640 1 -- 172.21.15.88:0/277538713 >> v1:172.21.15.88:6789/0 conn(0x7f080c15d9f0 legacy=0x7f080c0cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:51.677 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.669+0000 7f0810813640 1 -- 172.21.15.88:0/277538713 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4132995911 0 0) 0x7f07f8003530 con 0x7f080c15d9f0 2023-10-16T20:07:51.677 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.669+0000 7f0812016640 1 -- 172.21.15.88:0/277538713 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f080c160970 con 0x7f080c0d26c0 2023-10-16T20:07:51.679 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.673+0000 7f0812016640 1 --2- 172.21.15.88:0/277538713 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f080c0d26c0 0x7f080c161450 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0804009dc0 tx=0x7f080401c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4610e8b2e7f82b36 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:51.679 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.673+0000 7f0810813640 1 -- 172.21.15.88:0/277538713 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc001680 con 0x7f080c0d26c0 2023-10-16T20:07:51.679 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.673+0000 7f0810813640 1 -- 172.21.15.88:0/277538713 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08040a5520 con 0x7f080c0d26c0 2023-10-16T20:07:51.679 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.673+0000 7f0810813640 1 -- 172.21.15.88:0/277538713 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08040a58b0 con 0x7f080c0d26c0 2023-10-16T20:07:51.680 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.673+0000 7f0813a79640 1 -- 172.21.15.88:0/277538713 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f080c0d26c0 msgr2=0x7f080c161450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:51.680 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.673+0000 7f0813a79640 1 --2- 172.21.15.88:0/277538713 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f080c0d26c0 0x7f080c161450 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0804009dc0 tx=0x7f080401c7d0 comp rx=0 tx=0).stop 2023-10-16T20:07:51.680 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.673+0000 7f0813a79640 1 -- 172.21.15.88:0/277538713 shutdown_connections 2023-10-16T20:07:51.680 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.673+0000 7f0813a79640 1 --2- 172.21.15.88:0/277538713 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f080c0d26c0 0x7f080c161450 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:51.680 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.673+0000 7f0813a79640 1 -- 172.21.15.88:0/277538713 >> 172.21.15.88:0/277538713 conn(0x7f080c0715c0 msgr2=0x7f080c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:51.681 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.673+0000 7f0813a79640 1 -- 172.21.15.88:0/277538713 shutdown_connections 2023-10-16T20:07:51.681 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.673+0000 7f0813a79640 1 -- 172.21.15.88:0/277538713 wait complete. 2023-10-16T20:07:51.683 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.677+0000 7f0813a79640 1 Processor -- start 2023-10-16T20:07:51.683 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.677+0000 7f0813a79640 1 -- start start 2023-10-16T20:07:51.683 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.677+0000 7f0813a79640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f080c0d22e0 0x7f080c0cc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:51.683 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.677+0000 7f0813a79640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f080c0d26c0 0x7f080c0cc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:51.684 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.677+0000 7f0813a79640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f080c15d9f0 0x7f080c0c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:51.684 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.677+0000 7f0813a79640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080c149050 con 0x7f080c0d26c0 2023-10-16T20:07:51.684 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.677+0000 7f0813a79640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080c0c64e0 con 0x7f080c15d9f0 2023-10-16T20:07:51.684 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.677+0000 7f0813a79640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f080c0c6a60 con 0x7f080c0d22e0 2023-10-16T20:07:51.685 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.677+0000 7f0811014640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f080c0d26c0 0x7f080c0cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:51.685 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.677+0000 7f0811014640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f080c0d26c0 0x7f080c0cc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45108/0 (socket says 172.21.15.88:45108) 2023-10-16T20:07:51.685 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.677+0000 7f0811815640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f080c0d22e0 0x7f080c0cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:51.685 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.677+0000 7f0811014640 1 -- 172.21.15.88:0/436659605 learned_addr learned my addr 172.21.15.88:0/436659605 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:51.685 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.677+0000 7f0812016640 1 --2- 172.21.15.88:0/436659605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f080c15d9f0 0x7f080c0c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:51.685 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.677+0000 7f0811014640 1 -- 172.21.15.88:0/436659605 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f080c0d22e0 msgr2=0x7f080c0cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:51.686 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.677+0000 7f0811014640 1 --2- 172.21.15.88:0/436659605 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f080c0d22e0 0x7f080c0cc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:51.686 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.677+0000 7f0811014640 1 -- 172.21.15.88:0/436659605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f080c15d9f0 msgr2=0x7f080c0c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:51.686 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.677+0000 7f0811014640 1 --2- 172.21.15.88:0/436659605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f080c15d9f0 0x7f080c0c7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:51.686 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.677+0000 7f0811014640 1 -- 172.21.15.88:0/436659605 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0804009a70 con 0x7f080c0d26c0 2023-10-16T20:07:51.686 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.677+0000 7f0811014640 1 --2- 172.21.15.88:0/436659605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f080c0d26c0 0x7f080c0cc690 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00c730 tx=0x7f07fc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:51.687 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.677+0000 7f08027fc640 1 -- 172.21.15.88:0/436659605 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc005ae0 con 0x7f080c0d26c0 2023-10-16T20:07:51.687 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.677+0000 7f08027fc640 1 -- 172.21.15.88:0/436659605 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07fc005c40 con 0x7f080c0d26c0 2023-10-16T20:07:51.687 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.677+0000 7f08027fc640 1 -- 172.21.15.88:0/436659605 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc007270 con 0x7f080c0d26c0 2023-10-16T20:07:51.687 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.677+0000 7f0813a79640 1 -- 172.21.15.88:0/436659605 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f080c1493e0 con 0x7f080c0d26c0 2023-10-16T20:07:51.687 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.677+0000 7f0813a79640 1 -- 172.21.15.88:0/436659605 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f080c0c66f0 con 0x7f080c0d26c0 2023-10-16T20:07:51.688 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.681+0000 7f0813a79640 1 -- 172.21.15.88:0/436659605 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f080c067010 con 0x7f080c0d26c0 2023-10-16T20:07:51.689 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.681+0000 7f08027fc640 1 -- 172.21.15.88:0/436659605 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f07fc0073d0 con 0x7f080c0d26c0 2023-10-16T20:07:51.689 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.681+0000 7f08027fc640 1 --2- 172.21.15.88:0/436659605 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f07f00664a0 0x7f07f0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:51.689 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.681+0000 7f08027fc640 1 -- 172.21.15.88:0/436659605 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f07fc083ce0 con 0x7f080c0d26c0 2023-10-16T20:07:51.689 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.681+0000 7f0811815640 1 --2- 172.21.15.88:0/436659605 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f07f00664a0 0x7f07f0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:51.689 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.681+0000 7f0811815640 1 --2- 172.21.15.88:0/436659605 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f07f00664a0 0x7f07f0068960 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f08080011f0 tx=0x7f0808009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:51.690 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.685+0000 7f08027fc640 1 -- 172.21.15.88:0/436659605 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f07fc051010 con 0x7f080c0d26c0 2023-10-16T20:07:51.839 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.833+0000 7f0813a79640 1 -- 172.21.15.88:0/436659605 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f080c04e9c0 con 0x7f080c0d26c0 2023-10-16T20:07:51.840 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.837+0000 7f08027fc640 1 -- 172.21.15.88:0/436659605 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16718 (secure 0 0 0) 0x7f07fc001a30 con 0x7f080c0d26c0 2023-10-16T20:07:51.841 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:07:51.841 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":58,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:07:27.206614+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T20:07:24.001121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"testcephfs1_data","create_time":"2023-10-16T20:07:24.636209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:07:51.843 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.837+0000 7f0813a79640 1 -- 172.21.15.88:0/436659605 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f07f00664a0 msgr2=0x7f07f0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:51.843 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.837+0000 7f0813a79640 1 --2- 172.21.15.88:0/436659605 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f07f00664a0 0x7f07f0068960 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f08080011f0 tx=0x7f0808009040 comp rx=0 tx=0).stop 2023-10-16T20:07:51.843 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.837+0000 7f0813a79640 1 -- 172.21.15.88:0/436659605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f080c0d26c0 msgr2=0x7f080c0cc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:51.844 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.837+0000 7f0813a79640 1 --2- 172.21.15.88:0/436659605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f080c0d26c0 0x7f080c0cc690 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00c730 tx=0x7f07fc00cbf0 comp rx=0 tx=0).stop 2023-10-16T20:07:51.844 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.837+0000 7f0813a79640 1 -- 172.21.15.88:0/436659605 shutdown_connections 2023-10-16T20:07:51.844 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.837+0000 7f0813a79640 1 --2- 172.21.15.88:0/436659605 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f07f00664a0 0x7f07f0068960 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:51.844 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.837+0000 7f0813a79640 1 --2- 172.21.15.88:0/436659605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f080c15d9f0 0x7f080c0c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:51.844 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.837+0000 7f0813a79640 1 --2- 172.21.15.88:0/436659605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f080c0d26c0 0x7f080c0cc690 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:51.845 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.837+0000 7f0813a79640 1 --2- 172.21.15.88:0/436659605 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f080c0d22e0 0x7f080c0cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:51.845 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.837+0000 7f0813a79640 1 -- 172.21.15.88:0/436659605 >> 172.21.15.88:0/436659605 conn(0x7f080c0715c0 msgr2=0x7f080c0b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:51.845 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.837+0000 7f0813a79640 1 -- 172.21.15.88:0/436659605 shutdown_connections 2023-10-16T20:07:51.845 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:51.837+0000 7f0813a79640 1 -- 172.21.15.88:0/436659605 wait complete. 2023-10-16T20:07:51.862 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'testcephfs1_metadata', 'create_time': '2023-10-16T20:07:24.001121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'testcephfs1_data', 'create_time': '2023-10-16T20:07:24.636209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T20:07:51.863 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:07:52.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.000+0000 7f639a73e640 1 Processor -- start 2023-10-16T20:07:52.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.000+0000 7f639a73e640 1 -- start start 2023-10-16T20:07:52.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.000+0000 7f639a73e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f63940b4260 0x7f639415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:52.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.000+0000 7f639a73e640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f639406d3f0 0x7f639406d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:52.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.000+0000 7f639a73e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f639415ad30 con 0x7f63940b0da0 2023-10-16T20:07:52.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.000+0000 7f639a73e640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f639406dd60 con 0x7f639406d3f0 2023-10-16T20:07:52.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.000+0000 7f639a73e640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63940b4640 con 0x7f63940b4260 2023-10-16T20:07:52.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.000+0000 7f6398cdb640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f639406d3f0 0x7f639406d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:52.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.000+0000 7f6398cdb640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f639406d3f0 0x7f639406d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57718/0 (socket says 172.21.15.88:57718) 2023-10-16T20:07:52.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.000+0000 7f6398cdb640 1 -- 172.21.15.88:0/3851495433 learned_addr learned my addr 172.21.15.88:0/3851495433 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:52.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.000+0000 7f6393fff640 1 --2- 172.21.15.88:0/3851495433 >> v2:172.21.15.94:3300/0 conn(0x7f63940b4260 0x7f639415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:52.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.000+0000 7f6398cdb640 1 -- 172.21.15.88:0/3851495433 >> v2:172.21.15.94:3300/0 conn(0x7f63940b4260 msgr2=0x7f639415dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:52.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.000+0000 7f6398cdb640 1 --2- 172.21.15.88:0/3851495433 >> v2:172.21.15.94:3300/0 conn(0x7f63940b4260 0x7f639415dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:52.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.000+0000 7f6398cdb640 1 -- 172.21.15.88:0/3851495433 >> v1:172.21.15.94:6789/0 conn(0x7f63940b0da0 legacy=0x7f6394159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:52.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.000+0000 7f6398cdb640 1 -- 172.21.15.88:0/3851495433 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f639415a770 con 0x7f639406d3f0 2023-10-16T20:07:52.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.004+0000 7f6398cdb640 1 --2- 172.21.15.88:0/3851495433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f639406d3f0 0x7f639406d820 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f63940b1a80 tx=0x7f638c01eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f18be23c68b6c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:52.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.004+0000 7f6392ffd640 1 -- 172.21.15.88:0/3851495433 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c00b2d0 con 0x7f639406d3f0 2023-10-16T20:07:52.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.004+0000 7f6392ffd640 1 -- 172.21.15.88:0/3851495433 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f638c00b430 con 0x7f639406d3f0 2023-10-16T20:07:52.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.004+0000 7f6392ffd640 1 -- 172.21.15.88:0/3851495433 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c0a7c00 con 0x7f639406d3f0 2023-10-16T20:07:52.010 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.004+0000 7f639a73e640 1 -- 172.21.15.88:0/3851495433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f639406d3f0 msgr2=0x7f639406d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:52.010 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.004+0000 7f639a73e640 1 --2- 172.21.15.88:0/3851495433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f639406d3f0 0x7f639406d820 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f63940b1a80 tx=0x7f638c01eb10 comp rx=0 tx=0).stop 2023-10-16T20:07:52.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.004+0000 7f639a73e640 1 -- 172.21.15.88:0/3851495433 shutdown_connections 2023-10-16T20:07:52.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.004+0000 7f639a73e640 1 --2- 172.21.15.88:0/3851495433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f639406d3f0 0x7f639406d820 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:52.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.004+0000 7f639a73e640 1 --2- 172.21.15.88:0/3851495433 >> v2:172.21.15.94:3300/0 conn(0x7f63940b4260 0x7f639415dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:52.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.004+0000 7f639a73e640 1 -- 172.21.15.88:0/3851495433 >> 172.21.15.88:0/3851495433 conn(0x7f63940cb080 msgr2=0x7f63940cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:52.011 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.004+0000 7f639a73e640 1 -- 172.21.15.88:0/3851495433 shutdown_connections 2023-10-16T20:07:52.012 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.004+0000 7f639a73e640 1 -- 172.21.15.88:0/3851495433 wait complete. 2023-10-16T20:07:52.013 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.004+0000 7f639a73e640 1 Processor -- start 2023-10-16T20:07:52.014 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.008+0000 7f639a73e640 1 -- start start 2023-10-16T20:07:52.014 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.008+0000 7f639a73e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f639406d3f0 0x7f6394153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:52.014 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.008+0000 7f639a73e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63940b0da0 0x7f639413ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:52.014 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.008+0000 7f639a73e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63940b4260 0x7f639406c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:52.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.008+0000 7f639a73e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6394155020 con 0x7f63940b0da0 2023-10-16T20:07:52.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.008+0000 7f639a73e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63941532e0 con 0x7f63940b4260 2023-10-16T20:07:52.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.008+0000 7f639a73e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63941f5ec0 con 0x7f639406d3f0 2023-10-16T20:07:52.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.008+0000 7f63937fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63940b0da0 0x7f639413ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:52.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.008+0000 7f63937fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63940b0da0 0x7f639413ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57734/0 (socket says 172.21.15.88:57734) 2023-10-16T20:07:52.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.008+0000 7f63937fe640 1 -- 172.21.15.88:0/2781907774 learned_addr learned my addr 172.21.15.88:0/2781907774 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:52.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.008+0000 7f6393fff640 1 --2- 172.21.15.88:0/2781907774 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f639406d3f0 0x7f6394153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:52.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.008+0000 7f6398cdb640 1 --2- 172.21.15.88:0/2781907774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63940b4260 0x7f639406c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:52.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.008+0000 7f63937fe640 1 -- 172.21.15.88:0/2781907774 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f639406d3f0 msgr2=0x7f6394153770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:52.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.008+0000 7f63937fe640 1 --2- 172.21.15.88:0/2781907774 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f639406d3f0 0x7f6394153770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:52.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.008+0000 7f63937fe640 1 -- 172.21.15.88:0/2781907774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63940b4260 msgr2=0x7f639406c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:52.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.008+0000 7f63937fe640 1 --2- 172.21.15.88:0/2781907774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63940b4260 0x7f639406c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:52.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.008+0000 7f63937fe640 1 -- 172.21.15.88:0/2781907774 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f638c00bd40 con 0x7f63940b0da0 2023-10-16T20:07:52.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.008+0000 7f6393fff640 1 --2- 172.21.15.88:0/2781907774 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f639406d3f0 0x7f6394153770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:07:52.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.008+0000 7f6398cdb640 1 --2- 172.21.15.88:0/2781907774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63940b4260 0x7f639406c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:07:52.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.008+0000 7f63937fe640 1 --2- 172.21.15.88:0/2781907774 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63940b0da0 0x7f639413ffc0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f638800c500 tx=0x7f638800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:52.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.008+0000 7f6390ff9640 1 -- 172.21.15.88:0/2781907774 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6388005220 con 0x7f63940b0da0 2023-10-16T20:07:52.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.008+0000 7f6390ff9640 1 -- 172.21.15.88:0/2781907774 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6388005380 con 0x7f63940b0da0 2023-10-16T20:07:52.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.008+0000 7f639a73e640 1 -- 172.21.15.88:0/2781907774 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63941f5be0 con 0x7f63940b0da0 2023-10-16T20:07:52.019 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.008+0000 7f6390ff9640 1 -- 172.21.15.88:0/2781907774 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6388007270 con 0x7f63940b0da0 2023-10-16T20:07:52.019 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.008+0000 7f639a73e640 1 -- 172.21.15.88:0/2781907774 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6394152f70 con 0x7f63940b0da0 2023-10-16T20:07:52.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.008+0000 7f639a73e640 1 -- 172.21.15.88:0/2781907774 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f639404e9c0 con 0x7f63940b0da0 2023-10-16T20:07:52.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.008+0000 7f6390ff9640 1 -- 172.21.15.88:0/2781907774 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f63880073d0 con 0x7f63940b0da0 2023-10-16T20:07:52.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.012+0000 7f6390ff9640 1 --2- 172.21.15.88:0/2781907774 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f63800664a0 0x7f6380068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:52.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.012+0000 7f6390ff9640 1 -- 172.21.15.88:0/2781907774 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f6388083960 con 0x7f63940b0da0 2023-10-16T20:07:52.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.012+0000 7f6393fff640 1 --2- 172.21.15.88:0/2781907774 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f63800664a0 0x7f6380068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:52.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.012+0000 7f6393fff640 1 --2- 172.21.15.88:0/2781907774 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f63800664a0 0x7f6380068960 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6384009840 tx=0x7f6384008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:52.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.016+0000 7f6390ff9640 1 -- 172.21.15.88:0/2781907774 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6388050bf0 con 0x7f63940b0da0 2023-10-16T20:07:52.170 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.164+0000 7f639a73e640 1 -- 172.21.15.88:0/2781907774 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6394071500 con 0x7f63940b0da0 2023-10-16T20:07:52.171 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.164+0000 7f6390ff9640 1 -- 172.21.15.88:0/2781907774 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6388054210 con 0x7f63940b0da0 2023-10-16T20:07:52.171 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:07:52.173 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.168+0000 7f639a73e640 1 -- 172.21.15.88:0/2781907774 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f63800664a0 msgr2=0x7f6380068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:52.173 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.168+0000 7f639a73e640 1 --2- 172.21.15.88:0/2781907774 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f63800664a0 0x7f6380068960 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6384009840 tx=0x7f6384008040 comp rx=0 tx=0).stop 2023-10-16T20:07:52.173 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.168+0000 7f639a73e640 1 -- 172.21.15.88:0/2781907774 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63940b0da0 msgr2=0x7f639413ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:52.174 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.168+0000 7f639a73e640 1 --2- 172.21.15.88:0/2781907774 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63940b0da0 0x7f639413ffc0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f638800c500 tx=0x7f638800c9c0 comp rx=0 tx=0).stop 2023-10-16T20:07:52.174 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.168+0000 7f639a73e640 1 -- 172.21.15.88:0/2781907774 shutdown_connections 2023-10-16T20:07:52.174 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.168+0000 7f639a73e640 1 --2- 172.21.15.88:0/2781907774 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f63800664a0 0x7f6380068960 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:52.174 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.168+0000 7f639a73e640 1 --2- 172.21.15.88:0/2781907774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63940b4260 0x7f639406c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:52.174 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.168+0000 7f639a73e640 1 --2- 172.21.15.88:0/2781907774 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63940b0da0 0x7f639413ffc0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:52.175 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.168+0000 7f639a73e640 1 --2- 172.21.15.88:0/2781907774 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f639406d3f0 0x7f6394153770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:52.175 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.168+0000 7f639a73e640 1 -- 172.21.15.88:0/2781907774 >> 172.21.15.88:0/2781907774 conn(0x7f63940cb080 msgr2=0x7f63940b01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:52.175 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.168+0000 7f639a73e640 1 -- 172.21.15.88:0/2781907774 shutdown_connections 2023-10-16T20:07:52.175 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.168+0000 7f639a73e640 1 -- 172.21.15.88:0/2781907774 wait complete. 2023-10-16T20:07:52.192 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_metadata pg_num 2023-10-16T20:07:52.334 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.328+0000 7fd72bcea640 1 Processor -- start 2023-10-16T20:07:52.334 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.328+0000 7fd72bcea640 1 -- start start 2023-10-16T20:07:52.335 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.328+0000 7fd72bcea640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd7240cdfe0 0x7fd724163e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:52.335 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.328+0000 7fd72bcea640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7240cb1b0 con 0x7fd72406d290 2023-10-16T20:07:52.335 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.328+0000 7fd72bcea640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7240ca630 con 0x7fd72406d670 2023-10-16T20:07:52.336 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.328+0000 7fd72bcea640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7240ca910 con 0x7fd7240cdfe0 2023-10-16T20:07:52.336 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.328+0000 7fd729a86640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fd72406d290 0x7fd7240c9de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:56360/0 (socket says 172.21.15.88:56360) 2023-10-16T20:07:52.336 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.328+0000 7fd729a86640 1 -- 172.21.15.88:0/534343232 learned_addr learned my addr 172.21.15.88:0/534343232 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:52.338 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.332+0000 7fd72a287640 1 --2- 172.21.15.88:0/534343232 >> v2:172.21.15.94:3300/0 conn(0x7fd7240cdfe0 0x7fd724163e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:52.338 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.332+0000 7fd728a84640 1 -- 172.21.15.88:0/534343232 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1242926963 0 0) 0x7fd7240cb1b0 con 0x7fd72406d290 2023-10-16T20:07:52.338 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.332+0000 7fd728a84640 1 -- 172.21.15.88:0/534343232 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd710005740 con 0x7fd72406d290 2023-10-16T20:07:52.338 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.332+0000 7fd728a84640 1 -- 172.21.15.88:0/534343232 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1038973209 0 0) 0x7fd710005740 con 0x7fd72406d290 2023-10-16T20:07:52.339 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.332+0000 7fd728a84640 1 -- 172.21.15.88:0/534343232 >> v2:172.21.15.94:3300/0 conn(0x7fd7240cdfe0 msgr2=0x7fd724163e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:52.339 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.332+0000 7fd728a84640 1 --2- 172.21.15.88:0/534343232 >> v2:172.21.15.94:3300/0 conn(0x7fd7240cdfe0 0x7fd724163e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:52.339 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.332+0000 7fd728a84640 1 -- 172.21.15.88:0/534343232 >> v1:172.21.15.94:6789/0 conn(0x7fd72406d670 legacy=0x7fd72415ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:52.339 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.332+0000 7fd728a84640 1 -- 172.21.15.88:0/534343232 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7240cabf0 con 0x7fd72406d290 2023-10-16T20:07:52.340 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.332+0000 7fd728a84640 1 -- 172.21.15.88:0/534343232 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7fd720001680 con 0x7fd72406d290 2023-10-16T20:07:52.340 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.332+0000 7fd728a84640 1 -- 172.21.15.88:0/534343232 >> v1:172.21.15.88:6789/0 conn(0x7fd72406d290 legacy=0x7fd7240c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:52.340 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.332+0000 7fd728a84640 1 --2- 172.21.15.88:0/534343232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd710005b90 0x7fd710005f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:52.340 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.332+0000 7fd728a84640 1 --2- 172.21.15.88:0/534343232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7100069e0 0x7fd710006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:52.340 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.332+0000 7fd728a84640 1 --2- 172.21.15.88:0/534343232 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd7100073a0 0x7fd71000dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:52.340 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.332+0000 7fd728a84640 1 -- 172.21.15.88:0/534343232 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7240ca910 con 0x7fd710005b90 2023-10-16T20:07:52.341 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.332+0000 7fd728a84640 1 -- 172.21.15.88:0/534343232 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71000e2a0 con 0x7fd7100069e0 2023-10-16T20:07:52.341 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.332+0000 7fd728a84640 1 -- 172.21.15.88:0/534343232 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd71000e460 con 0x7fd7100073a0 2023-10-16T20:07:52.341 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.332+0000 7fd729a86640 1 --2- 172.21.15.88:0/534343232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd710005b90 0x7fd710005f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:52.341 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.332+0000 7fd72a287640 1 --2- 172.21.15.88:0/534343232 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd7100073a0 0x7fd71000dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:52.341 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.332+0000 7fd729285640 1 --2- 172.21.15.88:0/534343232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7100069e0 0x7fd710006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:52.342 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.332+0000 7fd729a86640 1 -- 172.21.15.88:0/534343232 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd7100073a0 msgr2=0x7fd71000dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:52.342 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.332+0000 7fd729a86640 1 --2- 172.21.15.88:0/534343232 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd7100073a0 0x7fd71000dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:52.342 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.332+0000 7fd729a86640 1 -- 172.21.15.88:0/534343232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7100069e0 msgr2=0x7fd710006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:52.342 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.332+0000 7fd729a86640 1 --2- 172.21.15.88:0/534343232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7100069e0 0x7fd710006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:52.342 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.332+0000 7fd729a86640 1 -- 172.21.15.88:0/534343232 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd710005740 con 0x7fd710005b90 2023-10-16T20:07:52.343 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.332+0000 7fd72a287640 1 -- 172.21.15.88:0/534343232 reap_dead start 2023-10-16T20:07:52.345 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.336+0000 7fd729a86640 1 --2- 172.21.15.88:0/534343232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd710005b90 0x7fd710005f70 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fd7200050a0 tx=0x7fd72001d1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c748cde7dfde3e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:52.345 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.336+0000 7fd728a84640 1 -- 172.21.15.88:0/534343232 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd720005ae0 con 0x7fd710005b90 2023-10-16T20:07:52.345 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.336+0000 7fd728a84640 1 -- 172.21.15.88:0/534343232 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7200a8070 con 0x7fd710005b90 2023-10-16T20:07:52.346 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.336+0000 7fd728a84640 1 -- 172.21.15.88:0/534343232 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd720008110 con 0x7fd710005b90 2023-10-16T20:07:52.346 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.336+0000 7fd72bcea640 1 -- 172.21.15.88:0/534343232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd710005b90 msgr2=0x7fd710005f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:52.346 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.336+0000 7fd72bcea640 1 --2- 172.21.15.88:0/534343232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd710005b90 0x7fd710005f70 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fd7200050a0 tx=0x7fd72001d1b0 comp rx=0 tx=0).stop 2023-10-16T20:07:52.346 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.336+0000 7fd72bcea640 1 -- 172.21.15.88:0/534343232 shutdown_connections 2023-10-16T20:07:52.347 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.336+0000 7fd72bcea640 1 --2- 172.21.15.88:0/534343232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd710005b90 0x7fd710005f70 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:52.347 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.336+0000 7fd72bcea640 1 -- 172.21.15.88:0/534343232 >> 172.21.15.88:0/534343232 conn(0x7fd7240b0970 msgr2=0x7fd7240b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:52.347 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.336+0000 7fd72bcea640 1 -- 172.21.15.88:0/534343232 shutdown_connections 2023-10-16T20:07:52.347 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.336+0000 7fd72bcea640 1 -- 172.21.15.88:0/534343232 wait complete. 2023-10-16T20:07:52.347 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.336+0000 7fd72bcea640 1 Processor -- start 2023-10-16T20:07:52.347 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.336+0000 7fd72bcea640 1 -- start start 2023-10-16T20:07:52.348 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.336+0000 7fd72bcea640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd710005b90 0x7fd7240b3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:52.348 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.336+0000 7fd72bcea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7240b3ed0 0x7fd7241f4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:52.348 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.340+0000 7fd72bcea640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7241f5130 0x7fd72415aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:52.348 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.340+0000 7fd72bcea640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7241432c0 con 0x7fd7241f5130 2023-10-16T20:07:52.348 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.340+0000 7fd72bcea640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7240b4430 con 0x7fd7240b3ed0 2023-10-16T20:07:52.348 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.340+0000 7fd72bcea640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7240b4710 con 0x7fd710005b90 2023-10-16T20:07:52.349 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.340+0000 7fd72a287640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7241f5130 0x7fd72415aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:52.349 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.340+0000 7fd72a287640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7241f5130 0x7fd72415aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57754/0 (socket says 172.21.15.88:57754) 2023-10-16T20:07:52.349 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.340+0000 7fd72a287640 1 -- 172.21.15.88:0/1886980205 learned_addr learned my addr 172.21.15.88:0/1886980205 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:52.349 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.340+0000 7fd729285640 1 --2- 172.21.15.88:0/1886980205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7240b3ed0 0x7fd7241f4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:52.349 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.340+0000 7fd729a86640 1 --2- 172.21.15.88:0/1886980205 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd710005b90 0x7fd7240b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:52.350 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.340+0000 7fd72a287640 1 -- 172.21.15.88:0/1886980205 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd710005b90 msgr2=0x7fd7240b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:52.350 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.340+0000 7fd72a287640 1 --2- 172.21.15.88:0/1886980205 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd710005b90 0x7fd7240b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:52.350 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.340+0000 7fd72a287640 1 -- 172.21.15.88:0/1886980205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7240b3ed0 msgr2=0x7fd7241f4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:52.350 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.340+0000 7fd72a287640 1 --2- 172.21.15.88:0/1886980205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7240b3ed0 0x7fd7241f4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:52.350 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.340+0000 7fd72a287640 1 -- 172.21.15.88:0/1886980205 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7240b49f0 con 0x7fd7241f5130 2023-10-16T20:07:52.351 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.340+0000 7fd72a287640 1 --2- 172.21.15.88:0/1886980205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7241f5130 0x7fd72415aa00 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fd7240b4ff0 tx=0x7fd71c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:52.351 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.340+0000 7fd71a7fc640 1 -- 172.21.15.88:0/1886980205 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71c00e070 con 0x7fd7241f5130 2023-10-16T20:07:52.351 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.340+0000 7fd71a7fc640 1 -- 172.21.15.88:0/1886980205 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71c011db0 con 0x7fd7241f5130 2023-10-16T20:07:52.351 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.340+0000 7fd71a7fc640 1 -- 172.21.15.88:0/1886980205 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71c0039a0 con 0x7fd7241f5130 2023-10-16T20:07:52.351 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.340+0000 7fd72bcea640 1 -- 172.21.15.88:0/1886980205 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd724142f50 con 0x7fd7241f5130 2023-10-16T20:07:52.351 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.340+0000 7fd72bcea640 1 -- 172.21.15.88:0/1886980205 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd72415c240 con 0x7fd7241f5130 2023-10-16T20:07:52.352 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.340+0000 7fd72bcea640 1 -- 172.21.15.88:0/1886980205 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7240665f0 con 0x7fd7241f5130 2023-10-16T20:07:52.353 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.340+0000 7fd71a7fc640 1 -- 172.21.15.88:0/1886980205 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd71c003b00 con 0x7fd7241f5130 2023-10-16T20:07:52.353 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.340+0000 7fd71a7fc640 1 --2- 172.21.15.88:0/1886980205 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd7080664a0 0x7fd708068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:52.353 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.344+0000 7fd71a7fc640 1 -- 172.21.15.88:0/1886980205 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fd71c0847a0 con 0x7fd7241f5130 2023-10-16T20:07:52.353 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.344+0000 7fd729a86640 1 --2- 172.21.15.88:0/1886980205 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd7080664a0 0x7fd708068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:52.353 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.344+0000 7fd729a86640 1 --2- 172.21.15.88:0/1886980205 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd7080664a0 0x7fd708068960 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd72000a000 tx=0x7fd720001de0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:52.354 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.344+0000 7fd71a7fc640 1 -- 172.21.15.88:0/1886980205 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd71c051a20 con 0x7fd7241f5130 2023-10-16T20:07:52.501 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.496+0000 7fd72bcea640 1 -- 172.21.15.88:0/1886980205 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd72406e340 con 0x7fd7241f5130 2023-10-16T20:07:52.502 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.496+0000 7fd71a7fc640 1 -- 172.21.15.88:0/1886980205 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"}]=0 v58) v1 ==== 109+0+11 (secure 0 0 0) 0x7fd71c055040 con 0x7fd7241f5130 2023-10-16T20:07:52.503 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:07:52.504 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.500+0000 7fd72bcea640 1 -- 172.21.15.88:0/1886980205 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd7080664a0 msgr2=0x7fd708068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:52.504 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.500+0000 7fd72bcea640 1 --2- 172.21.15.88:0/1886980205 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd7080664a0 0x7fd708068960 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd72000a000 tx=0x7fd720001de0 comp rx=0 tx=0).stop 2023-10-16T20:07:52.504 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.500+0000 7fd72bcea640 1 -- 172.21.15.88:0/1886980205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7241f5130 msgr2=0x7fd72415aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:52.505 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.500+0000 7fd72bcea640 1 --2- 172.21.15.88:0/1886980205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7241f5130 0x7fd72415aa00 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fd7240b4ff0 tx=0x7fd71c0115a0 comp rx=0 tx=0).stop 2023-10-16T20:07:52.505 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.500+0000 7fd72bcea640 1 -- 172.21.15.88:0/1886980205 shutdown_connections 2023-10-16T20:07:52.505 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.500+0000 7fd72bcea640 1 --2- 172.21.15.88:0/1886980205 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd7080664a0 0x7fd708068960 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:52.505 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.500+0000 7fd72bcea640 1 --2- 172.21.15.88:0/1886980205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7241f5130 0x7fd72415aa00 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:52.505 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.500+0000 7fd72bcea640 1 --2- 172.21.15.88:0/1886980205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7240b3ed0 0x7fd7241f4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:52.506 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.500+0000 7fd72bcea640 1 --2- 172.21.15.88:0/1886980205 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd710005b90 0x7fd7240b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:52.506 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.500+0000 7fd72bcea640 1 -- 172.21.15.88:0/1886980205 >> 172.21.15.88:0/1886980205 conn(0x7fd7240b0970 msgr2=0x7fd7240af0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:52.506 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.500+0000 7fd72bcea640 1 -- 172.21.15.88:0/1886980205 shutdown_connections 2023-10-16T20:07:52.506 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.500+0000 7fd72bcea640 1 -- 172.21.15.88:0/1886980205 wait complete. 2023-10-16T20:07:52.523 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_data pg_num 2023-10-16T20:07:52.665 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.660+0000 7fcf632c9640 1 Processor -- start 2023-10-16T20:07:52.665 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.660+0000 7fcf632c9640 1 -- start start 2023-10-16T20:07:52.667 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.660+0000 7fcf632c9640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fcf5c154f30 0x7fcf5c0b0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:52.667 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.660+0000 7fcf632c9640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf5c15bd20 con 0x7fcf5c0b0a40 2023-10-16T20:07:52.667 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.660+0000 7fcf632c9640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf5c0702f0 con 0x7fcf5c15b700 2023-10-16T20:07:52.667 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.660+0000 7fcf632c9640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf5c0b0ef0 con 0x7fcf5c154f30 2023-10-16T20:07:52.667 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.660+0000 7fcf61065640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fcf5c154f30 0x7fcf5c0b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:52.668 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.660+0000 7fcf60864640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fcf5c0b0a40 0x7fcf5c15aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:56368/0 (socket says 172.21.15.88:56368) 2023-10-16T20:07:52.668 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.660+0000 7fcf61065640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fcf5c154f30 0x7fcf5c0b0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57770/0 (socket says 172.21.15.88:57770) 2023-10-16T20:07:52.668 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.660+0000 7fcf60864640 1 -- 172.21.15.88:0/3100503097 learned_addr learned my addr 172.21.15.88:0/3100503097 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:52.669 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.660+0000 7fcf61065640 1 -- 172.21.15.88:0/3100503097 >> v1:172.21.15.94:6789/0 conn(0x7fcf5c15b700 legacy=0x7fcf5c06c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:52.669 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.660+0000 7fcf61065640 1 -- 172.21.15.88:0/3100503097 >> v1:172.21.15.88:6789/0 conn(0x7fcf5c0b0a40 legacy=0x7fcf5c15aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:52.669 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.660+0000 7fcf53fff640 1 -- 172.21.15.88:0/3100503097 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2941665863 0 0) 0x7fcf5c15bd20 con 0x7fcf5c0b0a40 2023-10-16T20:07:52.670 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.660+0000 7fcf61065640 1 -- 172.21.15.88:0/3100503097 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf5c0705d0 con 0x7fcf5c154f30 2023-10-16T20:07:52.670 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.664+0000 7fcf61065640 1 --2- 172.21.15.88:0/3100503097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf5c154f30 0x7fcf5c0b0500 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fcf580097c0 tx=0x7fcf5801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188c3dd15c2c6de4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:52.670 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.664+0000 7fcf53fff640 1 -- 172.21.15.88:0/3100503097 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf580ab600 con 0x7fcf5c154f30 2023-10-16T20:07:52.671 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.664+0000 7fcf53fff640 1 -- 172.21.15.88:0/3100503097 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf580a7050 con 0x7fcf5c154f30 2023-10-16T20:07:52.671 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.664+0000 7fcf53fff640 1 -- 172.21.15.88:0/3100503097 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf580ab980 con 0x7fcf5c154f30 2023-10-16T20:07:52.672 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.664+0000 7fcf632c9640 1 -- 172.21.15.88:0/3100503097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf5c154f30 msgr2=0x7fcf5c0b0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:52.672 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.664+0000 7fcf632c9640 1 --2- 172.21.15.88:0/3100503097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf5c154f30 0x7fcf5c0b0500 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fcf580097c0 tx=0x7fcf5801c480 comp rx=0 tx=0).stop 2023-10-16T20:07:52.672 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.664+0000 7fcf632c9640 1 -- 172.21.15.88:0/3100503097 shutdown_connections 2023-10-16T20:07:52.672 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.664+0000 7fcf632c9640 1 --2- 172.21.15.88:0/3100503097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf5c154f30 0x7fcf5c0b0500 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:52.673 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.664+0000 7fcf632c9640 1 -- 172.21.15.88:0/3100503097 >> 172.21.15.88:0/3100503097 conn(0x7fcf5c0cb080 msgr2=0x7fcf5c0cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:52.673 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.664+0000 7fcf632c9640 1 -- 172.21.15.88:0/3100503097 shutdown_connections 2023-10-16T20:07:52.673 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.664+0000 7fcf632c9640 1 -- 172.21.15.88:0/3100503097 wait complete. 2023-10-16T20:07:52.675 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.668+0000 7fcf632c9640 1 Processor -- start 2023-10-16T20:07:52.675 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.668+0000 7fcf632c9640 1 -- start start 2023-10-16T20:07:52.676 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.668+0000 7fcf632c9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf5c0b0a40 0x7fcf5c14d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:52.676 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.668+0000 7fcf632c9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf5c154f30 0x7fcf5c14cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:52.676 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.668+0000 7fcf632c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf5c15b700 0x7fcf5c0ad150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:52.676 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.668+0000 7fcf632c9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c15d350 con 0x7fcf5c154f30 2023-10-16T20:07:52.676 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.668+0000 7fcf632c9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c15d770 con 0x7fcf5c15b700 2023-10-16T20:07:52.677 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.668+0000 7fcf632c9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c0cd930 con 0x7fcf5c0b0a40 2023-10-16T20:07:52.677 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.668+0000 7fcf60864640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf5c154f30 0x7fcf5c14cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:52.677 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.668+0000 7fcf60864640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf5c154f30 0x7fcf5c14cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57780/0 (socket says 172.21.15.88:57780) 2023-10-16T20:07:52.677 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.668+0000 7fcf60864640 1 -- 172.21.15.88:0/566344733 learned_addr learned my addr 172.21.15.88:0/566344733 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:52.677 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.668+0000 7fcf61866640 1 --2- 172.21.15.88:0/566344733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf5c15b700 0x7fcf5c0ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:52.677 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.668+0000 7fcf61065640 1 --2- 172.21.15.88:0/566344733 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf5c0b0a40 0x7fcf5c14d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:52.678 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.668+0000 7fcf60864640 1 -- 172.21.15.88:0/566344733 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf5c0b0a40 msgr2=0x7fcf5c14d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:52.678 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.668+0000 7fcf60864640 1 --2- 172.21.15.88:0/566344733 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf5c0b0a40 0x7fcf5c14d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:52.678 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.668+0000 7fcf60864640 1 -- 172.21.15.88:0/566344733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf5c15b700 msgr2=0x7fcf5c0ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:52.678 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.668+0000 7fcf60864640 1 --2- 172.21.15.88:0/566344733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf5c15b700 0x7fcf5c0ad150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:52.679 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.668+0000 7fcf60864640 1 -- 172.21.15.88:0/566344733 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf58009420 con 0x7fcf5c154f30 2023-10-16T20:07:52.679 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.668+0000 7fcf60864640 1 --2- 172.21.15.88:0/566344733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf5c154f30 0x7fcf5c14cdb0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00c4d0 tx=0x7fcf4c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:52.679 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.668+0000 7fcf51ffb640 1 -- 172.21.15.88:0/566344733 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c0074e0 con 0x7fcf5c154f30 2023-10-16T20:07:52.679 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.668+0000 7fcf51ffb640 1 -- 172.21.15.88:0/566344733 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf4c007640 con 0x7fcf5c154f30 2023-10-16T20:07:52.679 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.668+0000 7fcf632c9640 1 -- 172.21.15.88:0/566344733 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf5c14e320 con 0x7fcf5c154f30 2023-10-16T20:07:52.680 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.668+0000 7fcf51ffb640 1 -- 172.21.15.88:0/566344733 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c004a30 con 0x7fcf5c154f30 2023-10-16T20:07:52.680 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.668+0000 7fcf632c9640 1 -- 172.21.15.88:0/566344733 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf5c15d000 con 0x7fcf5c154f30 2023-10-16T20:07:52.681 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.672+0000 7fcf632c9640 1 -- 172.21.15.88:0/566344733 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf5c15d5c0 con 0x7fcf5c154f30 2023-10-16T20:07:52.681 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.672+0000 7fcf51ffb640 1 -- 172.21.15.88:0/566344733 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcf4c004b90 con 0x7fcf5c154f30 2023-10-16T20:07:52.681 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.672+0000 7fcf51ffb640 1 --2- 172.21.15.88:0/566344733 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcf400664a0 0x7fcf40068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:52.681 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.672+0000 7fcf51ffb640 1 -- 172.21.15.88:0/566344733 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fcf4c083540 con 0x7fcf5c154f30 2023-10-16T20:07:52.681 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.672+0000 7fcf61065640 1 --2- 172.21.15.88:0/566344733 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcf400664a0 0x7fcf40068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:52.682 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.672+0000 7fcf61065640 1 --2- 172.21.15.88:0/566344733 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcf400664a0 0x7fcf40068960 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcf580ad000 tx=0x7fcf580a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:52.682 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.676+0000 7fcf51ffb640 1 -- 172.21.15.88:0/566344733 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcf4c0507c0 con 0x7fcf5c154f30 2023-10-16T20:07:52.831 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.824+0000 7fcf632c9640 1 -- 172.21.15.88:0/566344733 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"} v 0) v1 -- 0x7fcf5c04e9c0 con 0x7fcf5c154f30 2023-10-16T20:07:52.831 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.824+0000 7fcf51ffb640 1 -- 172.21.15.88:0/566344733 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"}]=0 v58) v1 ==== 105+0+11 (secure 0 0 0) 0x7fcf4c053de0 con 0x7fcf5c154f30 2023-10-16T20:07:52.831 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2023-10-16T20:07:52.833 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.828+0000 7fcf632c9640 1 -- 172.21.15.88:0/566344733 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcf400664a0 msgr2=0x7fcf40068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:52.833 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.828+0000 7fcf632c9640 1 --2- 172.21.15.88:0/566344733 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcf400664a0 0x7fcf40068960 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcf580ad000 tx=0x7fcf580a6000 comp rx=0 tx=0).stop 2023-10-16T20:07:52.834 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.828+0000 7fcf632c9640 1 -- 172.21.15.88:0/566344733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf5c154f30 msgr2=0x7fcf5c14cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:52.834 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.828+0000 7fcf632c9640 1 --2- 172.21.15.88:0/566344733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf5c154f30 0x7fcf5c14cdb0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00c4d0 tx=0x7fcf4c00c990 comp rx=0 tx=0).stop 2023-10-16T20:07:52.834 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.828+0000 7fcf632c9640 1 -- 172.21.15.88:0/566344733 shutdown_connections 2023-10-16T20:07:52.834 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.828+0000 7fcf632c9640 1 --2- 172.21.15.88:0/566344733 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcf400664a0 0x7fcf40068960 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:52.835 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.828+0000 7fcf632c9640 1 --2- 172.21.15.88:0/566344733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf5c15b700 0x7fcf5c0ad150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:52.835 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.828+0000 7fcf632c9640 1 --2- 172.21.15.88:0/566344733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf5c154f30 0x7fcf5c14cdb0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:52.835 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.828+0000 7fcf632c9640 1 --2- 172.21.15.88:0/566344733 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf5c0b0a40 0x7fcf5c14d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:52.835 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.828+0000 7fcf632c9640 1 -- 172.21.15.88:0/566344733 >> 172.21.15.88:0/566344733 conn(0x7fcf5c0cb080 msgr2=0x7fcf5c1520e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:52.835 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.828+0000 7fcf632c9640 1 -- 172.21.15.88:0/566344733 shutdown_connections 2023-10-16T20:07:52.836 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.828+0000 7fcf632c9640 1 -- 172.21.15.88:0/566344733 wait complete. 2023-10-16T20:07:52.852 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T20:07:52.991 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.984+0000 7ffadfa9e640 1 Processor -- start 2023-10-16T20:07:52.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.984+0000 7ffadfa9e640 1 -- start start 2023-10-16T20:07:52.992 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.988+0000 7ffadfa9e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffad815e5a0 0x7ffad80b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:52.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.988+0000 7ffadfa9e640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffad80b2720 con 0x7ffad81584a0 2023-10-16T20:07:52.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.988+0000 7ffadfa9e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffad815b250 con 0x7ffad80cc5d0 2023-10-16T20:07:52.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.988+0000 7ffadfa9e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad815f4d0 con 0x7ffad815e5a0 2023-10-16T20:07:52.993 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.988+0000 7ffadd83a640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7ffad81584a0 0x7ffad815b6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:56374/0 (socket says 172.21.15.88:56374) 2023-10-16T20:07:52.994 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.988+0000 7ffadd83a640 1 -- 172.21.15.88:0/3827185591 learned_addr learned my addr 172.21.15.88:0/3827185591 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:52.995 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.988+0000 7ffadd039640 1 --2- 172.21.15.88:0/3827185591 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffad815e5a0 0x7ffad80b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:52.995 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.988+0000 7ffadc838640 1 -- 172.21.15.88:0/3827185591 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 821125870 0 0) 0x7ffad4001120 con 0x7ffad81584a0 2023-10-16T20:07:52.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.988+0000 7ffadc838640 1 -- 172.21.15.88:0/3827185591 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffac4003530 con 0x7ffad81584a0 2023-10-16T20:07:52.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.988+0000 7ffadd039640 1 -- 172.21.15.88:0/3827185591 >> v1:172.21.15.94:6789/0 conn(0x7ffad80cc5d0 legacy=0x7ffad80b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:52.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.988+0000 7ffadc838640 1 -- 172.21.15.88:0/3827185591 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 778057628 0 0) 0x7ffad815b250 con 0x7ffad80cc5d0 2023-10-16T20:07:52.996 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.988+0000 7ffadd039640 1 -- 172.21.15.88:0/3827185591 >> v1:172.21.15.88:6789/0 conn(0x7ffad81584a0 legacy=0x7ffad815b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:52.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.988+0000 7ffadd039640 1 -- 172.21.15.88:0/3827185591 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad815e9e0 con 0x7ffad815e5a0 2023-10-16T20:07:52.998 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.992+0000 7ffadd039640 1 --2- 172.21.15.88:0/3827185591 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffad815e5a0 0x7ffad80b21e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ffac80097f0 tx=0x7ffac801c270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a330b71d99cbbb9c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:52.998 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.992+0000 7ffadc838640 1 -- 172.21.15.88:0/3827185591 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad4001810 con 0x7ffad815e5a0 2023-10-16T20:07:52.999 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.992+0000 7ffadc838640 1 -- 172.21.15.88:0/3827185591 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac80a7050 con 0x7ffad815e5a0 2023-10-16T20:07:52.999 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.992+0000 7ffadc838640 1 -- 172.21.15.88:0/3827185591 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac80ab620 con 0x7ffad815e5a0 2023-10-16T20:07:52.999 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.992+0000 7ffadfa9e640 1 -- 172.21.15.88:0/3827185591 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffad815e5a0 msgr2=0x7ffad80b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:52.999 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.992+0000 7ffadfa9e640 1 --2- 172.21.15.88:0/3827185591 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffad815e5a0 0x7ffad80b21e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ffac80097f0 tx=0x7ffac801c270 comp rx=0 tx=0).stop 2023-10-16T20:07:52.999 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.992+0000 7ffadfa9e640 1 -- 172.21.15.88:0/3827185591 shutdown_connections 2023-10-16T20:07:53.000 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.992+0000 7ffadfa9e640 1 --2- 172.21.15.88:0/3827185591 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffad815e5a0 0x7ffad80b21e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:53.000 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.992+0000 7ffadfa9e640 1 -- 172.21.15.88:0/3827185591 >> 172.21.15.88:0/3827185591 conn(0x7ffad80ce5e0 msgr2=0x7ffad80ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:53.000 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.992+0000 7ffadfa9e640 1 -- 172.21.15.88:0/3827185591 shutdown_connections 2023-10-16T20:07:53.000 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.992+0000 7ffadfa9e640 1 -- 172.21.15.88:0/3827185591 wait complete. 2023-10-16T20:07:53.002 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.992+0000 7ffadfa9e640 1 Processor -- start 2023-10-16T20:07:53.002 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.992+0000 7ffadfa9e640 1 -- start start 2023-10-16T20:07:53.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.996+0000 7ffadfa9e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffad80cc5d0 0x7ffad81540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:53.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.996+0000 7ffadfa9e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffad81584a0 0x7ffad813e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:53.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.996+0000 7ffadfa9e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffad815e5a0 0x7ffad8071df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:53.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.996+0000 7ffadfa9e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad813e250 con 0x7ffad815e5a0 2023-10-16T20:07:53.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.996+0000 7ffadfa9e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad813d640 con 0x7ffad81584a0 2023-10-16T20:07:53.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.996+0000 7ffadfa9e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad813d920 con 0x7ffad80cc5d0 2023-10-16T20:07:53.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.996+0000 7ffade03b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffad815e5a0 0x7ffad8071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:53.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.996+0000 7ffadd83a640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffad80cc5d0 0x7ffad81540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:53.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.996+0000 7ffade03b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffad815e5a0 0x7ffad8071df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57788/0 (socket says 172.21.15.88:57788) 2023-10-16T20:07:53.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.996+0000 7ffadd039640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffad81584a0 0x7ffad813e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:53.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.996+0000 7ffade03b640 1 -- 172.21.15.88:0/254674975 learned_addr learned my addr 172.21.15.88:0/254674975 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:53.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.996+0000 7ffade03b640 1 -- 172.21.15.88:0/254674975 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffad80cc5d0 msgr2=0x7ffad81540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:53.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.996+0000 7ffade03b640 1 --2- 172.21.15.88:0/254674975 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffad80cc5d0 0x7ffad81540c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:53.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.996+0000 7ffade03b640 1 -- 172.21.15.88:0/254674975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffad81584a0 msgr2=0x7ffad813e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:53.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.996+0000 7ffade03b640 1 --2- 172.21.15.88:0/254674975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffad81584a0 0x7ffad813e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:53.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.996+0000 7ffade03b640 1 -- 172.21.15.88:0/254674975 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac8009420 con 0x7ffad815e5a0 2023-10-16T20:07:53.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.996+0000 7ffade03b640 1 --2- 172.21.15.88:0/254674975 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffad815e5a0 0x7ffad8071df0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ffad0001570 tx=0x7ffad000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:53.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.996+0000 7fface7fc640 1 -- 172.21.15.88:0/254674975 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad0003db0 con 0x7ffad815e5a0 2023-10-16T20:07:53.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.996+0000 7fface7fc640 1 -- 172.21.15.88:0/254674975 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad0005070 con 0x7ffad815e5a0 2023-10-16T20:07:53.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.996+0000 7ffadfa9e640 1 -- 172.21.15.88:0/254674975 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad813dc00 con 0x7ffad815e5a0 2023-10-16T20:07:53.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.996+0000 7fface7fc640 1 -- 172.21.15.88:0/254674975 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad0005420 con 0x7ffad815e5a0 2023-10-16T20:07:53.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.996+0000 7ffadfa9e640 1 -- 172.21.15.88:0/254674975 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad8072330 con 0x7ffad815e5a0 2023-10-16T20:07:53.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:52.996+0000 7ffadfa9e640 1 -- 172.21.15.88:0/254674975 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad806d180 con 0x7ffad815e5a0 2023-10-16T20:07:53.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.000+0000 7fface7fc640 1 -- 172.21.15.88:0/254674975 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ffad0005580 con 0x7ffad815e5a0 2023-10-16T20:07:53.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.000+0000 7fface7fc640 1 --2- 172.21.15.88:0/254674975 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ffabc0664a0 0x7ffabc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:53.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.000+0000 7fface7fc640 1 -- 172.21.15.88:0/254674975 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7ffad00838a0 con 0x7ffad815e5a0 2023-10-16T20:07:53.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.000+0000 7ffadd83a640 1 --2- 172.21.15.88:0/254674975 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ffabc0664a0 0x7ffabc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:53.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.000+0000 7ffadd83a640 1 --2- 172.21.15.88:0/254674975 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ffabc0664a0 0x7ffabc068960 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ffad4000f70 tx=0x7ffad40048b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:53.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.004+0000 7fface7fc640 1 -- 172.21.15.88:0/254674975 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ffad0050bd0 con 0x7ffad815e5a0 2023-10-16T20:07:53.167 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.160+0000 7ffadfa9e640 1 -- 172.21.15.88:0/254674975 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ffad804e9c0 con 0x7ffad815e5a0 2023-10-16T20:07:53.168 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.164+0000 7fface7fc640 1 -- 172.21.15.88:0/254674975 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v58) v1 ==== 80+0+0 (secure 0 0 0) 0x7ffad0017070 con 0x7ffad815e5a0 2023-10-16T20:07:53.168 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2023-10-16T20:07:53.170 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.164+0000 7ffadfa9e640 1 -- 172.21.15.88:0/254674975 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ffabc0664a0 msgr2=0x7ffabc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:53.170 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.164+0000 7ffadfa9e640 1 --2- 172.21.15.88:0/254674975 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ffabc0664a0 0x7ffabc068960 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ffad4000f70 tx=0x7ffad40048b0 comp rx=0 tx=0).stop 2023-10-16T20:07:53.170 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.164+0000 7ffadfa9e640 1 -- 172.21.15.88:0/254674975 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffad815e5a0 msgr2=0x7ffad8071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:53.171 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.164+0000 7ffadfa9e640 1 --2- 172.21.15.88:0/254674975 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffad815e5a0 0x7ffad8071df0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ffad0001570 tx=0x7ffad000cd20 comp rx=0 tx=0).stop 2023-10-16T20:07:53.171 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.164+0000 7ffadfa9e640 1 -- 172.21.15.88:0/254674975 shutdown_connections 2023-10-16T20:07:53.171 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.164+0000 7ffadfa9e640 1 --2- 172.21.15.88:0/254674975 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ffabc0664a0 0x7ffabc068960 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:53.171 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.164+0000 7ffadfa9e640 1 --2- 172.21.15.88:0/254674975 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffad815e5a0 0x7ffad8071df0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:53.171 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.164+0000 7ffadfa9e640 1 --2- 172.21.15.88:0/254674975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffad81584a0 0x7ffad813e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:53.172 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.164+0000 7ffadfa9e640 1 --2- 172.21.15.88:0/254674975 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffad80cc5d0 0x7ffad81540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:53.172 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.164+0000 7ffadfa9e640 1 -- 172.21.15.88:0/254674975 >> 172.21.15.88:0/254674975 conn(0x7ffad80ce5e0 msgr2=0x7ffad80b0500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:53.172 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.164+0000 7ffadfa9e640 1 -- 172.21.15.88:0/254674975 shutdown_connections 2023-10-16T20:07:53.172 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.164+0000 7ffadfa9e640 1 -- 172.21.15.88:0/254674975 wait complete. 2023-10-16T20:07:53.189 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi088.front.sepia.ceph.com... 2023-10-16T20:07:53.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:53.190 DEBUG:teuthology.orchestra.run.smithi088:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-16T20:07:53.209 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[52880]: fuse finished with error 0 and tester_r 0 2023-10-16T20:07:53.211 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2023-10-16T20:07:53.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:53.212 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T20:07:53.348 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:07:53.348 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:07:53.359 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354624k,nr_inodes=4088656,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5605 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T20:07:53.360 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2023-10-16T20:07:53.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:53.360 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-16T20:07:53.411 INFO:teuthology.orchestra.run.smithi094.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-16T20:07:53.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T20:07:53.412 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:07:53.413 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:07:53.422 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354668k,nr_inodes=4088667,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9372 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T20:07:53.422 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2023-10-16T20:07:53.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:53.423 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-16T20:07:53.471 INFO:teuthology.orchestra.run.smithi088.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-16T20:07:53.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T20:07:53.472 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:07:53.473 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:07:53.483 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354624k,nr_inodes=4088656,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5605 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T20:07:53.483 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2023-10-16T20:07:53.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:07:53.484 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-16T20:07:53.535 INFO:teuthology.orchestra.run.smithi094.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-16T20:07:53.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T20:07:53.537 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T20:07:53.673 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.668+0000 7f2b1bd3e640 1 Processor -- start 2023-10-16T20:07:53.674 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.668+0000 7f2b1bd3e640 1 -- start start 2023-10-16T20:07:53.676 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.668+0000 7f2b1bd3e640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f2b141599f0 0x7f2b140b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:53.676 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.668+0000 7f2b1bd3e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2b14069e90 0x7f2b1406a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:53.676 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.668+0000 7f2b1bd3e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2b140b05a0 0x7f2b140b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:53.676 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.668+0000 7f2b1bd3e640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b1406a830 con 0x7f2b141599f0 2023-10-16T20:07:53.676 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.668+0000 7f2b1bd3e640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b140b1260 con 0x7f2b14069e90 2023-10-16T20:07:53.677 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.668+0000 7f2b1bd3e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b140ae960 con 0x7f2b140b05a0 2023-10-16T20:07:53.677 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.668+0000 7f2b19ada640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f2b141599f0 0x7f2b140b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:53.677 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.668+0000 7f2b19ada640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f2b141599f0 0x7f2b140b2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57794/0 (socket says 172.21.15.88:57794) 2023-10-16T20:07:53.677 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.668+0000 7f2b19ada640 1 -- 172.21.15.88:0/2536783842 learned_addr learned my addr 172.21.15.88:0/2536783842 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:53.677 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.668+0000 7f2b192d9640 1 --2- 172.21.15.88:0/2536783842 >> v2:172.21.15.94:3300/0 conn(0x7f2b14069e90 0x7f2b1406a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:53.678 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.668+0000 7f2b1a2db640 1 --2- 172.21.15.88:0/2536783842 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2b140b05a0 0x7f2b140b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:53.678 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.672+0000 7f2b19ada640 1 -- 172.21.15.88:0/2536783842 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2b140b05a0 msgr2=0x7f2b140b0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:53.678 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.672+0000 7f2b19ada640 1 --2- 172.21.15.88:0/2536783842 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2b140b05a0 0x7f2b140b0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:53.678 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.672+0000 7f2b19ada640 1 -- 172.21.15.88:0/2536783842 >> v2:172.21.15.94:3300/0 conn(0x7f2b14069e90 msgr2=0x7f2b1406a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:53.678 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.672+0000 7f2b19ada640 1 --2- 172.21.15.88:0/2536783842 >> v2:172.21.15.94:3300/0 conn(0x7f2b14069e90 0x7f2b1406a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:53.679 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.672+0000 7f2b19ada640 1 -- 172.21.15.88:0/2536783842 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b140aec40 con 0x7f2b141599f0 2023-10-16T20:07:53.680 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.672+0000 7f2b19ada640 1 --2- 172.21.15.88:0/2536783842 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b141599f0 0x7f2b140b2a30 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f2b140ae500 tx=0x7f2b1001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=226ccb529f7ea3f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:53.680 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.672+0000 7f2b18ad8640 1 -- 172.21.15.88:0/2536783842 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b100ad680 con 0x7f2b141599f0 2023-10-16T20:07:53.680 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.672+0000 7f2b18ad8640 1 -- 172.21.15.88:0/2536783842 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b100ad7e0 con 0x7f2b141599f0 2023-10-16T20:07:53.680 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.672+0000 7f2b18ad8640 1 -- 172.21.15.88:0/2536783842 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b100adb70 con 0x7f2b141599f0 2023-10-16T20:07:53.681 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.672+0000 7f2b1bd3e640 1 -- 172.21.15.88:0/2536783842 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b141599f0 msgr2=0x7f2b140b2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:53.681 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.672+0000 7f2b1bd3e640 1 --2- 172.21.15.88:0/2536783842 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b141599f0 0x7f2b140b2a30 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f2b140ae500 tx=0x7f2b1001c4d0 comp rx=0 tx=0).stop 2023-10-16T20:07:53.681 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.672+0000 7f2b1bd3e640 1 -- 172.21.15.88:0/2536783842 shutdown_connections 2023-10-16T20:07:53.681 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.672+0000 7f2b1bd3e640 1 --2- 172.21.15.88:0/2536783842 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2b140b05a0 0x7f2b140b0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:53.681 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.672+0000 7f2b1bd3e640 1 --2- 172.21.15.88:0/2536783842 >> v2:172.21.15.94:3300/0 conn(0x7f2b14069e90 0x7f2b1406a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:53.682 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.672+0000 7f2b1bd3e640 1 --2- 172.21.15.88:0/2536783842 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b141599f0 0x7f2b140b2a30 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:53.682 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.672+0000 7f2b1bd3e640 1 -- 172.21.15.88:0/2536783842 >> 172.21.15.88:0/2536783842 conn(0x7f2b140ce5e0 msgr2=0x7f2b140ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:53.682 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.676+0000 7f2b1bd3e640 1 -- 172.21.15.88:0/2536783842 shutdown_connections 2023-10-16T20:07:53.682 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.676+0000 7f2b1bd3e640 1 -- 172.21.15.88:0/2536783842 wait complete. 2023-10-16T20:07:53.684 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.676+0000 7f2b1bd3e640 1 Processor -- start 2023-10-16T20:07:53.684 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.676+0000 7f2b1bd3e640 1 -- start start 2023-10-16T20:07:53.685 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.676+0000 7f2b1bd3e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b14069e90 0x7f2b141f3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:53.685 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.676+0000 7f2b1bd3e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b140b05a0 0x7f2b14140740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:53.685 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.676+0000 7f2b1bd3e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2b141599f0 0x7f2b14159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:53.685 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.676+0000 7f2b1bd3e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b141426f0 con 0x7f2b140b05a0 2023-10-16T20:07:53.685 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.676+0000 7f2b1bd3e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b141421d0 con 0x7f2b14069e90 2023-10-16T20:07:53.686 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.676+0000 7f2b1bd3e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b14149480 con 0x7f2b141599f0 2023-10-16T20:07:53.686 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.676+0000 7f2b192d9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b140b05a0 0x7f2b14140740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:53.686 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.676+0000 7f2b192d9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b140b05a0 0x7f2b14140740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57804/0 (socket says 172.21.15.88:57804) 2023-10-16T20:07:53.686 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.676+0000 7f2b192d9640 1 -- 172.21.15.88:0/1492693883 learned_addr learned my addr 172.21.15.88:0/1492693883 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:53.686 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.676+0000 7f2b19ada640 1 --2- 172.21.15.88:0/1492693883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b14069e90 0x7f2b141f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:53.687 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.676+0000 7f2b1a2db640 1 --2- 172.21.15.88:0/1492693883 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2b141599f0 0x7f2b14159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:53.687 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.676+0000 7f2b192d9640 1 -- 172.21.15.88:0/1492693883 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2b141599f0 msgr2=0x7f2b14159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:53.687 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.676+0000 7f2b192d9640 1 --2- 172.21.15.88:0/1492693883 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2b141599f0 0x7f2b14159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:53.687 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.676+0000 7f2b192d9640 1 -- 172.21.15.88:0/1492693883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b14069e90 msgr2=0x7f2b141f3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:53.687 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.676+0000 7f2b192d9640 1 --2- 172.21.15.88:0/1492693883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b14069e90 0x7f2b141f3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:53.688 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.676+0000 7f2b192d9640 1 -- 172.21.15.88:0/1492693883 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b100094d0 con 0x7f2b140b05a0 2023-10-16T20:07:53.688 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.676+0000 7f2b192d9640 1 --2- 172.21.15.88:0/1492693883 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b140b05a0 0x7f2b14140740 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400e870 tx=0x7f2b0400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:53.688 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.676+0000 7f2b0a7fc640 1 -- 172.21.15.88:0/1492693883 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0400ee60 con 0x7f2b140b05a0 2023-10-16T20:07:53.688 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.676+0000 7f2b0a7fc640 1 -- 172.21.15.88:0/1492693883 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b0400c940 con 0x7f2b140b05a0 2023-10-16T20:07:53.688 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.680+0000 7f2b0a7fc640 1 -- 172.21.15.88:0/1492693883 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0400ccf0 con 0x7f2b140b05a0 2023-10-16T20:07:53.689 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.680+0000 7f2b1bd3e640 1 -- 172.21.15.88:0/1492693883 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b14149760 con 0x7f2b140b05a0 2023-10-16T20:07:53.689 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.680+0000 7f2b1bd3e640 1 -- 172.21.15.88:0/1492693883 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b14149d20 con 0x7f2b140b05a0 2023-10-16T20:07:53.689 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.680+0000 7f2b1bd3e640 1 -- 172.21.15.88:0/1492693883 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b1404e9c0 con 0x7f2b140b05a0 2023-10-16T20:07:53.690 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.680+0000 7f2b0a7fc640 1 -- 172.21.15.88:0/1492693883 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2b04013470 con 0x7f2b140b05a0 2023-10-16T20:07:53.690 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.680+0000 7f2b0a7fc640 1 --2- 172.21.15.88:0/1492693883 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2af80664a0 0x7f2af8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:53.690 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.680+0000 7f2b0a7fc640 1 -- 172.21.15.88:0/1492693883 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f2b04083fc0 con 0x7f2b140b05a0 2023-10-16T20:07:53.691 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.680+0000 7f2b19ada640 1 --2- 172.21.15.88:0/1492693883 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2af80664a0 0x7f2af8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:53.691 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.680+0000 7f2b19ada640 1 --2- 172.21.15.88:0/1492693883 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2af80664a0 0x7f2af8068960 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2b100af000 tx=0x7f2b100a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:53.691 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.684+0000 7f2b0a7fc640 1 -- 172.21.15.88:0/1492693883 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2b04051380 con 0x7f2b140b05a0 2023-10-16T20:07:53.869 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.864+0000 7f2b1bd3e640 1 -- 172.21.15.88:0/1492693883 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b1406aa90 con 0x7f2b140b05a0 2023-10-16T20:07:53.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.864+0000 7f2b0a7fc640 1 -- 172.21.15.88:0/1492693883 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f2b040549a0 con 0x7f2b140b05a0 2023-10-16T20:07:53.871 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:07:53.871 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5531,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/2485638574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":2485638574},{"type":"v1","addr":"172.21.15.88:6837","nonce":2485638574}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5534,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3821390040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3821390040},{"type":"v1","addr":"172.21.15.94:6834","nonce":3821390040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5968,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2544673478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":2544673478},{"type":"v1","addr":"172.21.15.94:6835","nonce":2544673478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T20:07:25.650705+0000","modified":"2023-10-16T20:07:28.651868+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6210":{"gid":6210,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6836/1018125900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":1018125900},{"type":"v1","addr":"172.21.15.88:6836","nonce":1018125900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"testcephfs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-16T20:07:53.871 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 28 2023-10-16T20:07:53.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.868+0000 7f2b1bd3e640 1 -- 172.21.15.88:0/1492693883 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2af80664a0 msgr2=0x7f2af8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:53.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.868+0000 7f2b1bd3e640 1 --2- 172.21.15.88:0/1492693883 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2af80664a0 0x7f2af8068960 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2b100af000 tx=0x7f2b100a7000 comp rx=0 tx=0).stop 2023-10-16T20:07:53.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.868+0000 7f2b1bd3e640 1 -- 172.21.15.88:0/1492693883 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b140b05a0 msgr2=0x7f2b14140740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:53.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.868+0000 7f2b1bd3e640 1 --2- 172.21.15.88:0/1492693883 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b140b05a0 0x7f2b14140740 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400e870 tx=0x7f2b0400ed30 comp rx=0 tx=0).stop 2023-10-16T20:07:53.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.868+0000 7f2b1bd3e640 1 -- 172.21.15.88:0/1492693883 shutdown_connections 2023-10-16T20:07:53.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.868+0000 7f2b1bd3e640 1 --2- 172.21.15.88:0/1492693883 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f2af80664a0 0x7f2af8068960 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:53.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.868+0000 7f2b1bd3e640 1 --2- 172.21.15.88:0/1492693883 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2b141599f0 0x7f2b14159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:53.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.868+0000 7f2b1bd3e640 1 --2- 172.21.15.88:0/1492693883 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b140b05a0 0x7f2b14140740 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:53.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.868+0000 7f2b1bd3e640 1 --2- 172.21.15.88:0/1492693883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b14069e90 0x7f2b141f3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:53.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.868+0000 7f2b1bd3e640 1 -- 172.21.15.88:0/1492693883 >> 172.21.15.88:0/1492693883 conn(0x7f2b140ce5e0 msgr2=0x7f2b14071a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:53.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.868+0000 7f2b1bd3e640 1 -- 172.21.15.88:0/1492693883 shutdown_connections 2023-10-16T20:07:53.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:53.868+0000 7f2b1bd3e640 1 -- 172.21.15.88:0/1492693883 wait complete. 2023-10-16T20:07:53.891 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:07:54.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.024+0000 7f7d30165640 1 Processor -- start 2023-10-16T20:07:54.029 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.024+0000 7f7d30165640 1 -- start start 2023-10-16T20:07:54.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.024+0000 7f7d30165640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d280d22e0 0x7f7d28161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:54.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.024+0000 7f7d30165640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7d2815d9f0 0x7f7d280cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:54.032 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.024+0000 7f7d30165640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d280b87e0 con 0x7f7d280d26c0 2023-10-16T20:07:54.032 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.024+0000 7f7d30165640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d2815df10 con 0x7f7d2815d9f0 2023-10-16T20:07:54.032 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.024+0000 7f7d30165640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2815e1f0 con 0x7f7d280d22e0 2023-10-16T20:07:54.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.024+0000 7f7d2d700640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f7d280d26c0 0x7f7d2815d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:56380/0 (socket says 172.21.15.88:56380) 2023-10-16T20:07:54.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.024+0000 7f7d2d700640 1 -- 172.21.15.88:0/3697454089 learned_addr learned my addr 172.21.15.88:0/3697454089 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:54.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.024+0000 7f7d2df01640 1 --2- 172.21.15.88:0/3697454089 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d280d22e0 0x7f7d28161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:54.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.024+0000 7f7d2e702640 1 --2- 172.21.15.88:0/3697454089 >> v2:172.21.15.94:3300/0 conn(0x7f7d2815d9f0 0x7f7d280cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:54.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.024+0000 7f7d2ceff640 1 -- 172.21.15.88:0/3697454089 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812742759 0 0) 0x7f7d280b87e0 con 0x7f7d280d26c0 2023-10-16T20:07:54.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.024+0000 7f7d2ceff640 1 -- 172.21.15.88:0/3697454089 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d14003530 con 0x7f7d280d26c0 2023-10-16T20:07:54.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.024+0000 7f7d2ceff640 1 -- 172.21.15.88:0/3697454089 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3762548679 0 0) 0x7f7d14003530 con 0x7f7d280d26c0 2023-10-16T20:07:54.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.024+0000 7f7d2df01640 1 -- 172.21.15.88:0/3697454089 >> v2:172.21.15.94:3300/0 conn(0x7f7d2815d9f0 msgr2=0x7f7d280cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:54.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.024+0000 7f7d2df01640 1 --2- 172.21.15.88:0/3697454089 >> v2:172.21.15.94:3300/0 conn(0x7f7d2815d9f0 0x7f7d280cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:54.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.024+0000 7f7d2df01640 1 -- 172.21.15.88:0/3697454089 >> v1:172.21.15.88:6789/0 conn(0x7f7d280d26c0 legacy=0x7f7d2815d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:54.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.024+0000 7f7d2df01640 1 -- 172.21.15.88:0/3697454089 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d28160970 con 0x7f7d280d22e0 2023-10-16T20:07:54.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.024+0000 7f7d2e702640 1 --2- 172.21.15.88:0/3697454089 >> v2:172.21.15.94:3300/0 conn(0x7f7d2815d9f0 0x7f7d280cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:07:54.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.028+0000 7f7d2df01640 1 --2- 172.21.15.88:0/3697454089 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d280d22e0 0x7f7d28161450 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7d240097c0 tx=0x7f7d2401c2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a246b30e9612e156 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:54.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.028+0000 7f7d2ceff640 1 -- 172.21.15.88:0/3697454089 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d180016a0 con 0x7f7d280d22e0 2023-10-16T20:07:54.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.028+0000 7f7d2ceff640 1 -- 172.21.15.88:0/3697454089 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d240a7050 con 0x7f7d280d22e0 2023-10-16T20:07:54.041 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.028+0000 7f7d2ceff640 1 -- 172.21.15.88:0/3697454089 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d240ab620 con 0x7f7d280d22e0 2023-10-16T20:07:54.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.028+0000 7f7d30165640 1 -- 172.21.15.88:0/3697454089 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d280d22e0 msgr2=0x7f7d28161450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:54.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.028+0000 7f7d30165640 1 --2- 172.21.15.88:0/3697454089 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d280d22e0 0x7f7d28161450 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7d240097c0 tx=0x7f7d2401c2e0 comp rx=0 tx=0).stop 2023-10-16T20:07:54.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.028+0000 7f7d30165640 1 -- 172.21.15.88:0/3697454089 shutdown_connections 2023-10-16T20:07:54.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.028+0000 7f7d30165640 1 --2- 172.21.15.88:0/3697454089 >> v2:172.21.15.94:3300/0 conn(0x7f7d2815d9f0 0x7f7d280cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:54.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.028+0000 7f7d30165640 1 --2- 172.21.15.88:0/3697454089 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d280d22e0 0x7f7d28161450 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:54.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.028+0000 7f7d30165640 1 -- 172.21.15.88:0/3697454089 >> 172.21.15.88:0/3697454089 conn(0x7f7d280715c0 msgr2=0x7f7d280719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:54.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.028+0000 7f7d30165640 1 -- 172.21.15.88:0/3697454089 shutdown_connections 2023-10-16T20:07:54.044 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.028+0000 7f7d30165640 1 -- 172.21.15.88:0/3697454089 wait complete. 2023-10-16T20:07:54.044 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.032+0000 7f7d30165640 1 Processor -- start 2023-10-16T20:07:54.044 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.032+0000 7f7d30165640 1 -- start start 2023-10-16T20:07:54.044 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.032+0000 7f7d30165640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d280d22e0 0x7f7d2814ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:54.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.032+0000 7f7d30165640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d280d26c0 0x7f7d281529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:54.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.032+0000 7f7d30165640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d2815d9f0 0x7f7d28150bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:54.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.032+0000 7f7d30165640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d281516b0 con 0x7f7d280d22e0 2023-10-16T20:07:54.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.032+0000 7f7d30165640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2814f260 con 0x7f7d2815d9f0 2023-10-16T20:07:54.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.032+0000 7f7d30165640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d28151950 con 0x7f7d280d26c0 2023-10-16T20:07:54.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.032+0000 7f7d2df01640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d280d22e0 0x7f7d2814ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:54.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.032+0000 7f7d2df01640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d280d22e0 0x7f7d2814ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57812/0 (socket says 172.21.15.88:57812) 2023-10-16T20:07:54.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.032+0000 7f7d2df01640 1 -- 172.21.15.88:0/2842817206 learned_addr learned my addr 172.21.15.88:0/2842817206 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:54.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.032+0000 7f7d2d700640 1 --2- 172.21.15.88:0/2842817206 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d280d26c0 0x7f7d281529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:54.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.032+0000 7f7d2e702640 1 --2- 172.21.15.88:0/2842817206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d2815d9f0 0x7f7d28150bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:54.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.032+0000 7f7d2df01640 1 -- 172.21.15.88:0/2842817206 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d280d26c0 msgr2=0x7f7d281529f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:54.048 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.032+0000 7f7d2df01640 1 --2- 172.21.15.88:0/2842817206 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d280d26c0 0x7f7d281529f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:54.048 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.032+0000 7f7d2df01640 1 -- 172.21.15.88:0/2842817206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d2815d9f0 msgr2=0x7f7d28150bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:54.048 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.032+0000 7f7d2df01640 1 --2- 172.21.15.88:0/2842817206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d2815d9f0 0x7f7d28150bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:54.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.032+0000 7f7d2df01640 1 -- 172.21.15.88:0/2842817206 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d20009a70 con 0x7f7d280d22e0 2023-10-16T20:07:54.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.032+0000 7f7d2e702640 1 --2- 172.21.15.88:0/2842817206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d2815d9f0 0x7f7d28150bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:07:54.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.032+0000 7f7d2df01640 1 --2- 172.21.15.88:0/2842817206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d280d22e0 0x7f7d2814ed20 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f7d2401a990 tx=0x7f7d2401ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:54.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.032+0000 7f7d1e7fc640 1 -- 172.21.15.88:0/2842817206 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d240b7450 con 0x7f7d280d22e0 2023-10-16T20:07:54.050 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.032+0000 7f7d1e7fc640 1 -- 172.21.15.88:0/2842817206 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d240b75d0 con 0x7f7d280d22e0 2023-10-16T20:07:54.050 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.032+0000 7f7d1e7fc640 1 -- 172.21.15.88:0/2842817206 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d240b7980 con 0x7f7d280d22e0 2023-10-16T20:07:54.050 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.032+0000 7f7d30165640 1 -- 172.21.15.88:0/2842817206 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d24009420 con 0x7f7d280d22e0 2023-10-16T20:07:54.050 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.032+0000 7f7d30165640 1 -- 172.21.15.88:0/2842817206 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d28151f10 con 0x7f7d280d22e0 2023-10-16T20:07:54.051 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.036+0000 7f7d30165640 1 -- 172.21.15.88:0/2842817206 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d2804e9c0 con 0x7f7d280d22e0 2023-10-16T20:07:54.051 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.036+0000 7f7d1e7fc640 1 -- 172.21.15.88:0/2842817206 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7d240b7ae0 con 0x7f7d280d22e0 2023-10-16T20:07:54.052 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.036+0000 7f7d1e7fc640 1 --2- 172.21.15.88:0/2842817206 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7d0c0664a0 0x7f7d0c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:54.053 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.036+0000 7f7d1e7fc640 1 -- 172.21.15.88:0/2842817206 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f7d2411e770 con 0x7f7d280d22e0 2023-10-16T20:07:54.053 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.036+0000 7f7d2d700640 1 --2- 172.21.15.88:0/2842817206 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7d0c0664a0 0x7f7d0c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:54.053 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.036+0000 7f7d2d700640 1 --2- 172.21.15.88:0/2842817206 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7d0c0664a0 0x7f7d0c068960 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7d180014e0 tx=0x7f7d18001470 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:54.054 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.040+0000 7f7d1e7fc640 1 -- 172.21.15.88:0/2842817206 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7d240ebba0 con 0x7f7d280d22e0 2023-10-16T20:07:54.196 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.192+0000 7f7d30165640 1 -- 172.21.15.88:0/2842817206 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d280b84a0 con 0x7f7d280d22e0 2023-10-16T20:07:54.197 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.192+0000 7f7d1e7fc640 1 -- 172.21.15.88:0/2842817206 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16718 (secure 0 0 0) 0x7f7d240ef1c0 con 0x7f7d280d22e0 2023-10-16T20:07:54.197 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:07:54.198 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":58,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:07:27.206614+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T20:07:24.001121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"testcephfs1_data","create_time":"2023-10-16T20:07:24.636209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:07:54.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.192+0000 7f7d30165640 1 -- 172.21.15.88:0/2842817206 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7d0c0664a0 msgr2=0x7f7d0c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:54.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.192+0000 7f7d30165640 1 --2- 172.21.15.88:0/2842817206 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7d0c0664a0 0x7f7d0c068960 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7d180014e0 tx=0x7f7d18001470 comp rx=0 tx=0).stop 2023-10-16T20:07:54.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.192+0000 7f7d30165640 1 -- 172.21.15.88:0/2842817206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d280d22e0 msgr2=0x7f7d2814ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:54.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.192+0000 7f7d30165640 1 --2- 172.21.15.88:0/2842817206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d280d22e0 0x7f7d2814ed20 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f7d2401a990 tx=0x7f7d2401ab20 comp rx=0 tx=0).stop 2023-10-16T20:07:54.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.192+0000 7f7d30165640 1 -- 172.21.15.88:0/2842817206 shutdown_connections 2023-10-16T20:07:54.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.192+0000 7f7d30165640 1 --2- 172.21.15.88:0/2842817206 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f7d0c0664a0 0x7f7d0c068960 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:54.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.192+0000 7f7d30165640 1 --2- 172.21.15.88:0/2842817206 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7d280d26c0 0x7f7d281529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:54.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.192+0000 7f7d30165640 1 --2- 172.21.15.88:0/2842817206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d2815d9f0 0x7f7d28150bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:54.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.192+0000 7f7d30165640 1 --2- 172.21.15.88:0/2842817206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d280d22e0 0x7f7d2814ed20 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:54.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.192+0000 7f7d30165640 1 -- 172.21.15.88:0/2842817206 >> 172.21.15.88:0/2842817206 conn(0x7f7d280715c0 msgr2=0x7f7d280719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:54.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.192+0000 7f7d30165640 1 -- 172.21.15.88:0/2842817206 shutdown_connections 2023-10-16T20:07:54.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.192+0000 7f7d30165640 1 -- 172.21.15.88:0/2842817206 wait complete. 2023-10-16T20:07:54.218 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T20:04:44.508444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'testcephfs1_metadata', 'create_time': '2023-10-16T20:07:24.001121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'testcephfs1_data', 'create_time': '2023-10-16T20:07:24.636209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T20:07:54.219 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T20:07:54.359 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.352+0000 7f0a43c91640 1 Processor -- start 2023-10-16T20:07:54.360 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.356+0000 7f0a43c91640 1 -- start start 2023-10-16T20:07:54.361 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.356+0000 7f0a43c91640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a3c15b700 0x7f0a3c1653f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:54.361 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.356+0000 7f0a43c91640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a3c06d2f0 con 0x7f0a3c0b0c10 2023-10-16T20:07:54.362 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.356+0000 7f0a43c91640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a3c15bd20 con 0x7f0a3c154f30 2023-10-16T20:07:54.362 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.356+0000 7f0a43c91640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c155310 con 0x7f0a3c15b700 2023-10-16T20:07:54.362 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.356+0000 7f0a4122c640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f0a3c0b0c10 0x7f0a3c15aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:56390/0 (socket says 172.21.15.88:56390) 2023-10-16T20:07:54.362 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.356+0000 7f0a4122c640 1 -- 172.21.15.88:0/347790529 learned_addr learned my addr 172.21.15.88:0/347790529 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:54.362 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.356+0000 7f0a4222e640 1 --2- 172.21.15.88:0/347790529 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a3c15b700 0x7f0a3c1653f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:54.363 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.356+0000 7f0a40a2b640 1 -- 172.21.15.88:0/347790529 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2492802175 0 0) 0x7f0a3c06d2f0 con 0x7f0a3c0b0c10 2023-10-16T20:07:54.364 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.356+0000 7f0a40a2b640 1 -- 172.21.15.88:0/347790529 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a28003530 con 0x7f0a3c0b0c10 2023-10-16T20:07:54.364 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.356+0000 7f0a40a2b640 1 -- 172.21.15.88:0/347790529 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3229402692 0 0) 0x7f0a3c15bd20 con 0x7f0a3c154f30 2023-10-16T20:07:54.364 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.356+0000 7f0a40a2b640 1 -- 172.21.15.88:0/347790529 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a3c06d2f0 con 0x7f0a3c154f30 2023-10-16T20:07:54.364 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.356+0000 7f0a4222e640 1 -- 172.21.15.88:0/347790529 >> v1:172.21.15.94:6789/0 conn(0x7f0a3c154f30 legacy=0x7f0a3c0b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:54.365 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.356+0000 7f0a4222e640 1 -- 172.21.15.88:0/347790529 >> v1:172.21.15.88:6789/0 conn(0x7f0a3c0b0c10 legacy=0x7f0a3c15aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:54.365 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.356+0000 7f0a4222e640 1 -- 172.21.15.88:0/347790529 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3c0702f0 con 0x7f0a3c15b700 2023-10-16T20:07:54.366 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.360+0000 7f0a4222e640 1 --2- 172.21.15.88:0/347790529 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a3c15b700 0x7f0a3c1653f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0a34009dc0 tx=0x7f0a3401c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=159fbdf1fe6015c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:54.366 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.360+0000 7f0a40a2b640 1 -- 172.21.15.88:0/347790529 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a340a5a00 con 0x7f0a3c15b700 2023-10-16T20:07:54.366 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.360+0000 7f0a40a2b640 1 -- 172.21.15.88:0/347790529 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a340a5b60 con 0x7f0a3c15b700 2023-10-16T20:07:54.367 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.360+0000 7f0a40a2b640 1 -- 172.21.15.88:0/347790529 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a340ad440 con 0x7f0a3c15b700 2023-10-16T20:07:54.367 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.360+0000 7f0a43c91640 1 -- 172.21.15.88:0/347790529 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a3c15b700 msgr2=0x7f0a3c1653f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:54.367 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.360+0000 7f0a43c91640 1 --2- 172.21.15.88:0/347790529 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a3c15b700 0x7f0a3c1653f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0a34009dc0 tx=0x7f0a3401c940 comp rx=0 tx=0).stop 2023-10-16T20:07:54.367 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.360+0000 7f0a43c91640 1 -- 172.21.15.88:0/347790529 shutdown_connections 2023-10-16T20:07:54.367 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.360+0000 7f0a43c91640 1 --2- 172.21.15.88:0/347790529 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a3c15b700 0x7f0a3c1653f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:54.369 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.360+0000 7f0a43c91640 1 -- 172.21.15.88:0/347790529 >> 172.21.15.88:0/347790529 conn(0x7f0a3c0cb080 msgr2=0x7f0a3c0cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:54.369 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.360+0000 7f0a43c91640 1 -- 172.21.15.88:0/347790529 shutdown_connections 2023-10-16T20:07:54.369 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.360+0000 7f0a43c91640 1 -- 172.21.15.88:0/347790529 wait complete. 2023-10-16T20:07:54.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.360+0000 7f0a43c91640 1 Processor -- start 2023-10-16T20:07:54.372 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.364+0000 7f0a43c91640 1 -- start start 2023-10-16T20:07:54.372 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.364+0000 7f0a43c91640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a3c0b0c10 0x7f0a3c156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:54.372 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.364+0000 7f0a43c91640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a3c154f30 0x7f0a3c151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:54.372 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.364+0000 7f0a43c91640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a3c15b700 0x7f0a3c14c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:54.372 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.364+0000 7f0a43c91640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c14c950 con 0x7f0a3c0b0c10 2023-10-16T20:07:54.373 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.364+0000 7f0a43c91640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c158620 con 0x7f0a3c15b700 2023-10-16T20:07:54.373 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.364+0000 7f0a43c91640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c158900 con 0x7f0a3c154f30 2023-10-16T20:07:54.373 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.364+0000 7f0a41a2d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a3c0b0c10 0x7f0a3c156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:54.373 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.364+0000 7f0a41a2d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a3c0b0c10 0x7f0a3c156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57828/0 (socket says 172.21.15.88:57828) 2023-10-16T20:07:54.373 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.364+0000 7f0a41a2d640 1 -- 172.21.15.88:0/2550768443 learned_addr learned my addr 172.21.15.88:0/2550768443 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:54.374 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.364+0000 7f0a4222e640 1 --2- 172.21.15.88:0/2550768443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a3c15b700 0x7f0a3c14c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:54.374 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.364+0000 7f0a4122c640 1 --2- 172.21.15.88:0/2550768443 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a3c154f30 0x7f0a3c151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:54.374 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.364+0000 7f0a41a2d640 1 -- 172.21.15.88:0/2550768443 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a3c154f30 msgr2=0x7f0a3c151270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:54.374 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.364+0000 7f0a41a2d640 1 --2- 172.21.15.88:0/2550768443 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a3c154f30 0x7f0a3c151270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:54.375 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.364+0000 7f0a41a2d640 1 -- 172.21.15.88:0/2550768443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a3c15b700 msgr2=0x7f0a3c14c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:54.375 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.364+0000 7f0a41a2d640 1 --2- 172.21.15.88:0/2550768443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a3c15b700 0x7f0a3c14c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:54.375 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.364+0000 7f0a41a2d640 1 -- 172.21.15.88:0/2550768443 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a34009a70 con 0x7f0a3c0b0c10 2023-10-16T20:07:54.375 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.364+0000 7f0a41a2d640 1 --2- 172.21.15.88:0/2550768443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a3c0b0c10 0x7f0a3c156250 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f0a3800c490 tx=0x7f0a3800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:54.375 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.364+0000 7f0a327fc640 1 -- 172.21.15.88:0/2550768443 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a380052d0 con 0x7f0a3c0b0c10 2023-10-16T20:07:54.376 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.364+0000 7f0a327fc640 1 -- 172.21.15.88:0/2550768443 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a38005430 con 0x7f0a3c0b0c10 2023-10-16T20:07:54.376 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.364+0000 7f0a327fc640 1 -- 172.21.15.88:0/2550768443 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a38012870 con 0x7f0a3c0b0c10 2023-10-16T20:07:54.376 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.364+0000 7f0a43c91640 1 -- 172.21.15.88:0/2550768443 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a3c158be0 con 0x7f0a3c0b0c10 2023-10-16T20:07:54.376 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.364+0000 7f0a43c91640 1 -- 172.21.15.88:0/2550768443 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a3c1591a0 con 0x7f0a3c0b0c10 2023-10-16T20:07:54.376 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.364+0000 7f0a43c91640 1 -- 172.21.15.88:0/2550768443 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a3c159390 con 0x7f0a3c0b0c10 2023-10-16T20:07:54.376 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.368+0000 7f0a327fc640 1 -- 172.21.15.88:0/2550768443 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0a380129d0 con 0x7f0a3c0b0c10 2023-10-16T20:07:54.377 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.368+0000 7f0a327fc640 1 --2- 172.21.15.88:0/2550768443 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0a200664a0 0x7f0a20068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:54.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.368+0000 7f0a327fc640 1 -- 172.21.15.88:0/2550768443 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f0a38083a60 con 0x7f0a3c0b0c10 2023-10-16T20:07:54.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.368+0000 7f0a4122c640 1 --2- 172.21.15.88:0/2550768443 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0a200664a0 0x7f0a20068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:54.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.368+0000 7f0a4122c640 1 --2- 172.21.15.88:0/2550768443 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0a200664a0 0x7f0a20068960 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c0ca7f0 tx=0x7f0a2c007d80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:54.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.372+0000 7f0a327fc640 1 -- 172.21.15.88:0/2550768443 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0a38050e20 con 0x7f0a3c0b0c10 2023-10-16T20:07:54.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.520+0000 7f0a43c91640 1 -- 172.21.15.88:0/2550768443 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0a3c070b60 con 0x7f0a3c0b0c10 2023-10-16T20:07:54.527 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.520+0000 7f0a327fc640 1 -- 172.21.15.88:0/2550768443 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0a38054440 con 0x7f0a3c0b0c10 2023-10-16T20:07:54.527 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2023-10-16T20:07:54.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.524+0000 7f0a43c91640 1 -- 172.21.15.88:0/2550768443 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0a200664a0 msgr2=0x7f0a20068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:54.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.524+0000 7f0a43c91640 1 --2- 172.21.15.88:0/2550768443 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0a200664a0 0x7f0a20068960 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c0ca7f0 tx=0x7f0a2c007d80 comp rx=0 tx=0).stop 2023-10-16T20:07:54.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.524+0000 7f0a43c91640 1 -- 172.21.15.88:0/2550768443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a3c0b0c10 msgr2=0x7f0a3c156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:54.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.524+0000 7f0a43c91640 1 --2- 172.21.15.88:0/2550768443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a3c0b0c10 0x7f0a3c156250 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f0a3800c490 tx=0x7f0a3800c950 comp rx=0 tx=0).stop 2023-10-16T20:07:54.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.524+0000 7f0a43c91640 1 -- 172.21.15.88:0/2550768443 shutdown_connections 2023-10-16T20:07:54.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.524+0000 7f0a43c91640 1 --2- 172.21.15.88:0/2550768443 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f0a200664a0 0x7f0a20068960 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:54.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.524+0000 7f0a43c91640 1 --2- 172.21.15.88:0/2550768443 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a3c154f30 0x7f0a3c151270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:54.531 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.524+0000 7f0a43c91640 1 --2- 172.21.15.88:0/2550768443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a3c15b700 0x7f0a3c14c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:54.531 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.524+0000 7f0a43c91640 1 --2- 172.21.15.88:0/2550768443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a3c0b0c10 0x7f0a3c156250 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:54.531 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.524+0000 7f0a43c91640 1 -- 172.21.15.88:0/2550768443 >> 172.21.15.88:0/2550768443 conn(0x7f0a3c0cb080 msgr2=0x7f0a3c0b1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:54.531 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.524+0000 7f0a43c91640 1 -- 172.21.15.88:0/2550768443 shutdown_connections 2023-10-16T20:07:54.531 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.524+0000 7f0a43c91640 1 -- 172.21.15.88:0/2550768443 wait complete. 2023-10-16T20:07:54.548 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_metadata pg_num 2023-10-16T20:07:54.688 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.684+0000 7f556d9c4640 1 Processor -- start 2023-10-16T20:07:54.688 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.684+0000 7f556d9c4640 1 -- start start 2023-10-16T20:07:54.689 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.684+0000 7f556d9c4640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f55680d1be0 0x7f55680b4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:54.689 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.684+0000 7f556d9c4640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f55680b4a70 0x7f55680b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:54.690 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.684+0000 7f556d9c4640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55680b7d20 con 0x7f556815cb70 2023-10-16T20:07:54.690 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.684+0000 7f556d9c4640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55680b4f00 con 0x7f55680b4a70 2023-10-16T20:07:54.690 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.684+0000 7f556d9c4640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55680b7110 con 0x7f55680d1be0 2023-10-16T20:07:54.690 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.684+0000 7f5566ffd640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f556815cb70 0x7f55680b83d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:56392/0 (socket says 172.21.15.88:56392) 2023-10-16T20:07:54.690 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.684+0000 7f5566ffd640 1 -- 172.21.15.88:0/691216917 learned_addr learned my addr 172.21.15.88:0/691216917 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:54.691 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.684+0000 7f55667fc640 1 --2- 172.21.15.88:0/691216917 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f55680d1be0 0x7f55680b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:54.691 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.684+0000 7f55677fe640 1 --2- 172.21.15.88:0/691216917 >> v2:172.21.15.94:3300/0 conn(0x7f55680b4a70 0x7f55680b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:54.692 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.684+0000 7f5565ffb640 1 -- 172.21.15.88:0/691216917 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 244752187 0 0) 0x7f55680b7d20 con 0x7f556815cb70 2023-10-16T20:07:54.692 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.684+0000 7f5565ffb640 1 -- 172.21.15.88:0/691216917 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5550003530 con 0x7f556815cb70 2023-10-16T20:07:54.692 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.684+0000 7f55677fe640 1 -- 172.21.15.88:0/691216917 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f55680d1be0 msgr2=0x7f55680b4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:54.692 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.684+0000 7f55677fe640 1 --2- 172.21.15.88:0/691216917 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f55680d1be0 0x7f55680b4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:54.692 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.684+0000 7f55677fe640 1 -- 172.21.15.88:0/691216917 >> v1:172.21.15.88:6789/0 conn(0x7f556815cb70 legacy=0x7f55680b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:54.693 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.684+0000 7f55677fe640 1 -- 172.21.15.88:0/691216917 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55680b73f0 con 0x7f55680b4a70 2023-10-16T20:07:54.693 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.684+0000 7f55667fc640 1 --2- 172.21.15.88:0/691216917 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f55680d1be0 0x7f55680b4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:07:54.694 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.688+0000 7f55677fe640 1 --2- 172.21.15.88:0/691216917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55680b4a70 0x7f55680b5a70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5560009d90 tx=0x7f556001f130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b992f72dd8fac8d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:54.694 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.688+0000 7f5565ffb640 1 -- 172.21.15.88:0/691216917 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55600af600 con 0x7f55680b4a70 2023-10-16T20:07:54.694 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.688+0000 7f5565ffb640 1 -- 172.21.15.88:0/691216917 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55600ab050 con 0x7f55680b4a70 2023-10-16T20:07:54.695 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.688+0000 7f5565ffb640 1 -- 172.21.15.88:0/691216917 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55600af980 con 0x7f55680b4a70 2023-10-16T20:07:54.695 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.688+0000 7f556d9c4640 1 -- 172.21.15.88:0/691216917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55680b4a70 msgr2=0x7f55680b5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:54.695 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.688+0000 7f556d9c4640 1 --2- 172.21.15.88:0/691216917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55680b4a70 0x7f55680b5a70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5560009d90 tx=0x7f556001f130 comp rx=0 tx=0).stop 2023-10-16T20:07:54.695 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.688+0000 7f556d9c4640 1 -- 172.21.15.88:0/691216917 shutdown_connections 2023-10-16T20:07:54.695 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.688+0000 7f556d9c4640 1 --2- 172.21.15.88:0/691216917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55680b4a70 0x7f55680b5a70 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:54.695 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.688+0000 7f556d9c4640 1 --2- 172.21.15.88:0/691216917 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f55680d1be0 0x7f55680b4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:54.696 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.688+0000 7f556d9c4640 1 -- 172.21.15.88:0/691216917 >> 172.21.15.88:0/691216917 conn(0x7f556806b6e0 msgr2=0x7f556806baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:54.696 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.688+0000 7f556d9c4640 1 -- 172.21.15.88:0/691216917 shutdown_connections 2023-10-16T20:07:54.696 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.688+0000 7f556d9c4640 1 -- 172.21.15.88:0/691216917 wait complete. 2023-10-16T20:07:54.698 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.688+0000 7f556d9c4640 1 Processor -- start 2023-10-16T20:07:54.698 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.692+0000 7f556d9c4640 1 -- start start 2023-10-16T20:07:54.698 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.692+0000 7f556d9c4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55680b4a70 0x7f556815e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:54.699 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.692+0000 7f556d9c4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55680d1be0 0x7f55680b5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:54.699 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.692+0000 7f556d9c4640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f556815cb70 0x7f556814eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:54.699 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.692+0000 7f556d9c4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556815f7b0 con 0x7f55680d1be0 2023-10-16T20:07:54.699 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.692+0000 7f556d9c4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556815eba0 con 0x7f55680b4a70 2023-10-16T20:07:54.699 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.692+0000 7f556d9c4640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f556815ee80 con 0x7f556815cb70 2023-10-16T20:07:54.700 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.692+0000 7f55667fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55680d1be0 0x7f55680b5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:54.700 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.692+0000 7f55667fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55680d1be0 0x7f55680b5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57830/0 (socket says 172.21.15.88:57830) 2023-10-16T20:07:54.700 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.692+0000 7f55667fc640 1 -- 172.21.15.88:0/1274050530 learned_addr learned my addr 172.21.15.88:0/1274050530 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:54.700 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.692+0000 7f5566ffd640 1 --2- 172.21.15.88:0/1274050530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55680b4a70 0x7f556815e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:54.700 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.692+0000 7f55677fe640 1 --2- 172.21.15.88:0/1274050530 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f556815cb70 0x7f556814eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:54.701 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.692+0000 7f55667fc640 1 -- 172.21.15.88:0/1274050530 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f556815cb70 msgr2=0x7f556814eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:54.701 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.692+0000 7f55667fc640 1 --2- 172.21.15.88:0/1274050530 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f556815cb70 0x7f556814eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:54.701 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.692+0000 7f55667fc640 1 -- 172.21.15.88:0/1274050530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55680b4a70 msgr2=0x7f556815e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:54.701 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.692+0000 7f55667fc640 1 --2- 172.21.15.88:0/1274050530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55680b4a70 0x7f556815e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:54.701 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.692+0000 7f55667fc640 1 -- 172.21.15.88:0/1274050530 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f555c009420 con 0x7f55680d1be0 2023-10-16T20:07:54.701 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.692+0000 7f55667fc640 1 --2- 172.21.15.88:0/1274050530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55680d1be0 0x7f55680b5ed0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f555c00ebe0 tx=0x7f555c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:54.702 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.692+0000 7f554ffff640 1 -- 172.21.15.88:0/1274050530 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555c00eee0 con 0x7f55680d1be0 2023-10-16T20:07:54.702 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.692+0000 7f554ffff640 1 -- 172.21.15.88:0/1274050530 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f555c00cc30 con 0x7f55680d1be0 2023-10-16T20:07:54.702 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.692+0000 7f556d9c4640 1 -- 172.21.15.88:0/1274050530 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5560009a70 con 0x7f55680d1be0 2023-10-16T20:07:54.702 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.692+0000 7f554ffff640 1 -- 172.21.15.88:0/1274050530 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555c013470 con 0x7f55680d1be0 2023-10-16T20:07:54.702 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.692+0000 7f556d9c4640 1 -- 172.21.15.88:0/1274050530 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f556815f440 con 0x7f55680d1be0 2023-10-16T20:07:54.703 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.692+0000 7f556d9c4640 1 -- 172.21.15.88:0/1274050530 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f556815f920 con 0x7f55680d1be0 2023-10-16T20:07:54.704 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.696+0000 7f554ffff640 1 -- 172.21.15.88:0/1274050530 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f555c0135d0 con 0x7f55680d1be0 2023-10-16T20:07:54.704 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.696+0000 7f554ffff640 1 --2- 172.21.15.88:0/1274050530 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f55540664a0 0x7f5554068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:54.704 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.696+0000 7f554ffff640 1 -- 172.21.15.88:0/1274050530 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f555c084330 con 0x7f55680d1be0 2023-10-16T20:07:54.704 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.696+0000 7f5566ffd640 1 --2- 172.21.15.88:0/1274050530 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f55540664a0 0x7f5554068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:54.704 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.696+0000 7f5566ffd640 1 --2- 172.21.15.88:0/1274050530 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f55540664a0 0x7f5554068960 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f55580011f0 tx=0x7f5558007d40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:54.705 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.700+0000 7f554ffff640 1 -- 172.21.15.88:0/1274050530 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f555c051660 con 0x7f55680d1be0 2023-10-16T20:07:54.851 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.844+0000 7f556d9c4640 1 -- 172.21.15.88:0/1274050530 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"} v 0) v1 -- 0x7f556804e9c0 con 0x7f55680d1be0 2023-10-16T20:07:54.852 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.848+0000 7f554ffff640 1 -- 172.21.15.88:0/1274050530 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"}]=0 v58) v1 ==== 109+0+11 (secure 0 0 0) 0x7f555c054c80 con 0x7f55680d1be0 2023-10-16T20:07:54.852 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2023-10-16T20:07:54.853 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.848+0000 7f556d9c4640 1 -- 172.21.15.88:0/1274050530 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f55540664a0 msgr2=0x7f5554068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:54.854 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.848+0000 7f556d9c4640 1 --2- 172.21.15.88:0/1274050530 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f55540664a0 0x7f5554068960 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f55580011f0 tx=0x7f5558007d40 comp rx=0 tx=0).stop 2023-10-16T20:07:54.854 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.848+0000 7f556d9c4640 1 -- 172.21.15.88:0/1274050530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55680d1be0 msgr2=0x7f55680b5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:54.854 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.848+0000 7f556d9c4640 1 --2- 172.21.15.88:0/1274050530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55680d1be0 0x7f55680b5ed0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f555c00ebe0 tx=0x7f555c00c520 comp rx=0 tx=0).stop 2023-10-16T20:07:54.854 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.848+0000 7f556d9c4640 1 -- 172.21.15.88:0/1274050530 shutdown_connections 2023-10-16T20:07:54.855 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.848+0000 7f556d9c4640 1 --2- 172.21.15.88:0/1274050530 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f55540664a0 0x7f5554068960 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:54.855 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.848+0000 7f556d9c4640 1 --2- 172.21.15.88:0/1274050530 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f556815cb70 0x7f556814eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:54.855 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.848+0000 7f556d9c4640 1 --2- 172.21.15.88:0/1274050530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55680d1be0 0x7f55680b5ed0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:54.855 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.848+0000 7f556d9c4640 1 --2- 172.21.15.88:0/1274050530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55680b4a70 0x7f556815e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:54.855 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.848+0000 7f556d9c4640 1 -- 172.21.15.88:0/1274050530 >> 172.21.15.88:0/1274050530 conn(0x7f556806b6e0 msgr2=0x7f55680d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:54.855 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.848+0000 7f556d9c4640 1 -- 172.21.15.88:0/1274050530 shutdown_connections 2023-10-16T20:07:54.856 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:54.848+0000 7f556d9c4640 1 -- 172.21.15.88:0/1274050530 wait complete. 2023-10-16T20:07:54.873 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_data pg_num 2023-10-16T20:07:55.013 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.008+0000 7fcf960bb640 1 Processor -- start 2023-10-16T20:07:55.013 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.008+0000 7fcf960bb640 1 -- start start 2023-10-16T20:07:55.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.008+0000 7fcf960bb640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fcf900cde80 0x7fcf900d3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:55.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.008+0000 7fcf960bb640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fcf900cf230 0x7fcf900cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:55.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.008+0000 7fcf960bb640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf900d0c40 con 0x7fcf9015c9a0 2023-10-16T20:07:55.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.008+0000 7fcf960bb640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf900d37b0 con 0x7fcf900cde80 2023-10-16T20:07:55.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.008+0000 7fcf960bb640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf900ce7e0 con 0x7fcf900cf230 2023-10-16T20:07:55.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.008+0000 7fcf948b8640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fcf900cde80 0x7fcf900d3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:55.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.008+0000 7fcf948b8640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fcf900cde80 0x7fcf900d3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57844/0 (socket says 172.21.15.88:57844) 2023-10-16T20:07:55.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.008+0000 7fcf948b8640 1 -- 172.21.15.88:0/1354671547 learned_addr learned my addr 172.21.15.88:0/1354671547 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:55.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.008+0000 7fcf958ba640 1 --2- 172.21.15.88:0/1354671547 >> v2:172.21.15.94:3300/0 conn(0x7fcf900cf230 0x7fcf900cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:55.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.008+0000 7fcf948b8640 1 -- 172.21.15.88:0/1354671547 >> v2:172.21.15.94:3300/0 conn(0x7fcf900cf230 msgr2=0x7fcf900cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:55.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.008+0000 7fcf948b8640 1 --2- 172.21.15.88:0/1354671547 >> v2:172.21.15.94:3300/0 conn(0x7fcf900cf230 0x7fcf900cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:55.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.008+0000 7fcf948b8640 1 -- 172.21.15.88:0/1354671547 >> v1:172.21.15.94:6789/0 conn(0x7fcf9015c9a0 legacy=0x7fcf900d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:55.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.008+0000 7fcf948b8640 1 -- 172.21.15.88:0/1354671547 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf900ce2d0 con 0x7fcf900cde80 2023-10-16T20:07:55.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.012+0000 7fcf948b8640 1 --2- 172.21.15.88:0/1354671547 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf900cde80 0x7fcf900d3270 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fcf900c63a0 tx=0x7fcf8001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f903aa14db3f5fd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:55.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.012+0000 7fcf87fff640 1 -- 172.21.15.88:0/1354671547 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf800ac410 con 0x7fcf900cde80 2023-10-16T20:07:55.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.012+0000 7fcf87fff640 1 -- 172.21.15.88:0/1354671547 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf800ac570 con 0x7fcf900cde80 2023-10-16T20:07:55.019 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.012+0000 7fcf87fff640 1 -- 172.21.15.88:0/1354671547 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf800ac900 con 0x7fcf900cde80 2023-10-16T20:07:55.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.012+0000 7fcf960bb640 1 -- 172.21.15.88:0/1354671547 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf900cde80 msgr2=0x7fcf900d3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:55.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.012+0000 7fcf960bb640 1 --2- 172.21.15.88:0/1354671547 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf900cde80 0x7fcf900d3270 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fcf900c63a0 tx=0x7fcf8001c4c0 comp rx=0 tx=0).stop 2023-10-16T20:07:55.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.012+0000 7fcf960bb640 1 -- 172.21.15.88:0/1354671547 shutdown_connections 2023-10-16T20:07:55.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.012+0000 7fcf960bb640 1 --2- 172.21.15.88:0/1354671547 >> v2:172.21.15.94:3300/0 conn(0x7fcf900cf230 0x7fcf900cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:55.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.012+0000 7fcf960bb640 1 --2- 172.21.15.88:0/1354671547 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf900cde80 0x7fcf900d3270 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:55.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.012+0000 7fcf960bb640 1 -- 172.21.15.88:0/1354671547 >> 172.21.15.88:0/1354671547 conn(0x7fcf9009bc30 msgr2=0x7fcf900d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:55.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.012+0000 7fcf960bb640 1 -- 172.21.15.88:0/1354671547 shutdown_connections 2023-10-16T20:07:55.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.012+0000 7fcf960bb640 1 -- 172.21.15.88:0/1354671547 wait complete. 2023-10-16T20:07:55.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.016+0000 7fcf960bb640 1 Processor -- start 2023-10-16T20:07:55.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.016+0000 7fcf960bb640 1 -- start start 2023-10-16T20:07:55.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.016+0000 7fcf960bb640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf900cde80 0x7fcf900b83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:55.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.016+0000 7fcf960bb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf900cf230 0x7fcf900cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:55.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.016+0000 7fcf960bb640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf9015c9a0 0x7fcf9015b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:55.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.016+0000 7fcf960bb640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf900b54b0 con 0x7fcf900cde80 2023-10-16T20:07:55.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.016+0000 7fcf960bb640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf900ca3a0 con 0x7fcf900cf230 2023-10-16T20:07:55.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.016+0000 7fcf960bb640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf900ca5f0 con 0x7fcf9015c9a0 2023-10-16T20:07:55.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.016+0000 7fcf950b9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf900cde80 0x7fcf900b83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:55.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.016+0000 7fcf948b8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf900cf230 0x7fcf900cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:55.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.016+0000 7fcf950b9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf900cde80 0x7fcf900b83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57856/0 (socket says 172.21.15.88:57856) 2023-10-16T20:07:55.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.016+0000 7fcf950b9640 1 -- 172.21.15.88:0/4163823052 learned_addr learned my addr 172.21.15.88:0/4163823052 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:55.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.016+0000 7fcf948b8640 1 --2- 172.21.15.88:0/4163823052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf900cf230 0x7fcf900cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:50382/0 (socket says 172.21.15.88:50382) 2023-10-16T20:07:55.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.016+0000 7fcf958ba640 1 --2- 172.21.15.88:0/4163823052 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf9015c9a0 0x7fcf9015b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:55.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.016+0000 7fcf950b9640 1 -- 172.21.15.88:0/4163823052 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf9015c9a0 msgr2=0x7fcf9015b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:55.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.016+0000 7fcf950b9640 1 --2- 172.21.15.88:0/4163823052 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf9015c9a0 0x7fcf9015b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:55.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.016+0000 7fcf950b9640 1 -- 172.21.15.88:0/4163823052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf900cf230 msgr2=0x7fcf900cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:55.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.016+0000 7fcf950b9640 1 --2- 172.21.15.88:0/4163823052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf900cf230 0x7fcf900cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:55.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.016+0000 7fcf950b9640 1 -- 172.21.15.88:0/4163823052 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf800094d0 con 0x7fcf900cde80 2023-10-16T20:07:55.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.016+0000 7fcf948b8640 1 --2- 172.21.15.88:0/4163823052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf900cf230 0x7fcf900cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:07:55.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.016+0000 7fcf950b9640 1 --2- 172.21.15.88:0/4163823052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf900cde80 0x7fcf900b83f0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c00b3f0 tx=0x7fcf8c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:55.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.016+0000 7fcf85ffb640 1 -- 172.21.15.88:0/4163823052 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c011830 con 0x7fcf900cde80 2023-10-16T20:07:55.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.016+0000 7fcf85ffb640 1 -- 172.21.15.88:0/4163823052 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf8c0119b0 con 0x7fcf900cde80 2023-10-16T20:07:55.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.016+0000 7fcf960bb640 1 -- 172.21.15.88:0/4163823052 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf900ca8d0 con 0x7fcf900cde80 2023-10-16T20:07:55.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.016+0000 7fcf85ffb640 1 -- 172.21.15.88:0/4163823052 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c011d60 con 0x7fcf900cde80 2023-10-16T20:07:55.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.016+0000 7fcf960bb640 1 -- 172.21.15.88:0/4163823052 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf900cae90 con 0x7fcf900cde80 2023-10-16T20:07:55.029 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.020+0000 7fcf960bb640 1 -- 172.21.15.88:0/4163823052 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf900674f0 con 0x7fcf900cde80 2023-10-16T20:07:55.030 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.020+0000 7fcf85ffb640 1 -- 172.21.15.88:0/4163823052 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcf8c01a400 con 0x7fcf900cde80 2023-10-16T20:07:55.030 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.020+0000 7fcf85ffb640 1 --2- 172.21.15.88:0/4163823052 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcf740664a0 0x7fcf74068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:55.030 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.020+0000 7fcf85ffb640 1 -- 172.21.15.88:0/4163823052 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fcf8c084110 con 0x7fcf900cde80 2023-10-16T20:07:55.030 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.020+0000 7fcf948b8640 1 --2- 172.21.15.88:0/4163823052 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcf740664a0 0x7fcf74068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:55.030 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.020+0000 7fcf948b8640 1 --2- 172.21.15.88:0/4163823052 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcf740664a0 0x7fcf74068960 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fcf800ae000 tx=0x7fcf80009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:55.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.024+0000 7fcf85ffb640 1 -- 172.21.15.88:0/4163823052 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcf8c0513b0 con 0x7fcf900cde80 2023-10-16T20:07:55.178 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.172+0000 7fcf960bb640 1 -- 172.21.15.88:0/4163823052 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"} v 0) v1 -- 0x7fcf900cff40 con 0x7fcf900cde80 2023-10-16T20:07:55.179 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.172+0000 7fcf85ffb640 1 -- 172.21.15.88:0/4163823052 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"}]=0 v58) v1 ==== 105+0+11 (secure 0 0 0) 0x7fcf8c015020 con 0x7fcf900cde80 2023-10-16T20:07:55.179 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2023-10-16T20:07:55.181 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.176+0000 7fcf960bb640 1 -- 172.21.15.88:0/4163823052 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcf740664a0 msgr2=0x7fcf74068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:55.181 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.176+0000 7fcf960bb640 1 --2- 172.21.15.88:0/4163823052 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcf740664a0 0x7fcf74068960 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fcf800ae000 tx=0x7fcf80009040 comp rx=0 tx=0).stop 2023-10-16T20:07:55.181 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.176+0000 7fcf960bb640 1 -- 172.21.15.88:0/4163823052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf900cde80 msgr2=0x7fcf900b83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:55.181 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.176+0000 7fcf960bb640 1 --2- 172.21.15.88:0/4163823052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf900cde80 0x7fcf900b83f0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c00b3f0 tx=0x7fcf8c00b8b0 comp rx=0 tx=0).stop 2023-10-16T20:07:55.182 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.176+0000 7fcf960bb640 1 -- 172.21.15.88:0/4163823052 shutdown_connections 2023-10-16T20:07:55.182 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.176+0000 7fcf960bb640 1 --2- 172.21.15.88:0/4163823052 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fcf740664a0 0x7fcf74068960 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:55.182 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.176+0000 7fcf960bb640 1 --2- 172.21.15.88:0/4163823052 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf9015c9a0 0x7fcf9015b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:55.182 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.176+0000 7fcf960bb640 1 --2- 172.21.15.88:0/4163823052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf900cf230 0x7fcf900cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:55.182 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.176+0000 7fcf960bb640 1 --2- 172.21.15.88:0/4163823052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf900cde80 0x7fcf900b83f0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:55.183 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.176+0000 7fcf960bb640 1 -- 172.21.15.88:0/4163823052 >> 172.21.15.88:0/4163823052 conn(0x7fcf9009bc30 msgr2=0x7fcf900aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:55.183 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.176+0000 7fcf960bb640 1 -- 172.21.15.88:0/4163823052 shutdown_connections 2023-10-16T20:07:55.183 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.176+0000 7fcf960bb640 1 -- 172.21.15.88:0/4163823052 wait complete. 2023-10-16T20:07:55.200 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T20:07:55.337 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.332+0000 7fbc6a1aa640 1 Processor -- start 2023-10-16T20:07:55.337 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.332+0000 7fbc6a1aa640 1 -- start start 2023-10-16T20:07:55.338 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.332+0000 7fbc6a1aa640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fbc6415d9f0 0x7fbc64161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:55.338 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.332+0000 7fbc6a1aa640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbc640d26c0 0x7fbc640cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:55.339 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.332+0000 7fbc6a1aa640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc640b87e0 con 0x7fbc640d22e0 2023-10-16T20:07:55.339 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.332+0000 7fbc6a1aa640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc6415df10 con 0x7fbc6415d9f0 2023-10-16T20:07:55.339 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.332+0000 7fbc6a1aa640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6415e1f0 con 0x7fbc640d26c0 2023-10-16T20:07:55.340 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.332+0000 7fbc62ffd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fbc6415d9f0 0x7fbc64161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:55.340 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.332+0000 7fbc637fe640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fbc640d22e0 0x7fbc6415d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:54720/0 (socket says 172.21.15.88:54720) 2023-10-16T20:07:55.340 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.332+0000 7fbc637fe640 1 -- 172.21.15.88:0/1076079825 learned_addr learned my addr 172.21.15.88:0/1076079825 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:55.341 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.332+0000 7fbc63fff640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbc640d26c0 0x7fbc640cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:55.341 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.332+0000 7fbc627fc640 1 -- 172.21.15.88:0/1076079825 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4038577051 0 0) 0x7fbc640b87e0 con 0x7fbc640d22e0 2023-10-16T20:07:55.341 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.332+0000 7fbc627fc640 1 -- 172.21.15.88:0/1076079825 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc4c003530 con 0x7fbc640d22e0 2023-10-16T20:07:55.341 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.332+0000 7fbc62ffd640 1 -- 172.21.15.88:0/1076079825 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbc640d26c0 msgr2=0x7fbc640cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:55.341 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.332+0000 7fbc62ffd640 1 --2- 172.21.15.88:0/1076079825 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbc640d26c0 0x7fbc640cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:55.342 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.336+0000 7fbc62ffd640 1 -- 172.21.15.88:0/1076079825 >> v1:172.21.15.94:6789/0 conn(0x7fbc640d22e0 legacy=0x7fbc6415d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:55.342 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.336+0000 7fbc62ffd640 1 -- 172.21.15.88:0/1076079825 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc64160970 con 0x7fbc6415d9f0 2023-10-16T20:07:55.342 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.336+0000 7fbc63fff640 1 --2- 172.21.15.88:0/1076079825 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbc640d26c0 0x7fbc640cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:07:55.343 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.336+0000 7fbc62ffd640 1 --2- 172.21.15.88:0/1076079825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc6415d9f0 0x7fbc64161450 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbc580097c0 tx=0x7fbc5801c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd58aff35dafea6c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:55.344 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.336+0000 7fbc627fc640 1 -- 172.21.15.88:0/1076079825 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc580ac620 con 0x7fbc6415d9f0 2023-10-16T20:07:55.344 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.336+0000 7fbc627fc640 1 -- 172.21.15.88:0/1076079825 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc580ac780 con 0x7fbc6415d9f0 2023-10-16T20:07:55.344 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.336+0000 7fbc627fc640 1 -- 172.21.15.88:0/1076079825 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc580acb10 con 0x7fbc6415d9f0 2023-10-16T20:07:55.344 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.336+0000 7fbc6a1aa640 1 -- 172.21.15.88:0/1076079825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc6415d9f0 msgr2=0x7fbc64161450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:55.344 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.336+0000 7fbc6a1aa640 1 --2- 172.21.15.88:0/1076079825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc6415d9f0 0x7fbc64161450 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbc580097c0 tx=0x7fbc5801c3c0 comp rx=0 tx=0).stop 2023-10-16T20:07:55.345 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.336+0000 7fbc6a1aa640 1 -- 172.21.15.88:0/1076079825 shutdown_connections 2023-10-16T20:07:55.345 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.336+0000 7fbc6a1aa640 1 --2- 172.21.15.88:0/1076079825 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbc640d26c0 0x7fbc640cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:55.345 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.336+0000 7fbc6a1aa640 1 --2- 172.21.15.88:0/1076079825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc6415d9f0 0x7fbc64161450 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:55.345 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.336+0000 7fbc6a1aa640 1 -- 172.21.15.88:0/1076079825 >> 172.21.15.88:0/1076079825 conn(0x7fbc640715c0 msgr2=0x7fbc640719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:55.346 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.336+0000 7fbc6a1aa640 1 -- 172.21.15.88:0/1076079825 shutdown_connections 2023-10-16T20:07:55.346 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.336+0000 7fbc6a1aa640 1 -- 172.21.15.88:0/1076079825 wait complete. 2023-10-16T20:07:55.348 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.340+0000 7fbc6a1aa640 1 Processor -- start 2023-10-16T20:07:55.348 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.340+0000 7fbc6a1aa640 1 -- start start 2023-10-16T20:07:55.348 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.340+0000 7fbc6a1aa640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc640d22e0 0x7fbc640c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:55.348 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.340+0000 7fbc6a1aa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc640d26c0 0x7fbc641579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:55.349 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.340+0000 7fbc6a1aa640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbc6415d9f0 0x7fbc6414f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:55.349 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.340+0000 7fbc6a1aa640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc64158d90 con 0x7fbc640d22e0 2023-10-16T20:07:55.349 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.340+0000 7fbc6a1aa640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc64157ee0 con 0x7fbc640d26c0 2023-10-16T20:07:55.349 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.340+0000 7fbc6a1aa640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc64158110 con 0x7fbc6415d9f0 2023-10-16T20:07:55.349 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.340+0000 7fbc637fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc640d22e0 0x7fbc640c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:55.350 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.340+0000 7fbc637fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc640d22e0 0x7fbc640c90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57872/0 (socket says 172.21.15.88:57872) 2023-10-16T20:07:55.350 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.340+0000 7fbc637fe640 1 -- 172.21.15.88:0/621024713 learned_addr learned my addr 172.21.15.88:0/621024713 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:55.350 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.340+0000 7fbc63fff640 1 --2- 172.21.15.88:0/621024713 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbc6415d9f0 0x7fbc6414f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:55.350 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.340+0000 7fbc62ffd640 1 --2- 172.21.15.88:0/621024713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc640d26c0 0x7fbc641579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:55.350 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.340+0000 7fbc637fe640 1 -- 172.21.15.88:0/621024713 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbc6415d9f0 msgr2=0x7fbc6414f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:55.351 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.340+0000 7fbc637fe640 1 --2- 172.21.15.88:0/621024713 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbc6415d9f0 0x7fbc6414f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:55.351 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.340+0000 7fbc637fe640 1 -- 172.21.15.88:0/621024713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc640d26c0 msgr2=0x7fbc641579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:55.351 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.340+0000 7fbc637fe640 1 --2- 172.21.15.88:0/621024713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc640d26c0 0x7fbc641579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:55.351 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.340+0000 7fbc637fe640 1 -- 172.21.15.88:0/621024713 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc5c009a70 con 0x7fbc640d22e0 2023-10-16T20:07:55.351 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.340+0000 7fbc637fe640 1 --2- 172.21.15.88:0/621024713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc640d22e0 0x7fbc640c90a0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fbc5400c4d0 tx=0x7fbc5400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:55.352 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.340+0000 7fbc4bfff640 1 -- 172.21.15.88:0/621024713 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc54005cf0 con 0x7fbc640d22e0 2023-10-16T20:07:55.352 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.344+0000 7fbc4bfff640 1 -- 172.21.15.88:0/621024713 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc54005e50 con 0x7fbc640d22e0 2023-10-16T20:07:55.352 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.344+0000 7fbc4bfff640 1 -- 172.21.15.88:0/621024713 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc540072d0 con 0x7fbc640d22e0 2023-10-16T20:07:55.352 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.344+0000 7fbc6a1aa640 1 -- 172.21.15.88:0/621024713 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc58009420 con 0x7fbc640d22e0 2023-10-16T20:07:55.352 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.344+0000 7fbc6a1aa640 1 -- 172.21.15.88:0/621024713 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc641586d0 con 0x7fbc640d22e0 2023-10-16T20:07:55.353 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.344+0000 7fbc6a1aa640 1 -- 172.21.15.88:0/621024713 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc6404e9c0 con 0x7fbc640d22e0 2023-10-16T20:07:55.354 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.344+0000 7fbc4bfff640 1 -- 172.21.15.88:0/621024713 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbc54007430 con 0x7fbc640d22e0 2023-10-16T20:07:55.354 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.344+0000 7fbc4bfff640 1 --2- 172.21.15.88:0/621024713 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbc500664d0 0x7fbc50068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:55.354 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.344+0000 7fbc4bfff640 1 -- 172.21.15.88:0/621024713 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fbc54083570 con 0x7fbc640d22e0 2023-10-16T20:07:55.354 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.344+0000 7fbc62ffd640 1 --2- 172.21.15.88:0/621024713 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbc500664d0 0x7fbc50068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:55.354 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.344+0000 7fbc62ffd640 1 --2- 172.21.15.88:0/621024713 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbc500664d0 0x7fbc50068990 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbc580ae000 tx=0x7fbc58008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:55.355 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.348+0000 7fbc4bfff640 1 -- 172.21.15.88:0/621024713 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbc540508a0 con 0x7fbc640d22e0 2023-10-16T20:07:55.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.528+0000 7fbc6a1aa640 1 -- 172.21.15.88:0/621024713 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc640d0840 con 0x7fbc640d22e0 2023-10-16T20:07:55.535 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.528+0000 7fbc4bfff640 1 -- 172.21.15.88:0/621024713 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fbc54053ec0 con 0x7fbc640d22e0 2023-10-16T20:07:55.535 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:07:55.536 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5531,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/2485638574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":2485638574},{"type":"v1","addr":"172.21.15.88:6837","nonce":2485638574}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5534,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3821390040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3821390040},{"type":"v1","addr":"172.21.15.94:6834","nonce":3821390040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5968,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2544673478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":2544673478},{"type":"v1","addr":"172.21.15.94:6835","nonce":2544673478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T20:07:25.650705+0000","modified":"2023-10-16T20:07:28.651868+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6210":{"gid":6210,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6836/1018125900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":1018125900},{"type":"v1","addr":"172.21.15.88:6836","nonce":1018125900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"testcephfs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-16T20:07:55.536 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 28 2023-10-16T20:07:55.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.532+0000 7fbc6a1aa640 1 -- 172.21.15.88:0/621024713 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbc500664d0 msgr2=0x7fbc50068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:55.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.532+0000 7fbc6a1aa640 1 --2- 172.21.15.88:0/621024713 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbc500664d0 0x7fbc50068990 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbc580ae000 tx=0x7fbc58008040 comp rx=0 tx=0).stop 2023-10-16T20:07:55.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.532+0000 7fbc6a1aa640 1 -- 172.21.15.88:0/621024713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc640d22e0 msgr2=0x7fbc640c90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:55.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.532+0000 7fbc6a1aa640 1 --2- 172.21.15.88:0/621024713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc640d22e0 0x7fbc640c90a0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fbc5400c4d0 tx=0x7fbc5400c8d0 comp rx=0 tx=0).stop 2023-10-16T20:07:55.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.532+0000 7fbc6a1aa640 1 -- 172.21.15.88:0/621024713 shutdown_connections 2023-10-16T20:07:55.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.532+0000 7fbc6a1aa640 1 --2- 172.21.15.88:0/621024713 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbc500664d0 0x7fbc50068990 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:55.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.532+0000 7fbc6a1aa640 1 --2- 172.21.15.88:0/621024713 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbc6415d9f0 0x7fbc6414f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:55.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.532+0000 7fbc6a1aa640 1 --2- 172.21.15.88:0/621024713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc640d26c0 0x7fbc641579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:55.539 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.532+0000 7fbc6a1aa640 1 --2- 172.21.15.88:0/621024713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc640d22e0 0x7fbc640c90a0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:55.539 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.532+0000 7fbc6a1aa640 1 -- 172.21.15.88:0/621024713 >> 172.21.15.88:0/621024713 conn(0x7fbc640715c0 msgr2=0x7fbc6415be10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:55.539 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.532+0000 7fbc6a1aa640 1 -- 172.21.15.88:0/621024713 shutdown_connections 2023-10-16T20:07:55.539 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.532+0000 7fbc6a1aa640 1 -- 172.21.15.88:0/621024713 wait complete. 2023-10-16T20:07:55.556 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:07:55.694 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.688+0000 7fc943c79640 1 Processor -- start 2023-10-16T20:07:55.694 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.688+0000 7fc943c79640 1 -- start start 2023-10-16T20:07:55.695 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.688+0000 7fc943c79640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fc93c160cc0 0x7fc93c0ca800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:55.695 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.688+0000 7fc943c79640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc93c0cd610 0x7fc93c0cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:55.696 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.688+0000 7fc943c79640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc93c0cc570 con 0x7fc93c1610a0 2023-10-16T20:07:55.696 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.688+0000 7fc943c79640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc93c0cc850 con 0x7fc93c0cd610 2023-10-16T20:07:55.696 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.688+0000 7fc943c79640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc93c0ccb30 con 0x7fc93c160cc0 2023-10-16T20:07:55.697 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.692+0000 7fc941a15640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fc93c160cc0 0x7fc93c0ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:55.697 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.692+0000 7fc941a15640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fc93c160cc0 0x7fc93c0ca800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:50404/0 (socket says 172.21.15.88:50404) 2023-10-16T20:07:55.698 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.692+0000 7fc941a15640 1 -- 172.21.15.88:0/1396342991 learned_addr learned my addr 172.21.15.88:0/1396342991 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:55.698 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.692+0000 7fc941214640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fc93c1610a0 0x7fc93c0ae1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:56404/0 (socket says 172.21.15.88:56404) 2023-10-16T20:07:55.698 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.692+0000 7fc942216640 1 --2- 172.21.15.88:0/1396342991 >> v2:172.21.15.88:3300/0 conn(0x7fc93c0cd610 0x7fc93c0cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:55.699 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.692+0000 7fc941a15640 1 -- 172.21.15.88:0/1396342991 >> v2:172.21.15.88:3300/0 conn(0x7fc93c0cd610 msgr2=0x7fc93c0cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:55.699 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.692+0000 7fc941a15640 1 --2- 172.21.15.88:0/1396342991 >> v2:172.21.15.88:3300/0 conn(0x7fc93c0cd610 0x7fc93c0cdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:55.699 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.692+0000 7fc941a15640 1 -- 172.21.15.88:0/1396342991 >> v1:172.21.15.88:6789/0 conn(0x7fc93c1610a0 legacy=0x7fc93c0ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:55.699 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.692+0000 7fc941a15640 1 -- 172.21.15.88:0/1396342991 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93c0cce10 con 0x7fc93c160cc0 2023-10-16T20:07:55.701 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.692+0000 7fc941a15640 1 --2- 172.21.15.88:0/1396342991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc93c160cc0 0x7fc93c0ca800 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0cf570 tx=0x7fc93801c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e12c11cfa3de0d96 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:55.701 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.692+0000 7fc940a13640 1 -- 172.21.15.88:0/1396342991 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9380aa430 con 0x7fc93c160cc0 2023-10-16T20:07:55.701 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.692+0000 7fc940a13640 1 -- 172.21.15.88:0/1396342991 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9380aa590 con 0x7fc93c160cc0 2023-10-16T20:07:55.701 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.692+0000 7fc940a13640 1 -- 172.21.15.88:0/1396342991 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9380aa920 con 0x7fc93c160cc0 2023-10-16T20:07:55.701 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.696+0000 7fc943c79640 1 -- 172.21.15.88:0/1396342991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc93c160cc0 msgr2=0x7fc93c0ca800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:55.702 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.696+0000 7fc943c79640 1 --2- 172.21.15.88:0/1396342991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc93c160cc0 0x7fc93c0ca800 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0cf570 tx=0x7fc93801c4d0 comp rx=0 tx=0).stop 2023-10-16T20:07:55.702 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.696+0000 7fc943c79640 1 -- 172.21.15.88:0/1396342991 shutdown_connections 2023-10-16T20:07:55.702 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.696+0000 7fc943c79640 1 --2- 172.21.15.88:0/1396342991 >> v2:172.21.15.88:3300/0 conn(0x7fc93c0cd610 0x7fc93c0cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:55.703 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.696+0000 7fc943c79640 1 --2- 172.21.15.88:0/1396342991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc93c160cc0 0x7fc93c0ca800 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:55.703 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.696+0000 7fc943c79640 1 -- 172.21.15.88:0/1396342991 >> 172.21.15.88:0/1396342991 conn(0x7fc93c0ce5e0 msgr2=0x7fc93c0ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:55.703 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.696+0000 7fc943c79640 1 -- 172.21.15.88:0/1396342991 shutdown_connections 2023-10-16T20:07:55.703 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.696+0000 7fc943c79640 1 -- 172.21.15.88:0/1396342991 wait complete. 2023-10-16T20:07:55.706 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.696+0000 7fc943c79640 1 Processor -- start 2023-10-16T20:07:55.706 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.696+0000 7fc943c79640 1 -- start start 2023-10-16T20:07:55.706 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.696+0000 7fc943c79640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc93c0cd610 0x7fc93c157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:55.706 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.696+0000 7fc943c79640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc93c160cc0 0x7fc93c0b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:55.707 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.696+0000 7fc943c79640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc93c1610a0 0x7fc93c153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:55.707 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.696+0000 7fc943c79640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c158f80 con 0x7fc93c0cd610 2023-10-16T20:07:55.707 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.696+0000 7fc943c79640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c157f00 con 0x7fc93c1610a0 2023-10-16T20:07:55.707 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.696+0000 7fc943c79640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c1581e0 con 0x7fc93c160cc0 2023-10-16T20:07:55.707 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.696+0000 7fc941a15640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc93c0cd610 0x7fc93c157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:55.708 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.696+0000 7fc941a15640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc93c0cd610 0x7fc93c157600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57894/0 (socket says 172.21.15.88:57894) 2023-10-16T20:07:55.708 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.696+0000 7fc941a15640 1 -- 172.21.15.88:0/3150880557 learned_addr learned my addr 172.21.15.88:0/3150880557 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:55.708 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.696+0000 7fc941214640 1 --2- 172.21.15.88:0/3150880557 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc93c160cc0 0x7fc93c0b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:55.708 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.696+0000 7fc942216640 1 --2- 172.21.15.88:0/3150880557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc93c1610a0 0x7fc93c153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:55.708 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.700+0000 7fc941a15640 1 -- 172.21.15.88:0/3150880557 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc93c160cc0 msgr2=0x7fc93c0b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:55.708 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.700+0000 7fc941a15640 1 --2- 172.21.15.88:0/3150880557 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc93c160cc0 0x7fc93c0b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:55.709 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.700+0000 7fc941a15640 1 -- 172.21.15.88:0/3150880557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc93c1610a0 msgr2=0x7fc93c153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:55.709 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.700+0000 7fc941a15640 1 --2- 172.21.15.88:0/3150880557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc93c1610a0 0x7fc93c153a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:55.709 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.700+0000 7fc941a15640 1 -- 172.21.15.88:0/3150880557 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9380094d0 con 0x7fc93c0cd610 2023-10-16T20:07:55.709 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.700+0000 7fc942216640 1 --2- 172.21.15.88:0/3150880557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc93c1610a0 0x7fc93c153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:07:55.709 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.700+0000 7fc941a15640 1 --2- 172.21.15.88:0/3150880557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc93c0cd610 0x7fc93c157600 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fc9380190e0 tx=0x7fc93801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:55.710 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.700+0000 7fc9327fc640 1 -- 172.21.15.88:0/3150880557 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9380aa430 con 0x7fc93c0cd610 2023-10-16T20:07:55.710 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.700+0000 7fc9327fc640 1 -- 172.21.15.88:0/3150880557 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9380b3800 con 0x7fc93c0cd610 2023-10-16T20:07:55.710 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.700+0000 7fc9327fc640 1 -- 172.21.15.88:0/3150880557 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9380b3bb0 con 0x7fc93c0cd610 2023-10-16T20:07:55.710 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.700+0000 7fc943c79640 1 -- 172.21.15.88:0/3150880557 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc93c1584c0 con 0x7fc93c0cd610 2023-10-16T20:07:55.710 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.700+0000 7fc943c79640 1 -- 172.21.15.88:0/3150880557 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc93c158a80 con 0x7fc93c0cd610 2023-10-16T20:07:55.711 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.700+0000 7fc943c79640 1 -- 172.21.15.88:0/3150880557 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc93c04e9c0 con 0x7fc93c0cd610 2023-10-16T20:07:55.711 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.700+0000 7fc9327fc640 1 -- 172.21.15.88:0/3150880557 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc9380b3d10 con 0x7fc93c0cd610 2023-10-16T20:07:55.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.700+0000 7fc9327fc640 1 --2- 172.21.15.88:0/3150880557 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc9200664a0 0x7fc920068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:55.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.700+0000 7fc9327fc640 1 -- 172.21.15.88:0/3150880557 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fc93811e4b0 con 0x7fc93c0cd610 2023-10-16T20:07:55.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.700+0000 7fc941214640 1 --2- 172.21.15.88:0/3150880557 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc9200664a0 0x7fc920068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:55.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.704+0000 7fc941214640 1 --2- 172.21.15.88:0/3150880557 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc9200664a0 0x7fc920068960 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc93c162810 tx=0x7fc92c007e50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:55.712 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.704+0000 7fc9327fc640 1 -- 172.21.15.88:0/3150880557 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc9380eb7e0 con 0x7fc93c0cd610 2023-10-16T20:07:55.861 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.856+0000 7fc943c79640 1 -- 172.21.15.88:0/3150880557 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc93c06ad70 con 0x7fc93c0cd610 2023-10-16T20:07:55.862 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.856+0000 7fc9327fc640 1 -- 172.21.15.88:0/3150880557 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16718 (secure 0 0 0) 0x7fc9380eee00 con 0x7fc93c0cd610 2023-10-16T20:07:55.862 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:07:55.863 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":58,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:07:27.206614+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T20:07:24.001121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"testcephfs1_data","create_time":"2023-10-16T20:07:24.636209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:07:55.864 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.860+0000 7fc943c79640 1 -- 172.21.15.88:0/3150880557 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc9200664a0 msgr2=0x7fc920068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:55.864 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.860+0000 7fc943c79640 1 --2- 172.21.15.88:0/3150880557 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc9200664a0 0x7fc920068960 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc93c162810 tx=0x7fc92c007e50 comp rx=0 tx=0).stop 2023-10-16T20:07:55.864 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.860+0000 7fc943c79640 1 -- 172.21.15.88:0/3150880557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc93c0cd610 msgr2=0x7fc93c157600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:55.864 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.860+0000 7fc943c79640 1 --2- 172.21.15.88:0/3150880557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc93c0cd610 0x7fc93c157600 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fc9380190e0 tx=0x7fc93801ad10 comp rx=0 tx=0).stop 2023-10-16T20:07:55.865 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.860+0000 7fc943c79640 1 -- 172.21.15.88:0/3150880557 shutdown_connections 2023-10-16T20:07:55.865 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.860+0000 7fc943c79640 1 --2- 172.21.15.88:0/3150880557 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fc9200664a0 0x7fc920068960 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:55.865 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.860+0000 7fc943c79640 1 --2- 172.21.15.88:0/3150880557 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc93c160cc0 0x7fc93c0b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:55.865 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.860+0000 7fc943c79640 1 --2- 172.21.15.88:0/3150880557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc93c1610a0 0x7fc93c153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:55.865 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.860+0000 7fc943c79640 1 --2- 172.21.15.88:0/3150880557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc93c0cd610 0x7fc93c157600 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:55.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.860+0000 7fc943c79640 1 -- 172.21.15.88:0/3150880557 >> 172.21.15.88:0/3150880557 conn(0x7fc93c0ce5e0 msgr2=0x7fc93c0cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:55.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.860+0000 7fc943c79640 1 -- 172.21.15.88:0/3150880557 shutdown_connections 2023-10-16T20:07:55.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:55.860+0000 7fc943c79640 1 -- 172.21.15.88:0/3150880557 wait complete. 2023-10-16T20:07:55.883 INFO:tasks.cephfs.filesystem:Destroying file system testcephfs1 and related pools 2023-10-16T20:07:55.884 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T20:07:56.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.016+0000 7feb65809640 1 Processor -- start 2023-10-16T20:07:56.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.016+0000 7feb65809640 1 -- start start 2023-10-16T20:07:56.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.016+0000 7feb65809640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7feb60160040 0x7feb60071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:56.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.016+0000 7feb65809640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feb6006a5a0 0x7feb600c3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:56.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.016+0000 7feb65809640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb600cd7d0 con 0x7feb60160420 2023-10-16T20:07:56.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.016+0000 7feb65809640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb600cdab0 con 0x7feb60160040 2023-10-16T20:07:56.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.016+0000 7feb65809640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb600cdd90 con 0x7feb6006a5a0 2023-10-16T20:07:56.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.016+0000 7feb5effd640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7feb60160040 0x7feb60071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:56.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.016+0000 7feb567fc640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7feb60160420 0x7feb60069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:56416/0 (socket says 172.21.15.88:56416) 2023-10-16T20:07:56.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.016+0000 7feb567fc640 1 -- 172.21.15.88:0/329595878 learned_addr learned my addr 172.21.15.88:0/329595878 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:56.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.016+0000 7feb5f7fe640 1 --2- 172.21.15.88:0/329595878 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feb6006a5a0 0x7feb600c3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:56.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.016+0000 7feb5effd640 1 -- 172.21.15.88:0/329595878 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feb6006a5a0 msgr2=0x7feb600c3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:56.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.016+0000 7feb5effd640 1 --2- 172.21.15.88:0/329595878 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feb6006a5a0 0x7feb600c3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.016+0000 7feb5effd640 1 -- 172.21.15.88:0/329595878 >> v1:172.21.15.88:6789/0 conn(0x7feb60160420 legacy=0x7feb60069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:56.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.016+0000 7feb5effd640 1 -- 172.21.15.88:0/329595878 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb600ce070 con 0x7feb60160040 2023-10-16T20:07:56.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.016+0000 7feb5e7fc640 1 -- 172.21.15.88:0/329595878 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4070328434 0 0) 0x7feb600cd7d0 con 0x7feb60160420 2023-10-16T20:07:56.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.016+0000 7feb5f7fe640 1 --2- 172.21.15.88:0/329595878 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feb6006a5a0 0x7feb600c3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:07:56.026 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.020+0000 7feb5effd640 1 --2- 172.21.15.88:0/329595878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb60160040 0x7feb60071f10 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7feb50009d90 tx=0x7feb5001c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5442ee249ee652 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:56.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.020+0000 7feb5e7fc640 1 -- 172.21.15.88:0/329595878 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb500a5970 con 0x7feb60160040 2023-10-16T20:07:56.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.020+0000 7feb5e7fc640 1 -- 172.21.15.88:0/329595878 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb500a5ad0 con 0x7feb60160040 2023-10-16T20:07:56.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.020+0000 7feb5e7fc640 1 -- 172.21.15.88:0/329595878 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb500a5e60 con 0x7feb60160040 2023-10-16T20:07:56.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.020+0000 7feb65809640 1 -- 172.21.15.88:0/329595878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb60160040 msgr2=0x7feb60071f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:56.027 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.020+0000 7feb65809640 1 --2- 172.21.15.88:0/329595878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb60160040 0x7feb60071f10 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7feb50009d90 tx=0x7feb5001c8f0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.020+0000 7feb65809640 1 -- 172.21.15.88:0/329595878 shutdown_connections 2023-10-16T20:07:56.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.020+0000 7feb65809640 1 --2- 172.21.15.88:0/329595878 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feb6006a5a0 0x7feb600c3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.020+0000 7feb65809640 1 --2- 172.21.15.88:0/329595878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb60160040 0x7feb60071f10 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.020+0000 7feb65809640 1 -- 172.21.15.88:0/329595878 >> 172.21.15.88:0/329595878 conn(0x7feb6006d340 msgr2=0x7feb6006d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:56.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.020+0000 7feb65809640 1 -- 172.21.15.88:0/329595878 shutdown_connections 2023-10-16T20:07:56.028 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.020+0000 7feb65809640 1 -- 172.21.15.88:0/329595878 wait complete. 2023-10-16T20:07:56.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.024+0000 7feb65809640 1 Processor -- start 2023-10-16T20:07:56.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.024+0000 7feb65809640 1 -- start start 2023-10-16T20:07:56.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.024+0000 7feb65809640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feb6006a5a0 0x7feb600c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:56.032 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.024+0000 7feb65809640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb60160040 0x7feb600c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:56.032 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.024+0000 7feb65809640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb60160420 0x7feb600cc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:56.032 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.024+0000 7feb65809640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb600c60f0 con 0x7feb60160420 2023-10-16T20:07:56.032 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.024+0000 7feb65809640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb600c8810 con 0x7feb60160040 2023-10-16T20:07:56.032 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.024+0000 7feb65809640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb600c5470 con 0x7feb6006a5a0 2023-10-16T20:07:56.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.024+0000 7feb5f7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb60160420 0x7feb600cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:56.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.024+0000 7feb5f7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb60160420 0x7feb600cc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57918/0 (socket says 172.21.15.88:57918) 2023-10-16T20:07:56.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.024+0000 7feb5f7fe640 1 -- 172.21.15.88:0/1792589525 learned_addr learned my addr 172.21.15.88:0/1792589525 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:56.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.024+0000 7feb5effd640 1 --2- 172.21.15.88:0/1792589525 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feb6006a5a0 0x7feb600c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:56.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.024+0000 7feb567fc640 1 --2- 172.21.15.88:0/1792589525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb60160040 0x7feb600c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:56.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.024+0000 7feb5f7fe640 1 -- 172.21.15.88:0/1792589525 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feb6006a5a0 msgr2=0x7feb600c4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:56.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.024+0000 7feb5f7fe640 1 --2- 172.21.15.88:0/1792589525 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feb6006a5a0 0x7feb600c4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.024+0000 7feb5f7fe640 1 -- 172.21.15.88:0/1792589525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb60160040 msgr2=0x7feb600c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:56.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.024+0000 7feb5f7fe640 1 --2- 172.21.15.88:0/1792589525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb60160040 0x7feb600c82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.024+0000 7feb5f7fe640 1 -- 172.21.15.88:0/1792589525 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb50009a70 con 0x7feb60160420 2023-10-16T20:07:56.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.024+0000 7feb5f7fe640 1 --2- 172.21.15.88:0/1792589525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb60160420 0x7feb600cc750 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7feb5800d800 tx=0x7feb5800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:56.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.024+0000 7feb5cff9640 1 -- 172.21.15.88:0/1792589525 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb580096a0 con 0x7feb60160420 2023-10-16T20:07:56.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.024+0000 7feb5cff9640 1 -- 172.21.15.88:0/1792589525 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb58009800 con 0x7feb60160420 2023-10-16T20:07:56.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.024+0000 7feb65809640 1 -- 172.21.15.88:0/1792589525 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb600c5750 con 0x7feb60160420 2023-10-16T20:07:56.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.024+0000 7feb5cff9640 1 -- 172.21.15.88:0/1792589525 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb58009bb0 con 0x7feb60160420 2023-10-16T20:07:56.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.024+0000 7feb65809640 1 -- 172.21.15.88:0/1792589525 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb600c5d10 con 0x7feb60160420 2023-10-16T20:07:56.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.028+0000 7feb65809640 1 -- 172.21.15.88:0/1792589525 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb600c6230 con 0x7feb60160420 2023-10-16T20:07:56.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.028+0000 7feb5cff9640 1 -- 172.21.15.88:0/1792589525 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7feb58009d10 con 0x7feb60160420 2023-10-16T20:07:56.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.028+0000 7feb5cff9640 1 --2- 172.21.15.88:0/1792589525 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7feb400664a0 0x7feb40068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:56.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.028+0000 7feb5cff9640 1 -- 172.21.15.88:0/1792589525 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7feb580842f0 con 0x7feb60160420 2023-10-16T20:07:56.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.028+0000 7feb5effd640 1 --2- 172.21.15.88:0/1792589525 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7feb400664a0 0x7feb40068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:56.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.028+0000 7feb5effd640 1 --2- 172.21.15.88:0/1792589525 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7feb400664a0 0x7feb40068960 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7feb500af000 tx=0x7feb500a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:56.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.032+0000 7feb5cff9640 1 -- 172.21.15.88:0/1792589525 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7feb600c6230 con 0x7feb60160420 2023-10-16T20:07:56.217 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.212+0000 7feb65809640 1 -- 172.21.15.88:0/1792589525 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb600c6230 con 0x7feb60160420 2023-10-16T20:07:56.218 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.212+0000 7feb5cff9640 1 -- 172.21.15.88:0/1792589525 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4918 (secure 0 0 0) 0x7feb600c6230 con 0x7feb60160420 2023-10-16T20:07:56.218 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:07:56.219 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5531,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/2485638574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":2485638574},{"type":"v1","addr":"172.21.15.88:6837","nonce":2485638574}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5534,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3821390040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3821390040},{"type":"v1","addr":"172.21.15.94:6834","nonce":3821390040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5968,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2544673478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":2544673478},{"type":"v1","addr":"172.21.15.94:6835","nonce":2544673478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T20:07:25.650705+0000","modified":"2023-10-16T20:07:28.651868+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6210":{"gid":6210,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6836/1018125900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":1018125900},{"type":"v1","addr":"172.21.15.88:6836","nonce":1018125900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"testcephfs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-16T20:07:56.220 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 28 2023-10-16T20:07:56.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.212+0000 7feb65809640 1 -- 172.21.15.88:0/1792589525 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7feb400664a0 msgr2=0x7feb40068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:56.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.212+0000 7feb65809640 1 --2- 172.21.15.88:0/1792589525 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7feb400664a0 0x7feb40068960 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7feb500af000 tx=0x7feb500a7070 comp rx=0 tx=0).stop 2023-10-16T20:07:56.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.212+0000 7feb65809640 1 -- 172.21.15.88:0/1792589525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb60160420 msgr2=0x7feb600cc750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:56.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.212+0000 7feb65809640 1 --2- 172.21.15.88:0/1792589525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb60160420 0x7feb600cc750 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7feb5800d800 tx=0x7feb5800dcc0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.212+0000 7feb65809640 1 -- 172.21.15.88:0/1792589525 shutdown_connections 2023-10-16T20:07:56.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.212+0000 7feb65809640 1 --2- 172.21.15.88:0/1792589525 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7feb400664a0 0x7feb40068960 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.212+0000 7feb65809640 1 --2- 172.21.15.88:0/1792589525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb60160420 0x7feb600cc750 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.212+0000 7feb65809640 1 --2- 172.21.15.88:0/1792589525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb60160040 0x7feb600c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.222 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.212+0000 7feb65809640 1 --2- 172.21.15.88:0/1792589525 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feb6006a5a0 0x7feb600c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.222 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.212+0000 7feb65809640 1 -- 172.21.15.88:0/1792589525 >> 172.21.15.88:0/1792589525 conn(0x7feb6006d340 msgr2=0x7feb600ae330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:56.222 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.212+0000 7feb65809640 1 -- 172.21.15.88:0/1792589525 shutdown_connections 2023-10-16T20:07:56.222 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.212+0000 7feb65809640 1 -- 172.21.15.88:0/1792589525 wait complete. 2023-10-16T20:07:56.239 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T20:07:56.376 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f084d6ad640 1 Processor -- start 2023-10-16T20:07:56.376 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f084d6ad640 1 -- start start 2023-10-16T20:07:56.377 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f084d6ad640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f08480cde80 0x7f08480d3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:56.377 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f084d6ad640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f08480cf230 0x7f08480cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:56.377 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f084d6ad640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08480d0c40 con 0x7f084815c9a0 2023-10-16T20:07:56.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f084d6ad640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08480d37b0 con 0x7f08480cf230 2023-10-16T20:07:56.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f084d6ad640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08480ce7e0 con 0x7f08480cde80 2023-10-16T20:07:56.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f0847fff640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f084815c9a0 0x7f08480d10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:56430/0 (socket says 172.21.15.88:56430) 2023-10-16T20:07:56.378 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f0847fff640 1 -- 172.21.15.88:0/143568772 learned_addr learned my addr 172.21.15.88:0/143568772 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:56.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f084ceac640 1 --2- 172.21.15.88:0/143568772 >> v2:172.21.15.94:3300/0 conn(0x7f08480cf230 0x7f08480cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:56.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f08477fe640 1 --2- 172.21.15.88:0/143568772 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f08480cde80 0x7f08480d3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:56.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f0846ffd640 1 -- 172.21.15.88:0/143568772 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 175817010 0 0) 0x7f08480d0c40 con 0x7f084815c9a0 2023-10-16T20:07:56.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f0846ffd640 1 -- 172.21.15.88:0/143568772 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0830003530 con 0x7f084815c9a0 2023-10-16T20:07:56.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f0846ffd640 1 -- 172.21.15.88:0/143568772 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2021008342 0 0) 0x7f0830003530 con 0x7f084815c9a0 2023-10-16T20:07:56.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f0846ffd640 1 -- 172.21.15.88:0/143568772 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f08480cde80 msgr2=0x7f08480d3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:56.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f0846ffd640 1 --2- 172.21.15.88:0/143568772 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f08480cde80 0x7f08480d3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f0846ffd640 1 -- 172.21.15.88:0/143568772 >> v2:172.21.15.94:3300/0 conn(0x7f08480cf230 msgr2=0x7f08480cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:56.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f0846ffd640 1 --2- 172.21.15.88:0/143568772 >> v2:172.21.15.94:3300/0 conn(0x7f08480cf230 0x7f08480cf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f0846ffd640 1 -- 172.21.15.88:0/143568772 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08480ce2a0 con 0x7f084815c9a0 2023-10-16T20:07:56.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f0846ffd640 1 -- 172.21.15.88:0/143568772 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f0838001800 con 0x7f084815c9a0 2023-10-16T20:07:56.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f08477fe640 1 --2- 172.21.15.88:0/143568772 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f08480cde80 0x7f08480d3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:07:56.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f084ceac640 1 --2- 172.21.15.88:0/143568772 >> v2:172.21.15.94:3300/0 conn(0x7f08480cf230 0x7f08480cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:07:56.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f0846ffd640 1 -- 172.21.15.88:0/143568772 >> v1:172.21.15.88:6789/0 conn(0x7f084815c9a0 legacy=0x7f08480d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:56.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f0846ffd640 1 --2- 172.21.15.88:0/143568772 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f08480cde80 0x7f0830003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:56.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f0846ffd640 1 --2- 172.21.15.88:0/143568772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0830003ec0 0x7f08300047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:56.384 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f0846ffd640 1 --2- 172.21.15.88:0/143568772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0830004d10 0x7f0830005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:56.384 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f0846ffd640 1 -- 172.21.15.88:0/143568772 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08480d37b0 con 0x7f0830003ec0 2023-10-16T20:07:56.384 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f0846ffd640 1 -- 172.21.15.88:0/143568772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08480ce7e0 con 0x7f0830004d10 2023-10-16T20:07:56.384 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f0846ffd640 1 -- 172.21.15.88:0/143568772 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f083000b990 con 0x7f08480cde80 2023-10-16T20:07:56.384 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f08477fe640 1 --2- 172.21.15.88:0/143568772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0830003ec0 0x7f08300047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:56.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f0847fff640 1 --2- 172.21.15.88:0/143568772 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f08480cde80 0x7f0830003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:56.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f084ceac640 1 --2- 172.21.15.88:0/143568772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0830004d10 0x7f0830005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:56.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f08477fe640 1 -- 172.21.15.88:0/143568772 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f08480cde80 msgr2=0x7f0830003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:56.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f08477fe640 1 --2- 172.21.15.88:0/143568772 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f08480cde80 0x7f0830003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f08477fe640 1 -- 172.21.15.88:0/143568772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0830004d10 msgr2=0x7f0830005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:56.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f08477fe640 1 --2- 172.21.15.88:0/143568772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0830004d10 0x7f0830005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.372+0000 7f08477fe640 1 -- 172.21.15.88:0/143568772 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f083c009420 con 0x7f0830003ec0 2023-10-16T20:07:56.389 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.376+0000 7f08477fe640 1 --2- 172.21.15.88:0/143568772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0830003ec0 0x7f08300047d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f083c009cb0 tx=0x7f083c01cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf954e4cc8427f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:56.389 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.376+0000 7f0846ffd640 1 -- 172.21.15.88:0/143568772 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083c0a57a0 con 0x7f0830003ec0 2023-10-16T20:07:56.389 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.376+0000 7f0846ffd640 1 -- 172.21.15.88:0/143568772 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f083c0a5900 con 0x7f0830003ec0 2023-10-16T20:07:56.389 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.376+0000 7f0846ffd640 1 -- 172.21.15.88:0/143568772 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083c0a5cf0 con 0x7f0830003ec0 2023-10-16T20:07:56.389 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.376+0000 7f084d6ad640 1 -- 172.21.15.88:0/143568772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0830003ec0 msgr2=0x7f08300047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:56.390 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.376+0000 7f084d6ad640 1 --2- 172.21.15.88:0/143568772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0830003ec0 0x7f08300047d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f083c009cb0 tx=0x7f083c01cbf0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.390 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.376+0000 7f084ceac640 1 -- 172.21.15.88:0/143568772 reap_dead start 2023-10-16T20:07:56.390 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.376+0000 7f084d6ad640 1 -- 172.21.15.88:0/143568772 shutdown_connections 2023-10-16T20:07:56.390 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.376+0000 7f084d6ad640 1 -- 172.21.15.88:0/143568772 >> 172.21.15.88:0/143568772 conn(0x7f084809bc30 msgr2=0x7f08480d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:56.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.376+0000 7f084d6ad640 1 -- 172.21.15.88:0/143568772 shutdown_connections 2023-10-16T20:07:56.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.376+0000 7f084d6ad640 1 -- 172.21.15.88:0/143568772 wait complete. 2023-10-16T20:07:56.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.380+0000 7f084d6ad640 1 Processor -- start 2023-10-16T20:07:56.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.380+0000 7f084d6ad640 1 -- start start 2023-10-16T20:07:56.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.380+0000 7f084d6ad640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0830003ec0 0x7f08481f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:56.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.380+0000 7f084d6ad640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0848148820 0x7f08480c66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:56.392 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.380+0000 7f084d6ad640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08480c6c00 0x7f084814a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:56.392 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.380+0000 7f084d6ad640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08481f5060 con 0x7f08480c6c00 2023-10-16T20:07:56.392 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.380+0000 7f084d6ad640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08480b6660 con 0x7f0848148820 2023-10-16T20:07:56.392 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.380+0000 7f084d6ad640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08480b6940 con 0x7f0830003ec0 2023-10-16T20:07:56.393 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.380+0000 7f084ceac640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08480c6c00 0x7f084814a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:56.393 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.380+0000 7f084ceac640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08480c6c00 0x7f084814a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57930/0 (socket says 172.21.15.88:57930) 2023-10-16T20:07:56.393 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.380+0000 7f084ceac640 1 -- 172.21.15.88:0/910012017 learned_addr learned my addr 172.21.15.88:0/910012017 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:56.393 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.380+0000 7f0847fff640 1 --2- 172.21.15.88:0/910012017 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0830003ec0 0x7f08481f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:56.393 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.380+0000 7f08477fe640 1 --2- 172.21.15.88:0/910012017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0848148820 0x7f08480c66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:56.394 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.380+0000 7f084ceac640 1 -- 172.21.15.88:0/910012017 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0830003ec0 msgr2=0x7f08481f4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:56.394 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.380+0000 7f084ceac640 1 --2- 172.21.15.88:0/910012017 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0830003ec0 0x7f08481f4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.394 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.380+0000 7f084ceac640 1 -- 172.21.15.88:0/910012017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0848148820 msgr2=0x7f08480c66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:56.394 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.380+0000 7f084ceac640 1 --2- 172.21.15.88:0/910012017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0848148820 0x7f08480c66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.394 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.380+0000 7f084ceac640 1 -- 172.21.15.88:0/910012017 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08480b6c20 con 0x7f08480c6c00 2023-10-16T20:07:56.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.380+0000 7f084ceac640 1 --2- 172.21.15.88:0/910012017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08480c6c00 0x7f084814a800 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f084808ce90 tx=0x7f084000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:56.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.384+0000 7f0844ff9640 1 -- 172.21.15.88:0/910012017 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084000e040 con 0x7f08480c6c00 2023-10-16T20:07:56.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.384+0000 7f0844ff9640 1 -- 172.21.15.88:0/910012017 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f084000f040 con 0x7f08480c6c00 2023-10-16T20:07:56.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.384+0000 7f084d6ad640 1 -- 172.21.15.88:0/910012017 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08480b6f00 con 0x7f08480c6c00 2023-10-16T20:07:56.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.384+0000 7f0844ff9640 1 -- 172.21.15.88:0/910012017 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0840003be0 con 0x7f08480c6c00 2023-10-16T20:07:56.396 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.384+0000 7f084d6ad640 1 -- 172.21.15.88:0/910012017 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08480c9b50 con 0x7f08480c6c00 2023-10-16T20:07:56.396 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.384+0000 7f084d6ad640 1 -- 172.21.15.88:0/910012017 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08480cfb60 con 0x7f08480c6c00 2023-10-16T20:07:56.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.384+0000 7f0844ff9640 1 -- 172.21.15.88:0/910012017 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0840015430 con 0x7f08480c6c00 2023-10-16T20:07:56.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.384+0000 7f0844ff9640 1 --2- 172.21.15.88:0/910012017 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f08340664d0 0x7f0834068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:56.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.384+0000 7f0844ff9640 1 -- 172.21.15.88:0/910012017 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f08400848d0 con 0x7f08480c6c00 2023-10-16T20:07:56.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.384+0000 7f0847fff640 1 --2- 172.21.15.88:0/910012017 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f08340664d0 0x7f0834068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:56.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.388+0000 7f0847fff640 1 --2- 172.21.15.88:0/910012017 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f08340664d0 0x7f0834068990 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0838001de0 tx=0x7f08380078d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:56.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.388+0000 7f0844ff9640 1 -- 172.21.15.88:0/910012017 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0840051c90 con 0x7f08480c6c00 2023-10-16T20:07:56.577 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.572+0000 7f084d6ad640 1 -- 172.21.15.88:0/910012017 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08480674f0 con 0x7f08480c6c00 2023-10-16T20:07:56.578 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.572+0000 7f0844ff9640 1 -- 172.21.15.88:0/910012017 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f08400552b0 con 0x7f08480c6c00 2023-10-16T20:07:56.578 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:07:56.578 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5531,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/2485638574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":2485638574},{"type":"v1","addr":"172.21.15.88:6837","nonce":2485638574}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5534,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3821390040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3821390040},{"type":"v1","addr":"172.21.15.94:6834","nonce":3821390040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5968,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2544673478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":2544673478},{"type":"v1","addr":"172.21.15.94:6835","nonce":2544673478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T20:07:25.650705+0000","modified":"2023-10-16T20:07:28.651868+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6210":{"gid":6210,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6836/1018125900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":1018125900},{"type":"v1","addr":"172.21.15.88:6836","nonce":1018125900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"testcephfs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-16T20:07:56.579 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 28 2023-10-16T20:07:56.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.572+0000 7f084d6ad640 1 -- 172.21.15.88:0/910012017 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f08340664d0 msgr2=0x7f0834068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:56.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.572+0000 7f084d6ad640 1 --2- 172.21.15.88:0/910012017 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f08340664d0 0x7f0834068990 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0838001de0 tx=0x7f08380078d0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.572+0000 7f084d6ad640 1 -- 172.21.15.88:0/910012017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08480c6c00 msgr2=0x7f084814a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:56.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.572+0000 7f084d6ad640 1 --2- 172.21.15.88:0/910012017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08480c6c00 0x7f084814a800 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f084808ce90 tx=0x7f084000f580 comp rx=0 tx=0).stop 2023-10-16T20:07:56.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.572+0000 7f084d6ad640 1 -- 172.21.15.88:0/910012017 shutdown_connections 2023-10-16T20:07:56.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.572+0000 7f084d6ad640 1 --2- 172.21.15.88:0/910012017 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f08340664d0 0x7f0834068990 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.572+0000 7f084d6ad640 1 --2- 172.21.15.88:0/910012017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08480c6c00 0x7f084814a800 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.572+0000 7f084d6ad640 1 --2- 172.21.15.88:0/910012017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0848148820 0x7f08480c66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.572+0000 7f084d6ad640 1 --2- 172.21.15.88:0/910012017 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0830003ec0 0x7f08481f4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.572+0000 7f084d6ad640 1 -- 172.21.15.88:0/910012017 >> 172.21.15.88:0/910012017 conn(0x7f084809bc30 msgr2=0x7f0848159ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:56.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.572+0000 7f084d6ad640 1 -- 172.21.15.88:0/910012017 shutdown_connections 2023-10-16T20:07:56.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.576+0000 7f084d6ad640 1 -- 172.21.15.88:0/910012017 wait complete. 2023-10-16T20:07:56.599 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:07:56.736 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.732+0000 7fbd2f3b5640 1 Processor -- start 2023-10-16T20:07:56.736 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.732+0000 7fbd2f3b5640 1 -- start start 2023-10-16T20:07:56.738 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.732+0000 7fbd2f3b5640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbd2815c9a0 0x7fbd280cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:56.738 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.732+0000 7fbd2f3b5640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fbd280d3270 0x7fbd280d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:56.738 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.732+0000 7fbd2f3b5640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd280d0c40 con 0x7fbd280d1610 2023-10-16T20:07:56.738 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.732+0000 7fbd2f3b5640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd280ce7e0 con 0x7fbd280d3270 2023-10-16T20:07:56.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.732+0000 7fbd2f3b5640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd280d0630 con 0x7fbd2815c9a0 2023-10-16T20:07:56.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.732+0000 7fbd2dbb2640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fbd280d3270 0x7fbd280d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:56.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.732+0000 7fbd2dbb2640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fbd280d3270 0x7fbd280d10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57934/0 (socket says 172.21.15.88:57934) 2023-10-16T20:07:56.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.732+0000 7fbd2dbb2640 1 -- 172.21.15.88:0/1517417934 learned_addr learned my addr 172.21.15.88:0/1517417934 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:56.739 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.732+0000 7fbd2e3b3640 1 --2- 172.21.15.88:0/1517417934 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbd2815c9a0 0x7fbd280cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:56.740 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.732+0000 7fbd2dbb2640 1 -- 172.21.15.88:0/1517417934 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbd2815c9a0 msgr2=0x7fbd280cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:56.740 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.732+0000 7fbd2dbb2640 1 --2- 172.21.15.88:0/1517417934 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbd2815c9a0 0x7fbd280cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.740 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.732+0000 7fbd2dbb2640 1 -- 172.21.15.88:0/1517417934 >> v1:172.21.15.94:6789/0 conn(0x7fbd280d1610 legacy=0x7fbd280cf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:56.740 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.732+0000 7fbd2dbb2640 1 -- 172.21.15.88:0/1517417934 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd280ce520 con 0x7fbd280d3270 2023-10-16T20:07:56.741 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T20:07:56.742 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.736+0000 7fbd2dbb2640 1 --2- 172.21.15.88:0/1517417934 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd280d3270 0x7fbd280d10d0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fbd18009870 tx=0x7fbd1801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91e8f1a8612a7d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:56.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.736+0000 7fbd2d3b1640 1 -- 172.21.15.88:0/1517417934 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd180aa610 con 0x7fbd280d3270 2023-10-16T20:07:56.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.736+0000 7fbd2d3b1640 1 -- 172.21.15.88:0/1517417934 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd180a6040 con 0x7fbd280d3270 2023-10-16T20:07:56.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.736+0000 7fbd2d3b1640 1 -- 172.21.15.88:0/1517417934 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd180aa990 con 0x7fbd280d3270 2023-10-16T20:07:56.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.736+0000 7fbd2f3b5640 1 -- 172.21.15.88:0/1517417934 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd280d3270 msgr2=0x7fbd280d10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:56.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.736+0000 7fbd2f3b5640 1 --2- 172.21.15.88:0/1517417934 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd280d3270 0x7fbd280d10d0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fbd18009870 tx=0x7fbd1801c510 comp rx=0 tx=0).stop 2023-10-16T20:07:56.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.736+0000 7fbd2f3b5640 1 -- 172.21.15.88:0/1517417934 shutdown_connections 2023-10-16T20:07:56.744 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.736+0000 7fbd2f3b5640 1 --2- 172.21.15.88:0/1517417934 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd280d3270 0x7fbd280d10d0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.744 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.736+0000 7fbd2f3b5640 1 --2- 172.21.15.88:0/1517417934 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbd2815c9a0 0x7fbd280cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.744 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.736+0000 7fbd2f3b5640 1 -- 172.21.15.88:0/1517417934 >> 172.21.15.88:0/1517417934 conn(0x7fbd2809bc30 msgr2=0x7fbd280d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:56.744 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.736+0000 7fbd2f3b5640 1 -- 172.21.15.88:0/1517417934 shutdown_connections 2023-10-16T20:07:56.744 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.736+0000 7fbd2f3b5640 1 -- 172.21.15.88:0/1517417934 wait complete. 2023-10-16T20:07:56.746 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T20:07:56.748 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.740+0000 7fbd2f3b5640 1 Processor -- start 2023-10-16T20:07:56.748 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.740+0000 7fbd2f3b5640 1 -- start start 2023-10-16T20:07:56.748 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.740+0000 7fbd2f3b5640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbd280d1610 0x7fbd28158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:56.748 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.740+0000 7fbd2f3b5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd280d3270 0x7fbd2814e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:56.749 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.740+0000 7fbd2f3b5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd2815c9a0 0x7fbd2806bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:56.749 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.740+0000 7fbd2f3b5640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd28158e90 con 0x7fbd2815c9a0 2023-10-16T20:07:56.749 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.740+0000 7fbd2f3b5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd28150950 con 0x7fbd280d3270 2023-10-16T20:07:56.749 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.740+0000 7fbd2f3b5640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2814f8b0 con 0x7fbd280d1610 2023-10-16T20:07:56.749 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.740+0000 7fbd2ebb4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd2815c9a0 0x7fbd2806bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:56.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.740+0000 7fbd2ebb4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd2815c9a0 0x7fbd2806bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57940/0 (socket says 172.21.15.88:57940) 2023-10-16T20:07:56.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.740+0000 7fbd2ebb4640 1 -- 172.21.15.88:0/1316263855 learned_addr learned my addr 172.21.15.88:0/1316263855 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:56.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.740+0000 7fbd2dbb2640 1 --2- 172.21.15.88:0/1316263855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd280d3270 0x7fbd2814e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:56.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.740+0000 7fbd2e3b3640 1 --2- 172.21.15.88:0/1316263855 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbd280d1610 0x7fbd28158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:56.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.740+0000 7fbd2ebb4640 1 -- 172.21.15.88:0/1316263855 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbd280d1610 msgr2=0x7fbd28158950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:56.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.740+0000 7fbd2ebb4640 1 --2- 172.21.15.88:0/1316263855 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbd280d1610 0x7fbd28158950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.740+0000 7fbd2ebb4640 1 -- 172.21.15.88:0/1316263855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd280d3270 msgr2=0x7fbd2814e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:56.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.740+0000 7fbd2ebb4640 1 --2- 172.21.15.88:0/1316263855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd280d3270 0x7fbd2814e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.740+0000 7fbd2ebb4640 1 -- 172.21.15.88:0/1316263855 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd180094d0 con 0x7fbd2815c9a0 2023-10-16T20:07:56.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.740+0000 7fbd2dbb2640 1 --2- 172.21.15.88:0/1316263855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd280d3270 0x7fbd2814e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:07:56.752 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.740+0000 7fbd2ebb4640 1 --2- 172.21.15.88:0/1316263855 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd2815c9a0 0x7fbd2806bcb0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fbd2000c730 tx=0x7fbd2000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:56.752 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.740+0000 7fbd1effd640 1 -- 172.21.15.88:0/1316263855 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd20003b80 con 0x7fbd2815c9a0 2023-10-16T20:07:56.752 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.740+0000 7fbd1effd640 1 -- 172.21.15.88:0/1316263855 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd20003ce0 con 0x7fbd2815c9a0 2023-10-16T20:07:56.752 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.740+0000 7fbd2f3b5640 1 -- 172.21.15.88:0/1316263855 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd2815cf00 con 0x7fbd2815c9a0 2023-10-16T20:07:56.752 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.740+0000 7fbd1effd640 1 -- 172.21.15.88:0/1316263855 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd20005310 con 0x7fbd2815c9a0 2023-10-16T20:07:56.752 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.740+0000 7fbd2f3b5640 1 -- 172.21.15.88:0/1316263855 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd2814fe70 con 0x7fbd2815c9a0 2023-10-16T20:07:56.753 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.744+0000 7fbd2f3b5640 1 -- 172.21.15.88:0/1316263855 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd280674f0 con 0x7fbd2815c9a0 2023-10-16T20:07:56.754 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.744+0000 7fbd1effd640 1 -- 172.21.15.88:0/1316263855 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbd20005680 con 0x7fbd2815c9a0 2023-10-16T20:07:56.754 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.744+0000 7fbd1effd640 1 --2- 172.21.15.88:0/1316263855 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbd0c0664d0 0x7fbd0c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:56.754 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.744+0000 7fbd1effd640 1 -- 172.21.15.88:0/1316263855 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fbd20083df0 con 0x7fbd2815c9a0 2023-10-16T20:07:56.754 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.744+0000 7fbd2e3b3640 1 --2- 172.21.15.88:0/1316263855 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbd0c0664d0 0x7fbd0c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:56.755 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.744+0000 7fbd2e3b3640 1 --2- 172.21.15.88:0/1316263855 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbd0c0664d0 0x7fbd0c068990 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fbd24006fd0 tx=0x7fbd24008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:56.755 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.748+0000 7fbd1effd640 1 -- 172.21.15.88:0/1316263855 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbd200511b0 con 0x7fbd2815c9a0 2023-10-16T20:07:56.907 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.900+0000 7fbd2f3b5640 1 -- 172.21.15.88:0/1316263855 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd280c96c0 con 0x7fbd2815c9a0 2023-10-16T20:07:56.908 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.904+0000 7fbd1effd640 1 -- 172.21.15.88:0/1316263855 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16718 (secure 0 0 0) 0x7fbd200547d0 con 0x7fbd2815c9a0 2023-10-16T20:07:56.909 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:07:56.909 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":58,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:07:27.206614+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T20:07:24.001121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"testcephfs1_data","create_time":"2023-10-16T20:07:24.636209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:07:56.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.904+0000 7fbd2f3b5640 1 -- 172.21.15.88:0/1316263855 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbd0c0664d0 msgr2=0x7fbd0c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:56.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.904+0000 7fbd2f3b5640 1 --2- 172.21.15.88:0/1316263855 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbd0c0664d0 0x7fbd0c068990 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fbd24006fd0 tx=0x7fbd24008040 comp rx=0 tx=0).stop 2023-10-16T20:07:56.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.904+0000 7fbd2f3b5640 1 -- 172.21.15.88:0/1316263855 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd2815c9a0 msgr2=0x7fbd2806bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:56.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.904+0000 7fbd2f3b5640 1 --2- 172.21.15.88:0/1316263855 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd2815c9a0 0x7fbd2806bcb0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fbd2000c730 tx=0x7fbd2000cbf0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.911 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.904+0000 7fbd2f3b5640 1 -- 172.21.15.88:0/1316263855 shutdown_connections 2023-10-16T20:07:56.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.904+0000 7fbd2f3b5640 1 --2- 172.21.15.88:0/1316263855 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbd0c0664d0 0x7fbd0c068990 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.904+0000 7fbd2f3b5640 1 --2- 172.21.15.88:0/1316263855 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd2815c9a0 0x7fbd2806bcb0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.904+0000 7fbd2f3b5640 1 --2- 172.21.15.88:0/1316263855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd280d3270 0x7fbd2814e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.904+0000 7fbd2f3b5640 1 --2- 172.21.15.88:0/1316263855 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbd280d1610 0x7fbd28158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:56.912 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.904+0000 7fbd2f3b5640 1 -- 172.21.15.88:0/1316263855 >> 172.21.15.88:0/1316263855 conn(0x7fbd2809bc30 msgr2=0x7fbd281590e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:56.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.904+0000 7fbd2f3b5640 1 -- 172.21.15.88:0/1316263855 shutdown_connections 2023-10-16T20:07:56.913 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:56.904+0000 7fbd2f3b5640 1 -- 172.21.15.88:0/1316263855 wait complete. 2023-10-16T20:07:56.930 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail testcephfs1 2023-10-16T20:07:57.068 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.064+0000 7fdead365640 1 Processor -- start 2023-10-16T20:07:57.069 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.064+0000 7fdead365640 1 -- start start 2023-10-16T20:07:57.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.064+0000 7fdead365640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fdea8153ea0 0x7fdea80d1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:57.071 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.064+0000 7fdead365640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fdea80d2660 0x7fdea815d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:57.072 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.064+0000 7fdead365640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdea80b5510 con 0x7fdea815d7d0 2023-10-16T20:07:57.072 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.064+0000 7fdead365640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea80b4990 con 0x7fdea8153ea0 2023-10-16T20:07:57.072 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.064+0000 7fdead365640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea80b4c70 con 0x7fdea80d2660 2023-10-16T20:07:57.073 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.064+0000 7fdea6ffd640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fdea8153ea0 0x7fdea80d1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:57.073 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.064+0000 7fdea6ffd640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fdea8153ea0 0x7fdea80d1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57952/0 (socket says 172.21.15.88:57952) 2023-10-16T20:07:57.073 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.064+0000 7fdea6ffd640 1 -- 172.21.15.88:0/4085686316 learned_addr learned my addr 172.21.15.88:0/4085686316 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:57.074 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.064+0000 7fdea6ffd640 1 -- 172.21.15.88:0/4085686316 >> v2:172.21.15.94:3300/0 conn(0x7fdea80d2660 msgr2=0x7fdea815d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:57.074 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.064+0000 7fde9e7fc640 1 --2- 172.21.15.88:0/4085686316 >> v2:172.21.15.94:3300/0 conn(0x7fdea80d2660 0x7fdea815d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:57.074 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.064+0000 7fdea6ffd640 1 --2- 172.21.15.88:0/4085686316 >> v2:172.21.15.94:3300/0 conn(0x7fdea80d2660 0x7fdea815d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:57.074 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.064+0000 7fdea6ffd640 1 -- 172.21.15.88:0/4085686316 >> v1:172.21.15.94:6789/0 conn(0x7fdea815d7d0 legacy=0x7fdea80cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:57.075 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.064+0000 7fdea6ffd640 1 -- 172.21.15.88:0/4085686316 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea80b4f50 con 0x7fdea8153ea0 2023-10-16T20:07:57.075 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.064+0000 7fde9e7fc640 1 --2- 172.21.15.88:0/4085686316 >> v2:172.21.15.94:3300/0 conn(0x7fdea80d2660 0x7fdea815d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:07:57.076 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.068+0000 7fdea6ffd640 1 --2- 172.21.15.88:0/4085686316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdea8153ea0 0x7fdea80d1d10 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fdea8072520 tx=0x7fde9801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5591e7d565d0a296 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:57.077 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.068+0000 7fdea67fc640 1 -- 172.21.15.88:0/4085686316 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde980ab5f0 con 0x7fdea8153ea0 2023-10-16T20:07:57.077 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.068+0000 7fdea67fc640 1 -- 172.21.15.88:0/4085686316 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde980ab750 con 0x7fdea8153ea0 2023-10-16T20:07:57.077 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.068+0000 7fdea67fc640 1 -- 172.21.15.88:0/4085686316 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde980abae0 con 0x7fdea8153ea0 2023-10-16T20:07:57.077 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.068+0000 7fdead365640 1 -- 172.21.15.88:0/4085686316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdea8153ea0 msgr2=0x7fdea80d1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:57.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.068+0000 7fdead365640 1 --2- 172.21.15.88:0/4085686316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdea8153ea0 0x7fdea80d1d10 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fdea8072520 tx=0x7fde9801c490 comp rx=0 tx=0).stop 2023-10-16T20:07:57.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.068+0000 7fdead365640 1 -- 172.21.15.88:0/4085686316 shutdown_connections 2023-10-16T20:07:57.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.068+0000 7fdead365640 1 --2- 172.21.15.88:0/4085686316 >> v2:172.21.15.94:3300/0 conn(0x7fdea80d2660 0x7fdea815d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:57.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.068+0000 7fdead365640 1 --2- 172.21.15.88:0/4085686316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdea8153ea0 0x7fdea80d1d10 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:57.078 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.068+0000 7fdead365640 1 -- 172.21.15.88:0/4085686316 >> 172.21.15.88:0/4085686316 conn(0x7fdea80715c0 msgr2=0x7fdea80719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:57.079 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.068+0000 7fdead365640 1 -- 172.21.15.88:0/4085686316 shutdown_connections 2023-10-16T20:07:57.079 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.068+0000 7fdead365640 1 -- 172.21.15.88:0/4085686316 wait complete. 2023-10-16T20:07:57.079 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.072+0000 7fdead365640 1 Processor -- start 2023-10-16T20:07:57.079 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.072+0000 7fdead365640 1 -- start start 2023-10-16T20:07:57.081 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.072+0000 7fdead365640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdea80d2660 0x7fdea80ca630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:57.081 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.072+0000 7fdead365640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdea8153ea0 0x7fdea8150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:57.082 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.072+0000 7fdead365640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdea80cab70 0x7fdea80b6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:57.082 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.072+0000 7fdead365640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea8149e90 con 0x7fdea8153ea0 2023-10-16T20:07:57.082 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.072+0000 7fdead365640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea80cc960 con 0x7fdea80d2660 2023-10-16T20:07:57.082 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.072+0000 7fdead365640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea81498d0 con 0x7fdea80cab70 2023-10-16T20:07:57.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.072+0000 7fde9e7fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdea8153ea0 0x7fdea8150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:57.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.072+0000 7fde9e7fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdea8153ea0 0x7fdea8150f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57958/0 (socket says 172.21.15.88:57958) 2023-10-16T20:07:57.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.072+0000 7fde9e7fc640 1 -- 172.21.15.88:0/1835966464 learned_addr learned my addr 172.21.15.88:0/1835966464 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:57.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.072+0000 7fdea6ffd640 1 --2- 172.21.15.88:0/1835966464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdea80d2660 0x7fdea80ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:57.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.072+0000 7fdea77fe640 1 --2- 172.21.15.88:0/1835966464 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdea80cab70 0x7fdea80b6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:57.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.072+0000 7fde9e7fc640 1 -- 172.21.15.88:0/1835966464 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdea80cab70 msgr2=0x7fdea80b6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:57.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.072+0000 7fde9e7fc640 1 --2- 172.21.15.88:0/1835966464 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdea80cab70 0x7fdea80b6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:57.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.072+0000 7fde9e7fc640 1 -- 172.21.15.88:0/1835966464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdea80d2660 msgr2=0x7fdea80ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:57.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.072+0000 7fde9e7fc640 1 --2- 172.21.15.88:0/1835966464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdea80d2660 0x7fdea80ca630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:57.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.072+0000 7fde9e7fc640 1 -- 172.21.15.88:0/1835966464 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde980094d0 con 0x7fdea8153ea0 2023-10-16T20:07:57.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.072+0000 7fde9e7fc640 1 --2- 172.21.15.88:0/1835966464 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdea8153ea0 0x7fdea8150f30 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fde9000cf60 tx=0x7fde9000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:57.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.072+0000 7fde9ffff640 1 -- 172.21.15.88:0/1835966464 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9000d840 con 0x7fdea8153ea0 2023-10-16T20:07:57.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.072+0000 7fde9ffff640 1 -- 172.21.15.88:0/1835966464 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9000d9a0 con 0x7fdea8153ea0 2023-10-16T20:07:57.086 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.072+0000 7fde9ffff640 1 -- 172.21.15.88:0/1835966464 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9000dd30 con 0x7fdea8153ea0 2023-10-16T20:07:57.086 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.072+0000 7fdead365640 1 -- 172.21.15.88:0/1835966464 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea8149bb0 con 0x7fdea8153ea0 2023-10-16T20:07:57.086 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.072+0000 7fdead365640 1 -- 172.21.15.88:0/1835966464 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea80cc4f0 con 0x7fdea8153ea0 2023-10-16T20:07:57.086 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.072+0000 7fdead365640 1 -- 172.21.15.88:0/1835966464 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdea804e9c0 con 0x7fdea8153ea0 2023-10-16T20:07:57.086 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.076+0000 7fde9ffff640 1 -- 172.21.15.88:0/1835966464 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fde90015440 con 0x7fdea8153ea0 2023-10-16T20:07:57.087 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.076+0000 7fde9ffff640 1 --2- 172.21.15.88:0/1835966464 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fde880664a0 0x7fde88068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:57.088 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.076+0000 7fde9ffff640 1 -- 172.21.15.88:0/1835966464 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fde90084920 con 0x7fdea8153ea0 2023-10-16T20:07:57.088 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.076+0000 7fdea6ffd640 1 --2- 172.21.15.88:0/1835966464 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fde880664a0 0x7fde88068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:57.088 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.076+0000 7fdea6ffd640 1 --2- 172.21.15.88:0/1835966464 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fde880664a0 0x7fde88068960 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fde980af000 tx=0x7fde980b6040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:57.088 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.080+0000 7fde9ffff640 1 -- 172.21.15.88:0/1835966464 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fde90051d80 con 0x7fdea8153ea0 2023-10-16T20:07:57.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.260+0000 7fdead365640 1 -- 172.21.15.88:0/1835966464 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "testcephfs1"} v 0) v1 -- 0x7fdea80b6a70 con 0x7fdea8153ea0 2023-10-16T20:07:57.268 INFO:tasks.ceph.mon.a.smithi088.stderr:2023-10-16T20:07:57.260+0000 7f41ee15b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T20:07:57.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.264+0000 7fde9ffff640 1 -- 172.21.15.88:0/1835966464 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "testcephfs1"}]=0 testcephfs1 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) v1 ==== 171+0+0 (secure 0 0 0) 0x7fde9001a020 con 0x7fdea8153ea0 2023-10-16T20:07:57.270 INFO:teuthology.orchestra.run.smithi088.stderr:testcephfs1 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T20:07:57.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.268+0000 7fdead365640 1 -- 172.21.15.88:0/1835966464 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fde880664a0 msgr2=0x7fde88068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:57.272 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.268+0000 7fdead365640 1 --2- 172.21.15.88:0/1835966464 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fde880664a0 0x7fde88068960 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fde980af000 tx=0x7fde980b6040 comp rx=0 tx=0).stop 2023-10-16T20:07:57.273 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.268+0000 7fdead365640 1 -- 172.21.15.88:0/1835966464 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdea8153ea0 msgr2=0x7fdea8150f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:57.273 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.268+0000 7fdead365640 1 --2- 172.21.15.88:0/1835966464 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdea8153ea0 0x7fdea8150f30 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fde9000cf60 tx=0x7fde9000cf90 comp rx=0 tx=0).stop 2023-10-16T20:07:57.273 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.268+0000 7fdead365640 1 -- 172.21.15.88:0/1835966464 shutdown_connections 2023-10-16T20:07:57.273 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.268+0000 7fdead365640 1 --2- 172.21.15.88:0/1835966464 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fde880664a0 0x7fde88068960 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:57.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.268+0000 7fdead365640 1 --2- 172.21.15.88:0/1835966464 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdea80cab70 0x7fdea80b6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:57.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.268+0000 7fdead365640 1 --2- 172.21.15.88:0/1835966464 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdea8153ea0 0x7fdea8150f30 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:57.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.268+0000 7fdead365640 1 --2- 172.21.15.88:0/1835966464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdea80d2660 0x7fdea80ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:57.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.268+0000 7fdead365640 1 -- 172.21.15.88:0/1835966464 >> 172.21.15.88:0/1835966464 conn(0x7fdea80715c0 msgr2=0x7fdea80d1930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:57.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.268+0000 7fdead365640 1 -- 172.21.15.88:0/1835966464 shutdown_connections 2023-10-16T20:07:57.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.268+0000 7fdead365640 1 -- 172.21.15.88:0/1835966464 wait complete. 2023-10-16T20:07:57.278 INFO:tasks.ceph.mds.a.smithi088.stderr: -2527> 2023-10-16T20:07:32.237+0000 7f239fea7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T20:07:57.294 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm testcephfs1 --yes-i-really-mean-it 2023-10-16T20:07:57.326 INFO:tasks.ceph.mds.a.smithi088.stdout:starting mds.a at 2023-10-16T20:07:57.436 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.432+0000 7fbfb4017640 1 Processor -- start 2023-10-16T20:07:57.436 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.432+0000 7fbfb4017640 1 -- start start 2023-10-16T20:07:57.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.432+0000 7fbfb4017640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fbfac0b0c10 0x7fbfac15aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:57.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.432+0000 7fbfb4017640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fbfac15b530 0x7fbfac06c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:57.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.432+0000 7fbfb4017640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfac15bd20 con 0x7fbfac154f30 2023-10-16T20:07:57.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.432+0000 7fbfb4017640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfac155310 con 0x7fbfac15b530 2023-10-16T20:07:57.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.432+0000 7fbfb4017640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfac0702f0 con 0x7fbfac0b0c10 2023-10-16T20:07:57.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.432+0000 7fbfb25b4640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fbfac15b530 0x7fbfac06c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:57.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.432+0000 7fbfb25b4640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fbfac15b530 0x7fbfac06c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57978/0 (socket says 172.21.15.88:57978) 2023-10-16T20:07:57.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.432+0000 7fbfb25b4640 1 -- 172.21.15.88:0/2770993110 learned_addr learned my addr 172.21.15.88:0/2770993110 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:57.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.432+0000 7fbfb15b2640 1 --2- 172.21.15.88:0/2770993110 >> v2:172.21.15.94:3300/0 conn(0x7fbfac0b0c10 0x7fbfac15aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:57.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.432+0000 7fbfb25b4640 1 -- 172.21.15.88:0/2770993110 >> v2:172.21.15.94:3300/0 conn(0x7fbfac0b0c10 msgr2=0x7fbfac15aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:57.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.432+0000 7fbfb25b4640 1 --2- 172.21.15.88:0/2770993110 >> v2:172.21.15.94:3300/0 conn(0x7fbfac0b0c10 0x7fbfac15aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:57.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.432+0000 7fbfb25b4640 1 -- 172.21.15.88:0/2770993110 >> v1:172.21.15.94:6789/0 conn(0x7fbfac154f30 legacy=0x7fbfac0b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:57.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.432+0000 7fbfb25b4640 1 -- 172.21.15.88:0/2770993110 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfac0705d0 con 0x7fbfac15b530 2023-10-16T20:07:57.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.432+0000 7fbfb15b2640 1 --2- 172.21.15.88:0/2770993110 >> v2:172.21.15.94:3300/0 conn(0x7fbfac0b0c10 0x7fbfac15aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:07:57.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.436+0000 7fbfb25b4640 1 --2- 172.21.15.88:0/2770993110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfac15b530 0x7fbfac06c2e0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fbfac06e700 tx=0x7fbfa401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3442e8d6d87e0c0e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:57.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.436+0000 7fbfb0db1640 1 -- 172.21.15.88:0/2770993110 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa40a5860 con 0x7fbfac15b530 2023-10-16T20:07:57.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.436+0000 7fbfb0db1640 1 -- 172.21.15.88:0/2770993110 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa40a59c0 con 0x7fbfac15b530 2023-10-16T20:07:57.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.436+0000 7fbfb0db1640 1 -- 172.21.15.88:0/2770993110 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa40a5d50 con 0x7fbfac15b530 2023-10-16T20:07:57.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.436+0000 7fbfb4017640 1 -- 172.21.15.88:0/2770993110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfac15b530 msgr2=0x7fbfac06c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:57.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.436+0000 7fbfb4017640 1 --2- 172.21.15.88:0/2770993110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfac15b530 0x7fbfac06c2e0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fbfac06e700 tx=0x7fbfa401c9a0 comp rx=0 tx=0).stop 2023-10-16T20:07:57.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.436+0000 7fbfb4017640 1 -- 172.21.15.88:0/2770993110 shutdown_connections 2023-10-16T20:07:57.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.436+0000 7fbfb4017640 1 --2- 172.21.15.88:0/2770993110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfac15b530 0x7fbfac06c2e0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:57.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.436+0000 7fbfb4017640 1 --2- 172.21.15.88:0/2770993110 >> v2:172.21.15.94:3300/0 conn(0x7fbfac0b0c10 0x7fbfac15aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:57.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.436+0000 7fbfb4017640 1 -- 172.21.15.88:0/2770993110 >> 172.21.15.88:0/2770993110 conn(0x7fbfac0cb080 msgr2=0x7fbfac0cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:57.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.436+0000 7fbfb4017640 1 -- 172.21.15.88:0/2770993110 shutdown_connections 2023-10-16T20:07:57.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.436+0000 7fbfb4017640 1 -- 172.21.15.88:0/2770993110 wait complete. 2023-10-16T20:07:57.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.440+0000 7fbfb4017640 1 Processor -- start 2023-10-16T20:07:57.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.440+0000 7fbfb4017640 1 -- start start 2023-10-16T20:07:57.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.440+0000 7fbfb4017640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfac0b0c10 0x7fbfac15c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:57.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.440+0000 7fbfb4017640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbfac154f30 0x7fbfac14a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:57.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.440+0000 7fbfb4017640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfac15b530 0x7fbfac14af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:57.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.440+0000 7fbfb4017640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfac15da00 con 0x7fbfac0b0c10 2023-10-16T20:07:57.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.440+0000 7fbfb4017640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfac15cf30 con 0x7fbfac15b530 2023-10-16T20:07:57.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.440+0000 7fbfb4017640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfac14ad40 con 0x7fbfac154f30 2023-10-16T20:07:57.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.440+0000 7fbfb1db3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfac0b0c10 0x7fbfac15c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:57.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.440+0000 7fbfb1db3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfac0b0c10 0x7fbfac15c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57986/0 (socket says 172.21.15.88:57986) 2023-10-16T20:07:57.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.440+0000 7fbfb1db3640 1 -- 172.21.15.88:0/3467119400 learned_addr learned my addr 172.21.15.88:0/3467119400 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:57.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.440+0000 7fbfb25b4640 1 --2- 172.21.15.88:0/3467119400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfac15b530 0x7fbfac14af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:57.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.440+0000 7fbfb15b2640 1 --2- 172.21.15.88:0/3467119400 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbfac154f30 0x7fbfac14a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:57.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.440+0000 7fbfb1db3640 1 -- 172.21.15.88:0/3467119400 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbfac154f30 msgr2=0x7fbfac14a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:57.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.440+0000 7fbfb1db3640 1 --2- 172.21.15.88:0/3467119400 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbfac154f30 0x7fbfac14a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:57.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.440+0000 7fbfb1db3640 1 -- 172.21.15.88:0/3467119400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfac15b530 msgr2=0x7fbfac14af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:57.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.440+0000 7fbfb1db3640 1 --2- 172.21.15.88:0/3467119400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfac15b530 0x7fbfac14af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:57.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.440+0000 7fbfb1db3640 1 -- 172.21.15.88:0/3467119400 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa4009b30 con 0x7fbfac0b0c10 2023-10-16T20:07:57.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.440+0000 7fbfb1db3640 1 --2- 172.21.15.88:0/3467119400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfac0b0c10 0x7fbfac15c9f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800c530 tx=0x7fbfa800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:57.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.440+0000 7fbfa27fc640 1 -- 172.21.15.88:0/3467119400 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8005220 con 0x7fbfac0b0c10 2023-10-16T20:07:57.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.440+0000 7fbfa27fc640 1 -- 172.21.15.88:0/3467119400 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa8005380 con 0x7fbfac0b0c10 2023-10-16T20:07:57.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.440+0000 7fbfa27fc640 1 -- 172.21.15.88:0/3467119400 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8007270 con 0x7fbfac0b0c10 2023-10-16T20:07:57.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.440+0000 7fbfb4017640 1 -- 172.21.15.88:0/3467119400 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfac15d160 con 0x7fbfac0b0c10 2023-10-16T20:07:57.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.440+0000 7fbfb4017640 1 -- 172.21.15.88:0/3467119400 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfac15d720 con 0x7fbfac0b0c10 2023-10-16T20:07:57.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.440+0000 7fbfb4017640 1 -- 172.21.15.88:0/3467119400 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfac04e9c0 con 0x7fbfac0b0c10 2023-10-16T20:07:57.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.444+0000 7fbfa27fc640 1 -- 172.21.15.88:0/3467119400 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbfa80073d0 con 0x7fbfac0b0c10 2023-10-16T20:07:57.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.444+0000 7fbfa27fc640 1 --2- 172.21.15.88:0/3467119400 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbf900664a0 0x7fbf90068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:57.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.444+0000 7fbfa27fc640 1 -- 172.21.15.88:0/3467119400 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6135+0+0 (secure 0 0 0) 0x7fbfa8083770 con 0x7fbfac0b0c10 2023-10-16T20:07:57.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.444+0000 7fbfb15b2640 1 --2- 172.21.15.88:0/3467119400 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbf900664a0 0x7fbf90068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:57.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.444+0000 7fbfb15b2640 1 --2- 172.21.15.88:0/3467119400 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbf900664a0 0x7fbf90068960 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fbfac1598b0 tx=0x7fbf9c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:57.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.448+0000 7fbfa27fc640 1 -- 172.21.15.88:0/3467119400 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbfa8050b80 con 0x7fbfac0b0c10 2023-10-16T20:07:57.633 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:57.628+0000 7fbfb4017640 1 -- 172.21.15.88:0/3467119400 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "testcephfs1", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbfac0b1110 con 0x7fbfac0b0c10 2023-10-16T20:07:58.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.264+0000 7fbfa27fc640 1 -- 172.21.15.88:0/3467119400 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "testcephfs1", "yes_i_really_mean_it": true}]=0 v30) v1 ==== 109+0+0 (secure 0 0 0) 0x7fbfa80541a0 con 0x7fbfac0b0c10 2023-10-16T20:07:58.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.268+0000 7fbfb4017640 1 -- 172.21.15.88:0/3467119400 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbf900664a0 msgr2=0x7fbf90068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:58.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.268+0000 7fbfb4017640 1 --2- 172.21.15.88:0/3467119400 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbf900664a0 0x7fbf90068960 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fbfac1598b0 tx=0x7fbf9c008040 comp rx=0 tx=0).stop 2023-10-16T20:07:58.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.268+0000 7fbfb4017640 1 -- 172.21.15.88:0/3467119400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfac0b0c10 msgr2=0x7fbfac15c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:58.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.268+0000 7fbfb4017640 1 --2- 172.21.15.88:0/3467119400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfac0b0c10 0x7fbfac15c9f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800c530 tx=0x7fbfa800c9f0 comp rx=0 tx=0).stop 2023-10-16T20:07:58.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.268+0000 7fbfb4017640 1 -- 172.21.15.88:0/3467119400 shutdown_connections 2023-10-16T20:07:58.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.268+0000 7fbfb4017640 1 --2- 172.21.15.88:0/3467119400 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fbf900664a0 0x7fbf90068960 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:58.277 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.268+0000 7fbfb4017640 1 --2- 172.21.15.88:0/3467119400 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbfac154f30 0x7fbfac14a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:58.277 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.268+0000 7fbfb4017640 1 --2- 172.21.15.88:0/3467119400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfac15b530 0x7fbfac14af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:58.277 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.268+0000 7fbfb4017640 1 --2- 172.21.15.88:0/3467119400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfac0b0c10 0x7fbfac15c9f0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:58.278 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.268+0000 7fbfb4017640 1 -- 172.21.15.88:0/3467119400 >> 172.21.15.88:0/3467119400 conn(0x7fbfac0cb080 msgr2=0x7fbfac0b0500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:58.278 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.268+0000 7fbfb4017640 1 -- 172.21.15.88:0/3467119400 shutdown_connections 2023-10-16T20:07:58.278 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.268+0000 7fbfb4017640 1 -- 172.21.15.88:0/3467119400 wait complete. 2023-10-16T20:07:58.296 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm testcephfs1_metadata testcephfs1_metadata --yes-i-really-really-mean-it 2023-10-16T20:07:58.490 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.484+0000 7f137bc09640 1 Processor -- start 2023-10-16T20:07:58.490 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.484+0000 7f137bc09640 1 -- start start 2023-10-16T20:07:58.491 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.484+0000 7f137bc09640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f137415cdc0 0x7f137406de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:58.491 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.484+0000 7f137bc09640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f137406cb10 0x7f1374159f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:58.492 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.484+0000 7f137bc09640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f137415a480 con 0x7f137415d1a0 2023-10-16T20:07:58.492 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.484+0000 7f137bc09640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f137406cfa0 con 0x7f137415cdc0 2023-10-16T20:07:58.492 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.484+0000 7f137bc09640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13740cca60 con 0x7f137406cb10 2023-10-16T20:07:58.492 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.484+0000 7f13791a4640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f137415d1a0 0x7f13740cdb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:56448/0 (socket says 172.21.15.88:56448) 2023-10-16T20:07:58.492 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.484+0000 7f13791a4640 1 -- 172.21.15.88:0/2924733365 learned_addr learned my addr 172.21.15.88:0/2924733365 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:07:58.493 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.484+0000 7f13799a5640 1 --2- 172.21.15.88:0/2924733365 >> v2:172.21.15.88:3300/0 conn(0x7f137415cdc0 0x7f137406de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:58.493 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.484+0000 7f137a1a6640 1 --2- 172.21.15.88:0/2924733365 >> v2:172.21.15.94:3300/0 conn(0x7f137406cb10 0x7f1374159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:58.494 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.488+0000 7f13789a3640 1 -- 172.21.15.88:0/2924733365 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1017253681 0 0) 0x7f137415a480 con 0x7f137415d1a0 2023-10-16T20:07:58.494 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.488+0000 7f13789a3640 1 -- 172.21.15.88:0/2924733365 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1360003530 con 0x7f137415d1a0 2023-10-16T20:07:58.494 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.488+0000 7f13799a5640 1 -- 172.21.15.88:0/2924733365 >> v2:172.21.15.94:3300/0 conn(0x7f137406cb10 msgr2=0x7f1374159f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:58.494 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.488+0000 7f13799a5640 1 --2- 172.21.15.88:0/2924733365 >> v2:172.21.15.94:3300/0 conn(0x7f137406cb10 0x7f1374159f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:58.494 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.488+0000 7f13799a5640 1 -- 172.21.15.88:0/2924733365 >> v1:172.21.15.88:6789/0 conn(0x7f137415d1a0 legacy=0x7f13740cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:58.495 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.488+0000 7f13799a5640 1 -- 172.21.15.88:0/2924733365 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13740ccd40 con 0x7f137415cdc0 2023-10-16T20:07:58.495 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.488+0000 7f137a1a6640 1 --2- 172.21.15.88:0/2924733365 >> v2:172.21.15.94:3300/0 conn(0x7f137406cb10 0x7f1374159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:07:58.496 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.488+0000 7f13799a5640 1 --2- 172.21.15.88:0/2924733365 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f137415cdc0 0x7f137406de00 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f13740c2410 tx=0x7f137001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c08bbb79295671f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:58.496 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.488+0000 7f13789a3640 1 -- 172.21.15.88:0/2924733365 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13700ac640 con 0x7f137415cdc0 2023-10-16T20:07:58.497 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.488+0000 7f13789a3640 1 -- 172.21.15.88:0/2924733365 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13700ac7a0 con 0x7f137415cdc0 2023-10-16T20:07:58.497 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.488+0000 7f13789a3640 1 -- 172.21.15.88:0/2924733365 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13700acb30 con 0x7f137415cdc0 2023-10-16T20:07:58.497 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.492+0000 7f137bc09640 1 -- 172.21.15.88:0/2924733365 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f137415cdc0 msgr2=0x7f137406de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:58.497 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.492+0000 7f137bc09640 1 --2- 172.21.15.88:0/2924733365 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f137415cdc0 0x7f137406de00 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f13740c2410 tx=0x7f137001c460 comp rx=0 tx=0).stop 2023-10-16T20:07:58.497 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.492+0000 7f137bc09640 1 -- 172.21.15.88:0/2924733365 shutdown_connections 2023-10-16T20:07:58.498 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.492+0000 7f137bc09640 1 --2- 172.21.15.88:0/2924733365 >> v2:172.21.15.94:3300/0 conn(0x7f137406cb10 0x7f1374159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:58.498 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.492+0000 7f137bc09640 1 --2- 172.21.15.88:0/2924733365 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f137415cdc0 0x7f137406de00 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:58.498 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.492+0000 7f137bc09640 1 -- 172.21.15.88:0/2924733365 >> 172.21.15.88:0/2924733365 conn(0x7f13740b0980 msgr2=0x7f13740b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:58.498 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.492+0000 7f137bc09640 1 -- 172.21.15.88:0/2924733365 shutdown_connections 2023-10-16T20:07:58.498 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.492+0000 7f137bc09640 1 -- 172.21.15.88:0/2924733365 wait complete. 2023-10-16T20:07:58.501 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.492+0000 7f137bc09640 1 Processor -- start 2023-10-16T20:07:58.501 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.492+0000 7f137bc09640 1 -- start start 2023-10-16T20:07:58.501 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.492+0000 7f137bc09640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f137406cb10 0x7f1374158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:58.501 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.492+0000 7f137bc09640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f137415cdc0 0x7f137414bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:58.502 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.492+0000 7f137bc09640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f137415d1a0 0x7f13741f6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:58.502 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.492+0000 7f137bc09640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13741f5bf0 con 0x7f137415d1a0 2023-10-16T20:07:58.502 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.492+0000 7f137bc09640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137414c3e0 con 0x7f137415cdc0 2023-10-16T20:07:58.502 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.492+0000 7f137bc09640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13741f6550 con 0x7f137406cb10 2023-10-16T20:07:58.502 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.492+0000 7f137a1a6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f137415d1a0 0x7f13741f6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:58.503 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.492+0000 7f137a1a6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f137415d1a0 0x7f13741f6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58002/0 (socket says 172.21.15.88:58002) 2023-10-16T20:07:58.503 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.492+0000 7f137a1a6640 1 -- 172.21.15.88:0/2756789568 learned_addr learned my addr 172.21.15.88:0/2756789568 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:58.503 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.492+0000 7f13799a5640 1 --2- 172.21.15.88:0/2756789568 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f137406cb10 0x7f1374158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:58.503 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.492+0000 7f13791a4640 1 --2- 172.21.15.88:0/2756789568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f137415cdc0 0x7f137414bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:58.503 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.492+0000 7f137a1a6640 1 -- 172.21.15.88:0/2756789568 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f137406cb10 msgr2=0x7f1374158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:58.504 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.492+0000 7f137a1a6640 1 --2- 172.21.15.88:0/2756789568 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f137406cb10 0x7f1374158300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:58.504 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.492+0000 7f137a1a6640 1 -- 172.21.15.88:0/2756789568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f137415cdc0 msgr2=0x7f137414bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:58.504 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.492+0000 7f137a1a6640 1 --2- 172.21.15.88:0/2756789568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f137415cdc0 0x7f137414bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:58.504 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.492+0000 7f137a1a6640 1 -- 172.21.15.88:0/2756789568 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f136c009a70 con 0x7f137415d1a0 2023-10-16T20:07:58.504 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.492+0000 7f13791a4640 1 --2- 172.21.15.88:0/2756789568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f137415cdc0 0x7f137414bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:07:58.505 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.492+0000 7f137a1a6640 1 --2- 172.21.15.88:0/2756789568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f137415d1a0 0x7f13741f6010 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f136c009f00 tx=0x7f136c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:58.505 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.496+0000 7f136a7fc640 1 -- 172.21.15.88:0/2756789568 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c009d70 con 0x7f137415d1a0 2023-10-16T20:07:58.505 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.496+0000 7f136a7fc640 1 -- 172.21.15.88:0/2756789568 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f136c00f040 con 0x7f137415d1a0 2023-10-16T20:07:58.505 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.496+0000 7f136a7fc640 1 -- 172.21.15.88:0/2756789568 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c003bc0 con 0x7f137415d1a0 2023-10-16T20:07:58.505 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.496+0000 7f137bc09640 1 -- 172.21.15.88:0/2756789568 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1370009420 con 0x7f137415d1a0 2023-10-16T20:07:58.506 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.496+0000 7f137bc09640 1 -- 172.21.15.88:0/2756789568 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13741f5530 con 0x7f137415d1a0 2023-10-16T20:07:58.506 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.496+0000 7f137bc09640 1 -- 172.21.15.88:0/2756789568 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13741f5d30 con 0x7f137415d1a0 2023-10-16T20:07:58.507 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.496+0000 7f136a7fc640 1 -- 172.21.15.88:0/2756789568 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f136c014480 con 0x7f137415d1a0 2023-10-16T20:07:58.507 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.496+0000 7f136a7fc640 1 --2- 172.21.15.88:0/2756789568 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f13580664a0 0x7f1358068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:58.507 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.496+0000 7f136a7fc640 1 -- 172.21.15.88:0/2756789568 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6135+0+0 (secure 0 0 0) 0x7f136c084c60 con 0x7f137415d1a0 2023-10-16T20:07:58.508 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.496+0000 7f13799a5640 1 --2- 172.21.15.88:0/2756789568 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f13580664a0 0x7f1358068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:58.508 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.496+0000 7f13799a5640 1 --2- 172.21.15.88:0/2756789568 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f13580664a0 0x7f1358068960 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f13700ae000 tx=0x7f1370008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:58.508 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.500+0000 7f136a7fc640 1 -- 172.21.15.88:0/2756789568 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f13741f5d30 con 0x7f137415d1a0 2023-10-16T20:07:58.658 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:58.652+0000 7f137bc09640 1 -- 172.21.15.88:0/2756789568 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "testcephfs1_metadata", "pool2": "testcephfs1_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f13741f5d30 con 0x7f137415d1a0 2023-10-16T20:07:59.276 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.272+0000 7f136a7fc640 1 -- 172.21.15.88:0/2756789568 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "testcephfs1_metadata", "pool2": "testcephfs1_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'testcephfs1_metadata' removed v60) v1 ==== 196+0+0 (secure 0 0 0) 0x7f13741f5d30 con 0x7f137415d1a0 2023-10-16T20:07:59.277 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'testcephfs1_metadata' removed 2023-10-16T20:07:59.279 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.272+0000 7f137bc09640 1 -- 172.21.15.88:0/2756789568 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f13580664a0 msgr2=0x7f1358068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:59.279 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.272+0000 7f137bc09640 1 --2- 172.21.15.88:0/2756789568 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f13580664a0 0x7f1358068960 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f13700ae000 tx=0x7f1370008040 comp rx=0 tx=0).stop 2023-10-16T20:07:59.279 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.272+0000 7f137bc09640 1 -- 172.21.15.88:0/2756789568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f137415d1a0 msgr2=0x7f13741f6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:59.280 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.272+0000 7f137bc09640 1 --2- 172.21.15.88:0/2756789568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f137415d1a0 0x7f13741f6010 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f136c009f00 tx=0x7f136c00c490 comp rx=0 tx=0).stop 2023-10-16T20:07:59.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.272+0000 7f137bc09640 1 -- 172.21.15.88:0/2756789568 shutdown_connections 2023-10-16T20:07:59.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.276+0000 7f137bc09640 1 --2- 172.21.15.88:0/2756789568 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f13580664a0 0x7f1358068960 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:59.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.276+0000 7f137bc09640 1 --2- 172.21.15.88:0/2756789568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f137415d1a0 0x7f13741f6010 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:59.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.276+0000 7f137bc09640 1 --2- 172.21.15.88:0/2756789568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f137415cdc0 0x7f137414bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:59.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.276+0000 7f137bc09640 1 --2- 172.21.15.88:0/2756789568 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f137406cb10 0x7f1374158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:59.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.276+0000 7f137bc09640 1 -- 172.21.15.88:0/2756789568 >> 172.21.15.88:0/2756789568 conn(0x7f13740b0980 msgr2=0x7f137406a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:59.283 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.276+0000 7f137bc09640 1 -- 172.21.15.88:0/2756789568 shutdown_connections 2023-10-16T20:07:59.283 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.276+0000 7f137bc09640 1 -- 172.21.15.88:0/2756789568 wait complete. 2023-10-16T20:07:59.302 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm testcephfs1_data testcephfs1_data --yes-i-really-really-mean-it 2023-10-16T20:07:59.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.440+0000 7f63d03c4640 1 Processor -- start 2023-10-16T20:07:59.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.440+0000 7f63d03c4640 1 -- start start 2023-10-16T20:07:59.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.444+0000 7f63d03c4640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f63c815cdb0 0x7f63c80cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:59.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.444+0000 7f63d03c4640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63c8069e90 0x7f63c806a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:59.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.444+0000 7f63d03c4640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63c806a800 con 0x7f63c80cfe10 2023-10-16T20:07:59.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.444+0000 7f63d03c4640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63c815d190 con 0x7f63c815cdb0 2023-10-16T20:07:59.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.444+0000 7f63d03c4640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63c80b0a50 con 0x7f63c8069e90 2023-10-16T20:07:59.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.444+0000 7f63ce160640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f63c815cdb0 0x7f63c80cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:59.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.444+0000 7f63ce160640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f63c815cdb0 0x7f63c80cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58014/0 (socket says 172.21.15.88:58014) 2023-10-16T20:07:59.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.444+0000 7f63ce160640 1 -- 172.21.15.88:0/1731711971 learned_addr learned my addr 172.21.15.88:0/1731711971 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:59.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.444+0000 7f63ce961640 1 --2- 172.21.15.88:0/1731711971 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63c8069e90 0x7f63c806a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:59.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.444+0000 7f63ce160640 1 -- 172.21.15.88:0/1731711971 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63c8069e90 msgr2=0x7f63c806a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:59.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.444+0000 7f63ce160640 1 --2- 172.21.15.88:0/1731711971 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63c8069e90 0x7f63c806a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:59.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.444+0000 7f63ce160640 1 -- 172.21.15.88:0/1731711971 >> v1:172.21.15.94:6789/0 conn(0x7f63c80cfe10 legacy=0x7f63c80b17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:07:59.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.444+0000 7f63ce160640 1 -- 172.21.15.88:0/1731711971 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c80b0d30 con 0x7f63c815cdb0 2023-10-16T20:07:59.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.444+0000 7f63ce961640 1 --2- 172.21.15.88:0/1731711971 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63c8069e90 0x7f63c806a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:07:59.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.444+0000 7f63ce160640 1 --2- 172.21.15.88:0/1731711971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63c815cdb0 0x7f63c80cb9e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f63c8070f50 tx=0x7f63c401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=541c46b7c85df610 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:59.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.448+0000 7f63cd15e640 1 -- 172.21.15.88:0/1731711971 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c40ab650 con 0x7f63c815cdb0 2023-10-16T20:07:59.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.448+0000 7f63cd15e640 1 -- 172.21.15.88:0/1731711971 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c40ab7b0 con 0x7f63c815cdb0 2023-10-16T20:07:59.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.448+0000 7f63cd15e640 1 -- 172.21.15.88:0/1731711971 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c40abb40 con 0x7f63c815cdb0 2023-10-16T20:07:59.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.448+0000 7f63d03c4640 1 -- 172.21.15.88:0/1731711971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63c815cdb0 msgr2=0x7f63c80cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:59.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.448+0000 7f63d03c4640 1 --2- 172.21.15.88:0/1731711971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63c815cdb0 0x7f63c80cb9e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f63c8070f50 tx=0x7f63c401c490 comp rx=0 tx=0).stop 2023-10-16T20:07:59.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.448+0000 7f63d03c4640 1 -- 172.21.15.88:0/1731711971 shutdown_connections 2023-10-16T20:07:59.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.448+0000 7f63d03c4640 1 --2- 172.21.15.88:0/1731711971 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63c8069e90 0x7f63c806a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:59.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.448+0000 7f63d03c4640 1 --2- 172.21.15.88:0/1731711971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63c815cdb0 0x7f63c80cb9e0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:59.454 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.448+0000 7f63d03c4640 1 -- 172.21.15.88:0/1731711971 >> 172.21.15.88:0/1731711971 conn(0x7f63c8098a20 msgr2=0x7f63c80cf700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:07:59.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.448+0000 7f63d03c4640 1 -- 172.21.15.88:0/1731711971 shutdown_connections 2023-10-16T20:07:59.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.448+0000 7f63d03c4640 1 -- 172.21.15.88:0/1731711971 wait complete. 2023-10-16T20:07:59.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.448+0000 7f63d03c4640 1 Processor -- start 2023-10-16T20:07:59.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.448+0000 7f63d03c4640 1 -- start start 2023-10-16T20:07:59.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.448+0000 7f63d03c4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63c8069e90 0x7f63c814d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:59.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.448+0000 7f63d03c4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63c80cfe10 0x7f63c814dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:59.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.448+0000 7f63d03c4640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63c815cdb0 0x7f63c8149dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:59.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.448+0000 7f63d03c4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c8153220 con 0x7f63c80cfe10 2023-10-16T20:07:59.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.448+0000 7f63d03c4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c80ac630 con 0x7f63c8069e90 2023-10-16T20:07:59.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.448+0000 7f63d03c4640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63c8152c60 con 0x7f63c815cdb0 2023-10-16T20:07:59.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.448+0000 7f63cd95f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63c80cfe10 0x7f63c814dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:59.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.448+0000 7f63cd95f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63c80cfe10 0x7f63c814dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58028/0 (socket says 172.21.15.88:58028) 2023-10-16T20:07:59.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.448+0000 7f63cd95f640 1 -- 172.21.15.88:0/1041083950 learned_addr learned my addr 172.21.15.88:0/1041083950 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:07:59.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.452+0000 7f63ce160640 1 --2- 172.21.15.88:0/1041083950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63c8069e90 0x7f63c814d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:59.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.452+0000 7f63ce961640 1 --2- 172.21.15.88:0/1041083950 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63c815cdb0 0x7f63c8149dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:59.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.452+0000 7f63cd95f640 1 -- 172.21.15.88:0/1041083950 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63c815cdb0 msgr2=0x7f63c8149dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:59.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.452+0000 7f63cd95f640 1 --2- 172.21.15.88:0/1041083950 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63c815cdb0 0x7f63c8149dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:59.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.452+0000 7f63cd95f640 1 -- 172.21.15.88:0/1041083950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63c8069e90 msgr2=0x7f63c814d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:07:59.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.452+0000 7f63cd95f640 1 --2- 172.21.15.88:0/1041083950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63c8069e90 0x7f63c814d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:07:59.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.452+0000 7f63cd95f640 1 -- 172.21.15.88:0/1041083950 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c4009480 con 0x7f63c80cfe10 2023-10-16T20:07:59.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.452+0000 7f63ce160640 1 --2- 172.21.15.88:0/1041083950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63c8069e90 0x7f63c814d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:07:59.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.452+0000 7f63cd95f640 1 --2- 172.21.15.88:0/1041083950 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63c80cfe10 0x7f63c814dc90 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f63b800b410 tx=0x7f63b800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:59.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.452+0000 7f63beffd640 1 -- 172.21.15.88:0/1041083950 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b8004020 con 0x7f63c80cfe10 2023-10-16T20:07:59.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.452+0000 7f63beffd640 1 -- 172.21.15.88:0/1041083950 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b8004180 con 0x7f63c80cfe10 2023-10-16T20:07:59.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.452+0000 7f63d03c4640 1 -- 172.21.15.88:0/1041083950 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63c8152f40 con 0x7f63c80cfe10 2023-10-16T20:07:59.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.452+0000 7f63beffd640 1 -- 172.21.15.88:0/1041083950 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b8011630 con 0x7f63c80cfe10 2023-10-16T20:07:59.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.452+0000 7f63d03c4640 1 -- 172.21.15.88:0/1041083950 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63c80acb60 con 0x7f63c80cfe10 2023-10-16T20:07:59.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.452+0000 7f63d03c4640 1 -- 172.21.15.88:0/1041083950 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63c804e9c0 con 0x7f63c80cfe10 2023-10-16T20:07:59.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.452+0000 7f63beffd640 1 -- 172.21.15.88:0/1041083950 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f63b8011790 con 0x7f63c80cfe10 2023-10-16T20:07:59.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.452+0000 7f63beffd640 1 --2- 172.21.15.88:0/1041083950 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f63ac0664a0 0x7f63ac068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:07:59.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.452+0000 7f63beffd640 1 -- 172.21.15.88:0/1041083950 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f63b8083bf0 con 0x7f63c80cfe10 2023-10-16T20:07:59.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.456+0000 7f63ce160640 1 --2- 172.21.15.88:0/1041083950 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f63ac0664a0 0x7f63ac068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:07:59.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.456+0000 7f63ce160640 1 --2- 172.21.15.88:0/1041083950 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f63ac0664a0 0x7f63ac068960 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f63c40af000 tx=0x7f63c40a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:07:59.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.456+0000 7f63beffd640 1 -- 172.21.15.88:0/1041083950 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f63b8051060 con 0x7f63c80cfe10 2023-10-16T20:07:59.616 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:07:59.612+0000 7f63d03c4640 1 -- 172.21.15.88:0/1041083950 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "testcephfs1_data", "pool2": "testcephfs1_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f63c80acd20 con 0x7f63c80cfe10 2023-10-16T20:08:00.283 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.276+0000 7f63beffd640 1 -- 172.21.15.88:0/1041083950 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "testcephfs1_data", "pool2": "testcephfs1_data", "yes_i_really_really_mean_it": true}]=0 pool 'testcephfs1_data' removed v61) v1 ==== 184+0+0 (secure 0 0 0) 0x7f63b8054680 con 0x7f63c80cfe10 2023-10-16T20:08:00.283 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'testcephfs1_data' removed 2023-10-16T20:08:00.287 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.280+0000 7f63d03c4640 1 -- 172.21.15.88:0/1041083950 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f63ac0664a0 msgr2=0x7f63ac068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:00.288 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.280+0000 7f63d03c4640 1 --2- 172.21.15.88:0/1041083950 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f63ac0664a0 0x7f63ac068960 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f63c40af000 tx=0x7f63c40a7000 comp rx=0 tx=0).stop 2023-10-16T20:08:00.288 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.280+0000 7f63d03c4640 1 -- 172.21.15.88:0/1041083950 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63c80cfe10 msgr2=0x7f63c814dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:00.288 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.280+0000 7f63d03c4640 1 --2- 172.21.15.88:0/1041083950 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63c80cfe10 0x7f63c814dc90 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f63b800b410 tx=0x7f63b800b8d0 comp rx=0 tx=0).stop 2023-10-16T20:08:00.289 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.280+0000 7f63d03c4640 1 -- 172.21.15.88:0/1041083950 shutdown_connections 2023-10-16T20:08:00.289 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.280+0000 7f63d03c4640 1 --2- 172.21.15.88:0/1041083950 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f63ac0664a0 0x7f63ac068960 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:00.289 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.280+0000 7f63d03c4640 1 --2- 172.21.15.88:0/1041083950 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63c815cdb0 0x7f63c8149dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:00.289 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.280+0000 7f63d03c4640 1 --2- 172.21.15.88:0/1041083950 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63c80cfe10 0x7f63c814dc90 secure :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f63b800b410 tx=0x7f63b800b8d0 comp rx=0 tx=0).stop 2023-10-16T20:08:00.290 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.280+0000 7f63d03c4640 1 --2- 172.21.15.88:0/1041083950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63c8069e90 0x7f63c814d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:00.290 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.280+0000 7f63d03c4640 1 -- 172.21.15.88:0/1041083950 >> 172.21.15.88:0/1041083950 conn(0x7f63c8098a20 msgr2=0x7f63c806d450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:00.290 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.284+0000 7f63d03c4640 1 -- 172.21.15.88:0/1041083950 shutdown_connections 2023-10-16T20:08:00.291 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.284+0000 7f63d03c4640 1 -- 172.21.15.88:0/1041083950 wait complete. 2023-10-16T20:08:00.317 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T20:08:00.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.452+0000 7f83f09a1640 1 Processor -- start 2023-10-16T20:08:00.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.452+0000 7f83f09a1640 1 -- start start 2023-10-16T20:08:00.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.452+0000 7f83f09a1640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f83ec071a70 0x7f83ec161540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:00.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.452+0000 7f83f09a1640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83ec161a80 con 0x7f83ec0ca0b0 2023-10-16T20:08:00.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.452+0000 7f83f09a1640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83ec0ca840 con 0x7f83ec15cf90 2023-10-16T20:08:00.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.452+0000 7f83f09a1640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83ec0cab20 con 0x7f83ec071a70 2023-10-16T20:08:00.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.452+0000 7f83ead9d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f83ec071a70 0x7f83ec161540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:00.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.452+0000 7f83e3fff640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f83ec0ca0b0 0x7f83ec071360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:56460/0 (socket says 172.21.15.88:56460) 2023-10-16T20:08:00.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.452+0000 7f83e3fff640 1 -- 172.21.15.88:0/4165552200 learned_addr learned my addr 172.21.15.88:0/4165552200 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:08:00.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.452+0000 7f83ead9d640 1 -- 172.21.15.88:0/4165552200 >> v1:172.21.15.94:6789/0 conn(0x7f83ec15cf90 legacy=0x7f83ec0cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:00.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.452+0000 7f83ead9d640 1 -- 172.21.15.88:0/4165552200 >> v1:172.21.15.88:6789/0 conn(0x7f83ec0ca0b0 legacy=0x7f83ec071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:00.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.452+0000 7f83ead9d640 1 -- 172.21.15.88:0/4165552200 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ec0cae00 con 0x7f83ec071a70 2023-10-16T20:08:00.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.456+0000 7f83ead9d640 1 --2- 172.21.15.88:0/4165552200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83ec071a70 0x7f83ec161540 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f83e4009d90 tx=0x7f83e401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04cfb88dafb993a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:00.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.456+0000 7f83e9d9b640 1 -- 172.21.15.88:0/4165552200 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e40a5900 con 0x7f83ec071a70 2023-10-16T20:08:00.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.456+0000 7f83e9d9b640 1 -- 172.21.15.88:0/4165552200 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e40a5a60 con 0x7f83ec071a70 2023-10-16T20:08:00.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.456+0000 7f83e9d9b640 1 -- 172.21.15.88:0/4165552200 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e40a5df0 con 0x7f83ec071a70 2023-10-16T20:08:00.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.456+0000 7f83f09a1640 1 -- 172.21.15.88:0/4165552200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83ec071a70 msgr2=0x7f83ec161540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:00.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.456+0000 7f83f09a1640 1 --2- 172.21.15.88:0/4165552200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83ec071a70 0x7f83ec161540 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f83e4009d90 tx=0x7f83e401c950 comp rx=0 tx=0).stop 2023-10-16T20:08:00.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.456+0000 7f83f09a1640 1 -- 172.21.15.88:0/4165552200 shutdown_connections 2023-10-16T20:08:00.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.456+0000 7f83f09a1640 1 --2- 172.21.15.88:0/4165552200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83ec071a70 0x7f83ec161540 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:00.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.456+0000 7f83f09a1640 1 -- 172.21.15.88:0/4165552200 >> 172.21.15.88:0/4165552200 conn(0x7f83ec0cf710 msgr2=0x7f83ec0cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:00.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.456+0000 7f83f09a1640 1 -- 172.21.15.88:0/4165552200 shutdown_connections 2023-10-16T20:08:00.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.456+0000 7f83f09a1640 1 -- 172.21.15.88:0/4165552200 wait complete. 2023-10-16T20:08:00.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.460+0000 7f83f09a1640 1 Processor -- start 2023-10-16T20:08:00.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.460+0000 7f83f09a1640 1 -- start start 2023-10-16T20:08:00.469 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.460+0000 7f83f09a1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83ec071a70 0x7f83ec14a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:00.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.460+0000 7f83f09a1640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f83ec0ca0b0 0x7f83ec152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:00.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.460+0000 7f83f09a1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83ec15cf90 0x7f83ec1528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:00.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.460+0000 7f83f09a1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec0c6910 con 0x7f83ec15cf90 2023-10-16T20:08:00.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.460+0000 7f83f09a1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec0c6bf0 con 0x7f83ec071a70 2023-10-16T20:08:00.470 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.460+0000 7f83f09a1640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec0c6ed0 con 0x7f83ec0ca0b0 2023-10-16T20:08:00.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.464+0000 7f83ea59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83ec071a70 0x7f83ec14a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:00.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.464+0000 7f83ea59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83ec071a70 0x7f83ec14a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:50546/0 (socket says 172.21.15.88:50546) 2023-10-16T20:08:00.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.464+0000 7f83ea59c640 1 -- 172.21.15.88:0/63875180 learned_addr learned my addr 172.21.15.88:0/63875180 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:00.471 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.464+0000 7f83ead9d640 1 --2- 172.21.15.88:0/63875180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83ec15cf90 0x7f83ec1528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:00.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.464+0000 7f83ead9d640 1 -- 172.21.15.88:0/63875180 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f83ec0ca0b0 msgr2=0x7f83ec152380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:00.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.464+0000 7f83ead9d640 1 --2- 172.21.15.88:0/63875180 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f83ec0ca0b0 0x7f83ec152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:00.472 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.464+0000 7f83ead9d640 1 -- 172.21.15.88:0/63875180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83ec071a70 msgr2=0x7f83ec14a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:00.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.464+0000 7f83ead9d640 1 --2- 172.21.15.88:0/63875180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83ec071a70 0x7f83ec14a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:00.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.464+0000 7f83ead9d640 1 -- 172.21.15.88:0/63875180 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e4009a70 con 0x7f83ec15cf90 2023-10-16T20:08:00.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.464+0000 7f83ead9d640 1 --2- 172.21.15.88:0/63875180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83ec15cf90 0x7f83ec1528c0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f83e40ac000 tx=0x7f83e40196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:00.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.464+0000 7f83e2ffd640 1 -- 172.21.15.88:0/63875180 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e40b9860 con 0x7f83ec15cf90 2023-10-16T20:08:00.473 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.464+0000 7f83e2ffd640 1 -- 172.21.15.88:0/63875180 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e40b99c0 con 0x7f83ec15cf90 2023-10-16T20:08:00.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.464+0000 7f83e2ffd640 1 -- 172.21.15.88:0/63875180 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e40b9d30 con 0x7f83ec15cf90 2023-10-16T20:08:00.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.464+0000 7f83f09a1640 1 -- 172.21.15.88:0/63875180 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83ec14fb10 con 0x7f83ec15cf90 2023-10-16T20:08:00.474 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.464+0000 7f83f09a1640 1 -- 172.21.15.88:0/63875180 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83ec14ad60 con 0x7f83ec15cf90 2023-10-16T20:08:00.475 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.468+0000 7f83f09a1640 1 -- 172.21.15.88:0/63875180 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83ec04e9c0 con 0x7f83ec15cf90 2023-10-16T20:08:00.476 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.472+0000 7f83e2ffd640 1 -- 172.21.15.88:0/63875180 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f83e40ad440 con 0x7f83ec15cf90 2023-10-16T20:08:00.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.472+0000 7f83e2ffd640 1 --2- 172.21.15.88:0/63875180 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f83cc0664a0 0x7f83cc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:00.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.472+0000 7f83e2ffd640 1 -- 172.21.15.88:0/63875180 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f83e411f090 con 0x7f83ec15cf90 2023-10-16T20:08:00.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.472+0000 7f83ea59c640 1 --2- 172.21.15.88:0/63875180 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f83cc0664a0 0x7f83cc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:00.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.476+0000 7f83ea59c640 1 --2- 172.21.15.88:0/63875180 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f83cc0664a0 0x7f83cc068960 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f83dc001810 tx=0x7f83dc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:00.485 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.480+0000 7f83e2ffd640 1 -- 172.21.15.88:0/63875180 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f83e40ec760 con 0x7f83ec15cf90 2023-10-16T20:08:00.635 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.628+0000 7f83f09a1640 1 -- 172.21.15.88:0/63875180 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f83ec14ffc0 con 0x7f83ec15cf90 2023-10-16T20:08:00.639 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.632+0000 7f83e2ffd640 1 -- 172.21.15.88:0/63875180 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f83e40efd80 con 0x7f83ec15cf90 2023-10-16T20:08:00.642 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.636+0000 7f83f09a1640 1 -- 172.21.15.88:0/63875180 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f83cc0664a0 msgr2=0x7f83cc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:00.642 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.636+0000 7f83f09a1640 1 --2- 172.21.15.88:0/63875180 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f83cc0664a0 0x7f83cc068960 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f83dc001810 tx=0x7f83dc009040 comp rx=0 tx=0).stop 2023-10-16T20:08:00.642 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.636+0000 7f83f09a1640 1 -- 172.21.15.88:0/63875180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83ec15cf90 msgr2=0x7f83ec1528c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:00.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.636+0000 7f83f09a1640 1 --2- 172.21.15.88:0/63875180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83ec15cf90 0x7f83ec1528c0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f83e40ac000 tx=0x7f83e40196f0 comp rx=0 tx=0).stop 2023-10-16T20:08:00.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.636+0000 7f83f09a1640 1 -- 172.21.15.88:0/63875180 shutdown_connections 2023-10-16T20:08:00.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.636+0000 7f83f09a1640 1 --2- 172.21.15.88:0/63875180 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f83cc0664a0 0x7f83cc068960 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:00.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.636+0000 7f83f09a1640 1 --2- 172.21.15.88:0/63875180 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f83ec0ca0b0 0x7f83ec152380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:00.643 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.636+0000 7f83f09a1640 1 --2- 172.21.15.88:0/63875180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83ec15cf90 0x7f83ec1528c0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:00.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.636+0000 7f83f09a1640 1 --2- 172.21.15.88:0/63875180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83ec071a70 0x7f83ec14a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:00.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.636+0000 7f83f09a1640 1 -- 172.21.15.88:0/63875180 >> 172.21.15.88:0/63875180 conn(0x7f83ec0cf710 msgr2=0x7f83ec06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:00.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.636+0000 7f83f09a1640 1 -- 172.21.15.88:0/63875180 shutdown_connections 2023-10-16T20:08:00.644 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.636+0000 7f83f09a1640 1 -- 172.21.15.88:0/63875180 wait complete. 2023-10-16T20:08:00.661 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm' 2023-10-16T20:08:00.799 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.792+0000 7f5e146ea640 1 Processor -- start 2023-10-16T20:08:00.799 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.792+0000 7f5e146ea640 1 -- start start 2023-10-16T20:08:00.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.796+0000 7f5e146ea640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5e0c15cb80 0x7f5e0c0b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:00.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.796+0000 7f5e146ea640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e0c06b490 con 0x7f5e0c0b28b0 2023-10-16T20:08:00.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.796+0000 7f5e146ea640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e0c0cf180 con 0x7f5e0c0cebd0 2023-10-16T20:08:00.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.796+0000 7f5e146ea640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e0c0b2d20 con 0x7f5e0c15cb80 2023-10-16T20:08:00.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.796+0000 7f5e11c85640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f5e0c0b28b0 0x7f5e0c0ce4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:56470/0 (socket says 172.21.15.88:56470) 2023-10-16T20:08:00.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.796+0000 7f5e11c85640 1 -- 172.21.15.88:0/4239897099 learned_addr learned my addr 172.21.15.88:0/4239897099 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:08:00.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.796+0000 7f5e12486640 1 --2- 172.21.15.88:0/4239897099 >> v2:172.21.15.88:3300/0 conn(0x7f5e0c15cb80 0x7f5e0c0b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:00.803 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.796+0000 7f5e12486640 1 -- 172.21.15.88:0/4239897099 >> v1:172.21.15.94:6789/0 conn(0x7f5e0c0cebd0 legacy=0x7f5e0c06bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:00.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.796+0000 7f5e12486640 1 -- 172.21.15.88:0/4239897099 >> v1:172.21.15.88:6789/0 conn(0x7f5e0c0b28b0 legacy=0x7f5e0c0ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:00.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.796+0000 7f5e11484640 1 -- 172.21.15.88:0/4239897099 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1332112364 0 0) 0x7f5e0c06b490 con 0x7f5e0c0b28b0 2023-10-16T20:08:00.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.796+0000 7f5e12486640 1 -- 172.21.15.88:0/4239897099 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e0c06b700 con 0x7f5e0c15cb80 2023-10-16T20:08:00.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.796+0000 7f5e12486640 1 --2- 172.21.15.88:0/4239897099 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e0c15cb80 0x7f5e0c0b2370 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f5e080097c0 tx=0x7f5e0801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d69f15dd7230172a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:00.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.796+0000 7f5e11484640 1 -- 172.21.15.88:0/4239897099 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e080ab600 con 0x7f5e0c15cb80 2023-10-16T20:08:00.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.800+0000 7f5e11484640 1 -- 172.21.15.88:0/4239897099 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e080a7050 con 0x7f5e0c15cb80 2023-10-16T20:08:00.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.800+0000 7f5e11484640 1 -- 172.21.15.88:0/4239897099 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e080ab980 con 0x7f5e0c15cb80 2023-10-16T20:08:00.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.800+0000 7f5e146ea640 1 -- 172.21.15.88:0/4239897099 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e0c15cb80 msgr2=0x7f5e0c0b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:00.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.800+0000 7f5e146ea640 1 --2- 172.21.15.88:0/4239897099 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e0c15cb80 0x7f5e0c0b2370 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f5e080097c0 tx=0x7f5e0801c480 comp rx=0 tx=0).stop 2023-10-16T20:08:00.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.800+0000 7f5e146ea640 1 -- 172.21.15.88:0/4239897099 shutdown_connections 2023-10-16T20:08:00.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.800+0000 7f5e146ea640 1 --2- 172.21.15.88:0/4239897099 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e0c15cb80 0x7f5e0c0b2370 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:00.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.800+0000 7f5e146ea640 1 -- 172.21.15.88:0/4239897099 >> 172.21.15.88:0/4239897099 conn(0x7f5e0c0cd370 msgr2=0x7f5e0c0cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:00.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.800+0000 7f5e146ea640 1 -- 172.21.15.88:0/4239897099 shutdown_connections 2023-10-16T20:08:00.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.800+0000 7f5e146ea640 1 -- 172.21.15.88:0/4239897099 wait complete. 2023-10-16T20:08:00.809 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.800+0000 7f5e146ea640 1 Processor -- start 2023-10-16T20:08:00.810 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.800+0000 7f5e146ea640 1 -- start start 2023-10-16T20:08:00.810 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.800+0000 7f5e146ea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e0c0b28b0 0x7f5e0c0b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:00.810 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.800+0000 7f5e146ea640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5e0c0cebd0 0x7f5e0c15e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:00.810 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.800+0000 7f5e146ea640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e0c15cb80 0x7f5e0c153990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:00.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.800+0000 7f5e146ea640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c15f7f0 con 0x7f5e0c15cb80 2023-10-16T20:08:00.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.800+0000 7f5e146ea640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c0c7d80 con 0x7f5e0c0b28b0 2023-10-16T20:08:00.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.800+0000 7f5e146ea640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c0c8060 con 0x7f5e0c0cebd0 2023-10-16T20:08:00.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.800+0000 7f5e12c87640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e0c15cb80 0x7f5e0c153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:00.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.800+0000 7f5e11c85640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5e0c0cebd0 0x7f5e0c15e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:00.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.800+0000 7f5e12486640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e0c0b28b0 0x7f5e0c0b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:00.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.800+0000 7f5e11c85640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5e0c0cebd0 0x7f5e0c15e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:55688/0 (socket says 172.21.15.88:55688) 2023-10-16T20:08:00.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.800+0000 7f5e11c85640 1 -- 172.21.15.88:0/1114198529 learned_addr learned my addr 172.21.15.88:0/1114198529 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:00.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.804+0000 7f5e12c87640 1 -- 172.21.15.88:0/1114198529 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5e0c0cebd0 msgr2=0x7f5e0c15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:00.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.804+0000 7f5e12c87640 1 --2- 172.21.15.88:0/1114198529 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5e0c0cebd0 0x7f5e0c15e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:00.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.804+0000 7f5e12c87640 1 -- 172.21.15.88:0/1114198529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e0c0b28b0 msgr2=0x7f5e0c0b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:00.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.804+0000 7f5e12c87640 1 --2- 172.21.15.88:0/1114198529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e0c0b28b0 0x7f5e0c0b5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:00.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.804+0000 7f5e12c87640 1 -- 172.21.15.88:0/1114198529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e08009420 con 0x7f5e0c15cb80 2023-10-16T20:08:00.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.804+0000 7f5e12c87640 1 --2- 172.21.15.88:0/1114198529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e0c15cb80 0x7f5e0c153990 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f5e0400c700 tx=0x7f5e0400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:00.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.804+0000 7f5e02ffd640 1 -- 172.21.15.88:0/1114198529 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e04003b80 con 0x7f5e0c15cb80 2023-10-16T20:08:00.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.804+0000 7f5e02ffd640 1 -- 172.21.15.88:0/1114198529 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e04003ce0 con 0x7f5e0c15cb80 2023-10-16T20:08:00.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.804+0000 7f5e146ea640 1 -- 172.21.15.88:0/1114198529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e0c0c8340 con 0x7f5e0c15cb80 2023-10-16T20:08:00.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.804+0000 7f5e02ffd640 1 -- 172.21.15.88:0/1114198529 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0400aa30 con 0x7f5e0c15cb80 2023-10-16T20:08:00.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.804+0000 7f5e146ea640 1 -- 172.21.15.88:0/1114198529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e0c0b6320 con 0x7f5e0c15cb80 2023-10-16T20:08:00.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.804+0000 7f5e146ea640 1 -- 172.21.15.88:0/1114198529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e0c0b46d0 con 0x7f5e0c15cb80 2023-10-16T20:08:00.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.804+0000 7f5e02ffd640 1 -- 172.21.15.88:0/1114198529 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5e0400ab90 con 0x7f5e0c15cb80 2023-10-16T20:08:00.815 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.804+0000 7f5e02ffd640 1 --2- 172.21.15.88:0/1114198529 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5df00664a0 0x7f5df0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:00.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.804+0000 7f5e02ffd640 1 -- 172.21.15.88:0/1114198529 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f5e04083920 con 0x7f5e0c15cb80 2023-10-16T20:08:00.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.804+0000 7f5e12486640 1 --2- 172.21.15.88:0/1114198529 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5df00664a0 0x7f5df0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:00.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.808+0000 7f5e12486640 1 --2- 172.21.15.88:0/1114198529 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5df00664a0 0x7f5df0068960 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5e080ad000 tx=0x7f5e08008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:00.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:00.808+0000 7f5e02ffd640 1 -- 172.21.15.88:0/1114198529 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5e0c0b46d0 con 0x7f5e0c15cb80 2023-10-16T20:08:01.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:01.012+0000 7f5e146ea640 1 -- 172.21.15.88:0/1114198529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]} v 0) v1 -- 0x7f5e0c0b46d0 con 0x7f5e0c15cb80 2023-10-16T20:08:01.288 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:01.284+0000 7f5e02ffd640 1 -- 172.21.15.88:0/1114198529 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]}]=0 v187) v1 ==== 134+0+0 (secure 0 0 0) 0x7f5e0c0b46d0 con 0x7f5e0c15cb80 2023-10-16T20:08:01.290 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:01.284+0000 7f5e146ea640 1 -- 172.21.15.88:0/1114198529 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5df00664a0 msgr2=0x7f5df0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:01.291 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:01.284+0000 7f5e146ea640 1 --2- 172.21.15.88:0/1114198529 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5df00664a0 0x7f5df0068960 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5e080ad000 tx=0x7f5e08008040 comp rx=0 tx=0).stop 2023-10-16T20:08:01.291 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:01.284+0000 7f5e146ea640 1 -- 172.21.15.88:0/1114198529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e0c15cb80 msgr2=0x7f5e0c153990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:01.291 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:01.284+0000 7f5e146ea640 1 --2- 172.21.15.88:0/1114198529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e0c15cb80 0x7f5e0c153990 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f5e0400c700 tx=0x7f5e0400cbc0 comp rx=0 tx=0).stop 2023-10-16T20:08:01.291 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:01.284+0000 7f5e146ea640 1 -- 172.21.15.88:0/1114198529 shutdown_connections 2023-10-16T20:08:01.291 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:01.284+0000 7f5e146ea640 1 --2- 172.21.15.88:0/1114198529 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f5df00664a0 0x7f5df0068960 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:01.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:01.284+0000 7f5e146ea640 1 --2- 172.21.15.88:0/1114198529 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5e0c0cebd0 0x7f5e0c15e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:01.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:01.284+0000 7f5e146ea640 1 --2- 172.21.15.88:0/1114198529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e0c15cb80 0x7f5e0c153990 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:01.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:01.284+0000 7f5e146ea640 1 --2- 172.21.15.88:0/1114198529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e0c0b28b0 0x7f5e0c0b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:01.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:01.284+0000 7f5e146ea640 1 -- 172.21.15.88:0/1114198529 >> 172.21.15.88:0/1114198529 conn(0x7f5e0c0cd370 msgr2=0x7f5e0c0b1530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:01.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:01.284+0000 7f5e146ea640 1 -- 172.21.15.88:0/1114198529 shutdown_connections 2023-10-16T20:08:01.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:01.284+0000 7f5e146ea640 1 -- 172.21.15.88:0/1114198529 wait complete. 2023-10-16T20:08:01.310 INFO:tasks.cephfs_test_runner:test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) ... ERROR 2023-10-16T20:08:01.311 INFO:tasks.cephfs_test_runner: 2023-10-16T20:08:01.311 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-16T20:08:01.312 INFO:tasks.cephfs_test_runner:ERROR: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-10-16T20:08:01.312 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-16T20:08:01.312 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-16T20:08:01.312 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_19fd96fe3581520b6d9f50ffdabf29051aee5d4d/qa/tasks/cephfs/test_pool_perm.py", line 52, in test_pool_perm 2023-10-16T20:08:01.313 INFO:tasks.cephfs_test_runner: self.mount_a.run_python(remote_script.format(path=file_path, check_read=str(True))) 2023-10-16T20:08:01.313 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_19fd96fe3581520b6d9f50ffdabf29051aee5d4d/qa/tasks/cephfs/mount.py", line 775, in run_python 2023-10-16T20:08:01.313 INFO:tasks.cephfs_test_runner: p.wait() 2023-10-16T20:08:01.313 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait 2023-10-16T20:08:01.314 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-10-16T20:08:01.314 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-10-16T20:08:01.314 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-10-16T20:08:01.314 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi088 with status 1: 'stdin-killer -- python3 -c \'\nimport os\nimport errno\n\nfd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR)\ntry:\n if True:\n ret = os.read(fd, 1024)\n else:\n os.write(fd, b\'"\'"\'content\'"\'"\')\nexcept OSError as e:\n if e.errno != errno.EPERM:\n raise\nelse:\n raise RuntimeError("client does not check permission of data pool")\n\'' 2023-10-16T20:08:01.315 INFO:tasks.cephfs_test_runner: 2023-10-16T20:08:01.315 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-16T20:08:01.315 INFO:tasks.cephfs_test_runner:Ran 2 tests in 139.661s 2023-10-16T20:08:01.315 INFO:tasks.cephfs_test_runner: 2023-10-16T20:08:01.316 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-10-16T20:08:01.316 INFO:tasks.cephfs_test_runner: 2023-10-16T20:08:01.316 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-16T20:08:01.317 INFO:tasks.cephfs_test_runner:ERROR: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-10-16T20:08:01.317 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-16T20:08:01.317 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-16T20:08:01.317 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_19fd96fe3581520b6d9f50ffdabf29051aee5d4d/qa/tasks/cephfs/test_pool_perm.py", line 52, in test_pool_perm 2023-10-16T20:08:01.318 INFO:tasks.cephfs_test_runner: self.mount_a.run_python(remote_script.format(path=file_path, check_read=str(True))) 2023-10-16T20:08:01.318 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_19fd96fe3581520b6d9f50ffdabf29051aee5d4d/qa/tasks/cephfs/mount.py", line 775, in run_python 2023-10-16T20:08:01.318 INFO:tasks.cephfs_test_runner: p.wait() 2023-10-16T20:08:01.319 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait 2023-10-16T20:08:01.319 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-10-16T20:08:01.319 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-10-16T20:08:01.319 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-10-16T20:08:01.320 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi088 with status 1: 'stdin-killer -- python3 -c \'\nimport os\nimport errno\n\nfd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR)\ntry:\n if True:\n ret = os.read(fd, 1024)\n else:\n os.write(fd, b\'"\'"\'content\'"\'"\')\nexcept OSError as e:\n if e.errno != errno.EPERM:\n raise\nelse:\n raise RuntimeError("client does not check permission of data pool")\n\'' 2023-10-16T20:08:01.320 INFO:tasks.cephfs_test_runner: 2023-10-16T20:08:01.320 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_19fd96fe3581520b6d9f50ffdabf29051aee5d4d/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-10-16T20:08:01.392 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=1dfcb444471f47cebfef2520438b1ac0 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_19fd96fe3581520b6d9f50ffdabf29051aee5d4d/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-10-16T20:08:01.394 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-16T20:08:01.403 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-10-16T20:08:01.424 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-10-16T20:08:01.425 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:08:01.425 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:08:01.448 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354668k,nr_inodes=4088667,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9372 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T20:08:01.448 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-10-16T20:08:01.448 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:08:01.495 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354668k,nr_inodes=4088667,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9372 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T20:08:01.496 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:08:01.496 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:08:01.507 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354624k,nr_inodes=4088656,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5605 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T20:08:01.507 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2023-10-16T20:08:01.507 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T20:08:01.560 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354624k,nr_inodes=4088656,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5605 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T20:08:01.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:08:01.561 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2023-10-16T20:08:01.571 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T20:08:01.572 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T20:08:01.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:08:01.573 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T20:08:01.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:08:01.637 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T20:08:01.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:08:01.724 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link delete ceph-brx 2023-10-16T20:08:01.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:08:01.903 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2023-10-16T20:08:01.913 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-16T20:08:01.914 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-16T20:08:01.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:08:01.916 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T20:08:01.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:08:01.987 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-16T20:08:02.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T20:08:02.068 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2023-10-16T20:08:02.242 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-16T20:08:02.276 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-16T20:08:02.277 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T20:08:02.425 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.420+0000 7fb58a268640 1 Processor -- start 2023-10-16T20:08:02.426 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.420+0000 7fb58a268640 1 -- start start 2023-10-16T20:08:02.428 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.420+0000 7fb58a268640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb5840d22e0 0x7fb584161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:02.428 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.420+0000 7fb58a268640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb58415d9f0 0x7fb5840cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:02.429 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.420+0000 7fb58a268640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5840b87e0 con 0x7fb5840d26c0 2023-10-16T20:08:02.429 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.420+0000 7fb58a268640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb58415df10 con 0x7fb58415d9f0 2023-10-16T20:08:02.429 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.420+0000 7fb58a268640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb58415e1f0 con 0x7fb5840d22e0 2023-10-16T20:08:02.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.420+0000 7fb5837fe640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fb5840d26c0 0x7fb58415d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:46828/0 (socket says 172.21.15.88:46828) 2023-10-16T20:08:02.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.420+0000 7fb5837fe640 1 -- 172.21.15.88:0/3122563654 learned_addr learned my addr 172.21.15.88:0/3122563654 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:08:02.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.420+0000 7fb583fff640 1 --2- 172.21.15.88:0/3122563654 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb5840d22e0 0x7fb584161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:02.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.420+0000 7fb588805640 1 --2- 172.21.15.88:0/3122563654 >> v2:172.21.15.94:3300/0 conn(0x7fb58415d9f0 0x7fb5840cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:02.431 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.420+0000 7fb582ffd640 1 -- 172.21.15.88:0/3122563654 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3983129777 0 0) 0x7fb5840b87e0 con 0x7fb5840d26c0 2023-10-16T20:08:02.431 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.420+0000 7fb582ffd640 1 -- 172.21.15.88:0/3122563654 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb56c003530 con 0x7fb5840d26c0 2023-10-16T20:08:02.431 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.420+0000 7fb583fff640 1 -- 172.21.15.88:0/3122563654 >> v2:172.21.15.94:3300/0 conn(0x7fb58415d9f0 msgr2=0x7fb5840cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:08:02.432 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.420+0000 7fb583fff640 1 --2- 172.21.15.88:0/3122563654 >> v2:172.21.15.94:3300/0 conn(0x7fb58415d9f0 0x7fb5840cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:02.432 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.420+0000 7fb583fff640 1 -- 172.21.15.88:0/3122563654 >> v1:172.21.15.94:6789/0 conn(0x7fb5840d26c0 legacy=0x7fb58415d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:02.432 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.420+0000 7fb583fff640 1 -- 172.21.15.88:0/3122563654 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb584160970 con 0x7fb5840d22e0 2023-10-16T20:08:02.433 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.420+0000 7fb588805640 1 --2- 172.21.15.88:0/3122563654 >> v2:172.21.15.94:3300/0 conn(0x7fb58415d9f0 0x7fb5840cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:08:02.437 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.424+0000 7fb583fff640 1 --2- 172.21.15.88:0/3122563654 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb5840d22e0 0x7fb584161450 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb5740097c0 tx=0x7fb57401c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50e3d543f7fecb3a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:02.437 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.424+0000 7fb582ffd640 1 -- 172.21.15.88:0/3122563654 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5740ac620 con 0x7fb5840d22e0 2023-10-16T20:08:02.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.424+0000 7fb582ffd640 1 -- 172.21.15.88:0/3122563654 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5740ac780 con 0x7fb5840d22e0 2023-10-16T20:08:02.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.424+0000 7fb582ffd640 1 -- 172.21.15.88:0/3122563654 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5740acb10 con 0x7fb5840d22e0 2023-10-16T20:08:02.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.424+0000 7fb58a268640 1 -- 172.21.15.88:0/3122563654 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb5840d22e0 msgr2=0x7fb584161450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:02.438 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.424+0000 7fb58a268640 1 --2- 172.21.15.88:0/3122563654 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb5840d22e0 0x7fb584161450 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb5740097c0 tx=0x7fb57401c3c0 comp rx=0 tx=0).stop 2023-10-16T20:08:02.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.424+0000 7fb58a268640 1 -- 172.21.15.88:0/3122563654 shutdown_connections 2023-10-16T20:08:02.439 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.424+0000 7fb58a268640 1 --2- 172.21.15.88:0/3122563654 >> v2:172.21.15.94:3300/0 conn(0x7fb58415d9f0 0x7fb5840cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:02.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.424+0000 7fb58a268640 1 --2- 172.21.15.88:0/3122563654 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb5840d22e0 0x7fb584161450 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:02.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.424+0000 7fb58a268640 1 -- 172.21.15.88:0/3122563654 >> 172.21.15.88:0/3122563654 conn(0x7fb5840715c0 msgr2=0x7fb5840719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:02.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.424+0000 7fb58a268640 1 -- 172.21.15.88:0/3122563654 shutdown_connections 2023-10-16T20:08:02.440 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.424+0000 7fb58a268640 1 -- 172.21.15.88:0/3122563654 wait complete. 2023-10-16T20:08:02.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.428+0000 7fb58a268640 1 Processor -- start 2023-10-16T20:08:02.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.428+0000 7fb58a268640 1 -- start start 2023-10-16T20:08:02.441 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.428+0000 7fb58a268640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5840d22e0 0x7fb584156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:02.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.428+0000 7fb58a268640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5840d26c0 0x7fb5840b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:02.442 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.428+0000 7fb58a268640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb58415d9f0 0x7fb584156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:02.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.428+0000 7fb58a268640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5840ccbb0 con 0x7fb5840d22e0 2023-10-16T20:08:02.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.428+0000 7fb58a268640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5840cc5a0 con 0x7fb5840d26c0 2023-10-16T20:08:02.443 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.428+0000 7fb58a268640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb584148f40 con 0x7fb58415d9f0 2023-10-16T20:08:02.444 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.428+0000 7fb583fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5840d22e0 0x7fb584156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:02.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.428+0000 7fb583fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5840d22e0 0x7fb584156100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:59876/0 (socket says 172.21.15.88:59876) 2023-10-16T20:08:02.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.428+0000 7fb583fff640 1 -- 172.21.15.88:0/594199590 learned_addr learned my addr 172.21.15.88:0/594199590 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:02.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.428+0000 7fb588805640 1 --2- 172.21.15.88:0/594199590 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb58415d9f0 0x7fb584156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:02.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.428+0000 7fb583fff640 1 -- 172.21.15.88:0/594199590 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb58415d9f0 msgr2=0x7fb584156890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:02.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.428+0000 7fb5837fe640 1 --2- 172.21.15.88:0/594199590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5840d26c0 0x7fb5840b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:02.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.428+0000 7fb583fff640 1 --2- 172.21.15.88:0/594199590 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb58415d9f0 0x7fb584156890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:02.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.428+0000 7fb583fff640 1 -- 172.21.15.88:0/594199590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5840d26c0 msgr2=0x7fb5840b76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:02.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.428+0000 7fb583fff640 1 --2- 172.21.15.88:0/594199590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5840d26c0 0x7fb5840b76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:02.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.428+0000 7fb583fff640 1 -- 172.21.15.88:0/594199590 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb57c009a70 con 0x7fb5840d22e0 2023-10-16T20:08:02.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.428+0000 7fb588805640 1 --2- 172.21.15.88:0/594199590 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb58415d9f0 0x7fb584156890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:08:02.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.428+0000 7fb5837fe640 1 --2- 172.21.15.88:0/594199590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5840d26c0 0x7fb5840b76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:08:02.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.428+0000 7fb583fff640 1 --2- 172.21.15.88:0/594199590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5840d22e0 0x7fb584156100 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fb5740af040 tx=0x7fb57401ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:02.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.428+0000 7fb580ff9640 1 -- 172.21.15.88:0/594199590 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5740ac620 con 0x7fb5840d22e0 2023-10-16T20:08:02.450 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.428+0000 7fb580ff9640 1 -- 172.21.15.88:0/594199590 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5740b8450 con 0x7fb5840d22e0 2023-10-16T20:08:02.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.428+0000 7fb580ff9640 1 -- 172.21.15.88:0/594199590 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5740b87e0 con 0x7fb5840d22e0 2023-10-16T20:08:02.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.428+0000 7fb58a268640 1 -- 172.21.15.88:0/594199590 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb574009420 con 0x7fb5840d22e0 2023-10-16T20:08:02.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.428+0000 7fb58a268640 1 -- 172.21.15.88:0/594199590 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5840cbe10 con 0x7fb5840d22e0 2023-10-16T20:08:02.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.428+0000 7fb58a268640 1 -- 172.21.15.88:0/594199590 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb58404e9c0 con 0x7fb5840d22e0 2023-10-16T20:08:02.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.432+0000 7fb580ff9640 1 -- 172.21.15.88:0/594199590 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb5740aa070 con 0x7fb5840d22e0 2023-10-16T20:08:02.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.432+0000 7fb580ff9640 1 --2- 172.21.15.88:0/594199590 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb5700664a0 0x7fb570068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:02.455 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.432+0000 7fb580ff9640 1 -- 172.21.15.88:0/594199590 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fb57411e310 con 0x7fb5840d22e0 2023-10-16T20:08:02.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.432+0000 7fb5837fe640 1 --2- 172.21.15.88:0/594199590 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb5700664a0 0x7fb570068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:02.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.432+0000 7fb5837fe640 1 --2- 172.21.15.88:0/594199590 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb5700664a0 0x7fb570068960 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb578005890 tx=0x7fb578009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:02.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.436+0000 7fb580ff9640 1 -- 172.21.15.88:0/594199590 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb5740eb9d0 con 0x7fb5840d22e0 2023-10-16T20:08:02.589 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.584+0000 7fb58a268640 1 -- 172.21.15.88:0/594199590 --> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb5840cfc30 con 0x7fb5700664a0 2023-10-16T20:08:02.591 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.584+0000 7fb580ff9640 1 -- 172.21.15.88:0/594199590 <== mgr.4110 v2:172.21.15.88:6832/4078483333 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50984 (secure 0 0 0) 0x7fb578001420 con 0x7fb5700664a0 2023-10-16T20:08:02.593 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:08:02.597 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2023-10-16T20:08:02.597 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.588+0000 7fb58a268640 1 -- 172.21.15.88:0/594199590 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb5700664a0 msgr2=0x7fb570068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:02.600 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.588+0000 7fb58a268640 1 --2- 172.21.15.88:0/594199590 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb5700664a0 0x7fb570068960 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb578005890 tx=0x7fb578009040 comp rx=0 tx=0).stop 2023-10-16T20:08:02.600 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.588+0000 7fb58a268640 1 -- 172.21.15.88:0/594199590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5840d22e0 msgr2=0x7fb584156100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:02.602 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.588+0000 7fb58a268640 1 --2- 172.21.15.88:0/594199590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5840d22e0 0x7fb584156100 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fb5740af040 tx=0x7fb57401ac00 comp rx=0 tx=0).stop 2023-10-16T20:08:02.602 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.588+0000 7fb58a268640 1 -- 172.21.15.88:0/594199590 shutdown_connections 2023-10-16T20:08:02.603 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.588+0000 7fb58a268640 1 --2- 172.21.15.88:0/594199590 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fb5700664a0 0x7fb570068960 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:02.603 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.588+0000 7fb58a268640 1 --2- 172.21.15.88:0/594199590 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb58415d9f0 0x7fb584156890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:02.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.588+0000 7fb58a268640 1 --2- 172.21.15.88:0/594199590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5840d26c0 0x7fb5840b76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:02.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.588+0000 7fb58a268640 1 --2- 172.21.15.88:0/594199590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5840d22e0 0x7fb584156100 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:02.604 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.588+0000 7fb58a268640 1 -- 172.21.15.88:0/594199590 >> 172.21.15.88:0/594199590 conn(0x7fb5840715c0 msgr2=0x7fb58415bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:02.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.588+0000 7fb58a268640 1 -- 172.21.15.88:0/594199590 shutdown_connections 2023-10-16T20:08:02.605 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.588+0000 7fb58a268640 1 -- 172.21.15.88:0/594199590 wait complete. 2023-10-16T20:08:02.612 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2023-10-16T20:08:02.485024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239500,"kb_used_data":6696,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735220,"statfs":{"total":773094113280,"available":772848865280,"internally_reserved":0,"allocated":6856704,"data_stored":3714343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238276153},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14930,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-65,"num_write_kb":-38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001056"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":198,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213453+0000","last_change":"2023-10-16T20:04:46.335822+0000","last_active":"2023-10-16T20:07:27.213453+0000","last_peered":"2023-10-16T20:07:27.213453+0000","last_clean":"2023-10-16T20:07:27.213453+0000","last_became_active":"2023-10-16T20:04:46.334617+0000","last_became_peered":"2023-10-16T20:04:46.334617+0000","last_unstale":"2023-10-16T20:07:27.213453+0000","last_undegraded":"2023-10-16T20:07:27.213453+0000","last_fullsized":"2023-10-16T20:07:27.213453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:45.306240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:45.306240+0000","last_clean_scrub_stamp":"2023-10-16T20:04:45.306240+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:42:11.153721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509482,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1302528,"data_stored":905075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.69099999999999995,"15min":0.70499999999999996},"min":{"1min":0.45500000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.872,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.318},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.58299999999999996,"5min":0.495,"15min":0.495},"max":{"1min":0.96499999999999997,"5min":1.077,"15min":1.077},"last":0.42199999999999999}]},{"osd":1,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.59299999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98399999999999999,"5min":1.78,"15min":1.78},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.70699999999999996,"15min":0.71499999999999997},"min":{"1min":0.57399999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93100000000000005,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":0.504}]},{"osd":2,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.72399999999999998,"15min":0.74099999999999999},"min":{"1min":0.54800000000000004,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91600000000000004,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.443},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78400000000000003,"15min":0.79500000000000004},"min":{"1min":0.64000000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.94699999999999995,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.64400000000000002}]},{"osd":3,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70499999999999996,"15min":0.70699999999999996},"min":{"1min":0.55100000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.84999999999999998,"5min":1.994,"15min":1.994},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.75700000000000001,"15min":0.78500000000000003},"min":{"1min":0.53200000000000003,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91300000000000003,"5min":2.629,"15min":2.629},"last":0.57799999999999996}]},{"osd":4,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.64900000000000002,"15min":0.66100000000000003},"min":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.98599999999999999,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.64900000000000002,"15min":0.65900000000000003},"min":{"1min":0.48099999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.83499999999999996,"5min":1.944,"15min":1.944},"last":0.72399999999999998}]},{"osd":5,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.72199999999999998,"15min":0.746},"min":{"1min":0.505,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91300000000000003,"5min":2.8809999999999998,"15min":2.8809999999999998},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71099999999999997,"15min":0.71599999999999997},"min":{"1min":0.51900000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94199999999999995,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.68100000000000005}]},{"osd":6,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.755,"15min":0.75700000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.96699999999999997,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.748},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72699999999999998,"15min":0.746},"min":{"1min":0.54200000000000004,"5min":0.438,"15min":0.438},"max":{"1min":0.92900000000000005,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.61799999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509483,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":700416,"data_stored":314547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.81000000000000005,"15min":0.86399999999999999},"min":{"1min":0.46500000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79400000000000004,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.78600000000000003,"15min":0.85399999999999998},"min":{"1min":0.46899999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86299999999999999,"5min":4.6139999999999999,"15min":4.6139999999999999},"last":0.79500000000000004}]},{"osd":1,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.83799999999999997,"15min":0.89000000000000001},"min":{"1min":0.51400000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.202,"5min":4.6639999999999997,"15min":4.6639999999999997},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.95099999999999996,"15min":1.075},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.86099999999999999,"5min":4.8140000000000001,"15min":4.8140000000000001},"last":0.65100000000000002}]},{"osd":2,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.871,"15min":0.95299999999999996},"min":{"1min":0.502,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88100000000000001,"5min":5.3769999999999998,"15min":5.3769999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.68700000000000006,"15min":0.67500000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.94399999999999995,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.58599999999999997}]},{"osd":3,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.86699999999999999,"15min":0.92400000000000004},"min":{"1min":0.60999999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.004,"5min":4.5460000000000003,"15min":4.5460000000000003},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.748,"15min":0.76600000000000001},"min":{"1min":0.53300000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.92300000000000004,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.82199999999999995}]},{"osd":4,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.70899999999999996,"15min":0.74299999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80200000000000005,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.67900000000000005,"15min":0.69499999999999995},"min":{"1min":0.42399999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93500000000000005,"5min":1.355,"15min":1.355},"last":0.71599999999999997}]},{"osd":5,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.69899999999999995,"15min":0.71799999999999997},"min":{"1min":0.441,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95599999999999996,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.78800000000000003,"15min":0.80200000000000005},"min":{"1min":0.52000000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.89000000000000001,"5min":1.218,"15min":1.218},"last":0.77700000000000002}]},{"osd":7,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.73899999999999999,"15min":0.745},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.97099999999999997,"5min":1.052,"15min":1.052},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.99099999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.84199999999999997}]}]},{"osd":5,"up_from":15,"seq":64424509481,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31872,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339968,"statfs":{"total":96636764160,"available":96604127232,"internally_reserved":0,"allocated":708608,"data_stored":315199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.64300000000000002,"15min":0.63200000000000001},"min":{"1min":0.40100000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94599999999999995,"5min":1.264,"15min":1.264},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.51700000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.65500000000000003}]},{"osd":1,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.63300000000000001,"15min":0.61399999999999999},"min":{"1min":0.47299999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.65200000000000002,"15min":0.64200000000000002},"min":{"1min":0.499,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.55600000000000005}]},{"osd":2,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67300000000000004,"15min":0.67100000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67300000000000004,"15min":0.65700000000000003},"min":{"1min":0.47699999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92300000000000004,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.68200000000000005}]},{"osd":3,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.76400000000000001,"15min":0.77800000000000002},"min":{"1min":0.58399999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.96299999999999997,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.71599999999999997,"15min":0.70399999999999996},"min":{"1min":0.52500000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.77900000000000003}]},{"osd":4,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.45400000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92700000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.67900000000000005,"15min":0.69399999999999995},"min":{"1min":0.41199999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.73599999999999999,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.61099999999999999}]},{"osd":6,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.65200000000000002,"15min":0.66300000000000003},"min":{"1min":0.437,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81899999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.53600000000000003,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.78900000000000003,"5min":1.04,"15min":1.04},"last":0.72399999999999998}]},{"osd":7,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.73699999999999999,"15min":0.746},"min":{"1min":0.53700000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.83199999999999996,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.747},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.72699999999999998,"15min":0.74099999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.52800000000000002}]}]},{"osd":4,"up_from":15,"seq":64424509481,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31888,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339952,"statfs":{"total":96636764160,"available":96604110848,"internally_reserved":0,"allocated":724992,"data_stored":330842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.74199999999999999,"15min":0.78600000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79200000000000004,"5min":3.835,"15min":3.835},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.76000000000000001,"15min":0.80700000000000005},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.91700000000000004,"5min":3.7610000000000001,"15min":3.7610000000000001},"last":0.54700000000000004}]},{"osd":1,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66600000000000004,"15min":0.67400000000000004},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.875,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.67400000000000004,"15min":0.67200000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.89100000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.49299999999999999}]},{"osd":2,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.54000000000000004,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.91400000000000003,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.70299999999999996,"15min":0.70699999999999996},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.97099999999999997,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.64300000000000002}]},{"osd":3,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.78800000000000003,"15min":0.80000000000000004},"min":{"1min":0.54900000000000004,"5min":0.44,"15min":0.44},"max":{"1min":1.002,"5min":1.163,"15min":1.163},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.77700000000000002,"15min":0.79100000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.93999999999999995,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.61299999999999999}]},{"osd":5,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.65500000000000003,"15min":0.66800000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85299999999999998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.67500000000000004,"15min":0.69399999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.94799999999999995,"5min":1.089,"15min":1.089},"last":0.66800000000000004}]},{"osd":6,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71299999999999997,"15min":0.73099999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93100000000000005,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.73799999999999999,"15min":0.77500000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.82799999999999996,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.56399999999999995}]},{"osd":7,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.75,"15min":0.77900000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92200000000000004,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.81999999999999995,"15min":0.84699999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.93200000000000005,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.70099999999999996}]}]},{"osd":3,"up_from":15,"seq":64424509482,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":708608,"data_stored":314603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.44900000000000001,"15min":0.45900000000000002},"min":{"1min":0.28399999999999997,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46800000000000003,"15min":0.46600000000000003},"min":{"1min":0.36099999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73499999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.504}]},{"osd":1,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.441,"15min":0.45400000000000001},"min":{"1min":0.29399999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64700000000000002,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.503,"15min":0.51500000000000001},"min":{"1min":0.36099999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.67300000000000004,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.64000000000000001}]},{"osd":2,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.49399999999999999,"15min":0.499},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70799999999999996,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53900000000000003,"15min":0.54500000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72399999999999998,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.66600000000000004}]},{"osd":4,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.76300000000000001,"15min":0.77700000000000002},"min":{"1min":0.56399999999999995,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.014,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.145},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.79800000000000004,"15min":0.81499999999999995},"min":{"1min":0.61699999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.96099999999999997,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.2}]},{"osd":5,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.745,"15min":0.76300000000000001},"min":{"1min":0.56899999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.88,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.76900000000000002,"15min":0.78500000000000003},"min":{"1min":0.56699999999999995,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.89400000000000002,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.92400000000000004}]},{"osd":6,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.752,"15min":0.78000000000000003},"min":{"1min":0.52300000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.90100000000000002,"5min":1.774,"15min":1.774},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.78800000000000003,"15min":0.81200000000000006},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.93000000000000005,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":1}]},{"osd":7,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.78000000000000003,"15min":0.80200000000000005},"min":{"1min":0.52500000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94199999999999995,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":1.075},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.77300000000000002,"15min":0.79900000000000004},"min":{"1min":0.56799999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.84899999999999998,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.94799999999999995}]}]},{"osd":2,"up_from":15,"seq":64424509481,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":708608,"data_stored":314603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40400000000000003,"15min":0.41199999999999998},"min":{"1min":0.26200000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.47199999999999998,"15min":0.48299999999999998},"min":{"1min":0.313,"5min":0.251,"15min":0.251},"max":{"1min":0.60599999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.375}]},{"osd":1,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.497,"5min":0.49099999999999999,"15min":0.48999999999999999},"min":{"1min":0.34300000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.439}]},{"osd":3,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.47099999999999997,"15min":0.47599999999999998},"min":{"1min":0.34100000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.58599999999999997,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.64500000000000002,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.50900000000000001}]},{"osd":4,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.75,"15min":0.76700000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.95399999999999996,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.73899999999999999,"15min":0.755},"min":{"1min":0.55800000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.107,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.71399999999999997}]},{"osd":5,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72099999999999997,"15min":0.73099999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0920000000000001,"5min":1.145,"15min":1.145},"last":0.62},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.73599999999999999,"15min":0.75},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.995,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.65000000000000002}]},{"osd":6,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.74099999999999999,"15min":0.749},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.97299999999999998,"5min":1.165,"15min":1.165},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.754,"15min":0.76600000000000001},"min":{"1min":0.57499999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.056,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.74299999999999999}]},{"osd":7,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.76000000000000001,"15min":0.76700000000000002},"min":{"1min":0.57799999999999996,"5min":0.502,"15min":0.502},"max":{"1min":1.0249999999999999,"5min":1.167,"15min":1.167},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.76900000000000002,"15min":0.78000000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.01,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.69199999999999995}]}]},{"osd":1,"up_from":15,"seq":64424509482,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":692224,"data_stored":308216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.499,"15min":0.54100000000000004},"min":{"1min":0.29199999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.63500000000000001,"5min":3.867,"15min":3.867},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.47999999999999998,"15min":0.49199999999999999},"min":{"1min":0.32600000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.621,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.32300000000000001}]},{"osd":2,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.63100000000000001,"15min":0.71499999999999997},"min":{"1min":0.34799999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.59999999999999998,"5min":5.6040000000000001,"15min":5.6040000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.68899999999999995,"15min":0.78800000000000003},"min":{"1min":0.34999999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65700000000000003,"5min":7.8380000000000001,"15min":7.8380000000000001},"last":0.505}]},{"osd":3,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.64200000000000002,"15min":0.70399999999999996},"min":{"1min":0.39800000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.74099999999999999,"5min":5.2039999999999997,"15min":5.2039999999999997},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.59999999999999998,"15min":0.65400000000000003},"min":{"1min":0.36899999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.67000000000000004,"5min":3.9470000000000001,"15min":3.9470000000000001},"last":0.38700000000000001}]},{"osd":4,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.81799999999999995,"15min":0.85599999999999998},"min":{"1min":0.54300000000000004,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.92200000000000004,"5min":4.1500000000000004,"15min":4.1500000000000004},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.83199999999999996,"15min":0.86799999999999999},"min":{"1min":0.54700000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.93300000000000005,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":0.69399999999999995}]},{"osd":5,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.754,"15min":0.75800000000000001},"min":{"1min":0.56399999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93200000000000005,"5min":1.496,"15min":1.496},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.84199999999999997,"15min":0.88100000000000001},"min":{"1min":0.56999999999999995,"5min":0.501,"15min":0.501},"max":{"1min":0.96999999999999997,"5min":4.0910000000000002,"15min":4.0910000000000002},"last":0.85999999999999999}]},{"osd":6,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.84199999999999997,"15min":0.876},"min":{"1min":0.56999999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.01,"5min":4.21,"15min":4.21},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.75900000000000001,"15min":0.76400000000000001},"min":{"1min":0.57799999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.95299999999999996,"5min":1.401,"15min":1.401},"last":0.754}]},{"osd":7,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.79500000000000004,"15min":0.79800000000000004},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.90600000000000003,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.874},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.83699999999999997,"15min":0.875},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.94099999999999995,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":0.70799999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574891,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28304,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343536,"statfs":{"total":96636764160,"available":96607780864,"internally_reserved":0,"allocated":1310720,"data_stored":911258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.51600000000000001,"15min":0.57599999999999996},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.61499999999999999,"5min":3.9289999999999998,"15min":3.9289999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.54100000000000004,"15min":0.59699999999999998},"min":{"1min":0.34200000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.61099999999999999,"5min":4.0910000000000002,"15min":4.0910000000000002},"last":0.61699999999999999}]},{"osd":2,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.60499999999999998,"15min":0.66200000000000003},"min":{"1min":0.32100000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70999999999999996,"5min":4.1520000000000001,"15min":4.1520000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.63700000000000001,"15min":0.69599999999999995},"min":{"1min":0.42399999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.71399999999999997,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.71099999999999997}]},{"osd":3,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.58699999999999997,"15min":0.63600000000000001},"min":{"1min":0.36499999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.67800000000000005,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55800000000000005,"15min":0.56799999999999995},"min":{"1min":0.45100000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.73599999999999999,"5min":1.006,"15min":1.006},"last":0.75600000000000001}]},{"osd":4,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.79900000000000004,"15min":0.85599999999999998},"min":{"1min":0.52500000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.91100000000000003,"5min":4.5250000000000004,"15min":4.5250000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.73699999999999999,"15min":0.747},"min":{"1min":0.5,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.89500000000000002,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.77900000000000003}]},{"osd":5,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.83899999999999997,"15min":0.89500000000000002},"min":{"1min":0.58199999999999996,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.004,"5min":3.9990000000000001,"15min":3.9990000000000001},"last":1.01},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.877,"15min":0.94499999999999995},"min":{"1min":0.54200000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0189999999999999,"5min":4.4779999999999998,"15min":4.4779999999999998},"last":0.64600000000000002}]},{"osd":6,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.75600000000000001,"15min":0.76600000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.93100000000000005,"5min":1.363,"15min":1.363},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.85699999999999998,"15min":0.92900000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.97099999999999997,"5min":3.956,"15min":3.956},"last":1.052}]},{"osd":7,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.879,"15min":0.93700000000000006},"min":{"1min":0.624,"5min":0.435,"15min":0.435},"max":{"1min":1.034,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":1.085},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.85799999999999998,"15min":0.92600000000000005},"min":{"1min":0.56399999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.97899999999999998,"5min":4.5640000000000001,"15min":4.5640000000000001},"last":0.68500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T20:08:02.615 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T20:08:02.752 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.748+0000 7f971a96e640 1 Processor -- start 2023-10-16T20:08:02.753 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.748+0000 7f971a96e640 1 -- start start 2023-10-16T20:08:02.755 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.748+0000 7f971a96e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f97140cde80 0x7f97140d3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:02.755 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.748+0000 7f971a96e640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f97140cf230 0x7f97140cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:02.756 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.748+0000 7f971a96e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97140d0c40 con 0x7f971415c9a0 2023-10-16T20:08:02.756 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.748+0000 7f971a96e640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97140d37b0 con 0x7f97140cf230 2023-10-16T20:08:02.757 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.748+0000 7f971a96e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97140ce7e0 con 0x7f97140cde80 2023-10-16T20:08:02.757 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.748+0000 7f971916b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f97140cde80 0x7f97140d3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:02.757 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.748+0000 7f971996c640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f971415c9a0 0x7f97140d10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:46830/0 (socket says 172.21.15.88:46830) 2023-10-16T20:08:02.758 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.748+0000 7f971996c640 1 -- 172.21.15.88:0/2772196199 learned_addr learned my addr 172.21.15.88:0/2772196199 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:08:02.758 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.748+0000 7f971a16d640 1 --2- 172.21.15.88:0/2772196199 >> v2:172.21.15.88:3300/0 conn(0x7f97140cf230 0x7f97140cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:02.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.748+0000 7f971896a640 1 -- 172.21.15.88:0/2772196199 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1816637100 0 0) 0x7f97140d0c40 con 0x7f971415c9a0 2023-10-16T20:08:02.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.748+0000 7f971896a640 1 -- 172.21.15.88:0/2772196199 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9700003530 con 0x7f971415c9a0 2023-10-16T20:08:02.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.748+0000 7f971a16d640 1 -- 172.21.15.88:0/2772196199 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f97140cde80 msgr2=0x7f97140d3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:08:02.760 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.748+0000 7f971a16d640 1 --2- 172.21.15.88:0/2772196199 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f97140cde80 0x7f97140d3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:02.760 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.748+0000 7f971a16d640 1 -- 172.21.15.88:0/2772196199 >> v1:172.21.15.94:6789/0 conn(0x7f971415c9a0 legacy=0x7f97140d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:02.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.748+0000 7f971a16d640 1 -- 172.21.15.88:0/2772196199 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97140ce2a0 con 0x7f97140cf230 2023-10-16T20:08:02.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.748+0000 7f971916b640 1 --2- 172.21.15.88:0/2772196199 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f97140cde80 0x7f97140d3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:08:02.767 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.752+0000 7f971a16d640 1 --2- 172.21.15.88:0/2772196199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97140cf230 0x7f97140cf6a0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f970c009d90 tx=0x7f970c01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20505c5ef67fb342 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:02.768 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.752+0000 7f971896a640 1 -- 172.21.15.88:0/2772196199 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c0a5810 con 0x7f97140cf230 2023-10-16T20:08:02.768 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.752+0000 7f971896a640 1 -- 172.21.15.88:0/2772196199 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f970c0a5970 con 0x7f97140cf230 2023-10-16T20:08:02.768 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.752+0000 7f971896a640 1 -- 172.21.15.88:0/2772196199 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c0a5ce0 con 0x7f97140cf230 2023-10-16T20:08:02.769 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.752+0000 7f971a96e640 1 -- 172.21.15.88:0/2772196199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97140cf230 msgr2=0x7f97140cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:02.769 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.752+0000 7f971a96e640 1 --2- 172.21.15.88:0/2772196199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97140cf230 0x7f97140cf6a0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f970c009d90 tx=0x7f970c01c950 comp rx=0 tx=0).stop 2023-10-16T20:08:02.769 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.752+0000 7f971a96e640 1 -- 172.21.15.88:0/2772196199 shutdown_connections 2023-10-16T20:08:02.770 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.752+0000 7f971a96e640 1 --2- 172.21.15.88:0/2772196199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97140cf230 0x7f97140cf6a0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:02.770 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.752+0000 7f971a96e640 1 --2- 172.21.15.88:0/2772196199 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f97140cde80 0x7f97140d3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:02.771 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.752+0000 7f971a96e640 1 -- 172.21.15.88:0/2772196199 >> 172.21.15.88:0/2772196199 conn(0x7f971409bc30 msgr2=0x7f97140d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:02.771 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.752+0000 7f971a96e640 1 -- 172.21.15.88:0/2772196199 shutdown_connections 2023-10-16T20:08:02.771 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.752+0000 7f971a96e640 1 -- 172.21.15.88:0/2772196199 wait complete. 2023-10-16T20:08:02.772 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.756+0000 7f971a96e640 1 Processor -- start 2023-10-16T20:08:02.772 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.756+0000 7f971a96e640 1 -- start start 2023-10-16T20:08:02.772 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.756+0000 7f971a96e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97140cde80 0x7f97140c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:02.773 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.756+0000 7f971a96e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f97140cf230 0x7f97140cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:02.773 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.756+0000 7f971a96e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f971415c9a0 0x7f9714152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:02.774 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.756+0000 7f971a96e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9714154a70 con 0x7f97140cde80 2023-10-16T20:08:02.774 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.756+0000 7f971a96e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97140c6500 con 0x7f971415c9a0 2023-10-16T20:08:02.774 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.756+0000 7f971a96e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97140c76c0 con 0x7f97140cf230 2023-10-16T20:08:02.775 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.756+0000 7f971996c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97140cde80 0x7f97140c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:02.775 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.756+0000 7f971996c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97140cde80 0x7f97140c8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:59898/0 (socket says 172.21.15.88:59898) 2023-10-16T20:08:02.775 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.756+0000 7f971996c640 1 -- 172.21.15.88:0/1335800992 learned_addr learned my addr 172.21.15.88:0/1335800992 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:02.775 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.756+0000 7f971916b640 1 --2- 172.21.15.88:0/1335800992 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f97140cf230 0x7f97140cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:02.776 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.756+0000 7f971a16d640 1 --2- 172.21.15.88:0/1335800992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f971415c9a0 0x7f9714152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:02.776 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.756+0000 7f971996c640 1 -- 172.21.15.88:0/1335800992 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f97140cf230 msgr2=0x7f97140cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:02.776 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.756+0000 7f971996c640 1 --2- 172.21.15.88:0/1335800992 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f97140cf230 0x7f97140cbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:02.777 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.756+0000 7f971996c640 1 -- 172.21.15.88:0/1335800992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f971415c9a0 msgr2=0x7f9714152a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:02.777 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.756+0000 7f971996c640 1 --2- 172.21.15.88:0/1335800992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f971415c9a0 0x7f9714152a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:02.777 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.756+0000 7f971996c640 1 -- 172.21.15.88:0/1335800992 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9704009420 con 0x7f97140cde80 2023-10-16T20:08:02.777 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.756+0000 7f971916b640 1 --2- 172.21.15.88:0/1335800992 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f97140cf230 0x7f97140cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:08:02.778 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.756+0000 7f971a16d640 1 --2- 172.21.15.88:0/1335800992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f971415c9a0 0x7f9714152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:08:02.778 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.756+0000 7f971996c640 1 --2- 172.21.15.88:0/1335800992 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97140cde80 0x7f97140c8e90 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f971000c400 tx=0x7f971000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:02.778 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.756+0000 7f970a7fc640 1 -- 172.21.15.88:0/1335800992 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9710005d40 con 0x7f97140cde80 2023-10-16T20:08:02.778 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.756+0000 7f970a7fc640 1 -- 172.21.15.88:0/1335800992 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97100070d0 con 0x7f97140cde80 2023-10-16T20:08:02.779 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.756+0000 7f971a96e640 1 -- 172.21.15.88:0/1335800992 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f970c009a70 con 0x7f97140cde80 2023-10-16T20:08:02.779 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.756+0000 7f970a7fc640 1 -- 172.21.15.88:0/1335800992 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9710005ea0 con 0x7f97140cde80 2023-10-16T20:08:02.779 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.756+0000 7f971a96e640 1 -- 172.21.15.88:0/1335800992 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f971415cf00 con 0x7f97140cde80 2023-10-16T20:08:02.780 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.756+0000 7f971a96e640 1 -- 172.21.15.88:0/1335800992 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97140c6140 con 0x7f97140cde80 2023-10-16T20:08:02.780 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.760+0000 7f970a7fc640 1 -- 172.21.15.88:0/1335800992 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9710007460 con 0x7f97140cde80 2023-10-16T20:08:02.780 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.760+0000 7f970a7fc640 1 --2- 172.21.15.88:0/1335800992 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f96f80664a0 0x7f96f8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:02.780 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.760+0000 7f970a7fc640 1 -- 172.21.15.88:0/1335800992 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f9710083290 con 0x7f97140cde80 2023-10-16T20:08:02.781 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.760+0000 7f971916b640 1 --2- 172.21.15.88:0/1335800992 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f96f80664a0 0x7f96f8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:02.781 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.760+0000 7f971916b640 1 --2- 172.21.15.88:0/1335800992 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f96f80664a0 0x7f96f8068960 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f97140cae00 tx=0x7f9704008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:02.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.764+0000 7f970a7fc640 1 -- 172.21.15.88:0/1335800992 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f97100508c0 con 0x7f97140cde80 2023-10-16T20:08:02.915 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.908+0000 7f971a96e640 1 -- 172.21.15.88:0/1335800992 --> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f971415cd80 con 0x7f96f80664a0 2023-10-16T20:08:02.917 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.912+0000 7f970a7fc640 1 -- 172.21.15.88:0/1335800992 <== mgr.4110 v2:172.21.15.88:6832/4078483333 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50984 (secure 0 0 0) 0x7f9704009990 con 0x7f96f80664a0 2023-10-16T20:08:02.919 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:08:02.920 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2023-10-16T20:08:02.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.912+0000 7f971a96e640 1 -- 172.21.15.88:0/1335800992 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f96f80664a0 msgr2=0x7f96f8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:02.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.912+0000 7f971a96e640 1 --2- 172.21.15.88:0/1335800992 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f96f80664a0 0x7f96f8068960 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f97140cae00 tx=0x7f9704008040 comp rx=0 tx=0).stop 2023-10-16T20:08:02.921 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.912+0000 7f971a96e640 1 -- 172.21.15.88:0/1335800992 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97140cde80 msgr2=0x7f97140c8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:02.922 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.912+0000 7f971a96e640 1 --2- 172.21.15.88:0/1335800992 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97140cde80 0x7f97140c8e90 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f971000c400 tx=0x7f971000c8c0 comp rx=0 tx=0).stop 2023-10-16T20:08:02.922 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.912+0000 7f971a96e640 1 -- 172.21.15.88:0/1335800992 shutdown_connections 2023-10-16T20:08:02.922 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.912+0000 7f971a96e640 1 --2- 172.21.15.88:0/1335800992 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f96f80664a0 0x7f96f8068960 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:02.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.912+0000 7f971a96e640 1 --2- 172.21.15.88:0/1335800992 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f97140cf230 0x7f97140cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:02.923 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.912+0000 7f971a96e640 1 --2- 172.21.15.88:0/1335800992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f971415c9a0 0x7f9714152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:02.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.912+0000 7f971a96e640 1 --2- 172.21.15.88:0/1335800992 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97140cde80 0x7f97140c8e90 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:02.924 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.912+0000 7f971a96e640 1 -- 172.21.15.88:0/1335800992 >> 172.21.15.88:0/1335800992 conn(0x7f971409bc30 msgr2=0x7f9714155f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:02.925 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.912+0000 7f971a96e640 1 -- 172.21.15.88:0/1335800992 shutdown_connections 2023-10-16T20:08:02.925 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:02.912+0000 7f971a96e640 1 -- 172.21.15.88:0/1335800992 wait complete. 2023-10-16T20:08:02.938 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2023-10-16T20:08:02.485024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239500,"kb_used_data":6696,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735220,"statfs":{"total":773094113280,"available":772848865280,"internally_reserved":0,"allocated":6856704,"data_stored":3714343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238276153},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14930,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-65,"num_write_kb":-38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001056"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":198,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213453+0000","last_change":"2023-10-16T20:04:46.335822+0000","last_active":"2023-10-16T20:07:27.213453+0000","last_peered":"2023-10-16T20:07:27.213453+0000","last_clean":"2023-10-16T20:07:27.213453+0000","last_became_active":"2023-10-16T20:04:46.334617+0000","last_became_peered":"2023-10-16T20:04:46.334617+0000","last_unstale":"2023-10-16T20:07:27.213453+0000","last_undegraded":"2023-10-16T20:07:27.213453+0000","last_fullsized":"2023-10-16T20:07:27.213453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:45.306240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:45.306240+0000","last_clean_scrub_stamp":"2023-10-16T20:04:45.306240+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:42:11.153721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509482,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1302528,"data_stored":905075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.69099999999999995,"15min":0.70499999999999996},"min":{"1min":0.45500000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.872,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.318},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.58299999999999996,"5min":0.495,"15min":0.495},"max":{"1min":0.96499999999999997,"5min":1.077,"15min":1.077},"last":0.42199999999999999}]},{"osd":1,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.59299999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98399999999999999,"5min":1.78,"15min":1.78},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.70699999999999996,"15min":0.71499999999999997},"min":{"1min":0.57399999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93100000000000005,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":0.504}]},{"osd":2,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.72399999999999998,"15min":0.74099999999999999},"min":{"1min":0.54800000000000004,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91600000000000004,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.443},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78400000000000003,"15min":0.79500000000000004},"min":{"1min":0.64000000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.94699999999999995,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.64400000000000002}]},{"osd":3,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70499999999999996,"15min":0.70699999999999996},"min":{"1min":0.55100000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.84999999999999998,"5min":1.994,"15min":1.994},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.75700000000000001,"15min":0.78500000000000003},"min":{"1min":0.53200000000000003,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91300000000000003,"5min":2.629,"15min":2.629},"last":0.57799999999999996}]},{"osd":4,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.64900000000000002,"15min":0.66100000000000003},"min":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.98599999999999999,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.64900000000000002,"15min":0.65900000000000003},"min":{"1min":0.48099999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.83499999999999996,"5min":1.944,"15min":1.944},"last":0.72399999999999998}]},{"osd":5,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.72199999999999998,"15min":0.746},"min":{"1min":0.505,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91300000000000003,"5min":2.8809999999999998,"15min":2.8809999999999998},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71099999999999997,"15min":0.71599999999999997},"min":{"1min":0.51900000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94199999999999995,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.68100000000000005}]},{"osd":6,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.755,"15min":0.75700000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.96699999999999997,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.748},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72699999999999998,"15min":0.746},"min":{"1min":0.54200000000000004,"5min":0.438,"15min":0.438},"max":{"1min":0.92900000000000005,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.61799999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509483,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":700416,"data_stored":314547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.81000000000000005,"15min":0.86399999999999999},"min":{"1min":0.46500000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79400000000000004,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.78600000000000003,"15min":0.85399999999999998},"min":{"1min":0.46899999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86299999999999999,"5min":4.6139999999999999,"15min":4.6139999999999999},"last":0.79500000000000004}]},{"osd":1,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.83799999999999997,"15min":0.89000000000000001},"min":{"1min":0.51400000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.202,"5min":4.6639999999999997,"15min":4.6639999999999997},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.95099999999999996,"15min":1.075},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.86099999999999999,"5min":4.8140000000000001,"15min":4.8140000000000001},"last":0.65100000000000002}]},{"osd":2,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.871,"15min":0.95299999999999996},"min":{"1min":0.502,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88100000000000001,"5min":5.3769999999999998,"15min":5.3769999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.68700000000000006,"15min":0.67500000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.94399999999999995,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.58599999999999997}]},{"osd":3,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.86699999999999999,"15min":0.92400000000000004},"min":{"1min":0.60999999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.004,"5min":4.5460000000000003,"15min":4.5460000000000003},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.748,"15min":0.76600000000000001},"min":{"1min":0.53300000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.92300000000000004,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.82199999999999995}]},{"osd":4,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.70899999999999996,"15min":0.74299999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80200000000000005,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.67900000000000005,"15min":0.69499999999999995},"min":{"1min":0.42399999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93500000000000005,"5min":1.355,"15min":1.355},"last":0.71599999999999997}]},{"osd":5,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.69899999999999995,"15min":0.71799999999999997},"min":{"1min":0.441,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95599999999999996,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.78800000000000003,"15min":0.80200000000000005},"min":{"1min":0.52000000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.89000000000000001,"5min":1.218,"15min":1.218},"last":0.77700000000000002}]},{"osd":7,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.73899999999999999,"15min":0.745},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.97099999999999997,"5min":1.052,"15min":1.052},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.99099999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.84199999999999997}]}]},{"osd":5,"up_from":15,"seq":64424509481,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31872,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339968,"statfs":{"total":96636764160,"available":96604127232,"internally_reserved":0,"allocated":708608,"data_stored":315199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.64300000000000002,"15min":0.63200000000000001},"min":{"1min":0.40100000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94599999999999995,"5min":1.264,"15min":1.264},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.51700000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.65500000000000003}]},{"osd":1,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.63300000000000001,"15min":0.61399999999999999},"min":{"1min":0.47299999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.65200000000000002,"15min":0.64200000000000002},"min":{"1min":0.499,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.55600000000000005}]},{"osd":2,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67300000000000004,"15min":0.67100000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67300000000000004,"15min":0.65700000000000003},"min":{"1min":0.47699999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92300000000000004,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.68200000000000005}]},{"osd":3,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.76400000000000001,"15min":0.77800000000000002},"min":{"1min":0.58399999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.96299999999999997,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.71599999999999997,"15min":0.70399999999999996},"min":{"1min":0.52500000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.77900000000000003}]},{"osd":4,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.45400000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92700000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.67900000000000005,"15min":0.69399999999999995},"min":{"1min":0.41199999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.73599999999999999,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.61099999999999999}]},{"osd":6,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.65200000000000002,"15min":0.66300000000000003},"min":{"1min":0.437,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81899999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.53600000000000003,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.78900000000000003,"5min":1.04,"15min":1.04},"last":0.72399999999999998}]},{"osd":7,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.73699999999999999,"15min":0.746},"min":{"1min":0.53700000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.83199999999999996,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.747},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.72699999999999998,"15min":0.74099999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.52800000000000002}]}]},{"osd":4,"up_from":15,"seq":64424509481,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31888,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339952,"statfs":{"total":96636764160,"available":96604110848,"internally_reserved":0,"allocated":724992,"data_stored":330842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.74199999999999999,"15min":0.78600000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79200000000000004,"5min":3.835,"15min":3.835},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.76000000000000001,"15min":0.80700000000000005},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.91700000000000004,"5min":3.7610000000000001,"15min":3.7610000000000001},"last":0.54700000000000004}]},{"osd":1,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66600000000000004,"15min":0.67400000000000004},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.875,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.67400000000000004,"15min":0.67200000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.89100000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.49299999999999999}]},{"osd":2,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.54000000000000004,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.91400000000000003,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.70299999999999996,"15min":0.70699999999999996},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.97099999999999997,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.64300000000000002}]},{"osd":3,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.78800000000000003,"15min":0.80000000000000004},"min":{"1min":0.54900000000000004,"5min":0.44,"15min":0.44},"max":{"1min":1.002,"5min":1.163,"15min":1.163},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.77700000000000002,"15min":0.79100000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.93999999999999995,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.61299999999999999}]},{"osd":5,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.65500000000000003,"15min":0.66800000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85299999999999998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.67500000000000004,"15min":0.69399999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.94799999999999995,"5min":1.089,"15min":1.089},"last":0.66800000000000004}]},{"osd":6,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71299999999999997,"15min":0.73099999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93100000000000005,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.73799999999999999,"15min":0.77500000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.82799999999999996,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.56399999999999995}]},{"osd":7,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.75,"15min":0.77900000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92200000000000004,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.81999999999999995,"15min":0.84699999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.93200000000000005,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.70099999999999996}]}]},{"osd":3,"up_from":15,"seq":64424509482,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":708608,"data_stored":314603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.44900000000000001,"15min":0.45900000000000002},"min":{"1min":0.28399999999999997,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46800000000000003,"15min":0.46600000000000003},"min":{"1min":0.36099999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73499999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.504}]},{"osd":1,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.441,"15min":0.45400000000000001},"min":{"1min":0.29399999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64700000000000002,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.503,"15min":0.51500000000000001},"min":{"1min":0.36099999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.67300000000000004,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.64000000000000001}]},{"osd":2,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.49399999999999999,"15min":0.499},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70799999999999996,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53900000000000003,"15min":0.54500000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72399999999999998,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.66600000000000004}]},{"osd":4,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.76300000000000001,"15min":0.77700000000000002},"min":{"1min":0.56399999999999995,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.014,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.145},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.79800000000000004,"15min":0.81499999999999995},"min":{"1min":0.61699999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.96099999999999997,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.2}]},{"osd":5,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.745,"15min":0.76300000000000001},"min":{"1min":0.56899999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.88,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.76900000000000002,"15min":0.78500000000000003},"min":{"1min":0.56699999999999995,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.89400000000000002,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.92400000000000004}]},{"osd":6,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.752,"15min":0.78000000000000003},"min":{"1min":0.52300000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.90100000000000002,"5min":1.774,"15min":1.774},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.78800000000000003,"15min":0.81200000000000006},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.93000000000000005,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":1}]},{"osd":7,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.78000000000000003,"15min":0.80200000000000005},"min":{"1min":0.52500000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94199999999999995,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":1.075},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.77300000000000002,"15min":0.79900000000000004},"min":{"1min":0.56799999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.84899999999999998,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.94799999999999995}]}]},{"osd":2,"up_from":15,"seq":64424509481,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":708608,"data_stored":314603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40400000000000003,"15min":0.41199999999999998},"min":{"1min":0.26200000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.47199999999999998,"15min":0.48299999999999998},"min":{"1min":0.313,"5min":0.251,"15min":0.251},"max":{"1min":0.60599999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.375}]},{"osd":1,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.497,"5min":0.49099999999999999,"15min":0.48999999999999999},"min":{"1min":0.34300000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.439}]},{"osd":3,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.47099999999999997,"15min":0.47599999999999998},"min":{"1min":0.34100000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.58599999999999997,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.64500000000000002,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.50900000000000001}]},{"osd":4,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.75,"15min":0.76700000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.95399999999999996,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.73899999999999999,"15min":0.755},"min":{"1min":0.55800000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.107,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.71399999999999997}]},{"osd":5,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72099999999999997,"15min":0.73099999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0920000000000001,"5min":1.145,"15min":1.145},"last":0.62},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.73599999999999999,"15min":0.75},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.995,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.65000000000000002}]},{"osd":6,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.74099999999999999,"15min":0.749},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.97299999999999998,"5min":1.165,"15min":1.165},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.754,"15min":0.76600000000000001},"min":{"1min":0.57499999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.056,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.74299999999999999}]},{"osd":7,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.76000000000000001,"15min":0.76700000000000002},"min":{"1min":0.57799999999999996,"5min":0.502,"15min":0.502},"max":{"1min":1.0249999999999999,"5min":1.167,"15min":1.167},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.76900000000000002,"15min":0.78000000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.01,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.69199999999999995}]}]},{"osd":1,"up_from":15,"seq":64424509482,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":692224,"data_stored":308216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.499,"15min":0.54100000000000004},"min":{"1min":0.29199999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.63500000000000001,"5min":3.867,"15min":3.867},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.47999999999999998,"15min":0.49199999999999999},"min":{"1min":0.32600000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.621,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.32300000000000001}]},{"osd":2,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.63100000000000001,"15min":0.71499999999999997},"min":{"1min":0.34799999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.59999999999999998,"5min":5.6040000000000001,"15min":5.6040000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.68899999999999995,"15min":0.78800000000000003},"min":{"1min":0.34999999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65700000000000003,"5min":7.8380000000000001,"15min":7.8380000000000001},"last":0.505}]},{"osd":3,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.64200000000000002,"15min":0.70399999999999996},"min":{"1min":0.39800000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.74099999999999999,"5min":5.2039999999999997,"15min":5.2039999999999997},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.59999999999999998,"15min":0.65400000000000003},"min":{"1min":0.36899999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.67000000000000004,"5min":3.9470000000000001,"15min":3.9470000000000001},"last":0.38700000000000001}]},{"osd":4,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.81799999999999995,"15min":0.85599999999999998},"min":{"1min":0.54300000000000004,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.92200000000000004,"5min":4.1500000000000004,"15min":4.1500000000000004},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.83199999999999996,"15min":0.86799999999999999},"min":{"1min":0.54700000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.93300000000000005,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":0.69399999999999995}]},{"osd":5,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.754,"15min":0.75800000000000001},"min":{"1min":0.56399999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93200000000000005,"5min":1.496,"15min":1.496},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.84199999999999997,"15min":0.88100000000000001},"min":{"1min":0.56999999999999995,"5min":0.501,"15min":0.501},"max":{"1min":0.96999999999999997,"5min":4.0910000000000002,"15min":4.0910000000000002},"last":0.85999999999999999}]},{"osd":6,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.84199999999999997,"15min":0.876},"min":{"1min":0.56999999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.01,"5min":4.21,"15min":4.21},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.75900000000000001,"15min":0.76400000000000001},"min":{"1min":0.57799999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.95299999999999996,"5min":1.401,"15min":1.401},"last":0.754}]},{"osd":7,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.79500000000000004,"15min":0.79800000000000004},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.90600000000000003,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.874},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.83699999999999997,"15min":0.875},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.94099999999999995,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":0.70799999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574891,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28304,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343536,"statfs":{"total":96636764160,"available":96607780864,"internally_reserved":0,"allocated":1310720,"data_stored":911258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.51600000000000001,"15min":0.57599999999999996},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.61499999999999999,"5min":3.9289999999999998,"15min":3.9289999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.54100000000000004,"15min":0.59699999999999998},"min":{"1min":0.34200000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.61099999999999999,"5min":4.0910000000000002,"15min":4.0910000000000002},"last":0.61699999999999999}]},{"osd":2,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.60499999999999998,"15min":0.66200000000000003},"min":{"1min":0.32100000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70999999999999996,"5min":4.1520000000000001,"15min":4.1520000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.63700000000000001,"15min":0.69599999999999995},"min":{"1min":0.42399999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.71399999999999997,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.71099999999999997}]},{"osd":3,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.58699999999999997,"15min":0.63600000000000001},"min":{"1min":0.36499999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.67800000000000005,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55800000000000005,"15min":0.56799999999999995},"min":{"1min":0.45100000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.73599999999999999,"5min":1.006,"15min":1.006},"last":0.75600000000000001}]},{"osd":4,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.79900000000000004,"15min":0.85599999999999998},"min":{"1min":0.52500000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.91100000000000003,"5min":4.5250000000000004,"15min":4.5250000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.73699999999999999,"15min":0.747},"min":{"1min":0.5,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.89500000000000002,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.77900000000000003}]},{"osd":5,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.83899999999999997,"15min":0.89500000000000002},"min":{"1min":0.58199999999999996,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.004,"5min":3.9990000000000001,"15min":3.9990000000000001},"last":1.01},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.877,"15min":0.94499999999999995},"min":{"1min":0.54200000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0189999999999999,"5min":4.4779999999999998,"15min":4.4779999999999998},"last":0.64600000000000002}]},{"osd":6,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.75600000000000001,"15min":0.76600000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.93100000000000005,"5min":1.363,"15min":1.363},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.85699999999999998,"15min":0.92900000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.97099999999999997,"5min":3.956,"15min":3.956},"last":1.052}]},{"osd":7,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.879,"15min":0.93700000000000006},"min":{"1min":0.624,"5min":0.435,"15min":0.435},"max":{"1min":1.034,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":1.085},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.85799999999999998,"15min":0.92600000000000005},"min":{"1min":0.56399999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.97899999999999998,"5min":4.5640000000000001,"15min":4.5640000000000001},"last":0.68500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T20:08:02.941 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-16T20:08:02.942 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T20:08:03.079 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.072+0000 7f53c00c9640 1 Processor -- start 2023-10-16T20:08:03.080 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.072+0000 7f53c00c9640 1 -- start start 2023-10-16T20:08:03.081 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.076+0000 7f53c00c9640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f53b8069e90 0x7f53b806a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:03.082 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.076+0000 7f53c00c9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53b80b05a0 0x7f53b80b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:03.082 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.076+0000 7f53c00c9640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b806a830 con 0x7f53b81599f0 2023-10-16T20:08:03.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.076+0000 7f53c00c9640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b80b1260 con 0x7f53b8069e90 2023-10-16T20:08:03.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.076+0000 7f53c00c9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b80b3140 con 0x7f53b80b05a0 2023-10-16T20:08:03.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.076+0000 7f53bd664640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f53b8069e90 0x7f53b806a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:03.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.076+0000 7f53bd664640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f53b8069e90 0x7f53b806a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:59906/0 (socket says 172.21.15.88:59906) 2023-10-16T20:08:03.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.076+0000 7f53bd664640 1 -- 172.21.15.88:0/484193450 learned_addr learned my addr 172.21.15.88:0/484193450 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:03.084 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.076+0000 7f53be666640 1 --2- 172.21.15.88:0/484193450 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53b80b05a0 0x7f53b80b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:03.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.076+0000 7f53bd664640 1 -- 172.21.15.88:0/484193450 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53b80b05a0 msgr2=0x7f53b80b0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:08:03.085 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.076+0000 7f53bd664640 1 --2- 172.21.15.88:0/484193450 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53b80b05a0 0x7f53b80b0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:03.086 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.076+0000 7f53bd664640 1 -- 172.21.15.88:0/484193450 >> v1:172.21.15.94:6789/0 conn(0x7f53b81599f0 legacy=0x7f53b80b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:08:03.086 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.076+0000 7f53bd664640 1 -- 172.21.15.88:0/484193450 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b80ae960 con 0x7f53b8069e90 2023-10-16T20:08:03.086 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.076+0000 7f53be666640 1 --2- 172.21.15.88:0/484193450 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53b80b05a0 0x7f53b80b0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:08:03.091 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.076+0000 7f53bd664640 1 --2- 172.21.15.88:0/484193450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53b8069e90 0x7f53b806a2f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f53a8009870 tx=0x7f53a801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67fec49e6e43a43c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:03.092 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.080+0000 7f53bce63640 1 -- 172.21.15.88:0/484193450 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a80aa5f0 con 0x7f53b8069e90 2023-10-16T20:08:03.092 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.080+0000 7f53bce63640 1 -- 172.21.15.88:0/484193450 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a80aa750 con 0x7f53b8069e90 2023-10-16T20:08:03.093 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.080+0000 7f53bce63640 1 -- 172.21.15.88:0/484193450 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a80aaae0 con 0x7f53b8069e90 2023-10-16T20:08:03.093 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.080+0000 7f53c00c9640 1 -- 172.21.15.88:0/484193450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53b8069e90 msgr2=0x7f53b806a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:03.094 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.080+0000 7f53c00c9640 1 --2- 172.21.15.88:0/484193450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53b8069e90 0x7f53b806a2f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f53a8009870 tx=0x7f53a801c4d0 comp rx=0 tx=0).stop 2023-10-16T20:08:03.094 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.080+0000 7f53c00c9640 1 -- 172.21.15.88:0/484193450 shutdown_connections 2023-10-16T20:08:03.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.080+0000 7f53c00c9640 1 --2- 172.21.15.88:0/484193450 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53b80b05a0 0x7f53b80b0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:03.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.080+0000 7f53c00c9640 1 --2- 172.21.15.88:0/484193450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53b8069e90 0x7f53b806a2f0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:03.095 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.080+0000 7f53c00c9640 1 -- 172.21.15.88:0/484193450 >> 172.21.15.88:0/484193450 conn(0x7f53b80ce5e0 msgr2=0x7f53b80ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:03.096 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.080+0000 7f53c00c9640 1 -- 172.21.15.88:0/484193450 shutdown_connections 2023-10-16T20:08:03.096 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.080+0000 7f53c00c9640 1 -- 172.21.15.88:0/484193450 wait complete. 2023-10-16T20:08:03.096 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.080+0000 7f53c00c9640 1 Processor -- start 2023-10-16T20:08:03.097 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.080+0000 7f53c00c9640 1 -- start start 2023-10-16T20:08:03.097 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.080+0000 7f53c00c9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53b8069e90 0x7f53b8146030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:03.098 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.080+0000 7f53c00c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53b80b05a0 0x7f53b814b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:03.098 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.080+0000 7f53c00c9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53b81599f0 0x7f53b8147f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:03.099 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.080+0000 7f53c00c9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b8145c10 con 0x7f53b81599f0 2023-10-16T20:08:03.099 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.080+0000 7f53c00c9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b8144f90 con 0x7f53b80b05a0 2023-10-16T20:08:03.100 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.080+0000 7f53c00c9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b8145270 con 0x7f53b8069e90 2023-10-16T20:08:03.100 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.080+0000 7f53be666640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53b81599f0 0x7f53b8147f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:03.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.080+0000 7f53be666640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53b81599f0 0x7f53b8147f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:59922/0 (socket says 172.21.15.88:59922) 2023-10-16T20:08:03.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.080+0000 7f53be666640 1 -- 172.21.15.88:0/2177436970 learned_addr learned my addr 172.21.15.88:0/2177436970 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:03.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.084+0000 7f53bd664640 1 --2- 172.21.15.88:0/2177436970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53b80b05a0 0x7f53b814b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:03.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.084+0000 7f53bde65640 1 --2- 172.21.15.88:0/2177436970 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53b8069e90 0x7f53b8146030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:03.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.084+0000 7f53be666640 1 -- 172.21.15.88:0/2177436970 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53b8069e90 msgr2=0x7f53b8146030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:03.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.084+0000 7f53be666640 1 --2- 172.21.15.88:0/2177436970 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53b8069e90 0x7f53b8146030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:03.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.084+0000 7f53be666640 1 -- 172.21.15.88:0/2177436970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53b80b05a0 msgr2=0x7f53b814b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:03.104 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.084+0000 7f53be666640 1 --2- 172.21.15.88:0/2177436970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53b80b05a0 0x7f53b814b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:03.104 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.084+0000 7f53be666640 1 -- 172.21.15.88:0/2177436970 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a80094d0 con 0x7f53b81599f0 2023-10-16T20:08:03.105 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.084+0000 7f53be666640 1 --2- 172.21.15.88:0/2177436970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53b81599f0 0x7f53b8147f90 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f53b000cef0 tx=0x7f53b000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:03.105 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.084+0000 7f53ae7fc640 1 -- 172.21.15.88:0/2177436970 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b000d840 con 0x7f53b81599f0 2023-10-16T20:08:03.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.084+0000 7f53ae7fc640 1 -- 172.21.15.88:0/2177436970 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b000d9c0 con 0x7f53b81599f0 2023-10-16T20:08:03.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.084+0000 7f53c00c9640 1 -- 172.21.15.88:0/2177436970 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b8145550 con 0x7f53b81599f0 2023-10-16T20:08:03.106 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.084+0000 7f53ae7fc640 1 -- 172.21.15.88:0/2177436970 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b000dd70 con 0x7f53b81599f0 2023-10-16T20:08:03.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.084+0000 7f53c00c9640 1 -- 172.21.15.88:0/2177436970 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b8146570 con 0x7f53b81599f0 2023-10-16T20:08:03.107 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.084+0000 7f53c00c9640 1 -- 172.21.15.88:0/2177436970 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53b8145d50 con 0x7f53b81599f0 2023-10-16T20:08:03.108 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.084+0000 7f53ae7fc640 1 -- 172.21.15.88:0/2177436970 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f53b001e3b0 con 0x7f53b81599f0 2023-10-16T20:08:03.108 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.084+0000 7f53ae7fc640 1 --2- 172.21.15.88:0/2177436970 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f539c0664a0 0x7f539c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:03.108 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.084+0000 7f53ae7fc640 1 -- 172.21.15.88:0/2177436970 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f53b0084770 con 0x7f53b81599f0 2023-10-16T20:08:03.109 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.084+0000 7f53bde65640 1 --2- 172.21.15.88:0/2177436970 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f539c0664a0 0x7f539c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:03.110 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.088+0000 7f53bde65640 1 --2- 172.21.15.88:0/2177436970 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f539c0664a0 0x7f539c068960 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f53b400a750 tx=0x7f53b4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:03.110 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.088+0000 7f53ae7fc640 1 -- 172.21.15.88:0/2177436970 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f53b8145d50 con 0x7f53b81599f0 2023-10-16T20:08:03.242 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.236+0000 7f53c00c9640 1 -- 172.21.15.88:0/2177436970 --> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f53b80b0f70 con 0x7f539c0664a0 2023-10-16T20:08:03.245 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.240+0000 7f53ae7fc640 1 -- 172.21.15.88:0/2177436970 <== mgr.4110 v2:172.21.15.88:6832/4078483333 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50984 (secure 0 0 0) 0x7f53b400a380 con 0x7f539c0664a0 2023-10-16T20:08:03.245 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:08:03.247 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2023-10-16T20:08:03.247 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.240+0000 7f53c00c9640 1 -- 172.21.15.88:0/2177436970 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f539c0664a0 msgr2=0x7f539c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:03.248 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.240+0000 7f53c00c9640 1 --2- 172.21.15.88:0/2177436970 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f539c0664a0 0x7f539c068960 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f53b400a750 tx=0x7f53b4008040 comp rx=0 tx=0).stop 2023-10-16T20:08:03.248 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.240+0000 7f53c00c9640 1 -- 172.21.15.88:0/2177436970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53b81599f0 msgr2=0x7f53b8147f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:03.249 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.240+0000 7f53c00c9640 1 --2- 172.21.15.88:0/2177436970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53b81599f0 0x7f53b8147f90 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f53b000cef0 tx=0x7f53b000a710 comp rx=0 tx=0).stop 2023-10-16T20:08:03.249 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.240+0000 7f53c00c9640 1 -- 172.21.15.88:0/2177436970 shutdown_connections 2023-10-16T20:08:03.249 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.240+0000 7f53c00c9640 1 --2- 172.21.15.88:0/2177436970 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f539c0664a0 0x7f539c068960 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:03.250 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.240+0000 7f53c00c9640 1 --2- 172.21.15.88:0/2177436970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53b81599f0 0x7f53b8147f90 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:03.250 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.240+0000 7f53c00c9640 1 --2- 172.21.15.88:0/2177436970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53b80b05a0 0x7f53b814b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:03.250 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.240+0000 7f53c00c9640 1 --2- 172.21.15.88:0/2177436970 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f53b8069e90 0x7f53b8146030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:03.251 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.240+0000 7f53c00c9640 1 -- 172.21.15.88:0/2177436970 >> 172.21.15.88:0/2177436970 conn(0x7f53b80ce5e0 msgr2=0x7f53b8071a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:03.251 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.240+0000 7f53c00c9640 1 -- 172.21.15.88:0/2177436970 shutdown_connections 2023-10-16T20:08:03.252 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.240+0000 7f53c00c9640 1 -- 172.21.15.88:0/2177436970 wait complete. 2023-10-16T20:08:03.266 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2023-10-16T20:08:02.485024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239500,"kb_used_data":6696,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735220,"statfs":{"total":773094113280,"available":772848865280,"internally_reserved":0,"allocated":6856704,"data_stored":3714343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238276153},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14930,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-65,"num_write_kb":-38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001056"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":198,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T20:07:27.213453+0000","last_change":"2023-10-16T20:04:46.335822+0000","last_active":"2023-10-16T20:07:27.213453+0000","last_peered":"2023-10-16T20:07:27.213453+0000","last_clean":"2023-10-16T20:07:27.213453+0000","last_became_active":"2023-10-16T20:04:46.334617+0000","last_became_peered":"2023-10-16T20:04:46.334617+0000","last_unstale":"2023-10-16T20:07:27.213453+0000","last_undegraded":"2023-10-16T20:07:27.213453+0000","last_fullsized":"2023-10-16T20:07:27.213453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:45.306240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:45.306240+0000","last_clean_scrub_stamp":"2023-10-16T20:04:45.306240+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:42:11.153721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509482,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1302528,"data_stored":905075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.69099999999999995,"15min":0.70499999999999996},"min":{"1min":0.45500000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.872,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.318},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.58299999999999996,"5min":0.495,"15min":0.495},"max":{"1min":0.96499999999999997,"5min":1.077,"15min":1.077},"last":0.42199999999999999}]},{"osd":1,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.59299999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98399999999999999,"5min":1.78,"15min":1.78},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.70699999999999996,"15min":0.71499999999999997},"min":{"1min":0.57399999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93100000000000005,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":0.504}]},{"osd":2,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.72399999999999998,"15min":0.74099999999999999},"min":{"1min":0.54800000000000004,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91600000000000004,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.443},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78400000000000003,"15min":0.79500000000000004},"min":{"1min":0.64000000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.94699999999999995,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.64400000000000002}]},{"osd":3,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70499999999999996,"15min":0.70699999999999996},"min":{"1min":0.55100000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.84999999999999998,"5min":1.994,"15min":1.994},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.75700000000000001,"15min":0.78500000000000003},"min":{"1min":0.53200000000000003,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91300000000000003,"5min":2.629,"15min":2.629},"last":0.57799999999999996}]},{"osd":4,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.64900000000000002,"15min":0.66100000000000003},"min":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.98599999999999999,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.64900000000000002,"15min":0.65900000000000003},"min":{"1min":0.48099999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.83499999999999996,"5min":1.944,"15min":1.944},"last":0.72399999999999998}]},{"osd":5,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.72199999999999998,"15min":0.746},"min":{"1min":0.505,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91300000000000003,"5min":2.8809999999999998,"15min":2.8809999999999998},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71099999999999997,"15min":0.71599999999999997},"min":{"1min":0.51900000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94199999999999995,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.68100000000000005}]},{"osd":6,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.755,"15min":0.75700000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.96699999999999997,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.748},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72699999999999998,"15min":0.746},"min":{"1min":0.54200000000000004,"5min":0.438,"15min":0.438},"max":{"1min":0.92900000000000005,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.61799999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509483,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":700416,"data_stored":314547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.81000000000000005,"15min":0.86399999999999999},"min":{"1min":0.46500000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79400000000000004,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.78600000000000003,"15min":0.85399999999999998},"min":{"1min":0.46899999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86299999999999999,"5min":4.6139999999999999,"15min":4.6139999999999999},"last":0.79500000000000004}]},{"osd":1,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.83799999999999997,"15min":0.89000000000000001},"min":{"1min":0.51400000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.202,"5min":4.6639999999999997,"15min":4.6639999999999997},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.95099999999999996,"15min":1.075},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.86099999999999999,"5min":4.8140000000000001,"15min":4.8140000000000001},"last":0.65100000000000002}]},{"osd":2,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.871,"15min":0.95299999999999996},"min":{"1min":0.502,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88100000000000001,"5min":5.3769999999999998,"15min":5.3769999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.68700000000000006,"15min":0.67500000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.94399999999999995,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.58599999999999997}]},{"osd":3,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.86699999999999999,"15min":0.92400000000000004},"min":{"1min":0.60999999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.004,"5min":4.5460000000000003,"15min":4.5460000000000003},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.748,"15min":0.76600000000000001},"min":{"1min":0.53300000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.92300000000000004,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.82199999999999995}]},{"osd":4,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.70899999999999996,"15min":0.74299999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80200000000000005,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.67900000000000005,"15min":0.69499999999999995},"min":{"1min":0.42399999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93500000000000005,"5min":1.355,"15min":1.355},"last":0.71599999999999997}]},{"osd":5,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.69899999999999995,"15min":0.71799999999999997},"min":{"1min":0.441,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95599999999999996,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.78800000000000003,"15min":0.80200000000000005},"min":{"1min":0.52000000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.89000000000000001,"5min":1.218,"15min":1.218},"last":0.77700000000000002}]},{"osd":7,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.73899999999999999,"15min":0.745},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.97099999999999997,"5min":1.052,"15min":1.052},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.99099999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.84199999999999997}]}]},{"osd":5,"up_from":15,"seq":64424509481,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31872,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339968,"statfs":{"total":96636764160,"available":96604127232,"internally_reserved":0,"allocated":708608,"data_stored":315199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.64300000000000002,"15min":0.63200000000000001},"min":{"1min":0.40100000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94599999999999995,"5min":1.264,"15min":1.264},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.51700000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.65500000000000003}]},{"osd":1,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.63300000000000001,"15min":0.61399999999999999},"min":{"1min":0.47299999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.65200000000000002,"15min":0.64200000000000002},"min":{"1min":0.499,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.55600000000000005}]},{"osd":2,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67300000000000004,"15min":0.67100000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67300000000000004,"15min":0.65700000000000003},"min":{"1min":0.47699999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92300000000000004,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.68200000000000005}]},{"osd":3,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.76400000000000001,"15min":0.77800000000000002},"min":{"1min":0.58399999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.96299999999999997,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.71599999999999997,"15min":0.70399999999999996},"min":{"1min":0.52500000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.77900000000000003}]},{"osd":4,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.45400000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92700000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.67900000000000005,"15min":0.69399999999999995},"min":{"1min":0.41199999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.73599999999999999,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.61099999999999999}]},{"osd":6,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.65200000000000002,"15min":0.66300000000000003},"min":{"1min":0.437,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81899999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.53600000000000003,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.78900000000000003,"5min":1.04,"15min":1.04},"last":0.72399999999999998}]},{"osd":7,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.73699999999999999,"15min":0.746},"min":{"1min":0.53700000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.83199999999999996,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.747},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.72699999999999998,"15min":0.74099999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.52800000000000002}]}]},{"osd":4,"up_from":15,"seq":64424509481,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31888,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339952,"statfs":{"total":96636764160,"available":96604110848,"internally_reserved":0,"allocated":724992,"data_stored":330842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.74199999999999999,"15min":0.78600000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79200000000000004,"5min":3.835,"15min":3.835},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.76000000000000001,"15min":0.80700000000000005},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.91700000000000004,"5min":3.7610000000000001,"15min":3.7610000000000001},"last":0.54700000000000004}]},{"osd":1,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66600000000000004,"15min":0.67400000000000004},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.875,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.67400000000000004,"15min":0.67200000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.89100000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.49299999999999999}]},{"osd":2,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.54000000000000004,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.91400000000000003,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.70299999999999996,"15min":0.70699999999999996},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.97099999999999997,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.64300000000000002}]},{"osd":3,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.78800000000000003,"15min":0.80000000000000004},"min":{"1min":0.54900000000000004,"5min":0.44,"15min":0.44},"max":{"1min":1.002,"5min":1.163,"15min":1.163},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.77700000000000002,"15min":0.79100000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.93999999999999995,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.61299999999999999}]},{"osd":5,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.65500000000000003,"15min":0.66800000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85299999999999998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.67500000000000004,"15min":0.69399999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.94799999999999995,"5min":1.089,"15min":1.089},"last":0.66800000000000004}]},{"osd":6,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71299999999999997,"15min":0.73099999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93100000000000005,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.73799999999999999,"15min":0.77500000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.82799999999999996,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.56399999999999995}]},{"osd":7,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.75,"15min":0.77900000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92200000000000004,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.81999999999999995,"15min":0.84699999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.93200000000000005,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.70099999999999996}]}]},{"osd":3,"up_from":15,"seq":64424509482,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":708608,"data_stored":314603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.44900000000000001,"15min":0.45900000000000002},"min":{"1min":0.28399999999999997,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46800000000000003,"15min":0.46600000000000003},"min":{"1min":0.36099999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73499999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.504}]},{"osd":1,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.441,"15min":0.45400000000000001},"min":{"1min":0.29399999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64700000000000002,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.503,"15min":0.51500000000000001},"min":{"1min":0.36099999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.67300000000000004,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.64000000000000001}]},{"osd":2,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.49399999999999999,"15min":0.499},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70799999999999996,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53900000000000003,"15min":0.54500000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72399999999999998,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.66600000000000004}]},{"osd":4,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.76300000000000001,"15min":0.77700000000000002},"min":{"1min":0.56399999999999995,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.014,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.145},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.79800000000000004,"15min":0.81499999999999995},"min":{"1min":0.61699999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.96099999999999997,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.2}]},{"osd":5,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.745,"15min":0.76300000000000001},"min":{"1min":0.56899999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.88,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.76900000000000002,"15min":0.78500000000000003},"min":{"1min":0.56699999999999995,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.89400000000000002,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.92400000000000004}]},{"osd":6,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.752,"15min":0.78000000000000003},"min":{"1min":0.52300000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.90100000000000002,"5min":1.774,"15min":1.774},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.78800000000000003,"15min":0.81200000000000006},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.93000000000000005,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":1}]},{"osd":7,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.78000000000000003,"15min":0.80200000000000005},"min":{"1min":0.52500000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94199999999999995,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":1.075},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.77300000000000002,"15min":0.79900000000000004},"min":{"1min":0.56799999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.84899999999999998,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.94799999999999995}]}]},{"osd":2,"up_from":15,"seq":64424509481,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":708608,"data_stored":314603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40400000000000003,"15min":0.41199999999999998},"min":{"1min":0.26200000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.47199999999999998,"15min":0.48299999999999998},"min":{"1min":0.313,"5min":0.251,"15min":0.251},"max":{"1min":0.60599999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.375}]},{"osd":1,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.497,"5min":0.49099999999999999,"15min":0.48999999999999999},"min":{"1min":0.34300000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.439}]},{"osd":3,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.47099999999999997,"15min":0.47599999999999998},"min":{"1min":0.34100000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.58599999999999997,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.64500000000000002,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.50900000000000001}]},{"osd":4,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.75,"15min":0.76700000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.95399999999999996,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.73899999999999999,"15min":0.755},"min":{"1min":0.55800000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.107,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.71399999999999997}]},{"osd":5,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72099999999999997,"15min":0.73099999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0920000000000001,"5min":1.145,"15min":1.145},"last":0.62},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.73599999999999999,"15min":0.75},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.995,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.65000000000000002}]},{"osd":6,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.74099999999999999,"15min":0.749},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.97299999999999998,"5min":1.165,"15min":1.165},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.754,"15min":0.76600000000000001},"min":{"1min":0.57499999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.056,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.74299999999999999}]},{"osd":7,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.76000000000000001,"15min":0.76700000000000002},"min":{"1min":0.57799999999999996,"5min":0.502,"15min":0.502},"max":{"1min":1.0249999999999999,"5min":1.167,"15min":1.167},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.76900000000000002,"15min":0.78000000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.01,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.69199999999999995}]}]},{"osd":1,"up_from":15,"seq":64424509482,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":692224,"data_stored":308216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.499,"15min":0.54100000000000004},"min":{"1min":0.29199999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.63500000000000001,"5min":3.867,"15min":3.867},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.47999999999999998,"15min":0.49199999999999999},"min":{"1min":0.32600000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.621,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.32300000000000001}]},{"osd":2,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.63100000000000001,"15min":0.71499999999999997},"min":{"1min":0.34799999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.59999999999999998,"5min":5.6040000000000001,"15min":5.6040000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.68899999999999995,"15min":0.78800000000000003},"min":{"1min":0.34999999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65700000000000003,"5min":7.8380000000000001,"15min":7.8380000000000001},"last":0.505}]},{"osd":3,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.64200000000000002,"15min":0.70399999999999996},"min":{"1min":0.39800000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.74099999999999999,"5min":5.2039999999999997,"15min":5.2039999999999997},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.59999999999999998,"15min":0.65400000000000003},"min":{"1min":0.36899999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.67000000000000004,"5min":3.9470000000000001,"15min":3.9470000000000001},"last":0.38700000000000001}]},{"osd":4,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.81799999999999995,"15min":0.85599999999999998},"min":{"1min":0.54300000000000004,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.92200000000000004,"5min":4.1500000000000004,"15min":4.1500000000000004},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.83199999999999996,"15min":0.86799999999999999},"min":{"1min":0.54700000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.93300000000000005,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":0.69399999999999995}]},{"osd":5,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.754,"15min":0.75800000000000001},"min":{"1min":0.56399999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93200000000000005,"5min":1.496,"15min":1.496},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.84199999999999997,"15min":0.88100000000000001},"min":{"1min":0.56999999999999995,"5min":0.501,"15min":0.501},"max":{"1min":0.96999999999999997,"5min":4.0910000000000002,"15min":4.0910000000000002},"last":0.85999999999999999}]},{"osd":6,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.84199999999999997,"15min":0.876},"min":{"1min":0.56999999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.01,"5min":4.21,"15min":4.21},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.75900000000000001,"15min":0.76400000000000001},"min":{"1min":0.57799999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.95299999999999996,"5min":1.401,"15min":1.401},"last":0.754}]},{"osd":7,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.79500000000000004,"15min":0.79800000000000004},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.90600000000000003,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.874},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.83699999999999997,"15min":0.875},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.94099999999999995,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":0.70799999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574891,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28304,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343536,"statfs":{"total":96636764160,"available":96607780864,"internally_reserved":0,"allocated":1310720,"data_stored":911258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.51600000000000001,"15min":0.57599999999999996},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.61499999999999999,"5min":3.9289999999999998,"15min":3.9289999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.54100000000000004,"15min":0.59699999999999998},"min":{"1min":0.34200000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.61099999999999999,"5min":4.0910000000000002,"15min":4.0910000000000002},"last":0.61699999999999999}]},{"osd":2,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.60499999999999998,"15min":0.66200000000000003},"min":{"1min":0.32100000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70999999999999996,"5min":4.1520000000000001,"15min":4.1520000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.63700000000000001,"15min":0.69599999999999995},"min":{"1min":0.42399999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.71399999999999997,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.71099999999999997}]},{"osd":3,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.58699999999999997,"15min":0.63600000000000001},"min":{"1min":0.36499999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.67800000000000005,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55800000000000005,"15min":0.56799999999999995},"min":{"1min":0.45100000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.73599999999999999,"5min":1.006,"15min":1.006},"last":0.75600000000000001}]},{"osd":4,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.79900000000000004,"15min":0.85599999999999998},"min":{"1min":0.52500000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.91100000000000003,"5min":4.5250000000000004,"15min":4.5250000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.73699999999999999,"15min":0.747},"min":{"1min":0.5,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.89500000000000002,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.77900000000000003}]},{"osd":5,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.83899999999999997,"15min":0.89500000000000002},"min":{"1min":0.58199999999999996,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.004,"5min":3.9990000000000001,"15min":3.9990000000000001},"last":1.01},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.877,"15min":0.94499999999999995},"min":{"1min":0.54200000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0189999999999999,"5min":4.4779999999999998,"15min":4.4779999999999998},"last":0.64600000000000002}]},{"osd":6,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.75600000000000001,"15min":0.76600000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.93100000000000005,"5min":1.363,"15min":1.363},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.85699999999999998,"15min":0.92900000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.97099999999999997,"5min":3.956,"15min":3.956},"last":1.052}]},{"osd":7,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.879,"15min":0.93700000000000006},"min":{"1min":0.624,"5min":0.435,"15min":0.435},"max":{"1min":1.034,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":1.085},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.85799999999999998,"15min":0.92600000000000005},"min":{"1min":0.56399999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.97899999999999998,"5min":4.5640000000000001,"15min":4.5640000000000001},"last":0.68500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T20:08:03.269 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T20:08:03.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.404+0000 7feed1ec9640 1 Processor -- start 2023-10-16T20:08:03.409 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.404+0000 7feed1ec9640 1 -- start start 2023-10-16T20:08:03.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.404+0000 7feed1ec9640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7feecc15cc60 0x7feecc0caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:03.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.404+0000 7feed1ec9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feecc071450 0x7feecc0c39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:03.411 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.404+0000 7feed1ec9640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feecc0ca710 con 0x7feecc0cb0e0 2023-10-16T20:08:03.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.404+0000 7feed1ec9640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feecc0c9de0 con 0x7feecc15cc60 2023-10-16T20:08:03.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.404+0000 7feed1ec9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feecc0ca0c0 con 0x7feecc071450 2023-10-16T20:08:03.412 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.404+0000 7feecbfff640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7feecc0cb0e0 0x7feecc070710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:47212/0 (socket says 172.21.15.88:47212) 2023-10-16T20:08:03.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.404+0000 7feecbfff640 1 -- 172.21.15.88:0/2676198799 learned_addr learned my addr 172.21.15.88:0/2676198799 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:08:03.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.404+0000 7feecb7fe640 1 --2- 172.21.15.88:0/2676198799 >> v2:172.21.15.94:3300/0 conn(0x7feecc15cc60 0x7feecc0caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:03.413 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.404+0000 7feec3fff640 1 --2- 172.21.15.88:0/2676198799 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feecc071450 0x7feecc0c39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:03.414 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.404+0000 7feecaffd640 1 -- 172.21.15.88:0/2676198799 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2280233540 0 0) 0x7feecc0ca710 con 0x7feecc0cb0e0 2023-10-16T20:08:03.414 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.404+0000 7feecaffd640 1 -- 172.21.15.88:0/2676198799 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feeb8003530 con 0x7feecc0cb0e0 2023-10-16T20:08:03.414 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.404+0000 7feecb7fe640 1 -- 172.21.15.88:0/2676198799 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feecc071450 msgr2=0x7feecc0c39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:08:03.415 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.404+0000 7feecb7fe640 1 --2- 172.21.15.88:0/2676198799 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feecc071450 0x7feecc0c39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:03.415 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.404+0000 7feecb7fe640 1 -- 172.21.15.88:0/2676198799 >> v1:172.21.15.88:6789/0 conn(0x7feecc0cb0e0 legacy=0x7feecc070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:03.415 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.404+0000 7feecb7fe640 1 -- 172.21.15.88:0/2676198799 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feecc0ca3a0 con 0x7feecc15cc60 2023-10-16T20:08:03.415 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.404+0000 7feec3fff640 1 --2- 172.21.15.88:0/2676198799 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feecc071450 0x7feecc0c39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:08:03.418 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.408+0000 7feecb7fe640 1 --2- 172.21.15.88:0/2676198799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feecc15cc60 0x7feecc0caba0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7feebc0097f0 tx=0x7feebc01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e065869cd13eb70c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:03.418 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.408+0000 7feecaffd640 1 -- 172.21.15.88:0/2676198799 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc0ad640 con 0x7feecc15cc60 2023-10-16T20:08:03.419 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.408+0000 7feecaffd640 1 -- 172.21.15.88:0/2676198799 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feebc0ad7a0 con 0x7feecc15cc60 2023-10-16T20:08:03.419 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.408+0000 7feecaffd640 1 -- 172.21.15.88:0/2676198799 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc0adb30 con 0x7feecc15cc60 2023-10-16T20:08:03.419 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.408+0000 7feed1ec9640 1 -- 172.21.15.88:0/2676198799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feecc15cc60 msgr2=0x7feecc0caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:03.420 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.408+0000 7feed1ec9640 1 --2- 172.21.15.88:0/2676198799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feecc15cc60 0x7feecc0caba0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7feebc0097f0 tx=0x7feebc01c480 comp rx=0 tx=0).stop 2023-10-16T20:08:03.420 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.408+0000 7feed1ec9640 1 -- 172.21.15.88:0/2676198799 shutdown_connections 2023-10-16T20:08:03.420 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.408+0000 7feed1ec9640 1 --2- 172.21.15.88:0/2676198799 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feecc071450 0x7feecc0c39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:03.420 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.408+0000 7feed1ec9640 1 --2- 172.21.15.88:0/2676198799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feecc15cc60 0x7feecc0caba0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:03.421 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.408+0000 7feed1ec9640 1 -- 172.21.15.88:0/2676198799 >> 172.21.15.88:0/2676198799 conn(0x7feecc0ceb20 msgr2=0x7feecc0cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:03.421 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.408+0000 7feed1ec9640 1 -- 172.21.15.88:0/2676198799 shutdown_connections 2023-10-16T20:08:03.421 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.408+0000 7feed1ec9640 1 -- 172.21.15.88:0/2676198799 wait complete. 2023-10-16T20:08:03.422 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.412+0000 7feed1ec9640 1 Processor -- start 2023-10-16T20:08:03.422 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.412+0000 7feed1ec9640 1 -- start start 2023-10-16T20:08:03.422 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.412+0000 7feed1ec9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feecc071450 0x7feecc0c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:03.422 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.412+0000 7feed1ec9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feecc0cb0e0 0x7feecc158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:03.423 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.412+0000 7feed1ec9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feecc15cc60 0x7feecc150500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:03.423 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.412+0000 7feed1ec9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feecc158aa0 con 0x7feecc071450 2023-10-16T20:08:03.423 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.412+0000 7feed1ec9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feecc0c3e40 con 0x7feecc15cc60 2023-10-16T20:08:03.424 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.412+0000 7feed1ec9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feecc159290 con 0x7feecc0cb0e0 2023-10-16T20:08:03.424 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.412+0000 7feecb7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feecc071450 0x7feecc0c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:03.424 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.412+0000 7feecb7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feecc071450 0x7feecc0c85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:59928/0 (socket says 172.21.15.88:59928) 2023-10-16T20:08:03.424 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.412+0000 7feecb7fe640 1 -- 172.21.15.88:0/3097264418 learned_addr learned my addr 172.21.15.88:0/3097264418 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:03.425 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.412+0000 7feecb7fe640 1 -- 172.21.15.88:0/3097264418 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feecc0cb0e0 msgr2=0x7feecc158560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:03.425 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.412+0000 7feecb7fe640 1 --2- 172.21.15.88:0/3097264418 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feecc0cb0e0 0x7feecc158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:03.425 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.412+0000 7feecb7fe640 1 -- 172.21.15.88:0/3097264418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feecc15cc60 msgr2=0x7feecc150500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:03.426 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.412+0000 7feecb7fe640 1 --2- 172.21.15.88:0/3097264418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feecc15cc60 0x7feecc150500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:03.426 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.412+0000 7feecb7fe640 1 -- 172.21.15.88:0/3097264418 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb4009420 con 0x7feecc071450 2023-10-16T20:08:03.426 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.412+0000 7feecb7fe640 1 --2- 172.21.15.88:0/3097264418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feecc071450 0x7feecc0c85f0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7feebc0b0040 tx=0x7feebc019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:03.428 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.412+0000 7feec8ff9640 1 -- 172.21.15.88:0/3097264418 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc0b6610 con 0x7feecc071450 2023-10-16T20:08:03.428 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.412+0000 7feec8ff9640 1 -- 172.21.15.88:0/3097264418 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feebc0a6040 con 0x7feecc071450 2023-10-16T20:08:03.429 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.412+0000 7feed1ec9640 1 -- 172.21.15.88:0/3097264418 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feebc009420 con 0x7feecc071450 2023-10-16T20:08:03.429 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.412+0000 7feec8ff9640 1 -- 172.21.15.88:0/3097264418 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc0b69b0 con 0x7feecc071450 2023-10-16T20:08:03.429 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.412+0000 7feed1ec9640 1 -- 172.21.15.88:0/3097264418 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feecc158cd0 con 0x7feecc071450 2023-10-16T20:08:03.429 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.416+0000 7feed1ec9640 1 -- 172.21.15.88:0/3097264418 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feecc04e9c0 con 0x7feecc071450 2023-10-16T20:08:03.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.416+0000 7feec8ff9640 1 -- 172.21.15.88:0/3097264418 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7feebc0b4040 con 0x7feecc071450 2023-10-16T20:08:03.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.416+0000 7feec8ff9640 1 --2- 172.21.15.88:0/3097264418 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7feeac0664d0 0x7feeac068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:03.430 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.416+0000 7feec8ff9640 1 -- 172.21.15.88:0/3097264418 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5263+0+0 (secure 0 0 0) 0x7feebc11de60 con 0x7feecc071450 2023-10-16T20:08:03.431 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.416+0000 7feec3fff640 1 --2- 172.21.15.88:0/3097264418 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7feeac0664d0 0x7feeac068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:03.431 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.416+0000 7feec3fff640 1 --2- 172.21.15.88:0/3097264418 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7feeac0664d0 0x7feeac068990 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7feeb4006e90 tx=0x7feeb4006de0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:03.431 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.420+0000 7feec8ff9640 1 -- 172.21.15.88:0/3097264418 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7feebc0eb520 con 0x7feecc071450 2023-10-16T20:08:03.576 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.572+0000 7feed1ec9640 1 -- 172.21.15.88:0/3097264418 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feecc06e820 con 0x7feecc071450 2023-10-16T20:08:03.578 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.572+0000 7feec8ff9640 1 -- 172.21.15.88:0/3097264418 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+13119 (secure 0 0 0) 0x7feebc0b2070 con 0x7feecc071450 2023-10-16T20:08:03.579 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:08:03.579 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":61,"fsid":"d11187c0-3dd8-4c92-8d13-6ec536230748","created":"2023-10-16T20:04:31.382069+0000","modified":"2023-10-16T20:08:00.276512+0000","last_up_change":"2023-10-16T20:04:43.299932+0000","last_in_change":"2023-10-16T20:04:33.913430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T20:04:44.508444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41293e0f-eacc-49a7-a09b-5e06fc182edb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6801","nonce":685586021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6803","nonce":685586021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6807","nonce":685586021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":685586021},{"type":"v1","addr":"172.21.15.88:6805","nonce":685586021}]},"public_addr":"172.21.15.88:6801/685586021","cluster_addr":"172.21.15.88:6803/685586021","heartbeat_back_addr":"172.21.15.88:6807/685586021","heartbeat_front_addr":"172.21.15.88:6805/685586021","state":["exists","up"]},{"osd":1,"uuid":"445f9559-9221-476a-b9dd-2a542e126270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6809","nonce":2212808191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6811","nonce":2212808191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6815","nonce":2212808191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2212808191},{"type":"v1","addr":"172.21.15.88:6813","nonce":2212808191}]},"public_addr":"172.21.15.88:6809/2212808191","cluster_addr":"172.21.15.88:6811/2212808191","heartbeat_back_addr":"172.21.15.88:6815/2212808191","heartbeat_front_addr":"172.21.15.88:6813/2212808191","state":["exists","up"]},{"osd":2,"uuid":"31b450a7-9bd8-4501-b4a1-0ced1a05f331","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6817","nonce":3866598902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6819","nonce":3866598902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6823","nonce":3866598902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3866598902},{"type":"v1","addr":"172.21.15.88:6821","nonce":3866598902}]},"public_addr":"172.21.15.88:6817/3866598902","cluster_addr":"172.21.15.88:6819/3866598902","heartbeat_back_addr":"172.21.15.88:6823/3866598902","heartbeat_front_addr":"172.21.15.88:6821/3866598902","state":["exists","up"]},{"osd":3,"uuid":"eb397ce8-ae27-4102-a5da-e0db2311baa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6825","nonce":1777716082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6827","nonce":1777716082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6831","nonce":1777716082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1777716082},{"type":"v1","addr":"172.21.15.88:6829","nonce":1777716082}]},"public_addr":"172.21.15.88:6825/1777716082","cluster_addr":"172.21.15.88:6827/1777716082","heartbeat_back_addr":"172.21.15.88:6831/1777716082","heartbeat_front_addr":"172.21.15.88:6829/1777716082","state":["exists","up"]},{"osd":4,"uuid":"dcdd1519-0732-4d8c-a3b1-f553e26ffef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6809","nonce":722063332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6811","nonce":722063332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6815","nonce":722063332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":722063332},{"type":"v1","addr":"172.21.15.94:6813","nonce":722063332}]},"public_addr":"172.21.15.94:6809/722063332","cluster_addr":"172.21.15.94:6811/722063332","heartbeat_back_addr":"172.21.15.94:6815/722063332","heartbeat_front_addr":"172.21.15.94:6813/722063332","state":["exists","up"]},{"osd":5,"uuid":"131b0869-de39-42a9-9fb3-718426d9290f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6801","nonce":3097673529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6803","nonce":3097673529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6807","nonce":3097673529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3097673529},{"type":"v1","addr":"172.21.15.94:6805","nonce":3097673529}]},"public_addr":"172.21.15.94:6801/3097673529","cluster_addr":"172.21.15.94:6803/3097673529","heartbeat_back_addr":"172.21.15.94:6807/3097673529","heartbeat_front_addr":"172.21.15.94:6805/3097673529","state":["exists","up"]},{"osd":6,"uuid":"5a576a94-dbf3-4dbb-9017-021d0da8ca66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6817","nonce":175389267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6819","nonce":175389267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6823","nonce":175389267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":175389267},{"type":"v1","addr":"172.21.15.94:6821","nonce":175389267}]},"public_addr":"172.21.15.94:6817/175389267","cluster_addr":"172.21.15.94:6819/175389267","heartbeat_back_addr":"172.21.15.94:6823/175389267","heartbeat_front_addr":"172.21.15.94:6821/175389267","state":["exists","up"]},{"osd":7,"uuid":"058c9901-1985-45bd-afdd-893a577c9adf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6825","nonce":1057220797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6827","nonce":1057220797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6831","nonce":1057220797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1057220797},{"type":"v1","addr":"172.21.15.94:6829","nonce":1057220797}]},"public_addr":"172.21.15.94:6825/1057220797","cluster_addr":"172.21.15.94:6827/1057220797","heartbeat_back_addr":"172.21.15.94:6831/1057220797","heartbeat_front_addr":"172.21.15.94:6829/1057220797","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.351796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.489730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.518183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.718087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.657009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.430539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.618627+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T20:04:39.638525+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.88:6836/1018125900":"2023-10-17T20:07:57.266869+0000","172.21.15.88:6834/1018125900":"2023-10-17T20:07:57.266869+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T20:08:03.580 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.572+0000 7feed1ec9640 1 -- 172.21.15.88:0/3097264418 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7feeac0664d0 msgr2=0x7feeac068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:03.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.572+0000 7feed1ec9640 1 --2- 172.21.15.88:0/3097264418 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7feeac0664d0 0x7feeac068990 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7feeb4006e90 tx=0x7feeb4006de0 comp rx=0 tx=0).stop 2023-10-16T20:08:03.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.572+0000 7feed1ec9640 1 -- 172.21.15.88:0/3097264418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feecc071450 msgr2=0x7feecc0c85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:03.581 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.572+0000 7feed1ec9640 1 --2- 172.21.15.88:0/3097264418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feecc071450 0x7feecc0c85f0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7feebc0b0040 tx=0x7feebc019220 comp rx=0 tx=0).stop 2023-10-16T20:08:03.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.572+0000 7feed1ec9640 1 -- 172.21.15.88:0/3097264418 shutdown_connections 2023-10-16T20:08:03.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.572+0000 7feed1ec9640 1 --2- 172.21.15.88:0/3097264418 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7feeac0664d0 0x7feeac068990 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:03.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.572+0000 7feed1ec9640 1 --2- 172.21.15.88:0/3097264418 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feecc0cb0e0 0x7feecc158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:03.582 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.572+0000 7feed1ec9640 1 --2- 172.21.15.88:0/3097264418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feecc15cc60 0x7feecc150500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:03.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.572+0000 7feed1ec9640 1 --2- 172.21.15.88:0/3097264418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feecc071450 0x7feecc0c85f0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:03.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.572+0000 7feed1ec9640 1 -- 172.21.15.88:0/3097264418 >> 172.21.15.88:0/3097264418 conn(0x7feecc0ceb20 msgr2=0x7feecc0afe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:03.583 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.572+0000 7feed1ec9640 1 -- 172.21.15.88:0/3097264418 shutdown_connections 2023-10-16T20:08:03.584 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:03.572+0000 7feed1ec9640 1 -- 172.21.15.88:0/3097264418 wait complete. 2023-10-16T20:08:04.600 INFO:tasks.ceph:Scrubbing osd.0 2023-10-16T20:08:04.601 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-16T20:08:04.742 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.736+0000 7fe8a0eea640 1 Processor -- start 2023-10-16T20:08:04.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.736+0000 7fe8a0eea640 1 -- start start 2023-10-16T20:08:04.743 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.740+0000 7fe8a0eea640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe89c0c8cb0 0x7fe89c069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:04.744 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.740+0000 7fe8a0eea640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe89c159b10 0x7fe89c06a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:04.744 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.740+0000 7fe8a0eea640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe89c15d660 con 0x7fe89c159730 2023-10-16T20:08:04.744 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.740+0000 7fe8a0eea640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe89c15e2e0 con 0x7fe89c0c8cb0 2023-10-16T20:08:04.745 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.740+0000 7fe8a0eea640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c15d940 con 0x7fe89c159b10 2023-10-16T20:08:04.746 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.740+0000 7fe89ad9d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe89c159b10 0x7fe89c06a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:04.746 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.740+0000 7fe899d9b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe89c0c8cb0 0x7fe89c069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:04.746 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.740+0000 7fe89ad9d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe89c159b10 0x7fe89c06a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.88:34586/0 (socket says 172.21.15.88:34586) 2023-10-16T20:08:04.747 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.740+0000 7fe89ad9d640 1 -- 172.21.15.88:0/3256026785 learned_addr learned my addr 172.21.15.88:0/3256026785 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:04.747 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.740+0000 7fe89a59c640 1 --1- 172.21.15.88:0/3256026785 >> v1:172.21.15.94:6789/0 conn(0x7fe89c159730 0x7fe89c0ad700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:46854/0 (socket says 172.21.15.88:46854) 2023-10-16T20:08:04.747 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.740+0000 7fe89ad9d640 1 -- 172.21.15.88:0/3256026785 >> v2:172.21.15.94:3300/0 conn(0x7fe89c0c8cb0 msgr2=0x7fe89c069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:08:04.747 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.740+0000 7fe89ad9d640 1 --2- 172.21.15.88:0/3256026785 >> v2:172.21.15.94:3300/0 conn(0x7fe89c0c8cb0 0x7fe89c069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:04.748 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.740+0000 7fe89ad9d640 1 -- 172.21.15.88:0/3256026785 >> v1:172.21.15.94:6789/0 conn(0x7fe89c159730 legacy=0x7fe89c0ad700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:04.748 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.740+0000 7fe89ad9d640 1 -- 172.21.15.88:0/3256026785 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89c15dc20 con 0x7fe89c159b10 2023-10-16T20:08:04.748 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.740+0000 7fe89959a640 1 -- 172.21.15.88:0/3256026785 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4127086901 0 0) 0x7fe89c15d660 con 0x7fe89c159730 2023-10-16T20:08:04.749 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.740+0000 7fe899d9b640 1 --2- 172.21.15.88:0/3256026785 >> v2:172.21.15.94:3300/0 conn(0x7fe89c0c8cb0 0x7fe89c069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:08:04.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.744+0000 7fe89ad9d640 1 --2- 172.21.15.88:0/3256026785 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe89c159b10 0x7fe89c06a3d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe89400bb10 tx=0x7fe89401ebe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d0eb8548e963e09 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:04.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.744+0000 7fe89959a640 1 -- 172.21.15.88:0/3256026785 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8940a7b90 con 0x7fe89c159b10 2023-10-16T20:08:04.750 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.744+0000 7fe89959a640 1 -- 172.21.15.88:0/3256026785 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8940a7cf0 con 0x7fe89c159b10 2023-10-16T20:08:04.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.744+0000 7fe89959a640 1 -- 172.21.15.88:0/3256026785 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8940af660 con 0x7fe89c159b10 2023-10-16T20:08:04.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.744+0000 7fe8a0eea640 1 -- 172.21.15.88:0/3256026785 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe89c159b10 msgr2=0x7fe89c06a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:04.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.744+0000 7fe8a0eea640 1 --2- 172.21.15.88:0/3256026785 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe89c159b10 0x7fe89c06a3d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe89400bb10 tx=0x7fe89401ebe0 comp rx=0 tx=0).stop 2023-10-16T20:08:04.751 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.744+0000 7fe8a0eea640 1 -- 172.21.15.88:0/3256026785 shutdown_connections 2023-10-16T20:08:04.752 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.744+0000 7fe8a0eea640 1 --2- 172.21.15.88:0/3256026785 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe89c159b10 0x7fe89c06a3d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:04.752 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.744+0000 7fe8a0eea640 1 --2- 172.21.15.88:0/3256026785 >> v2:172.21.15.94:3300/0 conn(0x7fe89c0c8cb0 0x7fe89c069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:04.752 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.744+0000 7fe8a0eea640 1 -- 172.21.15.88:0/3256026785 >> 172.21.15.88:0/3256026785 conn(0x7fe89c0ce660 msgr2=0x7fe89c0cea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:04.753 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.744+0000 7fe8a0eea640 1 -- 172.21.15.88:0/3256026785 shutdown_connections 2023-10-16T20:08:04.753 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.744+0000 7fe8a0eea640 1 -- 172.21.15.88:0/3256026785 wait complete. 2023-10-16T20:08:04.755 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.744+0000 7fe8a0eea640 1 Processor -- start 2023-10-16T20:08:04.755 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.744+0000 7fe8a0eea640 1 -- start start 2023-10-16T20:08:04.756 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.744+0000 7fe8a0eea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe89c0c8cb0 0x7fe89c157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:04.756 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.748+0000 7fe8a0eea640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe89c159730 0x7fe89c0af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:04.756 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.748+0000 7fe8a0eea640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe89c159b10 0x7fe89c0af650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:04.756 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.748+0000 7fe8a0eea640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c0c5730 con 0x7fe89c159b10 2023-10-16T20:08:04.757 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.748+0000 7fe8a0eea640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c0c31f0 con 0x7fe89c0c8cb0 2023-10-16T20:08:04.757 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.748+0000 7fe8a0eea640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c0c34d0 con 0x7fe89c159730 2023-10-16T20:08:04.757 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.748+0000 7fe89ad9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe89c159b10 0x7fe89c0af650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:04.758 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.748+0000 7fe89ad9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe89c159b10 0x7fe89c0af650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:59944/0 (socket says 172.21.15.88:59944) 2023-10-16T20:08:04.758 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.748+0000 7fe89ad9d640 1 -- 172.21.15.88:0/3530301288 learned_addr learned my addr 172.21.15.88:0/3530301288 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:04.758 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.748+0000 7fe899d9b640 1 --2- 172.21.15.88:0/3530301288 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe89c159730 0x7fe89c0af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:04.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.748+0000 7fe89a59c640 1 --2- 172.21.15.88:0/3530301288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe89c0c8cb0 0x7fe89c157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:04.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.748+0000 7fe89ad9d640 1 -- 172.21.15.88:0/3530301288 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe89c159730 msgr2=0x7fe89c0af110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:04.759 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.748+0000 7fe89ad9d640 1 --2- 172.21.15.88:0/3530301288 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe89c159730 0x7fe89c0af110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:04.760 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.748+0000 7fe89ad9d640 1 -- 172.21.15.88:0/3530301288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe89c0c8cb0 msgr2=0x7fe89c157ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:04.760 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.748+0000 7fe89ad9d640 1 --2- 172.21.15.88:0/3530301288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe89c0c8cb0 0x7fe89c157ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:04.760 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.748+0000 7fe89ad9d640 1 -- 172.21.15.88:0/3530301288 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe890009420 con 0x7fe89c159b10 2023-10-16T20:08:04.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.748+0000 7fe89ad9d640 1 --2- 172.21.15.88:0/3530301288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe89c159b10 0x7fe89c0af650 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe89401c810 tx=0x7fe89401d420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:04.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.748+0000 7fe88affd640 1 -- 172.21.15.88:0/3530301288 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8940afd10 con 0x7fe89c159b10 2023-10-16T20:08:04.761 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.748+0000 7fe88affd640 1 -- 172.21.15.88:0/3530301288 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8940afe90 con 0x7fe89c159b10 2023-10-16T20:08:04.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.748+0000 7fe8a0eea640 1 -- 172.21.15.88:0/3530301288 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe89400b770 con 0x7fe89c159b10 2023-10-16T20:08:04.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.748+0000 7fe88affd640 1 -- 172.21.15.88:0/3530301288 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8940ba6c0 con 0x7fe89c159b10 2023-10-16T20:08:04.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.748+0000 7fe8a0eea640 1 -- 172.21.15.88:0/3530301288 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe89c0c3a90 con 0x7fe89c159b10 2023-10-16T20:08:04.762 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.748+0000 7fe8a0eea640 1 -- 172.21.15.88:0/3530301288 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe89c04e9c0 con 0x7fe89c159b10 2023-10-16T20:08:04.763 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.748+0000 7fe88affd640 1 -- 172.21.15.88:0/3530301288 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe8940ba820 con 0x7fe89c159b10 2023-10-16T20:08:04.764 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.752+0000 7fe88affd640 1 --2- 172.21.15.88:0/3530301288 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe87c0664a0 0x7fe87c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:04.764 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.752+0000 7fe88affd640 1 -- 172.21.15.88:0/3530301288 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fe89411f9a0 con 0x7fe89c159b10 2023-10-16T20:08:04.765 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.752+0000 7fe88affd640 1 --2- 172.21.15.88:0/3530301288 >> [v2:172.21.15.88:6800/685586021,v1:172.21.15.88:6801/685586021] conn(0x7fe87c06c7a0 0x7fe87c06ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:04.765 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.752+0000 7fe88affd640 1 -- 172.21.15.88:0/3530301288 --> [v2:172.21.15.88:6800/685586021,v1:172.21.15.88:6801/685586021] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe87c06f2b0 con 0x7fe87c06c7a0 2023-10-16T20:08:04.765 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.752+0000 7fe88affd640 1 -- 172.21.15.88:0/3530301288 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe89411fd10 con 0x7fe89c159b10 2023-10-16T20:08:04.765 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.752+0000 7fe89a59c640 1 --2- 172.21.15.88:0/3530301288 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe87c0664a0 0x7fe87c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:04.766 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.752+0000 7fe899d9b640 1 --2- 172.21.15.88:0/3530301288 >> [v2:172.21.15.88:6800/685586021,v1:172.21.15.88:6801/685586021] conn(0x7fe87c06c7a0 0x7fe87c06ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:04.766 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.752+0000 7fe89a59c640 1 --2- 172.21.15.88:0/3530301288 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe87c0664a0 0x7fe87c068960 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe88c004fd0 tx=0x7fe88c001770 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:04.766 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.752+0000 7fe899d9b640 1 --2- 172.21.15.88:0/3530301288 >> [v2:172.21.15.88:6800/685586021,v1:172.21.15.88:6801/685586021] conn(0x7fe87c06c7a0 0x7fe87c06ebe0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:04.767 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.752+0000 7fe88affd640 1 -- 172.21.15.88:0/3530301288 <== osd.0 v2:172.21.15.88:6800/685586021 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fe87c06f2b0 con 0x7fe87c06c7a0 2023-10-16T20:08:04.769 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.764+0000 7fe8a0eea640 1 -- 172.21.15.88:0/3530301288 --> [v2:172.21.15.88:6800/685586021,v1:172.21.15.88:6801/685586021] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe89c0c5900 con 0x7fe87c06c7a0 2023-10-16T20:08:04.770 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.764+0000 7fe88affd640 1 -- 172.21.15.88:0/3530301288 <== osd.0 v2:172.21.15.88:6800/685586021 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fe89c0c5900 con 0x7fe87c06c7a0 2023-10-16T20:08:04.771 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2023-10-16T20:08:04.771 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T20:08:04.771 INFO:teuthology.orchestra.run.smithi088.stdout:} 2023-10-16T20:08:04.772 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.764+0000 7fe8a0eea640 1 -- 172.21.15.88:0/3530301288 >> [v2:172.21.15.88:6800/685586021,v1:172.21.15.88:6801/685586021] conn(0x7fe87c06c7a0 msgr2=0x7fe87c06ebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:04.773 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.764+0000 7fe8a0eea640 1 --2- 172.21.15.88:0/3530301288 >> [v2:172.21.15.88:6800/685586021,v1:172.21.15.88:6801/685586021] conn(0x7fe87c06c7a0 0x7fe87c06ebe0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:04.773 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.764+0000 7fe8a0eea640 1 -- 172.21.15.88:0/3530301288 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe87c0664a0 msgr2=0x7fe87c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:04.773 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.764+0000 7fe8a0eea640 1 --2- 172.21.15.88:0/3530301288 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fe87c0664a0 0x7fe87c068960 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe88c004fd0 tx=0x7fe88c001770 comp rx=0 tx=0).stop 2023-10-16T20:08:04.774 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.764+0000 7fe8a0eea640 1 -- 172.21.15.88:0/3530301288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe89c159b10 msgr2=0x7fe89c0af650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:04.774 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.764+0000 7fe8a0eea640 1 --2- 172.21.15.88:0/3530301288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe89c159b10 0x7fe89c0af650 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe89401c810 tx=0x7fe89401d420 comp rx=0 tx=0).stop 2023-10-16T20:08:04.774 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.764+0000 7fe89ad9d640 1 -- 172.21.15.88:0/3530301288 reap_dead start 2023-10-16T20:08:04.774 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.764+0000 7fe8a0eea640 1 -- 172.21.15.88:0/3530301288 shutdown_connections 2023-10-16T20:08:04.775 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.764+0000 7fe8a0eea640 1 -- 172.21.15.88:0/3530301288 >> 172.21.15.88:0/3530301288 conn(0x7fe89c0ce660 msgr2=0x7fe89c0c9100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:04.775 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.764+0000 7fe8a0eea640 1 -- 172.21.15.88:0/3530301288 shutdown_connections 2023-10-16T20:08:04.775 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.764+0000 7fe8a0eea640 1 -- 172.21.15.88:0/3530301288 wait complete. 2023-10-16T20:08:04.788 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-16T20:08:04.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.920+0000 7f4681f0f640 1 Processor -- start 2023-10-16T20:08:04.928 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.920+0000 7f4681f0f640 1 -- start start 2023-10-16T20:08:04.930 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.924+0000 7f4681f0f640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f467c069e90 0x7f467c06a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:04.930 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.924+0000 7f4681f0f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f467c15d480 0x7f467c0afa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:04.930 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.924+0000 7f4681f0f640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f467c15e400 con 0x7f467c0adf80 2023-10-16T20:08:04.931 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.924+0000 7f4681f0f640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f467c159e40 con 0x7f467c069e90 2023-10-16T20:08:04.931 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.924+0000 7f4681f0f640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467c06a7b0 con 0x7f467c15d480 2023-10-16T20:08:04.931 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.924+0000 7f467b7fe640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f467c069e90 0x7f467c06a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:04.932 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.924+0000 7f4673fff640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f467c0adf80 0x7f467c15a830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:47224/0 (socket says 172.21.15.88:47224) 2023-10-16T20:08:04.933 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.924+0000 7f4673fff640 1 -- 172.21.15.88:0/2026762277 learned_addr learned my addr 172.21.15.88:0/2026762277 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:08:04.933 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.924+0000 7f467bfff640 1 --2- 172.21.15.88:0/2026762277 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f467c15d480 0x7f467c0afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:04.934 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.924+0000 7f467affd640 1 -- 172.21.15.88:0/2026762277 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189244228 0 0) 0x7f467c15e400 con 0x7f467c0adf80 2023-10-16T20:08:04.934 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.924+0000 7f467affd640 1 -- 172.21.15.88:0/2026762277 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4668003530 con 0x7f467c0adf80 2023-10-16T20:08:04.935 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.924+0000 7f467b7fe640 1 -- 172.21.15.88:0/2026762277 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f467c15d480 msgr2=0x7f467c0afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:08:04.935 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.924+0000 7f467b7fe640 1 --2- 172.21.15.88:0/2026762277 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f467c15d480 0x7f467c0afa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:04.936 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.924+0000 7f467b7fe640 1 -- 172.21.15.88:0/2026762277 >> v1:172.21.15.88:6789/0 conn(0x7f467c0adf80 legacy=0x7f467c15a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:04.937 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.924+0000 7f467b7fe640 1 -- 172.21.15.88:0/2026762277 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f467c15de40 con 0x7f467c069e90 2023-10-16T20:08:04.940 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.928+0000 7f467b7fe640 1 --2- 172.21.15.88:0/2026762277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f467c069e90 0x7f467c06a270 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f466c0097c0 tx=0x7f466c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b44e72a4c35a58d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:04.941 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.928+0000 7f467affd640 1 -- 172.21.15.88:0/2026762277 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466c0ab650 con 0x7f467c069e90 2023-10-16T20:08:04.941 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.928+0000 7f467affd640 1 -- 172.21.15.88:0/2026762277 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f466c0ab7b0 con 0x7f467c069e90 2023-10-16T20:08:04.941 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.928+0000 7f467affd640 1 -- 172.21.15.88:0/2026762277 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466c0abb40 con 0x7f467c069e90 2023-10-16T20:08:04.942 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.928+0000 7f4681f0f640 1 -- 172.21.15.88:0/2026762277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f467c069e90 msgr2=0x7f467c06a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:04.942 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.928+0000 7f4681f0f640 1 --2- 172.21.15.88:0/2026762277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f467c069e90 0x7f467c06a270 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f466c0097c0 tx=0x7f466c01c450 comp rx=0 tx=0).stop 2023-10-16T20:08:04.943 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.928+0000 7f4681f0f640 1 -- 172.21.15.88:0/2026762277 shutdown_connections 2023-10-16T20:08:04.943 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.928+0000 7f4681f0f640 1 --2- 172.21.15.88:0/2026762277 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f467c15d480 0x7f467c0afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:04.944 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.928+0000 7f4681f0f640 1 --2- 172.21.15.88:0/2026762277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f467c069e90 0x7f467c06a270 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:04.944 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.928+0000 7f4681f0f640 1 -- 172.21.15.88:0/2026762277 >> 172.21.15.88:0/2026762277 conn(0x7f467c0ce5e0 msgr2=0x7f467c0ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:04.944 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.928+0000 7f4681f0f640 1 -- 172.21.15.88:0/2026762277 shutdown_connections 2023-10-16T20:08:04.945 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.928+0000 7f4681f0f640 1 -- 172.21.15.88:0/2026762277 wait complete. 2023-10-16T20:08:04.945 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.928+0000 7f4681f0f640 1 Processor -- start 2023-10-16T20:08:04.945 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.928+0000 7f4681f0f640 1 -- start start 2023-10-16T20:08:04.945 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.932+0000 7f4681f0f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f467c069e90 0x7f467c14c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:04.946 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.932+0000 7f4681f0f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f467c0adf80 0x7f467c14bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:04.946 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.932+0000 7f4681f0f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f467c15d480 0x7f467c150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:04.946 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.932+0000 7f4681f0f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c152a00 con 0x7f467c15d480 2023-10-16T20:08:04.946 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.932+0000 7f4681f0f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c0ad3c0 con 0x7f467c069e90 2023-10-16T20:08:04.947 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.932+0000 7f4681f0f640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467c1520d0 con 0x7f467c0adf80 2023-10-16T20:08:04.947 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.932+0000 7f467bfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f467c15d480 0x7f467c150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:04.948 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.932+0000 7f467bfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f467c15d480 0x7f467c150f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:59964/0 (socket says 172.21.15.88:59964) 2023-10-16T20:08:04.948 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.932+0000 7f467bfff640 1 -- 172.21.15.88:0/3847629820 learned_addr learned my addr 172.21.15.88:0/3847629820 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:04.949 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.932+0000 7f467b7fe640 1 --2- 172.21.15.88:0/3847629820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f467c069e90 0x7f467c14c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:04.949 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.932+0000 7f4673fff640 1 --2- 172.21.15.88:0/3847629820 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f467c0adf80 0x7f467c14bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:04.949 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.932+0000 7f467bfff640 1 -- 172.21.15.88:0/3847629820 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f467c0adf80 msgr2=0x7f467c14bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:04.950 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.932+0000 7f467bfff640 1 --2- 172.21.15.88:0/3847629820 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f467c0adf80 0x7f467c14bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:04.951 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.932+0000 7f467bfff640 1 -- 172.21.15.88:0/3847629820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f467c069e90 msgr2=0x7f467c14c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:04.951 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.932+0000 7f467bfff640 1 --2- 172.21.15.88:0/3847629820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f467c069e90 0x7f467c14c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:04.951 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.932+0000 7f467bfff640 1 -- 172.21.15.88:0/3847629820 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466c009420 con 0x7f467c15d480 2023-10-16T20:08:04.952 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.932+0000 7f467bfff640 1 --2- 172.21.15.88:0/3847629820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f467c15d480 0x7f467c150f00 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f467400efc0 tx=0x7f467400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:04.952 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.932+0000 7f4678ff9640 1 -- 172.21.15.88:0/3847629820 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4674009ba0 con 0x7f467c15d480 2023-10-16T20:08:04.953 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.932+0000 7f4678ff9640 1 -- 172.21.15.88:0/3847629820 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467400f040 con 0x7f467c15d480 2023-10-16T20:08:04.953 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.932+0000 7f4678ff9640 1 -- 172.21.15.88:0/3847629820 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46740146c0 con 0x7f467c15d480 2023-10-16T20:08:04.954 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.932+0000 7f4681f0f640 1 -- 172.21.15.88:0/3847629820 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f467c1523b0 con 0x7f467c15d480 2023-10-16T20:08:04.955 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.932+0000 7f4681f0f640 1 -- 172.21.15.88:0/3847629820 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f467c14c4c0 con 0x7f467c15d480 2023-10-16T20:08:04.955 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.932+0000 7f4681f0f640 1 -- 172.21.15.88:0/3847629820 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f467c06d180 con 0x7f467c15d480 2023-10-16T20:08:04.955 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.932+0000 7f4678ff9640 1 -- 172.21.15.88:0/3847629820 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4674014820 con 0x7f467c15d480 2023-10-16T20:08:04.956 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.936+0000 7f4678ff9640 1 --2- 172.21.15.88:0/3847629820 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f465c0664a0 0x7f465c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:04.956 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.936+0000 7f4678ff9640 1 -- 172.21.15.88:0/3847629820 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f4674084820 con 0x7f467c15d480 2023-10-16T20:08:04.957 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.936+0000 7f467b7fe640 1 --2- 172.21.15.88:0/3847629820 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f465c0664a0 0x7f465c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:04.957 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.936+0000 7f467b7fe640 1 --2- 172.21.15.88:0/3847629820 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f465c0664a0 0x7f465c068960 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f466c0af000 tx=0x7f466c0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:04.958 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:04.940+0000 7f4678ff9640 1 -- 172.21.15.88:0/3847629820 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4674051dc0 con 0x7f467c15d480 2023-10-16T20:08:05.096 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.088+0000 7f4681f0f640 1 -- 172.21.15.88:0/3847629820 --> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f467c154ed0 con 0x7f465c0664a0 2023-10-16T20:08:05.097 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.092+0000 7f4678ff9640 1 -- 172.21.15.88:0/3847629820 <== mgr.4110 v2:172.21.15.88:6832/4078483333 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f467c154ed0 con 0x7f465c0664a0 2023-10-16T20:08:05.097 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 0 to deep-scrub 2023-10-16T20:08:05.099 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.092+0000 7f4681f0f640 1 -- 172.21.15.88:0/3847629820 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f465c0664a0 msgr2=0x7f465c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:05.100 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.092+0000 7f4681f0f640 1 --2- 172.21.15.88:0/3847629820 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f465c0664a0 0x7f465c068960 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f466c0af000 tx=0x7f466c0a7000 comp rx=0 tx=0).stop 2023-10-16T20:08:05.100 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.092+0000 7f4681f0f640 1 -- 172.21.15.88:0/3847629820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f467c15d480 msgr2=0x7f467c150f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:05.100 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.092+0000 7f4681f0f640 1 --2- 172.21.15.88:0/3847629820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f467c15d480 0x7f467c150f00 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f467400efc0 tx=0x7f467400c490 comp rx=0 tx=0).stop 2023-10-16T20:08:05.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.092+0000 7f4681f0f640 1 -- 172.21.15.88:0/3847629820 shutdown_connections 2023-10-16T20:08:05.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.092+0000 7f4681f0f640 1 --2- 172.21.15.88:0/3847629820 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f465c0664a0 0x7f465c068960 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:05.101 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.092+0000 7f4681f0f640 1 --2- 172.21.15.88:0/3847629820 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f467c0adf80 0x7f467c14bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:05.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.092+0000 7f4681f0f640 1 --2- 172.21.15.88:0/3847629820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f467c15d480 0x7f467c150f00 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:05.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.092+0000 7f4681f0f640 1 --2- 172.21.15.88:0/3847629820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f467c069e90 0x7f467c14c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:05.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.092+0000 7f4681f0f640 1 -- 172.21.15.88:0/3847629820 >> 172.21.15.88:0/3847629820 conn(0x7f467c0ce5e0 msgr2=0x7f467c0ca800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:05.102 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.092+0000 7f4681f0f640 1 -- 172.21.15.88:0/3847629820 shutdown_connections 2023-10-16T20:08:05.103 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.092+0000 7f4681f0f640 1 -- 172.21.15.88:0/3847629820 wait complete. 2023-10-16T20:08:05.117 INFO:tasks.ceph:Scrubbing osd.1 2023-10-16T20:08:05.118 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-16T20:08:05.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.252+0000 7f0297a46640 1 Processor -- start 2023-10-16T20:08:05.260 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.256+0000 7f0297a46640 1 -- start start 2023-10-16T20:08:05.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.256+0000 7f0297a46640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f029015cc20 0x7f02900d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:05.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.256+0000 7f0297a46640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f02900cd8b0 0x7f02900cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:05.262 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.256+0000 7f0297a46640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02900d1490 con 0x7f02900d2530 2023-10-16T20:08:05.263 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.256+0000 7f0297a46640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02900d2110 con 0x7f02900cd8b0 2023-10-16T20:08:05.263 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.256+0000 7f0297a46640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02900ce2a0 con 0x7f029015cc20 2023-10-16T20:08:05.263 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.256+0000 7f0294fe1640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f02900d2530 0x7f02900b8490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:47240/0 (socket says 172.21.15.88:47240) 2023-10-16T20:08:05.263 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.256+0000 7f0294fe1640 1 -- 172.21.15.88:0/1224651484 learned_addr learned my addr 172.21.15.88:0/1224651484 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:08:05.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.256+0000 7f0295fe3640 1 --2- 172.21.15.88:0/1224651484 >> v2:172.21.15.94:3300/0 conn(0x7f02900cd8b0 0x7f02900cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:05.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.256+0000 7f02957e2640 1 --2- 172.21.15.88:0/1224651484 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f029015cc20 0x7f02900d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:05.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.256+0000 7f0287fff640 1 -- 172.21.15.88:0/1224651484 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 225414234 0 0) 0x7f02900d1490 con 0x7f02900d2530 2023-10-16T20:08:05.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.256+0000 7f0287fff640 1 -- 172.21.15.88:0/1224651484 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f027c003530 con 0x7f02900d2530 2023-10-16T20:08:05.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.256+0000 7f0295fe3640 1 -- 172.21.15.88:0/1224651484 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f029015cc20 msgr2=0x7f02900d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:08:05.265 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.256+0000 7f0295fe3640 1 --2- 172.21.15.88:0/1224651484 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f029015cc20 0x7f02900d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:05.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.256+0000 7f0295fe3640 1 -- 172.21.15.88:0/1224651484 >> v1:172.21.15.88:6789/0 conn(0x7f02900d2530 legacy=0x7f02900b8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:05.266 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.256+0000 7f0295fe3640 1 -- 172.21.15.88:0/1224651484 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02900d1770 con 0x7f02900cd8b0 2023-10-16T20:08:05.267 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.260+0000 7f0295fe3640 1 --2- 172.21.15.88:0/1224651484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02900cd8b0 0x7f02900cdd60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0288009d90 tx=0x7f028801c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98aa78715ceb4d73 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:05.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.260+0000 7f0287fff640 1 -- 172.21.15.88:0/1224651484 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02880a59e0 con 0x7f02900cd8b0 2023-10-16T20:08:05.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.260+0000 7f0287fff640 1 -- 172.21.15.88:0/1224651484 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02880a5b40 con 0x7f02900cd8b0 2023-10-16T20:08:05.268 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.260+0000 7f0287fff640 1 -- 172.21.15.88:0/1224651484 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02880ad440 con 0x7f02900cd8b0 2023-10-16T20:08:05.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.260+0000 7f0297a46640 1 -- 172.21.15.88:0/1224651484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02900cd8b0 msgr2=0x7f02900cdd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:05.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.260+0000 7f0297a46640 1 --2- 172.21.15.88:0/1224651484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02900cd8b0 0x7f02900cdd60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0288009d90 tx=0x7f028801c920 comp rx=0 tx=0).stop 2023-10-16T20:08:05.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.260+0000 7f0297a46640 1 -- 172.21.15.88:0/1224651484 shutdown_connections 2023-10-16T20:08:05.269 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.260+0000 7f0297a46640 1 --2- 172.21.15.88:0/1224651484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02900cd8b0 0x7f02900cdd60 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:05.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.260+0000 7f0297a46640 1 --2- 172.21.15.88:0/1224651484 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f029015cc20 0x7f02900d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:05.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.260+0000 7f0297a46640 1 -- 172.21.15.88:0/1224651484 >> 172.21.15.88:0/1224651484 conn(0x7f029006b120 msgr2=0x7f029006b530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:05.270 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.260+0000 7f0297a46640 1 -- 172.21.15.88:0/1224651484 shutdown_connections 2023-10-16T20:08:05.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.260+0000 7f0297a46640 1 -- 172.21.15.88:0/1224651484 wait complete. 2023-10-16T20:08:05.271 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.264+0000 7f0297a46640 1 Processor -- start 2023-10-16T20:08:05.273 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.264+0000 7f0297a46640 1 -- start start 2023-10-16T20:08:05.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.264+0000 7f0297a46640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02900cd8b0 0x7f0290157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:05.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.264+0000 7f0297a46640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02900d2530 0x7f02901f4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:05.274 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.264+0000 7f0297a46640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f029015cc20 0x7f029015b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:05.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.264+0000 7f0297a46640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029015e6b0 con 0x7f02900cd8b0 2023-10-16T20:08:05.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.264+0000 7f0297a46640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029015e950 con 0x7f02900d2530 2023-10-16T20:08:05.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.264+0000 7f0297a46640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02901f55e0 con 0x7f029015cc20 2023-10-16T20:08:05.275 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.264+0000 7f02957e2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02900cd8b0 0x7f0290157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:05.276 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.264+0000 7f02957e2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02900cd8b0 0x7f0290157fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:59976/0 (socket says 172.21.15.88:59976) 2023-10-16T20:08:05.276 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.264+0000 7f02957e2640 1 -- 172.21.15.88:0/4032178900 learned_addr learned my addr 172.21.15.88:0/4032178900 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:05.276 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.264+0000 7f0295fe3640 1 --2- 172.21.15.88:0/4032178900 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f029015cc20 0x7f029015b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:05.277 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.264+0000 7f0294fe1640 1 --2- 172.21.15.88:0/4032178900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02900d2530 0x7f02901f4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:05.277 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.264+0000 7f02957e2640 1 -- 172.21.15.88:0/4032178900 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f029015cc20 msgr2=0x7f029015b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:05.277 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.264+0000 7f02957e2640 1 --2- 172.21.15.88:0/4032178900 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f029015cc20 0x7f029015b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:05.277 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.264+0000 7f02957e2640 1 -- 172.21.15.88:0/4032178900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02900d2530 msgr2=0x7f02901f4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:05.278 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.264+0000 7f02957e2640 1 --2- 172.21.15.88:0/4032178900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02900d2530 0x7f02901f4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:05.278 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.264+0000 7f02957e2640 1 -- 172.21.15.88:0/4032178900 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0288009a70 con 0x7f02900cd8b0 2023-10-16T20:08:05.278 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.264+0000 7f0294fe1640 1 --2- 172.21.15.88:0/4032178900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02900d2530 0x7f02901f4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:08:05.279 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.264+0000 7f02957e2640 1 --2- 172.21.15.88:0/4032178900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02900cd8b0 0x7f0290157fb0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f028c00e870 tx=0x7f028c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:05.279 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.264+0000 7f0285ffb640 1 -- 172.21.15.88:0/4032178900 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c009510 con 0x7f02900cd8b0 2023-10-16T20:08:05.279 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.264+0000 7f0285ffb640 1 -- 172.21.15.88:0/4032178900 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028c00eec0 con 0x7f02900cd8b0 2023-10-16T20:08:05.280 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.264+0000 7f0297a46640 1 -- 172.21.15.88:0/4032178900 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02901f58c0 con 0x7f02900cd8b0 2023-10-16T20:08:05.280 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.264+0000 7f0285ffb640 1 -- 172.21.15.88:0/4032178900 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c00cb40 con 0x7f02900cd8b0 2023-10-16T20:08:05.280 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.264+0000 7f0297a46640 1 -- 172.21.15.88:0/4032178900 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02901f52f0 con 0x7f02900cd8b0 2023-10-16T20:08:05.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.264+0000 7f0297a46640 1 -- 172.21.15.88:0/4032178900 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f029004e9c0 con 0x7f02900cd8b0 2023-10-16T20:08:05.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.268+0000 7f0285ffb640 1 -- 172.21.15.88:0/4032178900 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f028c00cca0 con 0x7f02900cd8b0 2023-10-16T20:08:05.281 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.268+0000 7f0285ffb640 1 --2- 172.21.15.88:0/4032178900 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f02740664d0 0x7f0274068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:05.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.268+0000 7f0285ffb640 1 -- 172.21.15.88:0/4032178900 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f028c083dd0 con 0x7f02900cd8b0 2023-10-16T20:08:05.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.268+0000 7f0294fe1640 1 --2- 172.21.15.88:0/4032178900 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f02740664d0 0x7f0274068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:05.282 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.268+0000 7f0285ffb640 1 --2- 172.21.15.88:0/4032178900 >> [v2:172.21.15.88:6808/2212808191,v1:172.21.15.88:6809/2212808191] conn(0x7f027406c7d0 0x7f027406ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:05.283 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.268+0000 7f0285ffb640 1 -- 172.21.15.88:0/4032178900 --> [v2:172.21.15.88:6808/2212808191,v1:172.21.15.88:6809/2212808191] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f027406f2e0 con 0x7f027406c7d0 2023-10-16T20:08:05.283 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.268+0000 7f0285ffb640 1 -- 172.21.15.88:0/4032178900 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f028c087020 con 0x7f02900cd8b0 2023-10-16T20:08:05.283 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.268+0000 7f0295fe3640 1 --2- 172.21.15.88:0/4032178900 >> [v2:172.21.15.88:6808/2212808191,v1:172.21.15.88:6809/2212808191] conn(0x7f027406c7d0 0x7f027406ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:05.284 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.268+0000 7f0294fe1640 1 --2- 172.21.15.88:0/4032178900 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f02740664d0 0x7f0274068990 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f02901584f0 tx=0x7f0280009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:05.284 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.268+0000 7f0295fe3640 1 --2- 172.21.15.88:0/4032178900 >> [v2:172.21.15.88:6808/2212808191,v1:172.21.15.88:6809/2212808191] conn(0x7f027406c7d0 0x7f027406ec10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:05.285 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.268+0000 7f0285ffb640 1 -- 172.21.15.88:0/4032178900 <== osd.1 v2:172.21.15.88:6808/2212808191 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f027406f2e0 con 0x7f027406c7d0 2023-10-16T20:08:05.287 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.280+0000 7f0297a46640 1 -- 172.21.15.88:0/4032178900 --> [v2:172.21.15.88:6808/2212808191,v1:172.21.15.88:6809/2212808191] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f029015ea90 con 0x7f027406c7d0 2023-10-16T20:08:05.288 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.284+0000 7f0285ffb640 1 -- 172.21.15.88:0/4032178900 <== osd.1 v2:172.21.15.88:6808/2212808191 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f0288009450 con 0x7f027406c7d0 2023-10-16T20:08:05.288 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2023-10-16T20:08:05.289 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T20:08:05.289 INFO:teuthology.orchestra.run.smithi088.stdout:} 2023-10-16T20:08:05.290 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.284+0000 7f0297a46640 1 -- 172.21.15.88:0/4032178900 >> [v2:172.21.15.88:6808/2212808191,v1:172.21.15.88:6809/2212808191] conn(0x7f027406c7d0 msgr2=0x7f027406ec10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:05.290 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.284+0000 7f0297a46640 1 --2- 172.21.15.88:0/4032178900 >> [v2:172.21.15.88:6808/2212808191,v1:172.21.15.88:6809/2212808191] conn(0x7f027406c7d0 0x7f027406ec10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:05.291 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.284+0000 7f0297a46640 1 -- 172.21.15.88:0/4032178900 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f02740664d0 msgr2=0x7f0274068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:05.291 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.284+0000 7f0297a46640 1 --2- 172.21.15.88:0/4032178900 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f02740664d0 0x7f0274068990 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f02901584f0 tx=0x7f0280009040 comp rx=0 tx=0).stop 2023-10-16T20:08:05.291 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.284+0000 7f0297a46640 1 -- 172.21.15.88:0/4032178900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02900cd8b0 msgr2=0x7f0290157fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:05.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.284+0000 7f0297a46640 1 --2- 172.21.15.88:0/4032178900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02900cd8b0 0x7f0290157fb0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f028c00e870 tx=0x7f028c00ec70 comp rx=0 tx=0).stop 2023-10-16T20:08:05.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.284+0000 7f0295fe3640 1 -- 172.21.15.88:0/4032178900 reap_dead start 2023-10-16T20:08:05.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.284+0000 7f0297a46640 1 -- 172.21.15.88:0/4032178900 shutdown_connections 2023-10-16T20:08:05.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.284+0000 7f0297a46640 1 -- 172.21.15.88:0/4032178900 >> 172.21.15.88:0/4032178900 conn(0x7f029006b120 msgr2=0x7f02900c68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:05.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.284+0000 7f0297a46640 1 -- 172.21.15.88:0/4032178900 shutdown_connections 2023-10-16T20:08:05.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.284+0000 7f0297a46640 1 -- 172.21.15.88:0/4032178900 wait complete. 2023-10-16T20:08:05.307 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-16T20:08:05.445 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.440+0000 7f90e7d3e640 1 Processor -- start 2023-10-16T20:08:05.446 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.440+0000 7f90e7d3e640 1 -- start start 2023-10-16T20:08:05.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.440+0000 7f90e7d3e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f90e00adf80 0x7f90e015a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:05.447 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.440+0000 7f90e7d3e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f90e015d480 0x7f90e00afa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:05.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.440+0000 7f90e7d3e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e015e400 con 0x7f90e0069e90 2023-10-16T20:08:05.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.440+0000 7f90e7d3e640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e0159e40 con 0x7f90e00adf80 2023-10-16T20:08:05.448 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.440+0000 7f90e7d3e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e015de40 con 0x7f90e015d480 2023-10-16T20:08:05.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.440+0000 7f90e62db640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f90e015d480 0x7f90e00afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:05.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.440+0000 7f90e5ada640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f90e0069e90 0x7f90e006a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:46858/0 (socket says 172.21.15.88:46858) 2023-10-16T20:08:05.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.440+0000 7f90e52d9640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f90e00adf80 0x7f90e015a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:05.449 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.440+0000 7f90e5ada640 1 -- 172.21.15.88:0/898149683 learned_addr learned my addr 172.21.15.88:0/898149683 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:08:05.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.444+0000 7f90e4ad8640 1 -- 172.21.15.88:0/898149683 <== mon.1 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 701106609 0 0) 0x7f90e015e400 con 0x7f90e0069e90 2023-10-16T20:08:05.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.444+0000 7f90e4ad8640 1 -- 172.21.15.88:0/898149683 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90cc003530 con 0x7f90e0069e90 2023-10-16T20:08:05.451 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.444+0000 7f90e52d9640 1 -- 172.21.15.88:0/898149683 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f90e015d480 msgr2=0x7f90e00afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:08:05.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.444+0000 7f90e52d9640 1 --2- 172.21.15.88:0/898149683 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f90e015d480 0x7f90e00afa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:05.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.444+0000 7f90e52d9640 1 -- 172.21.15.88:0/898149683 >> v1:172.21.15.94:6789/0 conn(0x7f90e0069e90 legacy=0x7f90e006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:05.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.444+0000 7f90e52d9640 1 -- 172.21.15.88:0/898149683 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e015e120 con 0x7f90e00adf80 2023-10-16T20:08:05.452 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.444+0000 7f90e52d9640 1 --2- 172.21.15.88:0/898149683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90e00adf80 0x7f90e015a830 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f90d00097c0 tx=0x7f90d001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79b5e2f641b983d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:05.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.444+0000 7f90e4ad8640 1 -- 172.21.15.88:0/898149683 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d00ab650 con 0x7f90e00adf80 2023-10-16T20:08:05.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.444+0000 7f90e4ad8640 1 -- 172.21.15.88:0/898149683 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d00ab7b0 con 0x7f90e00adf80 2023-10-16T20:08:05.453 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.444+0000 7f90e4ad8640 1 -- 172.21.15.88:0/898149683 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d00abb40 con 0x7f90e00adf80 2023-10-16T20:08:05.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.444+0000 7f90e7d3e640 1 -- 172.21.15.88:0/898149683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90e00adf80 msgr2=0x7f90e015a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:05.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.444+0000 7f90e7d3e640 1 --2- 172.21.15.88:0/898149683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90e00adf80 0x7f90e015a830 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f90d00097c0 tx=0x7f90d001c450 comp rx=0 tx=0).stop 2023-10-16T20:08:05.456 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.444+0000 7f90e7d3e640 1 -- 172.21.15.88:0/898149683 shutdown_connections 2023-10-16T20:08:05.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.444+0000 7f90e7d3e640 1 --2- 172.21.15.88:0/898149683 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f90e015d480 0x7f90e00afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:05.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.444+0000 7f90e7d3e640 1 --2- 172.21.15.88:0/898149683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90e00adf80 0x7f90e015a830 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:05.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.448+0000 7f90e7d3e640 1 -- 172.21.15.88:0/898149683 >> 172.21.15.88:0/898149683 conn(0x7f90e00ce5e0 msgr2=0x7f90e00ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:05.457 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.448+0000 7f90e7d3e640 1 -- 172.21.15.88:0/898149683 shutdown_connections 2023-10-16T20:08:05.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.448+0000 7f90e7d3e640 1 -- 172.21.15.88:0/898149683 wait complete. 2023-10-16T20:08:05.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.448+0000 7f90e7d3e640 1 Processor -- start 2023-10-16T20:08:05.458 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.448+0000 7f90e7d3e640 1 -- start start 2023-10-16T20:08:05.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.448+0000 7f90e7d3e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90e0069e90 0x7f90e015c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:05.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.448+0000 7f90e7d3e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90e00adf80 0x7f90e0148ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:05.459 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.448+0000 7f90e7d3e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f90e015d480 0x7f90e00b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:05.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.448+0000 7f90e7d3e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e012f8b0 con 0x7f90e0069e90 2023-10-16T20:08:05.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.448+0000 7f90e7d3e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e014aa20 con 0x7f90e00adf80 2023-10-16T20:08:05.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.448+0000 7f90e7d3e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e0144210 con 0x7f90e015d480 2023-10-16T20:08:05.460 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.448+0000 7f90e5ada640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90e0069e90 0x7f90e015c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:05.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.448+0000 7f90e5ada640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90e0069e90 0x7f90e015c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:59982/0 (socket says 172.21.15.88:59982) 2023-10-16T20:08:05.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.448+0000 7f90e5ada640 1 -- 172.21.15.88:0/2295552707 learned_addr learned my addr 172.21.15.88:0/2295552707 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:05.461 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.448+0000 7f90e62db640 1 --2- 172.21.15.88:0/2295552707 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f90e015d480 0x7f90e00b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:05.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.448+0000 7f90e52d9640 1 --2- 172.21.15.88:0/2295552707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90e00adf80 0x7f90e0148ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:05.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.448+0000 7f90e5ada640 1 -- 172.21.15.88:0/2295552707 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f90e015d480 msgr2=0x7f90e00b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:05.462 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.448+0000 7f90e5ada640 1 --2- 172.21.15.88:0/2295552707 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f90e015d480 0x7f90e00b06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:05.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.448+0000 7f90e5ada640 1 -- 172.21.15.88:0/2295552707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90e00adf80 msgr2=0x7f90e0148ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:05.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.448+0000 7f90e5ada640 1 --2- 172.21.15.88:0/2295552707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90e00adf80 0x7f90e0148ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:05.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.448+0000 7f90e5ada640 1 -- 172.21.15.88:0/2295552707 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d0009420 con 0x7f90e0069e90 2023-10-16T20:08:05.463 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.448+0000 7f90e52d9640 1 --2- 172.21.15.88:0/2295552707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90e00adf80 0x7f90e0148ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:08:05.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.448+0000 7f90e5ada640 1 --2- 172.21.15.88:0/2295552707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90e0069e90 0x7f90e015c960 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f90dc00c500 tx=0x7f90dc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:05.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.448+0000 7f90d67fc640 1 -- 172.21.15.88:0/2295552707 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc005cf0 con 0x7f90e0069e90 2023-10-16T20:08:05.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.452+0000 7f90d67fc640 1 -- 172.21.15.88:0/2295552707 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90dc005e50 con 0x7f90e0069e90 2023-10-16T20:08:05.464 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.452+0000 7f90d67fc640 1 -- 172.21.15.88:0/2295552707 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc0072f0 con 0x7f90e0069e90 2023-10-16T20:08:05.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.452+0000 7f90e7d3e640 1 -- 172.21.15.88:0/2295552707 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e012ed30 con 0x7f90e0069e90 2023-10-16T20:08:05.465 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.452+0000 7f90e7d3e640 1 -- 172.21.15.88:0/2295552707 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e012f2f0 con 0x7f90e0069e90 2023-10-16T20:08:05.466 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.452+0000 7f90e7d3e640 1 -- 172.21.15.88:0/2295552707 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e006d180 con 0x7f90e0069e90 2023-10-16T20:08:05.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.452+0000 7f90d67fc640 1 -- 172.21.15.88:0/2295552707 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f90dc007450 con 0x7f90e0069e90 2023-10-16T20:08:05.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.452+0000 7f90d67fc640 1 --2- 172.21.15.88:0/2295552707 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f90c40664a0 0x7f90c4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:05.467 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.452+0000 7f90d67fc640 1 -- 172.21.15.88:0/2295552707 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f90dc083220 con 0x7f90e0069e90 2023-10-16T20:08:05.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.452+0000 7f90e52d9640 1 --2- 172.21.15.88:0/2295552707 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f90c40664a0 0x7f90c4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:05.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.452+0000 7f90e52d9640 1 --2- 172.21.15.88:0/2295552707 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f90c40664a0 0x7f90c4068960 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f90d00af000 tx=0x7f90d0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:05.468 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.456+0000 7f90d67fc640 1 -- 172.21.15.88:0/2295552707 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f90dc050900 con 0x7f90e0069e90 2023-10-16T20:08:05.611 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.604+0000 7f90e7d3e640 1 -- 172.21.15.88:0/2295552707 --> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f90e00b0370 con 0x7f90c40664a0 2023-10-16T20:08:05.612 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.604+0000 7f90d67fc640 1 -- 172.21.15.88:0/2295552707 <== mgr.4110 v2:172.21.15.88:6832/4078483333 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f90d00bc090 con 0x7f90c40664a0 2023-10-16T20:08:05.612 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 1 to deep-scrub 2023-10-16T20:08:05.613 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.608+0000 7f90e7d3e640 1 -- 172.21.15.88:0/2295552707 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f90c40664a0 msgr2=0x7f90c4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:05.614 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.608+0000 7f90e7d3e640 1 --2- 172.21.15.88:0/2295552707 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f90c40664a0 0x7f90c4068960 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f90d00af000 tx=0x7f90d0008040 comp rx=0 tx=0).stop 2023-10-16T20:08:05.614 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.608+0000 7f90e7d3e640 1 -- 172.21.15.88:0/2295552707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90e0069e90 msgr2=0x7f90e015c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:05.614 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.608+0000 7f90e7d3e640 1 --2- 172.21.15.88:0/2295552707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90e0069e90 0x7f90e015c960 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f90dc00c500 tx=0x7f90dc00c9c0 comp rx=0 tx=0).stop 2023-10-16T20:08:05.615 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.608+0000 7f90e7d3e640 1 -- 172.21.15.88:0/2295552707 shutdown_connections 2023-10-16T20:08:05.615 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.608+0000 7f90e7d3e640 1 --2- 172.21.15.88:0/2295552707 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f90c40664a0 0x7f90c4068960 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:05.615 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.608+0000 7f90e7d3e640 1 --2- 172.21.15.88:0/2295552707 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f90e015d480 0x7f90e00b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:05.616 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.608+0000 7f90e7d3e640 1 --2- 172.21.15.88:0/2295552707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90e00adf80 0x7f90e0148ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:05.616 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.608+0000 7f90e7d3e640 1 --2- 172.21.15.88:0/2295552707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90e0069e90 0x7f90e015c960 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:05.616 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.608+0000 7f90e7d3e640 1 -- 172.21.15.88:0/2295552707 >> 172.21.15.88:0/2295552707 conn(0x7f90e00ce5e0 msgr2=0x7f90e00ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:05.616 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.608+0000 7f90e7d3e640 1 -- 172.21.15.88:0/2295552707 shutdown_connections 2023-10-16T20:08:05.617 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.608+0000 7f90e7d3e640 1 -- 172.21.15.88:0/2295552707 wait complete. 2023-10-16T20:08:05.633 INFO:tasks.ceph:Scrubbing osd.2 2023-10-16T20:08:05.633 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-16T20:08:05.774 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.768+0000 7efc68dd9640 1 Processor -- start 2023-10-16T20:08:05.775 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.768+0000 7efc68dd9640 1 -- start start 2023-10-16T20:08:05.776 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.768+0000 7efc68dd9640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7efc6415cb20 0x7efc640cca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:05.777 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.772+0000 7efc68dd9640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7efc6406e720 0x7efc6406d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:05.777 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.772+0000 7efc68dd9640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc64069f30 con 0x7efc640abd60 2023-10-16T20:08:05.777 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.772+0000 7efc68dd9640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc6406dc50 con 0x7efc6406e720 2023-10-16T20:08:05.777 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.772+0000 7efc68dd9640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc640cd160 con 0x7efc6415cb20 2023-10-16T20:08:05.778 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.772+0000 7efc62d9d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7efc6406e720 0x7efc6406d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:05.778 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.772+0000 7efc62d9d640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7efc6406e720 0x7efc6406d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:59990/0 (socket says 172.21.15.88:59990) 2023-10-16T20:08:05.778 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.772+0000 7efc62d9d640 1 -- 172.21.15.88:0/185800102 learned_addr learned my addr 172.21.15.88:0/185800102 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:05.779 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.772+0000 7efc6259c640 1 --2- 172.21.15.88:0/185800102 >> v2:172.21.15.94:3300/0 conn(0x7efc6415cb20 0x7efc640cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:05.779 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.772+0000 7efc62d9d640 1 -- 172.21.15.88:0/185800102 >> v2:172.21.15.94:3300/0 conn(0x7efc6415cb20 msgr2=0x7efc640cca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:08:05.779 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.772+0000 7efc62d9d640 1 --2- 172.21.15.88:0/185800102 >> v2:172.21.15.94:3300/0 conn(0x7efc6415cb20 0x7efc640cca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:05.779 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.772+0000 7efc62d9d640 1 -- 172.21.15.88:0/185800102 >> v1:172.21.15.94:6789/0 conn(0x7efc640abd60 legacy=0x7efc6406e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:08:05.780 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.772+0000 7efc62d9d640 1 -- 172.21.15.88:0/185800102 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc6406a1d0 con 0x7efc6406e720 2023-10-16T20:08:05.781 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.772+0000 7efc62d9d640 1 --2- 172.21.15.88:0/185800102 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc6406e720 0x7efc6406d710 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7efc6406c320 tx=0x7efc5c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d639526341d997e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:05.781 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.776+0000 7efc61d9b640 1 -- 172.21.15.88:0/185800102 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5c0a7070 con 0x7efc6406e720 2023-10-16T20:08:05.781 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.776+0000 7efc61d9b640 1 -- 172.21.15.88:0/185800102 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc5c0ab430 con 0x7efc6406e720 2023-10-16T20:08:05.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.776+0000 7efc61d9b640 1 -- 172.21.15.88:0/185800102 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5c0ab7c0 con 0x7efc6406e720 2023-10-16T20:08:05.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.776+0000 7efc68dd9640 1 -- 172.21.15.88:0/185800102 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc6406e720 msgr2=0x7efc6406d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:05.782 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.776+0000 7efc68dd9640 1 --2- 172.21.15.88:0/185800102 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc6406e720 0x7efc6406d710 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7efc6406c320 tx=0x7efc5c01c500 comp rx=0 tx=0).stop 2023-10-16T20:08:05.783 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.776+0000 7efc68dd9640 1 -- 172.21.15.88:0/185800102 shutdown_connections 2023-10-16T20:08:05.783 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.776+0000 7efc68dd9640 1 --2- 172.21.15.88:0/185800102 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc6406e720 0x7efc6406d710 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:05.783 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.776+0000 7efc68dd9640 1 --2- 172.21.15.88:0/185800102 >> v2:172.21.15.94:3300/0 conn(0x7efc6415cb20 0x7efc640cca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:05.783 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.776+0000 7efc68dd9640 1 -- 172.21.15.88:0/185800102 >> 172.21.15.88:0/185800102 conn(0x7efc640b0980 msgr2=0x7efc640b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:05.784 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.776+0000 7efc68dd9640 1 -- 172.21.15.88:0/185800102 shutdown_connections 2023-10-16T20:08:05.784 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.776+0000 7efc68dd9640 1 -- 172.21.15.88:0/185800102 wait complete. 2023-10-16T20:08:05.786 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.776+0000 7efc68dd9640 1 Processor -- start 2023-10-16T20:08:05.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.776+0000 7efc68dd9640 1 -- start start 2023-10-16T20:08:05.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.776+0000 7efc68dd9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc6406e720 0x7efc64155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:05.787 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.776+0000 7efc68dd9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc640abd60 0x7efc640ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:05.788 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.776+0000 7efc68dd9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efc6415cb20 0x7efc640aee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:05.788 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.776+0000 7efc68dd9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6415fd50 con 0x7efc640abd60 2023-10-16T20:08:05.788 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.776+0000 7efc68dd9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc64156480 con 0x7efc6406e720 2023-10-16T20:08:05.789 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.776+0000 7efc68dd9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc6415f4b0 con 0x7efc6415cb20 2023-10-16T20:08:05.789 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.776+0000 7efc59d9b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc640abd60 0x7efc640ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:05.789 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.776+0000 7efc59d9b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc640abd60 0x7efc640ae8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60000/0 (socket says 172.21.15.88:60000) 2023-10-16T20:08:05.790 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.776+0000 7efc59d9b640 1 -- 172.21.15.88:0/3010660644 learned_addr learned my addr 172.21.15.88:0/3010660644 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:05.790 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.780+0000 7efc6259c640 1 --2- 172.21.15.88:0/3010660644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc6406e720 0x7efc64155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:05.790 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.780+0000 7efc62d9d640 1 --2- 172.21.15.88:0/3010660644 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efc6415cb20 0x7efc640aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:05.790 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.780+0000 7efc59d9b640 1 -- 172.21.15.88:0/3010660644 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efc6415cb20 msgr2=0x7efc640aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:05.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.780+0000 7efc59d9b640 1 --2- 172.21.15.88:0/3010660644 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efc6415cb20 0x7efc640aee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:05.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.780+0000 7efc59d9b640 1 -- 172.21.15.88:0/3010660644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc6406e720 msgr2=0x7efc64155f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:05.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.780+0000 7efc59d9b640 1 --2- 172.21.15.88:0/3010660644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc6406e720 0x7efc64155f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:05.791 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.780+0000 7efc59d9b640 1 -- 172.21.15.88:0/3010660644 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5c0094d0 con 0x7efc640abd60 2023-10-16T20:08:05.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.780+0000 7efc59d9b640 1 --2- 172.21.15.88:0/3010660644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc640abd60 0x7efc640ae8c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7efc4c00c530 tx=0x7efc4c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:05.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.780+0000 7efc5b7fe640 1 -- 172.21.15.88:0/3010660644 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c005220 con 0x7efc640abd60 2023-10-16T20:08:05.792 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.780+0000 7efc5b7fe640 1 -- 172.21.15.88:0/3010660644 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4c005380 con 0x7efc640abd60 2023-10-16T20:08:05.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.780+0000 7efc68dd9640 1 -- 172.21.15.88:0/3010660644 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc6415f790 con 0x7efc640abd60 2023-10-16T20:08:05.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.780+0000 7efc5b7fe640 1 -- 172.21.15.88:0/3010660644 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c007270 con 0x7efc640abd60 2023-10-16T20:08:05.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.780+0000 7efc68dd9640 1 -- 172.21.15.88:0/3010660644 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc64156810 con 0x7efc640abd60 2023-10-16T20:08:05.793 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.780+0000 7efc68dd9640 1 -- 172.21.15.88:0/3010660644 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efc6406b1e0 con 0x7efc640abd60 2023-10-16T20:08:05.794 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.780+0000 7efc5b7fe640 1 -- 172.21.15.88:0/3010660644 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7efc4c0073d0 con 0x7efc640abd60 2023-10-16T20:08:05.795 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.780+0000 7efc5b7fe640 1 --2- 172.21.15.88:0/3010660644 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7efc440664a0 0x7efc44068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:05.795 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.780+0000 7efc5b7fe640 1 -- 172.21.15.88:0/3010660644 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5263+0+0 (secure 0 0 0) 0x7efc4c083510 con 0x7efc640abd60 2023-10-16T20:08:05.796 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.780+0000 7efc6259c640 1 --2- 172.21.15.88:0/3010660644 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7efc440664a0 0x7efc44068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:05.796 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.780+0000 7efc5b7fe640 1 --2- 172.21.15.88:0/3010660644 >> [v2:172.21.15.88:6816/3866598902,v1:172.21.15.88:6817/3866598902] conn(0x7efc4406c7a0 0x7efc4406ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:05.796 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.780+0000 7efc5b7fe640 1 -- 172.21.15.88:0/3010660644 --> [v2:172.21.15.88:6816/3866598902,v1:172.21.15.88:6817/3866598902] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efc4406f2b0 con 0x7efc4406c7a0 2023-10-16T20:08:05.797 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.780+0000 7efc5b7fe640 1 -- 172.21.15.88:0/3010660644 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc4c085af0 con 0x7efc640abd60 2023-10-16T20:08:05.797 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.784+0000 7efc62d9d640 1 --2- 172.21.15.88:0/3010660644 >> [v2:172.21.15.88:6816/3866598902,v1:172.21.15.88:6817/3866598902] conn(0x7efc4406c7a0 0x7efc4406ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:05.797 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.784+0000 7efc6259c640 1 --2- 172.21.15.88:0/3010660644 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7efc440664a0 0x7efc44068960 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7efc54003ab0 tx=0x7efc54009830 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:05.797 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.784+0000 7efc62d9d640 1 --2- 172.21.15.88:0/3010660644 >> [v2:172.21.15.88:6816/3866598902,v1:172.21.15.88:6817/3866598902] conn(0x7efc4406c7a0 0x7efc4406ebe0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:05.798 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.784+0000 7efc5b7fe640 1 -- 172.21.15.88:0/3010660644 <== osd.2 v2:172.21.15.88:6816/3866598902 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7efc4406f2b0 con 0x7efc4406c7a0 2023-10-16T20:08:05.801 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.796+0000 7efc68dd9640 1 -- 172.21.15.88:0/3010660644 --> [v2:172.21.15.88:6816/3866598902,v1:172.21.15.88:6817/3866598902] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7efc64156670 con 0x7efc4406c7a0 2023-10-16T20:08:05.802 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.796+0000 7efc5b7fe640 1 -- 172.21.15.88:0/3010660644 <== osd.2 v2:172.21.15.88:6816/3866598902 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7efc64156670 con 0x7efc4406c7a0 2023-10-16T20:08:05.802 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2023-10-16T20:08:05.803 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T20:08:05.803 INFO:teuthology.orchestra.run.smithi088.stdout:} 2023-10-16T20:08:05.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.796+0000 7efc68dd9640 1 -- 172.21.15.88:0/3010660644 >> [v2:172.21.15.88:6816/3866598902,v1:172.21.15.88:6817/3866598902] conn(0x7efc4406c7a0 msgr2=0x7efc4406ebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:05.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.796+0000 7efc68dd9640 1 --2- 172.21.15.88:0/3010660644 >> [v2:172.21.15.88:6816/3866598902,v1:172.21.15.88:6817/3866598902] conn(0x7efc4406c7a0 0x7efc4406ebe0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:05.804 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.796+0000 7efc68dd9640 1 -- 172.21.15.88:0/3010660644 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7efc440664a0 msgr2=0x7efc44068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:05.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.796+0000 7efc68dd9640 1 --2- 172.21.15.88:0/3010660644 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7efc440664a0 0x7efc44068960 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7efc54003ab0 tx=0x7efc54009830 comp rx=0 tx=0).stop 2023-10-16T20:08:05.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.796+0000 7efc68dd9640 1 -- 172.21.15.88:0/3010660644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc640abd60 msgr2=0x7efc640ae8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:05.805 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.796+0000 7efc68dd9640 1 --2- 172.21.15.88:0/3010660644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc640abd60 0x7efc640ae8c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7efc4c00c530 tx=0x7efc4c00c9f0 comp rx=0 tx=0).stop 2023-10-16T20:08:05.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.796+0000 7efc62d9d640 1 -- 172.21.15.88:0/3010660644 reap_dead start 2023-10-16T20:08:05.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.796+0000 7efc68dd9640 1 -- 172.21.15.88:0/3010660644 shutdown_connections 2023-10-16T20:08:05.806 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.796+0000 7efc68dd9640 1 -- 172.21.15.88:0/3010660644 >> 172.21.15.88:0/3010660644 conn(0x7efc640b0980 msgr2=0x7efc640afc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:05.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.796+0000 7efc68dd9640 1 -- 172.21.15.88:0/3010660644 shutdown_connections 2023-10-16T20:08:05.807 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.796+0000 7efc68dd9640 1 -- 172.21.15.88:0/3010660644 wait complete. 2023-10-16T20:08:05.820 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-16T20:08:05.960 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.952+0000 7fecd2205640 1 Processor -- start 2023-10-16T20:08:05.961 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.952+0000 7fecd2205640 1 -- start start 2023-10-16T20:08:05.963 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.956+0000 7fecd2205640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7feccc0b2a80 0x7feccc0ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:05.963 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.956+0000 7fecd2205640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7feccc0cea00 0x7feccc06bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:05.963 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.956+0000 7fecd2205640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feccc06b490 con 0x7feccc15cb80 2023-10-16T20:08:05.964 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.956+0000 7fecd2205640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feccc0cf180 con 0x7feccc0b2a80 2023-10-16T20:08:05.964 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.956+0000 7fecd2205640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feccc06b700 con 0x7feccc0cea00 2023-10-16T20:08:05.964 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.956+0000 7feccaffd640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7feccc0b2a80 0x7feccc0ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:05.965 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.956+0000 7feccaffd640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7feccc0b2a80 0x7feccc0ce4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60004/0 (socket says 172.21.15.88:60004) 2023-10-16T20:08:05.965 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.956+0000 7feccaffd640 1 -- 172.21.15.88:0/2810771530 learned_addr learned my addr 172.21.15.88:0/2810771530 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:05.965 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.956+0000 7feccbfff640 1 --2- 172.21.15.88:0/2810771530 >> v2:172.21.15.94:3300/0 conn(0x7feccc0cea00 0x7feccc06bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:05.965 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.956+0000 7feccaffd640 1 -- 172.21.15.88:0/2810771530 >> v2:172.21.15.94:3300/0 conn(0x7feccc0cea00 msgr2=0x7feccc06bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:08:05.966 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.956+0000 7feccaffd640 1 --2- 172.21.15.88:0/2810771530 >> v2:172.21.15.94:3300/0 conn(0x7feccc0cea00 0x7feccc06bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:05.966 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.956+0000 7feccaffd640 1 -- 172.21.15.88:0/2810771530 >> v1:172.21.15.94:6789/0 conn(0x7feccc15cb80 legacy=0x7feccc0b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:05.966 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.956+0000 7feccaffd640 1 -- 172.21.15.88:0/2810771530 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feccc06b9e0 con 0x7feccc0b2a80 2023-10-16T20:08:05.969 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.960+0000 7feccaffd640 1 --2- 172.21.15.88:0/2810771530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feccc0b2a80 0x7feccc0ce4c0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7feccc0c43f0 tx=0x7fecc001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a383074ed09ffd1a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:05.969 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.960+0000 7fecca7fc640 1 -- 172.21.15.88:0/2810771530 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc00a7070 con 0x7feccc0b2a80 2023-10-16T20:08:05.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.960+0000 7fecca7fc640 1 -- 172.21.15.88:0/2810771530 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc00ab430 con 0x7feccc0b2a80 2023-10-16T20:08:05.970 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.960+0000 7fecca7fc640 1 -- 172.21.15.88:0/2810771530 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc00ab7c0 con 0x7feccc0b2a80 2023-10-16T20:08:05.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.960+0000 7fecd2205640 1 -- 172.21.15.88:0/2810771530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feccc0b2a80 msgr2=0x7feccc0ce4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:05.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.960+0000 7fecd2205640 1 --2- 172.21.15.88:0/2810771530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feccc0b2a80 0x7feccc0ce4c0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7feccc0c43f0 tx=0x7fecc001c500 comp rx=0 tx=0).stop 2023-10-16T20:08:05.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.960+0000 7fecd2205640 1 -- 172.21.15.88:0/2810771530 shutdown_connections 2023-10-16T20:08:05.971 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.960+0000 7fecd2205640 1 --2- 172.21.15.88:0/2810771530 >> v2:172.21.15.94:3300/0 conn(0x7feccc0cea00 0x7feccc06bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:05.972 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.960+0000 7fecd2205640 1 --2- 172.21.15.88:0/2810771530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feccc0b2a80 0x7feccc0ce4c0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:05.972 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.960+0000 7fecd2205640 1 -- 172.21.15.88:0/2810771530 >> 172.21.15.88:0/2810771530 conn(0x7feccc0cd370 msgr2=0x7feccc0cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:05.972 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.960+0000 7fecd2205640 1 -- 172.21.15.88:0/2810771530 shutdown_connections 2023-10-16T20:08:05.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.960+0000 7fecd2205640 1 -- 172.21.15.88:0/2810771530 wait complete. 2023-10-16T20:08:05.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.960+0000 7fecd2205640 1 Processor -- start 2023-10-16T20:08:05.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.960+0000 7fecd2205640 1 -- start start 2023-10-16T20:08:05.973 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.964+0000 7fecd2205640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feccc0b2a80 0x7feccc155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:05.974 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.964+0000 7fecd2205640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feccc0cea00 0x7feccc15e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:05.974 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.964+0000 7fecd2205640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feccc15cb80 0x7feccc0c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:05.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.964+0000 7fecd2205640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feccc1560e0 con 0x7feccc0b2a80 2023-10-16T20:08:05.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.964+0000 7fecd2205640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feccc0b3830 con 0x7feccc15cb80 2023-10-16T20:08:05.975 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.964+0000 7fecd2205640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feccc15d300 con 0x7feccc0cea00 2023-10-16T20:08:05.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.964+0000 7feccb7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feccc0b2a80 0x7feccc155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:05.976 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.964+0000 7feccb7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feccc0b2a80 0x7feccc155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60016/0 (socket says 172.21.15.88:60016) 2023-10-16T20:08:05.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.964+0000 7feccb7fe640 1 -- 172.21.15.88:0/2214197072 learned_addr learned my addr 172.21.15.88:0/2214197072 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:05.977 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.964+0000 7feccbfff640 1 --2- 172.21.15.88:0/2214197072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feccc15cb80 0x7feccc0c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:05.978 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.964+0000 7feccaffd640 1 --2- 172.21.15.88:0/2214197072 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feccc0cea00 0x7feccc15e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:05.978 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.964+0000 7feccb7fe640 1 -- 172.21.15.88:0/2214197072 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feccc0cea00 msgr2=0x7feccc15e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:05.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.964+0000 7feccb7fe640 1 --2- 172.21.15.88:0/2214197072 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feccc0cea00 0x7feccc15e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:05.979 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.964+0000 7feccb7fe640 1 -- 172.21.15.88:0/2214197072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feccc15cb80 msgr2=0x7feccc0c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:05.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.964+0000 7feccb7fe640 1 --2- 172.21.15.88:0/2214197072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feccc15cb80 0x7feccc0c79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:05.980 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.964+0000 7feccb7fe640 1 -- 172.21.15.88:0/2214197072 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc00094d0 con 0x7feccc0b2a80 2023-10-16T20:08:05.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.964+0000 7feccb7fe640 1 --2- 172.21.15.88:0/2214197072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feccc0b2a80 0x7feccc155ba0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00c370 tx=0x7fecbc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:05.981 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.964+0000 7fecb3fff640 1 -- 172.21.15.88:0/2214197072 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecbc0052d0 con 0x7feccc0b2a80 2023-10-16T20:08:05.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.964+0000 7fecb3fff640 1 -- 172.21.15.88:0/2214197072 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecbc005430 con 0x7feccc0b2a80 2023-10-16T20:08:05.982 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.964+0000 7fecd2205640 1 -- 172.21.15.88:0/2214197072 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feccc15d5e0 con 0x7feccc0b2a80 2023-10-16T20:08:05.983 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.964+0000 7fecb3fff640 1 -- 172.21.15.88:0/2214197072 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecbc007260 con 0x7feccc0b2a80 2023-10-16T20:08:05.984 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.964+0000 7fecd2205640 1 -- 172.21.15.88:0/2214197072 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feccc15dba0 con 0x7feccc0b2a80 2023-10-16T20:08:05.985 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.964+0000 7fecd2205640 1 -- 172.21.15.88:0/2214197072 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feccc0670c0 con 0x7feccc0b2a80 2023-10-16T20:08:05.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.964+0000 7fecb3fff640 1 -- 172.21.15.88:0/2214197072 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fecbc0073c0 con 0x7feccc0b2a80 2023-10-16T20:08:05.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.968+0000 7fecb3fff640 1 --2- 172.21.15.88:0/2214197072 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fecb80664a0 0x7fecb8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:05.986 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.968+0000 7fecb3fff640 1 -- 172.21.15.88:0/2214197072 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fecbc082fa0 con 0x7feccc0b2a80 2023-10-16T20:08:05.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.968+0000 7feccaffd640 1 --2- 172.21.15.88:0/2214197072 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fecb80664a0 0x7fecb8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:05.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.968+0000 7feccaffd640 1 --2- 172.21.15.88:0/2214197072 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fecb80664a0 0x7fecb8068960 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fecc00af000 tx=0x7fecc00b6040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:05.987 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:05.972+0000 7fecb3fff640 1 -- 172.21.15.88:0/2214197072 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fecbc050680 con 0x7feccc0b2a80 2023-10-16T20:08:06.125 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.120+0000 7fecd2205640 1 -- 172.21.15.88:0/2214197072 --> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7feccc04e9c0 con 0x7fecb80664a0 2023-10-16T20:08:06.125 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.120+0000 7fecb3fff640 1 -- 172.21.15.88:0/2214197072 <== mgr.4110 v2:172.21.15.88:6832/4078483333 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7feccc04e9c0 con 0x7fecb80664a0 2023-10-16T20:08:06.126 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 2 to deep-scrub 2023-10-16T20:08:06.127 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.120+0000 7fecd2205640 1 -- 172.21.15.88:0/2214197072 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fecb80664a0 msgr2=0x7fecb8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:06.127 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.120+0000 7fecd2205640 1 --2- 172.21.15.88:0/2214197072 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fecb80664a0 0x7fecb8068960 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fecc00af000 tx=0x7fecc00b6040 comp rx=0 tx=0).stop 2023-10-16T20:08:06.128 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.120+0000 7fecd2205640 1 -- 172.21.15.88:0/2214197072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feccc0b2a80 msgr2=0x7feccc155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:06.128 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.120+0000 7fecd2205640 1 --2- 172.21.15.88:0/2214197072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feccc0b2a80 0x7feccc155ba0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00c370 tx=0x7fecbc00c830 comp rx=0 tx=0).stop 2023-10-16T20:08:06.128 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.120+0000 7fecd2205640 1 -- 172.21.15.88:0/2214197072 shutdown_connections 2023-10-16T20:08:06.128 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.120+0000 7fecd2205640 1 --2- 172.21.15.88:0/2214197072 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fecb80664a0 0x7fecb8068960 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:06.129 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.120+0000 7fecd2205640 1 --2- 172.21.15.88:0/2214197072 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feccc0cea00 0x7feccc15e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:06.129 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.120+0000 7fecd2205640 1 --2- 172.21.15.88:0/2214197072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feccc15cb80 0x7feccc0c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:06.129 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.120+0000 7fecd2205640 1 --2- 172.21.15.88:0/2214197072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feccc0b2a80 0x7feccc155ba0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:06.130 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.120+0000 7fecd2205640 1 -- 172.21.15.88:0/2214197072 >> 172.21.15.88:0/2214197072 conn(0x7feccc0cd370 msgr2=0x7feccc15f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:06.130 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.120+0000 7fecd2205640 1 -- 172.21.15.88:0/2214197072 shutdown_connections 2023-10-16T20:08:06.130 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.120+0000 7fecd2205640 1 -- 172.21.15.88:0/2214197072 wait complete. 2023-10-16T20:08:06.146 INFO:tasks.ceph:Scrubbing osd.3 2023-10-16T20:08:06.146 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-16T20:08:06.289 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.284+0000 7f9d59ff8640 1 Processor -- start 2023-10-16T20:08:06.290 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.284+0000 7f9d59ff8640 1 -- start start 2023-10-16T20:08:06.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.284+0000 7f9d59ff8640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d540cae90 0x7f9d540cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:06.292 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.284+0000 7f9d59ff8640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f9d540b5f00 0x7f9d540b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:06.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.284+0000 7f9d59ff8640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d540cbbf0 con 0x7f9d5415ab40 2023-10-16T20:08:06.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.284+0000 7f9d59ff8640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d540b68d0 con 0x7f9d540b5f00 2023-10-16T20:08:06.293 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.284+0000 7f9d59ff8640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d540cee30 con 0x7f9d540cae90 2023-10-16T20:08:06.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.284+0000 7f9d537fe640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f9d5415ab40 0x7f9d540b3720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:47252/0 (socket says 172.21.15.88:47252) 2023-10-16T20:08:06.294 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.284+0000 7f9d537fe640 1 -- 172.21.15.88:0/3586260048 learned_addr learned my addr 172.21.15.88:0/3586260048 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:08:06.295 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.284+0000 7f9d53fff640 1 --2- 172.21.15.88:0/3586260048 >> v2:172.21.15.88:3300/0 conn(0x7f9d540b5f00 0x7f9d540b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:06.295 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.284+0000 7f9d4bfff640 1 --2- 172.21.15.88:0/3586260048 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d540cae90 0x7f9d540cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:06.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.284+0000 7f9d52ffd640 1 -- 172.21.15.88:0/3586260048 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2898171827 0 0) 0x7f9d540cbbf0 con 0x7f9d5415ab40 2023-10-16T20:08:06.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.284+0000 7f9d52ffd640 1 -- 172.21.15.88:0/3586260048 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d40003530 con 0x7f9d5415ab40 2023-10-16T20:08:06.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.284+0000 7f9d53fff640 1 -- 172.21.15.88:0/3586260048 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d540cae90 msgr2=0x7f9d540cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:08:06.296 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.284+0000 7f9d53fff640 1 --2- 172.21.15.88:0/3586260048 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d540cae90 0x7f9d540cb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:06.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.284+0000 7f9d53fff640 1 -- 172.21.15.88:0/3586260048 >> v1:172.21.15.88:6789/0 conn(0x7f9d5415ab40 legacy=0x7f9d540b3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:06.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.284+0000 7f9d53fff640 1 -- 172.21.15.88:0/3586260048 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d540cb910 con 0x7f9d540b5f00 2023-10-16T20:08:06.297 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.284+0000 7f9d4bfff640 1 --2- 172.21.15.88:0/3586260048 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d540cae90 0x7f9d540cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:08:06.300 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.288+0000 7f9d53fff640 1 --2- 172.21.15.88:0/3586260048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d540b5f00 0x7f9d540b6390 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c009d90 tx=0x7f9d4c01c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6875c5ecf65530e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:06.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.288+0000 7f9d52ffd640 1 -- 172.21.15.88:0/3586260048 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c0a5870 con 0x7f9d540b5f00 2023-10-16T20:08:06.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.288+0000 7f9d52ffd640 1 -- 172.21.15.88:0/3586260048 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4c0a59d0 con 0x7f9d540b5f00 2023-10-16T20:08:06.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.288+0000 7f9d52ffd640 1 -- 172.21.15.88:0/3586260048 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c0a5d60 con 0x7f9d540b5f00 2023-10-16T20:08:06.301 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.288+0000 7f9d59ff8640 1 -- 172.21.15.88:0/3586260048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d540b5f00 msgr2=0x7f9d540b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:06.302 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.288+0000 7f9d59ff8640 1 --2- 172.21.15.88:0/3586260048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d540b5f00 0x7f9d540b6390 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c009d90 tx=0x7f9d4c01c8c0 comp rx=0 tx=0).stop 2023-10-16T20:08:06.302 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.288+0000 7f9d59ff8640 1 -- 172.21.15.88:0/3586260048 shutdown_connections 2023-10-16T20:08:06.302 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.288+0000 7f9d59ff8640 1 --2- 172.21.15.88:0/3586260048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d540b5f00 0x7f9d540b6390 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:06.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.288+0000 7f9d59ff8640 1 --2- 172.21.15.88:0/3586260048 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d540cae90 0x7f9d540cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:06.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.288+0000 7f9d59ff8640 1 -- 172.21.15.88:0/3586260048 >> 172.21.15.88:0/3586260048 conn(0x7f9d5406a060 msgr2=0x7f9d5406a440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:06.303 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.288+0000 7f9d59ff8640 1 -- 172.21.15.88:0/3586260048 shutdown_connections 2023-10-16T20:08:06.304 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.288+0000 7f9d59ff8640 1 -- 172.21.15.88:0/3586260048 wait complete. 2023-10-16T20:08:06.304 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.292+0000 7f9d59ff8640 1 Processor -- start 2023-10-16T20:08:06.304 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.292+0000 7f9d59ff8640 1 -- start start 2023-10-16T20:08:06.304 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.292+0000 7f9d59ff8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d540b5f00 0x7f9d541f7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:06.305 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.292+0000 7f9d59ff8640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d540cae90 0x7f9d54155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:06.305 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.292+0000 7f9d59ff8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d5415ab40 0x7f9d54156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:06.305 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.292+0000 7f9d59ff8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d540b4c00 con 0x7f9d540b5f00 2023-10-16T20:08:06.306 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.292+0000 7f9d59ff8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d541545e0 con 0x7f9d5415ab40 2023-10-16T20:08:06.306 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.292+0000 7f9d59ff8640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d541548c0 con 0x7f9d540cae90 2023-10-16T20:08:06.306 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.292+0000 7f9d537fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d540b5f00 0x7f9d541f7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:06.307 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.292+0000 7f9d537fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d540b5f00 0x7f9d541f7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60038/0 (socket says 172.21.15.88:60038) 2023-10-16T20:08:06.307 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.292+0000 7f9d537fe640 1 -- 172.21.15.88:0/3768786717 learned_addr learned my addr 172.21.15.88:0/3768786717 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:06.307 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.292+0000 7f9d4bfff640 1 --2- 172.21.15.88:0/3768786717 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d540cae90 0x7f9d54155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:06.308 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.292+0000 7f9d53fff640 1 --2- 172.21.15.88:0/3768786717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d5415ab40 0x7f9d54156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:06.308 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.292+0000 7f9d537fe640 1 -- 172.21.15.88:0/3768786717 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d540cae90 msgr2=0x7f9d54155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:06.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.292+0000 7f9d537fe640 1 --2- 172.21.15.88:0/3768786717 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d540cae90 0x7f9d54155ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:06.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.292+0000 7f9d537fe640 1 -- 172.21.15.88:0/3768786717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d5415ab40 msgr2=0x7f9d54156020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:06.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.292+0000 7f9d537fe640 1 --2- 172.21.15.88:0/3768786717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d5415ab40 0x7f9d54156020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:06.309 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.292+0000 7f9d537fe640 1 -- 172.21.15.88:0/3768786717 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4c009a70 con 0x7f9d540b5f00 2023-10-16T20:08:06.310 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.292+0000 7f9d537fe640 1 --2- 172.21.15.88:0/3768786717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d540b5f00 0x7f9d541f7a70 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400c4d0 tx=0x7f9d4400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:06.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.296+0000 7f9d50ff9640 1 -- 172.21.15.88:0/3768786717 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d44005cf0 con 0x7f9d540b5f00 2023-10-16T20:08:06.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.296+0000 7f9d50ff9640 1 -- 172.21.15.88:0/3768786717 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d44005e50 con 0x7f9d540b5f00 2023-10-16T20:08:06.312 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.296+0000 7f9d50ff9640 1 -- 172.21.15.88:0/3768786717 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d44007270 con 0x7f9d540b5f00 2023-10-16T20:08:06.313 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.296+0000 7f9d59ff8640 1 -- 172.21.15.88:0/3768786717 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d540b4890 con 0x7f9d540b5f00 2023-10-16T20:08:06.313 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.300+0000 7f9d59ff8640 1 -- 172.21.15.88:0/3768786717 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d540b4ee0 con 0x7f9d540b5f00 2023-10-16T20:08:06.313 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.300+0000 7f9d59ff8640 1 -- 172.21.15.88:0/3768786717 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9d540d0270 con 0x7f9d540b5f00 2023-10-16T20:08:06.314 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.300+0000 7f9d50ff9640 1 -- 172.21.15.88:0/3768786717 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9d440073d0 con 0x7f9d540b5f00 2023-10-16T20:08:06.314 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.300+0000 7f9d50ff9640 1 --2- 172.21.15.88:0/3768786717 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9d340664a0 0x7f9d34068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:06.314 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.300+0000 7f9d50ff9640 1 -- 172.21.15.88:0/3768786717 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f9d44083220 con 0x7f9d540b5f00 2023-10-16T20:08:06.315 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.300+0000 7f9d4bfff640 1 --2- 172.21.15.88:0/3768786717 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9d340664a0 0x7f9d34068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:06.315 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.300+0000 7f9d50ff9640 1 --2- 172.21.15.88:0/3768786717 >> [v2:172.21.15.88:6824/1777716082,v1:172.21.15.88:6825/1777716082] conn(0x7f9d3406c7a0 0x7f9d3406ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:06.315 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.300+0000 7f9d50ff9640 1 -- 172.21.15.88:0/3768786717 --> [v2:172.21.15.88:6824/1777716082,v1:172.21.15.88:6825/1777716082] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9d3406f2b0 con 0x7f9d3406c7a0 2023-10-16T20:08:06.315 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.300+0000 7f9d50ff9640 1 -- 172.21.15.88:0/3768786717 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d44085800 con 0x7f9d540b5f00 2023-10-16T20:08:06.316 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.300+0000 7f9d53fff640 1 --2- 172.21.15.88:0/3768786717 >> [v2:172.21.15.88:6824/1777716082,v1:172.21.15.88:6825/1777716082] conn(0x7f9d3406c7a0 0x7f9d3406ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:06.316 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.300+0000 7f9d4bfff640 1 --2- 172.21.15.88:0/3768786717 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9d340664a0 0x7f9d34068960 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9d541f93c0 tx=0x7f9d3c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:06.316 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.300+0000 7f9d53fff640 1 --2- 172.21.15.88:0/3768786717 >> [v2:172.21.15.88:6824/1777716082,v1:172.21.15.88:6825/1777716082] conn(0x7f9d3406c7a0 0x7f9d3406ebe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:06.316 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.304+0000 7f9d50ff9640 1 -- 172.21.15.88:0/3768786717 <== osd.3 v2:172.21.15.88:6824/1777716082 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f9d3406f2b0 con 0x7f9d3406c7a0 2023-10-16T20:08:06.321 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.316+0000 7f9d59ff8640 1 -- 172.21.15.88:0/3768786717 --> [v2:172.21.15.88:6824/1777716082,v1:172.21.15.88:6825/1777716082] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9d54152af0 con 0x7f9d3406c7a0 2023-10-16T20:08:06.321 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.316+0000 7f9d50ff9640 1 -- 172.21.15.88:0/3768786717 <== osd.3 v2:172.21.15.88:6824/1777716082 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f9d54152af0 con 0x7f9d3406c7a0 2023-10-16T20:08:06.322 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2023-10-16T20:08:06.322 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T20:08:06.322 INFO:teuthology.orchestra.run.smithi088.stdout:} 2023-10-16T20:08:06.323 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.316+0000 7f9d59ff8640 1 -- 172.21.15.88:0/3768786717 >> [v2:172.21.15.88:6824/1777716082,v1:172.21.15.88:6825/1777716082] conn(0x7f9d3406c7a0 msgr2=0x7f9d3406ebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:06.324 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.316+0000 7f9d59ff8640 1 --2- 172.21.15.88:0/3768786717 >> [v2:172.21.15.88:6824/1777716082,v1:172.21.15.88:6825/1777716082] conn(0x7f9d3406c7a0 0x7f9d3406ebe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:06.324 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.316+0000 7f9d59ff8640 1 -- 172.21.15.88:0/3768786717 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9d340664a0 msgr2=0x7f9d34068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:06.325 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.316+0000 7f9d59ff8640 1 --2- 172.21.15.88:0/3768786717 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f9d340664a0 0x7f9d34068960 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9d541f93c0 tx=0x7f9d3c008040 comp rx=0 tx=0).stop 2023-10-16T20:08:06.325 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.316+0000 7f9d59ff8640 1 -- 172.21.15.88:0/3768786717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d540b5f00 msgr2=0x7f9d541f7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:06.326 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.316+0000 7f9d59ff8640 1 --2- 172.21.15.88:0/3768786717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d540b5f00 0x7f9d541f7a70 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400c4d0 tx=0x7f9d4400c990 comp rx=0 tx=0).stop 2023-10-16T20:08:06.326 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.316+0000 7f9d53fff640 1 -- 172.21.15.88:0/3768786717 reap_dead start 2023-10-16T20:08:06.327 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.316+0000 7f9d59ff8640 1 -- 172.21.15.88:0/3768786717 shutdown_connections 2023-10-16T20:08:06.328 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.316+0000 7f9d59ff8640 1 -- 172.21.15.88:0/3768786717 >> 172.21.15.88:0/3768786717 conn(0x7f9d5406a060 msgr2=0x7f9d540b3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:06.328 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.316+0000 7f9d59ff8640 1 -- 172.21.15.88:0/3768786717 shutdown_connections 2023-10-16T20:08:06.329 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.316+0000 7f9d59ff8640 1 -- 172.21.15.88:0/3768786717 wait complete. 2023-10-16T20:08:06.339 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-16T20:08:06.478 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.472+0000 7f8ab4279640 1 Processor -- start 2023-10-16T20:08:06.479 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.472+0000 7f8ab4279640 1 -- start start 2023-10-16T20:08:06.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.472+0000 7f8ab4279640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8aac069e90 0x7f8aac06a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:06.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.472+0000 7f8ab4279640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f8aac0adf80 0x7f8aac15a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:06.481 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.472+0000 7f8ab4279640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8aac15d480 0x7f8aac0afa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:06.482 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.472+0000 7f8ab4279640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aac159e40 con 0x7f8aac0adf80 2023-10-16T20:08:06.482 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.472+0000 7f8ab4279640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aac15e400 con 0x7f8aac15d480 2023-10-16T20:08:06.482 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.472+0000 7f8ab4279640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac06a7b0 con 0x7f8aac069e90 2023-10-16T20:08:06.482 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.476+0000 7f8ab1814640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f8aac0adf80 0x7f8aac15a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:06.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.476+0000 7f8ab1814640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f8aac0adf80 0x7f8aac15a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60042/0 (socket says 172.21.15.88:60042) 2023-10-16T20:08:06.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.476+0000 7f8ab1814640 1 -- 172.21.15.88:0/3165573711 learned_addr learned my addr 172.21.15.88:0/3165573711 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:06.483 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.476+0000 7f8ab2816640 1 --2- 172.21.15.88:0/3165573711 >> v2:172.21.15.94:3300/0 conn(0x7f8aac15d480 0x7f8aac0afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:06.484 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.476+0000 7f8ab2015640 1 --2- 172.21.15.88:0/3165573711 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8aac069e90 0x7f8aac06a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:06.484 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.476+0000 7f8ab1814640 1 -- 172.21.15.88:0/3165573711 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8aac069e90 msgr2=0x7f8aac06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:08:06.484 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.476+0000 7f8ab1814640 1 --2- 172.21.15.88:0/3165573711 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8aac069e90 0x7f8aac06a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:06.484 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.476+0000 7f8ab1814640 1 -- 172.21.15.88:0/3165573711 >> v2:172.21.15.94:3300/0 conn(0x7f8aac15d480 msgr2=0x7f8aac0afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:08:06.485 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.476+0000 7f8ab1814640 1 --2- 172.21.15.88:0/3165573711 >> v2:172.21.15.94:3300/0 conn(0x7f8aac15d480 0x7f8aac0afa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:06.485 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.476+0000 7f8ab1814640 1 -- 172.21.15.88:0/3165573711 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aac15de40 con 0x7f8aac0adf80 2023-10-16T20:08:06.485 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.476+0000 7f8ab2816640 1 --2- 172.21.15.88:0/3165573711 >> v2:172.21.15.94:3300/0 conn(0x7f8aac15d480 0x7f8aac0afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:08:06.488 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.476+0000 7f8ab1814640 1 --2- 172.21.15.88:0/3165573711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8aac0adf80 0x7f8aac15a830 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0725c0 tx=0x7f8a9c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a32d9c978276a0c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:06.488 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.476+0000 7f8ab1013640 1 -- 172.21.15.88:0/3165573711 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c0aa5f0 con 0x7f8aac0adf80 2023-10-16T20:08:06.488 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.476+0000 7f8ab1013640 1 -- 172.21.15.88:0/3165573711 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a9c0aa750 con 0x7f8aac0adf80 2023-10-16T20:08:06.489 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.476+0000 7f8ab1013640 1 -- 172.21.15.88:0/3165573711 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c0aaae0 con 0x7f8aac0adf80 2023-10-16T20:08:06.489 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.480+0000 7f8ab4279640 1 -- 172.21.15.88:0/3165573711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8aac0adf80 msgr2=0x7f8aac15a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:06.489 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.480+0000 7f8ab4279640 1 --2- 172.21.15.88:0/3165573711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8aac0adf80 0x7f8aac15a830 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0725c0 tx=0x7f8a9c01c4d0 comp rx=0 tx=0).stop 2023-10-16T20:08:06.490 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.480+0000 7f8ab4279640 1 -- 172.21.15.88:0/3165573711 shutdown_connections 2023-10-16T20:08:06.490 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.480+0000 7f8ab4279640 1 --2- 172.21.15.88:0/3165573711 >> v2:172.21.15.94:3300/0 conn(0x7f8aac15d480 0x7f8aac0afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:06.490 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.480+0000 7f8ab4279640 1 --2- 172.21.15.88:0/3165573711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8aac0adf80 0x7f8aac15a830 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:06.491 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.480+0000 7f8ab4279640 1 --2- 172.21.15.88:0/3165573711 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8aac069e90 0x7f8aac06a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:06.491 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.480+0000 7f8ab4279640 1 -- 172.21.15.88:0/3165573711 >> 172.21.15.88:0/3165573711 conn(0x7f8aac0ce5e0 msgr2=0x7f8aac0ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:06.491 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.480+0000 7f8ab4279640 1 -- 172.21.15.88:0/3165573711 shutdown_connections 2023-10-16T20:08:06.492 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.480+0000 7f8ab4279640 1 -- 172.21.15.88:0/3165573711 wait complete. 2023-10-16T20:08:06.492 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.480+0000 7f8ab4279640 1 Processor -- start 2023-10-16T20:08:06.492 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.480+0000 7f8ab4279640 1 -- start start 2023-10-16T20:08:06.492 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.480+0000 7f8ab4279640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8aac069e90 0x7f8aac149780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:06.493 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.480+0000 7f8ab4279640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8aac0adf80 0x7f8aac157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:06.493 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.480+0000 7f8ab4279640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aac15d480 0x7f8aac156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:06.493 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.480+0000 7f8ab4279640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac158dc0 con 0x7f8aac0adf80 2023-10-16T20:08:06.494 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.480+0000 7f8ab4279640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac158140 con 0x7f8aac15d480 2023-10-16T20:08:06.494 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.480+0000 7f8ab4279640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac158420 con 0x7f8aac069e90 2023-10-16T20:08:06.494 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.480+0000 7f8ab1814640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8aac0adf80 0x7f8aac157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:06.494 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.480+0000 7f8ab1814640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8aac0adf80 0x7f8aac157ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60056/0 (socket says 172.21.15.88:60056) 2023-10-16T20:08:06.495 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.480+0000 7f8ab1814640 1 -- 172.21.15.88:0/641346138 learned_addr learned my addr 172.21.15.88:0/641346138 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:06.495 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.480+0000 7f8ab2816640 1 --2- 172.21.15.88:0/641346138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aac15d480 0x7f8aac156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:06.495 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.480+0000 7f8ab2015640 1 --2- 172.21.15.88:0/641346138 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8aac069e90 0x7f8aac149780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:06.496 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.484+0000 7f8ab1814640 1 -- 172.21.15.88:0/641346138 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8aac069e90 msgr2=0x7f8aac149780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:06.496 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.484+0000 7f8ab1814640 1 --2- 172.21.15.88:0/641346138 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8aac069e90 0x7f8aac149780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:06.496 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.484+0000 7f8ab1814640 1 -- 172.21.15.88:0/641346138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aac15d480 msgr2=0x7f8aac156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:06.496 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.484+0000 7f8ab1814640 1 --2- 172.21.15.88:0/641346138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aac15d480 0x7f8aac156960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:06.497 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.484+0000 7f8ab1814640 1 -- 172.21.15.88:0/641346138 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9c0094d0 con 0x7f8aac0adf80 2023-10-16T20:08:06.498 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.484+0000 7f8ab1814640 1 --2- 172.21.15.88:0/641346138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8aac0adf80 0x7f8aac157ab0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c01ab80 tx=0x7f8a9c01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:06.498 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.484+0000 7f8aa2ffd640 1 -- 172.21.15.88:0/641346138 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c0b3800 con 0x7f8aac0adf80 2023-10-16T20:08:06.499 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.484+0000 7f8aa2ffd640 1 -- 172.21.15.88:0/641346138 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a9c0b3960 con 0x7f8aac0adf80 2023-10-16T20:08:06.499 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.484+0000 7f8aa2ffd640 1 -- 172.21.15.88:0/641346138 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c0b3d10 con 0x7f8aac0adf80 2023-10-16T20:08:06.499 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.484+0000 7f8ab4279640 1 -- 172.21.15.88:0/641346138 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aac158700 con 0x7f8aac0adf80 2023-10-16T20:08:06.500 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.484+0000 7f8ab4279640 1 -- 172.21.15.88:0/641346138 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aac156ea0 con 0x7f8aac0adf80 2023-10-16T20:08:06.500 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.484+0000 7f8ab4279640 1 -- 172.21.15.88:0/641346138 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aac06d180 con 0x7f8aac0adf80 2023-10-16T20:08:06.500 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.484+0000 7f8aa2ffd640 1 -- 172.21.15.88:0/641346138 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8a9c0c73a0 con 0x7f8aac0adf80 2023-10-16T20:08:06.500 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.488+0000 7f8aa2ffd640 1 --2- 172.21.15.88:0/641346138 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f8a900664a0 0x7f8a90068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:06.501 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.488+0000 7f8aa2ffd640 1 -- 172.21.15.88:0/641346138 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f8a9c0a7070 con 0x7f8aac0adf80 2023-10-16T20:08:06.501 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.488+0000 7f8ab2015640 1 --2- 172.21.15.88:0/641346138 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f8a900664a0 0x7f8a90068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:06.501 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.488+0000 7f8ab2015640 1 --2- 172.21.15.88:0/641346138 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f8a900664a0 0x7f8a90068960 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8aa80045f0 tx=0x7f8aa8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:06.502 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.492+0000 7f8aa2ffd640 1 -- 172.21.15.88:0/641346138 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8a9c0ebc80 con 0x7f8aac0adf80 2023-10-16T20:08:06.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.640+0000 7f8ab4279640 1 -- 172.21.15.88:0/641346138 --> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f8aac15a310 con 0x7f8a900664a0 2023-10-16T20:08:06.645 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.640+0000 7f8aa2ffd640 1 -- 172.21.15.88:0/641346138 <== mgr.4110 v2:172.21.15.88:6832/4078483333 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8aa800b6a0 con 0x7f8a900664a0 2023-10-16T20:08:06.645 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 3 to deep-scrub 2023-10-16T20:08:06.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.640+0000 7f8ab4279640 1 -- 172.21.15.88:0/641346138 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f8a900664a0 msgr2=0x7f8a90068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:06.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.640+0000 7f8ab4279640 1 --2- 172.21.15.88:0/641346138 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f8a900664a0 0x7f8a90068960 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8aa80045f0 tx=0x7f8aa8008040 comp rx=0 tx=0).stop 2023-10-16T20:08:06.647 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.640+0000 7f8ab4279640 1 -- 172.21.15.88:0/641346138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8aac0adf80 msgr2=0x7f8aac157ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:06.648 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.640+0000 7f8ab4279640 1 --2- 172.21.15.88:0/641346138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8aac0adf80 0x7f8aac157ab0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c01ab80 tx=0x7f8a9c01ad10 comp rx=0 tx=0).stop 2023-10-16T20:08:06.648 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.640+0000 7f8ab4279640 1 -- 172.21.15.88:0/641346138 shutdown_connections 2023-10-16T20:08:06.648 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.640+0000 7f8ab4279640 1 --2- 172.21.15.88:0/641346138 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f8a900664a0 0x7f8a90068960 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:06.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.640+0000 7f8ab4279640 1 --2- 172.21.15.88:0/641346138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aac15d480 0x7f8aac156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:06.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.640+0000 7f8ab4279640 1 --2- 172.21.15.88:0/641346138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8aac0adf80 0x7f8aac157ab0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:06.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.640+0000 7f8ab4279640 1 --2- 172.21.15.88:0/641346138 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8aac069e90 0x7f8aac149780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:06.649 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.640+0000 7f8ab4279640 1 -- 172.21.15.88:0/641346138 >> 172.21.15.88:0/641346138 conn(0x7f8aac0ce5e0 msgr2=0x7f8aac0728b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:06.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.640+0000 7f8ab4279640 1 -- 172.21.15.88:0/641346138 shutdown_connections 2023-10-16T20:08:06.650 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.640+0000 7f8ab4279640 1 -- 172.21.15.88:0/641346138 wait complete. 2023-10-16T20:08:06.666 INFO:tasks.ceph:Scrubbing osd.4 2023-10-16T20:08:06.669 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-16T20:08:06.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.804+0000 7ff9eaba7640 1 Processor -- start 2023-10-16T20:08:06.811 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.804+0000 7ff9eaba7640 1 -- start start 2023-10-16T20:08:06.812 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9eaba7640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff9e415cb50 0x7ff9e40cd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:06.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9eaba7640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff9e40cd9c0 0x7ff9e40b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:06.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9eaba7640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9e406b0a0 con 0x7ff9e415dde0 2023-10-16T20:08:06.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9eaba7640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9e40d0390 con 0x7ff9e415cb50 2023-10-16T20:08:06.813 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9eaba7640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e415cf30 con 0x7ff9e40cd9c0 2023-10-16T20:08:06.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9ea3a6640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7ff9e415dde0 0x7ff9e406c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:47254/0 (socket says 172.21.15.88:47254) 2023-10-16T20:08:06.814 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9ea3a6640 1 -- 172.21.15.88:0/3239654137 learned_addr learned my addr 172.21.15.88:0/3239654137 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:08:06.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9e9ba5640 1 --2- 172.21.15.88:0/3239654137 >> v2:172.21.15.94:3300/0 conn(0x7ff9e415cb50 0x7ff9e40cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:06.816 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9e93a4640 1 --2- 172.21.15.88:0/3239654137 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff9e40cd9c0 0x7ff9e40b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:06.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9e8ba3640 1 -- 172.21.15.88:0/3239654137 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2862344404 0 0) 0x7ff9e406b0a0 con 0x7ff9e415dde0 2023-10-16T20:08:06.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9e8ba3640 1 -- 172.21.15.88:0/3239654137 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9d0003530 con 0x7ff9e415dde0 2023-10-16T20:08:06.817 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9e8ba3640 1 -- 172.21.15.88:0/3239654137 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2798620927 0 0) 0x7ff9d0003530 con 0x7ff9e415dde0 2023-10-16T20:08:06.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9e8ba3640 1 -- 172.21.15.88:0/3239654137 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff9e40cd9c0 msgr2=0x7ff9e40b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:08:06.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9e8ba3640 1 --2- 172.21.15.88:0/3239654137 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff9e40cd9c0 0x7ff9e40b42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:06.818 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9e8ba3640 1 -- 172.21.15.88:0/3239654137 >> v2:172.21.15.94:3300/0 conn(0x7ff9e415cb50 msgr2=0x7ff9e40cd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:08:06.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9e8ba3640 1 --2- 172.21.15.88:0/3239654137 >> v2:172.21.15.94:3300/0 conn(0x7ff9e415cb50 0x7ff9e40cd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:06.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9e8ba3640 1 -- 172.21.15.88:0/3239654137 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e40cde80 con 0x7ff9e415dde0 2023-10-16T20:08:06.819 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9e8ba3640 1 -- 172.21.15.88:0/3239654137 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7ff9dc001a00 con 0x7ff9e415dde0 2023-10-16T20:08:06.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9e8ba3640 1 -- 172.21.15.88:0/3239654137 >> v1:172.21.15.88:6789/0 conn(0x7ff9e415dde0 legacy=0x7ff9e406c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:06.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9e8ba3640 1 --2- 172.21.15.88:0/3239654137 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff9e40cd9c0 0x7ff9d0003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:06.820 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9e8ba3640 1 --2- 172.21.15.88:0/3239654137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9d0003f00 0x7ff9d00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:06.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9e8ba3640 1 --2- 172.21.15.88:0/3239654137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d0004d10 0x7ff9d0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:06.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9e8ba3640 1 -- 172.21.15.88:0/3239654137 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e40d0390 con 0x7ff9d0003f00 2023-10-16T20:08:06.821 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9e8ba3640 1 -- 172.21.15.88:0/3239654137 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e415cf30 con 0x7ff9d0004d10 2023-10-16T20:08:06.822 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9e8ba3640 1 -- 172.21.15.88:0/3239654137 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d000b990 con 0x7ff9e40cd9c0 2023-10-16T20:08:06.822 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9e93a4640 1 --2- 172.21.15.88:0/3239654137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9d0003f00 0x7ff9d00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:06.822 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9e9ba5640 1 --2- 172.21.15.88:0/3239654137 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff9e40cd9c0 0x7ff9d0003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:06.823 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9ea3a6640 1 --2- 172.21.15.88:0/3239654137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d0004d10 0x7ff9d0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:06.823 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9e93a4640 1 -- 172.21.15.88:0/3239654137 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff9e40cd9c0 msgr2=0x7ff9d0003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:08:06.823 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9e93a4640 1 --2- 172.21.15.88:0/3239654137 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff9e40cd9c0 0x7ff9d0003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:06.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9e93a4640 1 -- 172.21.15.88:0/3239654137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d0004d10 msgr2=0x7ff9d0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:08:06.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9e93a4640 1 --2- 172.21.15.88:0/3239654137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d0004d10 0x7ff9d0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:06.824 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.808+0000 7ff9e93a4640 1 -- 172.21.15.88:0/3239654137 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d0003530 con 0x7ff9d0003f00 2023-10-16T20:08:06.829 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.812+0000 7ff9e93a4640 1 --2- 172.21.15.88:0/3239654137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9d0003f00 0x7ff9d00047d0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4009ae0 tx=0x7ff9d401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f6e1c25ae48253 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:06.830 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.812+0000 7ff9e8ba3640 1 -- 172.21.15.88:0/3239654137 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d40a5530 con 0x7ff9d0003f00 2023-10-16T20:08:06.830 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.812+0000 7ff9e8ba3640 1 -- 172.21.15.88:0/3239654137 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d40a5690 con 0x7ff9d0003f00 2023-10-16T20:08:06.830 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.812+0000 7ff9e8ba3640 1 -- 172.21.15.88:0/3239654137 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d40a5a40 con 0x7ff9d0003f00 2023-10-16T20:08:06.831 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.812+0000 7ff9eaba7640 1 -- 172.21.15.88:0/3239654137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9d0003f00 msgr2=0x7ff9d00047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:06.831 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.812+0000 7ff9eaba7640 1 --2- 172.21.15.88:0/3239654137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9d0003f00 0x7ff9d00047d0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4009ae0 tx=0x7ff9d401c9f0 comp rx=0 tx=0).stop 2023-10-16T20:08:06.832 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.812+0000 7ff9ea3a6640 1 -- 172.21.15.88:0/3239654137 reap_dead start 2023-10-16T20:08:06.832 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.812+0000 7ff9eaba7640 1 -- 172.21.15.88:0/3239654137 shutdown_connections 2023-10-16T20:08:06.832 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.812+0000 7ff9eaba7640 1 -- 172.21.15.88:0/3239654137 >> 172.21.15.88:0/3239654137 conn(0x7ff9e40d2850 msgr2=0x7ff9e40d2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:06.832 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.812+0000 7ff9eaba7640 1 -- 172.21.15.88:0/3239654137 shutdown_connections 2023-10-16T20:08:06.833 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.812+0000 7ff9eaba7640 1 -- 172.21.15.88:0/3239654137 wait complete. 2023-10-16T20:08:06.833 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.812+0000 7ff9eaba7640 1 Processor -- start 2023-10-16T20:08:06.833 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.816+0000 7ff9eaba7640 1 -- start start 2023-10-16T20:08:06.834 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.816+0000 7ff9eaba7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d0003f00 0x7ff9e4148ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:06.834 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.816+0000 7ff9eaba7640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff9e414c400 0x7ff9e40b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:06.834 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.816+0000 7ff9eaba7640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9e415a820 0x7ff9e415acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:06.834 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.816+0000 7ff9eaba7640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e415b210 con 0x7ff9e415a820 2023-10-16T20:08:06.835 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.816+0000 7ff9eaba7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e4149430 con 0x7ff9d0003f00 2023-10-16T20:08:06.835 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.816+0000 7ff9eaba7640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e40b7440 con 0x7ff9e414c400 2023-10-16T20:08:06.835 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.816+0000 7ff9e9ba5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d0003f00 0x7ff9e4148ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:06.835 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.816+0000 7ff9ea3a6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9e415a820 0x7ff9e415acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:06.836 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.816+0000 7ff9e9ba5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d0003f00 0x7ff9e4148ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:55362/0 (socket says 172.21.15.88:55362) 2023-10-16T20:08:06.836 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.816+0000 7ff9e9ba5640 1 -- 172.21.15.88:0/23674999 learned_addr learned my addr 172.21.15.88:0/23674999 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:06.836 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.816+0000 7ff9e93a4640 1 --2- 172.21.15.88:0/23674999 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff9e414c400 0x7ff9e40b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:06.837 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.816+0000 7ff9e9ba5640 1 -- 172.21.15.88:0/23674999 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff9e414c400 msgr2=0x7ff9e40b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:06.837 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.816+0000 7ff9e9ba5640 1 --2- 172.21.15.88:0/23674999 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff9e414c400 0x7ff9e40b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:06.837 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.816+0000 7ff9e9ba5640 1 -- 172.21.15.88:0/23674999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9e415a820 msgr2=0x7ff9e415acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:06.837 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.816+0000 7ff9e9ba5640 1 --2- 172.21.15.88:0/23674999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9e415a820 0x7ff9e415acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:06.838 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.816+0000 7ff9e9ba5640 1 -- 172.21.15.88:0/23674999 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e40b7720 con 0x7ff9d0003f00 2023-10-16T20:08:06.838 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.816+0000 7ff9ea3a6640 1 --2- 172.21.15.88:0/23674999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9e415a820 0x7ff9e415acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:08:06.838 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.816+0000 7ff9e9ba5640 1 --2- 172.21.15.88:0/23674999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d0003f00 0x7ff9e4148ef0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0011a20 tx=0x7ff9e0011e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:06.838 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.816+0000 7ff9da7fc640 1 -- 172.21.15.88:0/23674999 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e000faf0 con 0x7ff9d0003f00 2023-10-16T20:08:06.839 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.816+0000 7ff9da7fc640 1 -- 172.21.15.88:0/23674999 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e000fc50 con 0x7ff9d0003f00 2023-10-16T20:08:06.839 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.816+0000 7ff9eaba7640 1 -- 172.21.15.88:0/23674999 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e40b7a00 con 0x7ff9d0003f00 2023-10-16T20:08:06.839 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.816+0000 7ff9da7fc640 1 -- 172.21.15.88:0/23674999 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e00059a0 con 0x7ff9d0003f00 2023-10-16T20:08:06.839 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.816+0000 7ff9eaba7640 1 -- 172.21.15.88:0/23674999 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e40b7fc0 con 0x7ff9d0003f00 2023-10-16T20:08:06.840 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.816+0000 7ff9eaba7640 1 -- 172.21.15.88:0/23674999 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff9e40ce440 con 0x7ff9d0003f00 2023-10-16T20:08:06.840 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.820+0000 7ff9da7fc640 1 -- 172.21.15.88:0/23674999 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff9e0005b00 con 0x7ff9d0003f00 2023-10-16T20:08:06.840 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.820+0000 7ff9da7fc640 1 --2- 172.21.15.88:0/23674999 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff9c80664d0 0x7ff9c8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:06.841 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.820+0000 7ff9da7fc640 1 -- 172.21.15.88:0/23674999 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5263+0+0 (secure 0 0 0) 0x7ff9e00847b0 con 0x7ff9d0003f00 2023-10-16T20:08:06.841 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.820+0000 7ff9e93a4640 1 --2- 172.21.15.88:0/23674999 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff9c80664d0 0x7ff9c8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:06.841 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.820+0000 7ff9da7fc640 1 --2- 172.21.15.88:0/23674999 >> [v2:172.21.15.94:6808/722063332,v1:172.21.15.94:6809/722063332] conn(0x7ff9c806c7d0 0x7ff9c806ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:06.841 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.820+0000 7ff9da7fc640 1 -- 172.21.15.88:0/23674999 --> [v2:172.21.15.94:6808/722063332,v1:172.21.15.94:6809/722063332] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff9c806f2e0 con 0x7ff9c806c7d0 2023-10-16T20:08:06.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.820+0000 7ff9da7fc640 1 -- 172.21.15.88:0/23674999 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9e0084b20 con 0x7ff9d0003f00 2023-10-16T20:08:06.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.820+0000 7ff9ea3a6640 1 --2- 172.21.15.88:0/23674999 >> [v2:172.21.15.94:6808/722063332,v1:172.21.15.94:6809/722063332] conn(0x7ff9c806c7d0 0x7ff9c806ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:06.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.820+0000 7ff9e93a4640 1 --2- 172.21.15.88:0/23674999 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff9c80664d0 0x7ff9c8068990 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff9d40af000 tx=0x7ff9d40046d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:06.842 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.820+0000 7ff9ea3a6640 1 --2- 172.21.15.88:0/23674999 >> [v2:172.21.15.94:6808/722063332,v1:172.21.15.94:6809/722063332] conn(0x7ff9c806c7d0 0x7ff9c806ec10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:06.843 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.820+0000 7ff9da7fc640 1 -- 172.21.15.88:0/23674999 <== osd.4 v2:172.21.15.94:6808/722063332 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7ff9c806f2e0 con 0x7ff9c806c7d0 2023-10-16T20:08:06.845 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.832+0000 7ff9eaba7640 1 -- 172.21.15.88:0/23674999 --> [v2:172.21.15.94:6808/722063332,v1:172.21.15.94:6809/722063332] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff9e40ce160 con 0x7ff9c806c7d0 2023-10-16T20:08:06.846 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.836+0000 7ff9da7fc640 1 -- 172.21.15.88:0/23674999 <== osd.4 v2:172.21.15.94:6808/722063332 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff9dc005ab0 con 0x7ff9c806c7d0 2023-10-16T20:08:06.846 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2023-10-16T20:08:06.847 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T20:08:06.847 INFO:teuthology.orchestra.run.smithi088.stdout:} 2023-10-16T20:08:06.848 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.836+0000 7ff9eaba7640 1 -- 172.21.15.88:0/23674999 >> [v2:172.21.15.94:6808/722063332,v1:172.21.15.94:6809/722063332] conn(0x7ff9c806c7d0 msgr2=0x7ff9c806ec10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:06.848 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.836+0000 7ff9eaba7640 1 --2- 172.21.15.88:0/23674999 >> [v2:172.21.15.94:6808/722063332,v1:172.21.15.94:6809/722063332] conn(0x7ff9c806c7d0 0x7ff9c806ec10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:06.848 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.836+0000 7ff9eaba7640 1 -- 172.21.15.88:0/23674999 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff9c80664d0 msgr2=0x7ff9c8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:06.849 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.836+0000 7ff9eaba7640 1 --2- 172.21.15.88:0/23674999 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7ff9c80664d0 0x7ff9c8068990 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff9d40af000 tx=0x7ff9d40046d0 comp rx=0 tx=0).stop 2023-10-16T20:08:06.849 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.836+0000 7ff9eaba7640 1 -- 172.21.15.88:0/23674999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d0003f00 msgr2=0x7ff9e4148ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:06.849 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.836+0000 7ff9eaba7640 1 --2- 172.21.15.88:0/23674999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d0003f00 0x7ff9e4148ef0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0011a20 tx=0x7ff9e0011e20 comp rx=0 tx=0).stop 2023-10-16T20:08:06.849 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.836+0000 7ff9ea3a6640 1 -- 172.21.15.88:0/23674999 reap_dead start 2023-10-16T20:08:06.850 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.836+0000 7ff9eaba7640 1 -- 172.21.15.88:0/23674999 shutdown_connections 2023-10-16T20:08:06.850 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.836+0000 7ff9eaba7640 1 -- 172.21.15.88:0/23674999 >> 172.21.15.88:0/23674999 conn(0x7ff9e40d2850 msgr2=0x7ff9e406ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:06.850 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.836+0000 7ff9eaba7640 1 -- 172.21.15.88:0/23674999 shutdown_connections 2023-10-16T20:08:06.851 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.836+0000 7ff9eaba7640 1 -- 172.21.15.88:0/23674999 wait complete. 2023-10-16T20:08:06.859 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-16T20:08:06.997 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.992+0000 7f23a9209640 1 Processor -- start 2023-10-16T20:08:06.998 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.992+0000 7f23a9209640 1 -- start start 2023-10-16T20:08:07.000 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.992+0000 7f23a9209640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f23a40ce080 0x7f23a40d1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:07.001 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.992+0000 7f23a9209640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23a40b5510 con 0x7f23a4153ea0 2023-10-16T20:08:07.001 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.992+0000 7f23a9209640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23a415d9a0 con 0x7f23a40d2660 2023-10-16T20:08:07.001 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.992+0000 7f23a9209640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a40b4990 con 0x7f23a40ce080 2023-10-16T20:08:07.002 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.992+0000 7f23a2d9d640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f23a4153ea0 0x7f23a415d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:47268/0 (socket says 172.21.15.88:47268) 2023-10-16T20:08:07.002 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.992+0000 7f23a2d9d640 1 -- 172.21.15.88:0/3880805617 learned_addr learned my addr 172.21.15.88:0/3880805617 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:08:07.002 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.992+0000 7f23a1d9b640 1 -- 172.21.15.88:0/3880805617 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3029713678 0 0) 0x7f23a40b5510 con 0x7f23a4153ea0 2023-10-16T20:08:07.002 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.992+0000 7f23a1d9b640 1 -- 172.21.15.88:0/3880805617 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f238c003530 con 0x7f23a4153ea0 2023-10-16T20:08:07.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.992+0000 7f23a359e640 1 --2- 172.21.15.88:0/3880805617 >> v2:172.21.15.94:3300/0 conn(0x7f23a40ce080 0x7f23a40d1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:07.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.992+0000 7f23a1d9b640 1 -- 172.21.15.88:0/3880805617 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1609690338 0 0) 0x7f238c003530 con 0x7f23a4153ea0 2023-10-16T20:08:07.003 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.992+0000 7f23a1d9b640 1 -- 172.21.15.88:0/3880805617 >> v2:172.21.15.94:3300/0 conn(0x7f23a40ce080 msgr2=0x7f23a40d1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:08:07.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.992+0000 7f23a1d9b640 1 --2- 172.21.15.88:0/3880805617 >> v2:172.21.15.94:3300/0 conn(0x7f23a40ce080 0x7f23a40d1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:07.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.992+0000 7f23a1d9b640 1 -- 172.21.15.88:0/3880805617 >> v1:172.21.15.94:6789/0 conn(0x7f23a40d2660 legacy=0x7f23a40cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:07.004 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.992+0000 7f23a1d9b640 1 -- 172.21.15.88:0/3880805617 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a40b4c70 con 0x7f23a4153ea0 2023-10-16T20:08:07.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.992+0000 7f23a1d9b640 1 -- 172.21.15.88:0/3880805617 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1593089355 0 0) 0x7f239c001a00 con 0x7f23a4153ea0 2023-10-16T20:08:07.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.996+0000 7f23a1d9b640 1 -- 172.21.15.88:0/3880805617 >> v1:172.21.15.88:6789/0 conn(0x7f23a4153ea0 legacy=0x7f23a415d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:07.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.996+0000 7f23a1d9b640 1 --2- 172.21.15.88:0/3880805617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f238c003980 0x7f238c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:07.005 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.996+0000 7f23a1d9b640 1 --2- 172.21.15.88:0/3880805617 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f238c0047d0 0x7f238c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:07.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.996+0000 7f23a1d9b640 1 --2- 172.21.15.88:0/3880805617 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f238c005190 0x7f238c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:07.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.996+0000 7f23a1d9b640 1 -- 172.21.15.88:0/3880805617 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a40b4990 con 0x7f238c005190 2023-10-16T20:08:07.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.996+0000 7f23a1d9b640 1 -- 172.21.15.88:0/3880805617 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238c00c030 con 0x7f238c003980 2023-10-16T20:08:07.006 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.996+0000 7f23a1d9b640 1 -- 172.21.15.88:0/3880805617 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f238c00c1f0 con 0x7f238c0047d0 2023-10-16T20:08:07.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.996+0000 7f23a359e640 1 --2- 172.21.15.88:0/3880805617 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f238c005190 0x7f238c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:07.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.996+0000 7f23a2d9d640 1 --2- 172.21.15.88:0/3880805617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f238c003980 0x7f238c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:07.007 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.996+0000 7f23a259c640 1 --2- 172.21.15.88:0/3880805617 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f238c0047d0 0x7f238c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:07.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.996+0000 7f23a359e640 1 -- 172.21.15.88:0/3880805617 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f238c0047d0 msgr2=0x7f238c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:08:07.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.996+0000 7f23a359e640 1 --2- 172.21.15.88:0/3880805617 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f238c0047d0 0x7f238c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:07.008 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.996+0000 7f23a359e640 1 -- 172.21.15.88:0/3880805617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f238c003980 msgr2=0x7f238c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:08:07.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.996+0000 7f23a359e640 1 --2- 172.21.15.88:0/3880805617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f238c003980 0x7f238c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:07.009 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.996+0000 7f23a359e640 1 -- 172.21.15.88:0/3880805617 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f238c003530 con 0x7f238c005190 2023-10-16T20:08:07.012 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.996+0000 7f23a359e640 1 -- 172.21.15.88:0/3880805617 reap_dead start 2023-10-16T20:08:07.013 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.996+0000 7f23a359e640 1 --2- 172.21.15.88:0/3880805617 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f238c005190 0x7f238c00b990 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f2398009ae0 tx=0x7f239801ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68acebb4bed83ba server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:07.013 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.996+0000 7f23a1d9b640 1 -- 172.21.15.88:0/3880805617 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2398004020 con 0x7f238c005190 2023-10-16T20:08:07.013 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.996+0000 7f23a1d9b640 1 -- 172.21.15.88:0/3880805617 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23980041c0 con 0x7f238c005190 2023-10-16T20:08:07.014 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:06.996+0000 7f23a1d9b640 1 -- 172.21.15.88:0/3880805617 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23980a5680 con 0x7f238c005190 2023-10-16T20:08:07.014 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.000+0000 7f23a9209640 1 -- 172.21.15.88:0/3880805617 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f238c005190 msgr2=0x7f238c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:07.014 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.000+0000 7f23a9209640 1 --2- 172.21.15.88:0/3880805617 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f238c005190 0x7f238c00b990 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f2398009ae0 tx=0x7f239801ca90 comp rx=0 tx=0).stop 2023-10-16T20:08:07.014 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.000+0000 7f23a9209640 1 -- 172.21.15.88:0/3880805617 shutdown_connections 2023-10-16T20:08:07.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.000+0000 7f23a9209640 1 --2- 172.21.15.88:0/3880805617 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f238c005190 0x7f238c00b990 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:07.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.000+0000 7f23a9209640 1 -- 172.21.15.88:0/3880805617 >> 172.21.15.88:0/3880805617 conn(0x7f23a40715c0 msgr2=0x7f23a40719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:07.015 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.000+0000 7f23a9209640 1 -- 172.21.15.88:0/3880805617 shutdown_connections 2023-10-16T20:08:07.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.000+0000 7f23a9209640 1 -- 172.21.15.88:0/3880805617 wait complete. 2023-10-16T20:08:07.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.000+0000 7f23a9209640 1 Processor -- start 2023-10-16T20:08:07.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.000+0000 7f23a9209640 1 -- start start 2023-10-16T20:08:07.016 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.000+0000 7f23a9209640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f238c005190 0x7f23a41511a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:07.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.000+0000 7f23a9209640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f23a4160c00 0x7f23a4161080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:07.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.000+0000 7f23a9209640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23a40cbe40 0x7f23a40cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:07.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.000+0000 7f23a9209640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a41615c0 con 0x7f238c005190 2023-10-16T20:08:07.017 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.000+0000 7f23a9209640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a4154c40 con 0x7f23a40cbe40 2023-10-16T20:08:07.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.000+0000 7f23a9209640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a4154ee0 con 0x7f23a4160c00 2023-10-16T20:08:07.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.000+0000 7f23a2d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f238c005190 0x7f23a41511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:07.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.000+0000 7f23a2d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f238c005190 0x7f23a41511a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60102/0 (socket says 172.21.15.88:60102) 2023-10-16T20:08:07.018 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.000+0000 7f23a2d9d640 1 -- 172.21.15.88:0/591631511 learned_addr learned my addr 172.21.15.88:0/591631511 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:07.019 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.000+0000 7f23a259c640 1 --2- 172.21.15.88:0/591631511 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f23a4160c00 0x7f23a4161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:07.019 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.004+0000 7f23a359e640 1 --2- 172.21.15.88:0/591631511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23a40cbe40 0x7f23a40cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:07.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.004+0000 7f23a2d9d640 1 -- 172.21.15.88:0/591631511 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f23a4160c00 msgr2=0x7f23a4161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:07.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.004+0000 7f23a2d9d640 1 --2- 172.21.15.88:0/591631511 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f23a4160c00 0x7f23a4161080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:07.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.004+0000 7f23a2d9d640 1 -- 172.21.15.88:0/591631511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23a40cbe40 msgr2=0x7f23a40cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:07.020 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.004+0000 7f23a2d9d640 1 --2- 172.21.15.88:0/591631511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23a40cbe40 0x7f23a40cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:07.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.004+0000 7f23a2d9d640 1 -- 172.21.15.88:0/591631511 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a41551c0 con 0x7f238c005190 2023-10-16T20:08:07.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.004+0000 7f23a2d9d640 1 --2- 172.21.15.88:0/591631511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f238c005190 0x7f23a41511a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f239c007970 tx=0x7f239c00a6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:07.021 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.004+0000 7f238b7fe640 1 -- 172.21.15.88:0/591631511 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239c007a60 con 0x7f238c005190 2023-10-16T20:08:07.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.004+0000 7f238b7fe640 1 -- 172.21.15.88:0/591631511 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f239c0133f0 con 0x7f238c005190 2023-10-16T20:08:07.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.004+0000 7f238b7fe640 1 -- 172.21.15.88:0/591631511 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239c0137a0 con 0x7f238c005190 2023-10-16T20:08:07.022 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.004+0000 7f23a9209640 1 -- 172.21.15.88:0/591631511 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a41554a0 con 0x7f238c005190 2023-10-16T20:08:07.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.004+0000 7f23a9209640 1 -- 172.21.15.88:0/591631511 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a40cc800 con 0x7f238c005190 2023-10-16T20:08:07.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.004+0000 7f23a9209640 1 -- 172.21.15.88:0/591631511 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a404e9c0 con 0x7f238c005190 2023-10-16T20:08:07.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.004+0000 7f238b7fe640 1 -- 172.21.15.88:0/591631511 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f239c013900 con 0x7f238c005190 2023-10-16T20:08:07.023 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.004+0000 7f238b7fe640 1 --2- 172.21.15.88:0/591631511 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f23900664a0 0x7f2390068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:07.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.004+0000 7f238b7fe640 1 -- 172.21.15.88:0/591631511 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f239c0847e0 con 0x7f238c005190 2023-10-16T20:08:07.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.004+0000 7f23a259c640 1 --2- 172.21.15.88:0/591631511 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f23900664a0 0x7f2390068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:07.024 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.008+0000 7f23a259c640 1 --2- 172.21.15.88:0/591631511 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f23900664a0 0x7f2390068960 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f23a41f5900 tx=0x7f2394009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:07.025 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.008+0000 7f238b7fe640 1 -- 172.21.15.88:0/591631511 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f239c051e30 con 0x7f238c005190 2023-10-16T20:08:07.166 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.160+0000 7f23a9209640 1 -- 172.21.15.88:0/591631511 --> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f23a40b5230 con 0x7f23900664a0 2023-10-16T20:08:07.167 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.160+0000 7f238b7fe640 1 -- 172.21.15.88:0/591631511 <== mgr.4110 v2:172.21.15.88:6832/4078483333 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f23940115a0 con 0x7f23900664a0 2023-10-16T20:08:07.168 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 4 to deep-scrub 2023-10-16T20:08:07.169 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.164+0000 7f23a9209640 1 -- 172.21.15.88:0/591631511 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f23900664a0 msgr2=0x7f2390068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:07.170 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.164+0000 7f23a9209640 1 --2- 172.21.15.88:0/591631511 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f23900664a0 0x7f2390068960 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f23a41f5900 tx=0x7f2394009040 comp rx=0 tx=0).stop 2023-10-16T20:08:07.170 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.164+0000 7f23a9209640 1 -- 172.21.15.88:0/591631511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f238c005190 msgr2=0x7f23a41511a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:07.170 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.164+0000 7f23a9209640 1 --2- 172.21.15.88:0/591631511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f238c005190 0x7f23a41511a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f239c007970 tx=0x7f239c00a6e0 comp rx=0 tx=0).stop 2023-10-16T20:08:07.171 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.164+0000 7f23a9209640 1 -- 172.21.15.88:0/591631511 shutdown_connections 2023-10-16T20:08:07.171 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.164+0000 7f23a9209640 1 --2- 172.21.15.88:0/591631511 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f23900664a0 0x7f2390068960 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:07.171 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.164+0000 7f23a9209640 1 --2- 172.21.15.88:0/591631511 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f23a4160c00 0x7f23a4161080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:07.171 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.164+0000 7f23a9209640 1 --2- 172.21.15.88:0/591631511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23a40cbe40 0x7f23a40cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:07.172 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.164+0000 7f23a9209640 1 --2- 172.21.15.88:0/591631511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f238c005190 0x7f23a41511a0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:07.172 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.164+0000 7f23a9209640 1 -- 172.21.15.88:0/591631511 >> 172.21.15.88:0/591631511 conn(0x7f23a40715c0 msgr2=0x7f23a415e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:07.172 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.164+0000 7f23a9209640 1 -- 172.21.15.88:0/591631511 shutdown_connections 2023-10-16T20:08:07.172 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.164+0000 7f23a9209640 1 -- 172.21.15.88:0/591631511 wait complete. 2023-10-16T20:08:07.187 INFO:tasks.ceph:Scrubbing osd.5 2023-10-16T20:08:07.188 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-16T20:08:07.331 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.324+0000 7f377a268640 1 Processor -- start 2023-10-16T20:08:07.332 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.324+0000 7f377a268640 1 -- start start 2023-10-16T20:08:07.333 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.328+0000 7f377a268640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3774159730 0x7f37740ad700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:07.334 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.328+0000 7f377a268640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f37740c8cb0 0x7f3774069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:07.334 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.328+0000 7f377a268640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f377415d660 con 0x7f37740adc40 2023-10-16T20:08:07.335 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.328+0000 7f377a268640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f377415e2e0 con 0x7f37740c8cb0 2023-10-16T20:08:07.335 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.328+0000 7f377a268640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f377415d940 con 0x7f3774159730 2023-10-16T20:08:07.335 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.328+0000 7f37737fe640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f37740c8cb0 0x7f3774069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:07.336 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.328+0000 7f37737fe640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f37740c8cb0 0x7f3774069e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60114/0 (socket says 172.21.15.88:60114) 2023-10-16T20:08:07.336 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.328+0000 7f37737fe640 1 -- 172.21.15.88:0/3658818487 learned_addr learned my addr 172.21.15.88:0/3658818487 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:07.336 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.328+0000 7f3773fff640 1 --2- 172.21.15.88:0/3658818487 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3774159730 0x7f37740ad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:07.337 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.328+0000 7f37737fe640 1 -- 172.21.15.88:0/3658818487 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3774159730 msgr2=0x7f37740ad700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:08:07.337 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.328+0000 7f37737fe640 1 --2- 172.21.15.88:0/3658818487 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3774159730 0x7f37740ad700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:07.337 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.328+0000 7f37737fe640 1 -- 172.21.15.88:0/3658818487 >> v1:172.21.15.94:6789/0 conn(0x7f37740adc40 legacy=0x7f37740cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:07.338 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.328+0000 7f37737fe640 1 -- 172.21.15.88:0/3658818487 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377415dc20 con 0x7f37740c8cb0 2023-10-16T20:08:07.341 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.332+0000 7f37737fe640 1 --2- 172.21.15.88:0/3658818487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37740c8cb0 0x7f3774069e90 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f376800bb10 tx=0x7f376801ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a886cd8d32294bd0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:07.341 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.332+0000 7f3772ffd640 1 -- 172.21.15.88:0/3658818487 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37680a7c90 con 0x7f37740c8cb0 2023-10-16T20:08:07.341 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.332+0000 7f3772ffd640 1 -- 172.21.15.88:0/3658818487 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37680a7df0 con 0x7f37740c8cb0 2023-10-16T20:08:07.341 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.332+0000 7f3772ffd640 1 -- 172.21.15.88:0/3658818487 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37680af660 con 0x7f37740c8cb0 2023-10-16T20:08:07.342 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.332+0000 7f377a268640 1 -- 172.21.15.88:0/3658818487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37740c8cb0 msgr2=0x7f3774069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:07.342 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.332+0000 7f377a268640 1 --2- 172.21.15.88:0/3658818487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37740c8cb0 0x7f3774069e90 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f376800bb10 tx=0x7f376801ebe0 comp rx=0 tx=0).stop 2023-10-16T20:08:07.342 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.332+0000 7f377a268640 1 -- 172.21.15.88:0/3658818487 shutdown_connections 2023-10-16T20:08:07.343 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.332+0000 7f377a268640 1 --2- 172.21.15.88:0/3658818487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37740c8cb0 0x7f3774069e90 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:07.343 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.332+0000 7f377a268640 1 --2- 172.21.15.88:0/3658818487 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3774159730 0x7f37740ad700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:07.343 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.332+0000 7f377a268640 1 -- 172.21.15.88:0/3658818487 >> 172.21.15.88:0/3658818487 conn(0x7f37740ce660 msgr2=0x7f37740cea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:07.344 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.332+0000 7f377a268640 1 -- 172.21.15.88:0/3658818487 shutdown_connections 2023-10-16T20:08:07.344 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.332+0000 7f377a268640 1 -- 172.21.15.88:0/3658818487 wait complete. 2023-10-16T20:08:07.344 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.332+0000 7f377a268640 1 Processor -- start 2023-10-16T20:08:07.345 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.332+0000 7f377a268640 1 -- start start 2023-10-16T20:08:07.345 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.332+0000 7f377a268640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37740adc40 0x7f37740c5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:07.345 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.332+0000 7f377a268640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37740c8cb0 0x7f377414a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:07.345 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.332+0000 7f377a268640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3774159730 0x7f377414a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:07.346 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.332+0000 7f377a268640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377414bd80 con 0x7f3774159730 2023-10-16T20:08:07.346 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.332+0000 7f377a268640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377414baa0 con 0x7f37740c8cb0 2023-10-16T20:08:07.346 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.332+0000 7f377a268640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3774149590 con 0x7f37740adc40 2023-10-16T20:08:07.346 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.336+0000 7f3778805640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3774159730 0x7f377414a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:07.347 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.336+0000 7f3778805640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3774159730 0x7f377414a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60126/0 (socket says 172.21.15.88:60126) 2023-10-16T20:08:07.347 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.336+0000 7f3778805640 1 -- 172.21.15.88:0/3603341016 learned_addr learned my addr 172.21.15.88:0/3603341016 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:07.347 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.336+0000 7f37737fe640 1 --2- 172.21.15.88:0/3603341016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37740c8cb0 0x7f377414a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:07.347 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.336+0000 7f3773fff640 1 --2- 172.21.15.88:0/3603341016 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37740adc40 0x7f37740c5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:07.348 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.336+0000 7f3778805640 1 -- 172.21.15.88:0/3603341016 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37740adc40 msgr2=0x7f37740c5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:07.348 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.336+0000 7f3778805640 1 --2- 172.21.15.88:0/3603341016 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37740adc40 0x7f37740c5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:07.348 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.336+0000 7f3778805640 1 -- 172.21.15.88:0/3603341016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37740c8cb0 msgr2=0x7f377414a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:07.348 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.336+0000 7f3778805640 1 --2- 172.21.15.88:0/3603341016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37740c8cb0 0x7f377414a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:07.349 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.336+0000 7f3778805640 1 -- 172.21.15.88:0/3603341016 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f376800b770 con 0x7f3774159730 2023-10-16T20:08:07.349 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.336+0000 7f3778805640 1 --2- 172.21.15.88:0/3603341016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3774159730 0x7f377414a5b0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f376c00c7a0 tx=0x7f376c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:07.349 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.336+0000 7f3770ff9640 1 -- 172.21.15.88:0/3603341016 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c003ba0 con 0x7f3774159730 2023-10-16T20:08:07.349 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.336+0000 7f377a268640 1 -- 172.21.15.88:0/3603341016 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3774149870 con 0x7f3774159730 2023-10-16T20:08:07.350 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.336+0000 7f3770ff9640 1 -- 172.21.15.88:0/3603341016 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f376c003d00 con 0x7f3774159730 2023-10-16T20:08:07.350 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.336+0000 7f3770ff9640 1 -- 172.21.15.88:0/3603341016 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c00aa30 con 0x7f3774159730 2023-10-16T20:08:07.350 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.336+0000 7f377a268640 1 -- 172.21.15.88:0/3603341016 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37740b3ed0 con 0x7f3774159730 2023-10-16T20:08:07.352 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.336+0000 7f377a268640 1 -- 172.21.15.88:0/3603341016 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f377406a7e0 con 0x7f3774159730 2023-10-16T20:08:07.352 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.336+0000 7f3770ff9640 1 -- 172.21.15.88:0/3603341016 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f376c00ab90 con 0x7f3774159730 2023-10-16T20:08:07.352 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.336+0000 7f3770ff9640 1 --2- 172.21.15.88:0/3603341016 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f37600664a0 0x7f3760068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:07.353 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.336+0000 7f3770ff9640 1 -- 172.21.15.88:0/3603341016 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f376c084470 con 0x7f3774159730 2023-10-16T20:08:07.353 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.336+0000 7f3770ff9640 1 --2- 172.21.15.88:0/3603341016 >> [v2:172.21.15.94:6800/3097673529,v1:172.21.15.94:6801/3097673529] conn(0x7f376006c7a0 0x7f376006ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:07.353 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.336+0000 7f3773fff640 1 --2- 172.21.15.88:0/3603341016 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f37600664a0 0x7f3760068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:07.354 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.340+0000 7f3770ff9640 1 -- 172.21.15.88:0/3603341016 --> [v2:172.21.15.94:6800/3097673529,v1:172.21.15.94:6801/3097673529] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f376006f2b0 con 0x7f376006c7a0 2023-10-16T20:08:07.354 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.340+0000 7f3770ff9640 1 -- 172.21.15.88:0/3603341016 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f376c0847e0 con 0x7f3774159730 2023-10-16T20:08:07.354 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.340+0000 7f37737fe640 1 --2- 172.21.15.88:0/3603341016 >> [v2:172.21.15.94:6800/3097673529,v1:172.21.15.94:6801/3097673529] conn(0x7f376006c7a0 0x7f376006ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:07.354 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.340+0000 7f3773fff640 1 --2- 172.21.15.88:0/3603341016 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f37600664a0 0x7f3760068960 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3774151380 tx=0x7f3764009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:07.355 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.340+0000 7f37737fe640 1 --2- 172.21.15.88:0/3603341016 >> [v2:172.21.15.94:6800/3097673529,v1:172.21.15.94:6801/3097673529] conn(0x7f376006c7a0 0x7f376006ebe0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:07.355 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.340+0000 7f3770ff9640 1 -- 172.21.15.88:0/3603341016 <== osd.5 v2:172.21.15.94:6800/3097673529 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f376006f2b0 con 0x7f376006c7a0 2023-10-16T20:08:07.358 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.352+0000 7f377a268640 1 -- 172.21.15.88:0/3603341016 --> [v2:172.21.15.94:6800/3097673529,v1:172.21.15.94:6801/3097673529] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f377404e9c0 con 0x7f376006c7a0 2023-10-16T20:08:07.358 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.352+0000 7f3770ff9640 1 -- 172.21.15.88:0/3603341016 <== osd.5 v2:172.21.15.94:6800/3097673529 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f377404e9c0 con 0x7f376006c7a0 2023-10-16T20:08:07.359 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2023-10-16T20:08:07.360 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T20:08:07.360 INFO:teuthology.orchestra.run.smithi088.stdout:} 2023-10-16T20:08:07.361 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.352+0000 7f377a268640 1 -- 172.21.15.88:0/3603341016 >> [v2:172.21.15.94:6800/3097673529,v1:172.21.15.94:6801/3097673529] conn(0x7f376006c7a0 msgr2=0x7f376006ebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:07.361 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.352+0000 7f377a268640 1 --2- 172.21.15.88:0/3603341016 >> [v2:172.21.15.94:6800/3097673529,v1:172.21.15.94:6801/3097673529] conn(0x7f376006c7a0 0x7f376006ebe0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:07.362 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.352+0000 7f377a268640 1 -- 172.21.15.88:0/3603341016 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f37600664a0 msgr2=0x7f3760068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:07.362 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.352+0000 7f377a268640 1 --2- 172.21.15.88:0/3603341016 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f37600664a0 0x7f3760068960 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3774151380 tx=0x7f3764009190 comp rx=0 tx=0).stop 2023-10-16T20:08:07.362 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.352+0000 7f377a268640 1 -- 172.21.15.88:0/3603341016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3774159730 msgr2=0x7f377414a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:07.363 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.352+0000 7f377a268640 1 --2- 172.21.15.88:0/3603341016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3774159730 0x7f377414a5b0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f376c00c7a0 tx=0x7f376c00cc60 comp rx=0 tx=0).stop 2023-10-16T20:08:07.363 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.352+0000 7f3778805640 1 -- 172.21.15.88:0/3603341016 reap_dead start 2023-10-16T20:08:07.363 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.352+0000 7f377a268640 1 -- 172.21.15.88:0/3603341016 shutdown_connections 2023-10-16T20:08:07.364 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.352+0000 7f377a268640 1 -- 172.21.15.88:0/3603341016 >> 172.21.15.88:0/3603341016 conn(0x7f37740ce660 msgr2=0x7f37740aeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:07.364 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.352+0000 7f377a268640 1 -- 172.21.15.88:0/3603341016 shutdown_connections 2023-10-16T20:08:07.364 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.352+0000 7f377a268640 1 -- 172.21.15.88:0/3603341016 wait complete. 2023-10-16T20:08:07.377 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-16T20:08:07.515 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.508+0000 7f4958df8640 1 Processor -- start 2023-10-16T20:08:07.516 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.512+0000 7f4958df8640 1 -- start start 2023-10-16T20:08:07.517 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.512+0000 7f4958df8640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f49541584a0 0x7f495415e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:07.517 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.512+0000 7f4958df8640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f49540cc5d0 0x7f49540cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:07.518 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.512+0000 7f4958df8640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f495415b250 con 0x7f49540b21e0 2023-10-16T20:08:07.518 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.512+0000 7f4958df8640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f495415f4d0 con 0x7f49540cc5d0 2023-10-16T20:08:07.518 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.512+0000 7f4958df8640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f495415afb0 con 0x7f49541584a0 2023-10-16T20:08:07.519 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.512+0000 7f494bfff640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f49540b21e0 0x7f495415b6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:47270/0 (socket says 172.21.15.88:47270) 2023-10-16T20:08:07.519 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.512+0000 7f494bfff640 1 -- 172.21.15.88:0/4152266099 learned_addr learned my addr 172.21.15.88:0/4152266099 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:08:07.519 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.512+0000 7f4952d9d640 1 --2- 172.21.15.88:0/4152266099 >> v2:172.21.15.88:3300/0 conn(0x7f49540cc5d0 0x7f49540cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:07.519 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.512+0000 7f495259c640 1 --2- 172.21.15.88:0/4152266099 >> v2:172.21.15.94:3300/0 conn(0x7f49541584a0 0x7f495415e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:07.520 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.512+0000 7f4951d9b640 1 -- 172.21.15.88:0/4152266099 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2415461011 0 0) 0x7f495415b250 con 0x7f49540b21e0 2023-10-16T20:08:07.521 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.512+0000 7f4951d9b640 1 -- 172.21.15.88:0/4152266099 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4940003530 con 0x7f49540b21e0 2023-10-16T20:08:07.521 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.512+0000 7f4952d9d640 1 -- 172.21.15.88:0/4152266099 >> v2:172.21.15.94:3300/0 conn(0x7f49541584a0 msgr2=0x7f495415e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:08:07.521 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.512+0000 7f4952d9d640 1 --2- 172.21.15.88:0/4152266099 >> v2:172.21.15.94:3300/0 conn(0x7f49541584a0 0x7f495415e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:07.521 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.512+0000 7f4952d9d640 1 -- 172.21.15.88:0/4152266099 >> v1:172.21.15.88:6789/0 conn(0x7f49540b21e0 legacy=0x7f495415b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:07.522 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.512+0000 7f4952d9d640 1 -- 172.21.15.88:0/4152266099 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49540ccef0 con 0x7f49540cc5d0 2023-10-16T20:08:07.522 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.516+0000 7f4952d9d640 1 --2- 172.21.15.88:0/4152266099 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49540cc5d0 0x7f49540cc9b0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f49540b1050 tx=0x7f494c01c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15ea7eb354d94a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:07.523 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.516+0000 7f4951d9b640 1 -- 172.21.15.88:0/4152266099 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494c0a5940 con 0x7f49540cc5d0 2023-10-16T20:08:07.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.516+0000 7f4951d9b640 1 -- 172.21.15.88:0/4152266099 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494c0a5aa0 con 0x7f49540cc5d0 2023-10-16T20:08:07.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.516+0000 7f4951d9b640 1 -- 172.21.15.88:0/4152266099 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494c0a5e30 con 0x7f49540cc5d0 2023-10-16T20:08:07.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.516+0000 7f4958df8640 1 -- 172.21.15.88:0/4152266099 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49540cc5d0 msgr2=0x7f49540cc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:07.524 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.516+0000 7f4958df8640 1 --2- 172.21.15.88:0/4152266099 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49540cc5d0 0x7f49540cc9b0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f49540b1050 tx=0x7f494c01c880 comp rx=0 tx=0).stop 2023-10-16T20:08:07.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.516+0000 7f4958df8640 1 -- 172.21.15.88:0/4152266099 shutdown_connections 2023-10-16T20:08:07.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.516+0000 7f4958df8640 1 --2- 172.21.15.88:0/4152266099 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49540cc5d0 0x7f49540cc9b0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:07.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.516+0000 7f4958df8640 1 --2- 172.21.15.88:0/4152266099 >> v2:172.21.15.94:3300/0 conn(0x7f49541584a0 0x7f495415e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:07.525 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.516+0000 7f4958df8640 1 -- 172.21.15.88:0/4152266099 >> 172.21.15.88:0/4152266099 conn(0x7f49540ce5e0 msgr2=0x7f49540ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:07.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.516+0000 7f4958df8640 1 -- 172.21.15.88:0/4152266099 shutdown_connections 2023-10-16T20:08:07.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.516+0000 7f4958df8640 1 -- 172.21.15.88:0/4152266099 wait complete. 2023-10-16T20:08:07.526 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.516+0000 7f4958df8640 1 Processor -- start 2023-10-16T20:08:07.527 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.520+0000 7f4958df8640 1 -- start start 2023-10-16T20:08:07.529 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.520+0000 7f4958df8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49540b21e0 0x7f49541645a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:07.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.520+0000 7f4958df8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49540cc5d0 0x7f49541f85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:07.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.520+0000 7f4958df8640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49541584a0 0x7f4954156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:07.530 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.520+0000 7f4958df8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49540ab420 con 0x7f49540cc5d0 2023-10-16T20:08:07.531 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.520+0000 7f4958df8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49541f8b20 con 0x7f49540b21e0 2023-10-16T20:08:07.531 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.520+0000 7f4958df8640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4954070130 con 0x7f49541584a0 2023-10-16T20:08:07.531 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.520+0000 7f494bfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49540cc5d0 0x7f49541f85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:07.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.520+0000 7f494bfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49540cc5d0 0x7f49541f85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60150/0 (socket says 172.21.15.88:60150) 2023-10-16T20:08:07.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.520+0000 7f494bfff640 1 -- 172.21.15.88:0/660550447 learned_addr learned my addr 172.21.15.88:0/660550447 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:07.532 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.520+0000 7f495259c640 1 --2- 172.21.15.88:0/660550447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49540b21e0 0x7f49541645a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:07.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.520+0000 7f4952d9d640 1 --2- 172.21.15.88:0/660550447 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49541584a0 0x7f4954156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:07.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.520+0000 7f494bfff640 1 -- 172.21.15.88:0/660550447 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49541584a0 msgr2=0x7f4954156ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:07.533 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.520+0000 7f494bfff640 1 --2- 172.21.15.88:0/660550447 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49541584a0 0x7f4954156ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:07.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.520+0000 7f494bfff640 1 -- 172.21.15.88:0/660550447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49540b21e0 msgr2=0x7f49541645a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:07.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.520+0000 7f494bfff640 1 --2- 172.21.15.88:0/660550447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49540b21e0 0x7f49541645a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:07.534 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.520+0000 7f494bfff640 1 -- 172.21.15.88:0/660550447 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f494c009a70 con 0x7f49540cc5d0 2023-10-16T20:08:07.535 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.520+0000 7f4952d9d640 1 --2- 172.21.15.88:0/660550447 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49541584a0 0x7f4954156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:08:07.535 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.520+0000 7f494bfff640 1 --2- 172.21.15.88:0/660550447 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49540cc5d0 0x7f49541f85e0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f493c00c4d0 tx=0x7f493c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:07.535 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.520+0000 7f494affd640 1 -- 172.21.15.88:0/660550447 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c005cf0 con 0x7f49540cc5d0 2023-10-16T20:08:07.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.520+0000 7f494affd640 1 -- 172.21.15.88:0/660550447 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493c005e50 con 0x7f49540cc5d0 2023-10-16T20:08:07.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.520+0000 7f494affd640 1 -- 172.21.15.88:0/660550447 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c007270 con 0x7f49540cc5d0 2023-10-16T20:08:07.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.520+0000 7f4958df8640 1 -- 172.21.15.88:0/660550447 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49541f8dc0 con 0x7f49540cc5d0 2023-10-16T20:08:07.536 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.520+0000 7f4958df8640 1 -- 172.21.15.88:0/660550447 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49541f9380 con 0x7f49540cc5d0 2023-10-16T20:08:07.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.520+0000 7f4958df8640 1 -- 172.21.15.88:0/660550447 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f495406d180 con 0x7f49540cc5d0 2023-10-16T20:08:07.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.524+0000 7f494affd640 1 -- 172.21.15.88:0/660550447 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f493c0073d0 con 0x7f49540cc5d0 2023-10-16T20:08:07.537 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.524+0000 7f494affd640 1 --2- 172.21.15.88:0/660550447 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f49340664a0 0x7f4934068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:07.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.524+0000 7f494affd640 1 -- 172.21.15.88:0/660550447 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f493c083210 con 0x7f49540cc5d0 2023-10-16T20:08:07.538 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.524+0000 7f495259c640 1 --2- 172.21.15.88:0/660550447 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f49340664a0 0x7f4934068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:07.539 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.524+0000 7f495259c640 1 --2- 172.21.15.88:0/660550447 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f49340664a0 0x7f4934068960 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4944009ac0 tx=0x7f4944009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:07.539 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.528+0000 7f494affd640 1 -- 172.21.15.88:0/660550447 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f493c050860 con 0x7f49540cc5d0 2023-10-16T20:08:07.680 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.676+0000 7f4958df8640 1 -- 172.21.15.88:0/660550447 --> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f495415f1f0 con 0x7f49340664a0 2023-10-16T20:08:07.681 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.676+0000 7f494affd640 1 -- 172.21.15.88:0/660550447 <== mgr.4110 v2:172.21.15.88:6832/4078483333 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f49440119e0 con 0x7f49340664a0 2023-10-16T20:08:07.681 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 5 to deep-scrub 2023-10-16T20:08:07.682 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.676+0000 7f4958df8640 1 -- 172.21.15.88:0/660550447 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f49340664a0 msgr2=0x7f4934068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:07.683 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.676+0000 7f4958df8640 1 --2- 172.21.15.88:0/660550447 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f49340664a0 0x7f4934068960 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4944009ac0 tx=0x7f4944009190 comp rx=0 tx=0).stop 2023-10-16T20:08:07.683 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.676+0000 7f4958df8640 1 -- 172.21.15.88:0/660550447 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49540cc5d0 msgr2=0x7f49541f85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:07.683 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.676+0000 7f4958df8640 1 --2- 172.21.15.88:0/660550447 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49540cc5d0 0x7f49541f85e0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f493c00c4d0 tx=0x7f493c00c990 comp rx=0 tx=0).stop 2023-10-16T20:08:07.684 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.676+0000 7f4958df8640 1 -- 172.21.15.88:0/660550447 shutdown_connections 2023-10-16T20:08:07.684 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.676+0000 7f4958df8640 1 --2- 172.21.15.88:0/660550447 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f49340664a0 0x7f4934068960 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:07.684 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.676+0000 7f4958df8640 1 --2- 172.21.15.88:0/660550447 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f49541584a0 0x7f4954156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:07.684 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.676+0000 7f4958df8640 1 --2- 172.21.15.88:0/660550447 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49540cc5d0 0x7f49541f85e0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:07.685 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.676+0000 7f4958df8640 1 --2- 172.21.15.88:0/660550447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49540b21e0 0x7f49541645a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:07.685 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.676+0000 7f4958df8640 1 -- 172.21.15.88:0/660550447 >> 172.21.15.88:0/660550447 conn(0x7f49540ce5e0 msgr2=0x7f495415b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:07.685 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.676+0000 7f4958df8640 1 -- 172.21.15.88:0/660550447 shutdown_connections 2023-10-16T20:08:07.685 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.676+0000 7f4958df8640 1 -- 172.21.15.88:0/660550447 wait complete. 2023-10-16T20:08:07.701 INFO:tasks.ceph:Scrubbing osd.6 2023-10-16T20:08:07.702 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-16T20:08:07.843 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.836+0000 7fd614bf8640 1 Processor -- start 2023-10-16T20:08:07.844 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.836+0000 7fd614bf8640 1 -- start start 2023-10-16T20:08:07.845 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.840+0000 7fd614bf8640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd61015cb80 0x7fd6100ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:07.845 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.840+0000 7fd614bf8640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6100cb3b0 con 0x7fd6100cdd30 2023-10-16T20:08:07.845 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.840+0000 7fd614bf8640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6100cecf0 con 0x7fd610161400 2023-10-16T20:08:07.846 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.840+0000 7fd614bf8640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6100cefd0 con 0x7fd61015cb80 2023-10-16T20:08:07.846 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.840+0000 7fd60e59c640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd61015cb80 0x7fd6100ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:07.846 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.840+0000 7fd60ed9d640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fd6100cdd30 0x7fd6100cb840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:47286/0 (socket says 172.21.15.88:47286) 2023-10-16T20:08:07.846 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.840+0000 7fd60ed9d640 1 -- 172.21.15.88:0/2299879344 learned_addr learned my addr 172.21.15.88:0/2299879344 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:08:07.847 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.840+0000 7fd60e59c640 1 -- 172.21.15.88:0/2299879344 >> v1:172.21.15.94:6789/0 conn(0x7fd610161400 legacy=0x7fd6100cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:07.848 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.840+0000 7fd60e59c640 1 -- 172.21.15.88:0/2299879344 >> v1:172.21.15.88:6789/0 conn(0x7fd6100cdd30 legacy=0x7fd6100cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:07.848 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.840+0000 7fd60e59c640 1 -- 172.21.15.88:0/2299879344 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6100cf2b0 con 0x7fd61015cb80 2023-10-16T20:08:07.848 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.840+0000 7fd60dd9b640 1 -- 172.21.15.88:0/2299879344 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3324804215 0 0) 0x7fd6100cb3b0 con 0x7fd6100cdd30 2023-10-16T20:08:07.849 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.844+0000 7fd60e59c640 1 --2- 172.21.15.88:0/2299879344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd61015cb80 0x7fd6100ce410 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fd600009d90 tx=0x7fd60001f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2eb896bbf6bc888 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:07.850 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.844+0000 7fd60dd9b640 1 -- 172.21.15.88:0/2299879344 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6000af600 con 0x7fd61015cb80 2023-10-16T20:08:07.850 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.844+0000 7fd60dd9b640 1 -- 172.21.15.88:0/2299879344 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6000ab050 con 0x7fd61015cb80 2023-10-16T20:08:07.850 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.844+0000 7fd60dd9b640 1 -- 172.21.15.88:0/2299879344 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6000af980 con 0x7fd61015cb80 2023-10-16T20:08:07.851 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.844+0000 7fd614bf8640 1 -- 172.21.15.88:0/2299879344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd61015cb80 msgr2=0x7fd6100ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:07.851 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.844+0000 7fd614bf8640 1 --2- 172.21.15.88:0/2299879344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd61015cb80 0x7fd6100ce410 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fd600009d90 tx=0x7fd60001f130 comp rx=0 tx=0).stop 2023-10-16T20:08:07.851 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.844+0000 7fd614bf8640 1 -- 172.21.15.88:0/2299879344 shutdown_connections 2023-10-16T20:08:07.851 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.844+0000 7fd614bf8640 1 --2- 172.21.15.88:0/2299879344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd61015cb80 0x7fd6100ce410 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:07.852 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.844+0000 7fd614bf8640 1 -- 172.21.15.88:0/2299879344 >> 172.21.15.88:0/2299879344 conn(0x7fd61006a060 msgr2=0x7fd61006a440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:07.852 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.844+0000 7fd614bf8640 1 -- 172.21.15.88:0/2299879344 shutdown_connections 2023-10-16T20:08:07.852 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.844+0000 7fd614bf8640 1 -- 172.21.15.88:0/2299879344 wait complete. 2023-10-16T20:08:07.854 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.844+0000 7fd614bf8640 1 Processor -- start 2023-10-16T20:08:07.855 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.844+0000 7fd614bf8640 1 -- start start 2023-10-16T20:08:07.855 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.848+0000 7fd614bf8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6100cdd30 0x7fd61014ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:07.856 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.848+0000 7fd614bf8640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd61015cb80 0x7fd61015e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:07.856 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.848+0000 7fd614bf8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd610161400 0x7fd61015ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:07.856 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.848+0000 7fd614bf8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61015f030 con 0x7fd6100cdd30 2023-10-16T20:08:07.856 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.848+0000 7fd614bf8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61015f310 con 0x7fd610161400 2023-10-16T20:08:07.857 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.848+0000 7fd614bf8640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd61015f5f0 con 0x7fd61015cb80 2023-10-16T20:08:07.857 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.848+0000 7fd60e59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6100cdd30 0x7fd61014ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:07.857 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.848+0000 7fd60e59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6100cdd30 0x7fd61014ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60166/0 (socket says 172.21.15.88:60166) 2023-10-16T20:08:07.858 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.848+0000 7fd60e59c640 1 -- 172.21.15.88:0/3148223261 learned_addr learned my addr 172.21.15.88:0/3148223261 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:07.858 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.848+0000 7fd60ed9d640 1 --2- 172.21.15.88:0/3148223261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd610161400 0x7fd61015ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:07.858 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.848+0000 7fd605d9b640 1 --2- 172.21.15.88:0/3148223261 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd61015cb80 0x7fd61015e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:07.859 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.848+0000 7fd60e59c640 1 -- 172.21.15.88:0/3148223261 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd61015cb80 msgr2=0x7fd61015e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:07.859 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.848+0000 7fd60e59c640 1 --2- 172.21.15.88:0/3148223261 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd61015cb80 0x7fd61015e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:07.860 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.848+0000 7fd60e59c640 1 -- 172.21.15.88:0/3148223261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd610161400 msgr2=0x7fd61015ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:07.860 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.848+0000 7fd60e59c640 1 --2- 172.21.15.88:0/3148223261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd610161400 0x7fd61015ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:07.861 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.848+0000 7fd60e59c640 1 -- 172.21.15.88:0/3148223261 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd600009a70 con 0x7fd6100cdd30 2023-10-16T20:08:07.862 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.848+0000 7fd605d9b640 1 --2- 172.21.15.88:0/3148223261 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd61015cb80 0x7fd61015e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:08:07.863 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.848+0000 7fd60e59c640 1 --2- 172.21.15.88:0/3148223261 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6100cdd30 0x7fd61014ed20 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fd6000b2040 tx=0x7fd60001bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:07.863 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.848+0000 7fd6077fe640 1 -- 172.21.15.88:0/3148223261 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6000b9610 con 0x7fd6100cdd30 2023-10-16T20:08:07.864 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.848+0000 7fd6077fe640 1 -- 172.21.15.88:0/3148223261 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6000afe90 con 0x7fd6100cdd30 2023-10-16T20:08:07.864 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.848+0000 7fd6077fe640 1 -- 172.21.15.88:0/3148223261 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6000b99b0 con 0x7fd6100cdd30 2023-10-16T20:08:07.865 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.848+0000 7fd614bf8640 1 -- 172.21.15.88:0/3148223261 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd61015f8d0 con 0x7fd6100cdd30 2023-10-16T20:08:07.866 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.848+0000 7fd614bf8640 1 -- 172.21.15.88:0/3148223261 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd61015d430 con 0x7fd6100cdd30 2023-10-16T20:08:07.867 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.848+0000 7fd614bf8640 1 -- 172.21.15.88:0/3148223261 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd61015f480 con 0x7fd6100cdd30 2023-10-16T20:08:07.870 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.848+0000 7fd6077fe640 1 -- 172.21.15.88:0/3148223261 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd6000b6050 con 0x7fd6100cdd30 2023-10-16T20:08:07.871 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.852+0000 7fd6077fe640 1 --2- 172.21.15.88:0/3148223261 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd5f00664a0 0x7fd5f0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:07.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.852+0000 7fd6077fe640 1 -- 172.21.15.88:0/3148223261 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fd6001222d0 con 0x7fd6100cdd30 2023-10-16T20:08:07.872 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.852+0000 7fd605d9b640 1 --2- 172.21.15.88:0/3148223261 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd5f00664a0 0x7fd5f0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:07.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.852+0000 7fd6077fe640 1 --2- 172.21.15.88:0/3148223261 >> [v2:172.21.15.94:6816/175389267,v1:172.21.15.94:6817/175389267] conn(0x7fd5f006c7a0 0x7fd5f006ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:07.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.852+0000 7fd6077fe640 1 -- 172.21.15.88:0/3148223261 --> [v2:172.21.15.94:6816/175389267,v1:172.21.15.94:6817/175389267] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd5f006f2b0 con 0x7fd5f006c7a0 2023-10-16T20:08:07.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.852+0000 7fd6077fe640 1 -- 172.21.15.88:0/3148223261 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd600124990 con 0x7fd6100cdd30 2023-10-16T20:08:07.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.852+0000 7fd60ed9d640 1 --2- 172.21.15.88:0/3148223261 >> [v2:172.21.15.94:6816/175389267,v1:172.21.15.94:6817/175389267] conn(0x7fd5f006c7a0 0x7fd5f006ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:07.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.852+0000 7fd605d9b640 1 --2- 172.21.15.88:0/3148223261 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd5f00664a0 0x7fd5f0068960 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd6100c6e50 tx=0x7fd5f8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:07.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.852+0000 7fd60ed9d640 1 --2- 172.21.15.88:0/3148223261 >> [v2:172.21.15.94:6816/175389267,v1:172.21.15.94:6817/175389267] conn(0x7fd5f006c7a0 0x7fd5f006ebe0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:07.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.852+0000 7fd6077fe640 1 -- 172.21.15.88:0/3148223261 <== osd.6 v2:172.21.15.94:6816/175389267 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fd5f006f2b0 con 0x7fd5f006c7a0 2023-10-16T20:08:07.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.864+0000 7fd614bf8640 1 -- 172.21.15.88:0/3148223261 --> [v2:172.21.15.94:6816/175389267,v1:172.21.15.94:6817/175389267] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd61015cf60 con 0x7fd5f006c7a0 2023-10-16T20:08:07.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.864+0000 7fd6077fe640 1 -- 172.21.15.88:0/3148223261 <== osd.6 v2:172.21.15.94:6816/175389267 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fd6080047f0 con 0x7fd5f006c7a0 2023-10-16T20:08:07.878 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2023-10-16T20:08:07.879 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T20:08:07.879 INFO:teuthology.orchestra.run.smithi088.stdout:} 2023-10-16T20:08:07.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.868+0000 7fd614bf8640 1 -- 172.21.15.88:0/3148223261 >> [v2:172.21.15.94:6816/175389267,v1:172.21.15.94:6817/175389267] conn(0x7fd5f006c7a0 msgr2=0x7fd5f006ebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:07.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.868+0000 7fd614bf8640 1 --2- 172.21.15.88:0/3148223261 >> [v2:172.21.15.94:6816/175389267,v1:172.21.15.94:6817/175389267] conn(0x7fd5f006c7a0 0x7fd5f006ebe0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:07.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.868+0000 7fd614bf8640 1 -- 172.21.15.88:0/3148223261 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd5f00664a0 msgr2=0x7fd5f0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:07.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.868+0000 7fd614bf8640 1 --2- 172.21.15.88:0/3148223261 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd5f00664a0 0x7fd5f0068960 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd6100c6e50 tx=0x7fd5f8009040 comp rx=0 tx=0).stop 2023-10-16T20:08:07.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.868+0000 7fd614bf8640 1 -- 172.21.15.88:0/3148223261 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6100cdd30 msgr2=0x7fd61014ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:07.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.868+0000 7fd614bf8640 1 --2- 172.21.15.88:0/3148223261 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6100cdd30 0x7fd61014ed20 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fd6000b2040 tx=0x7fd60001bed0 comp rx=0 tx=0).stop 2023-10-16T20:08:07.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.868+0000 7fd60ed9d640 1 -- 172.21.15.88:0/3148223261 reap_dead start 2023-10-16T20:08:07.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.868+0000 7fd614bf8640 1 -- 172.21.15.88:0/3148223261 shutdown_connections 2023-10-16T20:08:07.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.868+0000 7fd614bf8640 1 -- 172.21.15.88:0/3148223261 >> 172.21.15.88:0/3148223261 conn(0x7fd61006a060 msgr2=0x7fd610160cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:07.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.868+0000 7fd614bf8640 1 -- 172.21.15.88:0/3148223261 shutdown_connections 2023-10-16T20:08:07.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:07.868+0000 7fd614bf8640 1 -- 172.21.15.88:0/3148223261 wait complete. 2023-10-16T20:08:07.890 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-16T20:08:08.029 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.024+0000 7fa8ff59e640 1 Processor -- start 2023-10-16T20:08:08.030 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.024+0000 7fa8ff59e640 1 -- start start 2023-10-16T20:08:08.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.024+0000 7fa8ff59e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fa9000d0e20 0x7fa9000cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:08.031 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.024+0000 7fa8ff59e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa9000d0430 0x7fa9000d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:08.032 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.024+0000 7fa8ff59e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9000d1740 con 0x7fa9000cd370 2023-10-16T20:08:08.032 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.024+0000 7fa8ff59e640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9000d0bd0 con 0x7fa9000d0e20 2023-10-16T20:08:08.032 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.024+0000 7fa8ff59e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9000d08f0 con 0x7fa9000d0430 2023-10-16T20:08:08.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.024+0000 7fa8fe59c640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fa9000d0e20 0x7fa9000cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:08.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.024+0000 7fa8fed9d640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fa9000cd370 0x7fa9000cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.88:46914/0 (socket says 172.21.15.88:46914) 2023-10-16T20:08:08.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.024+0000 7fa8fdd9b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa9000d0430 0x7fa9000d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:08.033 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.024+0000 7fa8fed9d640 1 -- 172.21.15.88:0/2690523779 learned_addr learned my addr 172.21.15.88:0/2690523779 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:08:08.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.024+0000 7fa8fe59c640 1 -- 172.21.15.88:0/2690523779 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa9000d0430 msgr2=0x7fa9000d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:08:08.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.024+0000 7fa8fe59c640 1 --2- 172.21.15.88:0/2690523779 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa9000d0430 0x7fa9000d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:08.034 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.024+0000 7fa8fe59c640 1 -- 172.21.15.88:0/2690523779 >> v1:172.21.15.94:6789/0 conn(0x7fa9000cd370 legacy=0x7fa9000cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:08.035 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.024+0000 7fa8fe59c640 1 -- 172.21.15.88:0/2690523779 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90006b160 con 0x7fa9000d0e20 2023-10-16T20:08:08.036 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.028+0000 7fa8fe59c640 1 --2- 172.21.15.88:0/2690523779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9000d0e20 0x7fa9000cfa80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa8f00097a0 tx=0x7fa8f001c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22cf0132fd7418bf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:08.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.028+0000 7fa8fd59a640 1 -- 172.21.15.88:0/2690523779 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f00a7070 con 0x7fa9000d0e20 2023-10-16T20:08:08.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.028+0000 7fa8fd59a640 1 -- 172.21.15.88:0/2690523779 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f00ab430 con 0x7fa9000d0e20 2023-10-16T20:08:08.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.028+0000 7fa8fd59a640 1 -- 172.21.15.88:0/2690523779 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f00ab7c0 con 0x7fa9000d0e20 2023-10-16T20:08:08.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.028+0000 7fa8ff59e640 1 -- 172.21.15.88:0/2690523779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9000d0e20 msgr2=0x7fa9000cfa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:08.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.028+0000 7fa8ff59e640 1 --2- 172.21.15.88:0/2690523779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9000d0e20 0x7fa9000cfa80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa8f00097a0 tx=0x7fa8f001c430 comp rx=0 tx=0).stop 2023-10-16T20:08:08.038 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.028+0000 7fa8ff59e640 1 -- 172.21.15.88:0/2690523779 shutdown_connections 2023-10-16T20:08:08.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.028+0000 7fa8ff59e640 1 --2- 172.21.15.88:0/2690523779 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa9000d0430 0x7fa9000d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:08.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.028+0000 7fa8ff59e640 1 --2- 172.21.15.88:0/2690523779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9000d0e20 0x7fa9000cfa80 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:08.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.028+0000 7fa8ff59e640 1 -- 172.21.15.88:0/2690523779 >> 172.21.15.88:0/2690523779 conn(0x7fa90009bbf0 msgr2=0x7fa9000d2860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:08.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.028+0000 7fa8ff59e640 1 -- 172.21.15.88:0/2690523779 shutdown_connections 2023-10-16T20:08:08.040 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.028+0000 7fa8ff59e640 1 -- 172.21.15.88:0/2690523779 wait complete. 2023-10-16T20:08:08.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.032+0000 7fa8ff59e640 1 Processor -- start 2023-10-16T20:08:08.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.032+0000 7fa8ff59e640 1 -- start start 2023-10-16T20:08:08.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.032+0000 7fa8ff59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9000cd370 0x7fa9001574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:08.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.032+0000 7fa8ff59e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa9000d0430 0x7fa9001546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:08.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.032+0000 7fa8ff59e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9000d0e20 0x7fa90014b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:08.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.032+0000 7fa8ff59e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa900156a80 con 0x7fa9000d0e20 2023-10-16T20:08:08.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.032+0000 7fa8ff59e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa900154ed0 con 0x7fa9000cd370 2023-10-16T20:08:08.044 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.032+0000 7fa8ff59e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9001551b0 con 0x7fa9000d0430 2023-10-16T20:08:08.044 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.032+0000 7fa8fed9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9000d0e20 0x7fa90014b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:08.044 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.032+0000 7fa8fe59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9000cd370 0x7fa9001574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:08.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.032+0000 7fa8fdd9b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa9000d0430 0x7fa9001546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:08.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.032+0000 7fa8fe59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9000cd370 0x7fa9001574d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.88:55440/0 (socket says 172.21.15.88:55440) 2023-10-16T20:08:08.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.032+0000 7fa8fed9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9000d0e20 0x7fa90014b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60168/0 (socket says 172.21.15.88:60168) 2023-10-16T20:08:08.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.032+0000 7fa8fe59c640 1 -- 172.21.15.88:0/3421016261 learned_addr learned my addr 172.21.15.88:0/3421016261 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:08.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.032+0000 7fa8fe59c640 1 -- 172.21.15.88:0/3421016261 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa9000d0430 msgr2=0x7fa9001546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:08.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.032+0000 7fa8fe59c640 1 --2- 172.21.15.88:0/3421016261 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa9000d0430 0x7fa9001546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:08.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.032+0000 7fa8fe59c640 1 -- 172.21.15.88:0/3421016261 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9000d0e20 msgr2=0x7fa90014b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:08.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.032+0000 7fa8fe59c640 1 --2- 172.21.15.88:0/3421016261 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9000d0e20 0x7fa90014b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:08.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.032+0000 7fa8fe59c640 1 -- 172.21.15.88:0/3421016261 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f00093b0 con 0x7fa9000cd370 2023-10-16T20:08:08.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.032+0000 7fa8fed9d640 1 --2- 172.21.15.88:0/3421016261 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9000d0e20 0x7fa90014b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T20:08:08.047 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.032+0000 7fa8fe59c640 1 --2- 172.21.15.88:0/3421016261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9000cd370 0x7fa9001574d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa8f00ad040 tx=0x7fa8f00191d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:08.048 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.032+0000 7fa8eeffd640 1 -- 172.21.15.88:0/3421016261 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f0005ba0 con 0x7fa9000cd370 2023-10-16T20:08:08.048 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.032+0000 7fa8eeffd640 1 -- 172.21.15.88:0/3421016261 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f0005d20 con 0x7fa9000cd370 2023-10-16T20:08:08.048 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.032+0000 7fa8eeffd640 1 -- 172.21.15.88:0/3421016261 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f0006260 con 0x7fa9000cd370 2023-10-16T20:08:08.048 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.032+0000 7fa8ff59e640 1 -- 172.21.15.88:0/3421016261 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa900156cd0 con 0x7fa9000cd370 2023-10-16T20:08:08.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.036+0000 7fa8ff59e640 1 -- 172.21.15.88:0/3421016261 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa900154be0 con 0x7fa9000cd370 2023-10-16T20:08:08.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.036+0000 7fa8ff59e640 1 -- 172.21.15.88:0/3421016261 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9000674f0 con 0x7fa9000cd370 2023-10-16T20:08:08.049 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.036+0000 7fa8eeffd640 1 -- 172.21.15.88:0/3421016261 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa8f00065d0 con 0x7fa9000cd370 2023-10-16T20:08:08.050 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.036+0000 7fa8eeffd640 1 --2- 172.21.15.88:0/3421016261 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa8e00664a0 0x7fa8e0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:08.051 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.036+0000 7fa8eeffd640 1 -- 172.21.15.88:0/3421016261 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fa8f011d410 con 0x7fa9000cd370 2023-10-16T20:08:08.051 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.036+0000 7fa8fdd9b640 1 --2- 172.21.15.88:0/3421016261 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa8e00664a0 0x7fa8e0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:08.051 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.036+0000 7fa8fdd9b640 1 --2- 172.21.15.88:0/3421016261 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa8e00664a0 0x7fa8e0068960 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa9000b1d80 tx=0x7fa8f4009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:08.052 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.040+0000 7fa8eeffd640 1 -- 172.21.15.88:0/3421016261 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa8f00eaae0 con 0x7fa9000cd370 2023-10-16T20:08:08.199 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.192+0000 7fa8ff59e640 1 -- 172.21.15.88:0/3421016261 --> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fa90015cc90 con 0x7fa8e00664a0 2023-10-16T20:08:08.200 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.192+0000 7fa8eeffd640 1 -- 172.21.15.88:0/3421016261 <== mgr.4110 v2:172.21.15.88:6832/4078483333 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa8f4011a30 con 0x7fa8e00664a0 2023-10-16T20:08:08.200 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 6 to deep-scrub 2023-10-16T20:08:08.201 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.196+0000 7fa8ff59e640 1 -- 172.21.15.88:0/3421016261 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa8e00664a0 msgr2=0x7fa8e0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:08.202 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.196+0000 7fa8ff59e640 1 --2- 172.21.15.88:0/3421016261 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa8e00664a0 0x7fa8e0068960 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa9000b1d80 tx=0x7fa8f4009190 comp rx=0 tx=0).stop 2023-10-16T20:08:08.202 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.196+0000 7fa8ff59e640 1 -- 172.21.15.88:0/3421016261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9000cd370 msgr2=0x7fa9001574d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:08.202 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.196+0000 7fa8ff59e640 1 --2- 172.21.15.88:0/3421016261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9000cd370 0x7fa9001574d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa8f00ad040 tx=0x7fa8f00191d0 comp rx=0 tx=0).stop 2023-10-16T20:08:08.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.196+0000 7fa8ff59e640 1 -- 172.21.15.88:0/3421016261 shutdown_connections 2023-10-16T20:08:08.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.196+0000 7fa8ff59e640 1 --2- 172.21.15.88:0/3421016261 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fa8e00664a0 0x7fa8e0068960 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:08.203 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.196+0000 7fa8ff59e640 1 --2- 172.21.15.88:0/3421016261 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa9000d0430 0x7fa9001546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:08.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.196+0000 7fa8ff59e640 1 --2- 172.21.15.88:0/3421016261 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9000d0e20 0x7fa90014b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:08.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.196+0000 7fa8ff59e640 1 --2- 172.21.15.88:0/3421016261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9000cd370 0x7fa9001574d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:08.204 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.196+0000 7fa8ff59e640 1 -- 172.21.15.88:0/3421016261 >> 172.21.15.88:0/3421016261 conn(0x7fa90009bbf0 msgr2=0x7fa9000b49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:08.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.196+0000 7fa8ff59e640 1 -- 172.21.15.88:0/3421016261 shutdown_connections 2023-10-16T20:08:08.205 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.196+0000 7fa8ff59e640 1 -- 172.21.15.88:0/3421016261 wait complete. 2023-10-16T20:08:08.221 INFO:tasks.ceph:Scrubbing osd.7 2023-10-16T20:08:08.221 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-16T20:08:08.364 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.360+0000 7f6622a88640 1 Processor -- start 2023-10-16T20:08:08.364 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.360+0000 7f6622a88640 1 -- start start 2023-10-16T20:08:08.365 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.360+0000 7f6622a88640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f661c06cb90 0x7f661c0d33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:08.365 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.360+0000 7f6622a88640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f661c06b6f0 con 0x7f661c15cea0 2023-10-16T20:08:08.366 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.360+0000 7f6622a88640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f661c0ce370 con 0x7f661c15cac0 2023-10-16T20:08:08.366 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.360+0000 7f6622a88640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f661c0ce180 con 0x7f661c06cb90 2023-10-16T20:08:08.366 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.360+0000 7f6621285640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f661c15cea0 0x7f661c0cda70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:47298/0 (socket says 172.21.15.88:47298) 2023-10-16T20:08:08.367 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.360+0000 7f6622287640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f661c06cb90 0x7f661c0d33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:08.367 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.360+0000 7f6621285640 1 -- 172.21.15.88:0/1690003044 learned_addr learned my addr 172.21.15.88:0/1690003044 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:08:08.368 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.360+0000 7f6622287640 1 -- 172.21.15.88:0/1690003044 >> v1:172.21.15.94:6789/0 conn(0x7f661c15cac0 legacy=0x7f661c0cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:08.368 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.360+0000 7f6622287640 1 -- 172.21.15.88:0/1690003044 >> v1:172.21.15.88:6789/0 conn(0x7f661c15cea0 legacy=0x7f661c0cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:08.368 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.360+0000 7f6622287640 1 -- 172.21.15.88:0/1690003044 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661c0ce650 con 0x7f661c06cb90 2023-10-16T20:08:08.369 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.360+0000 7f6620a84640 1 -- 172.21.15.88:0/1690003044 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2190681219 0 0) 0x7f661c06b6f0 con 0x7f661c15cea0 2023-10-16T20:08:08.370 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.364+0000 7f6622287640 1 --2- 172.21.15.88:0/1690003044 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f661c06cb90 0x7f661c0d33e0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f6614009d90 tx=0x7f661401c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6290a6b083b267 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:08.370 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.364+0000 7f6620a84640 1 -- 172.21.15.88:0/1690003044 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66140a5950 con 0x7f661c06cb90 2023-10-16T20:08:08.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.364+0000 7f6620a84640 1 -- 172.21.15.88:0/1690003044 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66140a5ab0 con 0x7f661c06cb90 2023-10-16T20:08:08.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.364+0000 7f6620a84640 1 -- 172.21.15.88:0/1690003044 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66140a5e40 con 0x7f661c06cb90 2023-10-16T20:08:08.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.364+0000 7f6622a88640 1 -- 172.21.15.88:0/1690003044 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f661c06cb90 msgr2=0x7f661c0d33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:08.371 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.364+0000 7f6622a88640 1 --2- 172.21.15.88:0/1690003044 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f661c06cb90 0x7f661c0d33e0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f6614009d90 tx=0x7f661401c890 comp rx=0 tx=0).stop 2023-10-16T20:08:08.372 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.364+0000 7f6622a88640 1 -- 172.21.15.88:0/1690003044 shutdown_connections 2023-10-16T20:08:08.372 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.364+0000 7f6622a88640 1 --2- 172.21.15.88:0/1690003044 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f661c06cb90 0x7f661c0d33e0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:08.372 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.364+0000 7f6622a88640 1 -- 172.21.15.88:0/1690003044 >> 172.21.15.88:0/1690003044 conn(0x7f661c0d28c0 msgr2=0x7f661c0d2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:08.373 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.364+0000 7f6622a88640 1 -- 172.21.15.88:0/1690003044 shutdown_connections 2023-10-16T20:08:08.373 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.364+0000 7f6622a88640 1 -- 172.21.15.88:0/1690003044 wait complete. 2023-10-16T20:08:08.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.364+0000 7f6622a88640 1 Processor -- start 2023-10-16T20:08:08.380 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.364+0000 7f6622a88640 1 -- start start 2023-10-16T20:08:08.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.368+0000 7f6622a88640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f661c06cb90 0x7f661c1f6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:08.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.368+0000 7f6622a88640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f661c15cac0 0x7f661c0b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:08.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.368+0000 7f6622a88640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f661c15cea0 0x7f661c15dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:08.382 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.368+0000 7f6622a88640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661c15dbb0 con 0x7f661c15cac0 2023-10-16T20:08:08.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.368+0000 7f6622a88640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661c15d2a0 con 0x7f661c15cea0 2023-10-16T20:08:08.383 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.368+0000 7f6622a88640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f661c0b0de0 con 0x7f661c06cb90 2023-10-16T20:08:08.384 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.368+0000 7f6621285640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f661c15cac0 0x7f661c0b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:08.384 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.368+0000 7f6621285640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f661c15cac0 0x7f661c0b08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60192/0 (socket says 172.21.15.88:60192) 2023-10-16T20:08:08.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.368+0000 7f6621285640 1 -- 172.21.15.88:0/1099949881 learned_addr learned my addr 172.21.15.88:0/1099949881 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:08.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.368+0000 7f6621a86640 1 --2- 172.21.15.88:0/1099949881 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f661c06cb90 0x7f661c1f6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:08.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.368+0000 7f6622287640 1 --2- 172.21.15.88:0/1099949881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f661c15cea0 0x7f661c15dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:08.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.368+0000 7f6621285640 1 -- 172.21.15.88:0/1099949881 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f661c06cb90 msgr2=0x7f661c1f6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:08.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.368+0000 7f6621285640 1 --2- 172.21.15.88:0/1099949881 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f661c06cb90 0x7f661c1f6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:08.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.368+0000 7f6621285640 1 -- 172.21.15.88:0/1099949881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f661c15cea0 msgr2=0x7f661c15dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:08.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.368+0000 7f6621285640 1 --2- 172.21.15.88:0/1099949881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f661c15cea0 0x7f661c15dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:08.388 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.368+0000 7f6621285640 1 -- 172.21.15.88:0/1099949881 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6614009a70 con 0x7f661c15cac0 2023-10-16T20:08:08.388 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.368+0000 7f6621285640 1 --2- 172.21.15.88:0/1099949881 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f661c15cac0 0x7f661c0b08a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f660c00c440 tx=0x7f660c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:08.389 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.368+0000 7f66127fc640 1 -- 172.21.15.88:0/1099949881 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c0074e0 con 0x7f661c15cac0 2023-10-16T20:08:08.389 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.368+0000 7f66127fc640 1 -- 172.21.15.88:0/1099949881 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f660c007640 con 0x7f661c15cac0 2023-10-16T20:08:08.390 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.368+0000 7f6622a88640 1 -- 172.21.15.88:0/1099949881 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f661c153e00 con 0x7f661c15cac0 2023-10-16T20:08:08.390 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.368+0000 7f66127fc640 1 -- 172.21.15.88:0/1099949881 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c004a10 con 0x7f661c15cac0 2023-10-16T20:08:08.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.368+0000 7f6622a88640 1 -- 172.21.15.88:0/1099949881 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f661c15d7d0 con 0x7f661c15cac0 2023-10-16T20:08:08.391 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.368+0000 7f6622a88640 1 -- 172.21.15.88:0/1099949881 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f661c16a410 con 0x7f661c15cac0 2023-10-16T20:08:08.392 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.372+0000 7f66127fc640 1 -- 172.21.15.88:0/1099949881 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f660c01c3b0 con 0x7f661c15cac0 2023-10-16T20:08:08.392 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.372+0000 7f66127fc640 1 --2- 172.21.15.88:0/1099949881 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f66000664d0 0x7f6600068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:08.393 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.372+0000 7f66127fc640 1 -- 172.21.15.88:0/1099949881 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f660c083900 con 0x7f661c15cac0 2023-10-16T20:08:08.393 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.372+0000 7f6621a86640 1 --2- 172.21.15.88:0/1099949881 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f66000664d0 0x7f6600068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:08.393 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.372+0000 7f66127fc640 1 --2- 172.21.15.88:0/1099949881 >> [v2:172.21.15.94:6824/1057220797,v1:172.21.15.94:6825/1057220797] conn(0x7f660006c7d0 0x7f660006ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:08.394 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.372+0000 7f66127fc640 1 -- 172.21.15.88:0/1099949881 --> [v2:172.21.15.94:6824/1057220797,v1:172.21.15.94:6825/1057220797] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f660006f2e0 con 0x7f660006c7d0 2023-10-16T20:08:08.394 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.372+0000 7f66127fc640 1 -- 172.21.15.88:0/1099949881 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f660c083c70 con 0x7f661c15cac0 2023-10-16T20:08:08.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.372+0000 7f6622287640 1 --2- 172.21.15.88:0/1099949881 >> [v2:172.21.15.94:6824/1057220797,v1:172.21.15.94:6825/1057220797] conn(0x7f660006c7d0 0x7f660006ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:08.395 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.372+0000 7f6621a86640 1 --2- 172.21.15.88:0/1099949881 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f66000664d0 0x7f6600068990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6618007fd0 tx=0x7f6618009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:08.396 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.372+0000 7f6622287640 1 --2- 172.21.15.88:0/1099949881 >> [v2:172.21.15.94:6824/1057220797,v1:172.21.15.94:6825/1057220797] conn(0x7f660006c7d0 0x7f660006ec10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:08.396 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.372+0000 7f66127fc640 1 -- 172.21.15.88:0/1099949881 <== osd.7 v2:172.21.15.94:6824/1057220797 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f660006f2e0 con 0x7f660006c7d0 2023-10-16T20:08:08.397 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.384+0000 7f6622a88640 1 -- 172.21.15.88:0/1099949881 --> [v2:172.21.15.94:6824/1057220797,v1:172.21.15.94:6825/1057220797] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f661c0674f0 con 0x7f660006c7d0 2023-10-16T20:08:08.398 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.388+0000 7f66127fc640 1 -- 172.21.15.88:0/1099949881 <== osd.7 v2:172.21.15.94:6824/1057220797 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f661c0674f0 con 0x7f660006c7d0 2023-10-16T20:08:08.399 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2023-10-16T20:08:08.399 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T20:08:08.399 INFO:teuthology.orchestra.run.smithi088.stdout:} 2023-10-16T20:08:08.400 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.388+0000 7f6622a88640 1 -- 172.21.15.88:0/1099949881 >> [v2:172.21.15.94:6824/1057220797,v1:172.21.15.94:6825/1057220797] conn(0x7f660006c7d0 msgr2=0x7f660006ec10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:08.401 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.388+0000 7f6622a88640 1 --2- 172.21.15.88:0/1099949881 >> [v2:172.21.15.94:6824/1057220797,v1:172.21.15.94:6825/1057220797] conn(0x7f660006c7d0 0x7f660006ec10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:08.401 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.388+0000 7f6622a88640 1 -- 172.21.15.88:0/1099949881 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f66000664d0 msgr2=0x7f6600068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:08.401 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.388+0000 7f6622a88640 1 --2- 172.21.15.88:0/1099949881 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f66000664d0 0x7f6600068990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6618007fd0 tx=0x7f6618009040 comp rx=0 tx=0).stop 2023-10-16T20:08:08.401 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.388+0000 7f6622a88640 1 -- 172.21.15.88:0/1099949881 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f661c15cac0 msgr2=0x7f661c0b08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:08.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.388+0000 7f6622a88640 1 --2- 172.21.15.88:0/1099949881 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f661c15cac0 0x7f661c0b08a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f660c00c440 tx=0x7f660c00c900 comp rx=0 tx=0).stop 2023-10-16T20:08:08.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.388+0000 7f6622287640 1 -- 172.21.15.88:0/1099949881 reap_dead start 2023-10-16T20:08:08.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.388+0000 7f6622a88640 1 -- 172.21.15.88:0/1099949881 shutdown_connections 2023-10-16T20:08:08.402 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.388+0000 7f6622a88640 1 -- 172.21.15.88:0/1099949881 >> 172.21.15.88:0/1099949881 conn(0x7f661c0d28c0 msgr2=0x7f661c0c8f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:08.403 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.388+0000 7f6622a88640 1 -- 172.21.15.88:0/1099949881 shutdown_connections 2023-10-16T20:08:08.403 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.388+0000 7f6622a88640 1 -- 172.21.15.88:0/1099949881 wait complete. 2023-10-16T20:08:08.410 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-16T20:08:08.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.544+0000 7f5007ad6640 1 Processor -- start 2023-10-16T20:08:08.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.544+0000 7f5007ad6640 1 -- start start 2023-10-16T20:08:08.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.544+0000 7f5007ad6640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f50000cfaa0 0x7f5000161330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:08.552 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.544+0000 7f5007ad6640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f500015cde0 0x7f500015dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:08.552 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.544+0000 7f5007ad6640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f500015e210 con 0x7f500015ca00 2023-10-16T20:08:08.552 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.544+0000 7f5007ad6640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50001608c0 con 0x7f50000cfaa0 2023-10-16T20:08:08.552 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.544+0000 7f5007ad6640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50000cdfa0 con 0x7f500015cde0 2023-10-16T20:08:08.553 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.544+0000 7f5005071640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f50000cfaa0 0x7f5000161330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:08.553 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.544+0000 7f5005071640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f50000cfaa0 0x7f5000161330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60204/0 (socket says 172.21.15.88:60204) 2023-10-16T20:08:08.553 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.544+0000 7f5005071640 1 -- 172.21.15.88:0/2348755525 learned_addr learned my addr 172.21.15.88:0/2348755525 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:08.553 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.544+0000 7f5006073640 1 --2- 172.21.15.88:0/2348755525 >> v2:172.21.15.94:3300/0 conn(0x7f500015cde0 0x7f500015dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:08.554 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.544+0000 7f5005071640 1 -- 172.21.15.88:0/2348755525 >> v2:172.21.15.94:3300/0 conn(0x7f500015cde0 msgr2=0x7f500015dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:08:08.554 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.544+0000 7f5005071640 1 --2- 172.21.15.88:0/2348755525 >> v2:172.21.15.94:3300/0 conn(0x7f500015cde0 0x7f500015dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:08.554 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.544+0000 7f5005071640 1 -- 172.21.15.88:0/2348755525 >> v1:172.21.15.94:6789/0 conn(0x7f500015ca00 legacy=0x7f50000cf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:08.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.544+0000 7f5005071640 1 -- 172.21.15.88:0/2348755525 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5000160b30 con 0x7f50000cfaa0 2023-10-16T20:08:08.556 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.548+0000 7f5005071640 1 --2- 172.21.15.88:0/2348755525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50000cfaa0 0x7f5000161330 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f50000b5b80 tx=0x7f4ff001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f3caa1412ba0a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:08.556 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.548+0000 7f5004870640 1 -- 172.21.15.88:0/2348755525 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff00ab660 con 0x7f50000cfaa0 2023-10-16T20:08:08.557 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.548+0000 7f5004870640 1 -- 172.21.15.88:0/2348755525 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff00ab7c0 con 0x7f50000cfaa0 2023-10-16T20:08:08.557 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.548+0000 7f5004870640 1 -- 172.21.15.88:0/2348755525 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff00abb50 con 0x7f50000cfaa0 2023-10-16T20:08:08.557 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.548+0000 7f5007ad6640 1 -- 172.21.15.88:0/2348755525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50000cfaa0 msgr2=0x7f5000161330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:08.558 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.548+0000 7f5007ad6640 1 --2- 172.21.15.88:0/2348755525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50000cfaa0 0x7f5000161330 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f50000b5b80 tx=0x7f4ff001c440 comp rx=0 tx=0).stop 2023-10-16T20:08:08.558 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.548+0000 7f5007ad6640 1 -- 172.21.15.88:0/2348755525 shutdown_connections 2023-10-16T20:08:08.558 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.548+0000 7f5007ad6640 1 --2- 172.21.15.88:0/2348755525 >> v2:172.21.15.94:3300/0 conn(0x7f500015cde0 0x7f500015dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:08.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.548+0000 7f5007ad6640 1 --2- 172.21.15.88:0/2348755525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50000cfaa0 0x7f5000161330 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:08.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.548+0000 7f5007ad6640 1 -- 172.21.15.88:0/2348755525 >> 172.21.15.88:0/2348755525 conn(0x7f50000715c0 msgr2=0x7f50000719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:08.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.548+0000 7f5007ad6640 1 -- 172.21.15.88:0/2348755525 shutdown_connections 2023-10-16T20:08:08.560 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.548+0000 7f5007ad6640 1 -- 172.21.15.88:0/2348755525 wait complete. 2023-10-16T20:08:08.562 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.552+0000 7f5007ad6640 1 Processor -- start 2023-10-16T20:08:08.563 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.552+0000 7f5007ad6640 1 -- start start 2023-10-16T20:08:08.563 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.552+0000 7f5007ad6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50000cfaa0 0x7f50000b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:08.563 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.552+0000 7f5007ad6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f500015ca00 0x7f50000c6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:08.564 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.552+0000 7f5007ad6640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f500015cde0 0x7f5000155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:08.564 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.552+0000 7f5007ad6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5000156740 con 0x7f500015ca00 2023-10-16T20:08:08.564 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.552+0000 7f5007ad6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50000c71a0 con 0x7f50000cfaa0 2023-10-16T20:08:08.564 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.552+0000 7f5007ad6640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50000cccb0 con 0x7f500015cde0 2023-10-16T20:08:08.565 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.552+0000 7f5005071640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f500015ca00 0x7f50000c6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:08.565 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.552+0000 7f5005071640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f500015ca00 0x7f50000c6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60218/0 (socket says 172.21.15.88:60218) 2023-10-16T20:08:08.565 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.552+0000 7f5005071640 1 -- 172.21.15.88:0/3892351592 learned_addr learned my addr 172.21.15.88:0/3892351592 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:08.566 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.552+0000 7f5006073640 1 --2- 172.21.15.88:0/3892351592 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f500015cde0 0x7f5000155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:08.566 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.552+0000 7f5005872640 1 --2- 172.21.15.88:0/3892351592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50000cfaa0 0x7f50000b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:08.566 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.552+0000 7f5005071640 1 -- 172.21.15.88:0/3892351592 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f500015cde0 msgr2=0x7f5000155f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:08.567 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.552+0000 7f5005071640 1 --2- 172.21.15.88:0/3892351592 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f500015cde0 0x7f5000155f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:08.567 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.552+0000 7f5005071640 1 -- 172.21.15.88:0/3892351592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50000cfaa0 msgr2=0x7f50000b5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:08.567 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.552+0000 7f5005071640 1 --2- 172.21.15.88:0/3892351592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50000cfaa0 0x7f50000b5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:08.567 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.552+0000 7f5005071640 1 -- 172.21.15.88:0/3892351592 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff00094d0 con 0x7f500015ca00 2023-10-16T20:08:08.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.552+0000 7f5005071640 1 --2- 172.21.15.88:0/3892351592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f500015ca00 0x7f50000c6c60 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0019050 tx=0x7f4ff001ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:08.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.552+0000 7f4ff67fc640 1 -- 172.21.15.88:0/3892351592 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff00b4800 con 0x7f500015ca00 2023-10-16T20:08:08.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.552+0000 7f4ff67fc640 1 -- 172.21.15.88:0/3892351592 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff00b4960 con 0x7f500015ca00 2023-10-16T20:08:08.568 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.552+0000 7f4ff67fc640 1 -- 172.21.15.88:0/3892351592 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff00b4d10 con 0x7f500015ca00 2023-10-16T20:08:08.569 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.552+0000 7f5007ad6640 1 -- 172.21.15.88:0/3892351592 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50000ccf90 con 0x7f500015ca00 2023-10-16T20:08:08.569 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.552+0000 7f5007ad6640 1 -- 172.21.15.88:0/3892351592 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5000156990 con 0x7f500015ca00 2023-10-16T20:08:08.569 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.556+0000 7f5007ad6640 1 -- 172.21.15.88:0/3892351592 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f500004e9c0 con 0x7f500015ca00 2023-10-16T20:08:08.570 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.556+0000 7f4ff67fc640 1 -- 172.21.15.88:0/3892351592 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4ff00a9070 con 0x7f500015ca00 2023-10-16T20:08:08.570 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.556+0000 7f4ff67fc640 1 --2- 172.21.15.88:0/3892351592 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4fe40664d0 0x7f4fe4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:08.570 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.556+0000 7f4ff67fc640 1 -- 172.21.15.88:0/3892351592 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f4ff011e560 con 0x7f500015ca00 2023-10-16T20:08:08.571 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.556+0000 7f5005872640 1 --2- 172.21.15.88:0/3892351592 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4fe40664d0 0x7f4fe4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:08.571 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.556+0000 7f5005872640 1 --2- 172.21.15.88:0/3892351592 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4fe40664d0 0x7f4fe4068990 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc007fd0 tx=0x7f4ffc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:08.572 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.560+0000 7f4ff67fc640 1 -- 172.21.15.88:0/3892351592 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4ff00ebd60 con 0x7f500015ca00 2023-10-16T20:08:08.715 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.708+0000 7f5007ad6640 1 -- 172.21.15.88:0/3892351592 --> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f50000ce280 con 0x7f4fe40664d0 2023-10-16T20:08:08.715 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.708+0000 7f4ff67fc640 1 -- 172.21.15.88:0/3892351592 <== mgr.4110 v2:172.21.15.88:6832/4078483333 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4ffc00f820 con 0x7f4fe40664d0 2023-10-16T20:08:08.716 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 7 to deep-scrub 2023-10-16T20:08:08.717 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.712+0000 7f5007ad6640 1 -- 172.21.15.88:0/3892351592 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4fe40664d0 msgr2=0x7f4fe4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:08.717 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.712+0000 7f5007ad6640 1 --2- 172.21.15.88:0/3892351592 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4fe40664d0 0x7f4fe4068990 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc007fd0 tx=0x7f4ffc009040 comp rx=0 tx=0).stop 2023-10-16T20:08:08.718 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.712+0000 7f5007ad6640 1 -- 172.21.15.88:0/3892351592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f500015ca00 msgr2=0x7f50000c6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:08.718 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.712+0000 7f5007ad6640 1 --2- 172.21.15.88:0/3892351592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f500015ca00 0x7f50000c6c60 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0019050 tx=0x7f4ff001ac80 comp rx=0 tx=0).stop 2023-10-16T20:08:08.719 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.712+0000 7f5007ad6640 1 -- 172.21.15.88:0/3892351592 shutdown_connections 2023-10-16T20:08:08.719 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.712+0000 7f5007ad6640 1 --2- 172.21.15.88:0/3892351592 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f4fe40664d0 0x7f4fe4068990 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:08.719 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.712+0000 7f5007ad6640 1 --2- 172.21.15.88:0/3892351592 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f500015cde0 0x7f5000155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:08.720 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.712+0000 7f5007ad6640 1 --2- 172.21.15.88:0/3892351592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f500015ca00 0x7f50000c6c60 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:08.720 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.712+0000 7f5007ad6640 1 --2- 172.21.15.88:0/3892351592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50000cfaa0 0x7f50000b5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:08.720 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.712+0000 7f5007ad6640 1 -- 172.21.15.88:0/3892351592 >> 172.21.15.88:0/3892351592 conn(0x7f50000715c0 msgr2=0x7f50000d1620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:08.721 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.712+0000 7f5007ad6640 1 -- 172.21.15.88:0/3892351592 shutdown_connections 2023-10-16T20:08:08.721 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.712+0000 7f5007ad6640 1 -- 172.21.15.88:0/3892351592 wait complete. 2023-10-16T20:08:08.736 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T20:08:08.873 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.868+0000 7f81223d9640 1 Processor -- start 2023-10-16T20:08:08.874 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.868+0000 7f81223d9640 1 -- start start 2023-10-16T20:08:08.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.868+0000 7f81223d9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f811c15ca30 0x7f811c160880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:08.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.868+0000 7f81223d9640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f811c0b8530 0x7f811c0cd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:08.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.868+0000 7f81223d9640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f811c0d0d00 con 0x7f811c15ce10 2023-10-16T20:08:08.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.868+0000 7f81223d9640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f811c160dc0 con 0x7f811c0b8530 2023-10-16T20:08:08.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.868+0000 7f81223d9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811c161040 con 0x7f811c15ca30 2023-10-16T20:08:08.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.868+0000 7f8120976640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f811c15ce10 0x7f811c0b4620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:47314/0 (socket says 172.21.15.88:47314) 2023-10-16T20:08:08.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.868+0000 7f8120976640 1 -- 172.21.15.88:0/1681599998 learned_addr learned my addr 172.21.15.88:0/1681599998 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:08:08.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.868+0000 7f811b7fe640 1 --2- 172.21.15.88:0/1681599998 >> v2:172.21.15.88:3300/0 conn(0x7f811c0b8530 0x7f811c0cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:08.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.868+0000 7f811bfff640 1 --2- 172.21.15.88:0/1681599998 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f811c15ca30 0x7f811c160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:08.878 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.868+0000 7f811b7fe640 1 -- 172.21.15.88:0/1681599998 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f811c15ca30 msgr2=0x7f811c160880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:08:08.878 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.868+0000 7f811b7fe640 1 --2- 172.21.15.88:0/1681599998 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f811c15ca30 0x7f811c160880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:08.878 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.868+0000 7f811b7fe640 1 -- 172.21.15.88:0/1681599998 >> v1:172.21.15.88:6789/0 conn(0x7f811c15ce10 legacy=0x7f811c0b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:08.878 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.868+0000 7f811b7fe640 1 -- 172.21.15.88:0/1681599998 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811c161320 con 0x7f811c0b8530 2023-10-16T20:08:08.879 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.868+0000 7f811affd640 1 -- 172.21.15.88:0/1681599998 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2796165238 0 0) 0x7f811c0d0d00 con 0x7f811c15ce10 2023-10-16T20:08:08.879 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.868+0000 7f811bfff640 1 --2- 172.21.15.88:0/1681599998 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f811c15ca30 0x7f811c160880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T20:08:08.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.872+0000 7f811b7fe640 1 --2- 172.21.15.88:0/1681599998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f811c0b8530 0x7f811c0cd870 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f81100097c0 tx=0x7f811001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327df4950c91a929 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:08.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.872+0000 7f811affd640 1 -- 172.21.15.88:0/1681599998 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81100ab650 con 0x7f811c0b8530 2023-10-16T20:08:08.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.872+0000 7f811affd640 1 -- 172.21.15.88:0/1681599998 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81100ab7b0 con 0x7f811c0b8530 2023-10-16T20:08:08.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.872+0000 7f811affd640 1 -- 172.21.15.88:0/1681599998 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81100abb40 con 0x7f811c0b8530 2023-10-16T20:08:08.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.872+0000 7f81223d9640 1 -- 172.21.15.88:0/1681599998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f811c0b8530 msgr2=0x7f811c0cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:08.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.872+0000 7f81223d9640 1 --2- 172.21.15.88:0/1681599998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f811c0b8530 0x7f811c0cd870 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f81100097c0 tx=0x7f811001c450 comp rx=0 tx=0).stop 2023-10-16T20:08:08.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.872+0000 7f81223d9640 1 -- 172.21.15.88:0/1681599998 shutdown_connections 2023-10-16T20:08:08.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.872+0000 7f81223d9640 1 --2- 172.21.15.88:0/1681599998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f811c0b8530 0x7f811c0cd870 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:08.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.872+0000 7f81223d9640 1 --2- 172.21.15.88:0/1681599998 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f811c15ca30 0x7f811c160880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:08.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.872+0000 7f81223d9640 1 -- 172.21.15.88:0/1681599998 >> 172.21.15.88:0/1681599998 conn(0x7f811c0715c0 msgr2=0x7f811c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:08.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.872+0000 7f81223d9640 1 -- 172.21.15.88:0/1681599998 shutdown_connections 2023-10-16T20:08:08.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.872+0000 7f81223d9640 1 -- 172.21.15.88:0/1681599998 wait complete. 2023-10-16T20:08:08.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.876+0000 7f81223d9640 1 Processor -- start 2023-10-16T20:08:08.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.876+0000 7f81223d9640 1 -- start start 2023-10-16T20:08:08.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.876+0000 7f81223d9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f811c0b8530 0x7f811c150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:08.886 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.876+0000 7f81223d9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f811c15ca30 0x7f811c0cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:08.887 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.876+0000 7f81223d9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f811c15ce10 0x7f811c14d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:08.887 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.876+0000 7f81223d9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811c0cb300 con 0x7f811c0b8530 2023-10-16T20:08:08.887 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.876+0000 7f81223d9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811c0cb5e0 con 0x7f811c15ce10 2023-10-16T20:08:08.887 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.876+0000 7f81223d9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811c0cb8c0 con 0x7f811c15ca30 2023-10-16T20:08:08.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.876+0000 7f811bfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f811c0b8530 0x7f811c150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:08.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.876+0000 7f811bfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f811c0b8530 0x7f811c150760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60232/0 (socket says 172.21.15.88:60232) 2023-10-16T20:08:08.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.876+0000 7f811bfff640 1 -- 172.21.15.88:0/3824124175 learned_addr learned my addr 172.21.15.88:0/3824124175 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:08.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.876+0000 7f811bfff640 1 -- 172.21.15.88:0/3824124175 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f811c15ca30 msgr2=0x7f811c0cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:08.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.876+0000 7f811bfff640 1 --2- 172.21.15.88:0/3824124175 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f811c15ca30 0x7f811c0cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:08.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.876+0000 7f811bfff640 1 -- 172.21.15.88:0/3824124175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f811c15ce10 msgr2=0x7f811c14d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:08.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.876+0000 7f811bfff640 1 --2- 172.21.15.88:0/3824124175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f811c15ce10 0x7f811c14d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:08.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.876+0000 7f811bfff640 1 -- 172.21.15.88:0/3824124175 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8110009420 con 0x7f811c0b8530 2023-10-16T20:08:08.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.876+0000 7f811bfff640 1 --2- 172.21.15.88:0/3824124175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f811c0b8530 0x7f811c150760 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f810c00d800 tx=0x7f810c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:08.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.876+0000 7f8118ff9640 1 -- 172.21.15.88:0/3824124175 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c0096c0 con 0x7f811c0b8530 2023-10-16T20:08:08.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.876+0000 7f8118ff9640 1 -- 172.21.15.88:0/3824124175 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f810c009820 con 0x7f811c0b8530 2023-10-16T20:08:08.891 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.876+0000 7f81223d9640 1 -- 172.21.15.88:0/3824124175 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f811c0cbba0 con 0x7f811c0b8530 2023-10-16T20:08:08.891 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.876+0000 7f8118ff9640 1 -- 172.21.15.88:0/3824124175 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c009bd0 con 0x7f811c0b8530 2023-10-16T20:08:08.891 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.876+0000 7f81223d9640 1 -- 172.21.15.88:0/3824124175 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f811c14e100 con 0x7f811c0b8530 2023-10-16T20:08:08.892 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.880+0000 7f81223d9640 1 -- 172.21.15.88:0/3824124175 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f811c04e9c0 con 0x7f811c0b8530 2023-10-16T20:08:08.892 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.880+0000 7f8118ff9640 1 -- 172.21.15.88:0/3824124175 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f810c009d30 con 0x7f811c0b8530 2023-10-16T20:08:08.892 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.880+0000 7f8118ff9640 1 --2- 172.21.15.88:0/3824124175 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f81080664d0 0x7f8108068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:08.892 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.880+0000 7f8118ff9640 1 -- 172.21.15.88:0/3824124175 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f810c084380 con 0x7f811c0b8530 2023-10-16T20:08:08.893 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.880+0000 7f811b7fe640 1 --2- 172.21.15.88:0/3824124175 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f81080664d0 0x7f8108068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:08.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.880+0000 7f811b7fe640 1 --2- 172.21.15.88:0/3824124175 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f81080664d0 0x7f8108068990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f81100af000 tx=0x7f8110008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:08.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:08.884+0000 7f8118ff9640 1 -- 172.21.15.88:0/3824124175 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f810c0519b0 con 0x7f811c0b8530 2023-10-16T20:08:09.037 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:09.032+0000 7f81223d9640 1 -- 172.21.15.88:0/3824124175 --> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f811c0ce1c0 con 0x7f81080664d0 2023-10-16T20:08:09.039 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:09.036+0000 7f8118ff9640 1 -- 172.21.15.88:0/3824124175 <== mgr.4110 v2:172.21.15.88:6832/4078483333 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50994 (secure 0 0 0) 0x7f81100ab5b0 con 0x7f81080664d0 2023-10-16T20:08:09.040 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:08:09.041 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2023-10-16T20:08:09.042 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:09.036+0000 7f81223d9640 1 -- 172.21.15.88:0/3824124175 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f81080664d0 msgr2=0x7f8108068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:09.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:09.036+0000 7f81223d9640 1 --2- 172.21.15.88:0/3824124175 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f81080664d0 0x7f8108068990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f81100af000 tx=0x7f8110008040 comp rx=0 tx=0).stop 2023-10-16T20:08:09.043 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:09.036+0000 7f81223d9640 1 -- 172.21.15.88:0/3824124175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f811c0b8530 msgr2=0x7f811c150760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:09.044 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:09.036+0000 7f81223d9640 1 --2- 172.21.15.88:0/3824124175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f811c0b8530 0x7f811c150760 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f810c00d800 tx=0x7f810c00dcc0 comp rx=0 tx=0).stop 2023-10-16T20:08:09.044 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:09.036+0000 7f81223d9640 1 -- 172.21.15.88:0/3824124175 shutdown_connections 2023-10-16T20:08:09.044 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:09.036+0000 7f81223d9640 1 --2- 172.21.15.88:0/3824124175 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f81080664d0 0x7f8108068990 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:09.044 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:09.036+0000 7f81223d9640 1 --2- 172.21.15.88:0/3824124175 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f811c15ca30 0x7f811c0cacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:09.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:09.036+0000 7f81223d9640 1 --2- 172.21.15.88:0/3824124175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f811c15ce10 0x7f811c14d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:09.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:09.036+0000 7f81223d9640 1 --2- 172.21.15.88:0/3824124175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f811c0b8530 0x7f811c150760 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:09.045 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:09.036+0000 7f81223d9640 1 -- 172.21.15.88:0/3824124175 >> 172.21.15.88:0/3824124175 conn(0x7f811c0715c0 msgr2=0x7f811c15bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:09.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:09.036+0000 7f81223d9640 1 -- 172.21.15.88:0/3824124175 shutdown_connections 2023-10-16T20:08:09.046 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:09.036+0000 7f81223d9640 1 -- 172.21.15.88:0/3824124175 wait complete. 2023-10-16T20:08:09.061 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2023-10-16T20:08:08.486003+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":239704,"kb_used_data":6888,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735016,"statfs":{"total":773094113280,"available":772848656384,"internally_reserved":0,"allocated":7053312,"data_stored":3795016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238276153},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.209518"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":200,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-16T20:07:59.682136+0000","last_change":"2023-10-16T20:04:46.335822+0000","last_active":"2023-10-16T20:07:59.682136+0000","last_peered":"2023-10-16T20:07:59.682136+0000","last_clean":"2023-10-16T20:07:59.682136+0000","last_became_active":"2023-10-16T20:04:46.334617+0000","last_became_peered":"2023-10-16T20:04:46.334617+0000","last_unstale":"2023-10-16T20:07:59.682136+0000","last_undegraded":"2023-10-16T20:07:59.682136+0000","last_fullsized":"2023-10-16T20:07:59.682136+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T20:04:45.306240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T20:04:45.306240+0000","last_clean_scrub_stamp":"2023-10-16T20:04:45.306240+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:42:11.153721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509484,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1327104,"data_stored":917153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.69099999999999995,"15min":0.70499999999999996},"min":{"1min":0.45500000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.872,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.58299999999999996,"5min":0.495,"15min":0.495},"max":{"1min":0.96499999999999997,"5min":1.077,"15min":1.077},"last":0.623}]},{"osd":1,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.59299999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98399999999999999,"5min":1.78,"15min":1.78},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.70699999999999996,"15min":0.71499999999999997},"min":{"1min":0.57399999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93100000000000005,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":0.56499999999999995}]},{"osd":2,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.72399999999999998,"15min":0.74099999999999999},"min":{"1min":0.54800000000000004,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91600000000000004,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78400000000000003,"15min":0.79500000000000004},"min":{"1min":0.64000000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.94699999999999995,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.72899999999999998}]},{"osd":3,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70499999999999996,"15min":0.70699999999999996},"min":{"1min":0.55100000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.84999999999999998,"5min":1.994,"15min":1.994},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.75700000000000001,"15min":0.78500000000000003},"min":{"1min":0.53200000000000003,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91300000000000003,"5min":2.629,"15min":2.629},"last":0.64200000000000002}]},{"osd":4,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.64900000000000002,"15min":0.66100000000000003},"min":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.98599999999999999,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.64900000000000002,"15min":0.65900000000000003},"min":{"1min":0.48099999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.83499999999999996,"5min":1.944,"15min":1.944},"last":0.53600000000000003}]},{"osd":5,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.72199999999999998,"15min":0.746},"min":{"1min":0.505,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91300000000000003,"5min":2.8809999999999998,"15min":2.8809999999999998},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71099999999999997,"15min":0.71599999999999997},"min":{"1min":0.51900000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94199999999999995,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.71599999999999997}]},{"osd":6,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.755,"15min":0.75700000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.96699999999999997,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72699999999999998,"15min":0.746},"min":{"1min":0.54200000000000004,"5min":0.438,"15min":0.438},"max":{"1min":0.92900000000000005,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.746}]}]},{"osd":6,"up_from":15,"seq":64424509485,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31900,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339940,"statfs":{"total":96636764160,"available":96604098560,"internally_reserved":0,"allocated":733184,"data_stored":326785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.81000000000000005,"15min":0.86399999999999999},"min":{"1min":0.46500000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79400000000000004,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.78600000000000003,"15min":0.85399999999999998},"min":{"1min":0.46899999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86299999999999999,"5min":4.6139999999999999,"15min":4.6139999999999999},"last":0.71299999999999997}]},{"osd":1,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.83799999999999997,"15min":0.89000000000000001},"min":{"1min":0.51400000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.202,"5min":4.6639999999999997,"15min":4.6639999999999997},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.95099999999999996,"15min":1.075},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.86099999999999999,"5min":4.8140000000000001,"15min":4.8140000000000001},"last":0.66400000000000003}]},{"osd":2,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.871,"15min":0.95299999999999996},"min":{"1min":0.502,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88100000000000001,"5min":5.3769999999999998,"15min":5.3769999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.68700000000000006,"15min":0.67500000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.94399999999999995,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.81999999999999995}]},{"osd":3,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.86699999999999999,"15min":0.92400000000000004},"min":{"1min":0.60999999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.004,"5min":4.5460000000000003,"15min":4.5460000000000003},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.748,"15min":0.76600000000000001},"min":{"1min":0.53300000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.92300000000000004,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.63900000000000001}]},{"osd":4,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.70899999999999996,"15min":0.74299999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80200000000000005,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.67900000000000005,"15min":0.69499999999999995},"min":{"1min":0.42399999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93500000000000005,"5min":1.355,"15min":1.355},"last":0.60799999999999998}]},{"osd":5,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.69899999999999995,"15min":0.71799999999999997},"min":{"1min":0.441,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95599999999999996,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.78800000000000003,"15min":0.80200000000000005},"min":{"1min":0.52000000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.89000000000000001,"5min":1.218,"15min":1.218},"last":0.68999999999999995}]},{"osd":7,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.73899999999999999,"15min":0.745},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.97099999999999997,"5min":1.052,"15min":1.052},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.99099999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.76400000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509483,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31896,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339944,"statfs":{"total":96636764160,"available":96604102656,"internally_reserved":0,"allocated":733184,"data_stored":326785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.64300000000000002,"15min":0.63200000000000001},"min":{"1min":0.40100000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94599999999999995,"5min":1.264,"15min":1.264},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.51700000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.56100000000000005}]},{"osd":1,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.63300000000000001,"15min":0.61399999999999999},"min":{"1min":0.47299999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.65200000000000002,"15min":0.64200000000000002},"min":{"1min":0.499,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.64600000000000002}]},{"osd":2,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67300000000000004,"15min":0.67100000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67300000000000004,"15min":0.65700000000000003},"min":{"1min":0.47699999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92300000000000004,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.53500000000000003}]},{"osd":3,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.76400000000000001,"15min":0.77800000000000002},"min":{"1min":0.58399999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.96299999999999997,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.71599999999999997,"15min":0.70399999999999996},"min":{"1min":0.52500000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.51900000000000002}]},{"osd":4,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.45400000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92700000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.67900000000000005,"15min":0.69399999999999995},"min":{"1min":0.41199999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.73599999999999999,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.69599999999999995}]},{"osd":6,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.65200000000000002,"15min":0.66300000000000003},"min":{"1min":0.437,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81899999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.53600000000000003,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.78900000000000003,"5min":1.04,"15min":1.04},"last":0.78600000000000003}]},{"osd":7,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.73699999999999999,"15min":0.746},"min":{"1min":0.53700000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.83199999999999996,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.72699999999999998,"15min":0.74099999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.68000000000000005}]}]},{"osd":4,"up_from":15,"seq":64424509483,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31896,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339944,"statfs":{"total":96636764160,"available":96604102656,"internally_reserved":0,"allocated":733184,"data_stored":326785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.74199999999999999,"15min":0.78600000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79200000000000004,"5min":3.835,"15min":3.835},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.76000000000000001,"15min":0.80700000000000005},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.91700000000000004,"5min":3.7610000000000001,"15min":3.7610000000000001},"last":0.49299999999999999}]},{"osd":1,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66600000000000004,"15min":0.67400000000000004},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.875,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.67400000000000004,"15min":0.67200000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.89100000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.55200000000000005}]},{"osd":2,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.54000000000000004,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.91400000000000003,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.70299999999999996,"15min":0.70699999999999996},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.97099999999999997,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.622}]},{"osd":3,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.78800000000000003,"15min":0.80000000000000004},"min":{"1min":0.54900000000000004,"5min":0.44,"15min":0.44},"max":{"1min":1.002,"5min":1.163,"15min":1.163},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.77700000000000002,"15min":0.79100000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.93999999999999995,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.64600000000000002}]},{"osd":5,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.65500000000000003,"15min":0.66800000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85299999999999998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.67500000000000004,"15min":0.69399999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.94799999999999995,"5min":1.089,"15min":1.089},"last":0.60499999999999998}]},{"osd":6,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71299999999999997,"15min":0.73099999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93100000000000005,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.73799999999999999,"15min":0.77500000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.82799999999999996,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.82599999999999996}]},{"osd":7,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.75,"15min":0.77900000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92200000000000004,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.755},{"interface":"front","average":{"1min":0.747,"5min":0.81999999999999995,"15min":0.84699999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.93200000000000005,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.77100000000000002}]}]},{"osd":3,"up_from":15,"seq":64424509484,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27736,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":733184,"data_stored":326785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.44900000000000001,"15min":0.45900000000000002},"min":{"1min":0.28399999999999997,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46800000000000003,"15min":0.46600000000000003},"min":{"1min":0.36099999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73499999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.45000000000000001}]},{"osd":1,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.441,"15min":0.45400000000000001},"min":{"1min":0.29399999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64700000000000002,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.503,"15min":0.51500000000000001},"min":{"1min":0.36099999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.67300000000000004,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.38900000000000001}]},{"osd":2,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.49399999999999999,"15min":0.499},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70799999999999996,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53900000000000003,"15min":0.54500000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72399999999999998,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.42399999999999999}]},{"osd":4,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.76300000000000001,"15min":0.77700000000000002},"min":{"1min":0.56399999999999995,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.014,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.79800000000000004,"15min":0.81499999999999995},"min":{"1min":0.61699999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.96099999999999997,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.66300000000000003}]},{"osd":5,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.745,"15min":0.76300000000000001},"min":{"1min":0.56899999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.88,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.76900000000000002,"15min":0.78500000000000003},"min":{"1min":0.56699999999999995,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.89400000000000002,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.622}]},{"osd":6,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.752,"15min":0.78000000000000003},"min":{"1min":0.52300000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.90100000000000002,"5min":1.774,"15min":1.774},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.78800000000000003,"15min":0.81200000000000006},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.93000000000000005,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.59099999999999997}]},{"osd":7,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.78000000000000003,"15min":0.80200000000000005},"min":{"1min":0.52500000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94199999999999995,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.77300000000000002,"15min":0.79900000000000004},"min":{"1min":0.56799999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.84899999999999998,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.61099999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509483,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27736,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":733184,"data_stored":326785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40400000000000003,"15min":0.41199999999999998},"min":{"1min":0.26200000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.47199999999999998,"15min":0.48299999999999998},"min":{"1min":0.313,"5min":0.251,"15min":0.251},"max":{"1min":0.60599999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.39200000000000002}]},{"osd":1,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.497,"5min":0.49099999999999999,"15min":0.48999999999999999},"min":{"1min":0.34300000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.45100000000000001}]},{"osd":3,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.47099999999999997,"15min":0.47599999999999998},"min":{"1min":0.34100000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.58599999999999997,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.64500000000000002,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.51800000000000002}]},{"osd":4,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.75,"15min":0.76700000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.95399999999999996,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.73899999999999999,"15min":0.755},"min":{"1min":0.55800000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.107,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.59399999999999997}]},{"osd":5,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72099999999999997,"15min":0.73099999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0920000000000001,"5min":1.145,"15min":1.145},"last":0.629},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.73599999999999999,"15min":0.75},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.995,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.60999999999999999}]},{"osd":6,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.74099999999999999,"15min":0.749},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.97299999999999998,"5min":1.165,"15min":1.165},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.754,"15min":0.76600000000000001},"min":{"1min":0.57499999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.056,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.57699999999999996}]},{"osd":7,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.76000000000000001,"15min":0.76700000000000002},"min":{"1min":0.57799999999999996,"5min":0.502,"15min":0.502},"max":{"1min":1.0249999999999999,"5min":1.167,"15min":1.167},"last":0.623},{"interface":"front","average":{"1min":0.745,"5min":0.76900000000000002,"15min":0.78000000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.01,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.67700000000000005}]}]},{"osd":1,"up_from":15,"seq":64424509484,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27736,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":733184,"data_stored":326785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.499,"15min":0.54100000000000004},"min":{"1min":0.29199999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.63500000000000001,"5min":3.867,"15min":3.867},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.47999999999999998,"15min":0.49199999999999999},"min":{"1min":0.32600000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.621,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.41699999999999998}]},{"osd":2,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.63100000000000001,"15min":0.71499999999999997},"min":{"1min":0.34799999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.59999999999999998,"5min":5.6040000000000001,"15min":5.6040000000000001},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.68899999999999995,"15min":0.78800000000000003},"min":{"1min":0.34999999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65700000000000003,"5min":7.8380000000000001,"15min":7.8380000000000001},"last":0.58699999999999997}]},{"osd":3,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.64200000000000002,"15min":0.70399999999999996},"min":{"1min":0.39800000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.74099999999999999,"5min":5.2039999999999997,"15min":5.2039999999999997},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.59999999999999998,"15min":0.65400000000000003},"min":{"1min":0.36899999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.67000000000000004,"5min":3.9470000000000001,"15min":3.9470000000000001},"last":0.46800000000000003}]},{"osd":4,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.81799999999999995,"15min":0.85599999999999998},"min":{"1min":0.54300000000000004,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.92200000000000004,"5min":4.1500000000000004,"15min":4.1500000000000004},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.83199999999999996,"15min":0.86799999999999999},"min":{"1min":0.54700000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.93300000000000005,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":0.61099999999999999}]},{"osd":5,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.754,"15min":0.75800000000000001},"min":{"1min":0.56399999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93200000000000005,"5min":1.496,"15min":1.496},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.84199999999999997,"15min":0.88100000000000001},"min":{"1min":0.56999999999999995,"5min":0.501,"15min":0.501},"max":{"1min":0.96999999999999997,"5min":4.0910000000000002,"15min":4.0910000000000002},"last":0.67900000000000005}]},{"osd":6,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.84199999999999997,"15min":0.876},"min":{"1min":0.56999999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.01,"5min":4.21,"15min":4.21},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.753,"5min":0.75900000000000001,"15min":0.76400000000000001},"min":{"1min":0.57799999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.95299999999999996,"5min":1.401,"15min":1.401},"last":0.57899999999999996}]},{"osd":7,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.79500000000000004,"15min":0.79800000000000004},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.90600000000000003,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.62},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.83699999999999997,"15min":0.875},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.94099999999999995,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":0.72899999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28324,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343516,"statfs":{"total":96636764160,"available":96607760384,"internally_reserved":0,"allocated":1327104,"data_stored":917153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.51600000000000001,"15min":0.57599999999999996},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.61499999999999999,"5min":3.9289999999999998,"15min":3.9289999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.54100000000000004,"15min":0.59699999999999998},"min":{"1min":0.34200000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.61099999999999999,"5min":4.0910000000000002,"15min":4.0910000000000002},"last":0.46200000000000002}]},{"osd":2,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.60499999999999998,"15min":0.66200000000000003},"min":{"1min":0.32100000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70999999999999996,"5min":4.1520000000000001,"15min":4.1520000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.63700000000000001,"15min":0.69599999999999995},"min":{"1min":0.42399999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.71399999999999997,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.41499999999999998}]},{"osd":3,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.58699999999999997,"15min":0.63600000000000001},"min":{"1min":0.36499999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.67800000000000005,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55800000000000005,"15min":0.56799999999999995},"min":{"1min":0.45100000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.73599999999999999,"5min":1.006,"15min":1.006},"last":0.64400000000000002}]},{"osd":4,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.79900000000000004,"15min":0.85599999999999998},"min":{"1min":0.52500000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.91100000000000003,"5min":4.5250000000000004,"15min":4.5250000000000004},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.73699999999999999,"15min":0.747},"min":{"1min":0.5,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.89500000000000002,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.67900000000000005}]},{"osd":5,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.83899999999999997,"15min":0.89500000000000002},"min":{"1min":0.58199999999999996,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.004,"5min":3.9990000000000001,"15min":3.9990000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.877,"15min":0.94499999999999995},"min":{"1min":0.54200000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0189999999999999,"5min":4.4779999999999998,"15min":4.4779999999999998},"last":0.60199999999999998}]},{"osd":6,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.75600000000000001,"15min":0.76600000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.93100000000000005,"5min":1.363,"15min":1.363},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.85699999999999998,"15min":0.92900000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.97099999999999997,"5min":3.956,"15min":3.956},"last":0.58099999999999996}]},{"osd":7,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.879,"15min":0.93700000000000006},"min":{"1min":0.624,"5min":0.435,"15min":0.435},"max":{"1min":1.034,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.621},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.85799999999999998,"15min":0.92600000000000005},"min":{"1min":0.56399999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.97899999999999998,"5min":4.5640000000000001,"15min":4.5640000000000001},"last":0.56399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T20:08:09.065 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-16T20:04:45.306240+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=20, tm_min=4, tm_sec=45, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=20, tm_min=8, tm_sec=3, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T20:08:09.065 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-16T20:08:26.811 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T20:08:26.816 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T20:08:29.067 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T20:08:29.215 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.208+0000 7fac29017640 1 Processor -- start 2023-10-16T20:08:29.216 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.208+0000 7fac29017640 1 -- start start 2023-10-16T20:08:29.217 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.212+0000 7fac29017640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fac241599f0 0x7fac240b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:29.218 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.212+0000 7fac29017640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fac240b0cb0 0x7fac2406a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:29.218 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.212+0000 7fac29017640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac2406a830 con 0x7fac24069e90 2023-10-16T20:08:29.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.212+0000 7fac29017640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac240b1260 con 0x7fac240b0cb0 2023-10-16T20:08:29.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.212+0000 7fac29017640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac240ae960 con 0x7fac241599f0 2023-10-16T20:08:29.219 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.212+0000 7fac2359e640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fac240b0cb0 0x7fac2406a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:29.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.212+0000 7fac1bfff640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fac24069e90 0x7fac240b05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:47562/0 (socket says 172.21.15.88:47562) 2023-10-16T20:08:29.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.212+0000 7fac1bfff640 1 -- 172.21.15.88:0/3904737977 learned_addr learned my addr 172.21.15.88:0/3904737977 (peer_addr_for_me v1:172.21.15.88:0/0) 2023-10-16T20:08:29.220 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.212+0000 7fac22d9d640 1 --2- 172.21.15.88:0/3904737977 >> v2:172.21.15.94:3300/0 conn(0x7fac241599f0 0x7fac240b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:29.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.212+0000 7fac2359e640 1 -- 172.21.15.88:0/3904737977 >> v2:172.21.15.94:3300/0 conn(0x7fac241599f0 msgr2=0x7fac240b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:08:29.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.212+0000 7fac2359e640 1 --2- 172.21.15.88:0/3904737977 >> v2:172.21.15.94:3300/0 conn(0x7fac241599f0 0x7fac240b2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:29.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.212+0000 7fac2359e640 1 -- 172.21.15.88:0/3904737977 >> v1:172.21.15.88:6789/0 conn(0x7fac24069e90 legacy=0x7fac240b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:29.221 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.212+0000 7fac2359e640 1 -- 172.21.15.88:0/3904737977 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac240aec40 con 0x7fac240b0cb0 2023-10-16T20:08:29.222 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.212+0000 7fac22d9d640 1 --2- 172.21.15.88:0/3904737977 >> v2:172.21.15.94:3300/0 conn(0x7fac241599f0 0x7fac240b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T20:08:29.222 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.212+0000 7fac2259c640 1 -- 172.21.15.88:0/3904737977 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 795792497 0 0) 0x7fac2406a830 con 0x7fac24069e90 2023-10-16T20:08:29.225 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.216+0000 7fac2359e640 1 --2- 172.21.15.88:0/3904737977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac240b0cb0 0x7fac2406a2f0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fac2406d650 tx=0x7fac1c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba6dee52a4be296a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:29.225 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.216+0000 7fac2259c640 1 -- 172.21.15.88:0/3904737977 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c0a59a0 con 0x7fac240b0cb0 2023-10-16T20:08:29.226 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.216+0000 7fac2259c640 1 -- 172.21.15.88:0/3904737977 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1c0a5b00 con 0x7fac240b0cb0 2023-10-16T20:08:29.226 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.216+0000 7fac2259c640 1 -- 172.21.15.88:0/3904737977 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c0a5e90 con 0x7fac240b0cb0 2023-10-16T20:08:29.226 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.216+0000 7fac29017640 1 -- 172.21.15.88:0/3904737977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac240b0cb0 msgr2=0x7fac2406a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:29.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.216+0000 7fac29017640 1 --2- 172.21.15.88:0/3904737977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac240b0cb0 0x7fac2406a2f0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fac2406d650 tx=0x7fac1c01c8e0 comp rx=0 tx=0).stop 2023-10-16T20:08:29.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.216+0000 7fac29017640 1 -- 172.21.15.88:0/3904737977 shutdown_connections 2023-10-16T20:08:29.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.216+0000 7fac29017640 1 --2- 172.21.15.88:0/3904737977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac240b0cb0 0x7fac2406a2f0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:29.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.216+0000 7fac29017640 1 --2- 172.21.15.88:0/3904737977 >> v2:172.21.15.94:3300/0 conn(0x7fac241599f0 0x7fac240b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:29.229 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.216+0000 7fac29017640 1 -- 172.21.15.88:0/3904737977 >> 172.21.15.88:0/3904737977 conn(0x7fac240ce5e0 msgr2=0x7fac240ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:29.229 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.216+0000 7fac29017640 1 -- 172.21.15.88:0/3904737977 shutdown_connections 2023-10-16T20:08:29.230 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.216+0000 7fac29017640 1 -- 172.21.15.88:0/3904737977 wait complete. 2023-10-16T20:08:29.230 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.216+0000 7fac29017640 1 Processor -- start 2023-10-16T20:08:29.230 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.216+0000 7fac29017640 1 -- start start 2023-10-16T20:08:29.230 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.216+0000 7fac29017640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fac24069e90 0x7fac2414a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:29.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.220+0000 7fac29017640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac240b0cb0 0x7fac241f8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:29.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.220+0000 7fac29017640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac241599f0 0x7fac241f8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:29.231 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.220+0000 7fac29017640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2414f9c0 con 0x7fac240b0cb0 2023-10-16T20:08:29.232 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.220+0000 7fac29017640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2414f730 con 0x7fac241599f0 2023-10-16T20:08:29.232 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.220+0000 7fac29017640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac2414ca00 con 0x7fac24069e90 2023-10-16T20:08:29.232 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.220+0000 7fac1bfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac240b0cb0 0x7fac241f8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:29.232 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.220+0000 7fac1bfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac240b0cb0 0x7fac241f8420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42234/0 (socket says 172.21.15.88:42234) 2023-10-16T20:08:29.233 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.220+0000 7fac1bfff640 1 -- 172.21.15.88:0/3478478419 learned_addr learned my addr 172.21.15.88:0/3478478419 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:29.233 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.220+0000 7fac22d9d640 1 --2- 172.21.15.88:0/3478478419 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fac24069e90 0x7fac2414a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:29.233 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.220+0000 7fac2359e640 1 --2- 172.21.15.88:0/3478478419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac241599f0 0x7fac241f8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:29.234 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.220+0000 7fac1bfff640 1 -- 172.21.15.88:0/3478478419 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fac24069e90 msgr2=0x7fac2414a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:29.234 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.220+0000 7fac1bfff640 1 --2- 172.21.15.88:0/3478478419 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fac24069e90 0x7fac2414a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:29.234 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.220+0000 7fac1bfff640 1 -- 172.21.15.88:0/3478478419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac241599f0 msgr2=0x7fac241f8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:29.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.220+0000 7fac1bfff640 1 --2- 172.21.15.88:0/3478478419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac241599f0 0x7fac241f8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:29.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.220+0000 7fac1bfff640 1 -- 172.21.15.88:0/3478478419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1c009a70 con 0x7fac240b0cb0 2023-10-16T20:08:29.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.220+0000 7fac1bfff640 1 --2- 172.21.15.88:0/3478478419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac240b0cb0 0x7fac241f8420 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fac0c0011f0 tx=0x7fac0c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:29.235 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.220+0000 7fac1b7fe640 1 -- 172.21.15.88:0/3478478419 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0c0074e0 con 0x7fac240b0cb0 2023-10-16T20:08:29.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.220+0000 7fac1b7fe640 1 -- 172.21.15.88:0/3478478419 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac0c007640 con 0x7fac240b0cb0 2023-10-16T20:08:29.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.220+0000 7fac29017640 1 -- 172.21.15.88:0/3478478419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac24071e40 con 0x7fac240b0cb0 2023-10-16T20:08:29.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.220+0000 7fac1b7fe640 1 -- 172.21.15.88:0/3478478419 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0c004a10 con 0x7fac240b0cb0 2023-10-16T20:08:29.236 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.220+0000 7fac29017640 1 -- 172.21.15.88:0/3478478419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac240abd10 con 0x7fac240b0cb0 2023-10-16T20:08:29.238 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.220+0000 7fac29017640 1 -- 172.21.15.88:0/3478478419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac240ab890 con 0x7fac240b0cb0 2023-10-16T20:08:29.238 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.220+0000 7fac1b7fe640 1 -- 172.21.15.88:0/3478478419 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fac0c004b70 con 0x7fac240b0cb0 2023-10-16T20:08:29.238 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.224+0000 7fac1b7fe640 1 --2- 172.21.15.88:0/3478478419 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fac040664a0 0x7fac04068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:29.239 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.224+0000 7fac1b7fe640 1 -- 172.21.15.88:0/3478478419 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fac0c083130 con 0x7fac240b0cb0 2023-10-16T20:08:29.239 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.224+0000 7fac22d9d640 1 --2- 172.21.15.88:0/3478478419 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fac040664a0 0x7fac04068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:29.239 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.224+0000 7fac22d9d640 1 --2- 172.21.15.88:0/3478478419 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fac040664a0 0x7fac04068960 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fac14006fd0 tx=0x7fac14008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:29.240 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.228+0000 7fac1b7fe640 1 -- 172.21.15.88:0/3478478419 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fac240ab890 con 0x7fac240b0cb0 2023-10-16T20:08:29.379 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.372+0000 7fac29017640 1 -- 172.21.15.88:0/3478478419 --> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fac240aff30 con 0x7fac040664a0 2023-10-16T20:08:29.381 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.376+0000 7fac1b7fe640 1 -- 172.21.15.88:0/3478478419 <== mgr.4110 v2:172.21.15.88:6832/4078483333 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+51022 (secure 0 0 0) 0x7fac1400c040 con 0x7fac040664a0 2023-10-16T20:08:29.382 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:08:29.384 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2023-10-16T20:08:29.384 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.376+0000 7fac29017640 1 -- 172.21.15.88:0/3478478419 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fac040664a0 msgr2=0x7fac04068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:29.384 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.376+0000 7fac29017640 1 --2- 172.21.15.88:0/3478478419 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fac040664a0 0x7fac04068960 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fac14006fd0 tx=0x7fac14008040 comp rx=0 tx=0).stop 2023-10-16T20:08:29.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.376+0000 7fac29017640 1 -- 172.21.15.88:0/3478478419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac240b0cb0 msgr2=0x7fac241f8420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:29.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.376+0000 7fac29017640 1 --2- 172.21.15.88:0/3478478419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac240b0cb0 0x7fac241f8420 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fac0c0011f0 tx=0x7fac0c00c9b0 comp rx=0 tx=0).stop 2023-10-16T20:08:29.385 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.376+0000 7fac29017640 1 -- 172.21.15.88:0/3478478419 shutdown_connections 2023-10-16T20:08:29.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.376+0000 7fac29017640 1 --2- 172.21.15.88:0/3478478419 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fac040664a0 0x7fac04068960 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:29.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.376+0000 7fac29017640 1 --2- 172.21.15.88:0/3478478419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac241599f0 0x7fac241f8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:29.386 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.376+0000 7fac29017640 1 --2- 172.21.15.88:0/3478478419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac240b0cb0 0x7fac241f8420 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:29.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.376+0000 7fac29017640 1 --2- 172.21.15.88:0/3478478419 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fac24069e90 0x7fac2414a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:29.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.376+0000 7fac29017640 1 -- 172.21.15.88:0/3478478419 >> 172.21.15.88:0/3478478419 conn(0x7fac240ce5e0 msgr2=0x7fac240b05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:29.387 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.380+0000 7fac29017640 1 -- 172.21.15.88:0/3478478419 shutdown_connections 2023-10-16T20:08:29.388 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.380+0000 7fac29017640 1 -- 172.21.15.88:0/3478478419 wait complete. 2023-10-16T20:08:29.403 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2023-10-16T20:08:28.490288+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":239684,"kb_used_data":6888,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735036,"statfs":{"total":773094113280,"available":772848676864,"internally_reserved":0,"allocated":7053312,"data_stored":3795016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238276153},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002719"},"pg_stats":[{"pgid":"1.0","version":"61'119","reported_seq":211,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-16T20:08:12.950070+0000","last_change":"2023-10-16T20:08:12.950157+0000","last_active":"2023-10-16T20:08:12.950070+0000","last_peered":"2023-10-16T20:08:12.950070+0000","last_clean":"2023-10-16T20:08:12.950070+0000","last_became_active":"2023-10-16T20:04:46.334617+0000","last_became_peered":"2023-10-16T20:04:46.334617+0000","last_unstale":"2023-10-16T20:08:12.950070+0000","last_undegraded":"2023-10-16T20:08:12.950070+0000","last_fullsized":"2023-10-16T20:08:12.950070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'119","last_scrub_stamp":"2023-10-16T20:08:12.949967+0000","last_deep_scrub":"61'119","last_deep_scrub_stamp":"2023-10-16T20:08:12.949967+0000","last_clean_scrub_stamp":"2023-10-16T20:08:12.949967+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:17:51.448695+0000","scrub_duration":0.0075297960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509488,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1327104,"data_stored":917153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.69099999999999995,"15min":0.70499999999999996},"min":{"1min":0.45500000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.872,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.58299999999999996,"5min":0.495,"15min":0.495},"max":{"1min":0.96499999999999997,"5min":1.077,"15min":1.077},"last":0.76600000000000001}]},{"osd":1,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.59299999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98399999999999999,"5min":1.78,"15min":1.78},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.70699999999999996,"15min":0.71499999999999997},"min":{"1min":0.57399999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93100000000000005,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":0.72699999999999998}]},{"osd":2,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.72399999999999998,"15min":0.74099999999999999},"min":{"1min":0.54800000000000004,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91600000000000004,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78400000000000003,"15min":0.79500000000000004},"min":{"1min":0.64000000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.94699999999999995,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.74299999999999999}]},{"osd":3,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70499999999999996,"15min":0.70699999999999996},"min":{"1min":0.55100000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.84999999999999998,"5min":1.994,"15min":1.994},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.75700000000000001,"15min":0.78500000000000003},"min":{"1min":0.53200000000000003,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91300000000000003,"5min":2.629,"15min":2.629},"last":0.78700000000000003}]},{"osd":4,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.64900000000000002,"15min":0.66100000000000003},"min":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.98599999999999999,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.64900000000000002,"15min":0.65900000000000003},"min":{"1min":0.48099999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.83499999999999996,"5min":1.944,"15min":1.944},"last":0.44900000000000001}]},{"osd":5,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.72199999999999998,"15min":0.746},"min":{"1min":0.505,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91300000000000003,"5min":2.8809999999999998,"15min":2.8809999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71099999999999997,"15min":0.71599999999999997},"min":{"1min":0.51900000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94199999999999995,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.68600000000000005}]},{"osd":6,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.755,"15min":0.75700000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.96699999999999997,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72699999999999998,"15min":0.746},"min":{"1min":0.54200000000000004,"5min":0.438,"15min":0.438},"max":{"1min":0.92900000000000005,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.52900000000000003}]}]},{"osd":6,"up_from":15,"seq":64424509489,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31900,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339940,"statfs":{"total":96636764160,"available":96604098560,"internally_reserved":0,"allocated":733184,"data_stored":326785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.81000000000000005,"15min":0.86399999999999999},"min":{"1min":0.46500000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79400000000000004,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.78600000000000003,"15min":0.85399999999999998},"min":{"1min":0.46899999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86299999999999999,"5min":4.6139999999999999,"15min":4.6139999999999999},"last":0.91500000000000004}]},{"osd":1,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.83799999999999997,"15min":0.89000000000000001},"min":{"1min":0.51400000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.202,"5min":4.6639999999999997,"15min":4.6639999999999997},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.95099999999999996,"15min":1.075},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.86099999999999999,"5min":4.8140000000000001,"15min":4.8140000000000001},"last":0.90100000000000002}]},{"osd":2,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.871,"15min":0.95299999999999996},"min":{"1min":0.502,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88100000000000001,"5min":5.3769999999999998,"15min":5.3769999999999998},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.68700000000000006,"15min":0.67500000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.94399999999999995,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.79700000000000004}]},{"osd":3,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.86699999999999999,"15min":0.92400000000000004},"min":{"1min":0.60999999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.004,"5min":4.5460000000000003,"15min":4.5460000000000003},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.748,"15min":0.76600000000000001},"min":{"1min":0.53300000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.92300000000000004,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.81299999999999994}]},{"osd":4,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.70899999999999996,"15min":0.74299999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80200000000000005,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.67900000000000005,"15min":0.69499999999999995},"min":{"1min":0.42399999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93500000000000005,"5min":1.355,"15min":1.355},"last":0.57599999999999996}]},{"osd":5,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.69899999999999995,"15min":0.71799999999999997},"min":{"1min":0.441,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95599999999999996,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.78800000000000003,"15min":0.80200000000000005},"min":{"1min":0.52000000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.89000000000000001,"5min":1.218,"15min":1.218},"last":0.66800000000000004}]},{"osd":7,"last update":"Mon Oct 16 20:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.73899999999999999,"15min":0.745},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.97099999999999997,"5min":1.052,"15min":1.052},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.99099999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.63500000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509487,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31896,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339944,"statfs":{"total":96636764160,"available":96604102656,"internally_reserved":0,"allocated":733184,"data_stored":326785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.64300000000000002,"15min":0.63200000000000001},"min":{"1min":0.40100000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94599999999999995,"5min":1.264,"15min":1.264},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.51700000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.45800000000000002}]},{"osd":1,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.63300000000000001,"15min":0.61399999999999999},"min":{"1min":0.47299999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.498},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.65200000000000002,"15min":0.64200000000000002},"min":{"1min":0.499,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.66000000000000003}]},{"osd":2,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67300000000000004,"15min":0.67100000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67300000000000004,"15min":0.65700000000000003},"min":{"1min":0.47699999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92300000000000004,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.63700000000000001}]},{"osd":3,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.76400000000000001,"15min":0.77800000000000002},"min":{"1min":0.58399999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.96299999999999997,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.71599999999999997,"15min":0.70399999999999996},"min":{"1min":0.52500000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.76300000000000001}]},{"osd":4,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.45400000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92700000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.67900000000000005,"15min":0.69399999999999995},"min":{"1min":0.41199999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.73599999999999999,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.67800000000000005}]},{"osd":6,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.65200000000000002,"15min":0.66300000000000003},"min":{"1min":0.437,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81899999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.53600000000000003,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.78900000000000003,"5min":1.04,"15min":1.04},"last":0.79200000000000004}]},{"osd":7,"last update":"Mon Oct 16 20:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.73699999999999999,"15min":0.746},"min":{"1min":0.53700000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.83199999999999996,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.72699999999999998,"15min":0.74099999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.71899999999999997}]}]},{"osd":4,"up_from":15,"seq":64424509487,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31896,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339944,"statfs":{"total":96636764160,"available":96604102656,"internally_reserved":0,"allocated":733184,"data_stored":326785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.74199999999999999,"15min":0.78600000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79200000000000004,"5min":3.835,"15min":3.835},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.76000000000000001,"15min":0.80700000000000005},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.91700000000000004,"5min":3.7610000000000001,"15min":3.7610000000000001},"last":0.622}]},{"osd":1,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66600000000000004,"15min":0.67400000000000004},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.875,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.67400000000000004,"15min":0.67200000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.89100000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.82599999999999996}]},{"osd":2,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.54000000000000004,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.91400000000000003,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.70299999999999996,"15min":0.70699999999999996},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.97099999999999997,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.73899999999999999}]},{"osd":3,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.78800000000000003,"15min":0.80000000000000004},"min":{"1min":0.54900000000000004,"5min":0.44,"15min":0.44},"max":{"1min":1.002,"5min":1.163,"15min":1.163},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.77700000000000002,"15min":0.79100000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.93999999999999995,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.88100000000000001}]},{"osd":5,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.65500000000000003,"15min":0.66800000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85299999999999998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.67500000000000004,"15min":0.69399999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.94799999999999995,"5min":1.089,"15min":1.089},"last":0.755}]},{"osd":6,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71299999999999997,"15min":0.73099999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93100000000000005,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.73799999999999999,"15min":0.77500000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.82799999999999996,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.70899999999999996}]},{"osd":7,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.75,"15min":0.77900000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92200000000000004,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.81999999999999995,"15min":0.84699999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.93200000000000005,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.80300000000000005}]}]},{"osd":3,"up_from":15,"seq":64424509488,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27736,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":733184,"data_stored":326785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.44900000000000001,"15min":0.45900000000000002},"min":{"1min":0.28399999999999997,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46800000000000003,"15min":0.46600000000000003},"min":{"1min":0.36099999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73499999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.51700000000000002}]},{"osd":1,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.441,"15min":0.45400000000000001},"min":{"1min":0.29399999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64700000000000002,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.503,"15min":0.51500000000000001},"min":{"1min":0.36099999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.67300000000000004,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.40600000000000003}]},{"osd":2,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.49399999999999999,"15min":0.499},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70799999999999996,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53900000000000003,"15min":0.54500000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72399999999999998,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.504}]},{"osd":4,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.76300000000000001,"15min":0.77700000000000002},"min":{"1min":0.56399999999999995,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.014,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.79800000000000004,"15min":0.81499999999999995},"min":{"1min":0.61699999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.96099999999999997,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.82099999999999995}]},{"osd":5,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.745,"15min":0.76300000000000001},"min":{"1min":0.56899999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.88,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.76900000000000002,"15min":0.78500000000000003},"min":{"1min":0.56699999999999995,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.89400000000000002,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.65800000000000003}]},{"osd":6,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.752,"15min":0.78000000000000003},"min":{"1min":0.52300000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.90100000000000002,"5min":1.774,"15min":1.774},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.78800000000000003,"15min":0.81200000000000006},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.93000000000000005,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.77800000000000002}]},{"osd":7,"last update":"Mon Oct 16 20:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.78000000000000003,"15min":0.80200000000000005},"min":{"1min":0.52500000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94199999999999995,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.77300000000000002,"15min":0.79900000000000004},"min":{"1min":0.56799999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.84899999999999998,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.79800000000000004}]}]},{"osd":2,"up_from":15,"seq":64424509487,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27736,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":733184,"data_stored":326785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40400000000000003,"15min":0.41199999999999998},"min":{"1min":0.26200000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.47199999999999998,"15min":0.48299999999999998},"min":{"1min":0.313,"5min":0.251,"15min":0.251},"max":{"1min":0.60599999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.48299999999999998}]},{"osd":1,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.49099999999999999,"15min":0.48999999999999999},"min":{"1min":0.34300000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.434}]},{"osd":3,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.47099999999999997,"15min":0.47599999999999998},"min":{"1min":0.34100000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.58599999999999997,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.64500000000000002,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.46300000000000002}]},{"osd":4,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.75,"15min":0.76700000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.95399999999999996,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.73899999999999999,"15min":0.755},"min":{"1min":0.55800000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.107,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.61099999999999999}]},{"osd":5,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72099999999999997,"15min":0.73099999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0920000000000001,"5min":1.145,"15min":1.145},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.73599999999999999,"15min":0.75},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.995,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.626}]},{"osd":6,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.74099999999999999,"15min":0.749},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.97299999999999998,"5min":1.165,"15min":1.165},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.754,"15min":0.76600000000000001},"min":{"1min":0.57499999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.056,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.84099999999999997}]},{"osd":7,"last update":"Mon Oct 16 20:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.76000000000000001,"15min":0.76700000000000002},"min":{"1min":0.57799999999999996,"5min":0.502,"15min":0.502},"max":{"1min":1.0249999999999999,"5min":1.167,"15min":1.167},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.76900000000000002,"15min":0.78000000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.01,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.89200000000000002}]}]},{"osd":1,"up_from":15,"seq":64424509488,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27736,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":733184,"data_stored":326785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.499,"15min":0.54100000000000004},"min":{"1min":0.29199999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.63500000000000001,"5min":3.867,"15min":3.867},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.47999999999999998,"15min":0.49199999999999999},"min":{"1min":0.32600000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.621,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.443}]},{"osd":2,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.63100000000000001,"15min":0.71499999999999997},"min":{"1min":0.34799999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.59999999999999998,"5min":5.6040000000000001,"15min":5.6040000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.68899999999999995,"15min":0.78800000000000003},"min":{"1min":0.34999999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65700000000000003,"5min":7.8380000000000001,"15min":7.8380000000000001},"last":0.59799999999999998}]},{"osd":3,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.64200000000000002,"15min":0.70399999999999996},"min":{"1min":0.39800000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.74099999999999999,"5min":5.2039999999999997,"15min":5.2039999999999997},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.59999999999999998,"15min":0.65400000000000003},"min":{"1min":0.36899999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.67000000000000004,"5min":3.9470000000000001,"15min":3.9470000000000001},"last":0.64500000000000002}]},{"osd":4,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.81799999999999995,"15min":0.85599999999999998},"min":{"1min":0.54300000000000004,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.92200000000000004,"5min":4.1500000000000004,"15min":4.1500000000000004},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.83199999999999996,"15min":0.86799999999999999},"min":{"1min":0.54700000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.93300000000000005,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":0.67000000000000004}]},{"osd":5,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.754,"15min":0.75800000000000001},"min":{"1min":0.56399999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93200000000000005,"5min":1.496,"15min":1.496},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.84199999999999997,"15min":0.88100000000000001},"min":{"1min":0.56999999999999995,"5min":0.501,"15min":0.501},"max":{"1min":0.96999999999999997,"5min":4.0910000000000002,"15min":4.0910000000000002},"last":0.77400000000000002}]},{"osd":6,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.84199999999999997,"15min":0.876},"min":{"1min":0.56999999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.01,"5min":4.21,"15min":4.21},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.753,"5min":0.75900000000000001,"15min":0.76400000000000001},"min":{"1min":0.57799999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.95299999999999996,"5min":1.401,"15min":1.401},"last":0.752}]},{"osd":7,"last update":"Mon Oct 16 20:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.79500000000000004,"15min":0.79800000000000004},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.90600000000000003,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.83699999999999997,"15min":0.875},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.94099999999999995,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":0.72899999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574896,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28312,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343528,"statfs":{"total":96636764160,"available":96607772672,"internally_reserved":0,"allocated":1327104,"data_stored":917153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.51600000000000001,"15min":0.57599999999999996},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.61499999999999999,"5min":3.9289999999999998,"15min":3.9289999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.54100000000000004,"15min":0.59699999999999998},"min":{"1min":0.34200000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.61099999999999999,"5min":4.0910000000000002,"15min":4.0910000000000002},"last":0.58799999999999997}]},{"osd":2,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.60499999999999998,"15min":0.66200000000000003},"min":{"1min":0.32100000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70999999999999996,"5min":4.1520000000000001,"15min":4.1520000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.63700000000000001,"15min":0.69599999999999995},"min":{"1min":0.42399999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.71399999999999997,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.76700000000000002}]},{"osd":3,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.58699999999999997,"15min":0.63600000000000001},"min":{"1min":0.36499999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.67800000000000005,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55800000000000005,"15min":0.56799999999999995},"min":{"1min":0.45100000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.73599999999999999,"5min":1.006,"15min":1.006},"last":0.69099999999999995}]},{"osd":4,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.79900000000000004,"15min":0.85599999999999998},"min":{"1min":0.52500000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.91100000000000003,"5min":4.5250000000000004,"15min":4.5250000000000004},"last":0.755},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.73699999999999999,"15min":0.747},"min":{"1min":0.5,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.89500000000000002,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.86699999999999999}]},{"osd":5,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.83899999999999997,"15min":0.89500000000000002},"min":{"1min":0.58199999999999996,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.004,"5min":3.9990000000000001,"15min":3.9990000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.877,"15min":0.94499999999999995},"min":{"1min":0.54200000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0189999999999999,"5min":4.4779999999999998,"15min":4.4779999999999998},"last":0.83999999999999997}]},{"osd":6,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.75600000000000001,"15min":0.76600000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.93100000000000005,"5min":1.363,"15min":1.363},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.85699999999999998,"15min":0.92900000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.97099999999999997,"5min":3.956,"15min":3.956},"last":0.89200000000000002}]},{"osd":7,"last update":"Mon Oct 16 20:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.879,"15min":0.93700000000000006},"min":{"1min":0.624,"5min":0.435,"15min":0.435},"max":{"1min":1.034,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.85799999999999998,"15min":0.92600000000000005},"min":{"1min":0.56399999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.97899999999999998,"5min":4.5640000000000001,"15min":4.5640000000000001},"last":0.91200000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T20:08:29.406 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-16T20:08:29.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.536+0000 7fd504c46640 1 Processor -- start 2023-10-16T20:08:29.543 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.536+0000 7fd504c46640 1 -- start start 2023-10-16T20:08:29.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.540+0000 7fd504c46640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd50015cc80 0x7fd5000cd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:29.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.540+0000 7fd504c46640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd50006c830 0x7fd500070730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:29.545 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.540+0000 7fd504c46640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5000c4870 con 0x7fd500070c70 2023-10-16T20:08:29.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.540+0000 7fd504c46640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5000cd8b0 con 0x7fd50006c830 2023-10-16T20:08:29.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.540+0000 7fd504c46640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd500071100 con 0x7fd50015cc80 2023-10-16T20:08:29.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.540+0000 7fd4f5d9b640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd50006c830 0x7fd500070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:29.546 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.540+0000 7fd4f5d9b640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd50006c830 0x7fd500070730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42238/0 (socket says 172.21.15.88:42238) 2023-10-16T20:08:29.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.540+0000 7fd4f5d9b640 1 -- 172.21.15.88:0/1889184789 learned_addr learned my addr 172.21.15.88:0/1889184789 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:29.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.540+0000 7fd4fe59c640 1 --2- 172.21.15.88:0/1889184789 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd50015cc80 0x7fd5000cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:29.547 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.540+0000 7fd4f5d9b640 1 -- 172.21.15.88:0/1889184789 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd50015cc80 msgr2=0x7fd5000cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:08:29.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.540+0000 7fd4f5d9b640 1 --2- 172.21.15.88:0/1889184789 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd50015cc80 0x7fd5000cd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:29.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.540+0000 7fd4f5d9b640 1 -- 172.21.15.88:0/1889184789 >> v1:172.21.15.94:6789/0 conn(0x7fd500070c70 legacy=0x7fd5000c4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:29.548 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.540+0000 7fd4f5d9b640 1 -- 172.21.15.88:0/1889184789 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5000717c0 con 0x7fd50006c830 2023-10-16T20:08:29.549 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.544+0000 7fd4f5d9b640 1 --2- 172.21.15.88:0/1889184789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd50006c830 0x7fd500070730 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0009870 tx=0x7fd4f001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c55cb97d7fd69b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:29.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.544+0000 7fd4fdd9b640 1 -- 172.21.15.88:0/1889184789 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f00ad640 con 0x7fd50006c830 2023-10-16T20:08:29.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.544+0000 7fd4fdd9b640 1 -- 172.21.15.88:0/1889184789 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f00ad7a0 con 0x7fd50006c830 2023-10-16T20:08:29.550 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.544+0000 7fd4fdd9b640 1 -- 172.21.15.88:0/1889184789 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f00adb30 con 0x7fd50006c830 2023-10-16T20:08:29.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.544+0000 7fd504c46640 1 -- 172.21.15.88:0/1889184789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd50006c830 msgr2=0x7fd500070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:29.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.544+0000 7fd504c46640 1 --2- 172.21.15.88:0/1889184789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd50006c830 0x7fd500070730 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0009870 tx=0x7fd4f001c480 comp rx=0 tx=0).stop 2023-10-16T20:08:29.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.544+0000 7fd504c46640 1 -- 172.21.15.88:0/1889184789 shutdown_connections 2023-10-16T20:08:29.551 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.544+0000 7fd504c46640 1 --2- 172.21.15.88:0/1889184789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd50006c830 0x7fd500070730 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:29.552 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.544+0000 7fd504c46640 1 --2- 172.21.15.88:0/1889184789 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd50015cc80 0x7fd5000cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:29.552 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.544+0000 7fd504c46640 1 -- 172.21.15.88:0/1889184789 >> 172.21.15.88:0/1889184789 conn(0x7fd5000ceb40 msgr2=0x7fd5000cef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:29.552 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.544+0000 7fd504c46640 1 -- 172.21.15.88:0/1889184789 shutdown_connections 2023-10-16T20:08:29.552 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.544+0000 7fd504c46640 1 -- 172.21.15.88:0/1889184789 wait complete. 2023-10-16T20:08:29.554 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.544+0000 7fd504c46640 1 Processor -- start 2023-10-16T20:08:29.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.544+0000 7fd504c46640 1 -- start start 2023-10-16T20:08:29.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.548+0000 7fd504c46640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd50006c830 0x7fd500146d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:29.555 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.548+0000 7fd504c46640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd500070c70 0x7fd500147240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:29.556 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.548+0000 7fd504c46640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd50015cc80 0x7fd5001519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:29.556 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.548+0000 7fd504c46640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd500147830 con 0x7fd50015cc80 2023-10-16T20:08:29.556 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.548+0000 7fd504c46640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5001555d0 con 0x7fd500070c70 2023-10-16T20:08:29.556 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.548+0000 7fd504c46640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5001558b0 con 0x7fd50006c830 2023-10-16T20:08:29.557 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.548+0000 7fd4fed9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd50015cc80 0x7fd5001519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:29.557 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.548+0000 7fd4f5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd500070c70 0x7fd500147240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:29.557 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.548+0000 7fd4fe59c640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd50006c830 0x7fd500146d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:29.558 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.548+0000 7fd4fed9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd50015cc80 0x7fd5001519a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42242/0 (socket says 172.21.15.88:42242) 2023-10-16T20:08:29.558 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.548+0000 7fd4fed9d640 1 -- 172.21.15.88:0/1909249571 learned_addr learned my addr 172.21.15.88:0/1909249571 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:29.558 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.548+0000 7fd4fed9d640 1 -- 172.21.15.88:0/1909249571 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd50006c830 msgr2=0x7fd500146d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:29.558 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.548+0000 7fd4fed9d640 1 --2- 172.21.15.88:0/1909249571 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd50006c830 0x7fd500146d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:29.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.548+0000 7fd4fed9d640 1 -- 172.21.15.88:0/1909249571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd500070c70 msgr2=0x7fd500147240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:29.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.548+0000 7fd4fed9d640 1 --2- 172.21.15.88:0/1909249571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd500070c70 0x7fd500147240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:29.559 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.548+0000 7fd4fed9d640 1 -- 172.21.15.88:0/1909249571 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f00094d0 con 0x7fd50015cc80 2023-10-16T20:08:29.560 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.548+0000 7fd4fed9d640 1 --2- 172.21.15.88:0/1909249571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd50015cc80 0x7fd5001519a0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800c770 tx=0x7fd4f800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:29.560 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.548+0000 7fd4f77fe640 1 -- 172.21.15.88:0/1909249571 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8003b80 con 0x7fd50015cc80 2023-10-16T20:08:29.560 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.548+0000 7fd4f77fe640 1 -- 172.21.15.88:0/1909249571 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f8003ce0 con 0x7fd50015cc80 2023-10-16T20:08:29.561 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.548+0000 7fd504c46640 1 -- 172.21.15.88:0/1909249571 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd500155b90 con 0x7fd50015cc80 2023-10-16T20:08:29.561 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.548+0000 7fd4f77fe640 1 -- 172.21.15.88:0/1909249571 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f800aa90 con 0x7fd50015cc80 2023-10-16T20:08:29.561 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.548+0000 7fd504c46640 1 -- 172.21.15.88:0/1909249571 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd500151ee0 con 0x7fd50015cc80 2023-10-16T20:08:29.561 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.548+0000 7fd504c46640 1 -- 172.21.15.88:0/1909249571 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5001479a0 con 0x7fd50015cc80 2023-10-16T20:08:29.562 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.548+0000 7fd4f77fe640 1 -- 172.21.15.88:0/1909249571 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd4f800abf0 con 0x7fd50015cc80 2023-10-16T20:08:29.563 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.552+0000 7fd4f77fe640 1 --2- 172.21.15.88:0/1909249571 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd4e00664a0 0x7fd4e0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:29.563 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.552+0000 7fd4f77fe640 1 -- 172.21.15.88:0/1909249571 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fd4f8083920 con 0x7fd50015cc80 2023-10-16T20:08:29.563 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.552+0000 7fd4fe59c640 1 --2- 172.21.15.88:0/1909249571 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd4e00664a0 0x7fd4e0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:29.564 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.552+0000 7fd4fe59c640 1 --2- 172.21.15.88:0/1909249571 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd4e00664a0 0x7fd4e0068960 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd5000ae500 tx=0x7fd4e8009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:29.564 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.556+0000 7fd4f77fe640 1 -- 172.21.15.88:0/1909249571 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd5001479a0 con 0x7fd50015cc80 2023-10-16T20:08:29.709 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.704+0000 7fd504c46640 1 -- 172.21.15.88:0/1909249571 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fd500147970 con 0x7fd50015cc80 2023-10-16T20:08:29.715 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.708+0000 7fd4f77fe640 1 -- 172.21.15.88:0/1909249571 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v14)=0 v14) v1 ==== 123+0+0 (secure 0 0 0) 0x7fd4f80510a0 con 0x7fd50015cc80 2023-10-16T20:08:29.716 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.712+0000 7fd4f77fe640 1 -- 172.21.15.88:0/1909249571 <== mon.0 v2:172.21.15.88:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fd4f800ce80 con 0x7fd50015cc80 2023-10-16T20:08:29.718 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.712+0000 7fd504c46640 1 -- 172.21.15.88:0/1909249571 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd4e00664a0 msgr2=0x7fd4e0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:29.718 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.712+0000 7fd504c46640 1 --2- 172.21.15.88:0/1909249571 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd4e00664a0 0x7fd4e0068960 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd5000ae500 tx=0x7fd4e8009190 comp rx=0 tx=0).stop 2023-10-16T20:08:29.719 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.712+0000 7fd504c46640 1 -- 172.21.15.88:0/1909249571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd50015cc80 msgr2=0x7fd5001519a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:29.719 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.712+0000 7fd504c46640 1 --2- 172.21.15.88:0/1909249571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd50015cc80 0x7fd5001519a0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800c770 tx=0x7fd4f800cc30 comp rx=0 tx=0).stop 2023-10-16T20:08:29.719 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.712+0000 7fd504c46640 1 -- 172.21.15.88:0/1909249571 shutdown_connections 2023-10-16T20:08:29.720 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.712+0000 7fd504c46640 1 --2- 172.21.15.88:0/1909249571 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7fd4e00664a0 0x7fd4e0068960 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:29.720 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.712+0000 7fd504c46640 1 --2- 172.21.15.88:0/1909249571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd50015cc80 0x7fd5001519a0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:29.720 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.712+0000 7fd504c46640 1 --2- 172.21.15.88:0/1909249571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd500070c70 0x7fd500147240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:29.721 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.712+0000 7fd504c46640 1 --2- 172.21.15.88:0/1909249571 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd50006c830 0x7fd500146d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:29.721 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.712+0000 7fd504c46640 1 -- 172.21.15.88:0/1909249571 >> 172.21.15.88:0/1909249571 conn(0x7fd5000ceb40 msgr2=0x7fd5000cd140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:29.721 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.712+0000 7fd504c46640 1 -- 172.21.15.88:0/1909249571 shutdown_connections 2023-10-16T20:08:29.722 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.712+0000 7fd504c46640 1 -- 172.21.15.88:0/1909249571 wait complete. 2023-10-16T20:08:29.738 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T20:08:29.739 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T20:08:29.875 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.868+0000 7f63d793e640 1 Processor -- start 2023-10-16T20:08:29.876 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.872+0000 7f63d793e640 1 -- start start 2023-10-16T20:08:29.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.872+0000 7f63d793e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63d015a2f0 0x7f63d00af330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:29.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.872+0000 7f63d793e640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f63d00afca0 0x7f63d00b0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:29.877 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.872+0000 7f63d793e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d00ae290 con 0x7f63d0153540 2023-10-16T20:08:29.878 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.872+0000 7f63d793e640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d00b1bf0 con 0x7f63d00afca0 2023-10-16T20:08:29.878 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.872+0000 7f63d793e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d00b0670 con 0x7f63d015a2f0 2023-10-16T20:08:29.879 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.872+0000 7f63d5edb640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f63d00afca0 0x7f63d00b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:29.879 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.872+0000 7f63d5edb640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f63d00afca0 0x7f63d00b0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42252/0 (socket says 172.21.15.88:42252) 2023-10-16T20:08:29.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.872+0000 7f63d5edb640 1 -- 172.21.15.88:0/2918385331 learned_addr learned my addr 172.21.15.88:0/2918385331 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:29.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.872+0000 7f63d4ed9640 1 --2- 172.21.15.88:0/2918385331 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63d015a2f0 0x7f63d00af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:29.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.872+0000 7f63d5edb640 1 -- 172.21.15.88:0/2918385331 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63d015a2f0 msgr2=0x7f63d00af330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T20:08:29.880 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.872+0000 7f63d5edb640 1 --2- 172.21.15.88:0/2918385331 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63d015a2f0 0x7f63d00af330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:29.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.872+0000 7f63d5edb640 1 -- 172.21.15.88:0/2918385331 >> v1:172.21.15.94:6789/0 conn(0x7f63d0153540 legacy=0x7f63d006a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:29.881 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.872+0000 7f63d5edb640 1 -- 172.21.15.88:0/2918385331 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d00ae570 con 0x7f63d00afca0 2023-10-16T20:08:29.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.876+0000 7f63d5edb640 1 --2- 172.21.15.88:0/2918385331 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63d00afca0 0x7f63d00b0130 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f63c8009e50 tx=0x7f63c801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4147fb8f348a2de5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:29.882 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.876+0000 7f63c7fff640 1 -- 172.21.15.88:0/2918385331 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c80a5750 con 0x7f63d00afca0 2023-10-16T20:08:29.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.876+0000 7f63c7fff640 1 -- 172.21.15.88:0/2918385331 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f63c80a58b0 con 0x7f63d00afca0 2023-10-16T20:08:29.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.876+0000 7f63c7fff640 1 -- 172.21.15.88:0/2918385331 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c80a5c60 con 0x7f63d00afca0 2023-10-16T20:08:29.883 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.876+0000 7f63d793e640 1 -- 172.21.15.88:0/2918385331 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63d00afca0 msgr2=0x7f63d00b0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:29.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.876+0000 7f63d793e640 1 --2- 172.21.15.88:0/2918385331 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63d00afca0 0x7f63d00b0130 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f63c8009e50 tx=0x7f63c801ca10 comp rx=0 tx=0).stop 2023-10-16T20:08:29.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.876+0000 7f63d793e640 1 -- 172.21.15.88:0/2918385331 shutdown_connections 2023-10-16T20:08:29.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.876+0000 7f63d793e640 1 --2- 172.21.15.88:0/2918385331 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63d00afca0 0x7f63d00b0130 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:29.884 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.876+0000 7f63d793e640 1 --2- 172.21.15.88:0/2918385331 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63d015a2f0 0x7f63d00af330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:29.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.876+0000 7f63d793e640 1 -- 172.21.15.88:0/2918385331 >> 172.21.15.88:0/2918385331 conn(0x7f63d00ce5e0 msgr2=0x7f63d00ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:29.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.876+0000 7f63d793e640 1 -- 172.21.15.88:0/2918385331 shutdown_connections 2023-10-16T20:08:29.885 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.876+0000 7f63d793e640 1 -- 172.21.15.88:0/2918385331 wait complete. 2023-10-16T20:08:29.887 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.876+0000 7f63d793e640 1 Processor -- start 2023-10-16T20:08:29.887 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.876+0000 7f63d793e640 1 -- start start 2023-10-16T20:08:29.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.880+0000 7f63d793e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63d00afca0 0x7f63d0159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:29.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.880+0000 7f63d793e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63d0153540 0x7f63d013d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:29.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.880+0000 7f63d793e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63d015a2f0 0x7f63d013d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:29.888 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.880+0000 7f63d793e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d0147de0 con 0x7f63d00afca0 2023-10-16T20:08:29.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.880+0000 7f63d793e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d013d600 con 0x7f63d015a2f0 2023-10-16T20:08:29.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.880+0000 7f63d793e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d01474d0 con 0x7f63d0153540 2023-10-16T20:08:29.889 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.880+0000 7f63d56da640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63d00afca0 0x7f63d0159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:29.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.880+0000 7f63d56da640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63d00afca0 0x7f63d0159760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42264/0 (socket says 172.21.15.88:42264) 2023-10-16T20:08:29.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.880+0000 7f63d56da640 1 -- 172.21.15.88:0/2737335382 learned_addr learned my addr 172.21.15.88:0/2737335382 (peer_addr_for_me v2:172.21.15.88:0/0) 2023-10-16T20:08:29.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.880+0000 7f63d5edb640 1 --2- 172.21.15.88:0/2737335382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63d015a2f0 0x7f63d013d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:29.890 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.880+0000 7f63d4ed9640 1 --2- 172.21.15.88:0/2737335382 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63d0153540 0x7f63d013d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:29.891 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.880+0000 7f63d56da640 1 -- 172.21.15.88:0/2737335382 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63d0153540 msgr2=0x7f63d013d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:29.891 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.880+0000 7f63d56da640 1 --2- 172.21.15.88:0/2737335382 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63d0153540 0x7f63d013d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:29.891 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.880+0000 7f63d56da640 1 -- 172.21.15.88:0/2737335382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63d015a2f0 msgr2=0x7f63d013d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:29.892 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.880+0000 7f63d56da640 1 --2- 172.21.15.88:0/2737335382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63d015a2f0 0x7f63d013d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:29.892 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.880+0000 7f63d56da640 1 -- 172.21.15.88:0/2737335382 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c8009b30 con 0x7f63d00afca0 2023-10-16T20:08:29.892 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.880+0000 7f63d56da640 1 --2- 172.21.15.88:0/2737335382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63d00afca0 0x7f63d0159760 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f63cc00c530 tx=0x7f63cc00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:29.892 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.880+0000 7f63c5ffb640 1 -- 172.21.15.88:0/2737335382 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc005220 con 0x7f63d00afca0 2023-10-16T20:08:29.893 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.880+0000 7f63c5ffb640 1 -- 172.21.15.88:0/2737335382 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f63cc005380 con 0x7f63d00afca0 2023-10-16T20:08:29.893 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.880+0000 7f63c5ffb640 1 -- 172.21.15.88:0/2737335382 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc007290 con 0x7f63d00afca0 2023-10-16T20:08:29.893 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.880+0000 7f63d793e640 1 -- 172.21.15.88:0/2737335382 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d0071c60 con 0x7f63d00afca0 2023-10-16T20:08:29.893 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.880+0000 7f63d793e640 1 -- 172.21.15.88:0/2737335382 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d0147720 con 0x7f63d00afca0 2023-10-16T20:08:29.894 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.880+0000 7f63d793e640 1 -- 172.21.15.88:0/2737335382 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d0147f20 con 0x7f63d00afca0 2023-10-16T20:08:29.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.884+0000 7f63c5ffb640 1 -- 172.21.15.88:0/2737335382 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f63cc0073f0 con 0x7f63d00afca0 2023-10-16T20:08:29.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.884+0000 7f63c5ffb640 1 --2- 172.21.15.88:0/2737335382 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f63b40664a0 0x7f63b4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T20:08:29.895 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.884+0000 7f63c5ffb640 1 -- 172.21.15.88:0/2737335382 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f63cc083490 con 0x7f63d00afca0 2023-10-16T20:08:29.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.884+0000 7f63d4ed9640 1 --2- 172.21.15.88:0/2737335382 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f63b40664a0 0x7f63b4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T20:08:29.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.884+0000 7f63d4ed9640 1 --2- 172.21.15.88:0/2737335382 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f63b40664a0 0x7f63b4068960 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f63d00719e0 tx=0x7f63c0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T20:08:29.896 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:29.888+0000 7f63c5ffb640 1 -- 172.21.15.88:0/2737335382 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f63d0147f20 con 0x7f63d00afca0 2023-10-16T20:08:30.076 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:30.072+0000 7f63d793e640 1 -- 172.21.15.88:0/2737335382 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63d0147f20 con 0x7f63d00afca0 2023-10-16T20:08:30.077 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:30.072+0000 7f63c5ffb640 1 -- 172.21.15.88:0/2737335382 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f63d0147f20 con 0x7f63d00afca0 2023-10-16T20:08:30.078 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-10-16T20:08:30.078 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":30,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5531,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/2485638574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":2485638574},{"type":"v1","addr":"172.21.15.88:6837","nonce":2485638574}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5534,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3821390040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3821390040},{"type":"v1","addr":"172.21.15.94:6834","nonce":3821390040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5968,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2544673478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":2544673478},{"type":"v1","addr":"172.21.15.94:6835","nonce":2544673478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":6648,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/3676538862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3676538862},{"type":"v1","addr":"172.21.15.88:6836","nonce":3676538862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[]} 2023-10-16T20:08:30.080 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2023-10-16T20:08:30.080 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:30.072+0000 7f63d793e640 1 -- 172.21.15.88:0/2737335382 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f63b40664a0 msgr2=0x7f63b4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:30.080 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:30.072+0000 7f63d793e640 1 --2- 172.21.15.88:0/2737335382 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f63b40664a0 0x7f63b4068960 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f63d00719e0 tx=0x7f63c0008040 comp rx=0 tx=0).stop 2023-10-16T20:08:30.081 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:30.072+0000 7f63d793e640 1 -- 172.21.15.88:0/2737335382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63d00afca0 msgr2=0x7f63d0159760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T20:08:30.081 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:30.072+0000 7f63d793e640 1 --2- 172.21.15.88:0/2737335382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63d00afca0 0x7f63d0159760 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f63cc00c530 tx=0x7f63cc00c9f0 comp rx=0 tx=0).stop 2023-10-16T20:08:30.081 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:30.072+0000 7f63d793e640 1 -- 172.21.15.88:0/2737335382 shutdown_connections 2023-10-16T20:08:30.082 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:30.072+0000 7f63d793e640 1 --2- 172.21.15.88:0/2737335382 >> [v2:172.21.15.88:6832/4078483333,v1:172.21.15.88:6833/4078483333] conn(0x7f63b40664a0 0x7f63b4068960 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:30.082 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:30.072+0000 7f63d793e640 1 --2- 172.21.15.88:0/2737335382 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f63d0153540 0x7f63d013d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:30.082 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:30.072+0000 7f63d793e640 1 --2- 172.21.15.88:0/2737335382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63d015a2f0 0x7f63d013d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:30.082 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:30.072+0000 7f63d793e640 1 --2- 172.21.15.88:0/2737335382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63d00afca0 0x7f63d0159760 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T20:08:30.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:30.072+0000 7f63d793e640 1 -- 172.21.15.88:0/2737335382 >> 172.21.15.88:0/2737335382 conn(0x7f63d00ce5e0 msgr2=0x7f63d015c050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T20:08:30.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:30.072+0000 7f63d793e640 1 -- 172.21.15.88:0/2737335382 shutdown_connections 2023-10-16T20:08:30.083 INFO:teuthology.orchestra.run.smithi088.stderr:2023-10-16T20:08:30.072+0000 7f63d793e640 1 -- 172.21.15.88:0/2737335382 wait complete. 2023-10-16T20:08:30.098 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-16T20:08:30.099 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-16T20:08:30.100 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T20:08:30.100 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:08:30.112 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T20:08:30.112 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T20:08:30.113 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:08:30.168 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T20:08:30.169 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T20:08:30.169 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:08:30.184 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T20:08:30.184 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T20:08:30.184 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:08:30.241 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T20:08:30.241 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-16T20:08:30.242 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-16T20:08:30.242 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:08:30.311 INFO:tasks.ceph.osd.0:Stopped 2023-10-16T20:08:30.311 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-16T20:08:30.312 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:08:30.402 INFO:tasks.ceph.osd.1:Stopped 2023-10-16T20:08:30.403 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-16T20:08:30.403 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:08:30.480 INFO:tasks.ceph.osd.2:Stopped 2023-10-16T20:08:30.481 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-16T20:08:30.481 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:08:30.557 INFO:tasks.ceph.osd.3:Stopped 2023-10-16T20:08:30.558 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-16T20:08:30.558 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:08:30.626 INFO:tasks.ceph.osd.4:Stopped 2023-10-16T20:08:30.627 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-16T20:08:30.627 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:08:30.718 INFO:tasks.ceph.osd.5:Stopped 2023-10-16T20:08:30.718 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-16T20:08:30.719 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:08:30.796 INFO:tasks.ceph.osd.6:Stopped 2023-10-16T20:08:30.797 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-16T20:08:30.797 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:08:30.865 INFO:tasks.ceph.osd.7:Stopped 2023-10-16T20:08:30.866 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-16T20:08:30.866 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-16T20:08:30.866 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:08:30.920 INFO:tasks.ceph.mgr.y:Stopped 2023-10-16T20:08:30.921 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-16T20:08:30.921 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:08:30.970 INFO:tasks.ceph.mgr.x:Stopped 2023-10-16T20:08:30.970 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-16T20:08:30.971 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-16T20:08:30.971 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:08:30.989 INFO:tasks.ceph.mon.a:Stopped 2023-10-16T20:08:30.990 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-16T20:08:30.990 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:08:31.005 INFO:tasks.ceph.mon.b:Stopped 2023-10-16T20:08:31.005 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-16T20:08:31.006 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T20:08:31.066 INFO:tasks.ceph.mon.c:Stopped 2023-10-16T20:08:31.067 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-16T20:08:31.067 DEBUG:teuthology.orchestra.run.smithi088:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-10-16T20:08:31.097 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi088.front.sepia.ceph.com 2023-10-16T20:08:31.097 DEBUG:teuthology.orchestra.run.smithi088:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-16T20:08:31.395 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi088.front.sepia.ceph.com 2023-10-16T20:08:31.396 DEBUG:teuthology.orchestra.run.smithi088:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-16T20:08:31.611 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi088.front.sepia.ceph.com 2023-10-16T20:08:31.612 DEBUG:teuthology.orchestra.run.smithi088:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-16T20:08:31.779 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi088.front.sepia.ceph.com 2023-10-16T20:08:31.780 DEBUG:teuthology.orchestra.run.smithi088:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-16T20:08:31.928 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi094.front.sepia.ceph.com 2023-10-16T20:08:31.929 DEBUG:teuthology.orchestra.run.smithi094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-16T20:08:32.295 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi094.front.sepia.ceph.com 2023-10-16T20:08:32.296 DEBUG:teuthology.orchestra.run.smithi094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-16T20:08:32.531 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi094.front.sepia.ceph.com 2023-10-16T20:08:32.531 DEBUG:teuthology.orchestra.run.smithi094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-16T20:08:32.738 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi094.front.sepia.ceph.com 2023-10-16T20:08:32.739 DEBUG:teuthology.orchestra.run.smithi094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-16T20:08:32.939 INFO:tasks.ceph:Archiving mon data... 2023-10-16T20:08:32.940 DEBUG:teuthology.misc:Transferring archived files from smithi088:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2023-10-16_18:49:15-fs-rishabh-2023oct13-testing-default-smithi/7430091/data/mon.a.tgz 2023-10-16T20:08:32.940 DEBUG:teuthology.orchestra.run.smithi088:> mktemp 2023-10-16T20:08:32.950 INFO:teuthology.orchestra.run.smithi088.stdout:/tmp/tmp.0F0OTGmXQl 2023-10-16T20:08:32.951 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.0F0OTGmXQl 2023-10-16T20:08:33.087 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 0666 /tmp/tmp.0F0OTGmXQl 2023-10-16T20:08:33.165 DEBUG:teuthology.orchestra.remote:smithi088:/tmp/tmp.0F0OTGmXQl is 367KB 2023-10-16T20:08:33.201 DEBUG:teuthology.orchestra.run.smithi088:> rm -fr /tmp/tmp.0F0OTGmXQl 2023-10-16T20:08:33.211 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2023-10-16_18:49:15-fs-rishabh-2023oct13-testing-default-smithi/7430091/data/mon.b.tgz 2023-10-16T20:08:33.212 DEBUG:teuthology.orchestra.run.smithi094:> mktemp 2023-10-16T20:08:33.223 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/tmp.tZuRZEiC1e 2023-10-16T20:08:33.224 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.tZuRZEiC1e 2023-10-16T20:08:33.391 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0666 /tmp/tmp.tZuRZEiC1e 2023-10-16T20:08:33.473 DEBUG:teuthology.orchestra.remote:smithi094:/tmp/tmp.tZuRZEiC1e is 477KB 2023-10-16T20:08:33.508 DEBUG:teuthology.orchestra.run.smithi094:> rm -fr /tmp/tmp.tZuRZEiC1e 2023-10-16T20:08:33.518 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2023-10-16_18:49:15-fs-rishabh-2023oct13-testing-default-smithi/7430091/data/mon.c.tgz 2023-10-16T20:08:33.518 DEBUG:teuthology.orchestra.run.smithi094:> mktemp 2023-10-16T20:08:33.567 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/tmp.tPInXaCiyw 2023-10-16T20:08:33.567 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.tPInXaCiyw 2023-10-16T20:08:33.725 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0666 /tmp/tmp.tPInXaCiyw 2023-10-16T20:08:33.805 DEBUG:teuthology.orchestra.remote:smithi094:/tmp/tmp.tPInXaCiyw is 477KB 2023-10-16T20:08:33.885 DEBUG:teuthology.orchestra.run.smithi094:> rm -fr /tmp/tmp.tPInXaCiyw 2023-10-16T20:08:33.893 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-16T20:08:33.894 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-16T20:08:33.898 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-16T20:08:33.954 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-16T20:08:33.957 DEBUG:teuthology.orchestra.run.smithi088:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-16T20:08:33.966 DEBUG:teuthology.orchestra.run.smithi094:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-16T20:08:34.009 INFO:teuthology.orchestra.run.smithi088.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-16T20:08:34.026 INFO:teuthology.orchestra.run.smithi094.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-16T20:08:34.032 INFO:tasks.ceph:Archiving crash dumps... 2023-10-16T20:08:34.034 DEBUG:teuthology.misc:Transferring archived files from smithi088:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-10-16_18:49:15-fs-rishabh-2023oct13-testing-default-smithi/7430091/remote/smithi088/crash 2023-10-16T20:08:34.035 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-16T20:08:34.085 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-10-16_18:49:15-fs-rishabh-2023oct13-testing-default-smithi/7430091/remote/smithi094/crash 2023-10-16T20:08:34.086 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-16T20:08:34.106 INFO:tasks.ceph:Shutting down logrotate 2023-10-16T20:08:34.106 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T20:08:34.134 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T20:08:34.202 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-16T20:08:34.224 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-16T20:08:34.244 INFO:tasks.ceph:Compressing logs... 2023-10-16T20:08:34.244 DEBUG:teuthology.orchestra.run.smithi088:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-16T20:08:34.270 DEBUG:teuthology.orchestra.run.smithi094:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-16T20:08:34.302 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47905.log 2023-10-16T20:08:34.303 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50722.log 2023-10-16T20:08:34.304 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46477.log 2023-10-16T20:08:34.304 INFO:teuthology.orchestra.run.smithi088.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47905.log.gz 2023-10-16T20:08:34.305 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50722.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50722.log.gzgzip 2023-10-16T20:08:34.305 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50319.log 2023-10-16T20:08:34.306 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50946.log 2023-10-16T20:08:34.306 INFO:teuthology.orchestra.run.smithi088.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46477.log.gz 2023-10-16T20:08:34.307 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50319.log: 89.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.50319.log.gz 2023-10-16T20:08:34.307 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49359.log 2023-10-16T20:08:34.308 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50946.log: gzip -5 --verbose 88.3% -- -- replaced with /var/log/ceph/ceph-client.admin.50946.log.gz /var/log/ceph/ceph-client.admin.53125.log 2023-10-16T20:08:34.308 INFO:teuthology.orchestra.run.smithi088.stderr: 2023-10-16T20:08:34.309 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53066.log 2023-10-16T20:08:34.310 INFO:teuthology.orchestra.run.smithi088.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49359.log.gz 2023-10-16T20:08:34.310 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53125.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53125.log.gz 2023-10-16T20:08:34.311 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49866.log 2023-10-16T20:08:34.312 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38277.log 2023-10-16T20:08:34.312 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-10-16T20:08:34.313 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53066.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53066.log.gz 2023-10-16T20:08:34.313 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51102.log 2023-10-16T20:08:34.315 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38277.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-10-16T20:08:34.315 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38277.log.gz 2023-10-16T20:08:34.315 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38253.log 2023-10-16T20:08:34.315 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38157.log 2023-10-16T20:08:34.316 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45277.log 2023-10-16T20:08:34.317 INFO:teuthology.orchestra.run.smithi088.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49866.log.gz 2023-10-16T20:08:34.317 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43524.log 2023-10-16T20:08:34.317 INFO:teuthology.orchestra.run.smithi088.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51102.log.gz 2023-10-16T20:08:34.317 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52239.log 2023-10-16T20:08:34.318 INFO:teuthology.orchestra.run.smithi088.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45277.log.gz 2023-10-16T20:08:34.318 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43524.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.43524.log.gz 2023-10-16T20:08:34.318 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50748.log 2023-10-16T20:08:34.321 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52239.log: gzip -5 --verbose -- 89.5% /var/log/ceph/ceph-client.admin.51310.log -- replaced with /var/log/ceph/ceph-client.admin.52239.log.gz 2023-10-16T20:08:34.322 INFO:teuthology.orchestra.run.smithi088.stderr: 2023-10-16T20:08:34.322 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50748.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50748.log.gz 2023-10-16T20:08:34.322 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-10-16T20:08:34.323 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51310.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.51310.log.gz 2023-10-16T20:08:34.323 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46139.log 2023-10-16T20:08:34.323 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53458.log 2023-10-16T20:08:34.323 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46139.log: gzip -5 --verbose 88.8% -- -- replaced with /var/log/ceph/ceph-client.admin.46139.log.gz /var/log/ceph/ceph-client.admin.50805.log 2023-10-16T20:08:34.324 INFO:teuthology.orchestra.run.smithi088.stderr: 2023-10-16T20:08:34.325 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38253.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-10-16T20:08:34.325 INFO:teuthology.orchestra.run.smithi094.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38253.log.gz 2023-10-16T20:08:34.326 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38205.log 2023-10-16T20:08:34.326 INFO:teuthology.orchestra.run.smithi094.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.38157.log.gz 2023-10-16T20:08:34.326 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-10-16T20:08:34.326 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38205.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38205.log.gz 2023-10-16T20:08:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-16T20:08:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42669.log 2023-10-16T20:08:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38301.log 2023-10-16T20:08:34.327 INFO:teuthology.orchestra.run.smithi094.stderr: 91.9% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-16T20:08:34.328 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.1.42669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42777.log 2023-10-16T20:08:34.328 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38301.log: 90.3% -- replaced with /var/log/ceph/ceph-client.1.42669.log.gz 2023-10-16T20:08:34.328 INFO:teuthology.orchestra.run.smithi094.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38301.log.gz 2023-10-16T20:08:34.328 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-10-16T20:08:34.332 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.3.42777.log: 93.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-10-16T20:08:34.332 INFO:teuthology.orchestra.run.smithi094.stderr: 91.3%gzip -- replaced with /var/log/ceph/ceph-client.3.42777.log.gz 2023-10-16T20:08:34.332 INFO:teuthology.orchestra.run.smithi094.stderr: -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-16T20:08:34.333 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-10-16T20:08:34.333 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38181.log 2023-10-16T20:08:34.333 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38229.log 2023-10-16T20:08:34.334 INFO:teuthology.orchestra.run.smithi094.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-16T20:08:34.334 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38181.log: 91.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-10-16T20:08:34.334 INFO:teuthology.orchestra.run.smithi094.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38181.log.gz 2023-10-16T20:08:34.334 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-10-16T20:08:34.338 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38229.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-10-16T20:08:34.339 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38229.log.gz 2023-10-16T20:08:34.339 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38133.log 2023-10-16T20:08:34.339 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-10-16T20:08:34.340 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38133.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38133.log.gz 2023-10-16T20:08:34.342 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48373.log 2023-10-16T20:08:34.342 INFO:teuthology.orchestra.run.smithi088.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53458.log.gz 2023-10-16T20:08:34.342 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45563.log 2023-10-16T20:08:34.343 INFO:teuthology.orchestra.run.smithi088.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50805.log.gz 2023-10-16T20:08:34.343 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48373.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48373.log.gz 2023-10-16T20:08:34.343 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46451.log 2023-10-16T20:08:34.343 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46347.log 2023-10-16T20:08:34.344 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45563.log.gz 2023-10-16T20:08:34.344 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46451.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46451.log.gz 2023-10-16T20:08:34.344 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53361.log 2023-10-16T20:08:34.344 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48269.log 2023-10-16T20:08:34.345 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46347.log.gz 2023-10-16T20:08:34.345 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53361.log: gzip 88.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.53361.log.gz 2023-10-16T20:08:34.345 INFO:teuthology.orchestra.run.smithi088.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53151.log 2023-10-16T20:08:34.345 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46848.log 2023-10-16T20:08:34.346 INFO:teuthology.orchestra.run.smithi088.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48269.log.gz 2023-10-16T20:08:34.346 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53762.log 2023-10-16T20:08:34.346 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53151.log.gz 2023-10-16T20:08:34.346 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51570.log 2023-10-16T20:08:34.347 INFO:teuthology.orchestra.run.smithi088.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46848.log.gz 2023-10-16T20:08:34.347 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47271.log 2023-10-16T20:08:34.347 INFO:teuthology.orchestra.run.smithi088.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53762.log.gz 2023-10-16T20:08:34.348 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52584.log 87.6% 2023-10-16T20:08:34.348 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51570.log.gz 2023-10-16T20:08:34.348 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43715.log 2023-10-16T20:08:34.348 INFO:teuthology.orchestra.run.smithi088.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47271.log.gz 2023-10-16T20:08:34.349 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50972.log 2023-10-16T20:08:34.349 INFO:teuthology.orchestra.run.smithi088.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52584.log.gz 2023-10-16T20:08:34.349 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43715.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43715.log.gz 2023-10-16T20:08:34.350 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44575.log 2023-10-16T20:08:34.350 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50240.log 2023-10-16T20:08:34.350 INFO:teuthology.orchestra.run.smithi088.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50972.log.gz 2023-10-16T20:08:34.350 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45069.log 2023-10-16T20:08:34.351 INFO:teuthology.orchestra.run.smithi088.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44575.log.gz 2023-10-16T20:08:34.351 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50240.log: gzip -5 --verbose -- 88.6% /var/log/ceph/ceph-client.admin.44127.log -- replaced with /var/log/ceph/ceph-client.admin.50240.log.gz 2023-10-16T20:08:34.351 INFO:teuthology.orchestra.run.smithi088.stderr: 2023-10-16T20:08:34.351 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48139.log 2023-10-16T20:08:34.352 INFO:teuthology.orchestra.run.smithi088.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45069.log.gz 2023-10-16T20:08:34.352 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44127.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-10-16T20:08:34.352 INFO:teuthology.orchestra.run.smithi088.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44127.log.gz 2023-10-16T20:08:34.357 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51336.log 2023-10-16T20:08:34.357 INFO:teuthology.orchestra.run.smithi088.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48139.log.gz 2023-10-16T20:08:34.357 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47931.log 2023-10-16T20:08:34.358 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51336.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51336.log.gz 2023-10-16T20:08:34.358 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46555.log 2023-10-16T20:08:34.358 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46874.log 2023-10-16T20:08:34.358 INFO:teuthology.orchestra.run.smithi088.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47931.log.gz 2023-10-16T20:08:34.359 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46555.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46555.log.gz 2023-10-16T20:08:34.359 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47245.log 2023-10-16T20:08:34.359 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46874.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46874.log.gz 2023-10-16T20:08:34.360 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49257.log 2023-10-16T20:08:34.360 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47245.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47245.log.gz 2023-10-16T20:08:34.360 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52029.log 2023-10-16T20:08:34.360 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49257.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49257.log.gz 2023-10-16T20:08:34.361 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51979.log 2023-10-16T20:08:34.361 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52029.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52029.log.gz 2023-10-16T20:08:34.361 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44678.log 2023-10-16T20:08:34.361 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43241.log 2023-10-16T20:08:34.362 INFO:teuthology.orchestra.run.smithi088.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.51979.log.gz 2023-10-16T20:08:34.362 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44678.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44678.log.gz 2023-10-16T20:08:34.362 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46633.log 2023-10-16T20:08:34.363 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43241.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43241.log.gz 2023-10-16T20:08:34.363 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47638.log 2023-10-16T20:08:34.363 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46633.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46633.log.gz 2023-10-16T20:08:34.363 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50513.log 2023-10-16T20:08:34.367 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47638.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47638.log.gz 2023-10-16T20:08:34.367 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44336.log 2023-10-16T20:08:34.367 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50513.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50513.log.gz 2023-10-16T20:08:34.368 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44471.log 2023-10-16T20:08:34.368 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44336.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44336.log.gz 2023-10-16T20:08:34.368 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53580.log 2023-10-16T20:08:34.368 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44471.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44471.log.gz 2023-10-16T20:08:34.369 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45407.log 2023-10-16T20:08:34.369 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51206.log 2023-10-16T20:08:34.369 INFO:teuthology.orchestra.run.smithi088.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53580.log.gz 2023-10-16T20:08:34.370 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45407.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45407.log.gz 2023-10-16T20:08:34.370 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48861.log 2023-10-16T20:08:34.370 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51623.log 2023-10-16T20:08:34.370 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51206.log.gz 2023-10-16T20:08:34.371 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48861.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48861.log.gz 2023-10-16T20:08:34.371 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47983.log 2023-10-16T20:08:34.371 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48451.log 2023-10-16T20:08:34.371 INFO:teuthology.orchestra.run.smithi088.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51623.log.gz 2023-10-16T20:08:34.374 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47983.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47983.log.gz 2023-10-16T20:08:34.374 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45511.log 2023-10-16T20:08:34.374 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48451.log: 91.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-10-16T20:08:34.375 INFO:teuthology.orchestra.run.smithi088.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48451.log.gz 2023-10-16T20:08:34.375 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47031.log 2023-10-16T20:08:34.375 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45511.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45511.log.gz 2023-10-16T20:08:34.376 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48087.log 2023-10-16T20:08:34.376 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47031.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47031.log.gz 2023-10-16T20:08:34.376 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52213.log 2023-10-16T20:08:34.376 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48087.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48087.log.gz 2023-10-16T20:08:34.377 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43606.log 2023-10-16T20:08:34.377 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52213.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52213.log.gz 2023-10-16T20:08:34.377 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52318.log 2023-10-16T20:08:34.379 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52930.log 2023-10-16T20:08:34.380 INFO:teuthology.orchestra.run.smithi088.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43606.log.gz 2023-10-16T20:08:34.380 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48009.log 2023-10-16T20:08:34.380 INFO:teuthology.orchestra.run.smithi088.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.52318.log.gz 2023-10-16T20:08:34.380 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45381.log 2023-10-16T20:08:34.381 INFO:teuthology.orchestra.run.smithi088.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52930.log.gz 2023-10-16T20:08:34.381 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48009.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48009.log.gz 2023-10-16T20:08:34.381 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49709.log 2023-10-16T20:08:34.381 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45381.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.45381.log.gz 2023-10-16T20:08:34.382 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48113.log 2023-10-16T20:08:34.383 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52265.log 2023-10-16T20:08:34.384 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49709.log.gz 2023-10-16T20:08:34.384 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48113.log: gzip 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48113.log.gz -5 2023-10-16T20:08:34.384 INFO:teuthology.orchestra.run.smithi088.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49073.log 2023-10-16T20:08:34.385 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44310.log 2023-10-16T20:08:34.385 INFO:teuthology.orchestra.run.smithi088.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52265.log.gz 2023-10-16T20:08:34.385 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51713.log 2023-10-16T20:08:34.385 INFO:teuthology.orchestra.run.smithi088.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49073.log.gz 2023-10-16T20:08:34.386 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52962.log 2023-10-16T20:08:34.386 INFO:teuthology.orchestra.run.smithi088.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44310.log.gz 2023-10-16T20:08:34.388 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50894.log 2023-10-16T20:08:34.388 INFO:teuthology.orchestra.run.smithi088.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51713.log.gz 2023-10-16T20:08:34.389 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48295.log 2023-10-16T20:08:34.389 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52962.log.gz 2023-10-16T20:08:34.389 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50894.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50894.log.gz 2023-10-16T20:08:34.389 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48165.log 2023-10-16T20:08:34.390 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49548.log 2023-10-16T20:08:34.390 INFO:teuthology.orchestra.run.smithi088.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48295.log.gz 2023-10-16T20:08:34.392 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44049.log 2023-10-16T20:08:34.392 INFO:teuthology.orchestra.run.smithi088.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48165.log.gz 2023-10-16T20:08:34.392 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47879.log 2023-10-16T20:08:34.393 INFO:teuthology.orchestra.run.smithi088.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49548.log.gz 2023-10-16T20:08:34.393 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44049.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44049.log.gz 2023-10-16T20:08:34.393 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-10-16T20:08:34.393 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46373.log 2023-10-16T20:08:34.394 INFO:teuthology.orchestra.run.smithi088.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47879.log.gz 2023-10-16T20:08:34.395 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43689.log 2023-10-16T20:08:34.395 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45097.log 2023-10-16T20:08:34.396 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46373.log.gz 2023-10-16T20:08:34.396 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43689.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43689.log.gz 2023-10-16T20:08:34.396 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45433.log 2023-10-16T20:08:34.397 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45097.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45097.log.gz 2023-10-16T20:08:34.398 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53684.log 2023-10-16T20:08:34.398 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45433.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45433.log.gz 2023-10-16T20:08:34.398 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49309.log 2023-10-16T20:08:34.399 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-osd.4.log: 91.6% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-10-16T20:08:34.400 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51362.log 2023-10-16T20:08:34.400 INFO:teuthology.orchestra.run.smithi088.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53684.log.gz 2023-10-16T20:08:34.401 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49657.log 2023-10-16T20:08:34.401 INFO:teuthology.orchestra.run.smithi088.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.49309.log.gz 2023-10-16T20:08:34.402 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44206.log 2023-10-16T20:08:34.402 INFO:teuthology.orchestra.run.smithi088.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51362.log.gz 2023-10-16T20:08:34.403 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49657.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49657.log.gz 2023-10-16T20:08:34.403 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49151.log 2023-10-16T20:08:34.404 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43365.log 2023-10-16T20:08:34.404 INFO:teuthology.orchestra.run.smithi088.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44206.log.gz 2023-10-16T20:08:34.404 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49151.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49151.log.gz 2023-10-16T20:08:34.405 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48652.log 2023-10-16T20:08:34.406 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43365.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43365.log.gz 2023-10-16T20:08:34.406 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50376.log 2023-10-16T20:08:34.406 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48652.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48652.log.gz 2023-10-16T20:08:34.407 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51953.log 2023-10-16T20:08:34.407 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50376.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50376.log.gz 2023-10-16T20:08:34.408 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48995.log 2023-10-16T20:08:34.409 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51953.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51953.log.gz 2023-10-16T20:08:34.409 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-10-16T20:08:34.410 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48995.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.48995.log.gz 2023-10-16T20:08:34.410 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47744.log 2023-10-16T20:08:34.411 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51927.log 2023-10-16T20:08:34.412 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47744.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47744.log.gz 2023-10-16T20:08:34.412 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46763.log 2023-10-16T20:08:34.413 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51927.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51927.log.gz 2023-10-16T20:08:34.414 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49177.log 2023-10-16T20:08:34.414 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44671.log 2023-10-16T20:08:34.415 INFO:teuthology.orchestra.run.smithi088.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46763.log.gz 2023-10-16T20:08:34.416 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49177.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49177.log.gz 2023-10-16T20:08:34.416 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47349.log 2023-10-16T20:08:34.417 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52187.log 2023-10-16T20:08:34.418 INFO:teuthology.orchestra.run.smithi088.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44671.log.gz 2023-10-16T20:08:34.418 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47349.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47349.log.gz 2023-10-16T20:08:34.419 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44601.log 2023-10-16T20:08:34.420 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49021.log 2023-10-16T20:08:34.421 INFO:teuthology.orchestra.run.smithi088.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52187.log.gz 2023-10-16T20:08:34.421 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44601.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44601.log.gz 2023-10-16T20:08:34.422 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.46037.log 2023-10-16T20:08:34.422 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48835.log 2023-10-16T20:08:34.423 INFO:teuthology.orchestra.run.smithi088.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49021.log.gz 2023-10-16T20:08:34.424 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.2.46037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52880.log 2023-10-16T20:08:34.424 INFO:teuthology.orchestra.run.smithi088.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.2.46037.log.gz 2023-10-16T20:08:34.425 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48835.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48835.log.gz 2023-10-16T20:08:34.426 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50920.log 2023-10-16T20:08:34.426 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.52880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44445.log 2023-10-16T20:08:34.428 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50920.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50920.log.gz 2023-10-16T20:08:34.428 INFO:teuthology.orchestra.run.smithi088.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.0.52880.log.gz 2023-10-16T20:08:34.429 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49814.log 2023-10-16T20:08:34.429 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44445.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44445.log.gz 2023-10-16T20:08:34.430 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48757.log 2023-10-16T20:08:34.430 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49814.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49814.log.gz 2023-10-16T20:08:34.431 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44258.log 2023-10-16T20:08:34.432 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51258.log 2023-10-16T20:08:34.433 INFO:teuthology.orchestra.run.smithi088.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48757.log.gz 2023-10-16T20:08:34.434 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44258.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44258.log.gz 2023-10-16T20:08:34.434 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48035.log 2023-10-16T20:08:34.435 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51258.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51258.log.gz 2023-10-16T20:08:34.435 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44075.log 2023-10-16T20:08:34.436 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51875.log 2023-10-16T20:08:34.437 INFO:teuthology.orchestra.run.smithi088.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48035.log.gz 2023-10-16T20:08:34.437 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44075.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44075.log.gz 2023-10-16T20:08:34.438 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53632.log 2023-10-16T20:08:34.439 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51875.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51875.log.gz 2023-10-16T20:08:34.439 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43339.log 2023-10-16T20:08:34.440 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43274.log 2023-10-16T20:08:34.441 INFO:teuthology.orchestra.run.smithi088.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53632.log.gz 2023-10-16T20:08:34.442 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43339.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43339.log.gz 2023-10-16T20:08:34.442 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53996.log 2023-10-16T20:08:34.442 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43274.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43274.log.gz 2023-10-16T20:08:34.443 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44672.log 2023-10-16T20:08:34.444 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51180.log 2023-10-16T20:08:34.445 INFO:teuthology.orchestra.run.smithi088.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53996.log.gz 2023-10-16T20:08:34.445 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44672.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44672.log.gz 2023-10-16T20:08:34.446 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50644.log 2023-10-16T20:08:34.446 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51180.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51180.log.gz 2023-10-16T20:08:34.447 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46321.log 2023-10-16T20:08:34.448 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49978.log 2023-10-16T20:08:34.449 INFO:teuthology.orchestra.run.smithi088.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50644.log.gz 2023-10-16T20:08:34.449 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46321.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46321.log.gz 2023-10-16T20:08:34.450 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52662.log 2023-10-16T20:08:34.451 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.49978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47481.log 2023-10-16T20:08:34.452 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52662.log: 91.0% -- replaced with /var/log/ceph/ceph-client.0.49978.log.gz 2023-10-16T20:08:34.453 INFO:teuthology.orchestra.run.smithi088.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52662.log.gz 2023-10-16T20:08:34.453 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50565.log 2023-10-16T20:08:34.454 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52482.log 2023-10-16T20:08:34.454 INFO:teuthology.orchestra.run.smithi088.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47481.log.gz 2023-10-16T20:08:34.455 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50565.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50565.log.gz 2023-10-16T20:08:34.455 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45096.log 2023-10-16T20:08:34.457 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.52482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43660.log 2023-10-16T20:08:34.457 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45096.log: 90.1% -- replaced with /var/log/ceph/ceph-client.0.52482.log.gz 2023-10-16T20:08:34.458 INFO:teuthology.orchestra.run.smithi088.stderr:gzip 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45096.log.gz 2023-10-16T20:08:34.458 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43869.log 2023-10-16T20:08:34.459 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50083.log 2023-10-16T20:08:34.460 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43660.log.gz 2023-10-16T20:08:34.461 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43869.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43869.log.gz 2023-10-16T20:08:34.461 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48535.log 2023-10-16T20:08:34.462 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50083.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50083.log.gz 2023-10-16T20:08:34.463 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46737.log 2023-10-16T20:08:34.464 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48535.log: gzip -5 88.2% --verbose -- replaced with /var/log/ceph/ceph-client.admin.48535.log.gz -- 2023-10-16T20:08:34.464 INFO:teuthology.orchestra.run.smithi088.stderr: /var/log/ceph/ceph-client.admin.53658.log 2023-10-16T20:08:34.465 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46737.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46737.log.gz 2023-10-16T20:08:34.466 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47323.log 2023-10-16T20:08:34.467 INFO:teuthology.orchestra.run.smithi094.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-10-16T20:08:34.467 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53658.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53658.log.gz 2023-10-16T20:08:34.468 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47401.log 2023-10-16T20:08:34.468 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44393.log 2023-10-16T20:08:34.469 INFO:teuthology.orchestra.run.smithi094.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-10-16T20:08:34.470 INFO:teuthology.orchestra.run.smithi088.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47323.log.gz 2023-10-16T20:08:34.470 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47401.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47401.log.gz 2023-10-16T20:08:34.471 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53432.log 2023-10-16T20:08:34.472 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45089.log 2023-10-16T20:08:34.472 INFO:teuthology.orchestra.run.smithi088.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.44393.log.gz 2023-10-16T20:08:34.473 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53432.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53432.log.gz 2023-10-16T20:08:34.473 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47083.log 2023-10-16T20:08:34.474 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45089.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45089.log.gz 2023-10-16T20:08:34.474 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52636.log 2023-10-16T20:08:34.475 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47083.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47083.log.gz 2023-10-16T20:08:34.476 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43392.log 2023-10-16T20:08:34.477 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52636.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52636.log.gz 2023-10-16T20:08:34.477 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47611.log 2023-10-16T20:08:34.478 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43392.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43392.log.gz 2023-10-16T20:08:34.479 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44549.log 2023-10-16T20:08:34.480 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47611.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47611.log.gz 2023-10-16T20:08:34.480 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52396.log 2023-10-16T20:08:34.482 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44549.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44549.log.gz 2023-10-16T20:08:34.482 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51849.log 2023-10-16T20:08:34.483 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52396.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52396.log.gz 2023-10-16T20:08:34.484 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51739.log 2023-10-16T20:08:34.484 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51849.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51849.log.gz 2023-10-16T20:08:34.485 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44883.log 2023-10-16T20:08:34.486 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53944.log 2023-10-16T20:08:34.487 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51739.log.gz 2023-10-16T20:08:34.487 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44883.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44883.log.gz 2023-10-16T20:08:34.488 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49683.log 2023-10-16T20:08:34.489 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53944.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53944.log.gz 2023-10-16T20:08:34.489 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-10-16T20:08:34.491 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49683.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49683.log.gz 2023-10-16T20:08:34.491 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51453.log 2023-10-16T20:08:34.493 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50868.log 2023-10-16T20:08:34.494 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51453.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51453.log.gz 2023-10-16T20:08:34.495 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51232.log 2023-10-16T20:08:34.496 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47957.log 88.2% 2023-10-16T20:08:34.496 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50868.log.gz 2023-10-16T20:08:34.497 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43580.log 2023-10-16T20:08:34.497 INFO:teuthology.orchestra.run.smithi088.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51232.log.gz 2023-10-16T20:08:34.499 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52689.log 2023-10-16T20:08:34.499 INFO:teuthology.orchestra.run.smithi088.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47957.log.gz 2023-10-16T20:08:34.500 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43793.log 2023-10-16T20:08:34.500 INFO:teuthology.orchestra.run.smithi088.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43580.log.gz 2023-10-16T20:08:34.501 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52689.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52689.log.gz 2023-10-16T20:08:34.502 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47827.log 2023-10-16T20:08:34.503 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38249.log 2023-10-16T20:08:34.503 INFO:teuthology.orchestra.run.smithi088.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43793.log.gz 2023-10-16T20:08:34.504 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47853.log 2023-10-16T20:08:34.505 INFO:teuthology.orchestra.run.smithi088.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47827.log.gz 2023-10-16T20:08:34.506 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.38249.log: gzip -5 --verbose -- 88.4% /var/log/ceph/ceph-client.admin.49047.log 2023-10-16T20:08:34.506 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38249.log.gz 2023-10-16T20:08:34.507 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47853.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47853.log.gz 2023-10-16T20:08:34.508 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48347.log 2023-10-16T20:08:34.509 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49047.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49047.log.gz 2023-10-16T20:08:34.509 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51597.log 2023-10-16T20:08:34.510 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53970.log 2023-10-16T20:08:34.511 INFO:teuthology.orchestra.run.smithi088.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48347.log.gz 2023-10-16T20:08:34.512 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45537.log 2023-10-16T20:08:34.512 INFO:teuthology.orchestra.run.smithi088.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.51597.log.gz 2023-10-16T20:08:34.513 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53970.log: gzip -5 87.4% --verbose -- replaced with /var/log/ceph/ceph-client.admin.53970.log.gz -- 2023-10-16T20:08:34.513 INFO:teuthology.orchestra.run.smithi088.stderr: /var/log/ceph/ceph-client.admin.53710.log 2023-10-16T20:08:34.515 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48503.log 2023-10-16T20:08:34.515 INFO:teuthology.orchestra.run.smithi088.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45537.log.gz 2023-10-16T20:08:34.516 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53710.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53710.log.gz 2023-10-16T20:08:34.517 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54048.log 2023-10-16T20:08:34.518 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44284.log 2023-10-16T20:08:34.518 INFO:teuthology.orchestra.run.smithi088.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48503.log.gz 2023-10-16T20:08:34.519 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.54048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50539.log 2023-10-16T20:08:34.519 INFO:teuthology.orchestra.run.smithi088.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54048.log.gz 2023-10-16T20:08:34.520 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51518.log 2023-10-16T20:08:34.521 INFO:teuthology.orchestra.run.smithi088.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44284.log.gz 2023-10-16T20:08:34.522 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50539.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50539.log.gz 2023-10-16T20:08:34.522 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48731.log 2023-10-16T20:08:34.523 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44153.log 2023-10-16T20:08:34.524 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51518.log.gz 2023-10-16T20:08:34.525 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53606.log 2023-10-16T20:08:34.525 INFO:teuthology.orchestra.run.smithi088.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48731.log.gz 2023-10-16T20:08:34.526 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44153.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-10-16T20:08:34.527 INFO:teuthology.orchestra.run.smithi088.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44153.log.gz 2023-10-16T20:08:34.528 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53606.log: gzip 88.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.53606.log.gz --verbose 2023-10-16T20:08:34.528 INFO:teuthology.orchestra.run.smithi088.stderr: -- /var/log/ceph/ceph-client.admin.53484.log 2023-10-16T20:08:34.529 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46269.log 2023-10-16T20:08:34.531 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53484.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.53484.log.gz 2023-10-16T20:08:34.531 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.51421.log 2023-10-16T20:08:34.532 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51024.log 2023-10-16T20:08:34.533 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46269.log.gz 2023-10-16T20:08:34.534 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51421.log: 87.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.51421.log.gz 2023-10-16T20:08:34.534 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47110.log 2023-10-16T20:08:34.535 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48600.log 2023-10-16T20:08:34.536 INFO:teuthology.orchestra.run.smithi088.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51024.log.gz 2023-10-16T20:08:34.536 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47110.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47110.log.gz 2023-10-16T20:08:34.537 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43819.log 2023-10-16T20:08:34.538 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50617.log 2023-10-16T20:08:34.538 INFO:teuthology.orchestra.run.smithi088.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48600.log.gz 2023-10-16T20:08:34.540 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52427.log 2023-10-16T20:08:34.540 INFO:teuthology.orchestra.run.smithi088.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.43819.log.gz 2023-10-16T20:08:34.541 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50617.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50617.log.gz 2023-10-16T20:08:34.541 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52370.log 2023-10-16T20:08:34.543 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49125.log 2023-10-16T20:08:34.543 INFO:teuthology.orchestra.run.smithi088.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52427.log.gz 2023-10-16T20:08:34.544 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52370.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52370.log.gz 2023-10-16T20:08:34.544 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44523.log 2023-10-16T20:08:34.545 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49125.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49125.log.gz 2023-10-16T20:08:34.546 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49203.log 2023-10-16T20:08:34.547 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44523.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44523.log.gz 2023-10-16T20:08:34.548 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49496.log 2023-10-16T20:08:34.549 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48917.log 2023-10-16T20:08:34.549 INFO:teuthology.orchestra.run.smithi088.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49203.log.gz 2023-10-16T20:08:34.550 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49496.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49496.log.gz 2023-10-16T20:08:34.551 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50214.log 2023-10-16T20:08:34.552 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48917.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48917.log.gz 2023-10-16T20:08:34.552 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44179.log 2023-10-16T20:08:34.554 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46243.log 2023-10-16T20:08:34.554 INFO:teuthology.orchestra.run.smithi088.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50214.log.gz 2023-10-16T20:08:34.555 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44179.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44179.log.gz 2023-10-16T20:08:34.557 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46243.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46243.log.gz 2023-10-16T20:08:34.557 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47057.log 2023-10-16T20:08:34.558 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45589.log 2023-10-16T20:08:34.560 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45329.log 2023-10-16T20:08:34.560 INFO:teuthology.orchestra.run.smithi088.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47057.log.gz 2023-10-16T20:08:34.561 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44877.log 2023-10-16T20:08:34.562 INFO:teuthology.orchestra.run.smithi088.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45589.log.gz 2023-10-16T20:08:34.563 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45329.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45329.log.gz 2023-10-16T20:08:34.564 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46900.log 2023-10-16T20:08:34.564 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47770.log 2023-10-16T20:08:34.565 INFO:teuthology.orchestra.run.smithi088.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44877.log.gz 2023-10-16T20:08:34.566 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48809.log 2023-10-16T20:08:34.566 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46900.log.gz 2023-10-16T20:08:34.567 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47770.log: gzip -5 --verbose 89.4% -- /var/log/ceph/ceph-client.admin.47690.log -- replaced with /var/log/ceph/ceph-client.admin.47770.log.gz 2023-10-16T20:08:34.568 INFO:teuthology.orchestra.run.smithi088.stderr: 2023-10-16T20:08:34.569 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43741.log 2023-10-16T20:08:34.569 INFO:teuthology.orchestra.run.smithi088.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48809.log.gz 2023-10-16T20:08:34.570 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47690.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47690.log.gz 2023-10-16T20:08:34.571 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43741.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43741.log.gz 2023-10-16T20:08:34.572 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38147.log 2023-10-16T20:08:34.573 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44888.log 2023-10-16T20:08:34.575 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.38147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43995.log 2023-10-16T20:08:34.575 INFO:teuthology.orchestra.run.smithi088.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38147.log.gz 2023-10-16T20:08:34.576 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52825.log 2023-10-16T20:08:34.577 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44888.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44888.log.gz/var/log/ceph/ceph-client.admin.43995.log: 2023-10-16T20:08:34.581 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52825.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52825.log.gz 2023-10-16T20:08:34.581 INFO:teuthology.orchestra.run.smithi088.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.43995.log.gz 2023-10-16T20:08:34.582 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45091.log 2023-10-16T20:08:34.582 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53814.log 2023-10-16T20:08:34.583 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47427.log 2023-10-16T20:08:34.583 INFO:teuthology.orchestra.run.smithi088.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45091.log.gz 2023-10-16T20:08:34.584 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53814.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53814.log.gz 2023-10-16T20:08:34.585 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47427.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47427.log.gz 2023-10-16T20:08:34.586 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52161.log 2023-10-16T20:08:34.587 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-16T20:08:34.589 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47136.log 2023-10-16T20:08:34.589 INFO:teuthology.orchestra.run.smithi088.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52161.log.gz 2023-10-16T20:08:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph.log: 85.2% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-16T20:08:34.591 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47136.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47136.log.gz 2023-10-16T20:08:34.592 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53014.log 2023-10-16T20:08:34.593 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54022.log 2023-10-16T20:08:34.595 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50135.log 2023-10-16T20:08:34.595 INFO:teuthology.orchestra.run.smithi088.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53014.log.gz 2023-10-16T20:08:34.596 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.54022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48969.log 2023-10-16T20:08:34.597 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54022.log.gz 2023-10-16T20:08:34.598 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54074.log 2023-10-16T20:08:34.598 INFO:teuthology.orchestra.run.smithi088.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50135.log.gz 2023-10-16T20:08:34.599 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52558.log 2023-10-16T20:08:34.600 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48969.log.gz 2023-10-16T20:08:34.600 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.54074.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54074.log.gz 2023-10-16T20:08:34.602 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52558.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52558.log.gz 2023-10-16T20:08:34.603 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45946.log 2023-10-16T20:08:34.604 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44675.log 2023-10-16T20:08:34.605 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54126.log 2023-10-16T20:08:34.606 INFO:teuthology.orchestra.run.smithi088.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45946.log.gz 2023-10-16T20:08:34.607 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44675.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44675.log.gz 2023-10-16T20:08:34.607 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45721.log 2023-10-16T20:08:34.608 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.54126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50188.log 2023-10-16T20:08:34.609 INFO:teuthology.orchestra.run.smithi088.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54126.log.gz 2023-10-16T20:08:34.610 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48477.log 2023-10-16T20:08:34.610 INFO:teuthology.orchestra.run.smithi088.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.45721.log.gz 2023-10-16T20:08:34.611 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50188.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50188.log.gz 2023-10-16T20:08:34.612 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50267.log 2023-10-16T20:08:34.613 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48477.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48477.log.gz 2023-10-16T20:08:34.614 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48705.log 2023-10-16T20:08:34.614 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50267.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50267.log.gz 2023-10-16T20:08:34.615 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45485.log 2023-10-16T20:08:34.617 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48626.log 2023-10-16T20:08:34.617 INFO:teuthology.orchestra.run.smithi088.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48705.log.gz 2023-10-16T20:08:34.618 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51284.log 87.9% 2023-10-16T20:08:34.618 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45485.log.gz 2023-10-16T20:08:34.620 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52084.log 2023-10-16T20:08:34.620 INFO:teuthology.orchestra.run.smithi088.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48626.log.gz 2023-10-16T20:08:34.621 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46685.log 2023-10-16T20:08:34.622 INFO:teuthology.orchestra.run.smithi088.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51284.log.gz 2023-10-16T20:08:34.623 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.52084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44662.log 2023-10-16T20:08:34.624 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46685.log: 88.0% 91.6% -- replaced with /var/log/ceph/ceph-client.0.52084.log.gz 2023-10-16T20:08:34.625 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46685.log.gz 2023-10-16T20:08:34.626 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44662.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44662.log.gz 2023-10-16T20:08:34.626 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51388.log 2023-10-16T20:08:34.627 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49923.log 2023-10-16T20:08:34.629 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51388.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51388.log.gz 2023-10-16T20:08:34.631 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49923.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49923.log.gz 2023-10-16T20:08:34.631 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53333.log 2023-10-16T20:08:34.632 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45080.log 2023-10-16T20:08:34.633 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51154.log 2023-10-16T20:08:34.634 INFO:teuthology.orchestra.run.smithi088.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53333.log.gz 2023-10-16T20:08:34.635 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45080.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45080.log.gz 2023-10-16T20:08:34.636 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51154.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51154.log.gz 2023-10-16T20:08:34.637 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52741.log 2023-10-16T20:08:34.638 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46978.log 2023-10-16T20:08:34.640 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52715.log 2023-10-16T20:08:34.640 INFO:teuthology.orchestra.run.smithi088.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52741.log.gz 2023-10-16T20:08:34.641 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46978.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46978.log.gz 2023-10-16T20:08:34.642 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52715.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52715.log.gz 2023-10-16T20:08:34.643 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47219.log 2023-10-16T20:08:34.644 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47559.log 2023-10-16T20:08:34.646 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48783.log 2023-10-16T20:08:34.646 INFO:teuthology.orchestra.run.smithi088.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47219.log.gz 2023-10-16T20:08:34.647 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47559.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47559.log.gz 2023-10-16T20:08:34.648 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48783.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48783.log.gz 2023-10-16T20:08:34.649 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43554.log 2023-10-16T20:08:34.650 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44872.log 2023-10-16T20:08:34.652 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52793.log 2023-10-16T20:08:34.652 INFO:teuthology.orchestra.run.smithi088.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43554.log.gz 2023-10-16T20:08:34.653 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54102.log 2023-10-16T20:08:34.654 INFO:teuthology.orchestra.run.smithi088.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44872.log.gz 2023-10-16T20:08:34.655 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52793.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52793.log.gz 2023-10-16T20:08:34.656 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.54102.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54102.log.gz 2023-10-16T20:08:34.657 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44664.log 2023-10-16T20:08:34.658 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46399.log 2023-10-16T20:08:34.660 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46217.log 2023-10-16T20:08:34.660 INFO:teuthology.orchestra.run.smithi088.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44664.log.gz 2023-10-16T20:08:34.661 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46399.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46399.log.gz 2023-10-16T20:08:34.661 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53387.log 2023-10-16T20:08:34.663 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53554.log 2023-10-16T20:08:34.663 INFO:teuthology.orchestra.run.smithi088.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46217.log.gz 2023-10-16T20:08:34.664 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47005.log 2023-10-16T20:08:34.665 INFO:teuthology.orchestra.run.smithi088.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53387.log.gz 2023-10-16T20:08:34.666 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53554.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53554.log.gz 2023-10-16T20:08:34.667 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47005.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47005.log.gz 2023-10-16T20:08:34.668 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44232.log 2023-10-16T20:08:34.669 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47664.log 2023-10-16T20:08:34.671 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44232.log: gzip 88.8% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.44232.log.gz -- 2023-10-16T20:08:34.671 INFO:teuthology.orchestra.run.smithi088.stderr: /var/log/ceph/ceph-client.admin.51679.log 2023-10-16T20:08:34.672 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47664.log: gzip -5 88.4% --verbose -- replaced with /var/log/ceph/ceph-client.admin.47664.log.gz -- 2023-10-16T20:08:34.672 INFO:teuthology.orchestra.run.smithi088.stderr: /var/log/ceph/ceph-client.admin.53255.log 2023-10-16T20:08:34.674 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50293.log 2023-10-16T20:08:34.674 INFO:teuthology.orchestra.run.smithi088.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51679.log.gz 2023-10-16T20:08:34.675 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53255.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53255.log.gz 2023-10-16T20:08:34.676 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50293.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50293.log.gz 2023-10-16T20:08:34.677 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49229.log 2023-10-16T20:08:34.678 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43444.log 2023-10-16T20:08:34.680 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49229.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49229.log.gz 2023-10-16T20:08:34.680 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50434.log 2023-10-16T20:08:34.681 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43444.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43444.log.gz 2023-10-16T20:08:34.681 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53281.log 2023-10-16T20:08:34.683 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.50434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49414.log 2023-10-16T20:08:34.684 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53281.log: 88.3% 91.3% -- replaced with /var/log/ceph/ceph-client.0.50434.log.gz 2023-10-16T20:08:34.685 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53281.log.gz 2023-10-16T20:08:34.688 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.49414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46711.log 2023-10-16T20:08:34.688 INFO:teuthology.orchestra.run.smithi088.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.0.49414.log.gz 2023-10-16T20:08:34.689 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44881.log 2023-10-16T20:08:34.691 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53203.log 2023-10-16T20:08:34.691 INFO:teuthology.orchestra.run.smithi088.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46711.log.gz 2023-10-16T20:08:34.692 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44881.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44881.log.gz 2023-10-16T20:08:34.693 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53203.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53203.log.gz 2023-10-16T20:08:34.695 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53892.log 2023-10-16T20:08:34.695 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49099.log 2023-10-16T20:08:34.697 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53892.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-16T20:08:34.698 INFO:teuthology.orchestra.run.smithi088.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53892.log.gz 2023-10-16T20:08:34.698 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49099.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49099.log.gz 2023-10-16T20:08:34.699 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45615.log 2023-10-16T20:08:34.701 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49605.log 2023-10-16T20:08:34.703 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47297.log 2023-10-16T20:08:34.703 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45615.log.gz 2023-10-16T20:08:34.704 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49605.log: 89.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-16T20:08:34.704 INFO:teuthology.orchestra.run.smithi088.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49605.log.gz 2023-10-16T20:08:34.706 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47297.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47297.log.gz 2023-10-16T20:08:34.706 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-10-16T20:08:34.707 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46425.log 2023-10-16T20:08:34.709 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48399.log 2023-10-16T20:08:34.710 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46425.log: 91.6% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 88.7% 2023-10-16T20:08:34.711 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46425.log.gz 2023-10-16T20:08:34.711 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48399.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48399.log.gz 2023-10-16T20:08:34.712 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49892.log 2023-10-16T20:08:34.713 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49578.log 2023-10-16T20:08:34.715 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53177.log 2023-10-16T20:08:34.715 INFO:teuthology.orchestra.run.smithi088.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49892.log.gz 2023-10-16T20:08:34.717 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49578.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49578.log.gz 2023-10-16T20:08:34.717 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50696.log 2023-10-16T20:08:34.718 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53177.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53177.log.gz 2023-10-16T20:08:34.719 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50696.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50696.log.gz 2023-10-16T20:08:34.720 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-10-16T20:08:34.721 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43767.log 2023-10-16T20:08:34.723 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47453.log 2023-10-16T20:08:34.725 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43767.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43767.log.gz 2023-10-16T20:08:34.725 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47453.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47453.log.gz 2023-10-16T20:08:34.726 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43418.log 2023-10-16T20:08:34.728 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45902.log 2023-10-16T20:08:34.729 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43418.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43418.log.gz 2023-10-16T20:08:34.731 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.45902.log: 90.3% -- replaced with /var/log/ceph/ceph-client.0.45902.log.gz 2023-10-16T20:08:34.732 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50998.log 2023-10-16T20:08:34.733 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52292.log 2023-10-16T20:08:34.735 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50998.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50998.log.gz 2023-10-16T20:08:34.736 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52292.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52292.log.gz 2023-10-16T20:08:34.736 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43496.log 2023-10-16T20:08:34.738 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48217.log 2023-10-16T20:08:34.739 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43496.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43496.log.gz 2023-10-16T20:08:34.741 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48217.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48217.log.gz 2023-10-16T20:08:34.742 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47801.log 2023-10-16T20:08:34.743 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48321.log 2023-10-16T20:08:34.744 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47801.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47801.log.gz 2023-10-16T20:08:34.746 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48321.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48321.log.gz 2023-10-16T20:08:34.747 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46581.log 2023-10-16T20:08:34.748 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45459.log 2023-10-16T20:08:34.749 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46581.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46581.log.gz 2023-10-16T20:08:34.750 INFO:teuthology.orchestra.run.smithi088.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-10-16T20:08:34.751 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45459.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45459.log.gz 2023-10-16T20:08:34.751 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46295.log 2023-10-16T20:08:34.753 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48191.log 2023-10-16T20:08:34.754 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46295.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46295.log.gz 2023-10-16T20:08:34.754 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45085.log 2023-10-16T20:08:34.756 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46191.log 2023-10-16T20:08:34.757 INFO:teuthology.orchestra.run.smithi088.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48191.log.gz 2023-10-16T20:08:34.757 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45085.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45085.log.gz 2023-10-16T20:08:34.758 INFO:teuthology.orchestra.run.smithi088.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-10-16T20:08:34.759 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46191.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46191.log.gz 2023-10-16T20:08:34.759 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43634.log 2023-10-16T20:08:34.760 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51901.log 2023-10-16T20:08:34.761 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43634.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43634.log.gz 2023-10-16T20:08:34.762 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44627.log 2023-10-16T20:08:34.763 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51901.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51901.log.gz 2023-10-16T20:08:34.764 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51823.log 2023-10-16T20:08:34.766 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49283.log 2023-10-16T20:08:34.766 INFO:teuthology.orchestra.run.smithi088.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44627.log.gz 2023-10-16T20:08:34.767 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45782.log 2023-10-16T20:08:34.768 INFO:teuthology.orchestra.run.smithi088.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51823.log.gz 2023-10-16T20:08:34.769 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52134.log 2023-10-16T20:08:34.769 INFO:teuthology.orchestra.run.smithi088.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49283.log.gz 2023-10-16T20:08:34.770 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45782.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45782.log.gz 2023-10-16T20:08:34.770 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49468.log 2023-10-16T20:08:34.771 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51769.log 2023-10-16T20:08:34.771 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52134.log.gz 2023-10-16T20:08:34.772 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49468.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49468.log.gz 2023-10-16T20:08:34.773 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47533.log 2023-10-16T20:08:34.773 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51769.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51769.log.gz 2023-10-16T20:08:34.774 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48243.log 2023-10-16T20:08:34.775 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47533.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47533.log.gz 2023-10-16T20:08:34.775 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50161.log 2023-10-16T20:08:34.777 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51076.log 2023-10-16T20:08:34.777 INFO:teuthology.orchestra.run.smithi088.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48243.log.gz 2023-10-16T20:08:34.778 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50161.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50161.log.gz 2023-10-16T20:08:34.778 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50774.log 2023-10-16T20:08:34.779 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48943.log 2023-10-16T20:08:34.780 INFO:teuthology.orchestra.run.smithi088.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51076.log.gz 2023-10-16T20:08:34.780 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50774.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50774.log.gz 2023-10-16T20:08:34.781 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47718.log 2023-10-16T20:08:34.782 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52988.log 2023-10-16T20:08:34.782 INFO:teuthology.orchestra.run.smithi088.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48943.log.gz 2023-10-16T20:08:34.786 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47718.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47718.log.gz 2023-10-16T20:08:34.787 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46503.log 2023-10-16T20:08:34.787 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52344.log 2023-10-16T20:08:34.788 INFO:teuthology.orchestra.run.smithi088.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52988.log.gz 2023-10-16T20:08:34.788 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46503.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46503.log.gz 2023-10-16T20:08:34.788 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51050.log 2023-10-16T20:08:34.790 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52344.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52344.log.gz 2023-10-16T20:08:34.790 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.33220.log 2023-10-16T20:08:34.790 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51050.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51050.log.gz 2023-10-16T20:08:34.790 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50670.log 2023-10-16T20:08:34.791 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph.tmp-client.admin.33220.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.33220.log.gz 2023-10-16T20:08:34.792 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45094.log 2023-10-16T20:08:34.792 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50670.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50670.log.gz 2023-10-16T20:08:34.793 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43173.log 2023-10-16T20:08:34.793 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45094.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45094.log.gz 2023-10-16T20:08:34.794 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50057.log 2023-10-16T20:08:34.795 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43173.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43173.log.gz 2023-10-16T20:08:34.795 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46081.log 2023-10-16T20:08:34.797 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50057.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50057.log.gz 2023-10-16T20:08:34.797 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45303.log 2023-10-16T20:08:34.798 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46081.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46081.log.gz 2023-10-16T20:08:34.798 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44680.log 2023-10-16T20:08:34.800 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45303.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45303.log.gz 2023-10-16T20:08:34.800 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53736.log 2023-10-16T20:08:34.801 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44680.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44680.log.gz 2023-10-16T20:08:34.802 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53840.log 2023-10-16T20:08:34.803 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53736.log: gzip -5 87.9% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.53736.log.gz /var/log/ceph/ceph-client.admin.53307.log 2023-10-16T20:08:34.803 INFO:teuthology.orchestra.run.smithi088.stderr: 2023-10-16T20:08:34.805 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53840.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53840.log.gz 2023-10-16T20:08:34.805 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50591.log 2023-10-16T20:08:34.806 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53307.log: 89.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.53307.log.gz 2023-10-16T20:08:34.806 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44889.log 2023-10-16T20:08:34.808 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47167.log 2023-10-16T20:08:34.808 INFO:teuthology.orchestra.run.smithi088.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50591.log.gz 2023-10-16T20:08:34.809 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50031.log 2023-10-16T20:08:34.810 INFO:teuthology.orchestra.run.smithi088.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44889.log.gz 2023-10-16T20:08:34.811 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47167.log: gzip 88.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.47167.log.gz --verbose 2023-10-16T20:08:34.811 INFO:teuthology.orchestra.run.smithi088.stderr: -- /var/log/ceph/ceph-client.admin.53099.log 2023-10-16T20:08:34.812 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46107.log 2023-10-16T20:08:34.812 INFO:teuthology.orchestra.run.smithi088.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50031.log.gz 2023-10-16T20:08:34.814 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53099.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53099.log.gz 2023-10-16T20:08:34.814 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48061.log 2023-10-16T20:08:34.815 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46107.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.46107.log.gz 2023-10-16T20:08:34.815 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.38028.log 2023-10-16T20:08:34.817 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48061.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48061.log.gz 2023-10-16T20:08:34.817 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45667.log 2023-10-16T20:08:34.818 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.38028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53229.log 2023-10-16T20:08:34.819 INFO:teuthology.orchestra.run.smithi088.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38028.log.gz 2023-10-16T20:08:34.820 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53040.log 2023-10-16T20:08:34.820 INFO:teuthology.orchestra.run.smithi088.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45667.log.gz 2023-10-16T20:08:34.821 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53229.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53229.log.gz 2023-10-16T20:08:34.821 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44869.log 2023-10-16T20:08:34.823 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48889.log 2023-10-16T20:08:34.823 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53040.log.gz 2023-10-16T20:08:34.823 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44869.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44869.log.gz 2023-10-16T20:08:34.824 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52531.log 2023-10-16T20:08:34.825 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48889.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48889.log.gz 2023-10-16T20:08:34.825 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46815.log 2023-10-16T20:08:34.827 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52531.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52531.log.gz 2023-10-16T20:08:34.827 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47507.log 2023-10-16T20:08:34.829 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46815.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46815.log.gz 2023-10-16T20:08:34.829 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45693.log 2023-10-16T20:08:34.830 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47507.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47507.log.gz 2023-10-16T20:08:34.830 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52610.log 2023-10-16T20:08:34.831 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45693.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45693.log.gz 2023-10-16T20:08:34.832 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46926.log 2023-10-16T20:08:34.833 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52610.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52610.log.gz 2023-10-16T20:08:34.833 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49522.log 2023-10-16T20:08:34.834 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46926.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46926.log.gz 2023-10-16T20:08:34.835 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54152.log 2023-10-16T20:08:34.837 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48679.log 2023-10-16T20:08:34.837 INFO:teuthology.orchestra.run.smithi088.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49522.log.gz 2023-10-16T20:08:34.838 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.54152.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54152.log.gz 2023-10-16T20:08:34.838 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46529.log 2023-10-16T20:08:34.839 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45641.log 2023-10-16T20:08:34.839 INFO:teuthology.orchestra.run.smithi088.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48679.log.gz 2023-10-16T20:08:34.840 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46529.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46529.log.gz 2023-10-16T20:08:34.841 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46789.log 2023-10-16T20:08:34.842 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45641.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45641.log.gz 2023-10-16T20:08:34.842 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47193.log 2023-10-16T20:08:34.843 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46789.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46789.log.gz 2023-10-16T20:08:34.844 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53918.log 2023-10-16T20:08:34.845 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47193.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47193.log.gz 2023-10-16T20:08:34.845 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49735.log 2023-10-16T20:08:34.846 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53918.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53918.log.gz 2023-10-16T20:08:34.847 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48425.log 2023-10-16T20:08:34.848 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49735.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49735.log.gz 2023-10-16T20:08:34.848 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53510.log 2023-10-16T20:08:34.850 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48425.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48425.log.gz 2023-10-16T20:08:34.850 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50842.log 2023-10-16T20:08:34.851 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53510.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53510.log.gz 2023-10-16T20:08:34.851 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53788.log 2023-10-16T20:08:34.852 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50842.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50842.log.gz 2023-10-16T20:08:34.853 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51797.log 2023-10-16T20:08:34.854 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50109.log 2023-10-16T20:08:34.854 INFO:teuthology.orchestra.run.smithi088.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53788.log.gz 2023-10-16T20:08:34.855 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51797.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51797.log.gz 2023-10-16T20:08:34.856 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44885.log 2023-10-16T20:08:34.856 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50109.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.50109.log.gz 2023-10-16T20:08:34.857 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49762.log 2023-10-16T20:08:34.858 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44885.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44885.log.gz 2023-10-16T20:08:34.859 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46607.log 2023-10-16T20:08:34.859 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44681.log 2023-10-16T20:08:34.860 INFO:teuthology.orchestra.run.smithi088.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49762.log.gz 2023-10-16T20:08:34.861 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46607.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46607.log.gz 2023-10-16T20:08:34.861 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49788.log 2023-10-16T20:08:34.862 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44681.log: gzip 88.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.44681.log.gz --verbose 2023-10-16T20:08:34.863 INFO:teuthology.orchestra.run.smithi088.stderr: -- /var/log/ceph/ceph-client.admin.44419.log 2023-10-16T20:08:34.864 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53866.log 2023-10-16T20:08:34.864 INFO:teuthology.orchestra.run.smithi088.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49788.log.gz 2023-10-16T20:08:34.865 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44419.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44419.log.gz 2023-10-16T20:08:34.866 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52767.log 2023-10-16T20:08:34.866 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49631.log 2023-10-16T20:08:34.867 INFO:teuthology.orchestra.run.smithi088.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53866.log.gz 2023-10-16T20:08:34.868 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52767.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52767.log.gz 2023-10-16T20:08:34.868 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46165.log 2023-10-16T20:08:34.869 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49631.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49631.log.gz 2023-10-16T20:08:34.870 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46952.log 2023-10-16T20:08:34.871 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46165.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46165.log.gz 2023-10-16T20:08:34.872 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46659.log 2023-10-16T20:08:34.872 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46952.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46952.log.gz 2023-10-16T20:08:34.873 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44497.log 2023-10-16T20:08:34.875 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50350.log 2023-10-16T20:08:34.875 INFO:teuthology.orchestra.run.smithi088.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46659.log.gz 2023-10-16T20:08:34.876 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44497.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44497.log.gz 2023-10-16T20:08:34.877 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50350.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50350.log.gz 2023-10-16T20:08:34.878 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51128.log 2023-10-16T20:08:34.878 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51544.log 2023-10-16T20:08:34.880 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51128.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51128.log.gz 2023-10-16T20:08:34.880 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44101.log 2023-10-16T20:08:34.882 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51544.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51544.log.gz 2023-10-16T20:08:34.882 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47375.log 2023-10-16T20:08:34.883 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45355.log 2023-10-16T20:08:34.884 INFO:teuthology.orchestra.run.smithi088.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44101.log.gz 2023-10-16T20:08:34.884 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47375.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47375.log.gz 2023-10-16T20:08:34.885 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47585.log 2023-10-16T20:08:34.886 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50487.log 2023-10-16T20:08:34.886 INFO:teuthology.orchestra.run.smithi088.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45355.log.gz 2023-10-16T20:08:34.887 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47585.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47585.log.gz 2023-10-16T20:08:34.887 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49840.log 2023-10-16T20:08:34.889 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51651.log 2023-10-16T20:08:34.889 INFO:teuthology.orchestra.run.smithi088.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50487.log.gz 2023-10-16T20:08:34.890 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49840.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49840.log.gz 2023-10-16T20:08:34.890 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44367.log 2023-10-16T20:08:34.891 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43470.log 2023-10-16T20:08:34.891 INFO:teuthology.orchestra.run.smithi088.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51651.log.gz 2023-10-16T20:08:34.892 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44367.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44367.log.gz 2023-10-16T20:08:34.893 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43470.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43470.log.gz 2023-10-16T20:08:35.043 INFO:teuthology.orchestra.run.smithi094.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-10-16T20:08:35.135 INFO:teuthology.orchestra.run.smithi094.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-10-16T20:08:35.141 INFO:teuthology.orchestra.run.smithi094.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-10-16T20:08:35.162 INFO:teuthology.orchestra.run.smithi088.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-10-16T20:08:35.173 INFO:teuthology.orchestra.run.smithi094.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-10-16T20:08:35.175 INFO:teuthology.orchestra.run.smithi094.stderr: 2023-10-16T20:08:35.175 INFO:teuthology.orchestra.run.smithi094.stderr:real 0m0.883s 2023-10-16T20:08:35.176 INFO:teuthology.orchestra.run.smithi094.stderr:user 0m3.283s 2023-10-16T20:08:35.176 INFO:teuthology.orchestra.run.smithi094.stderr:sys 0m0.358s 2023-10-16T20:08:35.270 INFO:teuthology.orchestra.run.smithi088.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-10-16T20:08:35.342 INFO:teuthology.orchestra.run.smithi088.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-10-16T20:08:35.384 INFO:teuthology.orchestra.run.smithi088.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-10-16T20:08:35.386 INFO:teuthology.orchestra.run.smithi088.stderr: 2023-10-16T20:08:35.386 INFO:teuthology.orchestra.run.smithi088.stderr:real 0m1.112s 2023-10-16T20:08:35.386 INFO:teuthology.orchestra.run.smithi088.stderr:user 0m3.981s 2023-10-16T20:08:35.387 INFO:teuthology.orchestra.run.smithi088.stderr:sys 0m1.063s 2023-10-16T20:08:35.388 INFO:tasks.ceph:Archiving logs... 2023-10-16T20:08:35.389 DEBUG:teuthology.misc:Transferring archived files from smithi088:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-10-16_18:49:15-fs-rishabh-2023oct13-testing-default-smithi/7430091/remote/smithi088/log 2023-10-16T20:08:35.390 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-16T20:08:36.120 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-10-16_18:49:15-fs-rishabh-2023oct13-testing-default-smithi/7430091/remote/smithi094/log 2023-10-16T20:08:36.121 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-16T20:08:36.522 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-16T20:08:36.532 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-16T20:08:36.532 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-16T20:08:36.536 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-16T20:08:36.612 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-16T20:08:36.621 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-16T20:08:36.622 DEBUG:teuthology.orchestra.run.smithi088:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-16T20:08:36.626 DEBUG:teuthology.orchestra.run.smithi094:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-16T20:08:36.638 INFO:teuthology.orchestra.run.smithi088.stdout: remote refid st t when poll reach delay offset jitter 2023-10-16T20:08:36.639 INFO:teuthology.orchestra.run.smithi088.stdout:============================================================================== 2023-10-16T20:08:36.639 INFO:teuthology.orchestra.run.smithi088.stdout:*hv01.front.sepi 44.4.53.2 2 u 52 64 177 0.147 -0.922 2.379 2023-10-16T20:08:36.639 INFO:teuthology.orchestra.run.smithi088.stdout:+hv02.front.sepi 96.245.170.99 3 u 55 64 177 0.127 +1.318 2.382 2023-10-16T20:08:36.640 INFO:teuthology.orchestra.run.smithi088.stdout:+hv03.front.sepi 74.6.168.72 3 u 52 64 177 0.142 -4.032 2.073 2023-10-16T20:08:36.640 INFO:teuthology.orchestra.run.smithi088.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-16T20:08:36.642 INFO:teuthology.orchestra.run.smithi094.stdout: remote refid st t when poll reach delay offset jitter 2023-10-16T20:08:36.643 INFO:teuthology.orchestra.run.smithi094.stdout:============================================================================== 2023-10-16T20:08:36.643 INFO:teuthology.orchestra.run.smithi094.stdout:*hv01.front.sepi 44.4.53.2 2 u 49 64 177 0.057 -0.587 1.704 2023-10-16T20:08:36.643 INFO:teuthology.orchestra.run.smithi094.stdout:+hv02.front.sepi 96.245.170.99 3 u 54 64 177 0.111 +2.234 1.381 2023-10-16T20:08:36.643 INFO:teuthology.orchestra.run.smithi094.stdout:+hv03.front.sepi 74.6.168.72 3 u 56 64 177 0.079 -3.334 1.698 2023-10-16T20:08:36.644 INFO:teuthology.orchestra.run.smithi094.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-16T20:08:36.645 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-16T20:08:36.653 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-16T20:08:36.654 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-16T20:08:36.663 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-16T20:08:36.678 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-16T20:08:36.687 INFO:teuthology.task.internal:Duration was 976.049840 seconds 2023-10-16T20:08:36.688 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-16T20:08:36.696 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-16T20:08:36.697 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-16T20:08:36.700 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-16T20:08:36.809 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-16T20:08:36.810 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi088.front.sepia.ceph.com 2023-10-16T20:08:36.810 DEBUG:teuthology.orchestra.run.smithi088:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-16T20:08:36.834 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi094.front.sepia.ceph.com 2023-10-16T20:08:36.835 DEBUG:teuthology.orchestra.run.smithi094:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-16T20:08:36.858 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-16T20:08:36.858 DEBUG:teuthology.orchestra.run.smithi088:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-16T20:08:36.882 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-16T20:08:36.937 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-16T20:08:36.938 DEBUG:teuthology.orchestra.run.smithi088:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-16T20:08:36.962 DEBUG:teuthology.orchestra.run.smithi094:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-16T20:08:37.050 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-16T20:08:37.069 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-16T20:08:37.070 DEBUG:teuthology.orchestra.run.smithi088:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-16T20:08:37.092 DEBUG:teuthology.orchestra.run.smithi094:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-16T20:08:37.113 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-16T20:08:37.128 DEBUG:teuthology.orchestra.run.smithi088:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-16T20:08:37.138 DEBUG:teuthology.orchestra.run.smithi094:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-16T20:08:37.153 INFO:teuthology.orchestra.run.smithi088.stdout:kernel.core_pattern = core 2023-10-16T20:08:37.176 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = core 2023-10-16T20:08:37.194 DEBUG:teuthology.orchestra.run.smithi088:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-16T20:08:37.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T20:08:37.227 DEBUG:teuthology.orchestra.run.smithi094:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-16T20:08:37.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T20:08:37.249 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-16T20:08:37.261 INFO:teuthology.task.internal:Transferring archived files... 2023-10-16T20:08:37.262 DEBUG:teuthology.misc:Transferring archived files from smithi088:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-10-16_18:49:15-fs-rishabh-2023oct13-testing-default-smithi/7430091/remote/smithi088 2023-10-16T20:08:37.262 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-16T20:08:37.296 DEBUG:teuthology.misc:Transferring archived files from smithi094:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-10-16_18:49:15-fs-rishabh-2023oct13-testing-default-smithi/7430091/remote/smithi094 2023-10-16T20:08:37.296 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-16T20:08:37.321 INFO:teuthology.task.internal:Removing archive directory... 2023-10-16T20:08:37.322 DEBUG:teuthology.orchestra.run.smithi088:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-16T20:08:37.338 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-16T20:08:37.366 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-16T20:08:37.378 INFO:teuthology.task.internal:Not uploading archives. 2023-10-16T20:08:37.379 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-16T20:08:37.389 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-16T20:08:37.389 DEBUG:teuthology.orchestra.run.smithi088:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-16T20:08:37.393 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-16T20:08:37.398 INFO:teuthology.orchestra.run.smithi088.stdout: 657970 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 16 20:08 /home/ubuntu/cephtest 2023-10-16T20:08:37.418 INFO:teuthology.orchestra.run.smithi094.stdout: 657971 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 16 20:08 /home/ubuntu/cephtest 2023-10-16T20:08:37.420 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-16T20:08:37.431 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-16T20:08:37.451 INFO:teuthology.nuke:Checking targets against current locks 2023-10-16T20:08:37.478 DEBUG:teuthology.nuke:shortname: smithi088 2023-10-16T20:08:37.479 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-16T20:08:37.502 DEBUG:teuthology.nuke:shortname: smithi094 2023-10-16T20:08:37.503 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-16T20:08:37.506 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi088.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-16_18:49:15-fs-rishabh-2023oct13-testing-default-smithi/7430091', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-16 19:42:32.886281', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO3u8ck309lL1eF1slemCDnL7evTbk1vSlxyuVhiJ0jwjFWr/00avoczBkwVMMA35fzd5AfNup9oIrkl12E+ihA='} 2023-10-16T20:08:37.526 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi094.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-16_18:49:15-fs-rishabh-2023oct13-testing-default-smithi/7430091', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-16 19:42:32.884956', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMmVItP9PTvUPm1HK+zbQSrkxi0N+SNhl6uG+l41BT706NG4MjhexTB9Ww7pdf+KZC/ZJNRNAt9HxF8lUnSjKy8='} 2023-10-16T20:08:37.558 INFO:teuthology.orchestra.console.smithi088:Power off 2023-10-16T20:08:37.559 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-16T20:08:37.580 DEBUG:teuthology.orchestra.console.smithi088:power off output: Chassis Power Control: Down/Off 2023-10-16T20:08:37.581 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T20:08:37.595 INFO:teuthology.orchestra.console.smithi094:Power off 2023-10-16T20:08:37.595 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-16T20:08:37.605 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is on 2023-10-16T20:08:37.617 DEBUG:teuthology.orchestra.console.smithi094:power off output: Chassis Power Control: Down/Off 2023-10-16T20:08:37.617 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T20:08:37.640 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is on 2023-10-16T20:08:41.607 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T20:08:41.642 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T20:08:41.717 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is on 2023-10-16T20:08:41.753 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is on 2023-10-16T20:08:45.718 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T20:08:45.755 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T20:08:45.829 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is off 2023-10-16T20:08:45.865 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is off 2023-10-16T20:08:45.930 INFO:teuthology.orchestra.console.smithi088:Power off completed 2023-10-16T20:08:45.967 INFO:teuthology.orchestra.console.smithi094:Power off completed 2023-10-16T20:08:46.072 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm} duration: 976.0498399734497 failure_reason: 'Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm)' flavor: default mon.a-kernel-sha1: 47672119149b103ec7cec2cc65c063ccbb001ac7 mon.b-kernel-sha1: 47672119149b103ec7cec2cc65c063ccbb001ac7 owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=1dfcb444471f47cebfef2520438b1ac0 status: fail success: false 2023-10-16T20:08:46.073 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-16T20:08:46.143 INFO:teuthology.run:FAIL